feat: Introduce OpenAiProviderDialect for flexible request shaping and remove reasoning properties from ModelProfile

- Added OpenAiProviderDialect to manage request features for OpenAI-compatible providers.
- Refactored ChatCompletionClient and TextCompletionClient to utilize the new dialect for handling reasoning and usage settings.
- Removed reasoningEffort and reasoningSummaryDetails from ModelProfile and related services.
- Updated README to reflect changes in request shaping and dialect usage.
This commit is contained in:
Stephan Birkl
2026-05-07 17:57:59 +02:00
parent 6235c49358
commit 244aa2574b
11 changed files with 357 additions and 71 deletions

View File

@@ -250,8 +250,6 @@ Each profile file is a `.properties` resource on the classpath.
| `cost1MCachedInTokens` | Double | No | Cost per 1 million cached input tokens. Defaults to `cost1MInTokens`. | `0.35`, `0` |
| `cost1MOutTokens` | Double | Yes | Cost per 1 million output tokens. | `0.71`, `15.00`, `0` |
| `cost1MReasoningOutTokens` | Double | No | Cost per 1 million reasoning output tokens. Defaults to `0`. | `0`, `60.00` |
| `reasoningEffort` | String | No | Reasoning effort for reasoning-capable models. | `low`, `medium`, `high` |
| `reasoningSummaryDetails` | String | No | How reasoning content should be summarized. | `auto`, `concise`, `detailed` |
Notes:
@@ -260,7 +258,6 @@ Notes:
- Cost properties are only used for statistics and accounting.
- `cost1MCachedInTokens` defaults to the normal input token cost when not provided.
- `cost1MReasoningOutTokens` defaults to `0` when not provided.
- Reasoning properties only apply to models that expose separate reasoning output.
- `tokenEncoding` is primarily relevant for tokenizer implementations that support multiple encodings, such as `jTokkit`.
### Example Profile Files
@@ -282,8 +279,6 @@ url=http://localhost:5001
apiKey=
cost1MInTokens=0
cost1MOutTokens=0
reasoningEffort=medium
reasoningSummaryDetails=auto
```
Local text completion:
@@ -322,6 +317,52 @@ Tool calling is currently supported only for chat completion models. `TextComple
These built-in providers are OpenAI-compatible today, but that is an implementation detail of the current client set, not a hard architectural limit of the library. Additional provider integrations can be added in the library itself, and applications can plug in their own client implementations where needed.
If you need to adapt request shaping for a specific OpenAI-compatible provider, you can optionally register an `OpenAiProviderDialect` bean. This allows applications to suppress request fields that a provider does not support while keeping `ModelProfile` provider-agnostic and runtime-driven.
The default dialect behavior is:
- `sendUsage(...) = true`
- `sendReasoning(...) = true`
- `sendMaxCompletionTokens(...) = true`
- `reasoningEffort(...) = "medium"`
- `reasoningSummaryDetails(...) = "concise"`
Example:
```java
@Bean
OpenAiProviderDialect azureOpenAiProviderDialect() {
return new OpenAiProviderDialect() {
@Override
public boolean sendUsage(final ModelProfile modelProfile) {
return false;
}
@Override
public boolean sendReasoning(final ModelProfile modelProfile) {
return false;
}
@Override
public boolean sendMaxCompletionTokens(final ModelProfile modelProfile) {
return false;
}
@Override
public String reasoningEffort(final ModelProfile modelProfile) {
return "medium";
}
@Override
public String reasoningSummaryDetails(final ModelProfile modelProfile) {
return "concise";
}
};
}
```
If no `OpenAiProviderDialect` bean is present, all OpenAI-compatible request features remain enabled by default.
For text completions, select a prompt strategy with the `prompts` property. lm-lib ships built-in strategies for common model families including Llama, GPT-OSS, Gemma, Mistral, Phi, Qwen, Vicuna, ChatML, Alpaca, and a no-formatting strategy.
Example: