feat: Introduce OpenAiProviderDialect for flexible request shaping and remove reasoning properties from ModelProfile
- Added OpenAiProviderDialect to manage request features for OpenAI-compatible providers. - Refactored ChatCompletionClient and TextCompletionClient to utilize the new dialect for handling reasoning and usage settings. - Removed reasoningEffort and reasoningSummaryDetails from ModelProfile and related services. - Updated README to reflect changes in request shaping and dialect usage.
This commit is contained in:
51
README.md
51
README.md
@@ -250,8 +250,6 @@ Each profile file is a `.properties` resource on the classpath.
|
||||
| `cost1MCachedInTokens` | Double | No | Cost per 1 million cached input tokens. Defaults to `cost1MInTokens`. | `0.35`, `0` |
|
||||
| `cost1MOutTokens` | Double | Yes | Cost per 1 million output tokens. | `0.71`, `15.00`, `0` |
|
||||
| `cost1MReasoningOutTokens` | Double | No | Cost per 1 million reasoning output tokens. Defaults to `0`. | `0`, `60.00` |
|
||||
| `reasoningEffort` | String | No | Reasoning effort for reasoning-capable models. | `low`, `medium`, `high` |
|
||||
| `reasoningSummaryDetails` | String | No | How reasoning content should be summarized. | `auto`, `concise`, `detailed` |
|
||||
|
||||
Notes:
|
||||
|
||||
@@ -260,7 +258,6 @@ Notes:
|
||||
- Cost properties are only used for statistics and accounting.
|
||||
- `cost1MCachedInTokens` defaults to the normal input token cost when not provided.
|
||||
- `cost1MReasoningOutTokens` defaults to `0` when not provided.
|
||||
- Reasoning properties only apply to models that expose separate reasoning output.
|
||||
- `tokenEncoding` is primarily relevant for tokenizer implementations that support multiple encodings, such as `jTokkit`.
|
||||
|
||||
### Example Profile Files
|
||||
@@ -282,8 +279,6 @@ url=http://localhost:5001
|
||||
apiKey=
|
||||
cost1MInTokens=0
|
||||
cost1MOutTokens=0
|
||||
reasoningEffort=medium
|
||||
reasoningSummaryDetails=auto
|
||||
```
|
||||
|
||||
Local text completion:
|
||||
@@ -322,6 +317,52 @@ Tool calling is currently supported only for chat completion models. `TextComple
|
||||
|
||||
These built-in providers are OpenAI-compatible today, but that is an implementation detail of the current client set, not a hard architectural limit of the library. Additional provider integrations can be added in the library itself, and applications can plug in their own client implementations where needed.
|
||||
|
||||
If you need to adapt request shaping for a specific OpenAI-compatible provider, you can optionally register an `OpenAiProviderDialect` bean. This allows applications to suppress request fields that a provider does not support while keeping `ModelProfile` provider-agnostic and runtime-driven.
|
||||
|
||||
The default dialect behavior is:
|
||||
|
||||
- `sendUsage(...) = true`
|
||||
- `sendReasoning(...) = true`
|
||||
- `sendMaxCompletionTokens(...) = true`
|
||||
- `reasoningEffort(...) = "medium"`
|
||||
- `reasoningSummaryDetails(...) = "concise"`
|
||||
|
||||
Example:
|
||||
|
||||
```java
|
||||
@Bean
|
||||
OpenAiProviderDialect azureOpenAiProviderDialect() {
|
||||
return new OpenAiProviderDialect() {
|
||||
@Override
|
||||
public boolean sendUsage(final ModelProfile modelProfile) {
|
||||
return false;
|
||||
}
|
||||
|
||||
@Override
|
||||
public boolean sendReasoning(final ModelProfile modelProfile) {
|
||||
return false;
|
||||
}
|
||||
|
||||
@Override
|
||||
public boolean sendMaxCompletionTokens(final ModelProfile modelProfile) {
|
||||
return false;
|
||||
}
|
||||
|
||||
@Override
|
||||
public String reasoningEffort(final ModelProfile modelProfile) {
|
||||
return "medium";
|
||||
}
|
||||
|
||||
@Override
|
||||
public String reasoningSummaryDetails(final ModelProfile modelProfile) {
|
||||
return "concise";
|
||||
}
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
If no `OpenAiProviderDialect` bean is present, all OpenAI-compatible request features remain enabled by default.
|
||||
|
||||
For text completions, select a prompt strategy with the `prompts` property. lm-lib ships built-in strategies for common model families including Llama, GPT-OSS, Gemma, Mistral, Phi, Qwen, Vicuna, ChatML, Alpaca, and a no-formatting strategy.
|
||||
|
||||
Example:
|
||||
|
||||
Reference in New Issue
Block a user