feat: Implement OpenRouter model discovery and endpoints

- Added OpenRouterModelDiscovery class for managing model listings and metadata resolution.
- Introduced OpenRouterModelEndpointsResponse and OpenRouterModelsResponse classes for handling API responses.
- Created OpenRouterModelQuery class to encapsulate query parameters for model listing.
- Updated ModelProfile to include OPENROUTER as a model provider.
- Enhanced ModelProfileService to handle default values for maxTokens and maxContextLength.
- Added OpenRouterClientTest for testing model discovery and chat completion functionalities.
- Configured application-test.properties and local-openrouter-chatcompletion.properties for OpenRouter integration.
This commit is contained in:
extio-de
2026-05-10 14:11:06 +02:00
parent 2028d9bea5
commit b95e725aad
16 changed files with 1563 additions and 24 deletions

View File

@@ -13,13 +13,13 @@ lm-lib is published under the Apache License 2.0.
lm-lib is designed for applications that need more than single prompt-response calls. Its center of gravity is the agent framework: structured multi-step flows such as classification, planning, fan-out analysis, synthesis, grading, feedback loops, and other graph-based execution patterns. lm-lib is designed for applications that need more than single prompt-response calls. Its center of gravity is the agent framework: structured multi-step flows such as classification, planning, fan-out analysis, synthesis, grading, feedback loops, and other graph-based execution patterns.
It also supports direct completion use cases through a client abstraction layer. The built-in clients currently cover OpenAI-compatible chat and text completion endpoints as well as Ollama's native chat API, and the abstraction remains open for additional provider implementations over time. Applications can also provide their own client implementations by implementing the relevant interfaces. It also supports direct completion use cases through a client abstraction layer. The built-in clients currently cover OpenAI-compatible chat and text completion endpoints, OpenRouter's OpenAI-compatible routing API, and Ollama's native chat API, and the abstraction remains open for additional provider implementations over time. Applications can also provide their own client implementations by implementing the relevant interfaces.
The current built-in client set supports OpenAI-compatible chat completions, legacy text completions, and native Ollama chat completions, with model-specific prompt strategies for text completion models. The current built-in client set supports OpenAI-compatible chat completions, OpenRouter chat completions with cached metadata discovery, legacy text completions, and native Ollama chat completions, with model-specific prompt strategies for text completion models.
## Key Capabilities ## Key Capabilities
- OpenAI-compatible chat and text completions plus native Ollama chat completions - OpenAI-compatible chat and text completions, OpenRouter chat completions, plus native Ollama chat completions
- Chat-completion tool calling with tool definitions, returned tool calls, and tool-result follow-up rounds - Chat-completion tool calling with tool definitions, returned tool calls, and tool-result follow-up rounds
- Streaming and non-streaming APIs - Streaming and non-streaming APIs
- Separate reasoning capture for models that expose reasoning output - Separate reasoning capture for models that expose reasoning output
@@ -242,7 +242,7 @@ Each profile file is a `.properties` resource on the classpath.
| `maxContextLength` | Integer | Yes | Maximum total context length. | `16000`, `32768`, `128000` | | `maxContextLength` | Integer | Yes | Maximum total context length. | `16000`, `32768`, `128000` |
| `temperature` | Double | Yes | Sampling temperature. Must be greater than zero. | `0.2`, `0.4`, `0.7` | | `temperature` | Double | Yes | Sampling temperature. Must be greater than zero. | `0.2`, `0.4`, `0.7` |
| `topP` | Double | Yes | Nucleus sampling parameter. Must be greater than zero. | `1.0`, `0.9` | | `topP` | Double | Yes | Nucleus sampling parameter. Must be greater than zero. | `1.0`, `0.9` |
| `modelProvider` | Enum | Yes | Completion client type. | `OAI_CHAT_COMPLETION`, `OAI_TEXT_COMPLETION`, `OLLAMA` | | `modelProvider` | Enum | Yes | Completion client type. | `OAI_CHAT_COMPLETION`, `OAI_TEXT_COMPLETION`, `OPENROUTER`, `OLLAMA` |
| `modelName` | String | No | Model name or deployment. Can be empty for endpoints where the model should be auto-detected. | `gpt-4`, `Llama-3.3-70B-Instruct`, `` | | `modelName` | String | No | Model name or deployment. Can be empty for endpoints where the model should be auto-detected. | `gpt-4`, `Llama-3.3-70B-Instruct`, `` |
| `url` | String | Yes | Base API URL. | `https://api.openai.com/v1`, `http://localhost:5001` | | `url` | String | Yes | Base API URL. | `https://api.openai.com/v1`, `http://localhost:5001` |
| `apiKey` | String | No | Bearer token or Spring placeholder. Can be empty for local servers. | `sk-...`, `${apikey}`, `` | | `apiKey` | String | No | Bearer token or Spring placeholder. Can be empty for local servers. | `sk-...`, `${apikey}`, `` |
@@ -330,6 +330,7 @@ The built-in provider implementations currently support these types:
```properties ```properties
modelProvider=OAI_CHAT_COMPLETION modelProvider=OAI_CHAT_COMPLETION
modelProvider=OAI_TEXT_COMPLETION modelProvider=OAI_TEXT_COMPLETION
modelProvider=OPENROUTER
modelProvider=OLLAMA modelProvider=OLLAMA
``` ```
@@ -391,6 +392,40 @@ OpenAiProviderDialect legacyOpenAiProviderDialect() {
} }
``` ```
### OpenRouter Provider
`OPENROUTER` targets OpenRouter's chat-completions API and its richer model discovery endpoints.
The OpenRouter provider uses `/v1/chat/completions` for completions and caches metadata from `/v1/models`, `/v1/models/user`, and `/v1/models/{author}/{slug}/endpoints`. The cached metadata is used to avoid repeating discovery calls on every request and to derive model-aware routing behavior such as tool support, reasoning support, token-limit parameter selection, context limits, and pricing.
Use `OPENROUTER` when you want OpenRouter's model catalog, user-filtered model availability, or provider-aggregated pricing and capability metadata while keeping the standard lm-lib chat-completion API.
Typical profile:
```properties
category=L
prompts=
tokenizer=jTokkit
tokenEncoding=cl100k_base
maxTokens=4000
maxContextLength=128000
temperature=0.7
topP=0.95
modelProvider=OPENROUTER
modelName=
url=
apiKey=${openrouter.apiKey}
cost1MInTokens=0
cost1MCachedInTokens=0
cost1MOutTokens=0
cost1MReasoningOutTokens=0
```
* `modelName` is the OpenRouter model id (example: `openai/gpt-chat-latest`). If `modelName` is left empty, lm-lib will use the first available free model.
* The `url` field is ignored by the OpenRouter client and always targets `https://openrouter.ai/api`.
* Token pricing is overridden by OpenRouter's model metadata
* `maxTokens` and `maxContextLength` are clamped at the actual model capabilities. Leave it 0 to use the model limits from OpenRouter's metadata.
### Ollama Provider ### Ollama Provider
`OLLAMA` targets Ollama's native API surface rather than the OpenAI-compatible transport. `OLLAMA` targets Ollama's native API surface rather than the OpenAI-compatible transport.
@@ -405,7 +440,7 @@ lm-lib caches `/api/show` model details per resolved model. Thinking and tool de
Tool calling is exposed through the same lm-lib API across providers that support it. The capability is model-aware rather than purely provider-wide, so callers should check `supportsToolCalling(modelProfile)` or `supportsToolCalling(modelCategory)` on `Client` or `ClientService` before enabling tool-aware flows. Tool calling is exposed through the same lm-lib API across providers that support it. The capability is model-aware rather than purely provider-wide, so callers should check `supportsToolCalling(modelProfile)` or `supportsToolCalling(modelCategory)` on `Client` or `ClientService` before enabling tool-aware flows.
In practice, text-completion clients do not support tools via api (but you have full control over the raw prompt), while chat-oriented clients may support them depending on the resolved backend and model. For Ollama, lm-lib probes cached `/api/show` capability metadata. For OpenAI-compatible chat providers, support is determined by the selected client type. In practice, text-completion clients do not support tools via api (but you have full control over the raw prompt), while chat-oriented clients may support them depending on the resolved backend and model. For Ollama, lm-lib probes cached `/api/show` capability metadata. For OpenRouter, lm-lib probes cached model and endpoint metadata from the OpenRouter model APIs. For OpenAI-compatible chat providers, support is determined by the selected client type.
The library keeps the caller-facing tool-calling flow uniform even when providers differ internally. You still pass `ToolDefinition` and `ToolCallData`, inspect `CompletionFinishReason.TOOL_CALLS`, and append tool results through the same conversation API. If a backend does not expose every request-side control, lm-lib keeps the public API stable and applies those controls on a best-effort basis. The library keeps the caller-facing tool-calling flow uniform even when providers differ internally. You still pass `ToolDefinition` and `ToolCallData`, inspect `CompletionFinishReason.TOOL_CALLS`, and append tool results through the same conversation API. If a backend does not expose every request-side control, lm-lib keeps the public API stable and applies those controls on a best-effort basis.

View File

@@ -6,6 +6,8 @@ import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Bean;
import de.extio.lmlib.client.openrouter.OpenRouterClient;
import de.extio.lmlib.client.openrouter.OpenRouterModelDiscovery;
import de.extio.lmlib.client.ollama.OllamaClient; import de.extio.lmlib.client.ollama.OllamaClient;
import de.extio.lmlib.client.oai.completion.chat.ChatCompletionClient; import de.extio.lmlib.client.oai.completion.chat.ChatCompletionClient;
import de.extio.lmlib.client.oai.completion.text.TextCompletionClient; import de.extio.lmlib.client.oai.completion.text.TextCompletionClient;
@@ -35,6 +37,18 @@ public class ClientAutoConfiguration {
return new ChatCompletionClient(); return new ChatCompletionClient();
} }
@Bean
@ConditionalOnMissingBean
OpenRouterModelDiscovery openRouterModelDiscovery() {
return new OpenRouterModelDiscovery();
}
@Bean
@ConditionalOnMissingBean
OpenRouterClient openRouterClient() {
return new OpenRouterClient();
}
@Bean @Bean
@ConditionalOnMissingBean @ConditionalOnMissingBean
OllamaClient ollamaClient() { OllamaClient ollamaClient() {

View File

@@ -265,13 +265,10 @@ public abstract class AbstractCompletionClient implements Client, DisposableBean
outTokens = this.tokenizerResolver.count(response, modelProfile); outTokens = this.tokenizerResolver.count(response, modelProfile);
} }
// Calculate cost - inTokens and outTokens are already exclusive of cached/reasoning tokens var cost = this.getUsageCost(usage);
BigDecimal cost = BigDecimal.ZERO; if (cost.compareTo(BigDecimal.ZERO) <= 0) {
cost = cost.add(new BigDecimal(inTokens).multiply(modelProfile.costPerInToken())); cost = this.calculateTokenCost(modelProfile, inTokens, cachedInTokens, outTokens, reasoningOutTokens);
cost = cost.add(new BigDecimal(cachedInTokens).multiply(modelProfile.costPerCachedInToken())); }
cost = cost.add(new BigDecimal(outTokens).multiply(modelProfile.costPerOutToken()));
cost = cost.add(new BigDecimal(reasoningOutTokens).multiply(modelProfile.costPerReasoningOutToken()));
final var statistics = new de.extio.lmlib.client.CompletionStatistics( final var statistics = new de.extio.lmlib.client.CompletionStatistics(
1, 1,
Duration.between(start, LocalDateTime.now()), Duration.between(start, LocalDateTime.now()),
@@ -288,6 +285,51 @@ public abstract class AbstractCompletionClient implements Client, DisposableBean
return statistics; return statistics;
} }
private BigDecimal getUsageCost(final Usage usage) {
if (usage == null) {
return BigDecimal.ZERO;
}
if (usage.getCost() != null && usage.getCost().compareTo(BigDecimal.ZERO) > 0) {
return usage.getCost();
}
final var costDetails = usage.getCostDetails();
if (costDetails != null) {
if (costDetails.getUpstreamInferenceCost() != null) {
return costDetails.getUpstreamInferenceCost();
}
var cost = BigDecimal.ZERO;
if (costDetails.getUpstreamInferencePromptCost() != null) {
cost = cost.add(costDetails.getUpstreamInferencePromptCost());
}
if (costDetails.getUpstreamInferenceCompletionsCost() != null) {
cost = cost.add(costDetails.getUpstreamInferenceCompletionsCost());
}
if (costDetails.getUpstreamInferenceInputCost() != null) {
cost = cost.add(costDetails.getUpstreamInferenceInputCost());
}
if (costDetails.getUpstreamInferenceOutputCost() != null) {
cost = cost.add(costDetails.getUpstreamInferenceOutputCost());
}
if (cost.compareTo(BigDecimal.ZERO) > 0) {
return cost;
}
}
return BigDecimal.ZERO;
}
private BigDecimal calculateTokenCost(final ModelProfile modelProfile, int inTokens, int cachedInTokens, int outTokens, int reasoningOutTokens) {
var cost = BigDecimal.ZERO;
cost = cost.add(new BigDecimal(inTokens).multiply(modelProfile.costPerInToken()));
cost = cost.add(new BigDecimal(cachedInTokens).multiply(modelProfile.costPerCachedInToken()));
cost = cost.add(new BigDecimal(outTokens).multiply(modelProfile.costPerOutToken()));
cost = cost.add(new BigDecimal(reasoningOutTokens).multiply(modelProfile.costPerReasoningOutToken()));
return cost;
}
@Override @Override
public void destroy() throws Exception { public void destroy() throws Exception {
if (this.collectStatistics && this.totalStatistics.requests > 0) { if (this.collectStatistics && this.totalStatistics.requests > 0) {

View File

@@ -1,5 +1,7 @@
package de.extio.lmlib.client.oai.completion; package de.extio.lmlib.client.oai.completion;
import java.math.BigDecimal;
import com.fasterxml.jackson.annotation.*; import com.fasterxml.jackson.annotation.*;
public final class Usage { public final class Usage {
@@ -14,6 +16,12 @@ public final class Usage {
private OutputTokensDetails outputTokensDetails; private OutputTokensDetails outputTokensDetails;
private BigDecimal cost;
private Boolean byok;
private CostDetails costDetails;
@JsonProperty("completion_tokens") @JsonProperty("completion_tokens")
public int getCompletionTokens() { public int getCompletionTokens() {
return this.completionTokens; return this.completionTokens;
@@ -60,6 +68,11 @@ public final class Usage {
this.inputTokensDetails = inputTokensDetails; this.inputTokensDetails = inputTokensDetails;
} }
@JsonProperty("prompt_tokens_details")
public void setPromptTokensDetails(final InputTokensDetails inputTokensDetails) {
this.inputTokensDetails = inputTokensDetails;
}
@JsonProperty("output_tokens_details") @JsonProperty("output_tokens_details")
public OutputTokensDetails getOutputTokensDetails() { public OutputTokensDetails getOutputTokensDetails() {
return this.outputTokensDetails; return this.outputTokensDetails;
@@ -69,6 +82,38 @@ public final class Usage {
this.outputTokensDetails = outputTokensDetails; this.outputTokensDetails = outputTokensDetails;
} }
@JsonProperty("completion_tokens_details")
public void setCompletionTokensDetails(final OutputTokensDetails outputTokensDetails) {
this.outputTokensDetails = outputTokensDetails;
}
@JsonProperty("cost")
public BigDecimal getCost() {
return this.cost;
}
public void setCost(final BigDecimal cost) {
this.cost = cost;
}
@JsonProperty("is_byok")
public Boolean getByok() {
return this.byok;
}
public void setByok(final Boolean byok) {
this.byok = byok;
}
@JsonProperty("cost_details")
public CostDetails getCostDetails() {
return this.costDetails;
}
public void setCostDetails(final CostDetails costDetails) {
this.costDetails = costDetails;
}
public int getCachedTokens() { public int getCachedTokens() {
return this.inputTokensDetails != null ? this.inputTokensDetails.getCachedTokens() : 0; return this.inputTokensDetails != null ? this.inputTokensDetails.getCachedTokens() : 0;
} }
@@ -79,7 +124,70 @@ public final class Usage {
@Override @Override
public String toString() { public String toString() {
return "Usage [completionTokens=" + this.completionTokens + ", promptTokens=" + this.promptTokens + ", totalTokens=" + this.totalTokens + ", inputTokensDetails=" + this.inputTokensDetails + ", outputTokensDetails=" + this.outputTokensDetails + "]"; return "Usage [completionTokens=" + this.completionTokens + ", promptTokens=" + this.promptTokens + ", totalTokens=" + this.totalTokens + ", inputTokensDetails=" + this.inputTokensDetails + ", outputTokensDetails=" + this.outputTokensDetails + ", cost=" + this.cost + ", byok=" + this.byok + ", costDetails=" + this.costDetails + "]";
}
public static final class CostDetails {
private BigDecimal upstreamInferenceCost;
private BigDecimal upstreamInferencePromptCost;
private BigDecimal upstreamInferenceCompletionsCost;
private BigDecimal upstreamInferenceInputCost;
private BigDecimal upstreamInferenceOutputCost;
@JsonProperty("upstream_inference_cost")
public BigDecimal getUpstreamInferenceCost() {
return this.upstreamInferenceCost;
}
public void setUpstreamInferenceCost(final BigDecimal upstreamInferenceCost) {
this.upstreamInferenceCost = upstreamInferenceCost;
}
@JsonProperty("upstream_inference_prompt_cost")
public BigDecimal getUpstreamInferencePromptCost() {
return this.upstreamInferencePromptCost;
}
public void setUpstreamInferencePromptCost(final BigDecimal upstreamInferencePromptCost) {
this.upstreamInferencePromptCost = upstreamInferencePromptCost;
}
@JsonProperty("upstream_inference_completions_cost")
public BigDecimal getUpstreamInferenceCompletionsCost() {
return this.upstreamInferenceCompletionsCost;
}
public void setUpstreamInferenceCompletionsCost(final BigDecimal upstreamInferenceCompletionsCost) {
this.upstreamInferenceCompletionsCost = upstreamInferenceCompletionsCost;
}
@JsonProperty("upstream_inference_input_cost")
public BigDecimal getUpstreamInferenceInputCost() {
return this.upstreamInferenceInputCost;
}
public void setUpstreamInferenceInputCost(final BigDecimal upstreamInferenceInputCost) {
this.upstreamInferenceInputCost = upstreamInferenceInputCost;
}
@JsonProperty("upstream_inference_output_cost")
public BigDecimal getUpstreamInferenceOutputCost() {
return this.upstreamInferenceOutputCost;
}
public void setUpstreamInferenceOutputCost(final BigDecimal upstreamInferenceOutputCost) {
this.upstreamInferenceOutputCost = upstreamInferenceOutputCost;
}
@Override
public String toString() {
return "CostDetails [upstreamInferenceCost=" + this.upstreamInferenceCost + ", upstreamInferencePromptCost=" + this.upstreamInferencePromptCost + ", upstreamInferenceCompletionsCost=" + this.upstreamInferenceCompletionsCost + ", upstreamInferenceInputCost=" + this.upstreamInferenceInputCost + ", upstreamInferenceOutputCost=" + this.upstreamInferenceOutputCost + "]";
}
} }
public static final class InputTokensDetails { public static final class InputTokensDetails {

View File

@@ -101,6 +101,9 @@ public class ChatCompletionClient extends AbstractCompletionClient {
if (line.startsWith("data: ")) { if (line.startsWith("data: ")) {
line = line.substring(6).trim(); line = line.substring(6).trim();
} }
if (line.startsWith(":")) {
continue;
}
if (line.isEmpty()) { if (line.isEmpty()) {
continue; continue;
} }

View File

@@ -44,10 +44,12 @@ final class ChatMessage {
} }
@JsonProperty("reasoning_content") @JsonProperty("reasoning_content")
@JsonAlias("reasoning")
public String getReasoningContent() { public String getReasoningContent() {
return reasoningContent; return reasoningContent;
} }
@JsonProperty("reasoning")
public void setReasoningContent(final String reasoningContent) { public void setReasoningContent(final String reasoningContent) {
this.reasoningContent = reasoningContent; this.reasoningContent = reasoningContent;
} }

View File

@@ -0,0 +1,185 @@
package de.extio.lmlib.client.openrouter;
import java.util.List;
import java.util.function.Consumer;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import de.extio.lmlib.client.Chunk;
import de.extio.lmlib.client.Completion;
import de.extio.lmlib.client.Conversation;
import de.extio.lmlib.client.ToolCallData;
import de.extio.lmlib.client.oai.OpenAiProviderDialect;
import de.extio.lmlib.client.oai.completion.chat.ChatCompletionClient;
import de.extio.lmlib.profile.ModelCategory;
import de.extio.lmlib.profile.ModelProfile;
import de.extio.lmlib.profile.ModelProfile.ModelProvider;
public class OpenRouterClient extends ChatCompletionClient {
private static final Logger LOGGER = LoggerFactory.getLogger(OpenRouterClient.class);
@Autowired
private OpenRouterModelDiscovery openRouterModelDiscovery;
@Override
public Completion conversation(final ModelCategory modelCategory, final Conversation conversation, final ToolCallData toolCallData, final boolean skipCache) {
return this.streamConversation(modelCategory, conversation, null, toolCallData, skipCache);
}
@Override
public Completion conversation(final ModelProfile modelProfile, final Conversation conversation, final ToolCallData toolCallData, final boolean skipCache) {
return this.streamConversation(modelProfile, conversation, null, toolCallData, skipCache);
}
@Override
public Completion streamConversation(final ModelCategory modelCategory, final Conversation conversation, final Consumer<Chunk> chunkConsumer, final ToolCallData toolCallData, final boolean skipCache) {
final var modelProfile = this.modelProfileService.getModelProfile(modelCategory.getModelProfile(), modelCategory);
return this.streamConversation(modelProfile, conversation, chunkConsumer, toolCallData, skipCache);
}
@Override
public Completion streamConversation(final ModelProfile modelProfile, final Conversation conversation, final Consumer<Chunk> chunkConsumer, final ToolCallData toolCallData, final boolean skipCache) {
if (modelProfile == null || modelProfile.modelProvider() != ModelProvider.OPENROUTER) {
throw new IllegalArgumentException("Invalid ModelProfile");
}
return this.requestCompletion(conversation, modelProfile, chunkConsumer, toolCallData);
}
@Override
public List<String> getModelNames(final ModelProfile modelProfile, final boolean forceReload) {
if (modelProfile == null || modelProfile.modelProvider() != ModelProvider.OPENROUTER) {
return List.of();
}
if (modelProfile.modelName() != null && !modelProfile.modelName().isBlank()) {
return List.of(modelProfile.modelName());
}
return this.openRouterModelDiscovery.listModels(modelProfile, new OpenRouterModelQuery(true, null, List.of(), List.of()), forceReload).stream()
.map(OpenRouterModelsResponse.OpenRouterModel::getId)
.filter(modelName -> modelName != null && !modelName.isBlank())
.toList();
}
@Override
public boolean supportsToolCalling(final ModelCategory modelCategory) {
final var modelProfile = this.modelProfileService.getModelProfile(modelCategory.getModelProfile(), modelCategory);
return this.supportsToolCalling(modelProfile);
}
@Override
public boolean supportsToolCalling(final ModelProfile modelProfile) {
if (modelProfile == null || modelProfile.modelProvider() != ModelProvider.OPENROUTER) {
return false;
}
final var metadata = this.resolveMetadata(modelProfile);
return metadata != null && metadata.supportsToolCalling();
}
@Override
public ModelProvider getModelProvider() {
return ModelProvider.OPENROUTER;
}
@Override
protected Completion requestCompletion(final Conversation conversation, final ModelProfile modelProfile, final Consumer<Chunk> chunkConsumer, final ToolCallData toolCallData) {
final var effectiveModelProfile = this.toEffectiveModelProfile(modelProfile);
LOGGER.debug("Requesting completion with OpenRouter model {} ", effectiveModelProfile.modelName());
return super.requestCompletion(conversation, effectiveModelProfile, chunkConsumer, toolCallData);
}
@Override
protected boolean sendUsage(final ModelProfile modelProfile) {
return false;
}
@Override
protected boolean sendReasoning(final ModelProfile modelProfile) {
final var metadata = this.resolveMetadata(modelProfile);
return metadata != null && metadata.supportsReasoning();
}
@Override
protected OpenAiProviderDialect.ChatTokenLimitParameterMode chatTokenLimitParameterMode(final ModelProfile modelProfile) {
final var metadata = this.resolveMetadata(modelProfile);
return metadata == null ? OpenAiProviderDialect.ChatTokenLimitParameterMode.MAX_COMPLETION_TOKENS : metadata.tokenLimitParameterMode();
}
@Override
protected OpenAiProviderDialect.ReasoningEffort reasoningEffort(final ModelProfile modelProfile) {
if (!this.sendReasoning(modelProfile)) {
return null;
}
final var reasoningEffort = super.reasoningEffort(modelProfile);
return reasoningEffort == null ? OpenAiProviderDialect.ReasoningEffort.MEDIUM : reasoningEffort;
}
private OpenRouterModelDiscovery.ResolvedOpenRouterModelMetadata resolveMetadata(final ModelProfile modelProfile) {
if (modelProfile == null || modelProfile.modelProvider() != ModelProvider.OPENROUTER) {
return null;
}
var modelName = modelProfile.modelName();
if (modelName == null || modelName.isBlank()) {
modelName = this.openRouterModelDiscovery.listModels(modelProfile, new OpenRouterModelQuery(true, null, List.of(), List.of()), false).stream()
.filter(model -> model.getPricing() != null && "0".equals(model.getPricing().getPrompt()) && "0".equals(model.getPricing().getCompletion())) // Prefer free models if no model name specified
.map(OpenRouterModelsResponse.OpenRouterModel::getId)
.filter(id -> id != null && !id.isBlank())
.findFirst()
.orElse("");
}
if (modelName.isBlank()) {
return null;
}
return this.openRouterModelDiscovery.resolveModelMetadata(modelProfile, modelName, false);
}
private ModelProfile toEffectiveModelProfile(final ModelProfile modelProfile) {
final var metadata = this.resolveMetadata(modelProfile);
if (metadata == null || metadata.modelId() == null || metadata.modelId().isBlank()) {
return modelProfile;
}
final var promptCost = metadata.promptCost();
final var cachedPromptCost = metadata.cachedPromptCost();
final var completionCost = metadata.completionCost();
final var reasoningCost = metadata.reasoningCost();
int maxContextLength;
if (modelProfile.maxContextLength() <= 0 && metadata.contextLength() <= 0) {
throw new IllegalArgumentException("OpenRouter metadata don't specify a valid context length. Therefore the model profile must specify a valid context length.");
}
else if (modelProfile.maxContextLength() > 0 && metadata.contextLength() > 0) {
maxContextLength = Math.min(modelProfile.maxContextLength(), metadata.contextLength());
}
else {
maxContextLength = modelProfile.maxContextLength() > 0 ? modelProfile.maxContextLength() : metadata.contextLength();
}
int maxTokens;
if (modelProfile.maxTokens() <= 0 && metadata.maxCompletionTokens() <= 0) {
throw new IllegalArgumentException("OpenRouter metadata don't specify a valid max completion tokens. Therefore the model profile must specify a valid max tokens.");
}
else if (modelProfile.maxTokens() > 0 && metadata.maxCompletionTokens() > 0) {
maxTokens = Math.min(modelProfile.maxTokens(), metadata.maxCompletionTokens());
}
else {
maxTokens = modelProfile.maxTokens() > 0 ? modelProfile.maxTokens() : metadata.maxCompletionTokens();
}
return new ModelProfile(
modelProfile.prompt(),
modelProfile.tokenizer(),
modelProfile.tokenEncoding(),
maxTokens,
maxContextLength,
modelProfile.temperature(),
modelProfile.topP(),
ModelProvider.OPENROUTER,
metadata.modelId(),
OpenRouterModelDiscovery.OPENROUTER_API_URL,
modelProfile.apiKey(),
promptCost != null ? promptCost : modelProfile.costPerInToken(),
cachedPromptCost != null ? cachedPromptCost : modelProfile.costPerCachedInToken(),
completionCost != null ? completionCost : modelProfile.costPerOutToken(),
reasoningCost != null ? reasoningCost : modelProfile.costPerReasoningOutToken(),
modelProfile.category());
}
}

View File

@@ -0,0 +1,351 @@
package de.extio.lmlib.client.openrouter;
import java.math.BigDecimal;
import java.net.URLEncoder;
import java.nio.charset.StandardCharsets;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.Objects;
import java.util.concurrent.ConcurrentHashMap;
import java.util.stream.Collectors;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.http.HttpMethod;
import org.springframework.web.client.RestClient;
import org.springframework.web.client.RestClientException;
import org.springframework.web.client.RestClientResponseException;
import de.extio.lmlib.client.oai.OpenAiProviderDialect;
import de.extio.lmlib.profile.ModelProfile;
import de.extio.lmlib.profile.ModelProfile.ModelProvider;
public class OpenRouterModelDiscovery {
static final String OPENROUTER_API_URL = "https://openrouter.ai/api";
private static final Logger LOGGER = LoggerFactory.getLogger(OpenRouterModelDiscovery.class);
@Autowired
@Qualifier("lmLibRestClientBuilder")
private RestClient.Builder restClientBuilder;
private final Map<String, List<OpenRouterModelsResponse.OpenRouterModel>> modelListCache = new ConcurrentHashMap<>();
private final Map<String, OpenRouterModelEndpointsResponse.OpenRouterModelEndpoints> endpointCache = new ConcurrentHashMap<>();
private final Map<String, ResolvedOpenRouterModelMetadata> metadataCache = new ConcurrentHashMap<>();
public List<OpenRouterModelsResponse.OpenRouterModel> listModels(final ModelProfile modelProfile, final OpenRouterModelQuery query_, final boolean forceReload) {
this.validateProfile(modelProfile);
final var query = query_ == null ? new OpenRouterModelQuery(false, null, List.of(), List.of()) : query_;
if (query.userFiltered() && query.hasServerSideFilters()) {
throw new IllegalArgumentException("OpenRouter /models/user does not support category, supported_parameters, or output_modalities filters");
}
final var cacheKey = this.modelListCacheKey(modelProfile, query);
if (forceReload) {
this.modelListCache.remove(cacheKey);
}
return this.modelListCache.computeIfAbsent(cacheKey, key -> this.loadModels(modelProfile, query));
}
public OpenRouterModelEndpointsResponse.OpenRouterModelEndpoints getModelEndpoints(final ModelProfile modelProfile, final String modelName, final boolean forceReload) {
this.validateProfile(modelProfile);
if (modelName == null || modelName.isBlank()) {
return new OpenRouterModelEndpointsResponse.OpenRouterModelEndpoints();
}
final var cacheKey = this.endpointCacheKey(modelProfile, modelName);
if (forceReload) {
final var endpoints = this.loadModelEndpoints(modelProfile, modelName, true);
this.endpointCache.put(cacheKey, endpoints);
return endpoints;
}
return this.endpointCache.computeIfAbsent(cacheKey, key -> this.loadModelEndpoints(modelProfile, modelName, false));
}
public ResolvedOpenRouterModelMetadata resolveModelMetadata(final ModelProfile modelProfile, final String modelName_, final boolean forceReload) {
this.validateProfile(modelProfile);
final var modelName = modelName_ == null || modelName_.isBlank() ? this.listModels(modelProfile, new OpenRouterModelQuery(true, null, List.of(), List.of()), forceReload)
.stream()
.map(OpenRouterModelsResponse.OpenRouterModel::getId)
.filter(Objects::nonNull)
.filter(id -> !id.isBlank())
.findFirst()
.orElse("") : modelName_;
if (modelName.isBlank()) {
return new ResolvedOpenRouterModelMetadata("", null, null, null, List.of(), 0, 0);
}
final var cacheKey = this.metadataCacheKey(modelProfile, modelName);
if (forceReload) {
final var metadata = this.loadResolvedMetadata(modelProfile, modelName, true);
this.metadataCache.put(cacheKey, metadata);
return metadata;
}
return this.metadataCache.computeIfAbsent(cacheKey, key -> this.loadResolvedMetadata(modelProfile, modelName, false));
}
private List<OpenRouterModelsResponse.OpenRouterModel> loadModels(final ModelProfile modelProfile, final OpenRouterModelQuery query) {
try {
var requestSpec = this.restClientBuilder.baseUrl(OPENROUTER_API_URL).build()
.method(HttpMethod.GET)
.uri(uriBuilder -> {
uriBuilder.path(query.userFiltered() ? "/v1/models/user" : "/v1/models");
if (!query.userFiltered() && query.category() != null && !query.category().isBlank()) {
uriBuilder.queryParam("category", query.category());
}
final var supportedParameters = this.join(query.supportedParameters());
if (!query.userFiltered() && !supportedParameters.isBlank()) {
uriBuilder.queryParam("supported_parameters", supportedParameters);
}
final var outputModalities = this.join(query.outputModalities());
if (!query.userFiltered() && !outputModalities.isBlank()) {
uriBuilder.queryParam("output_modalities", outputModalities);
}
return uriBuilder.build();
});
if (modelProfile.apiKey() != null && !modelProfile.apiKey().isBlank()) {
requestSpec = requestSpec.header("Authorization", "Bearer " + modelProfile.apiKey());
}
final var response = requestSpec.retrieve().body(OpenRouterModelsResponse.class);
return response == null || response.getData() == null ? List.of() : List.copyOf(response.getData());
}
catch (final RestClientResponseException e) {
LOGGER.warn("Failed to load OpenRouter models: {} {}", e.getStatusCode(), e.getStatusText());
return List.of();
}
catch (final RestClientException e) {
LOGGER.warn("Failed to load OpenRouter models: {}", e.getMessage());
return List.of();
}
catch (final RuntimeException e) {
LOGGER.warn("Failed to load OpenRouter models", e);
return List.of();
}
}
private OpenRouterModelEndpointsResponse.OpenRouterModelEndpoints loadModelEndpoints(final ModelProfile modelProfile, final String modelName, final boolean forceReload) {
try {
final var model = this.findModel(modelProfile, modelName, forceReload);
var requestSpec = this.restClientBuilder.baseUrl(OPENROUTER_API_URL).build()
.method(HttpMethod.GET)
.uri(this.resolveDetailsPath(model, modelName));
if (modelProfile.apiKey() != null && !modelProfile.apiKey().isBlank()) {
requestSpec = requestSpec.header("Authorization", "Bearer " + modelProfile.apiKey());
}
final var response = requestSpec.retrieve().body(OpenRouterModelEndpointsResponse.class);
return response == null || response.getData() == null ? new OpenRouterModelEndpointsResponse.OpenRouterModelEndpoints() : response.getData();
}
catch (final RestClientResponseException e) {
if (e.getStatusCode().value() != 404) {
LOGGER.warn("Failed to load OpenRouter endpoints for {}: {} {}", modelName, e.getStatusCode(), e.getStatusText());
}
return new OpenRouterModelEndpointsResponse.OpenRouterModelEndpoints();
}
catch (final RestClientException e) {
LOGGER.warn("Failed to load OpenRouter endpoints for {}: {}", modelName, e.getMessage());
return new OpenRouterModelEndpointsResponse.OpenRouterModelEndpoints();
}
catch (final RuntimeException e) {
LOGGER.warn("Failed to load OpenRouter endpoints for {}", modelName, e);
return new OpenRouterModelEndpointsResponse.OpenRouterModelEndpoints();
}
}
private ResolvedOpenRouterModelMetadata loadResolvedMetadata(final ModelProfile modelProfile, final String modelName, final boolean forceReload) {
final var model = this.findModel(modelProfile, modelName, forceReload);
final var endpoints = this.getModelEndpoints(modelProfile, modelName, forceReload);
final var endpoint = this.preferredEndpoint(endpoints);
final var pricing = endpoint != null && this.hasPricing(endpoint.getPricing()) ? endpoint.getPricing() : model != null ? model.getPricing() : null;
final List<String> supportedParameters = endpoint != null && endpoint.getSupportedParameters() != null && !endpoint.getSupportedParameters().isEmpty()
? List.copyOf(endpoint.getSupportedParameters())
: model != null && model.getSupportedParameters() != null ? List.copyOf(model.getSupportedParameters()) : List.of();
final var contextLength = this.firstPositive(
endpoint != null ? endpoint.getContextLength() : null,
model != null && model.getTopProvider() != null ? model.getTopProvider().getContextLength() : null,
model != null ? model.getContextLength() : null,
model != null && model.getPerRequestLimits() != null ? model.getPerRequestLimits().getPromptTokens() : null);
final var maxCompletionTokens = this.firstPositive(
endpoint != null ? endpoint.getMaxCompletionTokens() : null,
model != null && model.getTopProvider() != null ? model.getTopProvider().getMaxCompletionTokens() : null,
model != null && model.getPerRequestLimits() != null ? model.getPerRequestLimits().getCompletionTokens() : null);
return new ResolvedOpenRouterModelMetadata(modelName, model, endpoints, pricing, supportedParameters, contextLength, maxCompletionTokens);
}
private OpenRouterModelsResponse.OpenRouterModel findModel(final ModelProfile modelProfile, final String modelName, final boolean forceReload) {
final var model = this.findMatchingModel(this.listModels(modelProfile, new OpenRouterModelQuery(true, null, List.of(), List.of()), forceReload), modelName);
if (model != null) {
return model;
}
return this.findMatchingModel(this.listModels(modelProfile, new OpenRouterModelQuery(false, null, List.of(), List.of()), forceReload), modelName);
}
private OpenRouterModelsResponse.OpenRouterModel findMatchingModel(final List<OpenRouterModelsResponse.OpenRouterModel> models, final String modelName) {
if (modelName == null || modelName.isBlank() || models == null || models.isEmpty()) {
return null;
}
return models.stream()
.filter(Objects::nonNull)
.filter(model -> modelName.equalsIgnoreCase(model.getId()) || modelName.equalsIgnoreCase(model.getCanonicalSlug()))
.findFirst()
.orElse(null);
}
private OpenRouterModelEndpointsResponse.OpenRouterEndpoint preferredEndpoint(final OpenRouterModelEndpointsResponse.OpenRouterModelEndpoints endpoints) {
if (endpoints == null || endpoints.getEndpoints() == null || endpoints.getEndpoints().isEmpty()) {
return null;
}
return endpoints.getEndpoints().stream()
.filter(Objects::nonNull)
.filter(endpoint -> endpoint.getStatus() != null && endpoint.getStatus().equalsIgnoreCase("default"))
.findFirst()
.orElse(endpoints.getEndpoints().getFirst());
}
private String resolveDetailsPath(final OpenRouterModelsResponse.OpenRouterModel model, final String modelName) {
if (model != null && model.getLinks() != null && model.getLinks().getDetails() != null && !model.getLinks().getDetails().isBlank()) {
return model.getLinks().getDetails();
}
final var separator = modelName.indexOf('/');
if (separator < 0 || separator == modelName.length() - 1) {
return "/v1/models";
}
return "/v1/models/" + this.encode(modelName.substring(0, separator)) + "/" + this.encode(modelName.substring(separator + 1)) + "/endpoints";
}
private String encode(final String value) {
return URLEncoder.encode(value, StandardCharsets.UTF_8).replace("+", "%20");
}
private int firstPositive(final Integer... values) {
for (final var value : values) {
if (value != null && value > 0) {
return value;
}
}
return 0;
}
private boolean hasPricing(final OpenRouterModelsResponse.OpenRouterPricing pricing) {
return pricing != null
&& ((pricing.getPrompt() != null && !pricing.getPrompt().isBlank())
|| (pricing.getCompletion() != null && !pricing.getCompletion().isBlank())
|| (pricing.getInternalReasoning() != null && !pricing.getInternalReasoning().isBlank())
|| (pricing.getInputCacheRead() != null && !pricing.getInputCacheRead().isBlank())
|| (pricing.getInputCacheWrite() != null && !pricing.getInputCacheWrite().isBlank()));
}
private String join(final List<String> values) {
if (values == null || values.isEmpty()) {
return "";
}
return values.stream()
.filter(Objects::nonNull)
.map(String::trim)
.filter(value -> !value.isBlank())
.collect(Collectors.joining(","));
}
private String modelListCacheKey(final ModelProfile modelProfile, final OpenRouterModelQuery query) {
return this.baseKey(modelProfile) + "|user=" + query.userFiltered() + "|category=" + Objects.toString(query.category(), "") + "|supported=" + this.join(query.supportedParameters()) + "|modalities=" + this.join(query.outputModalities());
}
private String endpointCacheKey(final ModelProfile modelProfile, final String modelName) {
return this.baseKey(modelProfile) + "|endpoints|" + modelName;
}
private String metadataCacheKey(final ModelProfile modelProfile, final String modelName) {
return this.baseKey(modelProfile) + "|metadata|" + modelName;
}
private String baseKey(final ModelProfile modelProfile) {
return modelProfile.category() + "|" + OPENROUTER_API_URL + "|" + modelProfile.apiKey();
}
private void validateProfile(final ModelProfile modelProfile) {
if (modelProfile == null || modelProfile.modelProvider() != ModelProvider.OPENROUTER) {
throw new IllegalArgumentException("Invalid OpenRouter ModelProfile");
}
}
static BigDecimal parsePricing(final String value) {
if (value == null || value.isBlank()) {
return null;
}
try {
return new BigDecimal(value).divide(new BigDecimal(1000000.0));
}
catch (final NumberFormatException e) {
return null;
}
}
public record ResolvedOpenRouterModelMetadata(
String modelId,
OpenRouterModelsResponse.OpenRouterModel model,
OpenRouterModelEndpointsResponse.OpenRouterModelEndpoints endpoints,
OpenRouterModelsResponse.OpenRouterPricing pricing,
List<String> supportedParameters,
int contextLength,
int maxCompletionTokens) {
public ResolvedOpenRouterModelMetadata {
supportedParameters = supportedParameters == null ? List.of() : List.copyOf(supportedParameters);
}
public boolean supportsParameter(final String parameter) {
if (parameter == null || parameter.isBlank()) {
return false;
}
return this.supportedParameters.stream()
.filter(Objects::nonNull)
.map(value -> value.toLowerCase(Locale.ROOT).trim())
.anyMatch(value -> value.equals(parameter.toLowerCase(Locale.ROOT)));
}
public boolean supportsToolCalling() {
return this.supportsParameter("tools");
}
public boolean supportsReasoning() {
return this.supportsParameter("reasoning") || this.supportsParameter("reasoning_effort") || this.supportsParameter("include_reasoning");
}
public OpenAiProviderDialect.ChatTokenLimitParameterMode tokenLimitParameterMode() {
if (this.supportsParameter("max_completion_tokens")) {
return OpenAiProviderDialect.ChatTokenLimitParameterMode.MAX_COMPLETION_TOKENS;
}
if (this.supportsParameter("max_tokens")) {
return OpenAiProviderDialect.ChatTokenLimitParameterMode.MAX_TOKENS;
}
return OpenAiProviderDialect.ChatTokenLimitParameterMode.MAX_COMPLETION_TOKENS;
}
public BigDecimal promptCost() {
return OpenRouterModelDiscovery.parsePricing(this.pricing != null ? this.pricing.getPrompt() : null);
}
public BigDecimal completionCost() {
return OpenRouterModelDiscovery.parsePricing(this.pricing != null ? this.pricing.getCompletion() : null);
}
public BigDecimal cachedPromptCost() {
if (this.pricing == null) {
return null;
}
final var inputCacheReadCost = OpenRouterModelDiscovery.parsePricing(this.pricing.getInputCacheRead());
return inputCacheReadCost != null ? inputCacheReadCost : OpenRouterModelDiscovery.parsePricing(this.pricing.getInputCacheWrite());
}
public BigDecimal reasoningCost() {
return OpenRouterModelDiscovery.parsePricing(this.pricing != null ? this.pricing.getInternalReasoning() : null);
}
public BigDecimal requestCost() {
return OpenRouterModelDiscovery.parsePricing(this.pricing != null ? this.pricing.getRequest() : null);
}
}
}

View File

@@ -0,0 +1,224 @@
package de.extio.lmlib.client.openrouter;
import java.util.List;
import com.fasterxml.jackson.annotation.JsonProperty;
final class OpenRouterModelEndpointsResponse {
private OpenRouterModelEndpoints data;
@JsonProperty("data")
public OpenRouterModelEndpoints getData() {
return this.data;
}
public void setData(final OpenRouterModelEndpoints data) {
this.data = data;
}
static final class OpenRouterModelEndpoints {
private String id;
private String name;
private Integer created;
private String description;
private OpenRouterModelsResponse.OpenRouterModelArchitecture architecture;
private List<OpenRouterEndpoint> endpoints;
@JsonProperty("id")
public String getId() {
return this.id;
}
public void setId(final String id) {
this.id = id;
}
@JsonProperty("name")
public String getName() {
return this.name;
}
public void setName(final String name) {
this.name = name;
}
@JsonProperty("created")
public Integer getCreated() {
return this.created;
}
public void setCreated(final Integer created) {
this.created = created;
}
@JsonProperty("description")
public String getDescription() {
return this.description;
}
public void setDescription(final String description) {
this.description = description;
}
@JsonProperty("architecture")
public OpenRouterModelsResponse.OpenRouterModelArchitecture getArchitecture() {
return this.architecture;
}
public void setArchitecture(final OpenRouterModelsResponse.OpenRouterModelArchitecture architecture) {
this.architecture = architecture;
}
@JsonProperty("endpoints")
public List<OpenRouterEndpoint> getEndpoints() {
return this.endpoints;
}
public void setEndpoints(final List<OpenRouterEndpoint> endpoints) {
this.endpoints = endpoints;
}
}
static final class OpenRouterEndpoint {
private String name;
private String modelId;
private String modelName;
private String providerName;
private String status;
private String tag;
private Integer contextLength;
private Integer maxCompletionTokens;
private Integer maxPromptTokens;
private OpenRouterModelsResponse.OpenRouterPricing pricing;
private List<String> supportedParameters;
private boolean supportsImplicitCaching;
@JsonProperty("name")
public String getName() {
return this.name;
}
public void setName(final String name) {
this.name = name;
}
@JsonProperty("model_id")
public String getModelId() {
return this.modelId;
}
public void setModelId(final String modelId) {
this.modelId = modelId;
}
@JsonProperty("model_name")
public String getModelName() {
return this.modelName;
}
public void setModelName(final String modelName) {
this.modelName = modelName;
}
@JsonProperty("provider_name")
public String getProviderName() {
return this.providerName;
}
public void setProviderName(final String providerName) {
this.providerName = providerName;
}
@JsonProperty("status")
public String getStatus() {
return this.status;
}
public void setStatus(final String status) {
this.status = status;
}
@JsonProperty("tag")
public String getTag() {
return this.tag;
}
public void setTag(final String tag) {
this.tag = tag;
}
@JsonProperty("context_length")
public Integer getContextLength() {
return this.contextLength;
}
public void setContextLength(final Integer contextLength) {
this.contextLength = contextLength;
}
@JsonProperty("max_completion_tokens")
public Integer getMaxCompletionTokens() {
return this.maxCompletionTokens;
}
public void setMaxCompletionTokens(final Integer maxCompletionTokens) {
this.maxCompletionTokens = maxCompletionTokens;
}
@JsonProperty("max_prompt_tokens")
public Integer getMaxPromptTokens() {
return this.maxPromptTokens;
}
public void setMaxPromptTokens(final Integer maxPromptTokens) {
this.maxPromptTokens = maxPromptTokens;
}
@JsonProperty("pricing")
public OpenRouterModelsResponse.OpenRouterPricing getPricing() {
return this.pricing;
}
public void setPricing(final OpenRouterModelsResponse.OpenRouterPricing pricing) {
this.pricing = pricing;
}
@JsonProperty("supported_parameters")
public List<String> getSupportedParameters() {
return this.supportedParameters;
}
public void setSupportedParameters(final List<String> supportedParameters) {
this.supportedParameters = supportedParameters;
}
@JsonProperty("supports_implicit_caching")
public boolean isSupportsImplicitCaching() {
return this.supportsImplicitCaching;
}
public void setSupportsImplicitCaching(final boolean supportsImplicitCaching) {
this.supportsImplicitCaching = supportsImplicitCaching;
}
}
}

View File

@@ -0,0 +1,16 @@
package de.extio.lmlib.client.openrouter;
import java.util.List;
public record OpenRouterModelQuery(boolean userFiltered, String category, List<String> supportedParameters, List<String> outputModalities) {
public OpenRouterModelQuery {
supportedParameters = supportedParameters == null ? List.of() : List.copyOf(supportedParameters);
outputModalities = outputModalities == null ? List.of() : List.copyOf(outputModalities);
}
public boolean hasServerSideFilters() {
return this.category != null && !this.category.isBlank() || !this.supportedParameters.isEmpty() || !this.outputModalities.isEmpty();
}
}

View File

@@ -0,0 +1,428 @@
package de.extio.lmlib.client.openrouter;
import java.util.List;
import com.fasterxml.jackson.annotation.JsonProperty;
final class OpenRouterModelsResponse {
private List<OpenRouterModel> data;
@JsonProperty("data")
public List<OpenRouterModel> getData() {
return this.data;
}
public void setData(final List<OpenRouterModel> data) {
this.data = data;
}
static final class OpenRouterModel {
private String id;
private String canonicalSlug;
private String name;
private Integer created;
private Integer contextLength;
private OpenRouterModelArchitecture architecture;
private OpenRouterPricing pricing;
private List<String> supportedParameters;
private OpenRouterTopProviderInfo topProvider;
private OpenRouterPerRequestLimits perRequestLimits;
private OpenRouterModelLinks links;
private String description;
private String knowledgeCutoff;
private String expirationDate;
@JsonProperty("id")
public String getId() {
return this.id;
}
public void setId(final String id) {
this.id = id;
}
@JsonProperty("canonical_slug")
public String getCanonicalSlug() {
return this.canonicalSlug;
}
public void setCanonicalSlug(final String canonicalSlug) {
this.canonicalSlug = canonicalSlug;
}
@JsonProperty("name")
public String getName() {
return this.name;
}
public void setName(final String name) {
this.name = name;
}
@JsonProperty("created")
public Integer getCreated() {
return this.created;
}
public void setCreated(final Integer created) {
this.created = created;
}
@JsonProperty("context_length")
public Integer getContextLength() {
return this.contextLength;
}
public void setContextLength(final Integer contextLength) {
this.contextLength = contextLength;
}
@JsonProperty("architecture")
public OpenRouterModelArchitecture getArchitecture() {
return this.architecture;
}
public void setArchitecture(final OpenRouterModelArchitecture architecture) {
this.architecture = architecture;
}
@JsonProperty("pricing")
public OpenRouterPricing getPricing() {
return this.pricing;
}
public void setPricing(final OpenRouterPricing pricing) {
this.pricing = pricing;
}
@JsonProperty("supported_parameters")
public List<String> getSupportedParameters() {
return this.supportedParameters;
}
public void setSupportedParameters(final List<String> supportedParameters) {
this.supportedParameters = supportedParameters;
}
@JsonProperty("top_provider")
public OpenRouterTopProviderInfo getTopProvider() {
return this.topProvider;
}
public void setTopProvider(final OpenRouterTopProviderInfo topProvider) {
this.topProvider = topProvider;
}
@JsonProperty("per_request_limits")
public OpenRouterPerRequestLimits getPerRequestLimits() {
return this.perRequestLimits;
}
public void setPerRequestLimits(final OpenRouterPerRequestLimits perRequestLimits) {
this.perRequestLimits = perRequestLimits;
}
@JsonProperty("links")
public OpenRouterModelLinks getLinks() {
return this.links;
}
public void setLinks(final OpenRouterModelLinks links) {
this.links = links;
}
@JsonProperty("description")
public String getDescription() {
return this.description;
}
public void setDescription(final String description) {
this.description = description;
}
@JsonProperty("knowledge_cutoff")
public String getKnowledgeCutoff() {
return this.knowledgeCutoff;
}
public void setKnowledgeCutoff(final String knowledgeCutoff) {
this.knowledgeCutoff = knowledgeCutoff;
}
@JsonProperty("expiration_date")
public String getExpirationDate() {
return this.expirationDate;
}
public void setExpirationDate(final String expirationDate) {
this.expirationDate = expirationDate;
}
@Override
public String toString() {
return "OpenRouterModel [id=" + id + ", canonicalSlug=" + canonicalSlug + ", name=" + name + ", created=" + created + ", contextLength=" + contextLength + ", architecture=" + architecture + ", pricing=" + pricing + ", supportedParameters=" + supportedParameters + ", topProvider=" + topProvider + ", perRequestLimits=" + perRequestLimits + ", links=" + links + ", description=" + description + ", knowledgeCutoff=" + knowledgeCutoff + ", expirationDate=" + expirationDate + "]";
}
}
static final class OpenRouterModelArchitecture {
private String tokenizer;
private String instructType;
private String modality;
private List<String> inputModalities;
private List<String> outputModalities;
@JsonProperty("tokenizer")
public String getTokenizer() {
return this.tokenizer;
}
public void setTokenizer(final String tokenizer) {
this.tokenizer = tokenizer;
}
@JsonProperty("instruct_type")
public String getInstructType() {
return this.instructType;
}
public void setInstructType(final String instructType) {
this.instructType = instructType;
}
@JsonProperty("modality")
public String getModality() {
return this.modality;
}
public void setModality(final String modality) {
this.modality = modality;
}
@JsonProperty("input_modalities")
public List<String> getInputModalities() {
return this.inputModalities;
}
public void setInputModalities(final List<String> inputModalities) {
this.inputModalities = inputModalities;
}
@JsonProperty("output_modalities")
public List<String> getOutputModalities() {
return this.outputModalities;
}
public void setOutputModalities(final List<String> outputModalities) {
this.outputModalities = outputModalities;
}
@Override
public String toString() {
return "OpenRouterModelArchitecture [tokenizer=" + tokenizer + ", instructType=" + instructType + ", modality=" + modality + ", inputModalities=" + inputModalities + ", outputModalities=" + outputModalities + "]";
}
}
static final class OpenRouterPricing {
private String prompt;
private String completion;
private String internalReasoning;
private String inputCacheRead;
private String inputCacheWrite;
private String request;
private String image;
@JsonProperty("prompt")
public String getPrompt() {
return this.prompt;
}
public void setPrompt(final String prompt) {
this.prompt = prompt;
}
@JsonProperty("completion")
public String getCompletion() {
return this.completion;
}
public void setCompletion(final String completion) {
this.completion = completion;
}
@JsonProperty("internal_reasoning")
public String getInternalReasoning() {
return this.internalReasoning;
}
public void setInternalReasoning(final String internalReasoning) {
this.internalReasoning = internalReasoning;
}
@JsonProperty("input_cache_read")
public String getInputCacheRead() {
return this.inputCacheRead;
}
public void setInputCacheRead(final String inputCacheRead) {
this.inputCacheRead = inputCacheRead;
}
@JsonProperty("input_cache_write")
public String getInputCacheWrite() {
return this.inputCacheWrite;
}
public void setInputCacheWrite(final String inputCacheWrite) {
this.inputCacheWrite = inputCacheWrite;
}
@JsonProperty("request")
public String getRequest() {
return this.request;
}
public void setRequest(final String request) {
this.request = request;
}
@JsonProperty("image")
public String getImage() {
return this.image;
}
public void setImage(final String image) {
this.image = image;
}
@Override
public String toString() {
return "OpenRouterPricing [prompt=" + prompt + ", completion=" + completion + ", internalReasoning=" + internalReasoning + ", inputCacheRead=" + inputCacheRead + ", inputCacheWrite=" + inputCacheWrite + ", request=" + request + ", image=" + image + "]";
}
}
static final class OpenRouterTopProviderInfo {
private Integer contextLength;
private Integer maxCompletionTokens;
private boolean moderated;
@JsonProperty("context_length")
public Integer getContextLength() {
return this.contextLength;
}
public void setContextLength(final Integer contextLength) {
this.contextLength = contextLength;
}
@JsonProperty("max_completion_tokens")
public Integer getMaxCompletionTokens() {
return this.maxCompletionTokens;
}
public void setMaxCompletionTokens(final Integer maxCompletionTokens) {
this.maxCompletionTokens = maxCompletionTokens;
}
@JsonProperty("is_moderated")
public boolean isModerated() {
return this.moderated;
}
public void setModerated(final boolean moderated) {
this.moderated = moderated;
}
@Override
public String toString() {
return "OpenRouterTopProviderInfo [contextLength=" + contextLength + ", maxCompletionTokens=" + maxCompletionTokens + ", moderated=" + moderated + "]";
}
}
static final class OpenRouterPerRequestLimits {
private Integer promptTokens;
private Integer completionTokens;
@JsonProperty("prompt_tokens")
public Integer getPromptTokens() {
return this.promptTokens;
}
public void setPromptTokens(final Integer promptTokens) {
this.promptTokens = promptTokens;
}
@JsonProperty("completion_tokens")
public Integer getCompletionTokens() {
return this.completionTokens;
}
public void setCompletionTokens(final Integer completionTokens) {
this.completionTokens = completionTokens;
}
@Override
public String toString() {
return "OpenRouterPerRequestLimits [promptTokens=" + promptTokens + ", completionTokens=" + completionTokens + "]";
}
}
static final class OpenRouterModelLinks {
private String details;
@JsonProperty("details")
public String getDetails() {
return this.details;
}
public void setDetails(final String details) {
this.details = details;
}
@Override
public String toString() {
return "OpenRouterModelLinks [details=" + details + "]";
}
}
@Override
public String toString() {
return "OpenRouterModelsResponse [data=" + data + "]";
}
}

View File

@@ -21,11 +21,11 @@ public record ModelProfile(
String category) { String category) {
public ModelProfile { public ModelProfile {
if (maxTokens <= 0) { if (maxTokens < 0) {
throw new IllegalArgumentException("Model profile maxTokens must be greater than zero"); throw new IllegalArgumentException("Model profile maxTokens must be greater than or equal to zero");
} }
if (maxContextLength <= 0) { if (maxContextLength < 0) {
throw new IllegalArgumentException("Model profile maxContextLength must be greater than zero"); throw new IllegalArgumentException("Model profile maxContextLength must be greater than or equal to zero");
} }
if (temperature <= 0) { if (temperature <= 0) {
throw new IllegalArgumentException("Model profile temperature must be greater than zero"); throw new IllegalArgumentException("Model profile temperature must be greater than zero");
@@ -36,15 +36,12 @@ public record ModelProfile(
if (modelProvider == null) { if (modelProvider == null) {
throw new IllegalArgumentException("Model profile modelProvider is mandatory and cannot be null"); throw new IllegalArgumentException("Model profile modelProvider is mandatory and cannot be null");
} }
if (url == null || url.isBlank()) {
throw new IllegalArgumentException("Model profile url is mandatory and cannot be empty");
}
if (category == null || category.isBlank()) { if (category == null || category.isBlank()) {
throw new IllegalArgumentException("Model profile category is mandatory and cannot be empty"); throw new IllegalArgumentException("Model profile category is mandatory and cannot be empty");
} }
} }
public static enum ModelProvider { public static enum ModelProvider {
OAI_TEXT_COMPLETION, OAI_CHAT_COMPLETION, OLLAMA; OAI_TEXT_COMPLETION, OAI_CHAT_COMPLETION, OPENROUTER, OLLAMA;
} }
} }

View File

@@ -49,11 +49,11 @@ public class ModelProfileService {
final var maxTokens = Optional.ofNullable(resource.getProperty("maxTokens")) final var maxTokens = Optional.ofNullable(resource.getProperty("maxTokens"))
.map(Object::toString) .map(Object::toString)
.map(Integer::parseInt) .map(Integer::parseInt)
.orElseThrow(() -> new IllegalStateException("No model maxTokens found for model: " + modelName)); .orElse(Integer.valueOf(0));
final var maxContextLength = Optional.ofNullable(resource.getProperty("maxContextLength")) final var maxContextLength = Optional.ofNullable(resource.getProperty("maxContextLength"))
.map(Object::toString) .map(Object::toString)
.map(Integer::parseInt) .map(Integer::parseInt)
.orElseThrow(() -> new IllegalStateException("No model maxContextLength found for model: " + modelName)); .orElse(Integer.valueOf(0));
final var temperature = Optional.ofNullable(resource.getProperty("temperature")) final var temperature = Optional.ofNullable(resource.getProperty("temperature"))
.map(Object::toString) .map(Object::toString)
.map(Double::parseDouble) .map(Double::parseDouble)
@@ -71,7 +71,7 @@ public class ModelProfileService {
.orElse(null); .orElse(null);
final var url = Optional.ofNullable(resource.getProperty("url")) final var url = Optional.ofNullable(resource.getProperty("url"))
.map(Object::toString) .map(Object::toString)
.orElseThrow(() -> new IllegalStateException("No model url found for model: " + modelName)); .orElse(null);
final var costPerInToken = Optional.ofNullable(resource.getProperty("cost1MInTokens")) final var costPerInToken = Optional.ofNullable(resource.getProperty("cost1MInTokens"))
.map(Object::toString) .map(Object::toString)
.map(value -> new BigDecimal(value)) .map(value -> new BigDecimal(value))

View File

@@ -0,0 +1,117 @@
package de.extio.lmlib.client.openrouter;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertTrue;
import java.util.List;
import org.junit.jupiter.api.Disabled;
import org.junit.jupiter.api.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.SpringBootConfiguration;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.boot.test.context.SpringBootTest.WebEnvironment;
import org.springframework.test.context.DynamicPropertyRegistry;
import org.springframework.test.context.DynamicPropertySource;
import org.springframework.test.context.TestPropertySource;
import de.extio.lmlib.client.ClientService;
import de.extio.lmlib.client.Conversation;
import de.extio.lmlib.grader.Grader2;
import de.extio.lmlib.profile.ModelProfile;
import de.extio.lmlib.profile.ModelProfileService;
@Disabled("This test requires a valid OpenRouter API key")
@SpringBootTest(webEnvironment = WebEnvironment.NONE)
@SpringBootConfiguration
@EnableAutoConfiguration
@TestPropertySource("classpath:/application-test.properties")
public class OpenRouterClientTest {
private final static String API_KEY = ""; // Enter your OpenRouter API key here for testing
private static final Logger LOGGER = LoggerFactory.getLogger(OpenRouterClientTest.class);
@DynamicPropertySource
static void registerProperties(final DynamicPropertyRegistry registry) {
registry.add("OPENROUTER_API_KEY", () -> API_KEY);
}
@Autowired
private OpenRouterClient openRouterClient;
@Autowired
private OpenRouterModelDiscovery openRouterModelDiscovery;
@Autowired
private ModelProfileService modelProfileService;
@Autowired
private ClientService clientService;
@Test
void modelDiscovery() throws Exception {
final var modelProfile = this.modelProfile();
final var models = this.openRouterModelDiscovery.listModels(modelProfile, new OpenRouterModelQuery(true, null, List.of(), List.of()), true);
LOGGER.info("Models: {}", models);
assertFalse(models.isEmpty());
assertFalse(models.getFirst().getId().isBlank());
final var resolvedModelName = models.getFirst().getId();
LOGGER.info("Resolved model name: {}", resolvedModelName);
final var metadata = this.openRouterModelDiscovery.resolveModelMetadata(modelProfile, resolvedModelName, true);
LOGGER.info("Metadata: {}", metadata);
assertEquals(resolvedModelName, metadata.modelId());
assertTrue(metadata.contextLength() > 0);
assertEquals(ModelProfile.ModelProvider.OPENROUTER, this.clientService.getClient(modelProfile).getModelProvider());
}
@Test
void chatCompletion() throws Exception {
final var modelProfile = this.modelProfile();
final var completion = this.openRouterClient.conversation(
modelProfile,
Conversation.create("You are a helpful assistant", "Say the color is green"),
null,
false);
LOGGER.info(completion.reasoning());
LOGGER.info(completion.response());
assertTrue(Grader2.assessScoreBinary("Does the text mention the color green?", completion.response(), modelProfile, this.clientService));
}
@Test
void streamChatCompletion() throws Exception {
final var modelProfile = this.modelProfile();
final var conversation = Conversation.create("You are a helpful assistant", "I would like to play a game of chess.");
conversation.addTurn(new Conversation.Turn(Conversation.TurnType.ASSISTANT, "Which opening do you choose?"));
conversation.addTurn(new Conversation.Turn(Conversation.TurnType.USER, "You choose, but explain the opening to me."));
final var responseBuilder = new StringBuilder();
final var reasoningBuilder = new StringBuilder();
final var completion = this.openRouterClient.streamConversation(modelProfile, conversation, chunk -> {
if (chunk.reasoningContent() != null) {
System.out.print(chunk.reasoningContent());
reasoningBuilder.append(chunk.reasoningContent());
}
if (chunk.content() != null) {
System.out.print(chunk.content());
responseBuilder.append(chunk.content());
}
}, null, false);
assertEquals(completion.response(), responseBuilder.toString());
if (completion.reasoning() != null) {
assertEquals(completion.reasoning(), reasoningBuilder.toString());
}
assertTrue(Grader2.assessScoreBinary("Does the text explain a chess opening?", completion.response(), modelProfile, this.clientService));
}
private ModelProfile modelProfile() {
return this.modelProfileService.getModelProfile("profile.model.openrouter");
}
}

View File

@@ -7,6 +7,7 @@ profile.model.cold=local-chatcompletion
profile.model.textcompletion=local-completion profile.model.textcompletion=local-completion
profile.model.chatcompletion=local-chatcompletion profile.model.chatcompletion=local-chatcompletion
profile.model.ollama=local-ollama-chatcompletion profile.model.ollama=local-ollama-chatcompletion
profile.model.openrouter=local-openrouter-chatcompletion
#profile.model.azure=llama3.3-70b-azure #profile.model.azure=llama3.3-70b-azure
agent.threads=2 agent.threads=2

View File

@@ -0,0 +1,16 @@
category=OPENROUTER
prompts=
tokenizer=fallback
tokenEncoding=cl100k_base
maxTokens=2500
maxContextLength=32768
temperature=0.5
topP=0.95
modelProvider=OPENROUTER
modelName=
url=
apiKey=${OPENROUTER_API_KEY:}
cost1MInTokens=0
cost1MCachedInTokens=0
cost1MOutTokens=0
cost1MReasoningOutTokens=0