exo/lm-lib
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: add Anthropic model discovery and capability probing

Browse Source 
Uses GET /v1/models to list models and read advertised capabilities.
supportsToolCalling and extended-thinking requests honor the listed
capabilities (including tool_use/tool_calling on compatible servers).
Model listing is cached per profile; unreachable endpoints or missing
capabilities skip the checks rather than failing requests.
This commit is contained in:
extio-de
2026-10-17 20:39:04 +02:00
parent bb7ae456a8
commit cd99dfa360
9 changed files with 303 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
README.md
View File

@@ -514,7 +514,7 @@ lm-lib caches `/api/show` model details per resolved model. Thinking and tool de
`ANTHROPIC` targets the Anthropic Claude Messages API (`POST /v1/messages`), including Anthropic-compatible servers that expose the same endpoint shape.
The client sends system turns as the top-level `system` parameter, uses `x-api-key` and `anthropic-version` headers, and requires a positive `maxTokens` in the profile because the Messages API mandates `max_tokens`. Tool calling uses native `tool_use`/`tool_result` content blocks, and streamed responses are consumed through the standard server-sent event protocol (`message_start`, `content_block_delta`, `message_delta`, `message_stop`). Extended thinking is supported: thinking blocks are captured as reasoning and replayed on follow-up tool rounds. There is no model discovery endpoint, so `getModelNames(...)` returns an empty list and `modelName` must be configured.
The client sends system turns as the top-level `system` parameter, uses `x-api-key` and `anthropic-version` headers, and requires a positive `maxTokens` in the profile because the Messages API mandates `max_tokens`. Tool calling uses native `tool_use`/`tool_result` content blocks, and streamed responses are consumed through the standard server-sent event protocol (`message_start`, `content_block_delta`, `message_delta`, `message_stop`). Extended thinking is supported: thinking blocks are captured as reasoning and replayed on follow-up tool rounds. Model discovery uses `GET /v1/models` (cached per profile, refreshed with `forceReload`); `modelName` may be left empty to use the first listed model. Advertised model `capabilities` are used like Ollama's `/api/show` probing: `supportsToolCalling(...)` and extended-thinking requests honor the listed capabilities (including `tool_use`/`tool_calling` on compatible servers), while models or servers without capabilities or with an unreachable models endpoint simply skip these checks.
To enable extended thinking, register an `AnthropicDialect` bean and return a budget (at least 1024 tokens) from `thinkingBudgetTokens(...)`; the budget is clamped below the profile's `maxTokens`. Omitting the bean or returning `null` keeps thinking disabled.