feat: Update model profile and statistics tracking
- Updated model profile to support new cost parameters: costPerCachedInToken and costPerReasoningOutToken. - Enhanced AgentRequestStatistic to track cached input tokens and reasoning output tokens. - Modified CompletionStatistics to include cached input tokens and reasoning output tokens. - Updated AzureAiClient, CachedClient, and other clients to handle new statistics fields. - Introduced Timings class to capture detailed timing metrics for requests. - Adjusted Usage class to include input and output token details. - Updated properties files to include new cost parameters for cached and reasoning tokens.
This commit is contained in:
@@ -10,4 +10,6 @@ modelName=Llama-3.3-70B-Instruct
|
||||
url=XXX
|
||||
apiKey=XXX
|
||||
cost1MInTokens=0.71
|
||||
cost1MCachedInTokens=0.71
|
||||
cost1MOutTokens=0.71
|
||||
cost1MReasoningOutTokens=0
|
||||
|
||||
@@ -10,6 +10,8 @@ modelName=
|
||||
url=http://localhost:5001
|
||||
apiKey=
|
||||
cost1MInTokens=0
|
||||
cost1MCachedInTokens=0
|
||||
cost1MOutTokens=0
|
||||
cost1MReasoningOutTokens=0
|
||||
reasoningEffort=medium
|
||||
reasoningSummaryDetails=auto
|
||||
@@ -10,4 +10,6 @@ modelName=
|
||||
url=http://localhost:5001
|
||||
apiKey=
|
||||
cost1MInTokens=0
|
||||
cost1MCachedInTokens=0
|
||||
cost1MOutTokens=0
|
||||
cost1MReasoningOutTokens=0
|
||||
|
||||
Reference in New Issue
Block a user