Fix cost and token tracking between provider styles#8954
Conversation
Review Summary✅ No issues found - This PR successfully fixes cost and token tracking between provider styles. Changes ReviewedCore Changes:
Provider Updates: Test Coverage:
Verification✅ Type safety maintained across all changes |
Review TODOs
|
| // For OpenAI (or when protocol is not specified). | ||
| result.contextTokens = (tokensIn || 0) + (tokensOut || 0) | ||
| } | ||
| const { tokensIn, tokensOut } = parsedText |
There was a problem hiding this comment.
ParsedApiReqStartedTextType marks tokensIn/tokensOut/cacheWrites/cacheReads as required, but this block guards for missing fields with typeof checks. api_req_started payloads may not always include these keys (e.g., before usage aggregation), so the type should make them optional to match runtime reality and prevent misleading typings. Suggested change: make these fields optional in getApiMetrics.ts.
Before:

After:

This also fixes the real-time stats and telemetry sent to cloud.
Important
Refactor cost and token tracking across provider handlers to ensure consistent and accurate reporting for Anthropic and OpenAI protocols.
calculateApiCostAnthropicandcalculateApiCostOpenAIincost.tsto returnApiCostResultwithtotalInputTokens,totalOutputTokens, andtotalCost.calculateApiCostInternal()to handle both Anthropic and OpenAI protocols.AnthropicHandler,CerebrasHandler,DeepInfraHandler,GroqHandler,LiteLLMHandler,OpenAiNativeHandler, andRequestyHandlerto use the new cost calculation functions.totalCostis extracted and used consistently across handlers.getApiMetrics()ingetApiMetrics.tsto calculatecontextTokensusingtokensInandtokensOut.Task.tsto handle cost and token tracking using the newApiCostResultstructure.cost.spec.tsto validate new cost calculation logic for both Anthropic and OpenAI.This description was created by
for 6be9762. You can customize this summary. It will automatically update as commits are pushed.