Skip to content

Fix cost and token tracking between provider styles#8954

Merged
mrubens merged 1 commit intomainfrom
fix_cost_and_token_tracking
Oct 31, 2025
Merged

Fix cost and token tracking between provider styles#8954
mrubens merged 1 commit intomainfrom
fix_cost_and_token_tracking

Conversation

@mrubens
Copy link
Collaborator

@mrubens mrubens commented Oct 31, 2025

Before:
Screenshot 2025-10-31 at 10 53 41 AM

After:
Screenshot 2025-10-31 at 10 53 56 AM

This also fixes the real-time stats and telemetry sent to cloud.


Important

Refactor cost and token tracking across provider handlers to ensure consistent and accurate reporting for Anthropic and OpenAI protocols.

  • Cost Calculation:
    • Refactor calculateApiCostAnthropic and calculateApiCostOpenAI in cost.ts to return ApiCostResult with totalInputTokens, totalOutputTokens, and totalCost.
    • Update cost calculation logic in calculateApiCostInternal() to handle both Anthropic and OpenAI protocols.
  • Provider Handlers:
    • Update AnthropicHandler, CerebrasHandler, DeepInfraHandler, GroqHandler, LiteLLMHandler, OpenAiNativeHandler, and RequestyHandler to use the new cost calculation functions.
    • Ensure totalCost is extracted and used consistently across handlers.
  • Token Usage Tracking:
    • Modify getApiMetrics() in getApiMetrics.ts to calculate contextTokens using tokensIn and tokensOut.
    • Update Task.ts to handle cost and token tracking using the new ApiCostResult structure.
  • Testing:
    • Update tests in cost.spec.ts to validate new cost calculation logic for both Anthropic and OpenAI.

This description was created by Ellipsis for 6be9762. You can customize this summary. It will automatically update as commits are pushed.

@mrubens mrubens requested review from cte and jr as code owners October 31, 2025 14:54
@dosubot dosubot bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Oct 31, 2025
@roomote
Copy link
Contributor

roomote bot commented Oct 31, 2025

Review Summary

No issues found - This PR successfully fixes cost and token tracking between provider styles.

Changes Reviewed

Core Changes:

Provider Updates:

Test Coverage:

  • All tests in cost.spec.ts updated to verify new return structure

Verification

✅ Type safety maintained across all changes
✅ Correct token calculation for both Anthropic (excludes cache from base input) and OpenAI (includes cache in base input)
✅ All call sites updated consistently
✅ Test coverage updated appropriately
✅ Telemetry and cloud stats will now receive accurate total token counts

View Job

@dosubot dosubot bot added the bug Something isn't working label Oct 31, 2025
@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Oct 31, 2025
@roomote
Copy link
Contributor

roomote bot commented Oct 31, 2025

Review TODOs

  • Relax ParsedApiReqStartedTextType fields to optional in getApiMetrics.ts to match actual payloads; current implementation uses typeof guards implying keys can be absent.

Follow Along on Roo Code Cloud

@hannesrudolph hannesrudolph added the Issue/PR - Triage New issue. Needs quick review to confirm validity and assign labels. label Oct 31, 2025
@daniel-lxs daniel-lxs moved this from Triage to PR [Needs Review] in Roo Code Roadmap Oct 31, 2025
// For OpenAI (or when protocol is not specified).
result.contextTokens = (tokensIn || 0) + (tokensOut || 0)
}
const { tokensIn, tokensOut } = parsedText
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ParsedApiReqStartedTextType marks tokensIn/tokensOut/cacheWrites/cacheReads as required, but this block guards for missing fields with typeof checks. api_req_started payloads may not always include these keys (e.g., before usage aggregation), so the type should make them optional to match runtime reality and prevent misleading typings. Suggested change: make these fields optional in getApiMetrics.ts.

@hannesrudolph hannesrudolph added PR - Needs Review and removed Issue/PR - Triage New issue. Needs quick review to confirm validity and assign labels. labels Oct 31, 2025
@mrubens mrubens merged commit 416fa57 into main Oct 31, 2025
26 checks passed
@mrubens mrubens deleted the fix_cost_and_token_tracking branch October 31, 2025 19:14
@github-project-automation github-project-automation bot moved this from New to Done in Roo Code Roadmap Oct 31, 2025
@github-project-automation github-project-automation bot moved this from PR [Needs Review] to Done in Roo Code Roadmap Oct 31, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working lgtm This PR has been approved by a maintainer PR - Needs Review size:L This PR changes 100-499 lines, ignoring generated files.

Projects

No open projects
Archived in project

Development

Successfully merging this pull request may close these issues.

4 participants