-
Notifications
You must be signed in to change notification settings - Fork 12
Description
Summary
The OpenRouter API returns cost and cost_details in the usage object of chat completion responses, but the SDK's ChatGenerationTokenUsage model doesn't define these fields, so they get silently dropped during deserialization.
API response (raw HTTP)
{
"usage": {
"prompt_tokens": 158,
"completion_tokens": 152,
"total_tokens": 310,
"cost": 8.41005e-05,
"is_byok": false,
"cost_details": {
"upstream_inference_cost": 8.495e-05,
"upstream_inference_prompt_cost": 8.95e-06,
"upstream_inference_completions_cost": 7.6e-05
},
"completion_tokens_details": { "..." : "..." },
"prompt_tokens_details": { "..." : "..." }
}
}SDK result after model_dump()
{
"completion_tokens": 152,
"prompt_tokens": 158,
"total_tokens": 310,
"completion_tokens_details": { ... },
"prompt_tokens_details": { ... }
# cost and cost_details are gone
}Root cause
ChatGenerationTokenUsage in src/openrouter/components/chatgenerationtokenusage.py only defines completion_tokens, prompt_tokens, total_tokens, completion_tokens_details, and prompt_tokens_details. The cost and cost_details fields aren't in the OpenAPI spec, so they're not generated.
Expected fix
Add to the OpenAPI spec (ChatGenerationTokenUsage schema):
cost:
type: number
nullable: true
description: "Total cost of the generation in USD"
cost_details:
type: object
nullable: true
properties:
upstream_inference_cost:
type: number
nullable: true
upstream_inference_prompt_cost:
type: number
nullable: true
upstream_inference_completions_cost:
type: number
nullable: true
description: "Breakdown of generation cost"Verified locally
I patched the installed SDK to add cost: OptionalNullable[float] and cost_details: OptionalNullable[CostDetails] to ChatGenerationTokenUsage and confirmed the data flows through correctly after model_dump().
Impact
This blocks downstream packages (e.g. langchain-openrouter) from surfacing cost data, even though the langchain integration already has code to read cost from the usage dict. The SDK silently drops it before langchain ever sees it.