Skip to content

cost and cost_details missing from ChatGenerationTokenUsage schema #65

@untilhamza

Description

@untilhamza

Summary

The OpenRouter API returns cost and cost_details in the usage object of chat completion responses, but the SDK's ChatGenerationTokenUsage model doesn't define these fields, so they get silently dropped during deserialization.

API response (raw HTTP)

{
  "usage": {
    "prompt_tokens": 158,
    "completion_tokens": 152,
    "total_tokens": 310,
    "cost": 8.41005e-05,
    "is_byok": false,
    "cost_details": {
      "upstream_inference_cost": 8.495e-05,
      "upstream_inference_prompt_cost": 8.95e-06,
      "upstream_inference_completions_cost": 7.6e-05
    },
    "completion_tokens_details": { "..." : "..." },
    "prompt_tokens_details": { "..." : "..." }
  }
}

SDK result after model_dump()

{
    "completion_tokens": 152,
    "prompt_tokens": 158,
    "total_tokens": 310,
    "completion_tokens_details": { ... },
    "prompt_tokens_details": { ... }
    # cost and cost_details are gone
}

Root cause

ChatGenerationTokenUsage in src/openrouter/components/chatgenerationtokenusage.py only defines completion_tokens, prompt_tokens, total_tokens, completion_tokens_details, and prompt_tokens_details. The cost and cost_details fields aren't in the OpenAPI spec, so they're not generated.

Expected fix

Add to the OpenAPI spec (ChatGenerationTokenUsage schema):

cost:
  type: number
  nullable: true
  description: "Total cost of the generation in USD"
cost_details:
  type: object
  nullable: true
  properties:
    upstream_inference_cost:
      type: number
      nullable: true
    upstream_inference_prompt_cost:
      type: number
      nullable: true
    upstream_inference_completions_cost:
      type: number
      nullable: true
  description: "Breakdown of generation cost"

Verified locally

I patched the installed SDK to add cost: OptionalNullable[float] and cost_details: OptionalNullable[CostDetails] to ChatGenerationTokenUsage and confirmed the data flows through correctly after model_dump().

Impact

This blocks downstream packages (e.g. langchain-openrouter) from surfacing cost data, even though the langchain integration already has code to read cost from the usage dict. The SDK silently drops it before langchain ever sees it.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions