Skip to content

fix(opencode): set 1h prompt cache TTL for OpenRouter#16850

Open
rndmcnlly wants to merge 2 commits intoanomalyco:devfrom
rndmcnlly:fix/openrouter-cache-ttl
Open

fix(opencode): set 1h prompt cache TTL for OpenRouter#16850
rndmcnlly wants to merge 2 commits intoanomalyco:devfrom
rndmcnlly:fix/openrouter-cache-ttl

Conversation

@rndmcnlly
Copy link

@rndmcnlly rndmcnlly commented Mar 10, 2026

Issue for this PR

Closes #16848

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

Adds prompt_cache_ttl: 3600 alongside the existing prompt_cache_key for OpenRouter, extending cache lifetime from 5 minutes to 1 hour. Without it, any idle gap over 5 min between prompts triggers a full cache miss and rewrite.

Tested with curl against OpenRouter -- parameter is accepted for both Anthropic and non-Anthropic models. Non-Anthropic models silently ignore it. Anthropic models show cache hits (cached_tokens > 0) on follow-up requests.

How did you verify your code works?

  1. bun test test/provider/transform.test.ts -- 108 tests pass including the new one
  2. curl against OpenRouter with prompt_cache_key + prompt_cache_ttl: 3600 on anthropic/claude-sonnet-4 -- confirmed cache write on first request, cache hit on second
  3. curl against google/gemini-2.5-flash with same params -- 200 OK, param silently ignored

Screenshots / recordings

N/A -- no UI change.

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

@github-actions github-actions bot added needs:compliance This means the issue will auto-close after 2 hours. and removed needs:compliance This means the issue will auto-close after 2 hours. labels Mar 10, 2026
@github-actions
Copy link
Contributor

Thanks for updating your PR! It now meets our contributing guidelines. 👍

@rekram1-node
Copy link
Collaborator

Do they document this?

…ed prompt_cache_ttl

prompt_cache_ttl is silently ignored by OpenRouter. The documented approach
is cache_control: { type: "ephemeral", ttl: "1h" } on message content,
which empirically survives a 6-minute idle gap.
@rndmcnlly rndmcnlly force-pushed the fix/openrouter-cache-ttl branch from 732d370 to 2d2a432 Compare March 11, 2026 02:39
@rndmcnlly
Copy link
Author

Good question -- it caught a real bug.

Yes, they document it: https://openrouter.ai/docs/guides/best-practices/prompt-caching

Key quote from the Anthropic section:

By default, the cache expires after 5 minutes, but you can extend this to 1 hour by specifying "ttl": "1h" in the cache_control object.

The documented mechanism is cache_control: { type: "ephemeral", ttl: "1h" } on message content -- not a top-level prompt_cache_ttl parameter, which is what I originally submitted. prompt_cache_ttl is silently ignored (no error, no cache write, nothing).

I ran a 3-way empirical test with a 6-minute delay past the 5-min default TTL:

Approach                             R1 cached  R1 written  R2 cached  R2 written  Verdict
----------------------------------------------------------------------------------------------------
A (prompt_cache_ttl=3600)                    0           0          0           0  CACHE MISS
B (cache_control ttl=1h)                     0       13229      13229           0  CACHE HIT
C (baseline, no TTL ext)                     0           0          0           0  CACHE MISS

Only the documented cache_control approach survives. I've force-pushed a fix that adds ttl: "1h" to the existing cacheControl provider option on messages instead.

Verification script (~7 min, needs uv + OPENROUTER_API_KEY): https://gist.github.com/rndmcnlly/6680b63aeb26b5ea9ffbb56dab3030dc

Sorry for the original mistake -- my local test only checked that the parameter was accepted without error, not that it actually extended the TTL.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE]: Set prompt_cache_ttl for OpenRouter provider

2 participants