feat: lazy per-route cache seeding for Workers

[NathanDrake2406]

Summary

• On Workers with MemoryCacheHandler (the default when KV is not configured), pre-rendered routes now lazily seed the memory cache on first request via env.ASSETS.fetch()
• Seeding is per-route (not eager startup) to respect Workers' ephemeral isolate model and 128MB memory limit
• Concurrent cold hits for the same route are deduped via an in-flight promise map

Pre-requisite

Depends on #645 for the manifest format changes (buildId, router, trailingSlash fields).

How it works

1. Deploy time (deploy.ts): After prerendering, copies vinext-prerender.json and HTML/RSC files to dist/client/__prerender/ so they're deployed as static assets
2. Runtime (seed-cache-workers.ts): On each request, checks if the route has prerendered data. On first miss, fetches HTML/RSC from the assets binding and populates MemoryCacheHandler. Subsequent requests in the same isolate get cache HITs
3. Generated worker entry: Calls seedRouteFromAssets() before delegating to the RSC handler

Changes

File	Change
src/server/seed-cache-workers.ts	New — lazy per-route seeding with manifest caching + concurrent dedup
src/deploy.ts	Copy prerender files to assets dir + add seeding call to generated worker entry
package.json	Export ./server/seed-cache-workers
tests/seed-cache-workers.test.ts	9 unit tests

Scope

This is Workers memory stub support only, not the final durable caching story. For persistent caching, users should use KVCacheHandler (opt-in) with TPR for deploy-time KV population. Issue #562 tracks the generalised remote cache seeding.

Ref #561

Test plan

• 9 unit tests: basic seeding, index route, dynamic routes, missing manifest, non-prerendered routes, already-cached no-op, concurrent dedup, manifest caching, missing RSC graceful degradation
• 216 deploy tests pass unchanged
• Type-aware lint clean

[pkg-pr-new]

Open in StackBlitz

npm i https://pkg.pr.new/vinext@653

commit: 8f46d7f

[james-elicx]

/bigbonk review

[ask-bonk]

Good PR — the lazy per-route seeding approach is well-suited for Workers' ephemeral isolate model, and the shared extraction from seed-cache.ts is a clean refactor. The test coverage is solid. A few issues to address, one of which could cause request failures in production.

Summary of findings:

1. P0 (error resilience): doSeedRoute has no try/catch, and the generated worker entry awaits without a safety net — a transient env.ASSETS.fetch() failure or handler.set() error would crash the request instead of gracefully degrading.
2. P1 (permanent null caching): loadManifest permanently caches null on transient fetch failure — the isolate can never seed any route after a single hiccup.
3. Minor: copyPrerenderToAssets flattens the manifest into the same directory as the route files, which works but could cause filename collisions on a route literally named /vinext-prerender (extremely unlikely, but worth noting).

[ask-bonk]

Error resilience: doSeedRoute can throw on fetchAsset() network errors, handler.set() failures, etc. Since the generated worker entry (line 503 of deploy.ts) does await seedRouteFromAssets(...) without try/catch, any throw here becomes an unhandled error that crashes the request with a 500.

Seeding is a best-effort optimization — a failure should log and degrade gracefully, not break the request. Wrap this in try/catch:

Suggested change

	).finally(() => seedInFlight.delete(htmlKey));
	const promise = doSeedRoute(
	loaded.manifest,
	pathname,
	baseKey,
	revalidateSeconds,
	fetchAsset,
	).catch(() => {/* seeding is best-effort */}).finally(() => seedInFlight.delete(htmlKey));

Alternatively (and perhaps better), wrap the entire seedRouteFromAssets body in try/catch so that any unexpected error — including from getCacheHandler().get() — is swallowed. The loadManifest function already follows this pattern.

[ask-bonk]

Permanent null caching on transient failure: If fetchAsset("/__prerender/vinext-prerender.json") fails transiently (e.g., a brief ASSETS binding hiccup, timeout), loadedPromise is permanently set to a promise that resolves to null. Every subsequent call to seedRouteFromAssets in this isolate will get null and no-op — the isolate can never recover.

Since the manifest is immutable per deployment, a permanent cache of the successful result is correct. But caching the failure permanently is not — a retry on the next request would likely succeed.

Consider only caching successful results:

Suggested change

	return null;
	loadedPromise = (async () => {
	try {
	const res = await fetchAsset("/__prerender/vinext-prerender.json");
	if (!res.ok) return null;
	const manifest: PrerenderManifest = await res.json();
	if (!manifest.buildId || !Array.isArray(manifest.routes)) return null;
	const lookup = new Map<string, PrerenderManifestRoute>();
	for (const route of manifest.routes) {
	if (route.status !== "rendered") continue;
	if (route.router !== "app") continue;
	lookup.set(route.path ?? route.route, route);
	}
	return { manifest, lookup };
	} catch {
	loadedPromise = null; // allow retry on next request
	return null;
	}
	})();

Note: the !res.ok and validation-failure cases are arguably permanent (malformed deployment), so you could choose to only reset loadedPromise in the catch branch. Either way, the catch case (network/timeout) should not be permanently cached.

[ask-bonk]

Nit: fs.cpSync(sourceDir, targetDir, { recursive: true }) copies the contents of sourceDir into targetDir where targetDir already exists. This means vinext-prerender.json (copied on line 1219) and route files (e.g., about.html) all live at the same level under __prerender/. This works, but means a pre-rendered route literally named /vinext-prerender (producing vinext-prerender.html) would coexist fine — just calling it out.

A more robust layout would be __prerender/manifest.json + __prerender/routes/..., but that's a future cleanup, not a blocker.

[ask-bonk]

The rscData parameter is typed as ArrayBuffer, which matches the CachedAppPageValue field type. But in seed-cache.ts:142-145, the Node.js codepath does rscBuffer.buffer.slice(byteOffset, byteOffset + byteLength) to extract a clean ArrayBuffer from the Node Buffer. Worth verifying that the ArrayBuffer from Response.arrayBuffer() in the Workers path (line 179) doesn't need similar treatment — in practice Workers' Response.arrayBuffer() returns a standalone ArrayBuffer with byteOffset === 0, so this should be fine. Just flagging the asymmetry.

[ask-bonk]

This await is on the critical path of every request — it blocks the response until seeding completes for pre-rendered routes. For the first cold request this adds the latency of two env.ASSETS.fetch() calls (HTML + RSC) plus two handler.set() calls.

This is probably acceptable since env.ASSETS.fetch() is local to the colo and fast, but worth a comment noting that this is intentionally blocking (to guarantee the cache is populated before the RSC handler runs). An alternative would be to let the RSC handler serve the request normally while seeding in the background via ctx.waitUntil(), but that would mean the first request always re-renders — so blocking is the right call here.

Also: should this be wrapped in try/catch as defense-in-depth? If seedRouteFromAssets throws (see other comment), the request dies.

Suggested change

	await seedRouteFromAssets(url.pathname, (p) => env.ASSETS.fetch(new Request(new URL(p, request.url))));
	// Seed the memory cache from pre-rendered assets (lazy, per-route).
	// On first request for a pre-rendered page, fetches HTML/RSC from the
	// assets binding and inserts into the cache. Subsequent requests in the
	// same isolate get cache HITs. No-op for non-prerendered routes.
	// Blocking (not waitUntil) so the RSC handler sees the seeded cache.
	try {
	await seedRouteFromAssets(url.pathname, (p) => env.ASSETS.fetch(new Request(new URL(p, request.url))));
	} catch { /* seeding is best-effort — fall through to normal render */ }

[ask-bonk]

Good test, but the dedup assertion depends on the mock fetcher resolving synchronously (via microtask). In a real Workers environment, env.ASSETS.fetch() would yield to the event loop, and the timing of when the second caller hits the seedInFlight.get() check vs the getCacheHandler().get() check could differ.

Consider adding a test variant where the mock fetcher introduces a small delay (e.g., a Promise that resolves after a setTimeout(0)) to verify the dedup still works when fetches genuinely yield. This would give more confidence in the concurrency model.

[ask-bonk]

Review submitted successfully. Here's a summary of the feedback I posted:

Key findings:

1. P0 — Error resilience: doSeedRoute lacks error handling, and the generated worker entry awaits seedRouteFromAssets without try/catch. Any transient env.ASSETS.fetch() failure or handler.set() error would crash the request with a 500 instead of gracefully falling through to normal rendering. Suggested wrapping in try/catch both in doSeedRoute (via .catch()) and in the generated worker entry.

2. P1 — Permanent null caching in loadManifest: If the manifest fetch fails transiently (timeout, brief ASSETS binding hiccup), loadedPromise is permanently set to a null-resolving promise. The isolate can never recover and seed any route. Suggested resetting loadedPromise = null in the catch branch to allow retries.

3. Minor observations:

   • The copyPrerenderToAssets layout puts manifest and route files at the same level (works fine, just noting the design choice)
   • The ArrayBuffer handling asymmetry between Node.js (Buffer.slice) and Workers (Response.arrayBuffer()) paths — not a bug, just worth being aware of
   • The blocking await on the critical path is the correct choice (vs waitUntil) but should be defended with try/catch
   • The concurrency dedup test could be strengthened with a delayed mock fetcher to better simulate real async behavior

github run