Caching architecture

Summary#

Metric Vault is a thin orchestration layer over expensive third-party APIs, so almost every cache in the system exists to reduce a provider bill rather than to reduce latency. There are two independent cache layers on the main tool path and seven further feature caches, all of them tables in the one D1 database bound as MONITOR_DB.

Conflating the two main layers is the single most common source of confusion, so read the distinction first: mv_seo_cache stores a whole tool response and short-circuits the handler entirely, while dfs_cache stores a raw DataForSEO endpoint response and short-circuits one provider call inside a handler that still runs. Only the second of the two is reachable from the admin console.

Purpose#

Two things are true at once, and the design has to serve both.

Provider calls cost real money. A DataForSEO endpoint reports its own price back in the response, and the LLM providers bill per token. If two customers research the same domain on the same day, paying twice for identical data is pure waste.

A customer who runs a report has run a report. The credit they spend is the price of the product, not a pass-through of our provider cost. So the cache saves our bill, never the customer's credit. The design statement is written into the code at the top of the shared-cache module: "On a cache hit the customer is still charged their normal credit (they ran a report); only our provider bill is saved."

The customer-facing explanation of what that means in practice, including the freshness a reader should expect from each tool, lives in Result caching and freshness.

Architecture#

   POST /api/premium-ai
        │
        ▼
   mvSaveAroundPremiumAI          ── LAYER A ──────────────────────
        │   read  mv_seo_cache (key: vN|<params>)                  │
        │   HIT  → quota gate → charge credit → return stored body │
        │          X-MV-Cache: hit, X-MV-Cache-Age-Days: <n>       │
        │   MISS ↓                                                 │
        ──────────────────────────────────────────────────────────
        ▼
   handlePremiumAI  (kill switch, plan gate, quota, incrementUsage)
        │
        ▼
   callDataForSEOCached(endpoint, body, env, ttlSec, tool)
        │                          ── LAYER B ──────────────────────
        │   read  dfs_cache (key: <endpoint>:<json body>)          │
        │   HIT  → return stored provider payload                  │
        │   MISS → callDataForSEO → INSERT OR REPLACE              │
        ──────────────────────────────────────────────────────────
        ▼
   LLM provider call (never cached at this layer)
        │
        ▼
   write-through to mv_seo_cache · save to tool_results · metrics

Everything is D1. The KV namespace QUICKVIEW_CACHE is declared in wrangler.toml and referenced by nothing, because KV bindings on this Pages project reset on every deploy; the code comment that records that decision sits next to the quickview handler. Treat any documentation mentioning "the KV cache" as stale. See Data model.

Components#

Layer A: mv_seo_cache, the cross-customer tool-result cache#

Implemented entirely inside mvSaveAroundPremiumAI, the wrapper the router puts around POST /api/premium-ai.

Three preconditions must all hold before a lookup happens: the tool must appear in the MV_CACHE_TTL_MS map, the request must not carry forceFresh or _mvForceFresh, and body.query must be truthy. A tool absent from the map is never served from this cache, which makes the map the rollout control.

The MV_CACHE_TTL_MS map#

Thirty-nine tools are cached at this layer. The TTL is per tool because the underlying data changes at different speeds.

Layer B: dfs_cache, the raw DataForSEO endpoint cache#

Every cached provider call goes through callDataForSEOCached(endpoint, body, env, ttlSec, tool).

TTL resolution order, implemented in mvCacheTtlSec:

1. platform_config['cache_days:<tool>'], if the tool argument was passed and that key exists.
2. platform_config['cache_days_default'].
3. The caller's literal ttlSec argument.
4. 86400 (one day) when ttlSec is not a positive number.

Any error, unparseable value or value of zero or less falls through to the default. The code comment states the intent plainly: a bad config value can never break the core cache path or change data correctness, only freshness.

The remaining seven caches#

Edge cache#

caches.default is used for four static proxy routes only, never for application data: /blog, /legal, the two logo PNGs, and nine mapped /free-tools/* paths. The HTML routes cache for 300 seconds and the logos for 3,600.

Data flow#

A Layer A hit#

1. getToolCost(type) resolves the credit cost.
2. enforceAiQuota runs. The quota gate is not skipped.
3. incrementUsage runs. The customer is charged the full credit.
4. mvSeoCacheBumpHits increments the row's hits column.
5. mvBumpMetrics records runs: 1 and cache_hit: 1 on scope tool:<type>, and runs: 1 on scope all.
6. The stored body is returned with X-MV-Cache: hit, X-MV-Cache-Age-Days: <floor((now - fetched_at) / 86400000)> and an Access-Control-Expose-Headers list so the browser can read both.

A Layer A miss#

The real handler runs, metrics record cache_miss: 1, and on success the entire response text is written through with mvSeoCachePut. Only real payloads are stored: the write is skipped when the body has no data field or carries an error, so a degraded response can never be pinned for ten days.

Effect on credits and cost#

Administrator overrides#

The Tools and Cache panel controls Layer B only.

TOOL_CACHE_TTLS is the map the admin console reads to show real lifetimes. It holds the actual callDataForSEOCached TTL literals for 31 tools, taken from a source audit rather than guessed, and a tool can appear with several values because it hits several endpoints. A tool absent from that map makes no cached DataForSEO calls at all and is displayed as Not cached.

Eight tools cannot be tuned individually: seo_brief, article_outline, article_writer, content_template, content_optimizer, content_repurpose, meta_generator and press_pitch. All eight route through the single fetcher fetchRealContentContext, which calls callDataForSEOCached without the fifth tool argument, so their rows carry tool = NULL and are shared. The UI labels them Set via Apply to all with the tooltip Shares one cached fetch with related tools — use Apply to all above. The operator's view of all this is in Tools and cache management.

Failure modes#

The cache sweeper#

sweep_caches (mvSweepCaches) runs on every cron tick and deletes expired rows from all nine cache tables. Before it existed none of them had a delete path of any kind, and because they share one D1 instance with the live application tables, their growth was everyone's problem rather than the caches' own. idx_seo_cache_expires had been sitting on mv_seo_cache(expires_at) the whole time — an index whose only purpose is to make exactly this sweep cheap.

Each table keeps its own rule. Flattening them to one number would delete rows an operator deliberately configured to live longer, and the only symptom would be a larger provider bill.

Every delete is chunked at 5,000 rows per statement, the same shape as purge_watchtower, so a first run against a large backlog drains over several ticks instead of attempting one long-running statement.

Once every six hours the sweep also records a snapshot of what each table holds into platform_config under cache_sweep_last, readable through POST /api/admin/cache/stats. Nothing else reports these sizes, which is how nine tables came to have no delete path without anyone noticing. COUNT(*) is a scan, hence the six-hour throttle rather than a per-tick gauge.

See Background jobs and scheduling for the job registry and Data model for the full retention matrix.

Forcing a fresh result#

forceFresh on the request body bypasses Layer A only. Layer B still applies, so a forced run can still return provider data that is up to its TTL old. The recommendations panel sets forceFresh: true on every call, because its query is a findings blob rather than a user query and caching it would be meaningless.