Data model
Every one of the 60 D1 tables with its columns and indexes, plus the Supabase tables, the unused KV namespace, the R2 bucket and the edge cache.
Last updated 2026-09-14
Summary#
Almost all state lives in one Cloudflare D1 database, bound as MONITOR_DB and named metricvault-monitor. It holds 60 tables covering billing and usage, the Library, nine separate cache tables, telemetry, configuration, teams, Competitor Monitor, alerts and schedules, sharing, integrations, AI-visibility tracking, async jobs and the whole blog platform.
Supabase holds identity plus a small number of REST-accessed tables. R2 holds blog images only. The declared KV namespace is bound and read by nothing.
This is a complete enumeration, not a sample. Every table below is created by a CREATE TABLE IF NOT EXISTS inside the worker; see Schema migrations for how and when.
Overview#
Six conventions apply across the whole database.
Timestamps are not consistent. Some tables store UNIX seconds (unixepoch()), others store JavaScript milliseconds (Date.now()). The unit is noted per table because this is the most frequent source of confusion. Milliseconds: activity_log.created_at, tool_results.created_at, mv_seo_cache.fetched_at and .expires_at, article_jobs.*, admin_users.created_at, social_oauth_state.created_at, higgs_oauth_state.created_at, provider_tokens.access_expires_at, and every mvb_* timestamp. Seconds: everything using unixepoch(), plus dfs_cache.cached_at, all tier2_*, error_issues.*, metrics_hourly.bucket_ts, admin_audit_log.ts and user_plans.set_at.
Emails are the identity key. There is no user ID table. user_email is the join key everywhere. Writes lowercase it in most places; reads generally use LOWER(email).
Reads tolerate a missing table. Because schema is created lazily, most reads sit inside a try/catch that returns an empty result. A first-ever call on a fresh deployment can legitimately hit a table that does not exist yet.
Deletes are mostly manual and mostly cascade by hand. SQLite foreign keys are not used. Where a parent delete should remove children, the handler issues the child deletes itself.
Only one table has an automatic retention job. tool_results is purged at 90 days by the purge_results cron job. Three more tables self-sweep inside their own schema function. Everything else grows without bound.
There is no account-deletion endpoint. No handler in _worker.js deletes a customer's data. Account deletion is a support-ticket process.
1. Billing, plans, usage and quota#
user_plans#
The single authoritative record of a paid plan.
| Column | Type | Meaning |
|---|---|---|
email | TEXT PRIMARY KEY | Always stored lowercase. Compared directly, so the lookup uses the key's index |
plan | TEXT NOT NULL DEFAULT 'starter' | One of free, starter, pro, agency, enterprise, unlimited |
set_at | INTEGER NOT NULL DEFAULT (unixepoch()) | seconds |
set_by | TEXT | Provenance: stripe, admin, seed. Load-bearing: a Stripe sync only overwrites rows whose set_by = 'stripe' |
notes | TEXT | For example billing_sync, admin panel |
Single writer: mvSetUserPlan, which lowercases the address before it touches the table. Every read compares email directly.
That matters for speed, not just tidiness. resolveUserPlan is the hottest read in the application — almost every gated route calls it — and it used to ask for WHERE LOWER(email) = ?. Wrapping the column in a function makes the primary key's index unusable, so the query full-scanned the table and got slower for everybody as it grew. Measured on this schema, 2000 lookups:
| Rows | LOWER(email) = ? | email = ? | Per lookup |
|---|---|---|---|
| 1,000 | 727.8 ms | 16.3 ms | 363.9 µs |
| 10,000 | 5,911.5 ms | 18.4 ms | 2,955.7 µs |
| 100,000 | 60,264.4 ms | 19.5 ms | 30,132.2 µs |
A hundred times the rows made the scan 83× slower and the indexed lookup 1.2×.
The reads could only drop LOWER() once the stored data was uniform. Two early versions of the writer bound the address raw, and Stripe returns the case the customer typed, so rows like Foo@Bar.com exist from that era. mvEnsurePlanEmailsLower folds them once per database, guarded by the user_plans_email_lc marker in platform_config. It collapses any pair that differs only by case before lowercasing — email is the primary key, so lowercasing two rows onto one value would fail the constraint — and keeps the most recent decision by set_at. tests/plan-lookup-index.mjs is the gate.
usage_counters#
The monthly credit meter. Primary key (user_email, month).
| Column | Type | Meaning |
|---|---|---|
user_email | TEXT NOT NULL | |
month | TEXT NOT NULL | YYYY-MM, UTC |
ai_runs | INTEGER DEFAULT 0 | Credits consumed by AI and premium runs |
tool_runs | INTEGER DEFAULT 0 | Credits consumed by tool runs |
plan | TEXT DEFAULT 'starter' | Never written with a real plan. Retained for analytics only |
last_run_at | INTEGER | seconds |
ai_reco_runs | INTEGER DEFAULT 0 | Added by ALTER. The "Get Recommendations" meter |
usage_by_tool#
Per-tool monthly breakdown. Primary key (user_email, month, tool_type). Columns: user_email TEXT, month TEXT, tool_type TEXT, credits INTEGER DEFAULT 0, runs INTEGER DEFAULT 0. Written only by incrementUsage and described in code as best-effort.
usage_hourly#
The light-tool abuse limiter. Primary key (user_email, hour_key).
| Column | Type | Meaning |
|---|---|---|
user_email | TEXT NOT NULL | Also carries the synthetic key 'ip:<CF-Connecting-IP>' for anonymous callers |
hour_key | TEXT NOT NULL | YYYY-MM-DD-HH, UTC |
light_calls | INTEGER NOT NULL DEFAULT 0 | |
last_run_at | INTEGER | seconds |
Limit: 100 calls per key per hour.
api_keys#
| Column | Type | Meaning |
|---|---|---|
key | TEXT PRIMARY KEY | The API key in plaintext. mv_live_ plus 48 hex characters |
user_email | TEXT NOT NULL | Owner |
name | TEXT | User label, capped at 60 characters |
created_at | INTEGER NOT NULL DEFAULT (unixepoch()) | seconds |
last_used_at | INTEGER | seconds |
revoked | INTEGER NOT NULL DEFAULT 0 | Soft delete. Rows are never removed |
Index idx_api_keys_email(user_email). Maximum 5 active keys per account; creation requires the Enterprise plan; the list endpoint returns only a mask.
2. Library and saved work#
tool_results#
What a run produced.
| Column | Type | Meaning |
|---|---|---|
id | INTEGER PRIMARY KEY AUTOINCREMENT | Returned to the browser as X-MV-Saved-Id |
user_email | TEXT NOT NULL | Lowercased on write |
tool_type | TEXT NOT NULL | Capped at 80 characters |
query | TEXT | Capped at 300 characters |
params_key | TEXT NOT NULL | Dedupe key from mvToolParamsKey |
result_json | TEXT NOT NULL | Serialized payload |
credits | INTEGER DEFAULT 0 | Credits charged for that run |
created_at | INTEGER NOT NULL | milliseconds |
html_content | TEXT | Rendered report, added by ALTER |
Indexes: idx_tr_user_time(user_email, created_at DESC), idx_tr_lookup(user_email, params_key, created_at DESC).
Limits: a result_json over 400,000 bytes is silently not saved. Saved HTML over 900,000 characters is rejected with {"ok":false,"reason":"too_large"}. The save-run endpoint rejects data over 1,600,000 characters and silently drops HTML over 900,000 rather than failing the whole write. Retention is 90 days, enforced by the purge_results cron job.
activity_log#
The per-event timeline behind the Library.
| Column | Type | Meaning |
|---|---|---|
id | INTEGER PRIMARY KEY AUTOINCREMENT | |
user_email | TEXT NOT NULL | |
created_at | INTEGER NOT NULL | milliseconds |
category | TEXT NOT NULL | One of ai, projects, files, users, settings, auth, integrations, system; anything else is coerced to system |
action | TEXT NOT NULL | For example ai.run, tool.run, monitor.added |
title | TEXT NOT NULL | |
description | TEXT | Built from a whitelist of body and query keys |
module | TEXT | Pretty tool name |
status | TEXT NOT NULL DEFAULT 'success' | One of success, warning, failed, pending; anything else is coerced to success |
resource | TEXT | |
actor | TEXT | |
meta_json | TEXT | |
dedupe_key | TEXT | Capped at 160 characters |
Indexes: UNIQUE idx_activity_dedupe(dedupe_key), idx_activity_user_time(user_email, created_at DESC).
Two write modes: the default INSERT OR IGNORE (first write wins), and an upsert that replaces status, title and detail while keeping the original created_at, which lets a long job move from pending to success in place.
3. Caches#
Nine separate tables. The distinction between the first two is the single most common source of confusion and is explained in Caching architecture.
mv_seo_cache#
The shared, cross-customer tool-result cache.
| Column | Type | Meaning | |
|---|---|---|---|
cache_key | TEXT PRIMARY KEY | `'v1\ | ' + mvToolParamsKey(body)`. Contains no user identity |
tool | TEXT NOT NULL | The tool type | |
query | TEXT | Capped at 300 characters | |
data | TEXT NOT NULL | The entire /api/premium-ai response body, verbatim | |
fetched_at | INTEGER NOT NULL | milliseconds | |
expires_at | INTEGER NOT NULL | milliseconds | |
hits | INTEGER DEFAULT 0 | Incremented on every cross-customer hit |
Index idx_seo_cache_expires(expires_at). Expiry is checked at read time only and no DELETE statement for this table exists anywhere in the codebase.
dfs_cache#
Raw DataForSEO endpoint responses. Columns: cache_key TEXT PRIMARY KEY (endpoint + ':' + JSON.stringify(body)), response_json TEXT, cached_at INTEGER (seconds), tool TEXT (NULL when the call site omits the argument). Written with INSERT OR REPLACE. This is the only cache an administrator can clear or re-tune.
dfs_hist_cache#
Historical rank overview. cache_key TEXT PRIMARY KEY ('hist:' + JSON body), response_json TEXT, cached_at INTEGER (seconds). Hardcoded 7-day TTL, no admin override and no purge path.
pagespeed_cache#
cache_key TEXT PRIMARY KEY ('ps:<MOBILE|DESKTOP>:<url>'), response_json TEXT, cached_at INTEGER (seconds).
No longer read or written. It held PageSpeed results for 6 hours, with a 7-day fallback on a quota error, so a customer could press Run and be handed a result from earlier without anything on screen saying so. Every PageSpeed request now goes to Google. The table is left in place rather than dropped, so rows written before the change are still there and are simply never consulted.
benchmarks_cache#
domain TEXT PRIMARY KEY (normalised), response_json TEXT, cached_at INTEGER (seconds). 30-day TTL.
quickview_cache#
domain TEXT PRIMARY KEY, data TEXT NOT NULL, cached_at INTEGER NOT NULL DEFAULT (unixepoch()). 24-hour TTL expressed in the SELECT itself. The in-code note explains why this is D1 and not KV: KV bindings reset on every Pages deploy here, so MONITOR_DB is used because it is reliably bound.
mv_translation_cache#
cache_key TEXT PRIMARY KEY (sha256(src|tgt|text)), translated TEXT, created_at INTEGER (seconds). No TTL and no eviction. Entries are permanent.
brief_cache#
The daily AI brief widget. Primary key (user_email, day, target). Columns: user_email TEXT, day TEXT (YYYY-MM-DD, UTC), target TEXT, brief_json TEXT, created_at INTEGER. The brief fires on every dashboard load, so this path deliberately has no quota gate and no incrementUsage.
gsc_cache#
Google Search Console performance results.
| Column | Type | Meaning |
|---|---|---|
id | INTEGER PRIMARY KEY AUTOINCREMENT | |
user_email | TEXT | |
site_url | TEXT | |
cache_key | TEXT | perf:<site>:<start>:<end>:<dimensions> |
data_json | TEXT | |
cached_at | INTEGER DEFAULT (unixepoch()) | seconds |
Index idx_gsc_cache(user_email, cache_key). TTL 1 hour. Writes are a plain INSERT, never an upsert, so every miss adds a row forever and only the freshest is read. Cleared only when the user disconnects Google.
4. Telemetry and observability#
metrics_hourly#
RED metrics plus provider cost rollup. Primary key (bucket_ts, scope).
| Column | Type | Meaning |
|---|---|---|
bucket_ts | INTEGER NOT NULL | UNIX hour in seconds |
scope | TEXT NOT NULL | all, tool:<type>, provider:dataforseo, provider:openai, provider:anthropic |
tenant_id | TEXT NOT NULL DEFAULT 'global' | Carried from day one for future workspace scoping. Never written and not part of the key |
runs | INTEGER NOT NULL DEFAULT 0 | |
errors | INTEGER NOT NULL DEFAULT 0 | |
timeouts | INTEGER NOT NULL DEFAULT 0 | |
quota_hits | INTEGER NOT NULL DEFAULT 0 | |
dur_sum | INTEGER NOT NULL DEFAULT 0 | milliseconds; mean latency is dur_sum / runs |
cache_hit | INTEGER NOT NULL DEFAULT 0 | Shared-cache hits |
cache_miss | INTEGER NOT NULL DEFAULT 0 | |
cost_micros | INTEGER NOT NULL DEFAULT 0 | USD times 1e6 |
tokens_in | INTEGER NOT NULL DEFAULT 0 | |
tokens_out | INTEGER NOT NULL DEFAULT 0 |
Size is O(tools times hours), never O(users times events). DataForSEO cost is real, read from the provider's own cost field. LLM cost is estimated from a price table; unknown models record tokens with a zero-dollar estimate.
error_issues#
Grouped errors, described in code as "Sentry-lite on D1".
| Column | Type | Meaning |
|---|---|---|
fingerprint | TEXT PRIMARY KEY | Hash of tool plus a normalised message |
tool | TEXT | |
title | TEXT | First 180 characters of the message |
status | TEXT NOT NULL DEFAULT 'unresolved' | Also resolved, regressed |
count | INTEGER NOT NULL DEFAULT 0 | |
first_seen | INTEGER | seconds |
last_seen | INTEGER | seconds |
last_sample | TEXT | First 500 characters |
Normalisation lowercases and replaces URLs, emails, hex ids of 8 or more characters and digit runs with placeholders, collapses whitespace and caps at 200 characters. A new occurrence of a resolved issue flips it to regressed.
workflow_failures#
id INTEGER PRIMARY KEY AUTOINCREMENT, created_at INTEGER NOT NULL, user_email TEXT, workflow_key TEXT, step_key TEXT NOT NULL, step_label TEXT, input_seed TEXT, error_message TEXT, timed_out INTEGER NOT NULL DEFAULT 0, attempt_count INTEGER NOT NULL DEFAULT 1, resolved INTEGER NOT NULL DEFAULT 0, resolved_at INTEGER, notes TEXT. Indexes: wf_fail_created(created_at DESC), wf_fail_resolved(resolved, created_at DESC).
admin_audit_log#
id INTEGER PRIMARY KEY AUTOINCREMENT, ts INTEGER NOT NULL (seconds), actor TEXT, action TEXT, target TEXT, meta TEXT. No index. Written best-effort so it can never block the action it records.
5. Configuration and admin identity#
platform_config#
key TEXT PRIMARY KEY, value TEXT, updated_at INTEGER (seconds), updated_by TEXT. Read through mvConfigAll, which caches the whole map in module scope for 60,000 ms and fails open.
| Key | Purpose |
|---|---|
cache_days_default | Global DataForSEO cache lifetime override, in days |
cache_days:<tool> | Per-tool override, in days |
suspended:<email> | '1' means the account is suspended |
tool_off:<type> | '1' disables one tool with a 503 |
ai_reco_min_plan | Minimum plan for Get Recommendations, default pro |
ai_reco_quota:<plan> | Per-plan monthly recommendation allowance |
Writes are owner-only.
admin_users#
email TEXT PRIMARY KEY, role TEXT NOT NULL DEFAULT 'owner', status TEXT NOT NULL DEFAULT 'active', added_by TEXT, created_at INTEGER (milliseconds), last_seen_at INTEGER. Seeded from the code list as role='owner', status='active', added_by='seed' only when the table is empty.
6. Teams and workspaces#
team_memberships#
| Column | Type | Meaning |
|---|---|---|
id | INTEGER PRIMARY KEY AUTOINCREMENT | |
owner_email | TEXT NOT NULL | Workspace owner |
member_email | TEXT NOT NULL | Invitee |
role | TEXT DEFAULT 'member' | Display only; not enforced |
invite_token | TEXT | CSPRNG, 24 characters from an unambiguous alphabet. Never expires |
invited_at | INTEGER DEFAULT (unixepoch()) | |
accepted_at | INTEGER | NULL means pending |
blog_role | TEXT DEFAULT 'author' | owner, editor or author. The only enforced role |
UNIQUE (owner_email, member_email). Index idx_team_member(member_email). Seat caps count invited members only: starter: 0, pro: 4, agency: 14, enterprise: 100000, unlimited: 100000. Free has no key and therefore 0. Pending invites consume a seat.
7. Competitor Monitor#
monitored_urls#
id INTEGER PRIMARY KEY AUTOINCREMENT, user_email TEXT NOT NULL, competitor_domain TEXT, url TEXT NOT NULL, label TEXT, watch_type TEXT DEFAULT 'any', sensitivity TEXT DEFAULT 'medium', check_interval_hours INTEGER DEFAULT 6, notify_email INTEGER DEFAULT 0, custom_selector TEXT, enabled INTEGER DEFAULT 1, created_at INTEGER DEFAULT (unixepoch()), last_checked_at INTEGER, and four columns added by ALTER: slack_webhook_url TEXT, generic_webhook_url TEXT, notify_email_addr TEXT, min_severity TEXT DEFAULT "Low". No indexes.
url_snapshots#
One row per fetch. id INTEGER PRIMARY KEY AUTOINCREMENT, monitored_url_id INTEGER NOT NULL, status_code INTEGER, title, meta_description, h1, h2_list, canonical, og_image, og_title, og_description TEXT, prices_json TEXT, min_price, max_price, avg_price REAL, price_count INTEGER, product_names_json, nav_items_json TEXT, internal_links_count, external_links_count, images_count, word_count, content_length INTEGER, content_hash TEXT, schema_types_json TEXT, schema_count INTEGER, phones_json, emails_json, social_links_json TEXT, last_modified_header, etag TEXT, first_new_links_json TEXT, captured_at INTEGER DEFAULT (unixepoch()). No index; rows accumulate one per check per URL forever.
changes#
id INTEGER PRIMARY KEY AUTOINCREMENT, monitored_url_id INTEGER NOT NULL, change_type TEXT, severity TEXT (high, medium, low), old_value TEXT (capped at 500), new_value TEXT (capped at 500), numeric_delta REAL, description TEXT, detected_at INTEGER DEFAULT (unixepoch()), read_at INTEGER, dismissed_at INTEGER.
Deleting a monitored URL cascades manually: url_snapshots, then changes, then monitored_urls.
8. Alerts, schedules, notifications, editorial#
rank_alerts#
id INTEGER PRIMARY KEY AUTOINCREMENT, user_email TEXT NOT NULL, domain TEXT NOT NULL, keyword TEXT NOT NULL, country TEXT DEFAULT 'us', device TEXT DEFAULT 'desktop', last_position REAL, last_checked_at INTEGER, best_position REAL, worst_position REAL, enabled INTEGER DEFAULT 1, slack_webhook_url TEXT, generic_webhook_url TEXT, notify_email_addr TEXT, drop_threshold INTEGER DEFAULT 5 (clamped to 1-20 on write), created_at INTEGER DEFAULT (unixepoch()). An alert is due after 24 hours.
rank_history#
id INTEGER PRIMARY KEY AUTOINCREMENT, alert_id INTEGER NOT NULL, position REAL, recorded_at INTEGER DEFAULT (unixepoch()). Index idx_rank_history(alert_id, recorded_at).
scheduled_reports#
id INTEGER PRIMARY KEY AUTOINCREMENT, user_email TEXT NOT NULL, workflow_key TEXT NOT NULL, target TEXT NOT NULL, cadence TEXT NOT NULL (daily, weekly, monthly), recipient_email TEXT, enabled INTEGER DEFAULT 1, last_run_at INTEGER, next_run_at INTEGER, created_at INTEGER DEFAULT (unixepoch()), run_count INTEGER DEFAULT 0. Next run is +86400, +7 days or +30 days, defaulting to weekly. Creation requires plan starter or above.
schedule_runs#
id INTEGER PRIMARY KEY AUTOINCREMENT, schedule_id INTEGER NOT NULL, user_email TEXT NOT NULL, workflow_key TEXT NOT NULL, target TEXT NOT NULL, status TEXT NOT NULL, error TEXT, ran_at INTEGER DEFAULT (unixepoch()). No index. Note there is no created_at column.
notification_prefs#
user_email TEXT PRIMARY KEY, email_notifications INTEGER DEFAULT 1, weekly_digest INTEGER DEFAULT 1, competitor_alerts INTEGER DEFAULT 1, seo_score_alerts INTEGER DEFAULT 1, updated_at INTEGER DEFAULT (unixepoch()). Defaults are all on, returned both when no row exists and on any read error. weekly_digest is stored and returned but never read anywhere.
Note: there is no notifications table. The notifications feed is synthesised live from changes, rank_alerts, scheduled_reports and shared_reports.
editorial_items#
id INTEGER PRIMARY KEY AUTOINCREMENT, user_email TEXT NOT NULL, title TEXT NOT NULL, target_date TEXT (validated YYYY-MM-DD or NULL), status TEXT NOT NULL DEFAULT 'idea' (one of idea, draft, in-review, scheduled, published), owner TEXT, notes TEXT, sort_order INTEGER DEFAULT 0, created_at and updated_at INTEGER DEFAULT (unixepoch()). Index idx_editorial_user(user_email, target_date).
9. Sharing and white-label#
shared_reports#
| Column | Type | Meaning |
|---|---|---|
slug | TEXT PRIMARY KEY | 10 characters from an unambiguous alphabet, generated with Math.random() |
user_email | TEXT NOT NULL | |
title | TEXT NOT NULL | Capped at 200, default Metric Vault Report |
source_tool | TEXT | |
html_content | TEXT NOT NULL | Capped at 2,000,000 characters |
created_at | INTEGER DEFAULT (unixepoch()) | |
view_count | INTEGER DEFAULT 0 | Incremented on each public view |
expires_at | INTEGER | seconds, or NULL for never |
theme | TEXT | light, dark or NULL. Added by ALTER |
Index idx_shared_user(user_email). Slug generation retries up to 5 times on collision. An expired report returns HTTP 410; a missing one returns 404.
user_branding#
user_email TEXT PRIMARY KEY, company_name TEXT, logo_data_url TEXT (a data URL stored inline in D1), primary_color TEXT default '#7c5cfc', accent_color TEXT default '#D946A8', hide_mv_branding INTEGER default 0, custom_footer TEXT, updated_at INTEGER default (unixepoch()). Saving requires the Pro plan.
10. Integrations#
google_connections#
user_email TEXT PRIMARY KEY, google_email TEXT, access_token TEXT, refresh_token TEXT, token_expires_at INTEGER, scopes TEXT, connected_at INTEGER DEFAULT (unixepoch()), last_refreshed_at INTEGER.
Warning: token storage is base64 obfuscation, not encryption. The code states this explicitly and points at a possible future env.TOKEN_SECRET plus Web Crypto implementation.
social_connections#
Primary key (user_email, platform, account_id), so several accounts per platform are possible. Columns: user_email, platform, account_id (DEFAULT ''), account_name, account_handle, account_avatar, access_token NOT NULL, refresh_token, scopes, expires_at INTEGER, connected_at INTEGER NOT NULL, meta_json TEXT. Tokens are stored as provided, with no wrapping at all.
social_oauth_state#
state TEXT PRIMARY KEY, user_email TEXT NOT NULL, platform TEXT NOT NULL, code_verifier TEXT, created_at INTEGER NOT NULL (milliseconds). Self-cleaning at 15 minutes inside the schema function, and deleted single-use on callback.
social_scheduled_posts#
id TEXT PRIMARY KEY, user_email TEXT NOT NULL, text TEXT NOT NULL, platforms TEXT NOT NULL, scheduled_at INTEGER NOT NULL, status TEXT NOT NULL DEFAULT 'pending', created_at INTEGER NOT NULL, sent_at INTEGER, results_json TEXT. Indexes: idx_social_sched_due(status, scheduled_at), idx_social_sched_user(user_email, scheduled_at DESC). Drained by the social_schedule cron job, which claims up to 25 posts per run with an atomic pending to sending update so two runs cannot double-send.
provider_tokens#
The Higgsfield image provider's single platform-wide operator session. provider TEXT PRIMARY KEY, refresh_token TEXT, access_token TEXT, access_expires_at INTEGER (milliseconds), seed TEXT, client_id TEXT, updated_at INTEGER NOT NULL. Refresh tokens rotate on every use, which is why the diagnostic that inspects them is deliberately read-only.
higgs_oauth_state#
state TEXT PRIMARY KEY, verifier TEXT NOT NULL, client_id TEXT NOT NULL, redirect_uri TEXT NOT NULL, created_at INTEGER NOT NULL (milliseconds). Self-cleaning at 15 minutes and single-use on callback.
11. AI visibility, tier 2#
tier2_tracked_brands#
brand TEXT PRIMARY KEY (lowercased, trimmed), last_seen INTEGER NOT NULL (seconds), created_at INTEGER NOT NULL (seconds).
tier2_brand_owners#
user_email TEXT NOT NULL, brand TEXT NOT NULL, plan TEXT (the plan at the last enrolment, recorded for reference only), last_seen INTEGER NOT NULL (seconds), created_at INTEGER NOT NULL (seconds), alert_seeded INTEGER NOT NULL DEFAULT 0, PRIMARY KEY (user_email, brand). Index idx_t2owners_brand(brand, last_seen).
Written only for paid accounts, from the verified identity, when one of the three enrolling tools runs. tier2_tracked_brands has no owner column, which is why this table exists: the weekly cadence is a paid-plan behaviour and needs to know whose plan to read. alert_seeded records that the account's tier2_alerts row was created once, so an alert the customer deletes is not recreated on their next run. The stored plan is never trusted at scan time; the scheduler re-reads the live plan, so a downgrade stops the rescans.
tier2_history#
id INTEGER PRIMARY KEY AUTOINCREMENT, brand TEXT NOT NULL, engine TEXT NOT NULL, prompt TEXT NOT NULL (capped at 500), brand_appeared INTEGER NOT NULL, position INTEGER, sentiment TEXT, captured_at INTEGER NOT NULL (seconds). Index idx_t2h_brand_captured(brand, captured_at).
tier2_meta#
key TEXT PRIMARY KEY, value TEXT, updated_at INTEGER. Only key used: last_run_at.
tier2_custom_prompts#
id INTEGER PRIMARY KEY AUTOINCREMENT, brand TEXT NOT NULL, prompt TEXT NOT NULL, user_email TEXT, created_at INTEGER NOT NULL. Index idx_cp_brand(brand). A NULL or empty user_email makes a prompt visible to every user of that brand.
tier2_alerts#
id INTEGER PRIMARY KEY AUTOINCREMENT, user_email TEXT NOT NULL, brand TEXT NOT NULL, drop_threshold_pct INTEGER NOT NULL (clamped 10-99; 15 from the form, 10 when seeded automatically), last_value INTEGER, last_fired_at INTEGER, enabled INTEGER NOT NULL DEFAULT 1, created_at INTEGER NOT NULL. Indexes idx_alerts_brand(brand) and UNIQUE idx_alerts_user_brand(user_email, brand). One row per (user_email, brand): the add route updates an existing row rather than inserting a second, and the unique index enforces it, because a duplicate row is a second email about the same movement.
The tier-2 job looks for due brands every 6 hours, rescans a brand a week after its newest measurement, measures at most 2 brands per pass with 8 prompts each, and no-ops entirely without OPENAI_API_KEY. Brand selection is mvT2SelectDueBrands; the alert decision is mvT2DeltaVerdict, which fires only on a move of at least 10 points carried by at least 3 prompt-and-engine results.
12. Async jobs and standalone tools#
article_jobs#
id TEXT PRIMARY KEY, status TEXT NOT NULL, payload TEXT NOT NULL, created_at INTEGER NOT NULL (milliseconds), updated_at INTEGER NOT NULL. Self-pruning at 1 hour inside its schema function, which runs on every call.
responsive_analyses#
id INTEGER PRIMARY KEY AUTOINCREMENT, user_email TEXT, url TEXT NOT NULL, title TEXT, favicon TEXT, score INTEGER, summary TEXT, analysis_json TEXT, conversation_json TEXT, device TEXT default 'desktop', created_at and updated_at INTEGER default (unixepoch()). Index idx_ra_user(user_email, created_at DESC).
Warning: delete is by id alone. There is no user_email in the WHERE clause, unlike tool_results and shared_reports.
13. Blog#
Legacy identity tables#
| Table | Columns |
|---|---|
blog_sites (v1) | user_email TEXT PRIMARY KEY, site_id TEXT NOT NULL, signing_secret TEXT NOT NULL, site_name TEXT, site_slug TEXT, created_at INTEGER. Capped every account at one blog. Left in place deliberately so a rollback still works |
blog_sites_v2 | Same columns, primary key (user_email, site_id). One-time forward-fill from v1 |
blog_prefs | user_email TEXT PRIMARY KEY, current_site_id TEXT, updated_at INTEGER. Separate on purpose: selecting a site is a cheap write that can never disturb the signing secrets |
The native blog platform (mvb_*)#
| Table | Columns | Indexes |
|---|---|---|
mvb_sites | id TEXT PK, user_email TEXT NOT NULL, name TEXT NOT NULL, slug TEXT NOT NULL, default_locale TEXT DEFAULT 'en', settings_json TEXT, created_at, updated_at INTEGER | UNIQUE idx_mvb_sites_slug(slug), idx_mvb_sites_user(user_email, created_at) |
mvb_posts | id TEXT PK, site_id TEXT NOT NULL, title TEXT NOT NULL, slug TEXT NOT NULL, locale TEXT NOT NULL DEFAULT 'en', content TEXT, excerpt TEXT, status TEXT NOT NULL DEFAULT 'draft', visibility TEXT NOT NULL DEFAULT 'public', password TEXT, author_id TEXT, featured_image_id TEXT, seo_title TEXT, seo_description TEXT, canonical_url TEXT, sticky INTEGER NOT NULL DEFAULT 0, published_at INTEGER, newsletter_sent_at INTEGER, created_at, updated_at INTEGER NOT NULL | UNIQUE idx_mvb_posts_slug(site_id, locale, slug), idx_mvb_posts_site(site_id, updated_at DESC) |
mvb_media | id TEXT PK, site_id TEXT NOT NULL, filename TEXT, url TEXT NOT NULL, r2_key TEXT, mime TEXT, size INTEGER, alt_text TEXT (capped 200), caption TEXT, created_at INTEGER NOT NULL | idx_mvb_media_site(site_id, created_at DESC) |
mvb_categories | id TEXT PK, site_id TEXT NOT NULL, name TEXT NOT NULL, slug TEXT NOT NULL, description TEXT, parent_id TEXT, created_at INTEGER NOT NULL | UNIQUE idx_mvb_cat_slug(site_id, slug) |
mvb_tags | id TEXT PK, site_id TEXT NOT NULL, name TEXT NOT NULL, slug TEXT NOT NULL, created_at INTEGER NOT NULL | UNIQUE idx_mvb_tag_slug(site_id, slug) |
mvb_post_categories | post_id TEXT NOT NULL, category_id TEXT NOT NULL, PK (post_id, category_id) | - |
mvb_post_tags | post_id TEXT NOT NULL, tag_id TEXT NOT NULL, PK (post_id, tag_id) | - |
mvb_authors | id TEXT PK, site_id TEXT NOT NULL, external_id TEXT NOT NULL (the account email), display_name TEXT, email TEXT, avatar_url TEXT, created_at INTEGER NOT NULL | UNIQUE idx_mvb_authors_ext(site_id, external_id) |
mvb_subscribers | id TEXT PK, site_id TEXT NOT NULL, email TEXT NOT NULL, name TEXT, status TEXT NOT NULL DEFAULT 'confirmed', source TEXT, unsubscribe_token TEXT, created_at INTEGER NOT NULL, confirmed_at INTEGER | UNIQUE idx_mvb_subs(site_id, email) |
mvb_sites.slug is unique across all accounts, so it is salted with the last six characters of the site id. settings_json holds public_base_url and allowed_origins.
14. Index inventory#
30 indexes are created, 8 of them UNIQUE:
| Index | Table and columns | Unique |
|---|---|---|
idx_t2h_brand_captured | tier2_history(brand, captured_at) | |
idx_cp_brand | tier2_custom_prompts(brand) | |
idx_alerts_brand | tier2_alerts(brand) | |
idx_t2owners_brand | tier2_brand_owners(brand, last_seen) | |
idx_alerts_user_brand | tier2_alerts(user_email, brand) | UNIQUE |
idx_gsc_cache | gsc_cache(user_email, cache_key) | |
idx_social_sched_due | social_scheduled_posts(status, scheduled_at) | |
idx_social_sched_user | social_scheduled_posts(user_email, scheduled_at DESC) | |
idx_team_member | team_memberships(member_email) | |
idx_activity_dedupe | activity_log(dedupe_key) | UNIQUE |
idx_activity_user_time | activity_log(user_email, created_at DESC) | |
idx_tr_user_time | tool_results(user_email, created_at DESC) | |
idx_tr_lookup | tool_results(user_email, params_key, created_at DESC) | |
idx_seo_cache_expires | mv_seo_cache(expires_at) | |
idx_api_keys_email | api_keys(user_email) | |
idx_rank_history | rank_history(alert_id, recorded_at) | |
idx_editorial_user | editorial_items(user_email, target_date) | |
idx_shared_user | shared_reports(user_email) | |
wf_fail_created | workflow_failures(created_at DESC) | |
wf_fail_resolved | workflow_failures(resolved, created_at DESC) | |
idx_ra_user | responsive_analyses(user_email, created_at DESC) | |
idx_mvb_sites_slug | mvb_sites(slug) | UNIQUE |
idx_mvb_sites_user | mvb_sites(user_email, created_at) | |
idx_mvb_posts_slug | mvb_posts(site_id, locale, slug) | UNIQUE |
idx_mvb_posts_site | mvb_posts(site_id, updated_at DESC) | |
idx_mvb_media_site | mvb_media(site_id, created_at DESC) | |
idx_mvb_cat_slug | mvb_categories(site_id, slug) | UNIQUE |
idx_mvb_tag_slug | mvb_tags(site_id, slug) | UNIQUE |
idx_mvb_authors_ext | mvb_authors(site_id, external_id) | UNIQUE |
idx_mvb_subs | mvb_subscribers(site_id, email) | UNIQUE |
37 tables have no index at all beyond their primary key, including every cache table except mv_seo_cache and gsc_cache, all three usage tables, user_plans, platform_config, admin_users, admin_audit_log, error_issues, metrics_hourly, monitored_urls, url_snapshots and changes.
15. Retention#
Automatic cleanup exists for four tables#
| Table | Mechanism | Window |
|---|---|---|
tool_results | The purge_results cron job | 90 days |
article_jobs | Swept inside its schema function on every call | 1 hour |
social_oauth_state | Swept inside its schema function on every call | 15 minutes |
higgs_oauth_state | Swept inside the sign-in handler | 15 minutes |
Deleted only by a user or admin action#
dfs_cache (admin clear), platform_config cache keys (admin apply-to-all), google_connections and gsc_cache (disconnect), social_connections (disconnect), scheduled_reports with schedule_runs, rank_alerts with rank_history, monitored_urls with url_snapshots and changes, shared_reports, editorial_items, responsive_analyses, user_branding, team_memberships, tier2_alerts, tier2_brand_owners, tier2_custom_prompts, the mvb_* tables, and individual tool_results rows.
Grows without bound#
mv_seo_cache (no DELETE exists anywhere), mv_translation_cache (no TTL column at all), dfs_hist_cache, brief_cache (one row per user per UTC day per target), gsc_cache (a plain INSERT per miss), url_snapshots, changes, tier2_history, activity_log, metrics_hourly, error_issues, admin_audit_log, workflow_failures, schedule_runs, all three usage tables, and api_keys.
usage_hourly is the fastest-growing of the usage tables because it gains a row per user per hour and a row per IP per hour for /api/translate.
16. Supabase#
Supabase is used for two things only: authentication and identity, and a small number of REST-accessed tables. It is not the application data store.
| Table | Accessed from | Key used | Purpose |
|---|---|---|---|
analysis_history | The worker via PostgREST, and the browser via supabase-js | Service role from the worker; anon key plus the user session from the browser | Legacy analysis history. Reads filter user_id, order by created_at desc, default limit 50 |
contact_messages | admin.html only | Anon key plus an authenticated session | Messages from the founder widget |
bug_reports | admin.html only | Same | Bug, feature and UI reports |
usage_logs | Any page via a global logger | Anon key; RLS allows anon INSERT | Page-level usage events |
Two SQL files in the repository declare a wider schema (subscribers, blog_posts, team_members, team_invitations, site_settings and the tables above). They are not applied by any build or deploy step; the header says to run them in the Supabase SQL editor.
Warning: the declared RLS policies grant every authenticated Supabase user full read and write on contact_messages, bug_reports, subscribers, team_members and team_invitations. Admin-ness is not enforced in the database; the SQL comment says so. The worker's own /api/admin/* routes are a separate and stronger path that does enforce it server-side.
Nothing in this repository writes contact_messages or bug_reports, and analysis_history has no CREATE TABLE anywhere in the repository.
Note: the admin activity feed is not a Supabase table. It reads D1 activity_log through /api/admin/runs, which is what mvLogActivity writes on every routed action. The panel used to select from a Supabase table called admin_activity that nothing in the repository ever wrote, so it was permanently empty and read as "nothing happened today" either way.
17. KV, R2, Workers AI and the edge cache#
KV: QUICKVIEW_CACHE#
Declared in wrangler.toml and completely unused. A repository-wide search returns exactly one hit: the declaration itself. There is no env.QUICKVIEW_CACHE reference anywhere. The feature it was created for uses the D1 table quickview_cache instead, because KV bindings on this Pages project reset on every deploy.
There is therefore no KV key format, no KV TTL and no KV data. Any documentation describing "the KV cache" is stale.
R2: BLOG_MEDIA#
Bucket metricvault-blog-media. Stores blog images only: uploads, URL imports and AI-generated covers. Nothing else in the product uses R2.
| Property | Value |
|---|---|
| Key format | sites/<site_id>/media/<media_id>.<ext> |
| Index | The D1 table mvb_media, which carries r2_key. The two can drift |
| Public URL | /api/blog/media/file/<site_id>/<media_id>.<ext> |
| Serving | Unauthenticated and public, Cache-Control: public, max-age=31536000, immutable, Access-Control-Allow-Origin: * |
| Direct upload limit | 5 MB, {"code":"too_large","message":"Max 5MB"} |
| Allowed types | PNG, JPG, WEBP, GIF. Otherwise bad_type "Use PNG, JPG, WEBP, or GIF" |
| URL import limit | 15,000,000 bytes, HTTP 413 "Image is larger than 15MB" |
| AI generation | Maximum 4 images per batch, default wall-clock budget 75,000 ms |
| Missing binding | Writes return 503 media_not_configured, "Image storage is not set up yet (the BLOG_MEDIA R2 bucket is not bound)." Reads 404 |
Deletion nulls any mvb_posts.featured_image_id pointing at the image, deletes the R2 object best-effort inside a swallowing try/catch, then deletes the mvb_media row unconditionally. Body embeds are deliberately not rewritten.
Workers AI: AI#
One use: translation with @cf/meta/m2m100-1.2b, with DeepL as a fallback when DEEPL_API_KEY is set. Blog images do not use this binding. They come from Higgsfield (nano_banana_pro) only; the Flux image fallback was removed. /api/translate accepts at most 100 strings and 20,000 characters per request and fails open by returning the source text on a provider error.
Cloudflare edge cache (caches.default)#
Four uses, all for static content fetched from GitHub raw as a Pages workaround, none of them application data:
| Path | Cache key | TTL |
|---|---|---|
/blog, /blog.html | /__cache/blog | 300 s |
/legal, /legal.html | /__cache/legal | 300 s |
/logo-dark.png, /logo-light.png | /__cache<pathname> | 3600 s |
The 9 mapped /free-tools/* paths | /__cache/<filePath> | 300 s |
Browser storage#
Several preferences live only in localStorage and have no server equivalent, including mv-theme, mvLang, mv-country, mv-device, mv-range, mv_active_workspace, mvRankAlerts, mvBacklinkAlerts, mvScheduledCrawls, mvAiVisSchedule and mvSocTrackSchedule. The client-only alert and schedule keys are not the same thing as the server-side rank_alerts and scheduled_reports tables.
18. Known anomalies#
Each is reproducible from the code and should not be documented as intended behavior.
ai_visibilityappears twice in the credit-cost map, as 0 and as 1. The last value wins, so the effective cost is 1. The/api/toolsendpoint separately pins it to 0.forceFreshcannot be triggered from the shipped UI. Nothing indashboard.htmlever puts it on the request body, so the cross-customer cache cannot be bypassed by a user.X-MV-CacheandX-MV-Cache-Age-Daysare emitted but never read by the frontend, so a cached result is indistinguishable from a fresh one in the UI.- The Library backfill selects columns that do not exist on
workflow_failuresandschedule_runs. Both queries throw, are swallowed, and produce zero rows, and the "done" sentinel is written anyway so it never retries. - Per-tool cache clear is a no-op for the 8 shared content tools, whose
dfs_cacherows carrytool = NULL. responsive_analysesdelete has no ownership check.- API keys are stored in plaintext.
- Google OAuth tokens are base64-obfuscated; social tokens are stored raw.
user_plans.emailis stored lowercase and compared directly, so the lookup uses the primary key index.mvEnsurePlanEmailsLowerfolded the legacy mixed-case rows once.usage_counters.planis a decoy column that is never written with a real plan, yet the admin display still falls back to it.metrics_hourly.tenant_idis declared, defaulted and never used.monitored_urls.min_severityis declared with a double-quoted default, which SQLite tolerates but which is not standard SQL.
See also
Was this article helpful?
Thanks — feedback noted for the docs team.