Verification scripts

Summary#

tests/ holds Node scripts, baseline files and a README. None of them run in CI. The blocking list is tests/release-gates.txt, and you run it yourself before every push:

node tests/run-gates.mjs          # every check, reports every failure
node tests/run-gates.mjs --bail   # stop at the first failure

A push to main deploys whether or not these pass. Until 2026-09-14 the deploy workflow ran this list in a Run the release gates step and a failure stopped the deploy; the owner removed testing from GitHub Actions to save minutes, so the local run is now the only check.

tests/gates-wired.mjs keeps the list honest. Every script in tests/ must either be in the list or be declared as a deliberate exclusion, so a gate cannot be added and then quietly never run. A handful are not in the list because they need Playwright or a live deployment; those are declared, with the reason, in that file.

They are written against Node's standard library only, because the repository has no package.json and no node_modules. Run them with Node 24 from the repository root.

Overview#

The scripts fall into five groups.

Three of them write files: login-inline-sync.mjs --write, site-chrome-sync.mjs --write, i18n-wire.mjs, i18n-generate.mjs --emit and kb-build.mjs --write. Each also has a read-only form that exits 1 on drift. The contract is always the same: edit the source, run the tool, commit both sides.

Pre-commit gates#

Run the ones that apply to what you changed.

Full inventory#

Structural integrity#

guard.mjs — closing-tag tripwire#

node tests/guard.mjs                  # default targets
node tests/guard.mjs path/to/file.html

Default targets: dashboard.html, index.html, admin.html, login.html. Inside any inline <script> block it bans a literal </body, </html or </script. Exit 0 clean, 1 on any hit.

Why: this catches the class of bug that took production down on 2026-04-20. A literal closing-body or closing-script tag written inside a JavaScript template literal can be matched by any HTML rewriter doing string.replace('</body>', ...). The browser's script parser then terminates early and throws SyntaxError: Unexpected end of input, which breaks every inline-scripted feature on the page including authentication, leaving the whole site unclickable.

A failure means: split the string. The prescribed fix is printed: const CLOSE_BODY = '</' + 'body>'; and then use ${CLOSE_BODY}.

chkblk.mjs — inline <script> compile check#

node tests/chkblk.mjs                    # defaults to ../dashboard.html
node tests/chkblk.mjs dashboard.html

Uses vm.compileFunction in-process with no execution, so it catches unbalanced quotes, parentheses and braces introduced by an edit. It skips <script src=…> and application/json / application/ld+json blocks.

Output is Checked <n> <script> blocks in <FILE> — <k> broken. and it must report 0 broken. A failure lists block #<i> (line ~<line>) with the first line of the error and exits 1.

Sync contracts#

login-inline-sync.mjs — the inlined /login matches login.html#

node tests/login-inline-sync.mjs           # check, exits 1 on drift
node tests/login-inline-sync.mjs --write   # regenerate from login.html

_worker.js intercepts GET /login and /login.html and serves an inlined const LOGIN_HTML = \...\` template literal instead of the static asset, added to dodge a Pages caching bug that served stale or zero-byte HTML. Nothing regenerates that copy, so editing login.html` alone has no effect in production. That drift once silently swallowed a full set of authentication fixes.

The script evaluates the template literal rather than string-comparing it, so escaping bugs surface as ❌ inlined LOGIN_HTML does not evaluate: <msg>. Comparison is line-ending agnostic. A pass prints ✅ LOGIN_HTML matches login.html (<n> bytes). A drift report gives both byte counts, the first differing character index, 70-character excerpts from each side, and ends with Edits to login.html are NOT live until you run: node tests/login-inline-sync.mjs --write. --write re-verifies after writing and aborts if it still mismatches. Commit both files.

site-chrome-sync.mjs — one navbar and one footer#

node tests/site-chrome-sync.mjs           # drift check, exits 1 if stale
node tests/site-chrome-sync.mjs --write   # apply the partials to the pages
node tests/site-chrome-sync.mjs --list    # show the target set

Stamps partials/site-nav.html and partials/site-footer.html into 21 public pages between MV:NAV and MV:FOOTER marker comments. Per page it also fills <!--MV:SLOT name--> from partials/extras/, keeps <!--MV:IF lang--> only on pages that load js/mv-i18n-runtime.js, marks the current nav entry is-active, rewrites /#anchor to #anchor on index.html, and ensures /css/tokens.css, /css/site-chrome.css and /js/mv-site-chrome.js are linked.

Editing the chrome inside a page does nothing — the next sync overwrites it. Adding a public page means adding it to PAGES and running --write.

Why: the chrome had drifted into 20 distinct navbars and 12 distinct footers across 26 pages. On seven pages the "Free tools" nav link pointed at /free-tools/schema-validator instead of /free-tools, and the blog link was /blog on some pages and /blog.html on others. Detail in Centralized nav and footer.

kb-build.mjs — the knowledge-base gate#

node tests/kb-build.mjs             # validate only, exit 1 on error — the gate
node tests/kb-build.mjs --write     # validate, then regenerate help/
node tests/kb-build.mjs --stats     # article and word counts per category
node tests/kb-build.mjs --list      # every article id, category and file path

Source of truth is docs/kb/**/*.md; generated output is help/. The script carries its own front-matter parser, Markdown renderer and search-index builder, because the repository has no build step and no dependencies.

It validates front matter (id, title, summary, and a category that exists in its TAXONOMY), checks that every double-bracket cross-link and every related / seeAlso id resolves, and checks each article against the required section set for its type. Feature articles in the tools, billing and account categories are additionally checked against the full 17-section skeleton and warn if more than four sections are missing.

A failure means: a dead cross-link, a missing required section, a bad category, or malformed front matter. The build prints which file and which id. Never hand-edit help/ — the next --write wipes and regenerates it. help/ is not staged for deploy.

kb-manifest.mjs — plan versus reality#

node tests/kb-manifest.mjs             # full report
node tests/kb-manifest.mjs --missing   # just the ids still to write, one per line
node tests/kb-manifest.mjs --strict    # exit 1 if anything planned is missing

Compares docs/kb/_meta/manifest.json (the plan) with the articles that exist, in both directions:

It exits 1 with No manifest at <path> if the manifest is absent.

Ratchets and audits#

design-lint.mjs — the design-system ratchet#

node tests/design-lint.mjs                     # check against the baseline
node tests/design-lint.mjs --update-baseline   # lock in a reduction
node tests/design-lint.mjs --summary           # report without failing

Scanned files: dashboard.html, chat-widget.js, css/app.css, css/legacy-mvx.css, css/mvr.css, css/mvr-premium.css. css/tokens.css is deliberately excluded, because it is where tokens are supposed to be defined.

Pass prints ✅ PASS: no design-debt counts increased. The ratchet holds. Failure prints ❌ FAIL: new design debt was introduced (a tracked count went up):. A reduction prints ⬇ Debt reduced (nice — run --update-baseline to lock it in):.

A failure means: you added a hardcoded color, an inline style, an emoji, an off-scale radius, a left-accent border stripe, or a forked token definition. Fix it with a token or an .mvr-* component. Only re-baseline when you genuinely reduced debt. The rules themselves are law in docs/DESIGN-CONSTITUTION.md; The design system explains them.

theme-lint.mjs — one theme system#

node tests/theme-lint.mjs             # report, exit 1 if any page fails
node tests/theme-lint.mjs --todo      # just the remaining work
node tests/theme-lint.mjs --summary   # never fails

Twenty-nine pages, three checks each:

Output ends with <n>/<total> pages on the shared theme system. This was written before the migration on purpose, so it reports what is left rather than blocking work.

contrast-audit.mjs — WCAG AA#

node tests/contrast-audit.mjs                # light mode, AA, exit 1 on fail
node tests/contrast-audit.mjs --theme dark
node tests/contrast-audit.mjs --all          # both themes; this is what run-gates.mjs runs
node tests/contrast-audit.mjs --summary      # never exits non-zero
node tests/contrast-audit.mjs --min 3.0      # override the threshold

Walks the element tree, finds the nearest opaque-background ancestor, composites translucent layers, then measures, because the naive check is wrong often enough to be useless. Twenty-two pages are audited: index, pricing, blog, blog-post, legal.html, four legal/* and thirteen free-tools/*.

The release gate list runs --all, so a failure in either theme fails node tests/run-gates.mjs. It ran light only until 2026-09-11, because dark carried 25 findings of its own and a gate that is red on day one blocks everyone. Both themes are clean now.

A clean run is not a statement about the site. Twenty-two pages is fewer than the thirty-four seo-lint.mjs gates as public: about, contact, legal/cookies, the four solutions/* and the six platform/* are measured by nothing, and neither are dashboard, admin or reports/*. free-tools/llms-txt-validator was in that blind spot until 2026-09-11 and was sitting on a live 4.48:1 miss when it was added, which is what being unmeasured looks like.

Declared limits, stated in the script: gradients are measured at their first color stop; elements with a background-image: url(...) are unmeasurable and skipped but counted; :hover, :focus, ::before and other state selectors are skipped; unparseable selectors are skipped and counted. A clean run means "nothing measurable fails", never "everything is fine".

Behavior#

smoke.mjs — live end-to-end tool checks#

BASE_URL=https://metricvaultai.com node tests/smoke.mjs
BASE_URL=https://metricvaultai.com node tests/smoke.mjs --tools=audience_overlap,tech_stack
BASE_URL=https://metricvaultai.com node tests/smoke.mjs --concurrency=8 --timeout-ms=90000
BASE_URL=https://preview-abc123.metricvaultai.pages.dev node tests/smoke.mjs

Defaults: BASE_URL is https://metricvaultai.com, --concurrency is 4, --timeout-ms is 120000.

Forty-five cases: 35 premium-AI types posted to /api/premium-ai as {type, query}, and 10 real-data tools posted to /api/tools as {type, url}. Each case asserts that required fields are present (dotted paths supported) and that arrays meet a minimum length, for example domain_overview.topKeywords at least 3, keyword_magic.keywords at least 5, ai_visibility.platforms at least 3, tech_stack.technologies at least 1. Adding a tool means adding an entry to CASES.

Why: seventeen tools once looked "working" to users — the button clicked, the spinner showed — but silently threw a ReferenceError inside the renderer and never displayed data.

guide-rules.mjs — the recommendation rules#

node tests/guide-rules.mjs

Loads js/mv-guide-rules.js and tests every "Get recommendations" rule three ways: a realistic payload that should produce findings, a healthy payload that should produce none, and hostile inputs (null, wrong types, strings where numbers go, huge values) that must not throw and must not invent findings.

Shape assertions: the rule returns an array of at most 6 findings, each with a non-empty t, a sev of high, med or low, an area string and a numeric id, and the text must contain no undefined, NaN or [object. A stray undefined or NaN means a field was read that the payload did not have.

The two failure modes it guards: a rule reads a field the API never sends, so the button never appears and the tool looks like it was never wired up; or a rule fires on absent data, reading "0 issues" as "0 = bad".

guide-wiring.mjs — the panel wiring#

node tests/guide-wiring.mjs

Where guide-rules.mjs tests the rules in isolation, this pulls the real mvAttachGuide and mvBuildGuide out of dashboard.html, runs them against a minimal DOM shim, and checks that a button is inserted, that clicking it calls the AI with the right payload, and that a tool which already has a panel does not get a second one.

login-verify.cjs — real-browser login#

$env:MV_EMAIL='...'; $env:MV_PW='...'; node tests/login-verify.cjs [outPng] [gotoPath]

Requires a local dev server on 127.0.0.1:8788 and Playwright installed globally or in node_modules. Credentials come from the environment only, so they never appear in the file or on a command line.

Defaults: output tests/login-shot.png, target path /dashboard. It fills #signinEmail and #signinPassword, clicks #signinButton, waits for Supabase authentication, navigates, waits for the inline scripts to settle and screenshots at 1440x900. It prints LOGIN_OK:, final_url:, login_error_text:, the first 12 console_errors: and screenshot:. Exit 2 means MISSING MV_EMAIL / MV_PW env; exit 3 means CANNOT_LOAD_PLAYWRIGHT: set MV_PW_PKG to the playwright package path.

Pipelines and tooling#

The i18n pipeline#

Run all four in order whenever a UI string changed. Detail in Localization system.

node tests/i18n-extract.mjs
node tests/i18n-generate.mjs        # needs ANTHROPIC_API_KEY in .dev.vars; resumable
node tests/i18n-generate.mjs --emit
node tests/i18n-verify.mjs          # must print "all current UI strings are covered"

A verify failure means: a string is on screen that never reached the dictionary. Dictionary coverage is not the same as text swapping on screen, so also run a permissive loose scan and diff against i18n/dict.es.json; no visible English should remain except data, URLs, ids, domains or raw external-API values.

compare-repos.mjs — staging versus production drift#

node tests/compare-repos.mjs ../metricvaultai-seo-main
node tests/compare-repos.mjs <other-repo-dir> [this-repo-dir]

Reports what matches, what differs, and what is missing on each side, separating application code (which should match) from infrastructure (which is meant to differ). Exit code 1 if any non-infrastructure file differs, else 0. Full explanation in Production and staging repos.

higgsfield-auth.mjs — operator OAuth helper, not a test#

node tests/higgsfield-auth.mjs

Opens a browser sign-in against https://mcp.higgsfield.ai with a local callback on port 8792, then writes HIGGSFIELD_ACCESS_TOKEN, HIGGSFIELD_REFRESH_TOKEN, HIGGSFIELD_CLIENT_ID and AI_IMAGE_PROVIDER=higgsfield into ./.dev.vars.

Each run performs a fresh dynamic client registration so Metric Vault gets its own client_id and its own token family. That matters because Higgsfield rotates the refresh token on every renewal and has reuse detection: if two applications share one token family, whichever renews first invalidates the other permanently. For production, set the three values as Cloudflare Pages secrets instead. See Environment variables and secrets.

What is not covered#

Being straight about the gaps is more useful than implying coverage.

• No unit test harness for the worker. _worker.js is never imported and executed as a module. Well over a hundred gates do read it, and many lift a named function out of the source and run it against fixtures, but there is no harness that boots the worker and drives requests through it. smoke.mjs is the closest thing and it runs against a live deployment.
• No accessibility audit beyond contrast. contrast-audit.mjs measures color only.
• No performance budget check.