503 and service-unavailable errors

Summary#

A 503 from Metric Vault almost never means the platform is down. It means a safety gate refused to guess. Two designs produce it: a check that could not complete and therefore blocked the action rather than allowing it unmetered, and a dependency that is not configured on the environment being used. Both are deliberate. This page separates the two, lists every 503 the platform can emit, and covers the single configuration mistake that most often produces one: a missing internal secret.

Symptoms#

• A tool refuses with Usage check temporarily unavailable, please retry. or Plan check temporarily unavailable, please retry.
• A feature refuses with a message ending "is not configured", "is not set up yet" or "not configured".
• AI article generation starts, sits at queued, and eventually reports Job not found or expired. Please re-submit.
• Background jobs stop advancing: the Job Console shows jobs permanently Due now and last-run timestamps stop moving.
• Plans stop reconciling with the payment provider.
• A specific tool refuses for everyone with This tool is temporarily unavailable. Please try again shortly.
• A browser network panel shows HTTP 503 on a request that used to succeed.

Diagnosis#

Step 1. Decide which of the three kinds of 503 you have.

The first is transient and self-clears. The second and third persist until somebody changes configuration.

Step 2. Check whether it affects one account or everyone.

Open Admin console → System Health and set the range to the last 24 hours. Look at the Error rate tile and the Tool health table. A single customer's problem does not move a platform-wide error rate; a configuration problem does, and it will show a tool as down or degraded. Grouped issues underneath gives you the repeated message and a sample.

Step 3. Rule out a deliberate switch before anything else.

Open Admin console → Config. A row whose key begins tool_off: disables that tool for every customer and makes it answer This tool is temporarily unavailable. Please try again shortly. Deleting the row restores it immediately. See Runtime configuration.

Step 4. If jobs or article generation are involved, check the internal secret.

This is the highest-value check on the page and is covered in its own section below.

The internal-secret case#

Some endpoints exist only for the platform to call itself: the job runner, the billing reconciliation, and the second half of the AI article writer. They are protected by a shared secret, MV_INTERNAL_SECRET, and that check fails closed. When the secret is not set, those endpoints answer:

Internal jobs are not configured (MV_INTERNAL_SECRET is not set).

with HTTP 503. When the secret is set but does not match, they answer HTTP 401 with Unauthorized internal call.

Why AI article generation is the visible casualty

A full-length article takes far longer than a single request can safely stay open, so the writer splits the work in two. The first call records a queued job and returns a job id immediately. It then starts a second, separate call that does the actual generation, authenticating with the internal secret.

If the secret is missing, that second call is refused with 503. There is no fallback: the platform will not process the work with an unauthenticated call. The consequence for the customer is not an error message but a silence. The job stays at queued, never advances through its phases, and is removed after one hour. Polling it then returns Job not found or expired. Please re-submit.

Why background jobs stop

Scheduled reports, rank alerts, the competitor monitor and AI-visibility tracking are not driven by a clock inside the application. An external scheduler calls a protected endpoint roughly every 15 minutes, and that endpoint uses the same secret. If it is missing or mismatched, every call is rejected and nothing due is ever processed. Symptom: jobs sit at Due now in the Job Console and last-run timestamps freeze. See Background jobs console.

Where the secret must be set

The same value must exist in three places. A mismatch in any one of them produces 401 rather than 503, and the symptoms look identical from the customer's side.

After setting or changing it, redeploy so the application picks up the new value, then confirm from the Job Console that a job's last-run time advances. See Environment variables and secrets and Maintenance and health.

The fail-closed gates#

These block an action because a prerequisite lookup failed. They are transient. Retrying after a moment is the correct first response.

The not-configured responses#

These persist until an administrator sets the missing credential or binding. They are honest by design: the platform names what is absent rather than failing vaguely.

Email is the exception worth remembering: when the email provider is not configured, most sends are skipped silently rather than returning 503. A customer reporting "no email arrived" may therefore see no error at all. Only the blog newsletter refuses loudly. See My scheduled report never arrived.

Resolution#

1. Classify the 503 using the table in Step 1. Fail-closed and not-configured have completely different responses.
2. For a fail-closed message, retry once. If it repeats across accounts, treat it as a database availability incident and check System Health rather than the individual customer.
3. For a not-configured message, identify the missing credential from the message itself and set it in the Cloudflare Pages project's environment variables, then redeploy. Confirm with the affected feature, not with the configuration screen.
4. For a deliberate switch, delete the tool_off: row in Config. The change takes effect within about a minute, because configuration is cached briefly.
5. For the internal secret, set the same value in all three locations listed above, redeploy, then confirm a job advances in the Job Console and an article generation completes end to end.
6. Tell the customer what happened. A fail-closed 503 protected their credits; a not-configured 503 was our gap. Both deserve a plain answer. See Contacting support.

Things a 503 is not

• It is not a quota problem. Running out of credits returns 429 with quota_exceeded, not 503. See I ran out of credits.
• It is not a plan problem. A feature above the customer's tier returns 403 with upgrade_required. See "This feature requires the ... plan".
• It is not a diagnostics problem. The internal diagnostic endpoints answer 403 with forbidden when called without their key, never 503.
• On the blog interface, a 502 from an upstream provider is reported to the browser as 503, because the edge would otherwise replace a 502 body with its own page. A blog 503 can therefore mean "the provider failed", not "we are unconfigured".

Prevention#

• Treat the internal secret as a three-place value. Never rotate it in one place. The failure it produces is silent, and it takes the article writer and every background job with it.
• Verify after every deploy that changes environment variables, by confirming a job's last-run time advances in the Job Console rather than assuming.
• Watch the Job Console rather than waiting for reports. Jobs stuck at Due now is the earliest visible signal of an internal-secret problem.
• Record every tool_off: switch you set, including who set it and why. An undocumented switch reads as an outage to the next person on shift.
• Keep the customer-visible message in the ticket. The message names the missing dependency, which is most of the diagnosis.
• Monitor error rate, not error count. A configuration failure moves the rate; a single customer's bad input does not.