Duplicate Content Audit

Summary#

Duplicate Content Audit crawls up to twelve of your most important pages and cross-compares every title tag and meta description. It shows which pages are identical, which are near-identical, which have no tag at all, and what proportion of the crawled set is clean.

Purpose#

The duplicate content that hurts most is not scraped copies on other domains. It is your own pages carrying the same title.

When two of your pages say Products | Acme in the title, you have asked Google to choose between them for the same query. It will choose, and it may not choose the one you would. Meanwhile both pages compete, neither ranks as well as one strong page would, and the click-through rate on whichever wins is set by a title that describes neither page precisely.

This tool exists to find those collisions before they cost you anything, on the pages that actually matter — not across a whole site crawl you will never read.

Overview#

Enter a domain. The tool builds a short, high-value page list, then crawls it.

Discovery works in priority order:

1. The homepage.
2. Your highest-traffic organic pages, ranked by estimated traffic.
3. Whatever else is in your sitemap.

The list is filtered to your own domain and its subdomains only, deduplicated so that www and non-www variants of one page are not counted twice, and then capped at 12 pages. The cap is deliberate: the crawl runs live and in parallel, and twelve pages return in seconds.

Comparison then runs over the crawled titles and descriptions:

• Values are normalised before comparison — lowercased, whitespace collapsed, and trailing separators such as a dangling | or - removed. So Shoes | and shoes are treated as the same title.
• Exact groups are two or more pages sharing an identical normalised value.
• Near-duplicate groups are pages that are not identical but share at least 80% of their words, clustered together.
• Missing is any crawled page with no title, or no meta description, at all.

Every page in the report was fetched on this run. Nothing is estimated.

Benefits#

• Finds the collisions that matter. By crawling your top-traffic pages first, the report is about pages that earn something, not about pagination stubs.
• Near-duplicates, not just exact matches. Blue Running Shoes | Acme and Running Shoes Blue | Acme never match a string comparison, and they are exactly the pair you want to find.
• A clean-page percentage that tells you the size of the problem in one number.
• Groups, not a flat list. Each collision shows every page sharing the value, so you fix a set rather than a symptom.

Use Cases#

• Post-migration check. A template change has propagated one title across a whole section, and nobody noticed.
• E-commerce variant sprawl. Color and size variants of one product each get their own indexable URL with the same title.
• Content audit prep. You need a defensible list of pages to rewrite before a quarterly content review.
• Ranking cannibalisation. Two pages keep swapping positions for one query; this shows whether their titles are telling Google they are the same page.
• CMS validation. You confirm that a new template is actually emitting unique titles and descriptions before it ships site-wide.

Requirements#

• A paid plan. The audit costs 1 credit, and the Free plan cannot run any tool that costs credits.
• A publicly crawlable domain. A site that blocks automated crawlers returns an error rather than a partial report.
• Ideally a working sitemap, which widens the page list beyond your top organic pages. Check it with Sitemap Validator.
• No integration and no verified domain.

Permissions#

Available to every team role on a paid plan. On the Free plan the run is refused with:

This tool needs a paid plan. Free includes the 10 technical SEO tools; upgrade to Pro to unlock the rest.

At 1 credit the tool sits below the premium threshold of 3, so on a paid plan it is never blocked by your monthly quota. Each run still adds 1 to your recorded usage.

Cost#

1 credit per run.

The audit is not held in the shared data cache, so every run is a live crawl and every run is charged. Runs appear in the Library's Activity timeline as Ran a tool, but they are not stored as reopenable saved work, so there is no free re-open and no You already ran this prompt. Export the result if you need to keep it. See How credits work and Result caching and freshness.

Navigation Path#

Dashboard → Site Health → Duplicate Content Audit

Inputs#

Private, loopback and internal addresses are refused before any fetch happens.

Step-by-Step Guide#

1. Open Site Health in the left rail, then select Duplicate Content Audit.
2. Type your domain in the field marked e.g. nike.com, or select Try sample to run against nike.com.
3. Select Run Duplicate Audit, or press Enter.
4. Wait while the panel reads Analyzing… this can take a few seconds for live data. Twelve pages are fetched in parallel, so this is usually quick.
5. Read the tier badge in the header, then the four KPI tiles.
6. Work through Colliding Title Tags first, then Colliding Meta Descriptions, then Missing Tags.
7. Use All Crawled Pages to confirm which pages were actually inspected.
8. Export with Export PDF, Export Excel, Export CSV, Export JSON or Share public link at the top of the card.

Reading the Results#

The header#

The domain, a Live crawl verified badge, N top pages compared, and one tier badge summarizing the whole audit:

DUPLICATES FOUND is the one to act on today. NEEDS REVIEW is a backlog item.

The four KPI tiles#

Note that the duplicate counts are sets, not pages. Duplicate Titles: 2 can mean six pages across two collisions.

What we found#

A short list of plain-language findings, one per problem type, with the page counts attached — for example "3 set(s) of pages share an identical title tag (7 pages). Search engines may pick the wrong page to rank, or treat them as competing."

When nothing is wrong the section says so explicitly: "No duplicate or near-duplicate titles or descriptions found across the N pages crawled. Every page has a distinct title and description."

Colliding Title Tags#

One card per collision set, labelled N set(s) in the section header.

• An EXACT card shows the shared title in quotes, Shared by N pages, and the path of every page carrying it.
• A NEAR card is headed Near-identical across N pages, lists each page's own title so you can see how they differ, and then the paths.

Fix an EXACT set by giving each page a title that names what is unique about it. If the pages are genuinely the same content on different URLs, the fix is not a title rewrite — it is a canonical tag or a redirect.

Fix a NEAR set by asking whether the pages target different queries. If they do, make the titles reflect that difference. If they do not, you probably have two pages where one belongs.

Colliding Meta Descriptions#

The same card structure for meta descriptions. Duplicate descriptions rarely change rankings directly, but Google will often discard a description that does not match the query and write its own snippet, so you lose control of the result text. Treat these as a click-through problem.

Missing Tags#

Shown when any crawled page has no title or no meta description, split into two rows: N page(s) with no title tag and N page(s) with no meta description, each with the paths listed.

A missing title is more serious than a duplicate one. Google will invent something from the page content, usually the first heading, and you have no say in it.

All Crawled Pages#

A table with three columns — Page, Title and Description — and one row per crawled page. Each cell carries a status badge:

Read this table for two reasons. First, it confirms which twelve pages were actually inspected, which matters when the report looks cleaner than you expected. Second, it is the fastest way to hand a list to whoever is doing the rewriting.

Recommendations#

When collisions are found, the findings are handed to the Get recommendations panel below the result, ranked with exact duplicates and missing titles as high-severity, near-duplicates as medium, and missing descriptions as low. Each item can route you to the screen where the fix belongs. The panel is a paid feature; see Get Recommendations.

Examples#

Screenshots#

Tips#

• Run it after any template change, not before. The problems this finds are almost always template-shaped.
• Read Clean Pages first. A number under 70% means stop looking at individual pages and go and read the template.
• If Pages Crawled is low, fix the sitemap first and re-run. Discovery quality sets the value of the whole report.
• Export to CSV and hand the All Crawled Pages table straight to whoever writes the new titles.

Best Practices#

• Give every indexable page a title that would make sense on its own in a search result, without the rest of the site around it.
• Put what is unique about the page at the front of the title, and the brand at the end.
• Write a distinct meta description for pages that earn traffic; leave it off entirely rather than repeating boilerplate on pages that do not.
• Where two pages genuinely are the same content, use a canonical tag or a redirect. Rewriting the titles of duplicate pages hides the problem rather than solving it.
• Re-run after the rewrite to confirm the collisions have actually cleared.

Common Mistakes#

• Assuming the run is free despite the · 1 report chip. It charges 1 credit.
• Reading the duplicate counts as page counts. They are sets. One set can hold many pages.
• Expecting a whole-site crawl. Twelve pages is the cap, chosen so the crawl finishes live.
• Fixing the twelve pages listed and stopping. They are a sample of your most important pages. A collision here almost always repeats across pages that were not crawled.
• Rewriting near-duplicates without asking whether both pages should exist. Sometimes the correct fix is a merge.

Limitations#

• Maximum 12 pages per run. The page list is capped so the parallel crawl completes within the run window.
• Only title tags and meta descriptions are compared. Body content, headings and image alt text are not.
• Near-duplicate detection uses word overlap at an 80% threshold. Two titles that share few words but mean the same thing are not flagged.
• Discovery covers your own domain and subdomains only. Pages that are neither in the sitemap nor among your top organic pages are not seen.
• Results are not cached and are not stored as reopenable saved work, so every run costs a credit.
• The tool does not check canonical tags, so it cannot tell you whether a duplicate set is already correctly canonicalised. Use On-Page SEO Checker on an individual page for that.

Troubleshooting#

FAQs#

Why only twelve pages? Because the crawl is live. Every page is fetched at the moment you press the button, in parallel, and the cap keeps the run inside its time budget. Twelve of your highest-traffic pages will surface a template-level problem; a thousand pages would mostly repeat it.

Does it find content duplicated on other websites? No. This tool compares your pages against each other. For text copied elsewhere on the web, use the Plagiarism check inside Pre-Publish Trust Checks.

Why does it charge a credit when the button does not say so? The credit chip is missing from the button, which is a display gap. The run costs 1 credit because it uses a paid data provider both to rank your top pages and to crawl them.

Can I re-open a previous audit for free? No. Unlike the premium research tools, this one is not stored as reopenable saved work, so there is no free re-open. Export the result if you need to keep it.

What counts as "near-identical"? Two values that share at least 80% of their words after normalisation. That threshold catches reordered and lightly edited titles while ignoring pairs that merely share a brand suffix.