Review real URL responses, distinguish crawling from indexing and verify canonical, sitemap and structured-data signals with an evidence-based release checklist.
List representative indexable pages, duplicate variants, redirects, removed pages and private areas. Define which URL should represent each public document and make internal links use that choice consistently. Inspect actual response statuses and the rendered page content. Record the intended behavior alongside the observed result for each URL type.
Check robots.txt rules against the resources and pages that need crawling. Inspect HTML robots directives and X-Robots-Tag response headers for indexing intent. A robots.txt block is not an access control and can prevent a crawler from seeing a noindex instruction. Protect private content with appropriate authentication rather than relying on crawler preferences.
Choose consistent canonical URLs for equivalent content, use appropriate redirects for moved pages and check for chains or loops. Keep canonical links, internal links and sitemap entries aligned. Inspect actual rendered tags and the final response destination. Do not treat duplicate URLs as an automatic penalty or a canonical annotation as a guaranteed selection.
Write useful visible content with a descriptive title, heading, links and image alternatives. Give pages suitable descriptions without treating a character count as a guaranteed search-snippet length. Add only supported structured data that describes visible, accurate content and validate it. There is no universal word-count target that makes a page rank.
Generate a sitemap or sitemap index from the intended canonical public URLs. Confirm it is reachable, valid and free of unintended redirects or private entries, then submit it in the appropriate search-console account. Check selected pages with URL Inspection and structured-data tools. Sitemap submission is a discovery hint, not an indexing guarantee.
After the normal release process, inspect representative pages in Search Console and compare Google’s selected canonical and indexing state with your intent. Track discovered problems, changes and follow-up dates. Review search performance over defined periods while separating technical fixes from ranking outcomes. Download the URL inventory, checks and unresolved items.
Loading your checklist…
Google documents canonicalization methods and recommends consistent signals. Robots.txt is not a canonicalization method and blocked URLs can still appear in search.
Google documents HTML robots tags and X-Robots-Tag response headers. A crawler must be able to access the relevant response to discover its directives.
Google describes supported sitemap formats and submission methods. Submission provides a hint and does not guarantee crawling or indexing.
Google’s starter guide rejects a magic word-count target and explains that duplicate URLs alone are not a manual-action penalty.
Keep these task-specific records with the tested version and review date.
| Record | Include | Verify |
|---|---|---|
| Intended page | Canonical URL and public/indexing intent | Observed status, content and directives |
| Alternate URL | Duplicate, moved or removed reason | Correct destination or removal behavior |
| Search evidence | Inspection date and selected canonical | Intent compared with observed indexing state |
Crawler directives are public preferences, not authentication.
Inspect the content and tags served to users and crawlers; source templates alone do not prove the delivered page is correct.