Redirect / Canonical / Indexability Checker
Places the status chain, final URL, canonical, robots directives, hreflang, and sitemap evidence for one public URL in a single report.
Web / SEOA crawler does not judge a redirect, a canonical, or a robots directive in isolation. It assembles a case from every response, link, annotation, and repeated observation — and when those witnesses disagree, your preferred URL is merely one opinion in the room.
Indexability is not a property stored in one tag. A URL has an HTTP status, a response body, header directives, an HTML robots directive, a canonical target, internal links, sitemap membership, and possibly language annotations. The destination may have another set of all the same things. Search engines reconcile that evidence over repeated crawls to decide whether a document can be indexed and, separately, which URL should represent it. That second decision is canonicalization. Teams routinely blur the two and then wonder why an indexable page is omitted or a noindexed URL remains known.
Treat the system as a directed graph. Redirects are strong edges that move crawlers and users. Canonicals are hints that nominate a representative without moving anyone. noindex controls whether a fetched resource may appear in results; it does not nominate a replacement. Hreflang links equivalent regional or language pages but does not override their canonical choices. Sitemaps and internal links reinforce the URLs you operate as primary. The practical requirement is dull but strict: every layer should tell the same story. The Redirect / Canonical / Indexability Checker is useful because it places the served evidence for one public URL side by side instead of pretending one field settles the matter.
A 301 says the move is permanent: clients should use the target, caches may retain the mapping, and search engines should consolidate signals there. A 302 says the source remains the durable identity and the detour is temporary. Modern search engines can infer permanence despite the wrong code after enough time, but relying on that inference is sloppy engineering. Use a permanent redirect for domain migrations, normalized schemes and hostnames, retired paths with a true successor, and stable trailing-slash policy. Use a temporary redirect for maintenance, experiments, geo or login flows where the original URL is meant to return.
Status semantics are not the whole HTTP-method story. 301 and 302 have historically allowed a non-GET request to become GET; 307 and 308 preserve method and body, with temporary and permanent meaning respectively. That distinction matters to APIs and form endpoints even when SEO is not the concern. For ordinary document URLs, the larger failure is the accidental chain: http to https, apex to www, uppercase to lowercase, then no slash to slash. One hop is routine. Four hops add latency, consume crawl work, complicate caching, and give every future migration another place to break. Redirect directly from every known legacy form to the final canonical URL.
<link rel="canonical" href="…" > is designed for duplicate or substantially similar documents that must remain reachable. It says which URL you would like search engines to consolidate into; it does not force them to comply. A self-referencing canonical on the final 200 page is a clean default because it makes the intended identity explicit. A canonical pointing to a redirect, a 404, a noindexed page, another language with different content, or a URL outside the internal-link structure is weak testimony contradicted by the site itself.
Canonical loops and chains are avoidable configuration debt. If A canonicals to B and B canonicals to C, change A to C. If A redirects to B while B canonicals back to A, you have built a disagreement no crawler can honor literally. Cross-domain canonicals are legitimate for syndicated copies, but they require confidence that the destination is stable and actually equivalent. They are not a substitute for redirects during a site move because users, referrers, caches, and non-search clients still remain on the old host. Audit the final HTML as served, not just a framework component: client-side insertion is late, crawler-dependent, and invisible to tools that correctly inspect the raw response.
A robots meta tag applies to the HTML document that contains it. X-Robots-Tag carries comparable directives in the HTTP response and also works for PDFs, images, and other non-HTML resources. When both exist, do not expect an optimistic index,follow in markup to cancel a header-level noindex. Restrictive directives combine. The header often comes from a CDN rule, reverse proxy, security plugin, or staging configuration, which is why template reviews miss it. Inspect the actual response with an HTTP Header Analyzer, including every redirect hop.
noindex must be seen to be obeyed. Blocking the same URL in robots.txt prevents a compliant crawler from fetching the directive; the URL can then remain indexed from links and other evidence, often without a snippet. Allow crawling while the noindex is being processed, then remove the URL from sitemaps and internal navigation. Also distinguish nofollow from noindex: it concerns following links on the page, not whether the page itself may appear. Broad X-Robots-Tag: noindex rules attached to file extensions or entire CDN routes deserve special suspicion because their blast radius is larger than any visible template suggests.
Each hreflang URL should be a final, indexable, canonical 200 page. The annotation set should include the page itself and reciprocal return links; an optional x-default names the language-neutral selector or fallback. Pointing hreflang at redirects makes crawlers resolve an annotation before they can use it. Pointing English to a German page whose canonical selects English tells the engine that two language variants exist and that only one should represent both. It may discard the annotation, the canonical hint, or both.
Canonicalize within each language unless the pages are genuinely duplicates rather than translations. The English page should canonically select English and annotate German; German should select German and annotate English. Keep URL forms exact across HTML, sitemap, redirects, and internal links — scheme, hostname, escaping, case, and trailing slash included. Broken hreflang does not directly make a page non-indexable, but it prevents the right indexed page from being served to the right audience. From the outside, that often gets reported as “Google will not index the German page,” when the actual failure is an incoherent cluster.
Start with one exact URL, not the domain in general. Record every status and Location value until the final response. On that final URL, compare the canonical, robots meta, X-Robots-Tag, hreflang targets, sitemap membership, and prominent internal links. Repeat the check for a few variants you expect to normalize: HTTP, alternate host, legacy path, parameters, and slash form. The correct outcome is boring — each variant takes at most one intentional hop, and every remaining signal names the same final URL.
Then look for policy boundaries. A crawler blocked in robots.txt can neither confirm a canonical nor discover a noindex in the body; the AI Crawler robots.txt Checker can trace which group and source line wins for a concrete path, but remember that search, training, and user-triggered agents are different policy audiences. Finally, fix the source layer rather than adding another override. Remove the stale proxy header, change the redirect map, correct the canonical generator, and regenerate sitemaps. Search engines weigh signals because the web is inconsistent. Your job is not to guess the weighting formula; it is to stop asking the engine to arbitrate a dispute your own stack created.