How can you trace a URL’s source and redirect chain?

ProcrustesPR

When tracing a single URL, separate two questions: which page led the crawler to it, and where it redirects, step by step, when opened. To answer the first, examine incoming links and crawl paths. To answer the second, check the URL and HTTP status code at each hop, along with the final destination. These methods can trace links and redirects, but they cannot establish who created the URL or first published it. A crawler’s discovery path is not the same as a URL’s historical origin.

ProcrustesPRProcrustesPRDo I have to buy backlinks for SEO?

First, find which pages link to the URL

A 2023 article on Neil Patel’s website explains that after crawling a website with Screaming Frog, you can select the URL you want to investigate and check the “inlinks” tab below to see which pages link to it. If you want to know where the website still links to an old URL, this is more useful than looking only at the final redirect destination.

However, direct incoming links may not explain how the crawler reached the URL. A 2015 Screaming Frog article notes that incorrect relative links can generate URLs with multiple nested levels, requiring you to trace the path back step by step. A crawler may also discover a URL through a canonical element rather than a standard HTML link.

  • Start with the direct sources: which pages point to the URL being investigated?
  • If those source URLs also have an unusually nested structure, trace back further to see how the crawler reached them.
  • The “crawl path report” described in that historical article shows the shortest discovery path, not a complete list of all sources. At the time, the report was read from bottom to top; the current interface may differ.

Follow every hop, not just the first redirect

If you only want to check one URL, you can use a redirect checker. The 2023 Neil Patel article describes Httpstatus.io as a tool for checking redirect chains and HTTP status codes, while the Small SEO Tools checker can show the complete redirect path. These are the features documented in that article; whether the current plans and interfaces are the same has not been verified.

The key is to follow the redirects all the way to the end. For example, A returns a 301 and redirects to B, B then redirects to C with another 301, and C returns a 404. Checking only the first hop would miss the error at the final destination. Screaming Frog used this kind of scenario in its 2013 version 2.20 release notes to explain why redirects need to be followed to the end.

  • Keep the starting URL and record each subsequent URL and status code in order.
  • Verify the final URL reached rather than treating the first redirect target as the endpoint.
  • If no page is successfully reached, record the lack of response or the error status. Do not treat the presence of a redirect as a successful check.

Distinguish redirect chains, loops, and discovery paths

A redirect from A to B, followed by B to C, is a redirect chain. If a later redirect leads back to a URL already in the chain, it creates a loop. Screaming Frog’s 2022 documentation states that its loop filter at the time listed only cases that redirected back to a previous URL. Complete chains could be viewed and exported separately through reports.

When organizing your findings, record source pages separately from redirect hops. Source pages show where the crawler found the URL; redirect hops show where the request was sent. These records complement each other, but neither can replace the other.

ProcrustesPRProcrustesPRHow can I tell whether an SEO tactic breaks spam rules?

Tracing URL sources and redirect chains: perspectives from across the web

What should you do after finding a redirect chain or loop?

If you have permission to modify the website, first confirm the correct final page, then redirect the old URL directly to it to reduce intermediate hops. Also update internal links that still point to the old URL. If there is a loop, remove the redirects causing the cycle and confirm that the chosen endpoint does not redirect back into the chain. Long chains can delay crawling and slow loading, but that does not mean every chain has caused actual harm.

If a tool marks an external link as lost, does that mean the original link has been deleted?

Not necessarily. Ahrefs explains that a link passing through a redirect chain may be recorded as lost if a page in the chain stops responding, or if a redirect is removed or changed. The same can happen if the final URL starts redirecting elsewhere. Even when the source website changes its link to point directly to the original destination, the link may be reported as lost. Check the current link on the source page and the destination of each hop before deciding what actually happened.

After a website migration, how can you verify that old URLs redirect to the intended locations?

Compile a list of old URLs, follow their redirects one by one, and compare the final destinations in the report with the expected new URLs. Older Screaming Frog documentation described uploading old URLs in list mode and setting the tool to keep following redirects so you could check each subsequent hop and the status at the endpoint. This method is useful for checking the results of a website migration, but the option names and locations may have changed in the current interface.

ProcrustesPRWill Reposting Social Content on My Website Cause a Duplicate-Content Penalty?

References

  1. Redirects: What They Are & How to Use Themwww.semrush.com · 2020-07-20T11:00:00Z — Discusses: What should you do after finding a redirect chain or loop?
  2. At Least 66.5% of Links to Sites in the Last 9 Years Are Dead (Ahrefs Study on Link Rot)ahrefs.com · 2022-04-29T06:40:30+00:00 — Discusses: If a tool marks an external link as lost, does that mean the original link has been deleted?
  3. Redirect Chain Guide: How To Identify & Fix Redirect Chainsneilpatel.com · 2023-10-09T19:00:00+00:00 — Source used in the article above.
  4. Screaming Frog SEO Spider Update – Version 2.20www.screamingfrog.co.uk · 2013-07-15T11:06:00+00:00 — Discusses: After a website migration, how can you verify that old URLs redirect to the intended locations?
  5. Screaming Frog SEO Spider Update – Version 18.0www.screamingfrog.co.uk · 2022-12-05T08:37:04+00:00 — Source used in the article above.
  6. 10 Features In The SEO Spider You Should Really Knowwww.screamingfrog.co.uk · 2015-01-22T11:48:45+00:00 — Discusses: After a website migration, how can you verify that old URLs redirect to the intended locations?

Related articles

Do I have to buy backlinks for SEO?How can I tell whether an SEO tactic breaks spam rules?Will Reposting Social Content on My Website Cause a Duplicate-Content Penalty?Eight SEO essentials for beginners: BacklinksHow can you use multiple domains to build content and traffic while assessing the risks?How do you choose a new domain and maintain external links?Microsite or multiple landing pages? Separate branding needs from ranking expectations firstDoes Google Penalize Black-Hat SEO? First, Distinguish the Tactics and Scope of Impact
ProcrustesPR
ProcrustesPR