How can you use tools to audit redirects and trace how URLs were discovered?

ProcrustesPR

To audit redirects, first list the redirecting URLs and their responses, then check redirect chains, loops, and final destinations—not just whether the first hop works. To trace how a tool discovered a URL, start with inbound links pointing to it and consult the crawl path report if necessary. The instructions below come from different years, so note the dates and versions cited; current interfaces may differ.

ProcrustesPRProcrustesPRCan a One-Page Website Do SEO?

List redirects first, then identify chains and loops that need attention

A spreadsheet can track redirects, but maintaining it can become cumbersome. Unrecorded redirects are also easy to forget and may remain in place after they are no longer needed. A 2023 Semrush article explains that after creating a project and configuring an audit in Site Audit, you can view the redirect list under “Redirects,” then search for “redirect chain” under “Issues” to check for redirect chains and loops.

  • Check whether each redirect type suits its purpose, rather than simply counting redirects.
  • Track temporary 302 redirects, such as those used for maintenance or short-term campaigns, and remove them when they are no longer needed.
  • Where there are unnecessary consecutive hops, simplify the redirect to point directly to the final destination.

Redirect errors can affect the user experience and sometimes hinder indexing, but this does not mean every redirect chain necessarily causes a drop in rankings.

Trace how a URL was discovered, beyond the links pointing directly to it

Screaming Frog’s 2015 guidance recommends selecting the URL in question and checking the “in links” tab below. In most cases, this is a quick way to find links pointing to that URL.

However, incorrect relative links can generate large numbers of URLs, making it laborious to trace them back step by step. URLs may also be discovered through canonical elements rather than ordinary HTML links. According to the instructions from that year, you can right-click a URL and export a “crawl path report,” then read it from bottom to top to follow the shortest path by which the crawler discovered the URL.

The 2013 article on version 2.20 also offers a manual tracing technique: sort the relevant sequence of URLs alphabetically and identify the shortest URL within it. This technique applies to particular sequences, not to determining the source of every URL. Here, “source” refers only to the path by which the crawler discovered a URL.

Verify migration results all the way through to the final response

A 301 response on the first hop does not mean a migration is complete. The 2013 article gives an example in which a URL passes through two successive 301 redirects but ultimately returns a 404. Checking only the initial redirect would miss the problem.

Screaming Frog’s 2013 guidance for version 2.20 and its 2015 guidance describe uploading a list of old URLs and enabling “Always follow redirects” to trace them to their final destinations. The version 2.20 guidance states that this option also ignores crawl depth limits in list mode and continues following redirects.

  • Check whether old URLs reach the correct new destinations.
  • Inspect the final responses for results such as no response, 4XX, or 5XX.
  • Use the mapping report to check the response, hop count, and presence of loops for every uploaded URL, rather than only entries with redirect chains.

Semrush’s 2021 guidance recommends a comprehensive audit after a website redesign, migration, or domain update, as well as regular checks. If you find a loop, check for conflicting rules in plugins or custom scripts, but do not assume they are the cause without verification.

ProcrustesPRProcrustesPRHow to Use a Marketing Funnel: Fill Page Gaps at Each Stage, Then Build a Conversion Path

Redirect auditing and tracing URL discovery: perspectives from across the web

Which type of redirect can be used for A/B testing or serving different pages by device?

Semrush’s 2023 article lists A/B testing and directing users to different pages based on their location or device as use cases for 302 temporary redirects. A 302 indicates that a page has moved for a short period, and search engines may not update their results to the new URL. If the need is not temporary, this recommendation should not be applied merely for convenience.

How do redirect chain reports and migration mapping reports differ in coverage?

Screaming Frog’s 2013 notes for version 2.20 state that the redirect chain report identifies a chain’s starting point, number of hops, and whether it contains a loop. The migration mapping report introduced in 2015 includes every URL in the original uploaded list and its response, not just URLs with redirect chains. When checking old URLs individually, make sure the report covers the full list. These are descriptions of features at the time, not guarantees that the current interface is the same.

ProcrustesPRWhat types of URLs are there, and what are they used for?

References

  1. What Are Crawl Errors & How Do They Affect SEO?www.semrush.com · 2021-06-10T16:30:00Z — Source used in the article above.
  2. Screaming Frog SEO Spider Update – Version 2.20www.screamingfrog.co.uk · 2013-07-15T11:06:00+00:00 — Discusses: How do redirect chain reports and migration mapping reports differ in coverage?
  3. 10 Features In The SEO Spider You Should Really Knowwww.screamingfrog.co.uk · 2015-01-22T11:48:45+00:00 — Discusses: How do redirect chain reports and migration mapping reports differ in coverage?
  4. How to Set Up an HTML Redirect (+ Alternatives)www.semrush.com · 2023-07-24T08:57:00Z — Discusses: Which type of redirect can be used for A/B testing or serving different pages by device?
  5. What Is the ‘Too Many Redirects’ Error? & How to Fix Itwww.semrush.com · 2021-08-13T15:59:00Z — Source used in the article above.

Related articles

Can a One-Page Website Do SEO?How to Use a Marketing Funnel: Fill Page Gaps at Each Stage, Then Build a Conversion PathWhat types of URLs are there, and what are they used for?When should you use relative or absolute paths in WordPress?What are www subdomains and relative links used for?How to Create an HTML Sitemap: From Organizing Content to Publishing and UpdatingHow to troubleshoot and fix 404 errors: identify broken URLs first, then address the causeHow to improve on-site content and structure: audit pages first, then address content gaps and internal links
ProcrustesPR
ProcrustesPR