A cart URL blocked by robots.txt may still be listed as “indexed,” but that does not necessarily mean the block has failed: robots.txt restricts crawling; it does not guarantee that a URL will not be indexed. Google may discover a URL through links and include it in its index even if it cannot fetch the content. The right approach depends on your goal: do you only want to prevent crawling, or do you want to remove the URL from search results? It is also important to distinguish standalone cart pages from product URLs with add-to-cart parameters.
“Cannot be fetched” and “not indexed” are two different things
In June 2026, Search Engine Journal reported on a case involving URLs with add-to-cart parameters. The article quoted participants in the discussion speculating that Google may have discovered links containing these parameters on the site and added the URLs to its systems. This is one possible way Google could discover the URLs, not a confirmed explanation for every website. It also does not mean that robots.txt created these URLs.
The report quoted John Mueller as saying that these URLs do not need to be indexed and that blocking them with robots.txt is acceptable. In his assessment, even if they are reported as indexed, they are unlikely to appear in ordinary search results, although specific queries targeting the URLs may be an exception. The author, Roger Montti, questioned that assessment, emphasizing that blocking crawling alone cannot prevent URLs from appearing in search results. They differ on how likely these URLs are to appear in practice, so “unlikely” must not be treated as “guaranteed not to happen.”
To remove a URL from search results, Google must be able to read noindex
Relevant answers on Webmasters Stack Exchange note that if a page still exists but you want to remove it from search results, you can consider a meta robots noindex directive on the page or an X-Robots-Tag: noindex HTTP response header. This serves a different purpose from simply blocking crawling.
- If you only want to prevent crawling: robots.txt can restrict crawling, but it should not be treated as a guarantee of removal from the index.
- If you want Google to read noindex: you cannot also block crawling of that URL with robots.txt, or the crawler will be unable to read the noindex directive.
- If you have both blocked crawling and added noindex: do not assume this provides two layers of protection; a noindex directive that cannot be read cannot serve its intended purpose.
First, check whether the parameterized version can be configured separately
A URL with an add-to-cart parameter is not necessarily a standalone cart page. The 2026 case described above noted that the versions with and without parameters may use the same template. If WooCommerce cannot output noindex only for the parameterized version, you cannot apply this approach as described: normal product pages could also be affected. This limitation depends on whether the site allows separate settings for the two versions; it does not mean that all WooCommerce sites are unable to handle them.
Start by comparing the page tags, response headers, and blocking rules for both URL versions, then trace the sources of internal links. Montti suggested using Screaming Frog to help find these links, then assessing whether to remove unnecessary ones or add nofollow to necessary ones. However, this is the author’s recommendation; nofollow is only a hint, not an instruction to remove a URL from the index.
Cart URL indexing: perspectives from across the web
Why keep cart URLs blocked in robots.txt?
If the goal is to restrict crawling, keeping the block can be an acceptable approach. Semrush notes that blocking low-value pages such as cart, login, or filter pages helps search engines focus their crawling resources on important content, which is especially useful for sites with large numbers of URLs. However, this serves a different purpose from removing URLs from the index; blocking crawling cannot guarantee that a URL will no longer appear in search results.
Will indexed cart URLs disappear on their own if left alone?
They may drop out of the index over time, but this is not guaranteed and could take a long time. An answer on Webmasters Stack Exchange cautions that if a page still exists and links still point to it, it may not disappear entirely from search results even when blocked by robots.txt. Therefore, if you want a URL removed from search results, simply waiting for it to disappear naturally should not be treated as a reliable solution.
Besides cart URLs, what should you look for when reviewing blocking rules?
Also check whether robots.txt blocks important content that you intended search engines to access, and whether blocked content is still indexed. Search Engine Journal recommends using crawl statistics alongside page indexing reports to understand changes in crawling, rendering, and indexing status. This helps distinguish intentional cart restrictions from blocks on important content that may need attention.
References
- How To Audit Publisher Websites In 2026www.searchenginejournal.com · 2026-08-24T12:00:14+00:00 — Discusses: Besides cart URLs, what should you look for when reviewing blocking rules?
- Robots.txt Explained: Syntax, Best Practices, & SEOwww.semrush.com · 2018-07-10T11:00:00Z — Discusses: Why keep cart URLs blocked in robots.txt?
- Google Explains Why URLs Blocked By Robots.txt Can Still Be Indexedwww.searchenginejournal.com · 2026-06-17T10:58:48+00:00 — Discusses: Why keep cart URLs blocked in robots.txt?
- How to de-index pages from google using robots.txtwebmasters.stackexchange.com — Source used in the article above.
- How to de-index pages from google using robots.txtwebmasters.stackexchange.com — Discusses: Will indexed cart URLs disappear on their own if left alone?

