How to remove indexed content from search results: first distinguish between keeping it, deleting it, and restricting access

ProcrustesPR

To remove indexed content from search results, first check whether you control the source page and whether the content needs to remain available. For Google, use noindex if the page should stay public; remove pages that are no longer needed and have their URLs return 404; use password protection if only authorized readers should have access. Blocking crawling through robots.txt alone does not remove a page from search results, and temporary URL removal is not a permanent solution.

ProcrustesPRProcrustesPRHow Long Does It Take to Evaluate SEO Results?

First decide: who should still be able to see the content?

If you can modify the source website, choose an approach based on the content’s purpose, not just how quickly you want it to disappear from search results:

  • Keep the page public, but exclude it from search results: A 2024 Semrush article explains that you can add noindex. Public technical discussions also describe setting noindex through a page’s robots meta tag or the X-Robots-Tag HTTP response header.
  • The content is no longer needed: Public technical discussions recommend having the URL return 404 to signal that search engines should remove it from their indexes. What matters is the status the URL actually returns; do not assume that simply deleting a file will achieve this on every website.
  • The content must not be publicly accessible: For content such as internal documents or members-only areas, Semrush recommends password protection to limit access to authorized users. Pages that are already indexed will eventually be removed from Google’s search results as well.

After adding noindex, let crawlers read it

robots.txt controls whether crawlers can crawl a page, while noindex tells search engines not to include the page in their indexes. A 2026 Search Engine Journal overview clearly distinguishes the two. A public technical answer also warns that blocking crawling prevents crawlers from reading the page’s noindex directive. The URL may still appear in search results, but without a description.

For example, adding noindex, nofollow to a mobile version under /m/ is no guarantee that it will “definitely disappear after the next crawl.” First check that robots.txt is not also blocking the page. The guidance above supports using noindex to remove a page from the index but gives no exact completion time for this case. Finishing the setup does not mean the search results have already been updated.

After temporary removal, the source page still needs attention

In a 2022 article, awoo describes the URL removal tool as a temporary measure lasting about six months. For long-term removal, it recommends having the URL return 404 or using noindex. This reflects the article’s explanation at the time, not necessarily the tool’s full current rules.

Google’s guidance on documents whose content has not been fully redacted also explains that removal requests expire after the index is updated or after about six months. If other websites also host the document, the guidance recommends contacting them to request removal or using the Remove Outdated Content tool to request an update to search results. These recommendations apply to that document-handling scenario; they do not mean you can arbitrarily delete content from other websites.

ProcrustesPRProcrustesPRWhere should I start if I want to do SEO myself?

Removing content from search results: perspectives from across the web

When someone searches for a company’s domain, can you keep only the homepage in the results and remove the other pages?

You can target internal pages you do not want indexed, but these measures alone do not guarantee that only the homepage will appear when someone searches for the company’s name. If you control the website, consider adding noindex to those pages rather than simply blocking crawling through robots.txt. Some participants in public discussions also suggest explaining the trade-off to the client first: excluding internal pages reduces their visibility in search, so it may not be a good idea.

How do you remove only the AMP version while keeping the standard page?

If a website has both AMP and standard non-AMP versions, Google’s guidance is to first remove the rel="amphtml" link from the standard page’s source code, then redirect the AMP URL to the standard page using a 301 or 302 redirect. Afterward, you can search for the content to check whether the AMP version has been removed; for large numbers of pages, you can follow the guidance to monitor trends in the number of indexed AMP pages. This procedure applies to the AMP configuration described and is not a general rule for handling mobile versions of pages.

Why are there still crawl errors after a URL has disappeared from search results?

Google may still find a URL through links even after it disappears from search results. A public technical discussion notes that, as long as Google keeps finding links to the URL, it may still appear among crawl errors in Search Console even if it returns 404 or 410. The respondent considers this normal and says it will not affect other pages on the website. That explanation applies only to the situation described; it does not mean all crawl errors can be ignored.

ProcrustesPRShould I rewrite or update an old article that does not rank?

References

  1. 避免 Google 顯示受遮蓋的資訊 | Google 搜尋中心  |  Documentation  |  Google for Developersdevelopers.google.com — Source used in the article above.
  2. 如何從 Google 搜尋中移除您的 AMP 網頁 | Google 搜尋中心  |  Documentation  |  Google for Developersdevelopers.google.com — Discusses: How do you remove only the AMP version while keeping the standard page?
  3. How to Remove URLs from Google Search Enginewebmasters.stackexchange.com — Discusses: Why are there still crawl errors after a URL has disappeared from search results?
  4. What Opting Out Of Google’s AI Search Features Means Nowwww.searchenginejournal.com · 2026-08-01T12:00:39+00:00 — Source used in the article above.
  5. What Is Googlebot? How Google's Web Crawler Workswww.semrush.com · 2024-08-23T14:32:00Z — Source used in the article above.
  6. 【Google Search Console 完整教學】報表應用技巧、安裝驗證方式全收錄www.awoo.ai · 2022-02-18T10:01:42+00:00 — Source used in the article above.
  7. Multiple domain links on Google from one WordPress sitewebmasters.stackexchange.com — Discusses: When someone searches for a company’s domain, can you keep only the homepage in the results and remove the other pages?
  8. Effective combatting of 100's of spam links in Google Resultswebmasters.stackexchange.com — Source used in the article above.
  9. How to de-index pages from google using robots.txtwebmasters.stackexchange.com — Discusses: When someone searches for a company’s domain, can you keep only the homepage in the results and remove the other pages?

Related articles

How Long Does It Take to Evaluate SEO Results?Where should I start if I want to do SEO myself?Should I rewrite or update an old article that does not rank?What should I check when organic traffic suddenly drops?Why can I not find my published article on Google?Will Reposting Social Content on My Website Cause a Duplicate-Content Penalty?Eight SEO essentials for beginners: Getting your pages indexedWhat Is SEO? Understanding Optimization Priorities Through How Search Engines Work
ProcrustesPR
ProcrustesPR