To limit content appearing in Google Search, first decide whether you want to restrict indexing, snippet display, or access to the content. Use noindex to keep a page out of the index, data-nosnippet to keep specific portions of content out of snippets, and password protection for confidential or private content. robots.txt primarily controls crawling and should not be treated as a universal tool for removing pages from search. If content you own already appears in search results, you can also submit a removal request.
First, identify content that should not be publicly searchable
Limiting visibility is not just about keeping information private; it can also keep low-value content out of search. Google’s documentation lists the following situations:
- Information intended only for users of your site; when publishing files, also be mindful of metadata that may appear in search.
- Low-quality or spammy user-generated content; allowing this content to be indexed may negatively affect your site’s search rankings.
- Large amounts of duplicate content, or secondary pages on very large websites; limiting Google’s crawling of this content can help it focus on important content.
The last approach depends on the site’s size and how much duplicate content it has. It should not be applied indiscriminately to every website.
To prevent indexing, do not just block crawling
Crawling and indexing are separate processes. According to Google, robots.txt can prevent Googlebot from crawling a page, but it cannot guarantee that the page’s URL will not appear in search results.
If your goal is to prevent Google from indexing or displaying a page, use a noindex rule. For example, if you want visitors to be able to read a page through internal links but do not want it to appear in search results, use noindex rather than restricting everyone’s access. Users can still visit the page by entering its URL directly or following links from other pages, so noindex does not keep content private.
Private information needs access protection, not a hidden URL
Simply keeping links unpublished is generally not enough to keep a website hidden. Google’s documentation notes that when users follow outbound links from a website, the original site’s URLs may appear in referrer information and be logged and published by other websites.
Confidential or private content should be password-protected to ensure that only authorized users can access it. Google explains that this can also prevent the content from appearing in search or remove it from search results if it already appears there. If the content should no longer be available on your website, you can delete it. However, deleting it from your site does not mean every copy on the internet will also be removed.
Choose the right controls for restricting snippets or media
If you only want to keep a particular passage out of snippets, there is no need to remove the entire page from search. Google’s available methods include:
- data-nosnippet: Excludes specific portions of content from search snippets.
- max-snippet: Limits the number of characters displayed in search snippets.
- robots.txt rules for images and videos: Block Googlebot from accessing media files; Google says it only indexes images and videos it is allowed to crawl.
Search preview controls also apply to AI Overviews and AI Mode. However, limiting what appears in snippets is different from preventing users from opening the original content.
Limiting visibility in Google Search: perspectives from across the web
Can you restrict your site's appearance in AI Overviews and AI Mode while retaining eligibility for traditional search?
Yes, for eligible sites. According to reports from August 2026, Search Console's generative AI controls for search allowed site owners to opt their entire site out of AI Overviews, AI Mode, and generative Discover without affecting eligibility for traditional search. However, the opt-out setting was still being tested at the time, and only some UK site owners were eligible. Check your site's Search Console settings to see whether the option is available.
Does blocking Google-Extended opt you out of AI Overviews?
No. According to reporting from 2026, Google-Extended controls whether content can be used for specified AI training and grounding purposes. It does not affect indexing in Google Search, and blocking it does not opt a site out of AI Overviews or AI Mode. Search Console has separate controls for whether a site appears in generative AI search features. Before choosing a setting, decide whether you want to restrict how AI uses your content or whether your site appears in search features.
What are the trade-offs of using traditional search restrictions to reduce visibility in AI Overviews?
The effects may extend beyond AI Overviews. For example, noindex also removes a page from regular search results, while restricting crawling or snippets may affect visibility in regular search. Semrush therefore advises against using these methods without a compelling reason. If you only want to control snippet content, first consider the scope of the restrictions rather than removing the entire page from search outright.
Can you restrict content from appearing only in specific Google properties?
Yes. You can instruct Google not to include webpage content in specific properties, such as Google Shopping, Google Hotels, and vacation rentals. If you only want to limit visibility in these properties, you do not need to remove the entire page from search. Follow the applicable guidance for each property when choosing its settings.
References
- 訂閱和付費牆內容標記 | Google 搜尋中心 | Documentation | Google for Developersdevelopers.google.com — Discusses: What are the trade-offs of using traditional search restrictions to reduce visibility in AI Overviews?
- Googlebot 簡介 | Google 搜尋中心 | Documentation | Google for Developersdevelopers.google.com — Source used in the article above.
- 控管您在 Google 搜尋分享的內容 | Google 搜尋中心 | Documentation | Google for Developersdevelopers.google.com — Discusses: Can you restrict your site's appearance in AI Overviews and AI Mode while retaining eligibility for traditional search?; Can you restrict content from appearing only in specific Google properties?
- What Top Stories Inside AI Overviews Means For Publishers And Brands In 2026 And Beyondwww.searchenginejournal.com · 2026-08-04T19:00:12+00:00 — Discusses: Can you restrict your site's appearance in AI Overviews and AI Mode while retaining eligibility for traditional search?; Does blocking Google-Extended opt you out of AI Overviews?
- AI Overviews: What Are They & How to Optimize for Themwww.semrush.com · 2024-05-31T14:20:00Z — Discusses: What are the trade-offs of using traditional search restrictions to reduce visibility in AI Overviews?
- Google AI Payment Pilot, Search Profiles At 10,000 – SEO Pulsewww.searchenginejournal.com · 2026-09-19T12:30:42+00:00 — Discusses: Does blocking Google-Extended opt you out of AI Overviews?

