If a page needs to remain publicly accessible but should not appear in search results, you can use noindex. If the content should not be public, use password protection or take the page offline. To exclude a single page, configure that page; to exclude an entire site, cover every page you want to keep out of search results. You can set noindex before a page is indexed. For pages already appearing in search results, changing the settings does not make them disappear immediately, and temporary hiding is not permanent removal. Using Disallow in robots.txt alone does not guarantee that a URL will stay out of search results.
Blocking crawling does not prevent indexing
robots.txt and noindex serve different purposes. A Semrush article dated 2018 explains that robots.txt tells crawlers which paths they may or should not crawl. Even if crawling is blocked, a URL may still be indexed because of external links. Disallow therefore cannot be treated as a switch for hiding pages from search results.
Articles published by Welly and Semrush in 2024 explain that noindex asks search engines not to index a specified page. If you use an on-page noindex tag, crawlers must be able to read the page. Semrush also explicitly lists this as a requirement.
What do you need to configure for one page or an entire site?
- Excluding just one page: For example, you can set noindex on an expired event page that you still want people to be able to view. Welly provides the tag <meta name=”robots” content=”noindex”>.
- Excluding an entire website: Configuring only the homepage is not enough. You need to configure every page you do not want indexed. If the entire site is not yet ready to be public, consider password protection.
- Handling files such as PDFs: Semrush also describes using X-Robots-Tag in an HTTP response header to ask search engines not to index a file, rather than using an on-page tag.
A forum post mentions an “Allow search engines to index this site” toggle in a website control panel, but does not identify the platform. You therefore cannot assume that all websites have the same menu or setup process.
What if the page already appears in search results?
You can set noindex on test or draft pages before they are indexed. If the content should not be public in the first place, prioritize restricting access. If a page already appears in search results, you also need to adjust its indexing settings or decide whether to keep it public, rather than just hide the search result.
Older Search Console instructions preserved on a forum describe requesting that URLs be temporarily hidden. They state that the URLs must belong to a property you own and that requests are not guaranteed to be accepted. They also explicitly distinguish temporary hiding from permanent removal. Because these instructions describe an old interface, you cannot assume that the buttons, URL-matching behavior, or processing times are still the same.
Keeping content out of search results does not make it private
For pages that remain public, noindex controls whether search engines index them, not who can read them. Private content should be password-protected or the page taken offline. If information is published on someone else’s website, you can contact the site administrator to request removal of the original content. You cannot directly apply your own site’s settings to a website you have no authority to modify.
Keeping websites and pages out of search results: perspectives from across the web
How can you keep navigation links on a page while keeping the page out of search results?
Consider allowing search engines to crawl the page while setting noindex on it. One discussion specifically addresses the situation where someone wants links on a page to be followed without the page appearing in search results, and recommends removing the robots.txt block and using noindex. This is suitable for pages that still need to provide publicly accessible navigation. If the page contains private information, access should be restricted separately.
If your personal information appears on someone else’s website, can you ask Google to remove it?
You can try submitting a removal request to Google, but removal is assessed on a case-by-case basis; submitting a request does not guarantee approval. If the information is published on a single website, you can also contact the site administrator directly to request removal of the original content. This is more direct than addressing only one search engine’s results, since other search engines may also have the information.
What else should you consider when requesting that a URL already appearing in search results be temporarily hidden?
Be aware that the same page may have URL aliases or URLs with different capitalization, and check whether the request covers the versions you need to address. Older instructions preserved on a forum recommended submitting aliases and capitalization variants accepted by the server separately, and noted that requests were not guaranteed to be accepted. Do not assume those older instructions still describe the actual matching behavior or steps to follow; use the interface and rules in effect when you submit the request.
References
- 請教先進 : 如何讓姓名不被google搜尋資訊www.mobile01.com — Discusses: If your personal information appears on someone else’s website, can you ask Google to remove it?
- Robots.txt Explained: Syntax, Best Practices, & SEOwww.semrush.com · 2018-07-10T11:00:00Z — Discusses: How can you keep navigation links on a page while keeping the page out of search results?
- Noindex是什麼?了解3種設定方法與使用時機,讓你提升網站排名!welly.tw · 2024-11-11T08:07:06.770Z — Source used in the article above.
- How to hide links from search engineswww.warriorforum.com — Discusses: What else should you consider when requesting that a URL already appearing in search results be temporarily hidden?
- 請教先進 : 如何讓姓名不被google搜尋資訊www.mobile01.com — Discusses: If your personal information appears on someone else’s website, can you ask Google to remove it?
- Robots.txt and sitemapswebmasters.stackexchange.com — Discusses: How can you keep navigation links on a page while keeping the page out of search results?
- 請教先進 : 如何讓姓名不被google搜尋資訊www.mobile01.com — Discusses: If your personal information appears on someone else’s website, can you ask Google to remove it?
- 請教先進 : 如何讓姓名不被google搜尋資訊www.mobile01.com — Discusses: If your personal information appears on someone else’s website, can you ask Google to remove it?
- What Is Noindex Used for? An Overview + Best Practiceswww.semrush.com · 2024-07-31T12:14:00Z — Source used in the article above.
- How to hide links from search engineswww.warriorforum.com — Source used in the article above.
- How to hide links from search engineswww.warriorforum.com — Source used in the article above.



