If you want Google to index only the homepage, keep it crawlable and indexable while setting noindex on other pages. Restricting crawling through robots.txt alone does not confirm that other pages have been excluded from the index. Also, allowing the homepage to be indexed does not guarantee indexing: Google’s official documentation explicitly states that a page may not be indexed even if it meets the minimum technical requirements.
Keep the homepage eligible for indexing and set noindex on other pages
Answers on Webmasters Stack Exchange recommend setting noindex on pages other than the homepage through meta tags or HTTP response headers, rather than relying on robots.txt to control indexing. This is technical advice from the discussion, not an official guarantee from Google.
- Homepage: Do not apply settings that prohibit indexing, and confirm that the homepage is publicly accessible to Googlebot.
- Other pages: Use a noindex meta tag or return X-Robots-Tag: noindex, depending on which settings you can modify on the website.
- Scope: Keep homepage and internal-page settings separate so that shared settings do not exclude the homepage as well.
Semrush’s 2025 explanation also distinguishes crawling from indexing: after downloading a page, a search engine still needs to analyze its content. Being crawled does not guarantee indexing, and pages with noindex may be excluded from the index.
If the homepage redirects, check the destination URL
One scenario in the discussion involves /index.php redirecting to /, while the configuration allows crawling only of /index.php and blocks /. The answer points out that although the site owner considers both to be the same homepage, search engines still treat them as different URLs. Allowing only the path before the redirect therefore leaves the destination blocked.
Another historical answer proposed using Allow: /$ together with Disallow: / specifically for Googlebot. This creates an exception that allows crawling of the root path; the answer explains that $ represents the end of the URL. However, this historical approach restricts crawling and does not guarantee that other pages will be excluded from the index today. If you instead use noindex on all pages other than the homepage, base the exception on the actual homepage URL rather than simply checking whether the filename is index.php.
Check both access conditions and indexing results
According to Google’s official documentation, the homepage must, at a minimum, be accessible to Googlebot, return HTTP 200, and contain indexable content. Check these settings and results in three steps:
- Test the homepage with Search Console’s URL Inspection tool to check accessibility and the HTTP status code.
- Check the meta tags or response headers on other pages to confirm that noindex is applied to the correct pages.
- Use the Page indexing report and Crawl stats report to review indexing information and identify pages Google cannot access.
Passing the technical checks only means that the homepage meets the basic requirements for indexing. Neither completing the setup nor passing a test guarantees that Google will index it.
Homepage indexing controls: perspectives from across the web
Does having other pages indexed necessarily hurt SEO?
Not necessarily. One participant argues that having other pages indexed does not, in itself, hurt SEO, and that pages with very little value may not be indexed in the first place. This is an individual opinion, not a guarantee of results for every website. If your only concern is internal pages appearing in search results, first clarify whether you really need to exclude all internal pages, then decide the scope of the controls.
Should paginated pages and tag pages always be excluded from indexing too?
Not necessarily. Particularly on blogs or websites without a clear menu structure, paginated pages and tag pages may be important routes for search engines to discover articles. One participant recommends keeping these listing pages and separately addressing the endless combinations that repeatedly present the same content in different sort orders. This involves a trade-off with the goal of indexing only the homepage, so first confirm whether you are willing to limit the visibility of other content.
Is setting noindex through an HTTP header necessarily more effective than using a meta tag?
These discussions do not establish that HTTP headers are necessarily more effective than meta tags. Participants suggest using either X-Robots-Tag or meta tags to control indexing. The participant who described headers as more effective and preferred also said they could not find an article supporting that claim. You can choose a method based on which settings you can modify on your website, without treating a participant's preference as a proven difference in effectiveness.
References
- Google 搜尋技術相關規定 | Google 搜尋中心 | Documentation | Google for Developersdevelopers.google.com — Source used in the article above.
- Need only tens of pages to be indexed out of hundreds: Robots.txt is Okay to proceed with?www.warriorforum.com — Discusses: Does having other pages indexed necessarily hurt SEO?
- Will googlebot index my index.php even with an htaccess redirect?webmasters.stackexchange.com — Source used in the article above.
- How Search Engines Work [Explained]www.semrush.com · 2025-08-12T09:05:00Z — Source used in the article above.
- Will googlebot index my index.php even with an htaccess redirect?webmasters.stackexchange.com — Discusses: Is setting noindex through an HTTP header necessarily more effective than using a meta tag?
- SEO: Duplicate content caused by pagination/tag/index pageswebmasters.stackexchange.com — Discusses: Should paginated pages and tag pages always be excluded from indexing too?



