Google indexing

559 articles

Troubleshoot crawling, indexing, robots directives and canonicals when pages are missing from search.

ProcrustesPR
How does search engine crawl frequency change during a website migration?

During a website migration, crawl frequency does not necessarily keep increasing, and changes do not follow a fixed timeline. For Googlebot, a change in…

How should you handle old URLs and 301 redirects when migrating a website to a new CMS and changing its URLs?

When changing your CMS and URLs, first map the old URLs to the new ones, then use server-side 301 permanent redirects to send each old page to its…

What should you do if your website still hasn’t been indexed?

If your website has not been indexed, first check whether search engines have discovered and crawled its pages. Then investigate access restrictions,…

What search anomalies can appear when a hacked website generates large numbers of spam search pages?

Site owners' accounts suggest that when a hacked website generates large numbers of URLs through its internal search function, the number of unindexed pages…

Why does website speed matter, and how does it relate to rankings and crawling?

Website speed matters not only because it relates to search rankings, but also because slow loading can hinder browsing and may make it harder for search…

ProcrustesPRProcrustesPRProcrustesPRUnder what circumstances might backlinks affect rankings?
What are the pros and cons of letting search engines index user comments on a website?

Letting search engines index comments is more likely to offer benefits that outweigh the drawbacks when the comments add relevant, unique information and…

Should your website and sitemap use WWW or non-WWW URLs?

Both WWW and non-WWW URLs are valid choices; the key is to choose one primary version and keep your website settings, internal links, and sitemap…

How can you check for omissions when migrating a website’s URLs?

To check for omissions in a URL migration, start by compiling a list of old URLs, verify where each one should lead, and investigate anything missed after…

What should ongoing monitoring of website indexing and technical health cover?

To monitor website indexing and technical health over time, start by checking broken links, site speed, mobile browsing support, and indexing settings,…

How can you interpret, diagnose, and fix website indexing issues?

To troubleshoot website indexing issues, first check whether Google has crawled the page, then look at why it is not indexed. The page may not have been…

How does brief or prolonged website downtime affect Google rankings?

Brief downtime does not mean URLs will immediately disappear from Google's index. If a site remains inaccessible for a long time, rankings may suffer and…

How do web crawlers work, why do they matter for SEO, and how can they help?

Web crawlers are automated programs that visit and process web pages, gathering information through links, code, and content to help search engines discover…

ProcrustesPRProcrustesPRProcrustesPRWhat risks remain with large numbers of links, even from topically relevant websites?
How should you notify search engines when your website is ready to go public?

When your website is ready to go public, first check that search engines can access the pages you intend to publish, then create and submit a sitemap.…

How should website architecture and technical settings be optimized together?

Website architecture and technical settings should be reviewed together: first check how page content, URLs, and links are organized, then address URL…

What matters most in SEO-friendly website architecture? Make content readable and findable before focusing on rankings

Optimizing website architecture for SEO should start with confirming that search engines can read important content and follow links to find pages. Then…

Does your website need robots.txt, and how can you decide?

A website does not necessarily need robots.txt; the decision should depend on your crawl management needs, not simply on whether you are doing SEO. If you…

How should you plan and check redirects when redesigning a website or changing its URLs?

When redesigning a website or changing its URLs, first identify what is changing—the domain, paths, or HTTP/HTTPS—and map old URLs to their corresponding…

What should you do if rankings drop after a website redesign and URL changes?

If rankings drop after a simultaneous redesign and URL change, start with the change log. Check old-to-new URL mappings, redirect destinations, content, and…

How can website search boxes be abused, and how can sites prevent spam and endless crawling?

If a website’s search feature displays arbitrary query text on public, indexable results pages, it can be exploited to generate large volumes of spam. Some…

ProcrustesPRProcrustesPRProcrustesPRUsing Web 2.0 Sites for SEO: Content Exposure and the Effects and Risks of Link Manipulation
Will a Website or CMS Migration Affect SEO? Risks to Watch Even When URLs Stay the Same

A website or CMS (content management system) migration may cause short-term fluctuations in rankings and organic search traffic, but traffic loss is not…

What should a website audit cover for technical SEO and user experience?

A website audit should cover crawling and indexing, page markup and internal links, speed and mobile usability, accessibility, the inquiry process, and data…

How Can a Favicon Become Eligible to Appear in Google Search Results?

To be eligible to appear in Google's organic search results, a favicon must meet image specifications and content guidelines, and its URL must be specified…

How should websites adapt to mobile-first indexing? Key checks and adjustments

Adapting to mobile-first indexing takes more than shrinking the layout. First, check that the mobile version has the same primary content as the desktop…

Do you need to resubmit your sitemap every time you add a URL?

Generally, no. After adding a URL, update the sitemap’s contents; you do not need to resubmit it every time. This assumes that the sitemap has already been…

How can you make mobile website content crawlable and meet mobile indexing requirements?

To make mobile content crawlable and indexable by Google, the key is to ensure that Google can access and render all primary content, without being blocked…