What does unavailable_after do, and what are its recrawling limitations?

ProcrustesPR

unavailable_after tells Google to stop showing a page in search results after a specified date and time. If the expiration date is later changed, Google must recrawl the page to see the new date. According to Google documentation quoted by Search Engine Journal on July 29, 2026, Googlebot will also significantly reduce how often it crawls the URL after expiration. But if a page is repeatedly renewed, what happens to its search visibility when the old date Google last saw expires before Google discovers the new one? The article did not provide a clear answer.

ProcrustesPRProcrustesPRHow Long Does It Take to Evaluate SEO Results?

A deadline for search visibility, not a guarantee after renewal

The documentation explains how to have a page removed from search results after a specified deadline. It does not guarantee continued visibility each time the page is renewed and its expiration date extended. These are two different needs:

  • A fixed deadline: Having a page stop appearing in search results after a specified date and time matches the documented purpose.
  • Repeated renewals: Keeping a page visible in search while relying on the expiration setting to adjust crawling was a scenario that remained unresolved at the time.

The person asking the question in the article intended to use this setting for renewable, short-term advertising pages and hoped to avoid 404 errors and manage crawl budget. These were the questioner’s goals, not effects established by the article.

Changing the date on the site does not mean Google has seen it

The article notes that Google detects rules in robots meta tags or X-Robots-Tag headers when it crawls a URL. Wherever the date is placed, updating it on the website does not mean Google has adopted the new deadline.

For example, suppose an ad originally expires on Friday but is renewed on Thursday and extended until the following week. If Google does not crawl it again until Saturday, it will not see the new date until then. Will the page still appear in search results between the old deadline passing on Friday and the next crawl on Saturday? The article does not answer that question. Neither continued visibility nor removal can be assumed.

Nor should Google be expected to check only the headers without downloading the page content. The article quotes Gary Illyes as saying that Google rarely uses HEAD requests and will still download the content when fetching a page again.

How does Illyes’s preliminary assessment differ from the documentation?

Illyes’s intuition was that extending the date should be fine, but Google would need to crawl the page again to see the change. He also said the rule only affects index selection, acting as a signal that the page “can now be removed.” However, the article specifically notes that the statement about “only affecting index selection” does not appear in the documentation, which separately describes a reduction in crawl frequency after expiration.

His preliminary assessment therefore should not be treated as a complete set of rules for renewals. As of the article’s publication, the documentation still did not cover changing the expiration date with each renewal. Website owners also cannot control Google’s crawl frequency, so updating the date does not guarantee that Google will read and apply the new deadline in time.

ProcrustesPRProcrustesPRWhere should I start if I want to do SEO myself?

unavailable_after: perspectives from across the web

Can renewable advertising pages rely solely on unavailable_after to guarantee continued visibility in search results?

It should not be treated as a guarantee of continued visibility after renewal. When a website extends the date, Google cannot see the change until its next crawl. If the old date Google last saw expires before the page is recrawled, the July 2026 article does not answer what happens in the meantime. Illyes's intuition was that extending the date should be fine, but that was not confirmation of how this scenario would be handled.

Does putting the date in a header let Google confirm a renewal without downloading the page?

You should not expect Google to confirm a renewal by checking only the headers without downloading the page. Whether the date appears in a robots meta tag or an X-Robots-Tag header, Google detects the rule when it crawls the URL. Illyes also said that Google rarely uses HEAD requests and will still download the content when fetching the page again. This describes the crawling behavior quoted in the article; it does not mean every request is handled the same way.

Is an expiration setting suitable for precisely controlling Google's crawl frequency?

It should not be treated as a precise crawl-scheduling tool. The documentation quoted in the 2026 article says Googlebot will significantly reduce how often it crawls the URL after expiration, but it provides no crawl schedule that website owners can control. The documentation at the time also did not cover repeated renewals combined with using the tag to manage crawling, so timely discovery of a new date cannot be assumed.

ProcrustesPRShould I rewrite or update an old article that does not rank?

References

  1. Google’s Illyes Unsure On Shifting unavailable_after Dateswww.searchenginejournal.com · 2026-07-29T07:00:06+00:00 — Discusses: Can renewable advertising pages rely solely on unavailable_after to guarantee continued visibility in search results?; Does putting the date in a header let Google confirm a renewal without downloading the page?

Related articles

How Long Does It Take to Evaluate SEO Results?Where should I start if I want to do SEO myself?Should I rewrite or update an old article that does not rank?What should I check when organic traffic suddenly drops?Why can I not find my published article on Google?Will Reposting Social Content on My Website Cause a Duplicate-Content Penalty?Eight SEO essentials for beginners: Getting your pages indexedWhat Is SEO? Understanding Optimization Priorities Through How Search Engines Work
ProcrustesPR
ProcrustesPR