Developing
Google says Retry-After support is not new; the change clarifies documentation, not crawler behaviour.
In 60 seconds
- Google's changelog says it "Restructured the emergency crawl rate reduction section".
- The new text says a 503 or 429 response "can also include a Retry-After HTTP header" telling Google's crawlers when to retry.
- The wait can be expressed as seconds or a specific UTC date and time, following RFC 9110.
- Retry-After was already documented in "Temporarily pause or disable a website".
Impact Layer
- What changed
- The emergency crawl-reduction instructions now mention Retry-After explicitly.
- Why it matters
- If a server is overloaded, a clear retry time is a cleaner request to slow crawling than a bare error.
- Who is affected
- Developers, site reliability engineers and technical SEOs.
- Martech stack area
- SEO & Search, CMS
- Stack impact
- If you have to throttle Googlebot in an emergency, return 503 or 429 with a Retry-After header.
- Business impact
- Low Reported by Search Engine Roundtable from Google's changelog. The example code was a screenshot, and no maximum delay or crawler difference was stated.
- Related vendors
- What to watch
- Googlebot's actual response to Retry-After values in server logs.
Barry Schwartz reported on Search Engine Roundtable on October 6, 2026, citing Google’s changelog, that the “Reduce the Google crawl rate” documentation was restructured.
What changed
- Google’s changelog says it “Restructured the emergency crawl rate reduction section”.
- The section now says that when returning a 503 or 429 status code you “can also include a Retry-After HTTP header”.
- The header can express the wait as a number of seconds or a specific UTC date and time, following RFC 9110.
- Google also says Retry-After support is not new; it was already documented in “Temporarily pause or disable a website”.
Not stated
How much or how quickly crawling will slow, whether Google treats Retry-After differently from a bare 503 or 429, the maximum allowed delay, and the other organisational changes.
Our read
Housekeeping, but a useful reference: throttle with 503 or 429 and a Retry-After value, not with errors that look like content is gone.
Sources
- Search Engine Roundtable: Google Search Crawl Rate Doc Adds Retry-After HTTP & More (opens in a new tab)
Source for the changelog entry, wording and what is not stated.