Google Explains Error Codes And Image Support

Google recently reminded website owners to use a specific error code when taking their sites down for maintenance, according to Search Engine Journal. This error code helps preserve already indexed URLs in search results. The reminder was issued after PHP released emergency fixes for critical flaws.

The error code in question is the 503 status code, which should be used instead of general error pages. Google advises limiting the use of this code to no more than 1-2 days to avoid dropping URLs from its index. The company's guide to pausing a site also notes that triggering a 503 in response to robots.txt requests blocks all crawling.

In a separate development, Google's Gary Illyes stated that standard robots.txt rules only control what compliant crawlers can crawl, not how systems use content after it's crawled. Illyes described robots.txt as an access preference protocol, emphasizing that it does not dictate how content is used after crawling.

Illyes also mentioned that a draft proposal would add a Content-Usage rule to robots.txt, allowing website owners to state their usage preferences. However, this proposal is still in the working group stage and has not been approved. Meanwhile, website owners can use the Google-Extended token to manage whether content crawled by Google can be used for training future models.

In other news, a research pilot by Cloudflare and OpenAI discovered hundreds of millions of new web pages, which can now be used to improve ChatGPT's search capabilities. The pilot used signals from a participating group of Cloudflare's network to identify new pages and update existing ones.

Google's John Mueller also clarified that raw and encoded brackets in URLs create different URLs, which can affect how Google crawls and indexes websites. Mueller recommended using a consistent URL format across all pages to avoid duplicates and ensure proper indexing.

Additionally, Mueller noted that JPEG XL images are not currently supported in Google Images or Discover previews. However, website owners can use JPEG XL as a picture source with a supported format, such as JPEG or WebP, in the img element to ensure compatibility.