Google Crawling Times: New Pages Take Around 20 Hours

At a recent Google event in Barcelona, an analyst named Gary Illyes presented data on how long it takes for new web pages to be discovered by Google's search engine. According to Illyes' internal timing data, it typically takes around 20 hours for a new URL to be crawled and indexed, although the slowest cases can take weeks or even never finish.

The data showed that crawling and indexing are linked processes, with delays stacking up if one process is slower than expected. For example, a page cannot be indexed until it has been crawled, so delays in crawling can impact indexing times as well.

Illyes also highlighted the importance of quality in determining how long it takes for pages to be processed by Google's search engine. Pages that are deemed not worth the effort may never finish being crawled or indexed.