Google Blocks Scrapers with goto URL Parameter Rollout
Google is rolling out a new feature to prevent scraping of its search results by third-party tools and AI companies. The company has begun testing the 'google.com/goto' URL parameter, which acts as a passthrough URL within search engine results pages (SERPs). This means that when users click on a link in Google Search results, they will be redirected to the website through the 'goto' parameter instead of being taken directly to the URL.
Derek Perkins from Nozzle reported that he sees this change at nearly a 100% rollout across several residential IP providers. He shared a chart showing the spike in how much Google is using this feature, with a significant increase over the last ~4 months of testing. Perkins also mentioned that Google has been pushing this change live and that it seems to be working as an anti-bot measure.
According to Perkins, Google's goal with this new feature is to make it difficult for providers to decode the 'goto' links, which would require them to follow redirect links. This could potentially hinder third-party tools and AI companies from scraping search engine results. However, Perkins noted that while this works in small-scale tests, it may become tough as each SERP will have hundreds of links to decode.