Google explains why it crawls but barely indexes a major product site

Google's John Mueller recently addressed a question about why Google heavily crawls a large product aggregator website but indexes very few of its pages. The site in question, ponito.com, has been crawled 1.17 million times in recent weeks, yet only four of its 180,000 pages are indexed, including not even its homepage.

Mueller identified two main issues. First, he noted a technical problem with redirects, stating that many of the indexed URLs from the site redirect to the homepage, indicating unresolved technical issues. Second, he highlighted the challenges of maintaining a large product aggregator, which often involves programmatic, auto-generated content. He emphasized that convincing search engines and users of the value of such content is difficult.

Mueller also pointed out that Google is generally not a fan of programmatic SEO, suggesting that sites relying heavily on this approach may struggle to gain favorable search engine rankings. The discussion took place on Bluesky, where Mueller shared his insights in response to the user's query.