BigQuery Gets Unstructured Data Analysis Upgrade

Google has released new features for its BigQuery data warehouse, aiming to make unstructured data analysis more accessible within a single system. The updates include Autonomous Embedding Generation and AI.SEARCH in general availability, while Hybrid Search is in public preview.

The addition of these tools addresses the challenge of querying and analyzing documents, images, audio files, and free text alongside structured records in a single system. BigQuery has long been used for structured information, but organizations typically needed separate tools to extract and search meaning from unstructured material.

Autonomous Embedding Generation allows users to define a schema column for embeddings, which are then generated asynchronously as new data arrives. This process applies to text records and images through ObjectRefs. Users can choose external embedding models or use Gemma embedding models within BigQuery.

Ai.SEARCH is a function that lets users run natural language searches over embedded data. It has been tuned for single-query execution, which is particularly relevant for online applications and agent-style search tasks. Google claims gains of up to 133 times in slot efficiency for single-query searches.