New feature: Partitioning & Clustering for BigQuery
BigQuery is fast, but it gets even faster when your tables are organized. Weld now supports Partitioning and Clustering on materialized tables, giving you more control over performance and cost at scale.
Partitioning: Divide your data into sections
Partitioning is like splitting a huge table into folders based on a specific column, typically a date. When you run a query filtered on that column, BigQuery scans only the relevant folder, saving time and money.
In Weld, you can enable partitioning by defining a column expression. For example:
DATE(created_at)TIMESTAMP_TRUNC(_PARTITIONTIME, DAY)
This tells BigQuery how to organize and scan the table more efficiently.
Clustering: Sort data inside each section
Clustering organizes rows within each partition by the fields you query most often. Think of it as alphabetizing books inside each folder. It helps BigQuery quickly locate the exact rows it needs.
To enable clustering in Weld, just list the columns you want to use, like:
customer_id, order_date
Used together, Partitioning and Clustering give you tighter query scopes and faster results.
How to enable it in Weld
When publishing a model as a table to BigQuery:
- Toggle Advanced materialization settings
- Add your Partitioning expression
- Enter one or more Clustering columns
🎞️ Here’s how it looks:
Weld takes care of the configuration behind the scenes, you just define what matters to your use case.
Learn more
Available now
Partitioning and Clustering are now available for all BigQuery users in Weld. Ready to make your queries faster and cheaper? Try it out today!
