History Tables
History tables are one of the Slowly Changing Dimension (SCD) methods used in data warehousing. They allow you to keep track of changes to your data over time and are useful for auditing, debugging, and understanding how your data has evolved.
In Weld, history tables can be enabled for selected tables within the connector settings. When enabled, a history table is created with the same schema as the original table, but it includes recorded changes over time. In other words, each time a record in the original table is updated, a new entry is added to the history table with the previous state of the record.
Example: Customers Table with History Enabled
Current Table: customers
| id | name | phone_number | updated_at |
|---|---|---|---|
| 1 | Anna | 123-456-8888 | 2024-02-01 10:15:00 |
This table always contains the latest version of each record.
History Table: customers_history
| id | name | phone_number | updated_at |
|---|---|---|---|
| 1 | Anna | 123-456-7890 | 2024-01-01 09:00:00 |
| 1 | Anna | 123-456-8888 | 2024-02-01 10:15:00 |
This table stores previous versions of the record whenever changes occur.
Key Points
- The history table has the same schema as the original table.
- The main table always reflects the current state.
- The history table allows you to see what the data looked like at any point in time.
How to Enable History Tables in Weld
- Go to the connector's configuration settings in Weld.
- Check the box saying "Include history tables".
- Search for the tables you want to enable history for and select them.
- Save the configuration and update the sync.
Connectors Supporting History Tables in Weld
- Shopify
- Shopify Plus