Glossary

Account

Your designated Weld account is where you can set your data connectors and use your data.

Active rows

Active Row refers to any data row that is updated or inserted during the data syncing process between data sources using our platform. Active Rows typically represent 10-20% of the total rows synced. You can track your usage by navigating to the Settings page and selecting Billing in the Weld app.

There is a maximum number of Active Rows that may be synced under each subscription plan, described in our Fair Usage Policy.

The first 14 days of setting up any connector are not included in the count of active rows.

Alert

Alerts appear in your Weld account to give you important information about any issues. Alerts will appear when a sync has failed, if you are required to re-authenticate your connector, if a SQL table materialization has failed, or if an orchestration workflow error occurs.

Ed Logo AI Assistant (Ed)

Our AI SQL assistant, Ed Logo Ed, will support you in writing SQL, and can help you with syntax, suggest queries, and provide explanations for SQL errors. You can also include specific tables as context for your questions to Ed, so its answers are tailored to your problem.

Ed Logo Ed is available in the Weld platform by clicking on the icon in the bottom right corner while in your data Transform SQL editor or on the left-side menu in your Workspace.

Bulk Upload

We sync records in bulk instead of individually syncing them, which significantly speeds up sync times.

Column hashing

Column hashing allows for the anonymization of sensitive data, such as personable identifiable information (PII) while preserving the value for analytical purposes. We support column hashing for all of our integrations.

Common Table Expression (CTE)

A Common Table Expression (CTE) is a tool for keeping SQL queries simple. It creates a result set that exists only temporarily in a larger query. Read full article →

Connector

A Weld connector is a Data Pipeline that moves data from your source to your Destination. For example, you might have a HubSpot connector that moves your data from HubSpot to your Google BigQuery destination. Sometimes you might have multiple connectors of the same type. For example, you might have multiple Google Sheets connectors, each moving data from a different Google Sheet. Explore all our connectors here →

Custom Connector

Custom connectors are built and maintained by the user and can be used to connect to any data source that has an API. This allows you to connect to any data source that Weld does not currently support, increasing the flexibility of your data pipelines.

Custom Data

As part of Weld complete schema replication, we also support custom data. We replicate this whenever it is exposed and accessible in the data source. No action is needed to get this custom data. Custom data includes everything from metadata fields and tags, to entire tables that might be specific to your organization.

Customer Acquisition Cost

Customer Acquisition Cost is a metric that determines the amount of money your company spends to acquire a customer. Read full article →

Data Blocking

Data blocking is the complete exclusion of certain data from a sync. In some cases there might be restrictions on certain database columns, or excluding a column is preferable from a compliance perspective. We support column level data blocking for all of our integrations.

Data Pipeline

A data pipeline is a combination of data processes that move data from a source system to a target destination, usually through ELT or reverse-ETL. Read full article →

Data Source

A Data Source is the origin of the data that you want to replicate to a Destination, typically a data warehouse. We also support the creation of your own Custom Connector to allow for even more pipelines. Read full article →

Data Types

The type of data that can be stored and processed by a database system. Each data type has specific characteristics and operations associated with it.

  • Integer: Represents whole numbers (positive, negative, or zero) without any fractional part.
  • Floating-point or Double: Represents decimal numbers, allowing for fractional parts.
  • Character or Char: Represents a single character, such as a letter, digit, or special symbol.
  • String: Represents a sequence of characters or text.
  • Boolean: Represents true or false values, used for logical operations.
  • Date and Time: Represents dates, times, or a combination of both.

Destination

Weld connectors replicate your source data to a destination system. A Weld destination is a data warehouse. We currently support several data warehouses, such as Google BigQuery, Snowflake, Azure Synapse, Amazon RedShift, IBM Db2, Firebolt, MySQL, PostgreSQL, Microsoft SQL Server... Read full article →

Drafts

SQL Models that persist locally but are not stored in your data warehouse. These will be moved to the archive after 7 days if not used.

Editor / Transform

A SQL editor / Transform is a software tool that allows you to write, edit, and execute SQL code. SQL editors are widely used by data analysts, developers, and database administrators to manage databases, query data, and perform other tasks related to database management.

ELT

ELT stands for Extract, Load, and Transform. It's a process by which data is replicated from a Data Source to a Destination (often a data warehouse). Read full article →

Sync (Reverse ETL)

Involves moving data from a Destination back to operational systems or third-party applications for various purposes, such as data analysis, visualization, or activation. Read full article →

Full sync

Overwrites an entire table with the current version of that table from the source.

History

Weld uses the type 2 slowly changing dimension format to track table history in the Destination for certain tables. This information is useful to analyze changes over time.

Incremental Sync (Merge / Append)

Incremental syncs update only new or modified data. After the initial sync, Weld connectors sync most tables using incremental updates. We use a variety of mechanisms to capture the changes in the source data, depending on how the source provides change data. During incremental syncs, Weld maintains an internal set of pointers, which let us track the exact point where our last successful sync left off. Incremental syncs are efficient because they update only the changed data, instead of re-importing whole tables.

Incremental syncs can be merge syncs or append syncs:

  • Merge requires a primary key and a cursor. On every sync, it adds any new rows of data, while also updating the changed rows.
  • Append requires a cursor. On every sync, it will still add any new data after that last cursor's timestamp, but will not update any changed rows.

Initial sync

This is the first sync that runs when you set up your Connector. It syncs all historical data. It is common for the initial sync to take a long period of time to complete.

Integration

Integrations are the various methods and Connectors that allow users to connect different data sources to the Weld platform, enabling the creation of managed data pipelines without needing extensive coding skills. We offer many different types of integrations, for example in the Database, Application, and File categories. Explore all our connectors and destinations here →

Lineage

Data lineage refers to the ability to trace and understand the origin, movement, transformations, and relationships of data as it flows through various stages of its lifecycle. It involves tracking the entire journey of data, from its source systems to its destinations, including any transformations or modifications it undergoes along the way. Read full article →

Manual Data Import

Folders and tables can be manually imported from your destination using the manual data import. These tables can then be accessed and used with other data sources within Weld.

Normalize

Weld normalizes data by organizing it into tables and columns in a way that reduces data redundancy, storing it logically. Normalization divides larger tables into sub tables and links them using their specific relationships.

OAuth

OAuth is an open, industry-standard protocol for access delegation, which allows users to grant access to their different SaaS tools and databases without giving them the passwords.

Orchestration

Orchestration refers to the process of managing and coordinating the different Gather Syncs, SQL models, and Activate Syncs used within a database system based on synchronizing the flow by how they depend on each other. For example, the data connector will run first, then the SQL models that depend on the data source.

Published

SQL Models that have been saved as either views or tables in your Destination.

Re-sync

Weld supports a table level full re-sync, meaning it is possible to overwrite the data in a specific table. Occasionally this might be useful to solve integrity issues.

Read Replica

A read replica is a read-only copy of a primary database, used to distribute read operations and improve performance by reducing the load on the primary database. It is a common technique used to scale databases and improve query performance in high-traffic applications.

Schedule

The frequency set to determine how often an event occurs. Schedules can be set for when a sync runs or a table is materialized.

Schema

A table schema refers to the structure or blueprint of a specific table within a database. It defines the organization of data and the attributes (fields) that make up each row (record) in the table. It contains the table name, fields and Data types.

Schema Change

Schema changes are either due to APIs being updated or due to changes in the source data (such as column names or data types). A Full sync of the table is required to reflect these changes.

SQL

SQL (Structured Query Language) is a standardized programming language used for managing and manipulating relational databases. It provides a set of commands that enable users to interact with databases to create, retrieve, update, and delete data. SQL is widely used in various applications and industries, ranging from simple web applications to complex enterprise systems. Read full article →

Sync Status

Table Syncs can be in one of the following status:

  • Running: Currently syncing data.
  • Scheduled: This table is scheduled to be run shortly.
  • Completed: Sync has completed successfully.
  • Delayed: Issue occurred during the sync and it will automatically re-sync again later.
  • Failed: Recent sync failed and you will receive an alert with an error message informing you if there is an action needed on your side.

Table

Materialized tables are physical tablse, with data stored persistently on disk. They provide better query performance than views and allow direct data modifications.

Template

Templates are our pre-made SQL models, using the Weld Connectors to return common metrics. See our full list of available templates here →

View

A view is a virtual table that does not store data but instead retrieves it dynamically from underlying materialized tables. This simplifies query complexity and offer a dynamic representation of data without storing it separately.

Weld Managed Field

The _weld_synced field is a timestamp field that plays a crucial role in recording the synchronization time for each row of data within the Weld platform. It is automatically managed by Weld during the synchronization process and captures the exact moment when data is synchronized, providing valuable information for tracking data freshness and monitoring synchronization activities. For example, this is how the _weld_synced field might look in a JSON representation of a row of data:

{
  "id": 1,
  "name": "Example Data",
  "_weld_synced": "2024-01-18T12:30:00.000Z"
}

In this example, the _weld_synced field indicates that the row with ID 1 was last synchronized on January 18, 2024, at 12:30 PM (UTC).

The _weld_synced field was introduced on January 16, 2024. Any rows synchronized before this date will have a NULL value for this column instead of the actual synchronization time.

Was this page helpful?