Google Drive

Google Drive is a cloud-based storage and synchronization service provided by Google. It allows users to store files online, share them with others, and access them from any computer, smartphone, or tablet. Users can also collaborate on documents, spreadsheets, and presentations in real-time using Google's suite of productivity tools, which are integrated with Drive. This service is known for its ease of use, accessibility, and generous free storage allowance.

Feature nameSupported
Column HashingTrueField level
BlockingTrueField level
IncrementalTrue
Custom dataTrue
HistoryFalse
ReSyncTrueTable level
TemplatesFalse

The Google Drive connector combines all files within a single folder into one table. Therefore, it's best to keep only one kind of data in each folder. You can only sync one folder per data source - for multiple folders you will need to create new data sources.

🔧 Setup Guide - ELT

Step 1 - Begin Configuration

  1. In the connector setup form, enter the destination schema name of your choice.
  2. Click Authorize to allow Weld to access your Google account using OAuth. You will be redirected to your Google account.

Step 2 - Grant access to Weld

  1. Click Grant Access to allow Weld to connect to your Google app. When authorization is complete, you will be redirected back to the Weld connections overview. The configuration is complete.

Step 3 - Connection Settings

  1. Select the file type, sync mode and folder that you would like to sync data from.

Step 4 - Data To Sync

  1. Select the pre-built reports you wish to include in the sync.
    You can view the schema, remove columns or hash sensitive information.

Step 5 - Configure Sync

  1. Select how often you would like the data to sync.
  2. Provide a unique destination table name.

Weld will take over from here and commence syncing data from your Google Drive folder.

🔧 Setup Guide - Reverse ETL

Google Drive reverse ETL allows you to sync data on a schedule from your Weld App automatically into a file in a designated Google Drive folder.

Every sync creates a brand new file in the selected folder containing the full result of your model. There is no concept of inserting, updating, or upserting individual rows — each run is a fresh Mirror of your data.

Pre-requisites:

  1. You first must set up a connection to Google Drive via OAuth.
  2. You must publish your SQL model containing the data that you want to sync to Google Drive.

Step 1 - Begin Configuration

  1. Click on the Reverse ETL button in the side bar and click + New on the top right.
  2. Select the Google Drive connector from the drop down menu or else click + Add new connection to authorize the connector.

Step 2 - Destination Settings

  1. Folder: Select the Google Drive folder the file will be uploaded to.
  2. File type: Choose the format of the generated file — CSV, Parquet, NDJSON, or JSON array.
  3. Compression: Optionally compress the file with Gzip. Not available for Parquet, since it is already internally compressed.
  4. Custom file name prefix (optional): Override the default file name prefix. If not provided, files are named weld_{id}_{timestamp}.{extension}. With a custom prefix, files are named {custom_prefix}_{timestamp}.{extension}.

Step 3 - Model

  1. Choose the SQL model that contains the data you would like to send to Google Drive.
  2. Map the model columns to the file columns. Every column will be included as is, the column name and type will be preserved to the extent the file format allows.

Step 4 - Sync Operation

Google Drive only supports Mirror: every sync creates a brand new file containing all of the data from your model.

Step 5 - Schedule

Choose how often you would like the sync to run. You can choose to have the sync running on an independent schedule or as part of an orchestration workflow.

Was this page helpful?