Webhooks
Weld doesn't pull data from a webhook; your system pushes data to it. The Webhooks connector gives you a unique endpoint URL for every table you configure. Send a POST request with a JSON payload to that URL, and each request becomes a row in the corresponding table - making it easy to stream events from any system into your data warehouse in real time.
The payloads you send will be securely stored until the next sync, after which they will be permanently deleted. This also means that full syncs are not possible, and Weld will only sync the data that was sent to your endpoint since the last sync. If you need to re-sync data, you will need to re-send the entire dataset to your endpoint URLs before enabling the full sync.
Features
| Feature name | Supported | |
|---|---|---|
| Column Hashing | True | Column level |
| Blocking | True | Column level |
| Incremental | True | |
| Custom data | False | |
| History | False | |
| ReSync | False | |
| Templates | False |
Incremental syncs are supported, but each row gets a unique ID generated by Weld based on the payload. This means that any changes to the same row will be inserted as a new row.
Setup Guide - ELT
Step 1 - Add the connector
-
In the connector setup form, enter the connection name of your choice. This name will be used to identify the connection in future syncs.
-
Choose an Authentication Method for incoming requests:
-
None — no authentication required.
-
Basic Auth — requires an
Authorizationheader, with the following value:Basic <base64-encoded-credentials>.- Example: username
myuserand passwordmypasswordwould be sent asAuthorization: Basic bXl1c2VyOm15cGFzc3dvcmQ=wherebXl1c2VyOm15cGFzc3dvcmQ=is the base64-encoded string ofmyuser:mypassword.
- Example: username
-
Token — requires a static token sent in a header you define.
-
HMAC SHA-256 — requires requests to be signed with a shared secret, sent in a header you define.
- In order to get the correct signature, use a canonical JSON representation (ordered keys, non-significant white spaces removed).
- Example: if the request body is
{ "id": "12345", "event": "order.created"}and the shared secret ismysecret, the signature is generated as follows:
Python:
import hmac import hashlib import json_stable_stringify_python as json_stable body = { "id": "12345", "event": "order.created"} body_bytes = json_stable.stringify(body).encode("utf-8") secret = b'mysecret' signature = hmac.new(secret, body_bytes, hashlib.sha256).hexdigest() print(signature) # 2b7312c96e584db1fbe1abe54b3ea4dba342d841356592d2fa652de59e7b09caJavaScript:
const crypto = require("crypto"); const stringify = require("json-stable-stringify"); const body = { id: "12345", event: "order.created" }; const secret = "mysecret"; const signature = crypto .createHmac("sha256", secret) .update(stringify(body)) .digest("hex"); console.log(signature); // 2b7312c96e584db1fbe1abe54b3ea4dba342d841356592d2fa652de59e7b09caAny key order or white space differences will result in a different signature, and Weld will reject the request, so make sure to use a canonical JSON representation when generating the signature.
-
-
Depending on the method chosen, provide the associated username/password, token/header name, or HMAC secret/header name.
-
Click Connect to create the connection.
Step 2 - Define your tables
- Add one entry per event type or data source you want to keep separate — each table gets its own endpoint URL.
- Choose a Sync mode for how incoming payloads are processed:
- Unpacked — Weld parses the JSON payload and infers the schema from its fields. If the payload contains an array of records, each record becomes a separate row.
- Packed — Weld stores each payload as a single stringified JSON column, useful when the payload shape varies a lot.
Step 3 - Configure sync
- Select how often you would like the data to sync.
- Provide a unique destination schema name.
Step 4 - Send data to your endpoint
Weld generates a unique URL for each table you defined.
You will see these endpoints after the sync setup is complete, and you can also find them in the data source details page.
Configure the sending system (e.g. a SaaS product's webhook settings, or your own application) to POST JSON payloads to that URL, including the authentication credentials configured in Step 1.
curl https://elt-webhooks.weld.app/... \
--request POST \
--header 'Content-Type: application/json' \
--header 'authorization header key: authorization header value' \
--data '{
"event": "order.created",
"id": "12345"
}'
Weld will take over from here, and every request sent to your endpoint URLs will land as rows in your data warehouse.