What is a custom connector?

✅ What you’ll learn in this module:

• What a custom connector is;
• When and why you might need one;
• What Weld handles vs. what you’re responsible for;
• The two key endpoints: /schema and /.

What is a custom connector?

A custom connector in Weld is a lightweight, self-hosted integration that lets you bring data from any public API into your data warehouse, even if there isn’t a native Weld connector for it yet.

You write the logic. Weld handles the scheduling, loading, lineage, and monitoring.

What Weld handles for you:

Once your custom connector is set up and running, Weld takes care of:

• Scheduling syncs
• Schema generation and inference
• Loading data into your warehouse
• Error handling and retries
• Lineage tracking
• Alerting if something goes wrong

What you provide:

You build and host a simple HTTP API that follows Weld’s format, and with just two endpoints:

/schema

Returns the table names, primary keys, and (optionally) field types.

Example:

1{
2  "schema": {
3    "customers": {
4      "primary_key": "id",
5      "fields": [
6        {
7          "name": "id",
8          "type": "string"
9        },
10        {
11          "name": "email",
12          "type": "string"
13        }
14      ]
15    }
16  }
17}

/ (root)

Returns the data, sync state, and whether more pages are available.

Example:

1{
2  "insert": [
3    {
4      "id": "123",
5      "email": "test@example.com"
6    }
7  ],
8  "state": {
9    "page": 2
10  },
11  "hasMore": true
12}

🔒 Authentication

Weld includes a Bearer token in all requests to your connector. Your code should check for this token to validate that requests are coming from Weld.

When should you use a custom connector?

Use a custom connector when:

• You need data from a tool Weld doesn’t yet support
• You’re working with internal systems or niche APIs
• You want full control over how data is pulled, structured, and cleaned