✅ What you’ll learn in this module:

  • How to structure your data response;
  • What insert, state, and hasMore mean;
  • How Weld uses sync state for pagination;
  • How to build and test a simple root endpoint.

📦 What is the / endpoint?

This is the second required endpoint for every custom connector.

Weld calls this endpoint:

  • For each table listed in /schema
  • With a POST request
  • Sending the table name and the last known sync state

Your connector responds with:

  • A list of rows to insert (insert)
  • The new state (e.g. next page, timestamp, ID)
  • A flag (hasMore) telling Weld whether to request the next page

🧱 Example data response

1{
2  "insert": [
3    {
4      "id": "123",
5      "title": "Form A",
6      "created_at": "2024-12-01T10:00:00Z"
7    },
8    {
9      "id": "124",
10      "title": "Form B",
11      "created_at": "2024-12-02T11:00:00Z"
12    }
13  ],
14  "state": {
15    "page": 2
16  },
17  "hasMore": true
18}

🛠 Add it to your code

Update main.py to handle POST requests at the root / path:

1import functions_framework
2from flask import jsonify, Request
3
4
5@functions_framework.http
6def handler(request: Request):
7    if request.path == "/schema":
8        return jsonify(get_schema()), 200
9
10    if request.method == "POST":
11        request_data = request.get_json()
12        return jsonify(get_data(request_data)), 200
13
14    return "Hello, Weld!", 200
15
16
17def get_schema():
18    return {
19        "schema": {
20            "forms": {
21                "primary_key": "id",
22                "fields": [
23                    {"name": "id", "type": "string"},
24                    {"name": "title", "type": "string"},
25                    {"name": "created_at", "type": "datetime"}
26                ]
27            }
28        }
29    }
30
31
32def get_data(request_data):
33    table = request_data.get("name")
34    state = request_data.get("state", {})
35    current_page = state.get("page", 1)
36
37    # Simulate paginated data
38    all_data = [
39        {"id": "123", "title": "Form A", "created_at": "2024-12-01T10:00:00Z"},
40        {"id": "124", "title": "Form B", "created_at": "2024-12-02T11:00:00Z"}
41    ]
42
43    # Simple one-page example
44    has_more = False
45    next_state = {}
46
47    return {
48        "insert": all_data,
49        "state": next_state,
50        "hasMore": has_more
51    }
52

🧪 Test it in Postman

  1. Restart the server (if it’s not already running):
  functions-framework --target=handler --debug
  1. In Postman:
    • Make a POST request to http://localhost:8080
    • Set the Content-Type to application/json
    • Use this request body:
1{
2  "name": "forms",
3  "state": {}
4}

✅ If successful, you’ll see your insert data in the response!

Next up

Adding pagination to your custom connector

Break your data into pages so Weld can sync large datasets efficiently.

Go to module