Elasticsearch to Databricks integration

Connect Elasticsearch to Databricks and automatically sync your application data. Weld extracts data from Elasticsearch, loads it into Databricks, and lets you transform the data using SQL or dbt, no custom scripts required.

4.8 on G2
Image not found!
Source
WeldWeld
DatabricksWarehouse
Weld · New sync
Image not found!
SourceElasticsearchConnected
Tables to sync

Your own tables, picked when you set up the sync.

  • Extract
  • Load
  • Keep in sync
DatabricksDestinationDatabricksReady
elasticsearch schema
  • elasticsearch.Synced
  • elasticsearch.Syncing
  • elasticsearch.Synced
  • elasticsearch.Synced

Trusted by data and AI teams

  • Flatpay
  • Holafly
  • eComplete
  • Dishoom
  • Danish Endurance
  • Female Invest
  • Son of a Tailor
  • Onomondo
  • Medik8
  • Pas Normal Studios
How it works

How to set up Elasticsearch to Databricks in Weld.
In the app, with the API or an agent.

Weld · New Elasticsearch sync
ENew connectionElasticsearchAuthorised
Connection nameelasticsearch
AuthorizeOpens Elasticsearch to grant read access
Use cases

Why sync Elasticsearch to Databricks?
Ready for analysis.

Centralizing your Elasticsearch data in Databricks lets you join it with other data sources for cross-platform analytics and deeper insights.

Build reliable dashboards and reports by querying Elasticsearch data directly in Databricks using SQL or your preferred BI tool.

  • Centralized data warehouse

    Bring Elasticsearch data into Databricks alongside other sources to create a single source of truth for your organization.

  • Automated reporting

    Replace manual exports and spreadsheet-based reporting with automated pipelines from Elasticsearch to Databricks. Connect BI tools like Looker, Metabase, or Power BI.

Data

What data can you sync?
From Elasticsearch to Databricks.

Weld syncs the objects the Elasticsearch connector exposes into Databricks, and you choose which tables and fields to include. The connector's docs list what is available.

    Weld MCP server

    Ask your agent.
    It queries Databricks for you.

    Connect Claude, Cursor, ChatGPT or your own agent to Weld over MCP. It reads the Elasticsearch context, finds the tables it needs and runs the query in Databricks.

    Claude CodeOpus 5
    >

    Which Elasticsearch tables do we have in Databricks, and which is the largest?

    Looking up what Weld has synced from Elasticsearch.

    ●weld·search_available_references
    ⎿raw.elasticsearch.*
    tables, columns and freshness
    ●weld·get_reference_schema
    ⎿columns, types and nullability
    ●weld·run_query
    ⎿select count(*) from {{raw.elasticsearch.…}}
    submitted
    ●weld·get_query_status
    ⎿DONE
    ●weld·get_query_result
    ⎿rows returned
    ✓Answered from Elasticsearch data in Databricks
    >Ask about your Elasticsearch data
    Under the hood

    Technical considerations.
    What to expect.

    Incremental syncing
    Weld uses incremental syncing where supported by the Elasticsearch API, loading only new and updated records to minimize data transfer and warehouse costs.
    Schema handling
    Weld automatically manages schema changes. If Elasticsearch adds new fields, they're detected and added to the corresponding Databricks tables.
    Data freshness
    Sync frequency depends on your Weld plan. Most connectors support schedules from every 5 minutes up to daily intervals.
    Security
    All data is encrypted in transit (TLS) and at rest. Weld is SOC 2 Type II compliant and supports IP whitelisting for database connections.
    FAQ

    Common questions.
    About Elasticsearch to Databricks.

    Still have a question?
    Talk to us

    Start syncing Elasticsearch to Databricks.

    Set up your Elasticsearch data pipeline in minutes. No credit card required. Free for 14 days.

    Image not found!
    Source
    WeldWeld
    DatabricksWarehouse