Databricks

Install

cargo install faucet-cli --features sink-databricks
faucet init --sink databricks

# Cargo.toml — use it as a library
cargo add faucet-sink-databricks

Starter config

sink:
  type: databricks
  config:
    workspace_url: ""    # REQUIRED — Workspace base URL, e.g.
    warehouse_id: ""    # REQUIRED — Target SQL warehouse id.
    auth:
      type: pat    # one of: pat, token
      config:
        token: ""    # REQUIRED — The token string (use `${env:…}` / `${vault:…}` to inject).
      # --- Alternative variants — replace the block above with one of these ---
      # type: token
      # config:
        # token: ""    # REQUIRED
    # catalog: ""    # Unity Catalog catalog.
    schema: ""    # REQUIRED — Target schema.
    table: ""    # REQUIRED — Target table.
    # create_table: true    # Create the table from the first page's inferred schema when it does not exist.
    # load_method: null    # How a page is loaded into the table.
    # staging: null    # Staging location for `copy_into` / `auto`.
    # copy_threshold_bytes: 1048576    # `auto` stages a page whose estimated size is at least this many bytes.
    # batch_size: 1000    # Rows per `INSERT` / `MERGE` statement on the insert path (`0` = no row limit — statements are then bounded only by `max_…
    # max_statement_bytes: 8388608    # Upper bound on one statement's SQL text on the insert path; a page is split into several statements above it.
    # wait_timeout_secs: 50    # Server-side wait before a statement goes async (`0` or `5`–`50` s).
    # poll_interval_ms: 1000    # Poll cadence while a statement is queued or running (a serverless warehouse may take a minute to start).
    # statement_timeout_secs: 3600    # Client deadline for one statement; the statement is cancelled when it is exceeded.
    # max_retries: 5    # Retries on `429` / `503` and on Delta concurrent-write conflicts of idempotent statements.
    # retry_backoff_ms: 1000    # Base of the exponential retry backoff.
    # write_mode: null    # Write semantics for a sink.
    # key: []    # Key columns.
    # delete_marker: null    # Optional.
    # rollback: null    # Runtime rollback settings (#706): the run id to journal under, the run-id column, and whether to journal before-images / keep a replaced table.

Capabilities

Config fields