faucetstream
Get started

Connectors / Warehouses

ClickHouse

source

ClickHouse query source (HTTP interface, JSONEachRow streaming)

Install

cargo install faucet-cli --features source-clickhouse
faucet init --source clickhouse
# Cargo.toml — use it as a library
cargo add faucet-source-clickhouse

Starter config

source:
  type: clickhouse
  config:
    # url: ""    # Full base URL of the ClickHouse HTTP interface, e.g.
    # host: ""    # Hostname of the ClickHouse server.
    # http_port: 0    # HTTP-interface port used with `host`.
    # tls: false    # Use `https://` instead of `http://` when building the base URL from `host`.
    # database: "default"    # Target database.
    # user: ""    # ClickHouse user.
    # password: ""    # ClickHouse password.
    query: ""    # REQUIRED — SQL `SELECT` query to run.
    # batch_size: 1000    # Records per emitted `StreamPage`.
    # replication: { type: full }    # one of: full, incremental
      # --- Alternative variants — replace the block above with one of these ---
      # type: incremental
      # column: ""    # REQUIRED
      # initial_value: null    # REQUIRED
    # state_key: ""    # Explicit state-store key for the bookmark.

Capabilities

Replay Non-deterministic replay
Exactly-once replay No
Incremental Yes
Dataset discovery No
Reports lag No
Compression No

Config fields

FieldTypeDefaultDescription
query
required		 string — SQL SELECT query to run. The output format is set via the request settings (JSONCompactEachRowWithNamesAndTypes, decoded by column type), so **do not** append a FORMAT clause. Use the literal @bookmark token to push the incremental cursor down into the WHERE clause; use {key} tokens to inject parent-context values in a matrix child.
url string — Full base URL of the ClickHouse HTTP interface, e.g. "http://localhost:8123". Mutually exclusive with host. A trailing slash is trimmed.
host string — Hostname of the ClickHouse server. Mutually exclusive with url; combined with http_port and tls to build the base URL.
http_port integer — HTTP-interface port used with host. Defaults to DEFAULT_HTTP_PORT (8123).
tls boolean false Use https:// instead of http:// when building the base URL from host. Ignored when url is set. Defaults to false.
database string default Target database. Defaults to DEFAULT_DATABASE ("default").
user string — ClickHouse user. When set, sent as the X-ClickHouse-User header.
password string — ClickHouse password. When set, sent as the X-ClickHouse-Key header.
batch_size integer 1000 Records per emitted StreamPage. 0 emits the whole result set as a single page. Defaults to DEFAULT_BATCH_SIZE.
replication object type: full Replication mode. Defaults to ClickHouseReplication::Full.
state_key string — Explicit state-store key for the bookmark. When unset, a key is derived from the connection host and a query fingerprint.
Get started

Your first pipeline runs in five minutes.

Install the CLI, scaffold a config, and move real data — nothing external to stand up.

curl -LsSf https://github.com/faucet-hq/faucet-stream/releases/latest/download/faucet-cli-installer.sh | sh
brew install faucet-hq/faucet-stream/faucet-cli
Read the guide → Star on GitHub