faucetstream
Get started

Connectors / Databases

DuckDB

source

DuckDB query source (file or in-memory), rows streamed as JSON

Install

cargo install faucet-cli --features source-duckdb
faucet init --source duckdb
# Cargo.toml — use it as a library
cargo add faucet-source-duckdb

Starter config

source:
  type: duckdb
  config:
    database: ""    # REQUIRED — Path to the DuckDB database file, or `:memory:` for an in-memory database.
    query: ""    # REQUIRED — SQL query to execute.
    # read_only: false    # Open the database read-only.
    # batch_size: 1000    # Records per emitted `StreamPage`.
    # sandbox: false    # Confine the query to the opened database: no file or network access (`read_text`, `read_csv`, `ATTACH`, `COPY`, httpfs), no extension install/autoload, and the configuration locked so the query cannot `SET` it back.

Capabilities

Replay Non-deterministic replay
Exactly-once replay No
Incremental No
Dataset discovery No
Reports lag No
Compression No

Config fields

FieldTypeDefaultDescription
database
required		 string — Path to the DuckDB database file, or :memory: for an in-memory database. A duckdb:// / duckdb: scheme prefix is accepted and stripped.
query
required		 string — SQL query to execute.
read_only boolean false Open the database read-only. Defaults to false (read-write).
batch_size integer 1000 Records per emitted StreamPage. Rows are drained from the DuckDB result and yielded whenever the buffer reaches this size. Defaults to DEFAULT_BATCH_SIZE.
sandbox boolean — Confine the query to the opened database: no file or network access (read_text, read_csv, ATTACH, COPY, httpfs), no extension install/autoload, and the configuration locked so the query cannot SET it back. Defaults to false. faucet serve forces it on for every config it did not load from its own triggers file.
Get started

Your first pipeline runs in five minutes.

Install the CLI, scaffold a config, and move real data — nothing external to stand up.

curl -LsSf https://github.com/faucet-hq/faucet-stream/releases/latest/download/faucet-cli-installer.sh | sh
brew install faucet-hq/faucet-stream/faucet-cli
Read the guide → Star on GitHub