faucetstream
Get started

Connectors / Databases

SQLite

sink

SQLite sink

Install

cargo install faucet-cli --features sink-sqlite
faucet init --sink sqlite
# Cargo.toml — use it as a library
cargo add faucet-sink-sqlite

Starter config

sink:
  type: sqlite
  config:
    database_url: ""    # REQUIRED — SQLite database URL (file path or `:memory:`).
    table_name: ""    # REQUIRED — Target table name.
    column_mapping: null    # REQUIRED — How to map JSON records to table columns.
    # batch_size: 1000    # Maximum number of rows per multi-row INSERT.
    # max_connections: 1    # Maximum number of connections in the pool.
    # create_table: true    # Create the target table if it does not exist, inferring the columns from the first written page (#580).
    # busy_timeout_secs: 60    # How long a write waits for another connection's write lock before failing with `SQLITE_BUSY`, in seconds.
    # write_mode: null    # Write semantics for a sink.
    # key: []    # Key columns.
    # delete_marker: null    # Optional.
    # rollback: null    # Runtime rollback settings (#706): the run id to journal under, the run-id column, and whether to journal before-images / keep a replaced table.

Capabilities

Delivery Exactly-once (atomic watermark)
Write modes append, upsert, delete, overwrite
Exactly-once writes Yes
Schema evolution Yes
Scoped cleanup Yes
Staged bulk load No
Run rollback Yes
Compression No

Config fields

FieldTypeDefaultDescription
database_url
required		 string — SQLite database URL (file path or :memory:).
table_name
required		 string — Target table name.
column_mapping
required		 object — How to map JSON records to columns.
batch_size integer 1000 Maximum number of rows per multi-row INSERT. Defaults to DEFAULT_BATCH_SIZE (1000); the recommended value for SQLite.
max_connections integer 1 Maximum number of connections in the pool. Defaults to 1.
create_table boolean true Create the target table if it does not exist, inferring the columns from the first written page (#580). Enabled by default: a first-ever sync cannot assume the destination already exists.
busy_timeout_secs integer 60 How long a write waits for another connection's write lock before failing with SQLITE_BUSY, in seconds. Defaults to 60.
write_mode object append Append (default), upsert, or delete.
key array [] Key columns. Required and non-empty for upsert/delete; ignored for append.
delete_marker object — Optional. Upsert only: rows whose field matches one of values are deletes; all others are upserts. The marker field is stripped from upsert rows before writing.
rollback object — Runtime rollback settings (#706): the run id to journal under, the run-id column, and whether to journal before-images / keep a replaced table. Injected per invocation by the CLI when a config has a top-level rollback: block — not meant to be written by hand.
Get started

Your first pipeline runs in five minutes.

Install the CLI, scaffold a config, and move real data — nothing external to stand up.

curl -LsSf https://github.com/faucet-hq/faucet-stream/releases/latest/download/faucet-cli-installer.sh | sh
brew install faucet-hq/faucet-stream/faucet-cli
Read the guide → Star on GitHub