faucetstream
Get started

Connectors / Files & object stores

JSON Lines

sink

JSON Lines file sink

Install

cargo install faucet-cli --features sink-jsonl
faucet init --sink jsonl
# Cargo.toml — use it as a library
cargo add faucet-sink-jsonl

Starter config

sink:
  type: jsonl
  config:
    path: ""    # REQUIRED — Path to the output file.
    # append: false    # Whether to append to an existing file (default: false, truncates).
    # pretty: false    # Whether to pretty-print each JSON record (default: false, compact).
    # batch_size: 1000    # Records per upstream `StreamPage`.
    # compression: null    # User-facing compression config.
    # encryption: null    # Encryption at rest (#207): seal every record line with AES-256-GCM and write it base64-encoded (one sealed record per line, append-safe).

Capabilities

Delivery At-least-once
Write modes append
Exactly-once writes No
Schema evolution No
Scoped cleanup No
Staged bulk load No
Run rollback No
Compression Yes

Config fields

FieldTypeDefaultDescription
path
required		 string — Path to the output file.
append boolean false Whether to append to an existing file (default: false, truncates).
pretty boolean false Whether to pretty-print each JSON record (default: false, compact).
batch_size integer 1000 Records per upstream StreamPage. The JSONL sink writes records to disk one at a time through a buffered async writer, so this field has **no behavioural impact** at the sink — it is exposed purely for config parity across every sink in the workspace. Defaults to DEFAULT_BATCH_SIZE.
compression string | object auto Compression codec for the output file. Defaults to CompressionConfig::Auto — .gz / .zst suffix selects gzip / zstd, anything else writes uncompressed. Requires the crate-local compression feature.
encryption object — Encryption at rest (#207): seal every record line with AES-256-GCM and write it base64-encoded (one sealed record per line, append-safe). The natural fit for a file-backed DLQ holding sensitive failed rows; faucet dlq inspect/replay/discard decrypt transparently given the key. Mutually exclusive with compression. Requires the crate-local encryption feature.
Get started

Your first pipeline runs in five minutes.

Install the CLI, scaffold a config, and move real data — nothing external to stand up.

curl -LsSf https://github.com/faucet-hq/faucet-stream/releases/latest/download/faucet-cli-installer.sh | sh
brew install faucet-hq/faucet-stream/faucet-cli
Read the guide → Star on GitHub