Rust data-movement platform

Move data at Rust speed.
Govern it in flight.

A config-driven pipeline engine with 68 source & sink connectors, change-data-capture, effectively-once delivery, and governance built into the movement path — wired by a single faucet binary, or embedded in your own service. No Python runtime, nothing external to stand up.

brew install faucet-hq/faucet-stream/faucet-cli
curl -LsSf https://github.com/faucet-hq/faucet-stream/releases/latest/download/faucet-cli-installer.sh | sh
cargo install faucet-cli
docker run --rm -v "$PWD:/w" ghcr.io/faucet-hq/faucet-stream:full run /w/pipeline.yaml
helm install faucet oci://ghcr.io/faucet-hq/charts/faucet-stream

or embed the library — cargo add faucet-stream

deploy with Docker or Helm →

pipeline.yaml
Postgres source · CDC source
sql transform · DuckDB transform
BigQuery sink · MERGE sink
0 rows/s
11.8 MiB RAM bounded & streaming
68 connectors 38 sources · 30 sinks
712k rows / second 1M-row CSV → JSONL
~96× faster than Meltano exact row parity
1 binary to ship no daemon, no runtime
Why faucet-stream

One engine for the whole movement path.

Most stacks bolt movement, orchestration, and governance together from separate tools. faucet-stream does all three in a single, fast Rust runtime.

Built for throughput

Native streaming with bounded memory, connection pooling, multi-row inserts, bulk APIs, and parallel I/O. Throughput is a first-class design goal for every connector — not an afterthought.

A runtime, not just connectors

Incremental + resumable replication, change-data-capture, effectively-once delivery, dead-letter queues, automatic retries, adaptive batch sizing, secrets interpolation, cron scheduling, and built-in Prometheus metrics + tracing spans — with zero per-connector code.

Governance in the movement path

Data-quality checks, versioned contracts, PII masking applied before any sink sees a row, schema-drift policy, column-level lineage (OpenLineage), and freshness/volume SLAs — native and zero-config.

CLI, library, or HTTP API

Run faucet run pipeline.yaml, call Pipeline::new(&source, &sink).run().await? from Rust, or drive it over HTTP with faucet serve. Same engine three ways — a CLI you drop on any box, a library you compile in, or a control-plane API.

Bring your Singer taps

The singer source runs any existing Singer / Meltano tap unchanged, so you can adopt incrementally — start with the taps you already have and move to native connectors where throughput matters.

Compile only what you need

Almost every capability is a Cargo feature — connectors, transforms, state backends, governance, even the HTTP server — so your binary contains exactly what you switch on, nothing else. Ship a lean REST-to-JSONL build, or turn everything on with --features full.

Benchmarks

The same million rows — 1.4 seconds, not two minutes.

A reproducible 1M-row CSV → JSONL move: faucet-stream finishes in ~1.4s; Meltano takes ~2¼ minutes for the identical, row-for-row output. That's the best case (it maximally exposes Python's per-row overhead) — sink-bound moves like Postgres → Postgres narrow the gap, and every caveat is in the open.

See the full benchmarks →
Throughput higher is better
faucet-stream
712,000 rows/s
Meltano (Singer)
7,400 rows/s
Peak memory lower is better
faucet-stream
11.8 MiB
Meltano (Singer)
724 MiB

~96× faster · ~62× less memory · exact row parity · single binary, no daemon

Declare it once

A pipeline is a file, not a codebase.

Describe source, in-flight transforms, write mode, and durable state in YAML — then faucet run. Incremental replication, upsert, secrets, and bookmarks are config, not glue code. Prefer Rust? The same engine drops into your service through typed Source / Sink traits — or drive it over HTTP with faucet serve.

  • Resumable incremental & CDC replication
  • Upsert / delete write modes with effectively-once delivery
  • ${env:…} & secrets-manager interpolation
  • Page-level embedded-DuckDB sql transform
Full config grammar →

Running it as a service? The control plane ships a full OpenAPI 3 spec — import it into Postman, Insomnia, or Bruno (Import → Link → paste the spec URL).

# orders → warehouse, incremental + upsert
version: 1
name: orders_to_warehouse

pipeline:
  source:
    type: postgres
    config:
      connection_url: ${env:PG_URL}
      query: SELECT * FROM orders
      replication:
        method: incremental
        cursor_field: updated_at
  transforms:
    - sql: SELECT *, amount_cents/100.0 AS amount FROM batch
    - redact: { fields: [email] }
  sink:
    type: bigquery
    config:
      dataset: analytics
      table: orders
      write_mode: upsert
      primary_key: [id]
  state:
    type: file
    path: .faucet/state
// same engine, embedded in your service
use faucet_stream::prelude::*;

let source = PostgresSource::new(pg_cfg)?;
let sink   = BigQuerySink::new(bq_cfg)?;

Pipeline::new(&source, &sink)
    .with_state_store(state)   // durable bookmarks
    .run()
    .await?;

// → streams pages, checkpoints after each
//   confirmed write. Bounded memory throughout.
# run it over the HTTP control plane (faucet serve)
curl -X POST http://localhost:8080/v1/runs \
  -H 'authorization: Bearer $FAUCET_TOKEN' \
  -H 'content-type: application/json' \
  --data-binary @pipeline.json

# → 202 Accepted
#   { "id": "run_01H...", "status": "queued" }

# then poll / stream:
curl http://localhost:8080/v1/runs/run_01H...
curl -N .../v1/runs/run_01H.../logs   # SSE
Connectors

38 sources. 30 sinks. One config grammar.

Every connector shares the same auth, pagination, batching, and state model — so what you learn on one applies to all of them. Arrow-native fast paths run columnar end-to-end where both ends support it.

Browse the catalog →

Databases

  • Postgres
  • MySQL
  • SQL Server
  • SQLite
  • MongoDB
  • Redis
  • DuckDB

Change data capture

  • Postgres CDC
  • MySQL binlog
  • SQL Server CDC
  • MongoDB streams

Warehouses

  • BigQuery
  • Snowflake
  • Databricks
  • Redshift
  • ClickHouse
  • Spanner

Streaming & queues

  • Kafka
  • Kinesis
  • Pub/Sub
  • SQS
  • NATS
  • RabbitMQ

Files & object stores

  • S3
  • GCS
  • Azure Blob
  • Parquet
  • Delta Lake
  • Iceberg
  • SFTP
  • CSV
  • JSONL

APIs & search

  • REST
  • GraphQL
  • gRPC
  • XML / SOAP
  • WebSocket
  • Webhook
  • Elasticsearch
Governance in flight

The guardrails live in the pipe — not bolted on downstream.

Everything that usually needs a second platform runs inside the movement path, with zero per-connector code. A masked field never reaches the sink in the clear.

Quality checks

Assert row- and batch-level rules; route failures to a DLQ.

Data contracts

Versioned schemas enforced on the wire, not downstream.

PII masking

Redact or hash sensitive fields before any sink sees a row.

Schema-drift policy

Detect drift and decide: pass, quarantine, or stop.

Column lineage

OpenLineage events + a movement catalog, per column.

SLA monitors

Freshness and volume checks with built-in metrics.

Observability

Metrics and traces, built in.

Every source, sink, transform, and state op emits Prometheus metrics and OpenTelemetry spans automatically — labelled by pipeline, row, and connector. Scrape /metrics, ship spans over OTLP, point Grafana at it.

Prometheus metricsOTLP tracing spans120+ metricszero per-connector code
How it compares

An all-in-one platform — not a pile of services.

A single fast Rust binary and an embeddable library: connectors, orchestration, governance, and delivery in one place.

vs Meltano

Same config-driven ergonomics, ~2 orders of magnitude faster, no Python runtime.

vs Airbyte

No platform to stand up — one binary or a library, governance native not paywalled.

vs Fivetran

Self-hosted and open-source; your data never leaves your infra.

Get started

Your first pipeline runs in five minutes.

Install the CLI, scaffold a config, and move real data — nothing external to stand up.

curl -LsSf https://github.com/faucet-hq/faucet-stream/releases/latest/download/faucet-cli-installer.sh | sh
brew install faucet-hq/faucet-stream/faucet-cli