Constant memory at any scale
Decouple peak memory from dataset size across every source and sink, so a large table never OOMs the runner.
Roadmap
Where faucet-stream is headed
The runtime, CLI, and governance layer are feature-complete. What’s left is making every connector as fast and memory-bounded as the engine already is — and growing the ecosystem around it. This is the curated view; the live tracker is the roadmap epic on GitHub, where every issue is public and open to comment.
Track it live on GitHub — epic #38 →The foundation, shipped
Exactly-once and effectively-once delivery, unified write modes / upsert, CDC with
crash-safe bookmarks, a full in-flight governance layer (quality, contracts,
schema-drift, PII masking, SLAs), the faucet serve HTTP control plane
with triggers, clustering, RBAC, a web console, and a data-movement catalog —
plus multi-edge topology, cross-row joins, WASM transforms, and an MCP server.
The fleet ships 38 sources and 30 sinks today.
Now
What we’re building
The runtime and CLI are feature-complete; the active work is closing the one axis where a streaming data mover can still lose — memory — and going columnar end-to-end.
Arrow end-to-end
Vectorized, columnar-native transforms, quality, and masking — plus a byte-passthrough fast-load — so Arrow-to-Arrow moves never pay the row-conversion tax.
Warehouse & lakehouse loads
Bulk load jobs over per-page commits, and correctness fixes on the BigQuery and Delta paths.
Next
On deck
Scale the matrix runtime out further, and round out the serve control plane and operator DX.
Parallelism & scale
Within-object partitioned extraction, discovery-driven fan-out, and cost-aware dispatch to cut makespan on wide tenants.
Serve control plane & RBAC
A first-class read/write/admin token trio with a verified permission matrix, structured JSON logging, and per-matrix-row run timings.
Incremental everywhere
Bookmark-based incremental replication for the REST/Salesforce Bulk async-job path, and adaptive poll backoff.
Exploring
Further out
Ecosystem surface and adoption — the center of gravity now that the engine is done.
Connector & Template Hubs
An auto-generated, crates.io-backed connector catalog with conformance badges, and a source/sink-split template hub with pull-based hosting.
Reach
New platforms — prebuilt Windows binaries.
Adoption & DX
Docs, positioning, examples, community, and the technical white-paper program.
Priorities shift as the work and the community do. For the authoritative, always-current picture — and to weigh in — follow the roadmap epic and the open issues on GitHub.
Get started
Your first pipeline runs in five minutes.
Install the CLI, scaffold a config, and move real data — nothing external to stand up.
curl -LsSf https://github.com/faucet-hq/faucet-stream/releases/latest/download/faucet-cli-installer.sh | sh brew install faucet-hq/faucet-stream/faucet-cli