Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Benchmarks (vs Meltano)

Honest, reproducible evidence for the “built for throughput” claim. Every number below comes from BENCHMARKS.md — identical workloads, one machine (Apple M3 Pro, 12 cores, 18 GiB RAM), 1M rows, seed 42, median of 5 timed runs. faucet-stream is compared against Meltano (the most common Singer runtime).

Read the caveats first. This measures single-machine batch throughput of three specific moves. It does not measure distributed throughput, connector breadth, or correctness. The CSV→JSONL figure is a best case (upper bound), not the typical case — see vs. Meltano and BENCHMARKS.md for the full methodology, hardware capture, and the Postgres-row measurement caveat.

~96×
faster on CSV → JSONL (best case, parse-bound)
~16×
faster on a realistic DB → DB move (sink-bound)
~62×
less peak memory (11.8 vs 724 MiB)
1:1
exact row-count parity, every scenario
faucet-stream Meltano (Singer)
Throughput — rows/second (higher is better)
1,000,000 rows. Meltano's bar is a sliver on purpose — that is the result.
A — CSV → JSONL · parse-bound (best case)
faucet
712,403
Meltano
7,383
B — Postgres → JSONL · typed row decode
faucet
179,700
Meltano
7,184
C — Postgres → Postgres · sink-bound (the realistic move)
faucet (COPY)
123,200
faucet (INSERT)
99,000
Meltano
7,706
Peak memory — MiB (lower is better)
Here faucet is the sliver: bounded-memory streaming holds flat while Meltano buffers.
A — CSV → JSONL
faucet
11.8
Meltano
724.5
B — Postgres → JSONL
faucet
13.9
Meltano
743.0
C — Postgres → Postgres
faucet
35.9
Meltano
485.7
The gap collapses as the workload gets more I/O-bound
Speed-up multiplier (faucet ÷ Meltano). The best case is not the typical case.
A · parse-bound
~96×
B · row decode
~25×
C · sink-bound
~16×
Show the raw numbers as a table
ScenarioBottleneckfaucet (rows/s)Meltano (rows/s)Gapfaucet RSS (MiB)Meltano RSS (MiB)
A — CSV → JSONLparse/serialize (best case)712,4037,383~96×11.8724.5
B — Postgres → JSONLtyped row decode179,7007,184~25×13.9743.0
C — Postgres → Postgresdestination write (sink-bound)123,200 (COPY) / 99,000 (INSERT)7,706~16× / ~13×35.9485.7

All runs: 1,000,000 rows, seed 42, exact row-count parity (1,000,000 = 1,000,000).

How to reproduce

The harness never fabricates a number — a tool that won’t install/run is recorded as such, not faked. Regenerate everything on your own hardware:

make bench            # Scenario A (CSV → JSONL, 1M rows) — no infra
make bench-smoke      # 100k-row smoke run
make bench-postgres   # adds Scenarios B & C (needs Docker)

Results land in benchmarks/results/. One independent confirmation on your own hardware is worth more to this project than a new connector — open an issue or PR with your output, especially if faucet does not win. See benchmarks/README.md and Performance tuning for the levers behind these numbers.