Crash-safe by bookmark: when it's safe to move a CDC checkpoint
A change-data-capture checkpoint is a promise: 'everything up to here is safely delivered.' Move it too early and a crash silently drops committed rows. faucet-stream moves it from one place only — a durable bookmark, after the sink flushes.
The checkpoint is a promise
Every CDC connector keeps a position in the source's change log — a Postgres LSN, a MongoDB resume token, a SQL Server capture LSN. Advancing that position tells the source: "everything up to here is safely delivered — you can reclaim it." That's the whole danger. Move the checkpoint before the data is durable at the destination, and a crash in the gap loses committed changes that the source has already discarded. No error, no retry — the rows are just gone. This is the worst class of bug: silent downstream data loss.
One rule: advance only from a durable bookmark
faucet-stream advances a replication checkpoint from one place only: a durable bookmark persisted after the sink flushes a page — never from decoded WAL still in flight, never from a keepalive, never from "we read it off the wire." The ordering per page is fixed:
- Read a batch of changes from the source's log.
- Move it through the pipeline and flush it to the sink.
- Only then persist the bookmark for that position.
- Only from that persisted bookmark is the upstream checkpoint ever advanced.
Crash anywhere before step 3 and the bookmark still points at the last proven position; on restart the connector resumes from there and re-reads the un-flushed changes. Nothing committed is skipped.
Honest about the guarantee
Re-reading after a crash means a page can be delivered twice — so the default guarantee is at-least-once, not exactly-once. That's the correct default: it never loses data. Pair it with an idempotent sink (upsert by key) and the duplicates collapse on write, giving you effectively-once end-to-end. faucet is explicit about which you're getting rather than quietly promising more than the source can support.
Per-source, because the logs differ
Each source has its own resume unit, and faucet handles them on their own terms: Postgres
acknowledges a confirmed_flush_lsn back to its logical replication slot (so it
can free WAL); MongoDB and SQL Server are pull/poll-based and just resume from a stored token
or LSN map. The bookmark rule is the same everywhere; only the mechanism to persist and
resume it changes.
Read the details
The full treatment — per-source resume units, the acknowledgement flow, and the crash-resume tests — is in the paper: Crash-Safe Change Data Capture.
More on the blog, or read the documentation.