faucetstream
Get started

Connectors / Change data capture

SQL Server CDC

source

Microsoft SQL Server CDC source (change data capture, exactly-once capable)

Install

cargo install faucet-cli --features source-mssql-cdc
faucet init --source mssql-cdc
# Cargo.toml — use it as a library
cargo add faucet-source-mssql-cdc

Starter config

source:
  type: mssql-cdc
  config:
    # connection_url: ""    # `mssql://user:pass@host:1433/database` URL form.
    # connection_string: ""    # ADO.NET-style connection string handed straight to `tiberius`, e.g.
    # tls: { ... }    # TLS configuration for an MSSQL connection.
    capture_instances: []    # REQUIRED — Capture instances to poll (e.g.
    # start_position: { type: current }    # one of: current, earliest
      # --- Alternative variants — replace the block above with one of these ---
      # type: earliest
    # poll_interval: 1    # Seconds to wait between empty polls (no new changes).
    # idle_timeout: 30    # Terminator: end the fetch cycle after this much continuous quiet (no change rows across any capture instance).
    # max_staged_records: 1000000    # Max records buffered for a single in-progress transaction before the run aborts with a typed error (rather than risking unbounded memory growth).
    # batch_size: 1000    # Advisory per-page record count.
    # max_connections: 5    # Maximum pooled connections.
    # statement_timeout_secs: 300    # Per-query timeout in seconds (`0` disables).
    # state_key: ""    # Explicit state-store key for the LSN bookmark.
    # on_gap: null    # What to do when a run resumes behind the capture instance's retained minimum LSN — the CDC cleanup job purged changes be…

Capabilities

Replay Deterministic replay
Exactly-once replay Yes
Incremental Yes
Dataset discovery No
Reports lag Yes
Compression No

Config fields

FieldTypeDefaultDescription
capture_instances
required		 array — Capture instances to poll (e.g. dbo_Orders). **Required and non-empty.**
connection_url string — mssql://user:pass@host:1433/database URL form. Mutually exclusive with connection_string.
connection_string string — ADO.NET-style connection string handed straight to tiberius, e.g. Server=tcp:host,1433;Database=db;User Id=sa;Password=...;. Mutually exclusive with connection_url.
tls object type: prefer TLS / encryption settings. Defaults to MssqlTlsMode::Prefer.
start_position object type: current Start position on a fresh run (ignored once a bookmark exists). Default current (skip existing history).
poll_interval integer 1 Seconds to wait between empty polls (no new changes). Default 1s.
idle_timeout integer 30 Terminator: end the fetch cycle after this much continuous quiet (no change rows across any capture instance). Default 30s. A long-running runtime (faucet schedule / faucet serve) re-invokes the source to keep tailing.
max_staged_records integer 1000000 Max records buffered for a single in-progress transaction before the run aborts with a typed error (rather than risking unbounded memory growth). Defaults to 1,000,000; null = unbounded.
batch_size integer 1000 Advisory per-page record count. 0 accumulates every change into a single trailing page (snapshot/test convenience). Default DEFAULT_BATCH_SIZE.
max_connections integer 5 Maximum pooled connections. Defaults to 5; raised to the number of capture instances, since a poll reads every instance's changes at once to merge them in commit order.
statement_timeout_secs integer 300 Per-query timeout in seconds (0 disables). Defaults to 300.
state_key string — Explicit state-store key for the LSN bookmark. When unset, a key is derived from the database (or host) and the sorted capture-instance list.
on_gap object fail What to do when the bookmark predates the retained change history (downtime longer than the CDC cleanup retention): fail (default) or skip.
Get started

Your first pipeline runs in five minutes.

Install the CLI, scaffold a config, and move real data — nothing external to stand up.

curl -LsSf https://github.com/faucet-hq/faucet-stream/releases/latest/download/faucet-cli-installer.sh | sh
brew install faucet-hq/faucet-stream/faucet-cli
Read the guide → Star on GitHub