Connectors / Change data capture
Oracle CDC
sourceOracle CDC via LogMiner; committed-transaction change events, SCN bookmarks, exactly-once; needs Oracle Instant Client
Install
cargo install faucet-cli --features source-oracle-cdc faucet init --source oracle-cdc
# Cargo.toml — use it as a library cargo add faucet-source-oracle-cdc
Starter config
source:
type: oracle-cdc
config:
# connect_string: "" # Easy Connect (`host:1521/FREEPDB1`), TNS alias, or connect descriptor.
# host: "" # Database host.
# port: 0 # Listener port.
# service_name: "" # Service name (e.g.
# sid: "" # Legacy SID.
# username: "" # Database user.
# password: "" # Password for `username`.
# external_auth: false # Authenticate with external credentials (OS authentication or a wallet holding the credentials) instead of `username` / `…
# tls: { ... } # TCPS settings applied to a `host`-form connection.
tables: [] # REQUIRED — Tables to capture as `OWNER.TABLE`, in data-dictionary case (usually upper case).
# start_position: { type: current } # one of: current, earliest
# --- Alternative variants — replace the block above with one of these ---
# type: earliest
# poll_interval: 1 # Seconds to wait between polls that find nothing new.
# idle_timeout: 30 # End the fetch cycle after this long without a captured change, counted only once mining has reached the current SCN.
# max_scn_window: 500000 # Largest SCN range mined per LogMiner session.
# max_staged_records: 1000000 # Abort when one in-flight transaction buffers more than this many changes (bounded memory).
# max_staged_records_total: null # Abort when all open transactions together buffer more than this many changes, counting LOB pieces and folded updates.
# batch_size: 1000 # `0` aggregates committed transactions into pages of up to 100,000 records, emitted at least every `idle_timeout`, each with its bookmark; otherwise each committed transaction is its own page.
# max_connections: 2 # Maximum pooled sessions.
# statement_timeout_secs: 600 # Per-round-trip call timeout in seconds (`0` disables).
# flush_table: "FAUCET_LOGMNR_FLUSH" # Table the source commits to before each mining window, forcing the log writer to flush redo up to the window's end SCN.
# on_unsupported: null # What to do with a change LogMiner cannot render as SQL for a tracked table (`UNSUPPORTED` rows, or a *dictionary mismatc…
# state_key: "" # Explicit state-store key.
Capabilities
| Replay | Deterministic replay |
|---|---|
| Exactly-once replay | Yes |
| Incremental | Yes |
| Dataset discovery | No |
| Reports lag | Yes |
| Compression | No |
Config fields
| Field | Type | Default | Description |
|---|---|---|---|
tablesrequired | array | — | Tables to capture as OWNER.TABLE, in data-dictionary case (usually upper case). **Required and non-empty.** |
connect_string | string | — | Easy Connect (host:1521/FREEPDB1), TNS alias, or connect descriptor. Mutually exclusive with host. |
host | string | — | Database host. Mutually exclusive with connect_string. |
port | integer | — | Listener port. Defaults to 1521 (2484 when tls.enabled). |
service_name | string | — | Service name (e.g. a PDB such as FREEPDB1). Mutually exclusive with sid. |
sid | string | — | Legacy SID. Mutually exclusive with service_name. |
username | string | "" | Database user. Required unless external_auth is set. |
password | string | "" | Password for username. |
external_auth | boolean | false | Authenticate with external credentials (OS authentication or a wallet holding the credentials) instead of username / password. |
tls | object | enabled: falseserver_dn_match: true | Transport encryption (TCPS) for the host form. |
start_position | object | type: current | Start position on a fresh run. Default current. |
poll_interval | integer | 1 | Seconds to wait between polls that find nothing new. Default 1s. |
idle_timeout | integer | 30 | End the fetch cycle after this long without a captured change, counted only once mining has reached the current SCN. Default 30s. A long-running runtime re-invokes the source to keep tailing. |
max_scn_window | integer | 500000 | Largest SCN range mined per LogMiner session. Default 500000. |
max_staged_records | integer | 1000000 | Abort when one in-flight transaction buffers more than this many changes (bounded memory). Defaults to 1,000,000; null = unbounded. |
max_staged_records_total | integer | — | Abort when all open transactions together buffer more than this many changes, counting LOB pieces and folded updates. Unset or null = unbounded. |
batch_size | integer | 1000 | 0 aggregates committed transactions into pages of up to 100,000 records, emitted at least every idle_timeout, each with its bookmark; otherwise each committed transaction is its own page. Default DEFAULT_BATCH_SIZE. |
max_connections | integer | 2 | Maximum pooled sessions. Default 2. |
statement_timeout_secs | integer | 600 | Per-round-trip call timeout in seconds (0 disables). Default 600. |
flush_table | string | FAUCET_LOGMNR_FLUSH | Table the source commits to before each mining window, forcing the log writer to flush redo up to the window's end SCN. Created in the connecting user's schema when missing. Default FAUCET_LOGMNR_FLUSH. |
on_unsupported | object | fail | Handling of changes LogMiner cannot render. Default fail. |
state_key | string | — | Explicit state-store key. Derived as oracle-cdc:<database>:<table> (or a digest of several tables) when unset. |
Get started
Your first pipeline runs in five minutes.
Install the CLI, scaffold a config, and move real data — nothing external to stand up.
curl -LsSf https://github.com/faucet-hq/faucet-stream/releases/latest/download/faucet-cli-installer.sh | sh brew install faucet-hq/faucet-stream/faucet-cli