faucetstream
Get started

Connectors / Databases

MongoDB

source

MongoDB query source

Install

cargo install faucet-cli --features source-mongodb
faucet init --source mongodb
# Cargo.toml — use it as a library
cargo add faucet-source-mongodb

Starter config

source:
  type: mongodb
  config:
    connection_uri: ""    # REQUIRED — MongoDB connection URI (e.g.
    database: ""    # REQUIRED — Database name.
    collection: ""    # REQUIRED — Collection name.
    # filter: null    # Optional query filter as JSON, converted to a BSON `Document` at query time.
    # projection: null    # Optional field projection as JSON.
    # sort: null    # Optional sort specification as JSON.
    # limit: 0    # Maximum number of documents to return.
    # cursor_batch_size: 0    # Cursor batch size: documents per server round-trip on the MongoDB driver cursor.
    # batch_size: 1000    # Records per emitted `StreamPage`.

Capabilities

Replay Non-deterministic replay
Exactly-once replay No
Incremental No
Dataset discovery Yes
Reports lag No
Compression No

Config fields

FieldTypeDefaultDescription
connection_uri
required		 string — MongoDB connection URI (e.g. mongodb://localhost:27017).
database
required		 string — Database name.
collection
required		 string — Collection name.
filter any — Optional query filter as JSON, converted to a BSON Document at query time.
projection any — Optional field projection as JSON.
sort any — Optional sort specification as JSON.
limit integer — Maximum number of documents to return.
cursor_batch_size integer — Cursor batch size: documents per server round-trip on the MongoDB driver cursor. This is a wire-level tuning knob — distinct from Self::batch_size, which controls the size of each emitted StreamPage.
batch_size integer 1000 Records per emitted StreamPage. Documents are pulled from the cursor and yielded whenever the buffer reaches this size. Defaults to DEFAULT_BATCH_SIZE.
Get started

Your first pipeline runs in five minutes.

Install the CLI, scaffold a config, and move real data — nothing external to stand up.

curl -LsSf https://github.com/faucet-hq/faucet-stream/releases/latest/download/faucet-cli-installer.sh | sh
brew install faucet-hq/faucet-stream/faucet-cli
Read the guide → Star on GitHub