Connectors / Streaming & queues
NATS
sourceNATS source; subject/JetStream drain with idle/max termination
Install
cargo install faucet-cli --features source-nats faucet init --source nats
# Cargo.toml — use it as a library cargo add faucet-source-nats
Starter config
source:
type: nats
config:
# servers: ["nats://127.0.0.1:4222"] # One or more NATS server URLs, e.g.
# auth: { type: none } # one of: none, token, user_password, n_key, creds_file
# --- Alternative variants — replace the block above with one of these ---
# type: token
# config:
# token: "" # REQUIRED
#
# type: user_password
# config:
# username: "" # REQUIRED
# password: "" # REQUIRED
#
# type: n_key
# config:
# nkey: "" # REQUIRED
#
# type: creds_file
# config:
# path: "" # REQUIRED
# tls: null # `tls:` on the wire: a flag or a settings block.
# name: "" # Optional client connection name (surfaced in NATS server monitoring).
# ignore_discovered_servers: false # Connect only to `servers`, ignoring the cluster members a server advertises.
subject: "" # REQUIRED — Subject to subscribe to.
# queue_group: "" # Optional queue group for core-NATS load-balanced subscriptions — all subscribers sharing a group split the subject's messages.
# jetstream_stream: "" # JetStream stream name.
# jetstream_consumer: "" # Name of the durable JetStream (pull) consumer to bind to.
# max_messages: 0 # Stop after this many messages have been drained.
# idle_timeout_secs: 0 # Stop after this many seconds elapse with no new message.
# batch_size: 1000 # Messages per emitted `StreamPage`.
# progress_interval_secs: 10 # JetStream only: send an in-progress ack (`+WPI`) every this many seconds for every message pulled but not yet acked (while a page is assembled and while the sink writes it), resetting the consumer's `ack_wait` so a slow page is not redelivered into the same run.
# value_format: null # How `NatsSourceConfig::value_format` decodes a payload.
# include_metadata: false # Wrap each record as `{ subject, sequence, message_id, payload }` so a downstream consumer can deduplicate redeliveries: `sequence` is the JetStream stream sequence (`null` in core mode) and `message_id` the `Nats-Msg-Id` header (`null` when absent).
# on_decode_error: null # `NatsSourceConfig::on_decode_error`.
Capabilities
| Replay | Non-deterministic replay |
|---|---|
| Exactly-once replay | No |
| Incremental | No |
| Dataset discovery | No |
| Reports lag | No |
| Compression | No |
Config fields
| Field | Type | Default | Description |
|---|---|---|---|
subjectrequired | string | — | Subject to subscribe to. Supports NATS wildcards (* for one token, > for the remaining tokens). In JetStream mode the durable consumer's own filter subject governs delivery and this field is informational. |
servers | array | ["nats://127.0.0.1:4222"] | One or more NATS server URLs, e.g. ["nats://127.0.0.1:4222"]. The client connects to the first reachable server and uses the rest for failover. |
auth | object | type: none | Authentication mode. Defaults to none (anonymous). |
tls | boolean | object | false | TLS: true to require it with the platform roots, or a block with ca_cert_path / ca_cert_pem, client_cert_path + client_key_path and insecure_skip_verify (which also requires TLS). Defaults to false. |
name | string | — | Optional client connection name (surfaced in NATS server monitoring). |
ignore_discovered_servers | boolean | — | Connect only to servers, ignoring the cluster members a server advertises. Defaults to false. |
queue_group | string | — | Optional queue group for core-NATS load-balanced subscriptions — all subscribers sharing a group split the subject's messages. Ignored in JetStream mode. |
jetstream_stream | string | — | JetStream stream name. When set together with jetstream_consumer the source pulls from a durable JetStream consumer instead of a core-NATS subscription. |
jetstream_consumer | string | — | Name of the durable JetStream (pull) consumer to bind to. Required when jetstream_stream is set; the consumer must already exist on the server. |
max_messages | integer | — | Stop after this many messages have been drained. At least one of max_messages / idle_timeout_secs must be set so a run terminates. |
idle_timeout_secs | integer | — | Stop after this many seconds elapse with no new message. At least one of max_messages / idle_timeout_secs must be set so a run terminates. |
batch_size | integer | 1000 | Messages per emitted StreamPage. Drained messages accumulate in an in-memory buffer and a page is yielded when the buffer reaches this size (or when the run terminates with a partial buffer). Defaults to DEFAULT_BATCH_SIZE. |
progress_interval_secs | integer | 10 | JetStream only: send an in-progress ack (+WPI) every this many seconds for every message pulled but not yet acked (while a page is assembled and while the sink writes it), resetting the consumer's ack_wait so a slow page is not redelivered into the same run. Keep it below the consumer's ack_wait (30 s by default). 0 disables. Default 10. |
value_format | object | auto | How a message payload becomes a record: auto (default — JSON when it parses, else the UTF-8 text), json, string or bytes (base64). A payload the format cannot represent fails the run rather than being altered (#789 MSG-39). |
include_metadata | boolean | false | Wrap each record as { subject, sequence, message_id, payload } so a downstream consumer can deduplicate redeliveries: sequence is the JetStream stream sequence (null in core mode) and message_id the Nats-Msg-Id header (null when absent). Default false (the payload alone). |
on_decode_error | object | fail | What a payload value_format cannot decode does: fail (default) fails the run; skip logs a warning and drops it — a JetStream message is terminated so it is not redelivered. |
Get started
Your first pipeline runs in five minutes.
Install the CLI, scaffold a config, and move real data — nothing external to stand up.
curl -LsSf https://github.com/faucet-hq/faucet-stream/releases/latest/download/faucet-cli-installer.sh | sh brew install faucet-hq/faucet-stream/faucet-cli