Connectors / Streaming & queues
RabbitMQ
sinkRabbitMQ (AMQP 0.9.1) sink; exchange publish, per-record routing key, publisher confirms
Install
cargo install faucet-cli --features sink-rabbitmq faucet init --sink rabbitmq
# Cargo.toml — use it as a library cargo add faucet-sink-rabbitmq
Starter config
sink:
type: rabbitmq
config:
# url: "" # Full AMQP URI, e.g.
# host: "" # Broker host.
# port: 0 # Broker port.
# vhost: "" # Virtual host.
# auth: { type: none } # one of: none, plain, external
# --- Alternative variants — replace the block above with one of these ---
# type: plain
# config:
# username: "" # REQUIRED
# password: "" # REQUIRED
#
# type: external
# tls: { ... } # TLS (`amqps://`) settings.
# connection_name: "" # Client connection name shown in the RabbitMQ management UI.
# heartbeat_secs: 0 # Requested heartbeat interval in seconds (`0` disables heartbeats).
# connect_timeout_secs: 30 # Upper bound, in seconds, on establishing the connection (TCP + AMQP handshake, including any retries).
# connect_retries: 0 # Extra TCP connect attempts (exponential backoff) before giving up.
# exchange: "" # Exchange to publish to.
# exchange_kind: null # When set, `exchange` is declared (durable) with this type before the first publish; otherwise it must already exist.
# routing_key: "" # Static routing key for every message.
# routing_key_field: "" # Top-level record field whose value (string, number or bool) is the routing key.
# routing_key_jsonpath: "" # JSONPath (e.g.
# value_format: null # How a message body maps to and from a JSON record.
# persistent: true # Publish with delivery mode 2 (persisted to disk on durable queues).
# mandatory: true # Publish with the `mandatory` flag: a message no queue is bound to receive is returned by the broker and surfaces as a per-row error (DLQ-routable) instead of being silently dropped.
# publish_timeout_secs: 60 # Seconds one publish group (its publishes and their confirms) may take — a broker under a memory or disk alarm blocks publishers indefinitely.
# confirm: true # Enable publisher confirms: every batch waits for the broker to acknowledge each message before the write returns.
# batch_size: 1000 # Messages published before waiting on their confirms.
Capabilities
| Delivery | At-least-once |
|---|---|
| Write modes | append |
| Exactly-once writes | No |
| Schema evolution | No |
| Scoped cleanup | No |
| Staged bulk load | No |
| Run rollback | No |
| Compression | No |
Config fields
| Field | Type | Default | Description |
|---|---|---|---|
url | string | — | Full AMQP URI, e.g. amqp://user:pass@rabbit:5672/%2f (the vhost is URL-encoded; %2f is /). amqps:// selects TLS. Mutually exclusive with host/port/vhost. |
host | string | — | Broker host. Defaults to 127.0.0.1 when url is not set. |
port | integer | — | Broker port. Defaults to 5672 (5671 with TLS). |
vhost | string | — | Virtual host. Defaults to /. |
auth | object | type: none | Authentication. Defaults to RabbitMqAuth::None. |
tls | object | enabled: false | TLS settings. Disabled by default. |
connection_name | string | — | Client connection name shown in the RabbitMQ management UI. |
heartbeat_secs | integer | — | Requested heartbeat interval in seconds (0 disables heartbeats). Unset uses the broker's proposal. |
connect_timeout_secs | integer | 30 | Upper bound, in seconds, on establishing the connection (TCP + AMQP handshake, including any retries). Defaults to 30. |
connect_retries | integer | 0 | Extra TCP connect attempts (exponential backoff) before giving up. Defaults to 0 — an unreachable broker fails immediately. |
exchange | string | "" | Exchange to publish to. Defaults to "", the default exchange, which routes each message to the queue named by its routing key. |
exchange_kind | object | — | When set, exchange is declared (durable) with this type before the first publish; otherwise it must already exist. Not allowed with the default exchange. |
routing_key | string | — | Static routing key for every message. |
routing_key_field | string | — | Top-level record field whose value (string, number or bool) is the routing key. |
routing_key_jsonpath | string | — | JSONPath (e.g. $.meta.route) whose first match is the routing key. |
value_format | object | json | How records encode into message bodies. Defaults to json. |
persistent | boolean | true | Publish with delivery mode 2 (persisted to disk on durable queues). Defaults to true. |
mandatory | boolean | true | Publish with the mandatory flag: a message no queue is bound to receive is returned by the broker and surfaces as a per-row error (DLQ-routable) instead of being silently dropped. Takes effect with confirm (returns are matched to rows through the confirms) and is ignored without it. Defaults to true. |
publish_timeout_secs | integer | 60 | Seconds one publish group (its publishes and their confirms) may take — a broker under a memory or disk alarm blocks publishers indefinitely. On a timeout the rows not yet confirmed fail with a timeout-class row error: a resilience.poison policy resubmits them, otherwise they go to the DLQ; earlier groups keep their outcomes. Default 60. |
confirm | boolean | true | Enable publisher confirms: every batch waits for the broker to acknowledge each message before the write returns. Defaults to true (the reliable setting — without it a broker-side failure is invisible). |
batch_size | integer | 1000 | Messages published before waiting on their confirms. Defaults to DEFAULT_BATCH_SIZE. 0 publishes the whole batch handed over by the pipeline at once. |
Get started
Your first pipeline runs in five minutes.
Install the CLI, scaffold a config, and move real data — nothing external to stand up.
curl -LsSf https://github.com/faucet-hq/faucet-stream/releases/latest/download/faucet-cli-installer.sh | sh brew install faucet-hq/faucet-stream/faucet-cli