Connectors / Streaming & queues

RabbitMQ

sink

RabbitMQ (AMQP 0.9.1) sink; exchange publish, per-record routing key, publisher confirms

Install

cargo install faucet-cli --features sink-rabbitmq
faucet init --sink rabbitmq
# Cargo.toml — use it as a library
cargo add faucet-sink-rabbitmq

Starter config

sink:
  type: rabbitmq
  config:
    # url: ""    # Full AMQP URI, e.g.
    # host: ""    # Broker host.
    # port: 0    # Broker port.
    # vhost: ""    # Virtual host.
    # auth: { type: none }    # one of: none, plain, external
      # --- Alternative variants — replace the block above with one of these ---
      # type: plain
      # config:
        # username: ""    # REQUIRED
        # password: ""    # REQUIRED
      #
      # type: external
    # tls: { ... }    # TLS (`amqps://`) settings.
    # connection_name: ""    # Client connection name shown in the RabbitMQ management UI.
    # heartbeat_secs: 0    # Requested heartbeat interval in seconds (`0` disables heartbeats).
    # connect_timeout_secs: 30    # Upper bound, in seconds, on establishing the connection (TCP + AMQP handshake, including any retries).
    # connect_retries: 0    # Extra TCP connect attempts (exponential backoff) before giving up.
    # exchange: ""    # Exchange to publish to.
    # exchange_kind: null    # When set, `exchange` is declared (durable) with this type before the first publish; otherwise it must already exist.
    # routing_key: ""    # Static routing key for every message.
    # routing_key_field: ""    # Top-level record field whose value (string, number or bool) is the routing key.
    # routing_key_jsonpath: ""    # JSONPath (e.g.
    # value_format: null    # How a message body maps to and from a JSON record.
    # persistent: true    # Publish with delivery mode 2 (persisted to disk on durable queues).
    # mandatory: true    # Publish with the `mandatory` flag: a message no queue is bound to receive is returned by the broker and surfaces as a per-row error (DLQ-routable) instead of being silently dropped.
    # publish_timeout_secs: 60    # Seconds one publish group (its publishes and their confirms) may take — a broker under a memory or disk alarm blocks publishers indefinitely.
    # confirm: true    # Enable publisher confirms: every batch waits for the broker to acknowledge each message before the write returns.
    # batch_size: 1000    # Messages published before waiting on their confirms.

Capabilities

Delivery At-least-once
Write modes append
Exactly-once writes No
Schema evolution No
Scoped cleanup No
Staged bulk load No
Run rollback No
Compression No

Config fields

FieldTypeDefaultDescription
url string — Full AMQP URI, e.g. amqp://user:pass@rabbit:5672/%2f (the vhost is URL-encoded; %2f is /). amqps:// selects TLS. Mutually exclusive with host/port/vhost.
host string — Broker host. Defaults to 127.0.0.1 when url is not set.
port integer — Broker port. Defaults to 5672 (5671 with TLS).
vhost string — Virtual host. Defaults to /.
auth object type: none Authentication. Defaults to RabbitMqAuth::None.
tls object enabled: false TLS settings. Disabled by default.
connection_name string — Client connection name shown in the RabbitMQ management UI.
heartbeat_secs integer — Requested heartbeat interval in seconds (0 disables heartbeats). Unset uses the broker's proposal.
connect_timeout_secs integer 30 Upper bound, in seconds, on establishing the connection (TCP + AMQP handshake, including any retries). Defaults to 30.
connect_retries integer 0 Extra TCP connect attempts (exponential backoff) before giving up. Defaults to 0 — an unreachable broker fails immediately.
exchange string "" Exchange to publish to. Defaults to "", the default exchange, which routes each message to the queue named by its routing key.
exchange_kind object — When set, exchange is declared (durable) with this type before the first publish; otherwise it must already exist. Not allowed with the default exchange.
routing_key string — Static routing key for every message.
routing_key_field string — Top-level record field whose value (string, number or bool) is the routing key.
routing_key_jsonpath string — JSONPath (e.g. $.meta.route) whose first match is the routing key.
value_format object json How records encode into message bodies. Defaults to json.
persistent boolean true Publish with delivery mode 2 (persisted to disk on durable queues). Defaults to true.
mandatory boolean true Publish with the mandatory flag: a message no queue is bound to receive is returned by the broker and surfaces as a per-row error (DLQ-routable) instead of being silently dropped. Takes effect with confirm (returns are matched to rows through the confirms) and is ignored without it. Defaults to true.
publish_timeout_secs integer 60 Seconds one publish group (its publishes and their confirms) may take — a broker under a memory or disk alarm blocks publishers indefinitely. On a timeout the rows not yet confirmed fail with a timeout-class row error: a resilience.poison policy resubmits them, otherwise they go to the DLQ; earlier groups keep their outcomes. Default 60.
confirm boolean true Enable publisher confirms: every batch waits for the broker to acknowledge each message before the write returns. Defaults to true (the reliable setting — without it a broker-side failure is invisible).
batch_size integer 1000 Messages published before waiting on their confirms. Defaults to DEFAULT_BATCH_SIZE. 0 publishes the whole batch handed over by the pipeline at once.
Get started

Your first pipeline runs in five minutes.

Install the CLI, scaffold a config, and move real data — nothing external to stand up.

curl -LsSf https://github.com/faucet-hq/faucet-stream/releases/latest/download/faucet-cli-installer.sh | sh
brew install faucet-hq/faucet-stream/faucet-cli