Install
cargo install faucet-cli --features source-websocket faucet init --source websocket
# Cargo.toml — use it as a library cargo add faucet-source-websocket
Starter config
source:
type: websocket
config:
url: "" # REQUIRED — WebSocket endpoint, `ws://` or `wss://`.
# auth: { type: none } # one of: none, bearer, custom
# --- Alternative variants — replace the block above with one of these ---
# type: bearer
# config:
# token: "" # REQUIRED
#
# type: custom
# config:
# headers: {} # REQUIRED
# subscribe_messages: [] # Subscription frames sent (in order) immediately after every (re)connect.
# message_format: null # How each incoming frame is converted into a record.
# on_parse_error: null # What to do when a `Json`-mode frame is not valid JSON.
# envelope: false # `false` (default) emits the record raw; `true` wraps it as `{ data, received_at, url }`.
# ping_interval: 0 # If set, send a WebSocket Ping frame on this interval (seconds) to keep the connection alive through proxies/load balance…
# max_messages: 0 # Stop after this many messages.
# idle_timeout: 0 # Stop after this many seconds with no message.
# reconnect: false # Reconnect on transport error / non-1000 close.
# reconnect_backoff: 1 # Base wait (seconds) for reconnect backoff.
# reconnect_max_attempts: 0 # Cap on *consecutive* failed reconnects (resets on a received data frame — a handshake the server accepts and then closes still counts).
# max_message_bytes: 0 # Bound the max WebSocket message/frame size (bytes) to prevent runaway memory.
# batch_size: 1000 # Records per emitted `StreamPage`.
# connect_timeout: 30 # Seconds allowed for the TCP connect + upgrade handshake and for sending the subscribe frames (default 30).
Capabilities
| Replay | Non-deterministic replay |
|---|---|
| Exactly-once replay | No |
| Incremental | No |
| Dataset discovery | No |
| Reports lag | No |
| Compression | No |
Config fields
| Field | Type | Default | Description |
|---|---|---|---|
urlrequired | string | — | WebSocket endpoint, ws:// or wss://. Supports {placeholder} parent-matrix context substitution. |
auth | object | type: none | Authentication applied to the HTTP upgrade request. Either inline ({ type, config }) or a { ref: <name> } pointer to a shared provider in the CLI's top-level auth: catalog. |
subscribe_messages | array | [] | Subscription frames sent (in order) immediately after every (re)connect. Empty = send nothing. |
message_format | object | json | How to interpret each incoming frame. |
on_parse_error | object | fail | In Json mode, what to do when a frame is not valid JSON. |
envelope | boolean | false | false (default) emits the record raw; true wraps it as { data, received_at, url }. |
ping_interval | integer | — | If set, send a WebSocket Ping frame on this interval (seconds) to keep the connection alive through proxies/load balancers. |
max_messages | integer | — | Stop after this many messages. At least one of max_messages / idle_timeout must be set. |
idle_timeout | integer | — | Stop after this many seconds with no message. The idle clock keeps ticking across reconnect gaps, so it also caps a connection outage. |
reconnect | boolean | false | Reconnect on transport error / non-1000 close. |
reconnect_backoff | integer | 1 | Base wait (seconds) for reconnect backoff. Default 1s. |
reconnect_max_attempts | integer | — | Cap on *consecutive* failed reconnects (resets on a received data frame — a handshake the server accepts and then closes still counts). None = unlimited (then idle_timeout is the natural cap). |
max_message_bytes | integer | — | Bound the max WebSocket message/frame size (bytes) to prevent runaway memory. None = tungstenite default (64 MiB message / 16 MiB frame). |
batch_size | integer | 1000 | Records per emitted StreamPage. Default DEFAULT_BATCH_SIZE. 0 drains the entire run window into a single page (same sentinel as the Kafka source). |
connect_timeout | integer | 30 | Seconds allowed for the TCP connect + upgrade handshake and for sending the subscribe frames (default 30). Expiry counts as a connect failure, so reconnect retries it. |
Get started
Your first pipeline runs in five minutes.
Install the CLI, scaffold a config, and move real data — nothing external to stand up.
curl -LsSf https://github.com/faucet-hq/faucet-stream/releases/latest/download/faucet-cli-installer.sh | sh brew install faucet-hq/faucet-stream/faucet-cli