faucetstream
Get started

Connectors / Streaming & queues

AWS SQS

sink

AWS SQS sink; batched SendMessageBatch with per-entry retry

Install

cargo install faucet-cli --features sink-sqs
faucet init --sink sqs
# Cargo.toml — use it as a library
cargo add faucet-sink-sqs

Starter config

sink:
  type: sqs
  config:
    queue_url: ""    # REQUIRED — SQS queue URL (e.g.
    # region: ""    # AWS region.
    # endpoint_url: ""    # Custom endpoint URL for LocalStack / VPC endpoints.
    # credentials: { type: default }    # one of: default, profile, access_key, assume_role, web_identity
      # --- Alternative variants — replace the block above with one of these ---
      # type: profile
      # config:
        # name: ""    # REQUIRED
      #
      # type: access_key
      # config:
        # access_key_id: ""    # REQUIRED
        # secret_access_key: ""    # REQUIRED
        # session_token: ""
      #
      # type: assume_role
      # config:
        # role_arn: ""    # REQUIRED
        # session_name: ""
        # external_id: ""
      #
      # type: web_identity
    # message_group_id: ""    # FIFO message group id, applied to every message.
    # message_deduplication_id_field: ""    # Record field whose (stringified) value becomes each message's `MessageDeduplicationId` on a FIFO queue.
    # batch_size: 10    # Max entries per `SendMessageBatch` request (1–10).
    # concurrency: 4    # Bounded concurrent in-flight `SendMessageBatch` requests.
    # retry: null    # Per-record retry for partial failures, grouped (#654 M20).
    # retry_max_attempts: 5    # **Deprecated** — use `retry.max_attempts`.
    # retry_initial_backoff_ms: 100    # **Deprecated** — use `retry.initial_backoff_ms`.
    # retry_max_backoff_ms: 30000    # **Deprecated** — use `retry.max_backoff_ms`.

Capabilities

Delivery At-least-once
Write modes append
Exactly-once writes No
Schema evolution No
Scoped cleanup No
Staged bulk load No
Run rollback No
Compression No

Config fields

FieldTypeDefaultDescription
queue_url
required		 string — SQS queue URL (e.g. https://sqs.us-east-1.amazonaws.com/1234/my-q).
region string — AWS region. None uses the SDK default chain.
endpoint_url string — Custom endpoint URL for LocalStack / VPC endpoints.
credentials object type: default AWS credentials. Defaults to the SDK default provider chain.
message_group_id string — FIFO message group id, applied to every message. Required by FIFO queues; omit for standard queues. Default: none.
message_deduplication_id_field string — Record field whose (stringified) value becomes each message's MessageDeduplicationId on a FIFO queue. A record missing the field (or with a non-scalar value) fails per-record (DLQ-routable). Only used when set. Default: none.
batch_size integer 10 Max entries per SendMessageBatch request (1–10). Default 10.
concurrency integer 4 Bounded concurrent in-flight SendMessageBatch requests. Default 4. A FIFO queue (.fifo) always sends one request at a time, so a message group's order is preserved.
retry object — Per-record retry for partial failures, grouped (#654 M20). When present it supersedes the three deprecated flat retry_* keys below.
retry_max_attempts integer 5 **Deprecated** — use retry.max_attempts.
retry_initial_backoff_ms integer 100 **Deprecated** — use retry.initial_backoff_ms.
retry_max_backoff_ms integer 30000 **Deprecated** — use retry.max_backoff_ms.
Get started

Your first pipeline runs in five minutes.

Install the CLI, scaffold a config, and move real data — nothing external to stand up.

curl -LsSf https://github.com/faucet-hq/faucet-stream/releases/latest/download/faucet-cli-installer.sh | sh
brew install faucet-hq/faucet-stream/faucet-cli
Read the guide → Star on GitHub