faucetstream
Get started

Connectors / Warehouses

Amazon Redshift

sink

Amazon Redshift sink (COPY-from-S3 or multi-row INSERT)

Install

cargo install faucet-cli --features sink-redshift
faucet init --sink redshift
# Cargo.toml — use it as a library
cargo add faucet-sink-redshift

Starter config

sink:
  type: redshift
  config:
    # host: ""    # Cluster / endpoint host (e.g.
    # port: 5439    # Port.
    database: ""    # REQUIRED — Database name.
    # user: ""    # Database user.
    credentials:
      type: password    # one of: password, iam, redshift_data_api
      config:
        password: ""    # REQUIRED — The password (use `${env:…}` / `${vault:…}` to inject it, never a literal).
      # --- Alternative variants — replace the block above with one of these ---
      # type: iam
      # config:
        # region: ""
        # cluster_identifier: ""
        # db_user: ""
        # workgroup_name: ""
        # auto_create: false
        # db_groups: []
        # duration_secs: 0
        # aws_credentials: null
        # endpoint_url: ""
      #
      # type: redshift_data_api
      # config:
        # region: ""
        # cluster_identifier: ""
        # workgroup_name: ""
        # secret_arn: ""
        # db_user: ""
        # aws_credentials: null
        # endpoint_url: ""
    # tls: true    # Whether to require TLS.
    # tls_mode: null    # Explicit TLS mode; overrides `tls`.
    # ssl_root_cert: ""    # PEM file of the CA that signed the server certificate (the Redshift CA bundle), for `tls_mode: verify_ca | verify_full`.
    table_name: ""    # REQUIRED — Target table name.
    # create_table: true    # Create the target table (and its schema, when `schema:` is set) if it does not exist, inferring the columns from the first written page (#580).
    # commit_rows: 0    # Commit-group size for the cross-page accumulator (#617).
    # commit_bytes: 0    # Estimated-bytes counterpart of `commit_rows` (#617).
    # schema: ""    # Optional schema (namespace) qualifying `table_name`.
    # write_strategy: null    # How the sink loads rows into Redshift.
    # copy: null    # `COPY`-path settings, grouped (#654 M20).
    # copy_format: null    # Format of the staged file that `COPY` reads.
    # staging_bucket: ""    # **Deprecated** — use `copy.staging_bucket`.
    # staging_prefix: ""    # **Deprecated** — use `copy.staging_prefix`.
    # iam_role: ""    # **Deprecated** — use `copy.iam_role`.
    # region: ""    # **Deprecated** — use `copy.region`.
    # endpoint_url: ""    # **Deprecated** — use `copy.endpoint_url`.
    # batch_size: 1000    # Rows per load unit.
    # max_connections: 5    # Maximum number of connections in the pool.

Capabilities

Delivery At-least-once
Write modes append
Exactly-once writes No
Schema evolution No
Scoped cleanup No
Staged bulk load Yes
Run rollback No
Compression No

Config fields

FieldTypeDefaultDescription
database
required		 string — Database name.
credentials
required		 object — Authentication credentials.
table_name
required		 string — Target table name.
host string "" Cluster / endpoint host (e.g. my-cluster.abc123.us-east-1.redshift.amazonaws.com). Required except with credentials: { type: redshift_data_api }.
port integer 5439 Port. Defaults to DEFAULT_PORT (5439).
user string "" Database user. Required for password; for a provisioned iam cluster it is the default db_user; unused by redshift_data_api.
tls boolean true Whether to require TLS. Defaults to true (Redshift clusters require SSL by default). true maps to sslmode=require (encrypted, but the server certificate is **not** verified); false maps to sslmode=prefer (opportunistic TLS with plaintext fallback). Ignored when tls_mode is set.
tls_mode object — Explicit TLS mode; overrides tls. verify_ca / verify_full check the server certificate against ssl_root_cert (or, without one, the Mozilla root store bundled into the binary — not the operating system's), the latter also its host name.
ssl_root_cert string — PEM file of the CA that signed the server certificate (the Redshift CA bundle), for tls_mode: verify_ca | verify_full.
create_table boolean true Create the target table (and its schema, when schema: is set) if it does not exist, inferring the columns from the first written page (#580). Enabled by default: a first-ever sync cannot assume the destination already exists.
commit_rows integer — Commit-group size for the cross-page accumulator (#617).
commit_bytes integer — Estimated-bytes counterpart of commit_rows (#617). Rows are a poor proxy for how much work a warehouse commit is; this bounds the buffered size. 0 means no byte limit; see commit_rows for the default when both are unset.
schema string — Optional schema (namespace) qualifying table_name.
write_strategy object copy How rows are loaded. Defaults to RedshiftWriteStrategy::Copy.
copy object — COPY-path settings, grouped (#654 M20). Applies only to write_strategy: copy; when present it supersedes the six deprecated flat keys below. This mirrors the BigQuery sink's bulk_load: block, which is the house shape for a staged-load configuration.
copy_format object jsonl **Deprecated** — use copy.format.
staging_bucket string — **Deprecated** — use copy.staging_bucket.
staging_prefix string "" **Deprecated** — use copy.staging_prefix.
iam_role string — **Deprecated** — use copy.iam_role.
region string — **Deprecated** — use copy.region.
endpoint_url string — **Deprecated** — use copy.endpoint_url.
batch_size integer 1000 Rows per load unit. For insert, the per-statement multi-row chunk size; for copy, the number of rows per staged S3 object (a commit group's objects load in one COPY through a manifest). Defaults to DEFAULT_BATCH_SIZE. 0 = one unit for the whole page.
max_connections integer 5 Maximum number of connections in the pool. Defaults to 5.
Get started

Your first pipeline runs in five minutes.

Install the CLI, scaffold a config, and move real data — nothing external to stand up.

curl -LsSf https://github.com/faucet-hq/faucet-stream/releases/latest/download/faucet-cli-installer.sh | sh
brew install faucet-hq/faucet-stream/faucet-cli
Read the guide → Star on GitHub