Secrets that never touch your config
Connectors need credentials — a database URL, an API token, a service-account key. The worst place for those is a config file in git. faucet-stream keeps them out of the config entirely: you write a reference, and the value is fetched from your secret manager at run time.
A reference, not a value
Anywhere a connector takes a string, you can drop a directive that names a secret instead of embedding it. At run time faucet resolves it against the manager and hands the real value to the connector — the config on disk only ever holds the pointer.
# The config holds a *reference*, never the secret itself.
source:
type: postgres
config:
# whole secret →
connection_url: ${aws-sm:prod/faucet/pg-url}
sink:
type: bigquery
config:
# one JSON field of a secret →
service_account: ${gcp-sm:sa-key#private_key}
# Also supported: ${vault:secret/data/app#token},
# ${azure-kv:my-vault/api-key},
# ${env:PGPASSWORD}, ${file:/etc/secrets/token} Four managers, one syntax
${aws-sm:<name-or-ARN>[#field]}— AWS Secrets Manager${gcp-sm:<secret>[#field]}— GCP Secret Manager${azure-kv:<vault>/<secret>[/<version>]}— Azure Key Vault${vault:<path>[#field]}— HashiCorp Vault
The optional #field pulls a single key out of a JSON secret — so one stored
blob can hold a whole set of credentials and each connector takes just the piece it
needs. And ${env:…} / ${file:…} cover the plainer cases
(an environment variable, or a file mounted by your platform — e.g. a Kubernetes Secret
or a CSI-provided path).
Auth is the platform's, not another credential
Each resolver authenticates with the provider's own default credential chain — environment variables, then IAM roles / workload identity / managed identity. So on ECS, GKE, or an Azure VM, faucet reads secrets using the machine's own identity: there is no bootstrap secret to store just to fetch your other secrets.
The SDK for a backend is initialized lazily, on first resolve. A config that
never references aws-sm pays nothing for it — no client, no dependency
cost at run time. You only light up the managers you actually use.
Why it matters
- Nothing sensitive in git. Configs are safe to commit, diff, and share — they contain references, not credentials.
- Rotate without redeploying. Change the secret in the manager; the next run picks up the new value. No config change, no rebuild.
- Least privilege by default. Access is governed by the manager's own policy and the run's identity, not by who can read a file.
Takeaway
Secret handling should be boring: reference it, and let the platform you already trust hand it over at run time. faucet-stream makes the config the wrong place to keep a secret — which is exactly where you want the friction to be. See the secrets reference in the documentation.
More on the blog, or read the documentation.