Skip to content

Parameters & pipeline templates

Most fleets run the same pipeline shape over and over with a handful of values that change: a tenant id, a date window, a target table, a source URL. Copying the config once per tenant means N files to keep in sync; re-sending the whole config on every trigger means re-validating it every time and hoping the caller got the shape right.

faucet splits that into two pieces:

  • params: — a config declares its typed, trigger-time surface. Available everywhere, no build feature needed.
  • the template registry — register a parameterized config once, then trigger runs by {id, params}. Needs the templates build feature.
kind: pipeline # a complete config — see "Registering a template" for the other kinds
version: 1
name: tenant-sync
params:
tenant_id:
type: string
required: true
description: Tenant whose events to sync
since:
type: string
default: "1970-01-01"
page_size:
type: int
default: 500
api_token:
type: string
required: true
secret: true
region:
type: string
default: us
values: [us, eu, apac]
pipeline:
source:
type: rest
config:
base_url: https://api.example.com
path: "/tenants/${param.tenant_id}/events"
query_params: { since: "${param.since}" }
auth: { type: bearer, config: { token: "${param.api_token}" } }
pagination:
type: PageNumber
param_name: page
start_page: 1
page_size: "${param.page_size}"
page_size_param: per_page
sink:
type: file
config:
path: "./out/${param.tenant_id}/events.jsonl"

Fields per entry:

Field Meaning
type string (default) · int · float · bool
required The caller must supply a value. Mutually exclusive with default.
default Value when the caller supplies none. An ordinary config scalar, so default: "${env:SINCE}" works. On a typed param it may be text that resolves to the type (type: int, default: "${env:PORT}").
secret Registered for redaction the instant it is bound — never reaches a log, an error message, an API response, the audit log, or the registry.
description Shown by faucet template list / show, GET /v1/templates, and the MCP get_template tool.
values Closed set of acceptable values. Anything else is rejected at bind time, naming the set. A default must be one of them.

Reference a param anywhere in the config as ${param.NAME}. Write $${param.x} for a literal ${param.x}.

A param spliced into a longer string becomes plain text of that string — it is not bound as a SQL parameter or URL-encoded. So a value the caller supplies for a param without a values: set may not carry a quote (', ", `), ;, a backslash or a control character when it is spliced into a longer string (query: "… WHERE account = '${param.account}'"); such a trigger is refused. In a SQL field (query, sql, statement, *_query, and any field a connector’s schema marks as SQL) a splice that is not enclosed in quotes (WHERE id = ${param.id}) takes only a number (42, -7, 3.5) — text that could name a column or form an expression is refused, so quote the splice ('${param.id}') or declare the param int / float — and in a URL field (url, *_url, path, endpoint) a spliced value may not carry ?, #, &, whitespace or a .. segment. The same rules follow a value that reaches the longer string through a reference — vars: { acct: "${param.account}" } spliced as '${vars.acct}', or a template field read with ${sources.NAME.config.FIELD} — judged by the field the text finally lands in. For a param a lower-trust caller fills into a query, declare a closed values: set. A whole-value reference (table: ${param.table}) and a default are not restricted.

values: turns a param into an enumerable axis. Without it a typo’d value — region: ue — binds happily and surfaces as a 404 halfway through a run; with it the bind fails up front and names the three it will accept:

param 'region': 'ue' is not one of the allowed values: apac, eu, us

Being enumerable is the other half: a test suite can sweep every declared value without being told what they are, so the sweep stays correct as the set grows.

When ${param.NAME} is a scalar’s entire text, the declared type survives: page_size above arrives at the connector as the JSON number 500, not "500". Embedded in a longer string (.../events?since=${param.since}) it is stringified, like every other interpolation namespace.

Values are accepted in either wire shape, so a CLI --param page_size=500 and an HTTP {"page_size": 500} behave identically — and --param page_size=abc is rejected up front, naming the param.

On faucet serve a template’s body is the operator’s, but the values a caller supplies for its params are the caller’s. When one lands in a path, URL or command field — a sink path: ${param.out}, a source base_url: ${param.api_root}, a DLQ path — the server holds it to its caller reference policy (--allow-path-root, --allow-url-host, …) exactly as it would a config the caller submitted, and refuses the run with a 403 naming the param and the field. Defaults, closed values: sets, int / float / bool params and ${map:}-computed params are the author’s and are never checked; see the HTTP API reference.

When the author means a param’s value to reach such a field as given, mark it trusted:

params:
out_dir: { required: true, trusted: true, description: "Any directory on the host" }

faucet template show lists such a param as [trusted] and the console marks it with a trusted pill. Registering a template already needs the admin role, so only an admin can make that choice. trusted applies to a whole param (not a list’s items or one of their fields) and not to a computed param — mark the params it reads instead.

faucet run and faucet validate take --param:

Terminal window
faucet run tenant-sync.yaml --param tenant_id=acme --param since=2026-01-01 \
--param api_token="$TOKEN"
# Validate in CI without inventing values: required params bind to type-shaped
# placeholders, so the config's structure is still fully checked.
faucet validate tenant-sync.yaml
# Or check one concrete invocation end to end (strict binding).
faucet validate tenant-sync.yaml --param tenant_id=acme --param api_token=x

--param-env NAME=VALUE overrides an environment variable for that run’s ${env:VAR} resolution only; bare --param-env TOKEN takes the value from your own environment, so a secret never appears in the process arguments. The process environment itself is never modified — which is what makes this safe inside a concurrent server. A template trigger’s env overrides (HTTP / MCP) may only name variables the template reads (${env:NAME} / ${secret:NAME}) and are literal text — a value containing ${ is refused, and a tenant-scoped principal may not pass env at all.

Terminal window
faucet run tenant-sync.yaml --param tenant_id=acme --param-env API_HOST=eu.example.com

The registry stores four kinds of document, told apart by a kind: line:

kind: What it is How it runs
source-template One system: its connector, shared transforms, and streams with per-stream write preferences (Template Hub) Composed with a registered sink-template: faucet template run <source> --sink <sink> / POST …/runs {"sink": …}
sink-template One destination and how a stream is addressed (per_stream) Never on its own — named as the sink of a source template’s run
deployment The operational blocks of a composed run — state, dlq, notifications, sla, … (Deployment overlays) Never on its own — named as the overlay of a source template’s run
pipeline A complete config with params: Alone, as below

A source or sink template is registered under its own name (the hub id), is validated as a hub template, and goes through the publishability lint — a literal credential or a private hostname is refused, because a shared registry is a shared place. A template’s kind is fixed for its id: a later register under the same id with a different kind is refused. A document without kind: is still accepted as a pipeline but prints a deprecation notice — add kind: pipeline to a complete config.

A pipeline template’s name: is its state-key namespace ({name}::{row}). One registered without a name: runs under its template id — the same on faucet template run, a server trigger and MCP — so two unnamed templates sharing a state store never read each other’s bookmarks. Releases before this ran unnamed templates as serve / pipeline; see Upgrading for keeping their bookmarks.

Terminal window
faucet template register hub/source-templates/acme/billing.yaml --launch # id = acme/billing
faucet template register hub/sink-templates/faucet-hq/bigquery.yaml --launch # id = faucet-hq/bigquery
faucet template list --kind sink-template
faucet template run acme/billing --sink faucet-hq/bigquery \
--param api_token --param bq_project=my-project --param bq_sa_key # secrets from $API_TOKEN / $BQ_SA_KEY
# → composes the two, prints the per-stream plan (bills: overwrite, transactions: upsert[id], …), runs
faucet template register ops/prod.yaml --launch # kind: deployment
faucet template run acme/billing --sink faucet-hq/bigquery --overlay prod … # + state / DLQ / SLA

The rest of this page uses a complete pipeline template; everything about versions, channels, launching, and triggering applies to all three kinds.

Terminal window
faucet template register tenant-sync.yaml --store sqlite:./faucet-templates.db
# registered template 'tenant-sync' version 1
#
# params:
# api_token string required [secret]
# page_size int default 500
# since string default "1970-01-01"
# tenant_id string required — Tenant whose events to sync
faucet template list --store sqlite:./faucet-templates.db
faucet template show tenant-sync --store sqlite:./faucet-templates.db
faucet template run tenant-sync --store sqlite:./faucet-templates.db \
--param tenant_id=acme --param api_token="$TOKEN"
faucet template delete tenant-sync --store sqlite:./faucet-templates.db --version 1

--store accepts sqlite:<path>, a postgres://… URL, or memory (process-lifetime only, for a smoke test) — the same grammar as catalog.url and faucet serve --history, and it can be set once via FAUCET_TEMPLATE_STORE. SQL stores need the matching serve-history-sqlite / serve-history-postgres build feature.

faucet template run materializes the template and then runs it through the identical path as faucet run — observability, lineage, notifications, the catalog, SLA evaluation and row selection all behave the same.

Every register appends a new numeric version, auto-incrementing from 1 — and that is all it does. Registering never moves existing callers. A nightly build, a feature branch, a half-tested experiment: they all land as a new version while everyone who did not pin one keeps running exactly what they ran yesterday.

Making a version live is a separate, deliberate step: launch.

Terminal window
faucet template register tenant-sync.yaml # v4 exists; nobody is affected
faucet template launch tenant-sync # v4 is live — this moves callers

That split is the whole point. A deploy can register freely; promoting a build to “what production runs” stays a decision somebody makes on purpose.

A template is in exactly one of three states, and the state is derived from what has actually happened, so it can never disagree with the registry:

Status Meaning
draft Registered but never launched — the work-in-progress state. An unpinned run is refused (there is no blessed version); explicit selectors still work, so a draft is fully testable.
launched A version has been launched. Unpinned runs resolve to it.
deprecated Explicitly retired. Unpinned runs still work — retiring must not hard-break callers — but every trigger warns and listings mark it. delete is the hard stop.
Terminal window
faucet template register tenant-sync.yaml --launch # skip the draft stage
faucet template deprecate tenant-sync --reason "superseded by tenant-sync-v2"
faucet template deprecate tenant-sync --undo # revive it

A single bad build can be retired without deleting it, which would erase its history and break anything pinned to it:

Terminal window
faucet template deprecate tenant-sync --version 5 --reason "drops the invoices stream"
faucet template deprecate tenant-sync --version 5 --undo # revive it

A retired version:

  • still runs when pinned (--version 5), or when stable or a channel points at it, and every such trigger carries a deprecated warning ("v5 is deprecated: drops the invoices stream");
  • is skipped by newest, which resolves to the highest version that is not retired (when every version is retired, newest says so instead);
  • cannot be launched. Revive it first, or launch another version.

Deprecating the live version does not move stable; roll back separately. Deleting a version clears its marker. Over HTTP it is POST /v1/templates/{id}/versions/{version}/deprecate with {"reason":"…"} or {"undo":true}, and GET /v1/templates/{id} lists deprecated_versions. A sync from a Template Hub catalog retires a version here when the catalog deprecates the body it holds.

On top of the numbers sit named channels: pointers at one numeric version. Three are derived — computed from the launch log, never assigned:

Derived channel Resolves to
stable The launched version. The default when no version is given. Moves only via launch.
previous The version launched before the current one — the rollback target. Unset until a second launch.
newest The highest version number, launched or not. The build tip.

The rest are assignable — you point them wherever you like with promote:

Assignable channel Meaning
dev Day-to-day development
test QA / integration testing
staging Staging
pre-prod Pre-production / release-candidate soak
canary Partial-traffic canary ahead of prod
prod Production

The set is closed on purpose: an open-ended tag namespace becomes a second, unreviewable naming system in which a typo (prd) silently creates a channel nobody watches. An unknown name is rejected with the valid list. Names are forgiving about spelling — pre-prod, pre_prod, PreProd, and preprod are one channel — and if you need a free-form label, put it in the run’s labels, not in the registry.

Selector Resolves to
(omitted) stable — the launched version
a channel name (prod, newest, previous, …) whatever that channel points at
a number (2) exactly that version
Terminal window
faucet template run tenant-sync --param tenant_id=acme # stable
faucet template run tenant-sync --version prod --param … # whatever prod names
faucet template run tenant-sync --version newest --param … # the build tip
faucet template run tenant-sync --version 2 --param … # pinned

Asking for an unset channel is an error phrased for that channel, because the fix differs: stable needs a launch, previous needs a second launch, an environment channel needs a promote. Silently falling back would run the wrong code.

Terminal window
# Register v5 and point `dev` at it in one step.
faucet template register tenant-sync.yaml --tag dev
# Walk it up the channels — each promote copies another channel's current target.
faucet template promote tenant-sync --tag test --version dev
faucet template promote tenant-sync --tag pre-prod --version test
faucet template promote tenant-sync --tag prod --version pre-prod
# → template 'tenant-sync': prod → v5
# Bless whatever soaked in pre-prod as the new stable.
faucet template launch tenant-sync --version pre-prod
# → template 'tenant-sync': launched v5 (was v4; previous → v4)

launch defaults to newest, since launching what you just registered is the common case. Re-launching the already-live version is a no-op — which is what keeps previous a real rollback target rather than a copy of the current version. A promote from a channel resolves to a concrete version at that moment, so a pointer never silently follows future registrations. Derived channels cannot be assigned: faucet template promote … --tag stable is rejected and tells you to use launch.

Terminal window
faucet template rollback tenant-sync
# → template 'tenant-sync': rolled back to v4 (was v5; previous → v5)

Rollback re-launches previous, and it is an ordinary launch under the hood — so the launch log keeps the full audit trail and previous becomes the version you just rolled off (roll back twice and you are where you started).

Terminal window
curl -sX POST localhost:8080/v1/templates/tenant-sync/launch \
-H "Authorization: Bearer $TOKEN" -d '{"version":"pre-prod"}'
# → 200 {"id":"tenant-sync","version":5,"replaced":4,"already_launched":false,"status":"launched"}
curl -sX POST localhost:8080/v1/templates/tenant-sync/rollback \
-H "Authorization: Bearer $TOKEN" -d '{}'

faucet template list shows one row per id — its status, what is live, and the build tip:

ID STATUS LIVE NEWEST PARAMS DESCRIPTION
orders-export launched v2 v2 4 Nightly export of an orders table.

faucet template show reports one version in the context of the whole release state — every version with the channels pointing at it, and who launched what:

template orders-export [launched]
name orders-export
about Nightly export of an orders table.
created 2026-08-07T14:34:05Z
showing v2 (live)
versions:
v2 live, newest, dev, staging
v1 previous
launch history (newest first):
v2 2026-08-07T14:34:05Z
v1 2026-08-07T14:34:05Z

A description describes the template, so it carries forward: re-registering without --description keeps the previous one rather than blanking the listing.

GET /v1/templates/{id} returns the same picture as JSON, so a client can pin, promote, launch, or roll back without a second call:

{
"id": "tenant-sync", "version": 2, // the version returned
"status": "launched", // draft | launched | deprecated
"versions": [3, 2, 1], // everything stored, newest first
"stable": 2, // the launched version (unpinned runs)
"previous": 1, // the rollback target
"newest": 3, // the build tip
"is_stable": true, // the returned version is the live one
"tags": { "dev": 3, "prod": 1 }, // assignable channels only
"launches": [ { "seq": 2, "version": 2, "launched_at": "…", "launched_by": "ci" } ],
"body": "version: 1\nname: tenant-sync\n…"
}

Pass ?version=newest to open a draft template — it has no stable version yet.

A suite whose template: is a source template names the sink it should be tested against: sink: (a registered id, or a path when template: is a path) and optionally sink_select:. Every case then materializes the composed pipeline — the same document a trigger builds — and auto: cases sweep the merged parameter surface, so a sink param the source never declared is still covered. overlay: (and overlay_select:) adds a deployment overlay to that composition — a registered id, or a path in a file-based suite — and its params join the surface too.

version: 1
template: acme/billing
sink: faucet-hq/bigquery
suite:
auto: { enum_coverage: true }
cases:
- name: prod-shape
params: { bq_project: analytics, api_token: placeholder }

The web console (serve-ui) has a Templates view built around exactly this: a list showing each template’s status, live version, and build tip, and a per-template versions page with one row per version, the channels currently pointing at it, an assign-channel dropdown, and Launch / Config / Deprecate / Delete (a retired row shows a deprecated pill and cannot be launched) — plus Roll back, Deprecate for the whole template, and a typed trigger form generated from the template’s params:. The controls follow the signed-in role: a viewer sees no buttons that change anything, an operator can run templates but not manage them.

version accepts a channel name ("prod"), a numeric string ("2"), or a bare number (2), so a query string, a JSON body, and an MCP tool argument all mean the same thing. 0, latest, and unknown names are rejected rather than silently falling back.

Deleting: --version <N|channel> removes one version; faucet template delete <id> with no --version removes the template entirely. Channels pointing at a deleted version — and its launch-log entries — are dropped with it, so no pointer outlives its target. Runs already produced are untouched.

The 20 most recent versions of each id are kept, so a template re-registered on every deploy keeps a useful rollback window without growing without bound.

Practical pattern. Let deploys register --tag dev freely — nothing moves. Walk a version up the channels (dev → test → pre-prod → prod) as it earns trust. Bless it with launch when it should become what unpinned callers get, and keep rollback one command away. Point scheduled jobs at a channel (--version prod) when they must be pinned to a specific promotion train, and leave everything else unpinned so launch is your single release lever.

A template fans out across a parameter space, and a change to one version can silently break one corner of it — a param that no longer interpolates, a value that yields an invalid config, a required param whose failure stopped being clean. faucet template test turns that sweep into a red/green artifact:

Terminal window
faucet template test suite.yaml # template: is a path
faucet template test suite.yaml --store sqlite:./faucet-templates.db --select prod
version: 1
# A registered id, or — as here — a path, so a template can be tested *before*
# it is ever registered, which is when these failures are cheapest to fix.
template: ./tenant-sync.yaml
suite:
# Derived from the template's own `params:`, so these stay correct as the
# template gains params instead of going stale like a hand-written list.
auto:
enum_coverage: true # one case per declared value of every `values:` param
required_omitted: true # one per required param, omitted, expecting a named failure
defaults_baseline: true # the all-defaults combination
cases:
- name: eu-small-pages
params: { tenant_id: acme, api_token: t0ken, region: eu, page_size: 50 }
# `error:` implies the case must fail *and* that the message mentions the
# substring — a real assertion rather than "it failed somehow", which would
# also pass on an unrelated break.
- name: rejects-an-undeclared-region
params: { tenant_id: acme, api_token: t0ken, region: antarctica }
expect: { error: region }
combine:
params:
region: [us, eu, apac]
page_size: [1, 500]
exclude:
- { region: apac, page_size: 1 } # a genuinely-invalid pairing
pairwise: false # all-pairs instead of the full product
# Optional second tier: fixture records through the real pipeline, using
# `faucet test`'s matchers.
behavioral:
- name: shapes-a-record
params: { tenant_id: acme, api_token: t0ken }
input: [{ "Id": "1", "Name": "Acme" }]
expect: { records_written: 1 }
template ./tenant-sync.yaml
ok [explicit] eu-small-pages
ok [explicit] rejects-an-undeclared-region
ok [combine] page_size=1,region=us
…
ok [auto] auto:missing-api_token
ok [auto] auto:missing-tenant_id
13 case(s): 13 passed, 0 failed

Two tiers, both offline. The default validation tier materializes the template for a combination exactly as a real trigger would, then expands it, deserializes each row’s connector configs into their typed structs and compiles each row’s transform chain — the same checks faucet validate runs, so a param whose type the connector rejects (an int used as a whole query_params value) fails the case (in topology mode it validates the graph instead — skipping that would let a broken graph pass). No network, no data, no sink, which is what makes it cheap enough to run on every change. The behavioural tier feeds fixture records through the real pipeline via the faucet test harness, reusing its matchers rather than reimplementing them.

Case origins. Every case is labelled explicit, combine, auto, or behavioral in the report, because a red case nobody wrote is otherwise a mystery. Required params a combine: sweep does not name are filled automatically, so generated cases test the axes you listed and nothing else.

A secret param’s default never resolves in a test. A secret: true param a case leaves unset gets a typed placeholder when it has a default, so a default such as ${env:API_TOKEN} never pulls a real credential into a test run; one without a default stays missing and fails by name, as a real trigger would. A value a case names itself (a dummy like t0ken) is used, kept out of logs, and shown as *** in the report and in recorded failures.

Guard rails.

  • A cartesian product explodes quietly, so generation stops at 512 cases with an error rather than a truncation — a report covering a third of the space would read green.
  • Set pairwise: true to reduce to an all-pairs set: most param-interaction bugs involve two params, so this keeps the coverage that matters while turning a multiplicative count into roughly the product of the two largest lists.
  • An exclude: entry naming no swept param is rejected — it can never match, and it silently widens the tested space rather than narrowing it.
  • An empty suite is rejected. A suite with no cases reports green, which is worse than no suite.
  • A sweep whose full product passes 32,768 combinations is refused before it is built (narrow it or use pairwise: true, which is generated without the product).
  • A behavioural case on a config with several rows (a multi-stream source template, a matrix:) must name one with row:; fixture paths resolve against the suite file’s directory.
  • Duplicate case names are rejected (they make --filter ambiguous).

--filter '<pattern>' runs a subset (* wildcards; a bare name is an exact match, so --filter auto does not match auto:defaults). --json emits the machine-readable report. The exit code is the failed-case count, mirroring faucet test, so CI gates on it without parsing output. faucet schema template-test prints the suite schema.

A suite file lives next to a template; a test bundle lives inside it. Add a tests: block to any template document — kind: pipeline, source-template, sink-template or deployment — and the tests are stored verbatim with the version, covered by its content hash. Changing a test is a new version, exactly like changing the config, so “v3 passed its tests” always means v3’s tests.

# cli/examples/csv_to_jsonl_with_tests.yaml (abridged)
kind: pipeline
version: 1
name: orders_export
params:
region: { type: string, default: us, values: [us, eu] }
pipeline: { … }
tests:
suite: # the `faucet template test` grammar above
auto: { enum_coverage: true, defaults_baseline: true }
cases:
- { name: unknown-region-is-refused, params: { region: apac }, expect: { error: region } }
fixtures: # `faucet test` cases, run through this template
- name: rows-are-stamped
params: { region: eu }
input: [{ id: 1 }, { id: 2 }]
expect: { records: [{ id: 1, exported: true }, { id: 2, exported: true }] }
- name: inline-filter # or an inline `pipeline:` instead of the template's own
retries: 1 # explicit and recorded — never a silent retry
pipeline: { transforms: [{ type: filter, config: { path: status, op: eq, value: paid } }] }
input: [{ id: 1, status: paid }, { id: 2, status: open }]
expect: { records_written: 1 }
requires_suites: # shared suites, picked by semver range
- { name: rest-conformance, version: ">=1.2,<2" }
Field Meaning
suite Parameter-space cases (cases, combine, auto, behavioral), as in a suite file.
fixtures faucet test cases. Without pipeline: a fixture runs the template’s own materialized config (params: bind it, row: picks a row); with pipeline: it runs that inline logic. When no case touches the template — only inline-pipeline: fixtures, here and in the shared suites — the run adds an implicit implicit:auto:defaults case that materializes it with its defaults, expands it and compiles its transforms.
requires_suites Shared kind: test-suite documents, by name and semver version range over their release:.
sink / source / overlay (+ *_select) The companions a hub kind composes with: a source template names its sink (and optionally an overlay); a sink template names its source; a deployment overlay names both. A pipeline takes none. sink_select / source_select / overlay_select pick the companion’s version — a number or a channel (newest, prod, …); sink: files@newest is shorthand for sink: files + sink_select: newest (give the version once). Without a selector the companion’s launched (stable) version is used, or — while it has none, as for a pair that is new — its newest, which the recorded result notes (companion_notes). From a hub directory the selector is ignored: the hub holds one file per template.
retries (on any case) Extra attempts, at most 5. A case that passed on a retry says so (attempts).

Everything runs offline and repeatably: records are inline (a fixture path is refused — the version stores only the document), no connector is ever opened, every case runs at a fixed clock (2000-01-01T00:00:00Z unless the fixture sets clock:), and a secret: true param only ever gets a placeholder — its default is never resolved, and a case that names a value for one fails, saying so. A flaky test is a bug in the test; retries must be written into the case. Each case has a 60-second limit, a run a 10-minute budget, and a tests: block may be at most 1 MiB; hitting a limit records a failure with the reason.

A deployment overlay’s bundle is structural: naming its source and sink is enough, and the run composes the three and checks the result validates and expands. Fixture cases are optional.

Terminal window
faucet template test cli/examples/csv_to_jsonl_with_tests.yaml # a file — no registry
faucet template register orders.yaml --store sqlite:./faucet-templates.db # runs + records the bundle
faucet template test orders_export@3 --store sqlite:./faucet-templates.db # run + record on v3
faucet template test orders_export@3 --no-record --filter 'auto:*' --store … # a subset, not recorded

Registering a version runs its bundle. Every registered run is recorded on the version as evidence: a result id, per-case pass/fail, attempts and duration, the sha256 of the body it tested, the full faucet version, the connectors and transforms compiled in, any plugin connectors, the shared-suite releases and companion templates it ran with, the server instance (when a server ran it) and who asked. The newest 20 runs per version are kept; deleting a version (or the template) deletes its results. No API accepts a result — evidence comes only from a test run. Over HTTP: POST /v1/templates/{id}/versions/{version}/test and GET …/versions/{version}/tests; over MCP: test_template.

Each half of a new pair names the other in its tests:, and a launch needs passing tests — so neither could be tested against a launched companion. That is why a companion with no launched version is tested at its newest version: register both, then launch both, with no selector anywhere.

# billing.yaml # files.yaml
kind: source-template kind: sink-template
name: billing name: files
# … # …
tests: tests:
sink: files source: billing
suite: suite:
auto: { defaults_baseline: true } auto: { defaults_baseline: true }
Terminal window
curl -X POST "$FAUCET/v1/templates" -d '{"config": "<billing.yaml>"}' # tests: red — `files` is not registered yet
curl -X POST "$FAUCET/v1/templates" -d '{"config": "<files.yaml>"}' # tests: green (against billing v1, its newest)
# and billing v1 is re-tested: green
curl -X POST "$FAUCET/v1/templates/billing/launch" -d '{"version": 1}'
curl -X POST "$FAUCET/v1/templates/files/launch" -d '{"version": 1}'

The first registration’s result is red only because its companion did not exist; it says so (missing_companions, and an error naming the sink-template 'files' and the re-test command). Registering the companion re-runs those tests — the response lists them under companions_retested — so the second launch needs no manual step. To re-run by hand: faucet template test billing@1 or POST /v1/templates/billing/versions/1/test. Pin a companion explicitly (sink_select: 3, or sink: files@prod) once both are launched and the bundle should follow a specific build; a selector that names nothing (an unlaunched stable, an unset channel) fails the run, naming the companion and the fix.

A suite many templates share — a conformance suite for every REST source, say — is its own document, registered (or kept in a hub’s test-suites/ directory) and versioned by a semver release:

kind: test-suite
name: rest-conformance
release: 1.4.0
suite:
auto: { defaults_baseline: true, required_omitted: true }
fixtures: [ … ]

A requires_suites range resolves to the highest registered, non-deprecated release in range when the requiring version is registered, and that release is copied into the version (tests.locked_suites, written by the registry), like a lockfile. A later release, a retired release or a deleted suite never changes what an existing version tests. A release is immutable — registering the same release: twice is refused. A range nothing satisfies fails registration of the requiring template with a typed error naming the suite, the range and the releases that exist. A test-suite is never runnable on its own. faucet schema test-suite / faucet schema template-tests print the two schemas.

Every label — stable (launch), previous as a rollback target and every environment channel including dev and test — only moves onto a version whose status is passing:

Status Means Can take a label
passing Its tests ran and passed for this exact body under the running faucet major.minor yes
failing Its newest such result failed no
untested It has tests but no result for this body under this faucet major.minor after its tests run and pass
no_tests No tests: block, an empty one, or one that generates no cases no
unknown Its tests: block or its recorded results cannot be read no

One check covers every way a label moves — launch, rollback, promote, a register with launch / tags, template sync, approved change requests and MCP — and there is no override for anyone. Moving a label onto an untested version runs its tests first, records the result, and proceeds only if they pass; a failing version is never re-run automatically. A refusal is a 422 whose error.details.tests lists the status and failing cases, and is audited (template.launch, result refused). Each move records the result it relied on (evidence on the launch log and on each channel), re-read in the same transaction as the move, so a result recorded in between can never be skipped.

Status gates labels, never registration: a failing version is still registered, so its author sees the failing cases. Status is derived from the recorded results, never stored, and labels never move by themselves — a failing re-run, a faucet upgrade or a deleted suite only blocks future moves onto that version.

newest is derived, not moved. Before the first launch it is the highest version that is not deprecated. After it, newest is sticky: the highest non-deprecated version whose exact body has ever had a passing result, under any faucet release. A later failing run, a failed revalidation or a stale release line never moves it; only a newer version passing, or deprecating or deleting the holder, does. A version marked unknown never holds it. The newest holder is protected from version pruning like any label holder.

Drafts. A template with no launch in its history is a draft: its versions may have tests or not, passing or failing, and any of them runs when pinned. Deleting the whole template deletes its evidence, so registering it again starts a new draft.

After the first launch, every new version has tests. A version registered without a tests: block inherits the newest earlier version’s block, copied into the new version (tests.inherited_from records where from; the response carries tests_inherited_from). With nothing to inherit, the registration is refused with a message saying to add a tests: block. A tests: block that generates no cases (say auto: { enum_coverage: true } on a template with no values: params) tests nothing, so it is refused too, and the version is not kept. Registration always runs the version’s bundle. inherited_from and locked_suites are written by the registry; a submitted document that carries either is refused.

A run that resolves through a label — stable, a channel, or newest — always goes ahead, even after a faucet upgrade has made that version’s evidence stale: the label was justified when it moved. After the first launch, a run of a pinned version that holds no label and is not passing is refused unless the request sets allow_untested: true (--allow-untested on faucet template run, the allow_untested argument of MCP run_template, allow_untested on a trigger’s template:). Such a run is labelled allow_untested=true and audited (template.run, detail naming the status). It needs only the run permission. allow_untested never affects labels, and a tenant-scoped principal or a run made for a tenant can never use it (403, audited). A trigger that sets it is logged when the triggers file loads.

A composed run checks each part — source, sink and overlay — through its own label.

Evidence counts only under the faucet major.minor that produced it. After an upgrade the labels stay where they are and keep authorizing runs, but moving one needs fresh evidence. faucet template revalidate --store … re-runs the tests of every version that holds a label (stable, newest, each channel) or is a rollback target (previous), records the results, and exits with the number that no longer pass; it never moves a label. faucet serve does the same in the background when it starts on a release line newer than the one last revalidated under (kept in the history database). The marker is claimed with a compare-and-set, so of several instances starting on the new line only one revalidates, and an instance on an older line never claims it back; when a version could not be tested at all (error) the claim is released and the next start tries again. Metrics: faucet_serve_template_tests_total{outcome}, faucet_serve_template_label_refusals_total{reason}, faucet_serve_template_revalidations_total{outcome}.

A server runs the tests of registered versions only, which it already trusts to run, so they get the same trust as the template’s own runs: a subprocess connector (singer) is allowed, as it is when the template runs. Every document a test executes is still screened for the server’s own credentials (FAUCET_VAULT_KEY, the serve tokens, the auth-config sources) before any ${env:} / ${file:} resolves; a template that reads them fails its tests with the reason, so it can never take a label there. Each result records the server instance that produced it.

The console shows each template’s stable test status in the list, each version’s status and recent runs on the versions page, a Run tests button, and disables Launch and the channel selector on a version that cannot take a label.

A template with rows — the streams: of a source template, the matrix: of a pipeline template — does not have to run all of them. Backfill one stream, re-run the one that failed, run the cheap streams hourly and the heavy one nightly, or let an embedding product’s tenant pick which objects to sync, all from one registered template. Every trigger path takes the same selection, the model faucet run already uses (--select / --only / --skip / --tag / --status / --include-parents):

{ "selection": { "select": ["deals"], "only": ["act*"], "skip": ["audit"],
"tags": ["finance"], "status": ["available"],
"include_parents": "eligible" } }
surface how
CLI faucet template run crm --sink files --select deals --include-parents eligible
HTTP selection in the body of POST /v1/templates/{id}/runs, POST /v1/runs, POST /v1/tenants/{t}/runs, POST /v1/tenants/{t}/templates/{id}/runs and POST /v1/templates/{id}/fanout
MCP selection on run_template, run_pipeline and propose_run
triggers run: { selection: {…} } on a schedule / webhook / object-arrival / queue-depth trigger (reference)
console the Streams (or Rows) checkboxes on the trigger form

What runs is resolved the same way everywhere: status gate (mandatory / active, widened by status) → narrowed by tags, or picked by id with select / only → ancestors per include_parents → skip removed last. Omit selection and every row runs; an empty object applies only the status gate. For a source template the composition then includes only the selected streams — so a stream the chosen sink cannot run (say, one that needs upsert on a file sink) no longer blocks the others.

Parents and dependencies. A child stream (parent:) or a row with depends_on: needs its ancestors in the run:

include_parents a selected row whose ancestor is not selected
off (default) refused before anything runs, naming every missing ancestor
eligible runnable (mandatory / active) ancestors are pulled in; a parked (available / draft / archived) one is refused
all every ancestor is pulled in (parked ones with a warning)

Ordering rows from the request. A source template’s stream (like a matrix row) can declare depends_on: [other_stream] — it starts only after that stream completes, with no record hand-off. A caller who knows of an order the template does not encode adds it to the trigger instead:

Terminal window
curl -sX POST localhost:8080/v1/templates/crm/runs -H "Authorization: Bearer $TOKEN" \
-H 'content-type: application/json' \
-d '{"sink":"files","depends_on":{"deals":["accounts"]}}'

The request’s edges are added to the template’s own (never removing one), checked like them — every name must be a stream or row of the run (a for_each stream by its rendered name), and the combined parent: + depends_on: graph must stay acyclic, else a 400 naming the edge — and applied before the selection resolves, so a prerequisite the selection leaves out follows include_parents exactly as in the table above. An edge on a stream the selection leaves out is dropped with it. A failed prerequisite skips its dependents under the run’s on_error. The run record keeps the edges (depends_on), the run is labelled depends_on=deals:accounts, and they are part of the idempotency fingerprint and of a change request’s plan. POST /v1/runs takes the same field for a config’s matrix rows.

What a selection does not change. State keys are per row ({pipeline}::{row}), so running a subset never touches another row’s bookmark. The run is labelled selection=<canonical form> (for example select=deal_lines;include_parents=eligible), which is also part of the idempotency fingerprint — the same key with a different subset is a 409, not a replay. The selection is stored with the run, so a clustered, claimed or sharded run executes the same subset on whichever instance picks it up, and a change request fingerprints it: a different subset is a different change. A topology pipeline (pipeline.nodes) has no rows, so any selection there is refused.

Errors are 400s naming the problem — an unknown row or tag (with the valid ones listed), an empty run set, a missing ancestor under include_parents: off.

Selection is only usable if callers can see the valid names, so every registered template lists its rows:

Terminal window
curl -s "localhost:8080/v1/templates/crm/rows?sink=files" -H "Authorization: Bearer $TOKEN"
faucet template rows crm --sink files --store sqlite:./faucet-templates.db
faucet hub rows example-csv --sink jsonl # a catalog template, unregistered

Each row carries, where faucet can derive it:

group fields
identity id, kind (stream / row), description, status, tags, default_selected
hierarchy parent, parent_key, children, depends_on, depth, per_parent_record
write write.requested; with a sink write.resolved, write.supported, write.offered, write.alias_applied, write.unsupported_reason; primary_keys, delete_marker
read read.source_kind, read.replication (full / incremental + field), read.resumable, read.shardable, read.supports_discover
guarantees (with a sink) guarantees.delivery_guarantee, guarantees.cleanup_capable
shape shape.schema (from the contract, schema_source: contract), shape.transforms (count, kinds, whether any rename or drop fields)
inputs params_used
state state.health, last_success, last_failure, last_error, bookmark_age_secs, lag — from faucet status, when the state store is readable

Rows are expanded with placeholder-bound params, so a required param never blocks a listing; the state group needs the config’s defaults to name its store (a source template’s comes from the sink pairing and an overlay). A topology template returns rows: [] with selectable: false.

Previewing a selection. Pass the selection as query parameters (or flags) and the listing becomes a dry run: each row gains selected, pulled_in (with because), blocked (with the reason) or excluded, and the report gains run_set in execution order and the error a trigger would return.

Terminal window
curl -s "localhost:8080/v1/templates/crm/rows?select=deal_lines&include_parents=eligible"
# → {…,"run_set":["deals","deal_lines"],
# "rows":[…,{"id":"deals","pulled_in":{"because":"deal_lines"},…},
# {"id":"deal_lines","selected":true,…}]}
faucet template rows crm --select deal_lines --store sqlite:./faucet-templates.db
# ✗ deals … blocked: required by `deal_lines` (parent) but not selected — include_parents is off; …

The MCP list_template_rows tool returns the same report.

Point faucet serve --history at the same store and the same templates become triggerable over HTTP and MCP — one registry, not three:

Terminal window
faucet serve --history sqlite:./faucet-templates.db --auth-token "$TOKEN"
Terminal window
# Register (admin / TemplateAdmin)
curl -sX POST localhost:8080/v1/templates \
-H "Authorization: Bearer $TOKEN" \
-d '{"id":"tenant-sync","config":"'"$(sed 's/"/\\"/g;:a;N;$!ba;s/\n/\\n/g' tenant-sync.yaml)"'"}'
# Browse (viewer+ / TemplateRead)
curl -s localhost:8080/v1/templates -H "Authorization: Bearer $TOKEN"
curl -s localhost:8080/v1/templates/tenant-sync -H "Authorization: Bearer $TOKEN"
# Trigger (operator+ / RunWrite)
curl -sX POST localhost:8080/v1/templates/tenant-sync/runs \
-H "Authorization: Bearer $TOKEN" \
-d '{"params":{"tenant_id":"acme","api_token":"'"$TOKEN"'"},"env":{"API_HOST":"eu.example.com"}}'
# → 202 {"run_id":"…","status":"queued","template_id":"tenant-sync",
# "template_version":1,"params":{"tenant_id":"acme","api_token":"***",…}}
Terminal window
# A source template names its sink; params are the union of both halves.
curl -sX POST localhost:8080/v1/templates/acme%2Fbilling/runs \
-H "Authorization: Bearer $TOKEN" \
-d '{"sink":"faucet-hq/bigquery","sink_version":"stable","params":{"api_token":"…","bq_project":"my-project"}}'
# → 202 {…,"template_id":"acme/billing","template_version":1,"sink_template":"faucet-hq/bigquery",
# "sink_template_version":1,"streams":[{"stream":"bills","chosen":"overwrite",…},…]}
curl -s "localhost:8080/v1/templates?kind=source-template" -H "Authorization: Bearer $TOKEN"

A trigger is submitted through the same path as POST /v1/runs, so idempotency keys, doctor_first, queue limits, cluster dispatch, metrics, and the audit log all behave identically. The run is labelled template and template_version (and sink_template / sink_template_version for a composed run), so GET /v1/runs?… and your dashboards can group by provenance.

See the HTTP API reference for the full endpoint list.

With --mcp, an agent gets list_templates / get_template read-only, plus register_template / run_template behind --mcp-allow-mutations and the caller’s RunWrite scope:

Terminal window
faucet serve --history sqlite:./faucet-templates.db --mcp --mcp-allow-mutations
# or, over stdio:
faucet mcp --template-store sqlite:./faucet-templates.db --allow-mutations

Without a store the template tools are not advertised at all, so an agent never sees a tool it cannot use. run_template takes the same sink / sink_version pair as the HTTP trigger for a source template (and overlay / overlay_version for a deployment overlay), and its dry_run output carries the per-stream write-mode plan and what the overlay set. list_template_rows lists a template’s rows (and previews a selection); run_template and run_pipeline take a selection to run a subset.

The config body is stored verbatim. ${env:…} / ${vault:…} / ${secret:…} stay unresolved tokens and are resolved at trigger time, on the instance that runs the pipeline — the same privilege surface as any normally-submitted config. That is the recommended way to get a credential into a template: reference it from the body, don’t pass it as a param.

A caller-supplied secret: true param value is never persisted. It lives only for the duration of one trigger: bound into the materialized config, registered for redaction, and echoed back as "***".

One consequence is worth stating plainly: a clustered server persists the materialized config so a peer can execute the run, which would put a secret param value in the shared history database. A clustered server started with a vault key (--vault-key / FAUCET_VAULT_KEY, the same on every member) seals those values instead: the stored config carries ${sealed:N} tokens, the values are stored encrypted and bound to the run id, opened only by the member that runs it, and deleted once the run finishes (see clustered execution). Without a vault key, a clustered trigger that supplies a value for a secret: true param is refused with a 422 naming the key and the safe alternatives (reference the secret from the body, or trigger on a non-clustered server). A trigger that needs an approved change request is refused either way. A secret param left at a default: "${env:VAR}" is not refused: the reference stays unresolved in the persisted body and resolves on the executing instance. Non-clustered servers store no config body and are unaffected.

Hosting templates in a repo or bucket (sync)

Section titled “Hosting templates in a repo or bucket (sync)”

(requires the templates-sync build feature; S3 / GCS / Azure Blob origins additionally need templates-sync-object-store)

The registry is where templates run from; a repo or bucket is where they are authored — reviewed in a pull request, or dropped in a partner’s bucket. A sync file names those origins, and faucet serve --templates-sync pulls them into the registry: on start, on demand, and on a per-origin interval.

# sync.yaml — see cli/examples/templates/sync.yaml
version: 1
origins:
- name: platform
source:
type: github
config: { repo: acme/data-templates, ref: main, path: templates/, token: "${env:GITHUB_TOKEN}" }
prefix: platform- # this origin owns every `platform-*` id
launch: follow # a sidecar's `launch: true` moves `stable`
prune: deprecate # removed upstream → deprecated here (never deleted)
interval_secs: 300
- name: partner
source:
type: s3
config: { bucket: acme-templates, prefix: shared/, region: us-east-1 }
prefix: partner-

A template under an <owner>/ directory of an origin (a Template Hub catalog’s layout) is its owner’s: it registers only when it reads nothing of the server’s (${env:}, ${file:}, secret directives), references no other part of the composed config (${sources.*}, ${sinks.*}, ${vars.*}) and names no connector that runs a program — otherwise the pull reports and skips it. List the owners (or owner/name ids, or "*") you have reviewed under the origin’s trust:. The faucet-hq namespace of faucet-hq/template-hub at main needs no entry.

Terminal window
faucet serve --history sqlite:./faucet.db --templates-sync sync.yaml
faucet template sync --store sqlite:./faucet.db --config sync.yaml --dry-run # plan only
faucet template sync --store sqlite:./faucet.db --config sync.yaml --origin platform
faucet template publish platform-nightly --store sqlite:./faucet.db --config sync.yaml --origin platform

Over HTTP the same two verbs are POST /v1/templates/sync ({origin?, dry_run?}) and POST /v1/templates/{id}/publish ({origin, version?}), both TemplateAdmin (admin) and audited as template.sync / template.publish; the console’s Templates page grows a Sync from origins panel when the server has any.

Layout at an origin. The template id is the file stem, with the origin’s prefix prepended: templates/nightly.yaml under prefix: platform- registers as platform-nightly. Only *.yaml / *.yml / *.json files directly in the directory — plus one level of owner directories, whose files register as owner/name — are read, on GitHub and object-store origins alike; a GitHub origin may name several directories with paths: [source-templates, sink-templates] instead of one path (a Template Hub catalog — stems must be unique across them, and publish writes each template to the directory named for its kind: source-templates, sink-templates, deployments; a pipeline goes to the first). A GitHub pull resolves ref to one commit and reads every file at that commit, so a push during the pull cannot mix two revisions; a directory past the contents API’s 1000-entry limit is read from the git tree. An optional sidecar <stem>.faucet.yaml beside the template carries release intent, kept out of the config body so the body stays runnable with faucet run:

nightly.faucet.yaml
description: Nightly account sync
launch: true # make this version `stable` on pull (honoured under `launch: follow`)
tags: [staging] # assignable channels to point at the pulled version

What a pull does — and never does.

Upstream state Registry action
New file register a version (launched only if the policy says so)
Body changed (comments/whitespace ignored — the canonical body is hashed) register the next version
Body unchanged nothing — re-pulling is free and never inflates the version counter
Unchanged, but stable lags the policy (always, or a sidecar flipped launch) and that version was never launched launch the existing version
Unchanged, but an operator rolled back off it (the launch log already holds it) or the version is retired nothing — a pull never undoes a rollback; a new body is a new version and is launched as usual
File removed prune: keep → reported as orphaned; prune: deprecate → deprecated
Removed file returns under prune: deprecate the deprecation the origin placed is lifted; a deprecation a person placed is left alone, and nothing is launched into that template until they lift it
Bad id / unparseable body / bad or unreadable sidecar / ambiguous stem / a file that could not be read this pull skipped and reported — one broken file never blocks the origin, and it still counts as upstream, so prune: deprecate never retires it

A pull only appends: nothing is overwritten and nothing is deleted (a delete would cascade to the launch log and silently repoint stable). Under the default launch: ignore a pull moves nobody — exactly like a manual register; follow lets the sidecar decide; always is GitOps mode, where merging upstream is the release. Every register is attributed (created_by: sync:<origin>, or the principal who called the HTTP endpoint). Pulls of one origin are serialized in a process, and a body another pull registered since the plan was made is not registered again, so a rollback always has a different version to land on.

${env:…} / ${file:…} / ${secret:…} in the sync file resolve inside each value after it is parsed — an environment value can never add keys. A value that is exactly one directive may resolve to a number (interval_secs: ${env:SYNC_EVERY}).

One owner per template. Each origin owns the id namespace named by its prefix; two origins with overlapping prefixes (including an empty prefix beside any other) are refused when the file loads, so “who wins” never has to be decided at runtime, and an origin never touches ids outside its prefix.

Publish is manual. faucet template publish <id> --origin X [--version stable] writes one registered version back as <id minus prefix>.<yaml|json> — a deliberate operator step (audited), never automatic, so the registry can never overwrite a reviewed file on its own. The next pull sees the identical body and plans unchanged.

Credentials. GitHub uses the contents API — no git binary; a private repo needs only a token (use ${env:…} / ${secret:…}; the value is registered for log redaction). Object-store origins use the SDK default chain (AWS_*, Application Default Credentials, AZURE_*), like the trigger watchers. A transport failure on the initial pull is logged and counted, not fatal — the server comes up on the registry it has; an invalid sync file is fatal.

Metrics: faucet_serve_template_sync_runs_total{origin,outcome=ok|partial|error}, faucet_serve_template_sync_mutations_total{origin}, faucet_serve_template_sync_last_unix_seconds{origin}. Schema: faucet schema templates-sync.

  • Structure safety. Params are substituted per JSON/YAML scalar, before the typed parse — a value containing :, a newline, or - stays the single scalar it replaced and can never inject a key or an array element. That is a guarantee about the config’s structure, not about the text a param lands in: params are not bound as SQL parameters or URL-encoded (record fan-out tokens are). A caller-supplied free-text value spliced into a longer string is checked instead — see Declaring parameters — and for a value a lower-trust caller fills into a query, a closed values: set or an int / float type is the reliable guard.
  • No re-interpolation of caller input. Env/file/secret directives resolve before params bind, so a supplied value is never itself scanned for directives. A supplied value containing ${ is rejected outright: params are data, not directives.
  • Caller values meet the caller policy. On a server, a caller-supplied param value that reaches a path, URL or command is checked like a submitted config (see Params that reach paths and URLs); trusted: true opts a param out.
  • Typos fail loudly. An undeclared --param, an undeclared ${param.x} reference, a missing required param, or a type mismatch is an error naming the param — never a silent no-op.
  • Nothing leaks by accident. ${param.*} binding happens pre-parse on every load path, and a token that somehow survived to matrix expansion is rejected there as a backstop rather than reaching a connector as literal text.
Feature Enables
(none) The params: block, ${param.*}, --param / --param-env, faucet schema params
templates faucet template …, /v1/templates*, the MCP template tools (implies serve)
templates-sync faucet template sync|publish, faucet serve --templates-sync, POST /v1/templates/sync + /{id}/publish, GitHub origins (implies templates)
templates-sync-object-store S3 / GCS / Azure Blob origins (implies templates-sync; pulls object_store)
serve-history-sqlite / serve-history-postgres A registry that survives a restart

templates is in --features full, not in default.