Governance in flight The guardrails live in the pipe.
Everything that usually needs a second platform runs inside the movement path,
with zero per-connector code — and a masked field never reaches the sink in the
clear. Each guardrail is a config block; the engine applies them in order,
per page, before every write.
source mask → quality → contract → drift sink
PII masking
Redact, hash, tokenize or partially mask sensitive fields, matched by name, pattern or value detector. Runs first, before quality, contract, drift, the sink, the DLQ and lineage, so a masked field never leaves unmasked. Keyed hashing (HMAC-SHA256) stays deterministic and joinable.
Docs → masking:
key: ${env:MASKING_KEY} # keyed HMAC-SHA256
rules:
- name: emails
match: { fields: [email] }
action: { type: hash }
- name: ssn
match: { fields: [ssn] }
action: { type: redact }
Data-quality checks
Assert invariants on every page, per record or per batch: not-null, regex, ranges, sets, uniqueness, row counts, JSON Schema. Failures abort the run or quarantine to a dead-letter queue.
Docs → quality:
record:
- type: not_null
field: email
on_failure: abort
- type: compare
field: amount
op: gte
value: 0
on_failure: quarantine
Data contracts
A versioned promise about the output shape (types, nullability, enums, patterns, bounds) enforced before the sink. Export it as JSON Schema or OpenLineage with faucet contract.
Docs → contract:
version: "1"
on_breach: quarantine
fields:
- { name: id, type: integer }
- { name: status, type: string,
enum: [active, churned] }
Schema-drift policy
Detect when the source shape changes and decide what happens (warn, evolve, ignore, quarantine or fail) instead of silently corrupting downstream.
Docs → schema:
on_drift: quarantine
# warn | evolve | ignore
# | quarantine | fail
Data-flow policies
Classify columns and declare which sinks each label may reach, such as PII only to EU sinks. validate, plan and doctor report violations before any data moves, and run refuses them.
Docs → policy:
version: 1
classifications:
- label: pii
fields: [email, phone]
rules:
- name: pii-stays-in-eu
when: { label: pii }
require: { residency: [eu] }
Column-level lineage
Emit OpenLineage events with dataset schemas and column-level lineage derived from the transform chain, to Marquez or any OpenLineage endpoint, plus a movement catalog.
Docs → lineage:
namespace: prod.warehouse
include_schema_facet: true
include_column_lineage: true
transport:
type: http
config:
url: ${env:MARQUEZ_URL}
Change impact analysis
Before a change ships, faucet plan --impact walks the catalog lineage graph and lists the datasets, contracts, owners and consumers a dropped or retyped column would break, with a severity for each.
Docs → catalog:
url: sqlite:./faucet-catalog.db
# then, before merging a change:
# faucet plan orders.yaml --impact
Freshness & volume SLAs
Declare what healthy looks like: max staleness, minimum rows, or a learned volume baseline (±3σ). Violations emit a Prometheus counter you can alert on.
Docs → sla:
max_staleness_secs: 86400
min_rows_per_run: 1
volume_anomaly:
method: zscore
sensitivity: 3.0
Column profiling
Learn a per-column profile (null rate, type mix, distinct counts, distributions) every run and report statistically significant drift, with no thresholds to write.
Docs → profiling:
min_history: 5 # runs before detection starts
window: 20 # rolling baseline
on_drift: warn # warn | notify | fail
Plan → approve → run
Make runs and template changes wait for named approvers. faucet re-plans before it executes, refuses a change whose plan moved underneath it, and audits every step.
Docs → # auth.yaml (faucet serve --auth-config)
approvals:
rules:
- kinds: [run]
roles: [operator, admin]
min_approvers: 1
self_approve: false
# faucet serve --auth-config auth.yaml \
# --require-approval run
All of it is opt-in config on the same pipeline — no extra services, no
downstream bolt-ons. See the full guide
for every field and policy.
Get started Your first pipeline runs in five minutes.
Install the CLI, scaffold a config, and move real data — nothing external to stand up.