Prove your last N restores
all passed.

# The policy is the contract. Review it like code.
version: 1
policies:
  - source: orders-db
    schedule: "0 * * * *"
    rpo: 1h15m
    rto: 15m
    retention:
      hourly: 24
      daily: 30
      monthly: 12
    placement:
      residency: eu
      copies: 2
      separate_credentials: true
    verify:
      cadence: "0 4 * * *"

$ tilldev ark plan policies.yaml
Plan: 1 to change.
  ~ orders-db / default
      retention.daily                 7 → 30
      placement.residency             global → eu
      placement.copies                1 → 2
      placement.separate_credentials  false → true
      rto                             1h → 15m
      verify.cadence                  0 6 * * * → 0 4 * * *

$ tilldev ark apply policies.yaml

An owner or admin requests the failover, or arms automatic failover, and a different one approves it. One person, or one leaked token, can’t move your primary.

Automatic failover waits until a quorum of observers, one of them where your application runs, has not reached the primary for the whole confirm window. A partition or a blip is not an outage.

The old primary is fenced before promotion — on Neon its compute is suspended and its role password reset; on PostgreSQL you run, your application’s roles are locked out and it turns read-only — so your application can’t write to it even if it is still alive. Nothing is fenced unless a replica could be promoted, and if fencing fails, nothing is promoted.

A replica is promoted only with a lag measured in the last 5 minutes and at most 30 seconds, measured again just before promoting; in an outage, as it stood when the primary went down. Unknown lag is never treated as zero.

Opt in, with a second approver, and the replica’s agent can fail over on its own when TillArk itself is unreachable: on the same rules, under a grant TillArk signs with Ed25519 and ML-DSA-65, and rechecked from the observers’ signed reports once it reports back.

A cap per window and a cooldown after any failover stop ping-pong, and a failed failover stops automatic failover until someone re-arms it.

One failover at a time per source. Failback is a deliberate switchover once the old primary has caught up — never an automatic flip.

After fencing and before the promoted node takes writes, the agent points your Cloudflare DNS records at it and reads them back, or calls your signed webhook, which can refuse. If the promotion fails, clients are pointed back. Records that drift from the primary are flagged.

On PostgreSQL you run, see whether each replica holds every commit the primary acknowledged, and why not; let the agent keep synchronous replication set, on one standby or a quorum of several. When one of up to nine servers is promoted, the other standbys follow it, and it never inherits a wait for a standby that isn’t there.

Each source shows three durability tiers side by side: a synchronous replica, a replica on another provider that could take over, and a continuous WAL archive. When one lapses, an alarm opens; when it holds again, it closes.

restic, WAL-G and pgBackRest encrypt every backup, clickhouse-backup’s included, before upload — the storage vendor only ever sees ciphertext.

Hybrid-signed manifests (Ed25519 + ML-DSA-65) are checked before a single byte is trusted.

The agent confirms a target’s lock before every write; a locked backup can’t be deleted or overwritten inside its retention window.

A policy can keep its copies under different keys, so one leaked key can’t reach them all; under compliance-mode lock, no key can erase a backup early.

Every envelope names its suite, so primitives rotate without breaking old backups. Hybrid post-quantum ready.

Real restores, failover and unmasked clones need a second, distinct owner or admin to approve.