Take it, prove it, restore it.

How a backup run works

You queue a run (ark backups run or a policy schedule). The agent claims it atomically, runs the engine, encrypts client-side before any byte leaves your environment, streams ciphertext to the cross-vendor target, and reports a hybrid-signed manifest. The control plane records the manifest — never the data.

run ─▶ agent CLAIMS the job (atomic: pending→running)
        │
        ├─ engine snapshot   (restic / WAL-G / ch-backup, argv-only)
        ├─ encrypt CLIENT-SIDE   (AES-256-GCM, before bytes leave your env)
        ├─ stream ciphertext ─▶ cross-vendor target (WORM)
        └─ build + HYBRID-SIGN manifest  (Ed25519 + ML-DSA-65)
                │
                ▼
        report manifest ─▶ control plane records it   (NOT the bytes)
        status: succeeded · size · dedup ratio · consistency_epoch

Incremental-forever, dedup, and zstd compression come from the engine; TillArk adds scheduling, retention, the manifest, and a coordinated prune that never races a running backup (the classic dedup-GC corruption bug). A backup carries a status through pending → running → succeeded, then verifying → verified once a verification passes.

Restore-verification

On the cadence set in your policy, TillArk restores a real backup into an ephemeral, network-isolated, per-tenant sandbox, asserts integrity, records the observed RTO, emits green or red, and tears the sandbox down. It is non-destructive — nothing production is touched.

verify (scheduled or on demand)
   │
   ▼
 select freshest succeeded/verified backup   (never an in-flight one)
   │
   ▼
 provision EPHEMERAL, network-isolated, per-tenant sandbox
   │
   ▼
 VERIFY manifest signature ──✗──▶ red · executor never runs
   │ ✓
   ▼
 engine restore ─▶ assert integrity
     • row counts        • content checksum
     • schema hash       • manifest signature
   │
   ├─ empty check battery  ──▶  FAIL  (proved nothing ⇒ never silent green)
   │
   ▼
 record RTO + green/red ─▶ TillPulse   ·   promote backup → "verified"
   │
   ▼
 tear the sandbox down   (guaranteed — no residual data)

Run one by hand with tilldev ark verify <backup-id> and read the outcome with tilldev ark restores. The same green/red + RPO/RTO is a board in TillPulse.

Real restores are dangerous — and gated

A real restore overwrites a destination and, done carelessly, is a data-exfil vector (restore-to-arbitrary-target). So it carries the strongest gates in the product:

• Four-eyes approval. The requester must name a second, different owner/admin of the same org as approver, plus a non-empty approval_ref (a change ticket / incident id). The server re-verifies the approver against the org’s members — the client’s claim is never trusted, and a token cannot approve its own request.
• Allowlisted destination. You can only restore into a registered source in your own org — not an arbitrary URL.
• Verify before trust. The agent verifies the manifest signature — and reconciles the snapshot reference against the signed manifest — before the engine runs. A tampered manifest means the executor never runs, and the outcome is red.

# A REAL restore overwrites a destination, so it is four-eyes gated:
# a DIFFERENT owner/admin's user id + an approval reference, and the
# destination must be an ALLOWLISTED registered source (exfil guard).
tilldev ark restore <backup-id> \
  --into <destination-source-id> \
  --approved-by <second-approver-uuid> \
  --approval-ref INC-4821
# → authorized real restore <uuid> → source <dest> (pending)
#   the agent verifies the manifest signature before trusting a byte

The honest numbers

Per database, you choose and see the tradeoff; TillArk measures the live value and alarms on breach. Nothing here is a brochure number.

GFS retention

Retention is grandfather-father-son: keep the last N, plus a thinning tail of hourly / daily / weekly / monthly / yearly snapshots. Retention depth is a correctness property — it’s what lets you roll back past a logical corruption (a “time-bomb” a fresh backup would faithfully preserve).

# GFS defaults — override any bucket with a --keep-* flag on the policy.
tilldev ark policies create --source <id> \
  --keep-last 7 --keep-hourly 24 --keep-daily 7 \
  --keep-weekly 4 --keep-monthly 12 --keep-yearly 3