TILLARK · ARCHITECTURE

Own the control plane, not the bytes.

PREVIEW

TillArk splits into a control plane that owns policy, the manifest, and the failover machine, and an agent that moves data inside your environment. Engines encrypt and move bytes; TillArk owns policy, verification, and orchestration — and never sees plaintext backup data.

01Components

Control plane, agent, engines, targets

text

┌──────────────────────────────────────────────────────────────────┐
│  CONTROL PLANE  (apps/tillark — stateless, multi-tenant, org-scoped)│
│                                                                    │
│   • Policy engine        schedule · retention · RPO/RTO SLOs       │
│   • Anti-affinity        residency-aware, distinct-failure-domain  │
│   • Manifest / catalog   the source of truth (crown-jewel IP)      │
│   • State machine        failover / failback (§ Failover)          │
│   • Orchestrator         restore + restore-verification            │
│                                                                    │
│   NEVER in the data path · holds NO plaintext backup data          │
└───────────────┬──────────────────────────────────────────────────┘
                │  mTLS + hybrid-signed commands  ·  no secret transits
    ┌───────────▼────────────┐
    │  ark-agent (your env)   │   caches policy → can act if CP is down
    │  claim → run → sign →   │   (TillArk is never a SPOF for recovery)
    │  report                 │
    └───┬───────────┬─────────┘
        │           │
 ┌──────▼─────┐ ┌───▼──────────────────┐
 │ Engines     │ │ Storage targets       │
 │ restic      │ │ S3 · R2 · B2 · Wasabi │
 │ WAL-G       │ │ GCS · Azure · MinIO   │
 │ ch-backup   │ │ SFTP  — cross-vendor, │
 │ (argv-only) │ │ WORM / object-lock    │
 └────────────┘ └──────────────────────┘

Component	Responsibility
Control plane	Stateless API + workers. Owns policy, manifest/catalog, the state machine, and orchestration. Never in the data path; holds no plaintext backup data.
Agent	Runs in your environment. Drives engines, streams ciphertext to targets, executes failover, reports signed manifests. Caches its policy so it can act if the control plane is unreachable. See Agent.
Engine adapters	Thin, audited wrappers over restic / WAL-G / clickhouse-backup. Argv arrays only — never a shell string.
Manifest / catalog	The tillark_* tables — what was backed up, where, its hash + signature, and its verification history. TillArk’s source of truth.
Verification service	Scheduled ephemeral restores that prove recoverability and emit green/red into TillPulse.

02Front doors

Two front doors, one catalog

The same tillark_* catalog is written through two authenticated front doors — the same pattern the rest of the suite uses. A third, agent-facing door handles the machine loop (register, heartbeat, claim a job, report a result).

text

                     ┌──────────────────────────┐
                     │   schema.tillark_*        │   one catalog,
                     │   (manifest / policy /    │   org_id on every row
                     │    backups / failovers…)  │
                     └───▲───────────────▲──────┘
      machine / agent    │               │   human / session
   ┌─────────────────────┴──┐        ┌───┴──────────────────────┐
   │ apps/tillark            │        │ /api/ark/* (dashboard)    │
   │ /api/ark/v1/*           │        │ + tilldev ark … CLI       │
   │ tark_ bearer, hash-only │        │ session / tp_ API key     │
   │ read < operator < admin │        │ read=member, write=admin  │
   └─────────────────────────┘        └──────────────────────────┘
        (+ agent door: register · heartbeat · claim · report)

Machine / agent API (apps/tillark, REST under /api/ark/v1/*, OpenAPI 3.1 at /api/ark/v1/openapi.json). Auth is a tark_ bearer token, stored sha256-hash-only, on a three-tier scope ladder read < operator < admin plus a can_approve capability.
Human dashboard + CLI (/api/ark/* in the web app, and tilldev ark). Auth is your session; reads are open to any org member, writes need owner/admin. The finer operator tier collapses into admin on the human door.

One envelope, either door

Both doors envelope-encrypt credentials with the same crypto-agility envelope under the same per-org DEK, so either front door can decrypt what the other wrote. Every mutation on either door is org-scoped server-side and appended to the audit trail.

03Manifest

The manifest / catalog

Every backup produces a manifest — the engine snapshot id, content hashes, the WAL range, a monotonic consistency_epoch, and the source’s schema / migration hash. The manifest is hybrid-signed (Ed25519 + ML-DSA-65), and a restore verifies the signature before trusting a single byte. This is the anti-tamper spine — swapping ciphertext in the bucket can’t forge the manifest, and the consistency_epoch defeats a silent roll-back to an older (e.g. pre-patch) backup.

A shared consistency_epoch also tags backups taken in the same window across Postgres / ClickHouse / object storage, so a multi-store restore is mutually consistent — no dangling reference from a Postgres row to a missing object. More in Security.

04Durability

Three-tier cross-vendor durability

RPO is a measured, displayed number per database, not a hope. Durability is layered so each tier defends a different failure domain — and the tiers are explicit so a same-region sync replica dying with the primary can never be mistaken for real redundancy.

text

  app writes
      │
      ▼
 ┌──────────────┐  quorum-sync (ANY 1 of N, RPO≈0)  ┌────────────────────┐
 │  Neon         │ ─────────────────────────────────▶│ Same-region replica │ Tier 1
 │  PRIMARY role │                                   │ (promotable)        │ node failure
 └──────┬───────┘                                   └────────────────────┘
        │ logical replication (async, bounded lag)
        ▼
 ┌────────────────────┐  continuous WAL   ┌─────────────────────────┐
 │ Cross-vendor replica│ ──── ship ──────▶ │ Cross-vendor WAL archive │ Tier 2+3
 │ (diff provider+rgn, │                   │ (object store, WORM,     │ vendor death,
 │  promotable)        │                   │  residency-ok)           │ PITR
 └────────────────────┘                   └─────────────────────────┘

 Switch:  DNS flip (default, out-of-path)  |  ark-managed proxy (enterprise)
 Fence:   Neon compute suspend + role-password rotate  (managed-DB STONITH)

Quorum commit (ANY 1 of N) gives zero-loss durability without letting one slow remote replica stall the primary. For a managed DB we don’t control (Neon), fencing is a Neon-native STONITH substitute: rotate the role password and suspend the compute endpoint via the provider API, so the old primary rejects writes even if it is still alive. The full state machine is on the Failover page.

05Anti-affinity

Residency-aware anti-affinity

A copy only counts as a copy if it is on an independent failure domain. TillArk’s resolver requires a different provider and region than the source — but the same legal residency zone — and it counts distinct failure domains, so two buckets on the same vendor are not two copies. Cross-border replication that would violate a residency rule is refused, not silently performed.

Anti-affinity rule	Default
`differentProvider`	true — target must not share the source’s provider
`differentRegion`	true — target must not share the source’s region
`residencyZone`	global — a specific zone (e.g. eu) rejects a global-only target
`minCopies`	1 — minimum distinct failure-domains that must hold a copy

06Engines

Wrapped, not built

TillArk does not build a backup engine, chunker, dedup index, or crypto primitive — that is person-years of work whose failure mode is silent data loss. It orchestrates proven engines and adds scheduling, retention, the manifest, and verification on top.

Engine	Used for	What it brings
`restic`	Files, object storage, generic datastores	Content-addressed dedup, native client-side encryption, incremental-forever, object-store native.
`WAL-G`	Postgres physical + PITR	Continuous WAL archiving, parallel, point-in-time recovery.
`clickhouse-backup`	ClickHouse	FREEZE-consistent snapshots to S3-compatible storage; the first engine-specific non-Postgres target.

The agent invokes each engine with an argv array — never a shell string, never string-interpolated user input — so command injection is structurally unreachable, and secrets travel in the environment, never on the command line. MySQL / Mongo / Redis follow by demand.

Next: Backups & restore-verification, Failover, or the Security model. Back to the TillArk overview.