TILLARK · ARCHITECTURE

Own the control plane, not the bytes.

TillArk splits into a control plane that owns policy, placement, the catalog and verification, and an agent that moves data inside your environment. Proven engines capture, encrypt and store the bytes; your backups never pass through TillArk.

01Components

Control plane, agent, engines, targets

text
┌────────────────────────────────────────────────────────────────────┐
│  CONTROL PLANE  (multi-tenant; every row scoped to your org)        │
│                                                                    │
│   • Policies        UTC cron schedules · GFS retention · RPO/RTO    │
│   • Placement       residency-aware, distinct-failure-domain       │
│   • Catalog         every backup and its signed manifest           │
│   • Verification    scheduled restores that prove each backup      │
│   • Failover        the fenced-promotion state machine (preview)   │
│                                                                    │
│   never in the data path · backup bytes never reach it              │
└───────────────┬────────────────────────────────────────────────────┘
                │  HTTPS (optional mTLS) · tark_ token
                │  job credentials sealed to the agent's own key
    ┌───────────▼────────────┐
    │  ark-agent (your host)  │   approved by key fingerprint
    │  claim → run → sign →   │   before it receives any job
    │  report                 │
    └───┬───────────┬─────────┘
        │           │
 ┌──────▼───────┐ ┌─▼─────────────────────┐
 │ Engines       │ │ Storage targets        │
 │ restic        │ │ S3 · R2 · B2 · Wasabi  │
 │ WAL-G         │ │ GCS · Azure · MinIO    │
 │ pgBackRest    │ │ SFTP                   │
 │ clickhouse-   │ │                        │
 │   backup      │ │                        │
 │ + each store's│ │                        │
 │   own tools   │ │                        │
 └──────────────┘ └───────────────────────┘
ComponentResponsibility
Control planeOwns policies, placement, the catalog, verification scheduling and the failover state machine. Stores connection strings, storage credentials and repository keys only encrypted under your org’s key. Never in the data path.
AgentRuns on your host. Claims jobs, runs the engines, sends encrypted backups straight to your targets and reports a signed manifest for each. Receives only the jobs its host has the tools for. See Agent.
Enginesrestic for every source type, fed by each datastore’s own dump tools or protocol; WAL-G or pgBackRest for PostgreSQL point-in-time recovery; clickhouse-backup to copy ClickHouse table parts. Run from argument arrays, never a shell string.
CatalogEvery backup: where it is, its signed manifest, the run it belongs to and its verification history.
VerificationScheduled restores into a throwaway sandbox on the agent’s host that prove each backup restores, with the checks recorded.
02Front doors

Two front doors, one catalog

The catalog is written through two authenticated front doors. A third, agent-facing door handles the machine loop (register, heartbeat, claim a job, report a result).

text
                     ┌──────────────────────────┐
                     │   TillArk catalog         │   one catalog,
                     │   (sources / targets /    │   scoped to your org
                     │    policies / backups…)   │
                     └───▲───────────────▲──────┘
      machine / agent    │               │   human / session
   ┌─────────────────────┴──┐        ┌───┴──────────────────────┐
   │ agent / machine API     │        │ /api/ark/* (dashboard)    │
   │ /api/ark/v1/*           │        │ + tilldev ark … CLI       │
   │ tark_ bearer, hash-only │        │ session / tp_ API key     │
   │ read < operator < admin │        │ read=member, write=admin  │
   └─────────────────────────┘        └──────────────────────────┘
        (+ agent door: register · heartbeat · claim · report)
  • Machine / agent API (REST under /api/ark/v1/*, OpenAPI 3.1 at /api/ark/v1/openapi.json). Auth is a tark_ bearer token, stored sha256-hash-only, on a three-tier scope ladder read < operator < admin plus a can_approve capability.
  • Human dashboard + CLI (/api/ark/* in the web app, and tilldev ark). Auth is your session; reads are open to any org member, writes need owner/admin. There is no separate operator role here: operator actions need admin.
One envelope, either door
Both doors envelope-encrypt credentials with the same crypto-agility envelope under the same per-org key, so either door can read what the other wrote. Every mutation on either door is org-scoped server-side and appended to the audit trail.
03Manifest

The signed manifest

Every backup produces a manifest: the engine’s snapshot id, a content hash of what was captured, its size, the WAL range (WAL-G and pgBackRest), the schema hash (PostgreSQL), row and table counts for databases, the backup’s id, its run epoch and the moment it was signed. The agent that took the backup signs it with Ed25519 and ML-DSA-65.

  • On report, the control plane re-verifies both signatures against that agent’s registered keys, and checks that the manifest names the org, source, backup, epoch and snapshot it was reported for. A backup that fails either check is recorded but not marked signed.
  • Before any restore or verification touches a byte, the agent verifies the manifest again and refuses one that belongs to another backup or carries another epoch. Swapping bytes in the bucket, or an older backup’s manifest into a newer backup’s record, can’t pass.

More in Security.

04Runs

Runs: the stores backed up together

A backup’s consistency_epoch is its run’s window, in unix seconds, set when it is queued and signed into its manifest. Every policy that fires in the same scheduled slot gives its backups the same epoch, so the backups of your PostgreSQL, your object store and your search index taken together form one set. A source’s epochs only rise.

Open a run from any backup on the Backups page, or with tilldev ark runs or GET /api/ark/runs. A run is verified and restored as a set.

One moment, where each store allows it
A run is the set queued for one slot. Restoring it brings every store back as of the same moment where its backup allows it: PostgreSQL on WAL-G or pgBackRest, a snapshot of PostgreSQL, MySQL or a MongoDB replica set, and a versioned bucket can; stores read over time can’t, and the run says so. Declare the references from PostgreSQL columns to object stores, and TillArk reports every value that names a missing object whenever the run is verified or restored.
05Durability

Layers of protection

Each layer defends a different failure, and a copy only counts when it is somewhere the failure can’t reach.

LayerDefends againstProvided by
Backups on another vendor’s storageLosing the database, the account or the vendor; logical corruption you need to roll back pastTillArk backups, GFS retention and verification
Continuous WAL archiveLosing the writes made since the last backup (PostgreSQL)TillArk’s WAL-G and pgBackRest engines: point-in-time recovery
A replica in another region or providerThe hours a full restore takesYour replication. TillArk records the replica and runs the fenced promotion on Neon or on PostgreSQL you run (preview)
A synchronous replicaLosing even the last seconds of writesYour database’s synchronous replication
06Anti-affinity

Residency-aware anti-affinity

A copy only counts as a copy if it is on an independent failure domain. A policy’s rule can require a different provider and region than the source, within the same legal residency zone, and TillArk counts distinct failure domains, so two buckets in the same provider and region are not two copies. A target outside the residency zone is never used, and a policy that no target satisfies fails to queue with the reason. When the targets can hold fewer copies than minCopies, each backup makes the copies it can and the policy reports the shortfall as placement_problem.

Anti-affinity ruleDefault
differentProvidertrue — target must not share the source’s provider
differentRegiontrue — target must not share the source’s region
residencyZoneglobal — a specific zone (e.g. eu) rejects a global-only target
minCopies1 — minimum distinct failure-domains that must hold a copy
separateCredentialsfalse — when true, no two copies share a target credential domain

A target’s credential domain is a name you give the account that holds its keys. TillArk can’t see account boundaries, so it trusts the name: give two targets the same domain when one key or one person can delete both. With separateCredentials, placement finds the largest set of targets that differ in both failure domain and credential domain, so one leaked storage key reaches one copy at most. Object lock, not this rule, protects a copy from the key that wrote it.

An on-demand backup run without a policy applies no rule: it goes to the target already holding the source’s repository, or else the oldest active target.

07Engines

Wrapped, not built

TillArk does not build a backup engine, chunker, dedup index or crypto primitive, because a new one’s failure mode is silent data loss. It runs proven engines and adds scheduling, placement, retention, the manifest and verification on top.

EngineSourcesHow
resticEvery source type: PostgreSQL, MySQL / MariaDB, MongoDB, ClickHouse, Meilisearch, Redis / Valkey and filesThe agent captures each datastore with its own dump tools or protocol and streams the result into restic, which deduplicates, compresses and encrypts it (AES-256 with Poly1305) before it leaves the host. Files are read straight from disk.
WAL-GSelf-managed PostgreSQLPhysical base backups and a continuously archived WAL, for recovery to any moment. Runs on the database host; encrypted with libsodium.
pgBackRestSelf-managed PostgreSQLThe same base backups and continuous WAL archive, on object storage or an SFTP server. Runs on the database host; encrypted with AES-256-CBC, and every file and WAL segment is checked against its SHA-1 before it is used.
clickhouse-backupSelf-managed ClickHouseCopies MergeTree tables’ data parts from a snapshot of the server’s disks instead of reading every row; other tables travel as rows. Runs on the database host, and the result is stored with restic, encrypted the same way. Verification restores the parts in a throwaway ClickHouse server and runs CHECK TABLE on every table.

Each repository’s encryption key is minted by TillArk, stored under your org’s key, and sealed to the agent that runs the job. The tools each source type needs on the agent’s host are listed on the Agent page. Engines run from an argument array, never a shell string, and secrets travel in the environment or in a per-job file only the agent’s user can read, removed when the job ends; never on the command line.