tilldev ark — the full command surface.
TillArk lives inside the one tilldev CLI. Every group takes --json, secrets are read from a prompt or stdin (never bare on argv), and the dangerous verbs (restore, failover promote, failback and reseed) demand a four-eyes approval.
Command groups
tilldev ark <group> <verb> [flags]
sources register / list the resources you protect
targets register / list cross-vendor storage destinations
agents approve the agents that run your jobs
policies schedule + retention + RPO/RTO + verify schedule
backups list the catalog · queue a run
verify non-destructive restore-verification
restore real, four-eyes-gated restore INTO a source
restores restore + verification history
runs verify or restore a run's stores together
wal how far point-in-time recovery reaches
canary ransomware canary events
clone masked clones of a backup
failover status · promote · failback · finish · reseed · auto · autonomy
replicas list replicas across the org
tokens tark_ control-plane tokens (hash-only, shown once)
audit the hash-chained audit log · verify it on this machine
export · plan · apply every policy as one file: plan a change, apply it whole
--json everywhere · secrets via prompt / --*-stdin (never on argv)The CLI talks to the dashboard API (/api/ark/*) over HTTPS only, the same front door the web dashboard uses. Reads work for any org member; writes need owner/admin; dangerous verbs additionally need the approval flags below. With a tark_ token, operator may add sources, targets and policies and queue runs; changing, pausing or deleting them needs admin.
Sources — what you protect
tilldev ark sources ls
tilldev ark sources create --name <n> --type <postgres|mysql|mongo|clickhouse|meili|redis|object> \
--provider <neon|rds|supabase|self|s3|r2|…> \
[--region <r>] [--residency <zone>] [--engine <restic|walg|pgbackrest|chbackup>] \
[--unmanaged] [--with-dsn] [--byok <fingerprint>]
tilldev ark sources get <id>
tilldev ark sources update <id> [--with-dsn] [--engine <e> | --default-engine] \
[--managed | --unmanaged] [--pause | --resume] \
[--byok <fingerprint> | --no-byok] # customer key; see Security
tilldev ark sources rm <id> [--destroy-backups]
# PostgreSQL columns that name objects in an object source, checked when a run comes back:
tilldev ark sources refs <source>
tilldev ark sources refs <source> add --name <n> --column <[schema.]table.column> \
--to <object source> [--key <k>] [--strip <prefix>]
tilldev ark sources refs <source> rm <name>| Flag | Meaning |
|---|---|
--type | postgres · mysql · mongo · clickhouse · meili · redis · object |
--provider | Where the source lives (neon, rds, supabase, self, or an object-store provider). |
--residency | Legal residency zone (default global). A policy for a zone only places backups on targets in that zone. |
--engine | restic (every type, the default); walg or pgbackrest (self-managed PostgreSQL: base backups plus continuous WAL, for point-in-time recovery), where pgbackrest also writes to SFTP targets; or chbackup (self-managed ClickHouse: MergeTree table parts copied with clickhouse-backup). Changing it moves queued backups to the new engine and fails any bound for a target it can’t write to; backups already taken keep theirs. |
--unmanaged | Mark a self-managed source (default is managed). |
--with-dsn | Prompt for the connection string (hidden); stored envelope-encrypted, never returned. |
--pause / --resume | A paused source queues no runs. |
sources rm refuses while the source has backups. With --destroy-backups it destroys the repository keys, so every backup of that source becomes unreadable, including copies under object-lock. You type the source name to confirm; without a terminal, add --yes.Targets — where backups land
tilldev ark targets ls
tilldev ark targets create --name <n> --provider <s3|r2|b2|wasabi|gcs|azure|minio|sftp> \
--bucket <b> [--endpoint <url>] [--region <r>] [--residency <zone>] \
[--lock-days <n>] [--credential-domain <d>] [--private] [--with-creds]
tilldev ark targets get <id>
tilldev ark targets update <id> [--endpoint <url> | --default-endpoint] [--lock-days <n> | --mutable] \
[--credential-domain <d>] [--private | --public] [--with-creds] [--pause | --resume]
tilldev ark targets rm <id>| Flag | Meaning |
|---|---|
--provider | s3 · r2 · b2 · wasabi · gcs · azure · minio · sftp |
--lock-days | Require object lock for at least this many days (s3, b2, wasabi, minio). The agent reads the bucket’s lock before every write and refuses to write if it is shorter or off. |
--mutable | On update: drop the object-lock requirement. |
--credential-domain | A name for the account that holds this bucket’s keys (default: default). A policy with --separate-credentials puts each copy under a different one. Change it on update when you move the bucket’s keys. |
--endpoint | An https origin. Required for r2, b2 and minio; optional for s3, wasabi and gcs. A private address or internal name works; its jobs go only to agents that list it in TILLARK_PRIVATE_TARGETS. Loopback, link-local and cloud metadata addresses are refused. |
--private / --public | Send the target’s jobs only to agents that list its host in TILLARK_PRIVATE_TARGETS, for a public-looking name that resolves to a private address. Private addresses and names outside public DNS are private anyway. |
--with-creds | Prompt for each credential field (secrets hidden). In CI, pipe a JSON object to --creds-json-stdin. Stored envelope-encrypted, never returned. |
| Provider | Credential fields |
|---|---|
| s3 · r2 · b2 · wasabi · minio | access_key_id, secret_access_key, session_token (optional) |
| gcs | access_key_id, secret_access_key (an HMAC key) |
| azure | account_name, account_key |
| sftp | host, port (optional), username, host_key, private_key (PEM; the prompt asks for its file path) |
A target that holds backups can’t be deleted; pause it instead.
Agents — who runs your jobs
tilldev ark agents ls
tilldev ark agents approve <id> [--fingerprint <fp>] # prompts when the flag is left off
tilldev ark agents revoke <id>
tilldev ark agents reset-cert <id> # after rebuilding its host
tilldev ark cp-key # the key agents take work fromAn agent receives job credentials only after you approve its key. Run ark-agent fingerprint on the agent host and pass what it prints. TillArk shows the fingerprint only after approval, so the value you type must come from the host. Approval needs a signed-in owner or admin, not an API key. A rebuilt host has new keys and must be approved again.
Each agent’s client certificate is pinned when it registers. A rebuilt host presents a new one, so reset the pin with reset-cert (signed-in owner or admin) before starting it. cp-key prints the key TillArk signs agent work with; set it as TILLARK_CP_KEY on a new host to pin it from the first start.
Policies — schedule, retention, SLOs
tilldev ark policies ls [--source <id>] # CRON (UTC), NEXT run, schedule and placement problems
tilldev ark policies get <id>
tilldev ark policies create --source <id> \
[--name <n>] [--cron "<expr>"] [--rpo <sec>] [--rto <sec>] [--verify-cron "<expr>"] \
[--keep-last <n>] [--keep-hourly <n>] [--keep-daily <n>] \
[--keep-weekly <n>] [--keep-monthly <n>] [--keep-yearly <n>] \
[--min-copies <n>] [--residency <zone>] [--allow-same-provider] [--allow-same-region] \
[--separate-credentials] [--pause-on-alarm] \
[--verify-spread <min>] [--verify-older-every <n>] [--verify-read-parts <n>] \
[--incremental <off|differential|incremental>] [--full-every <days>]
tilldev ark policies update <id> [--cron "<expr>"] [--verify-cron "<expr>" | --no-verify] \
[--rpo <sec>] [--rto <sec>] [--enable | --disable] [--name <n>] [--keep-* <n>] \
[--min-copies <n>] [--residency <zone>] [--allow-same-provider | --different-provider] \
[--allow-same-region | --different-region] [--separate-credentials | --shared-credentials] \
[--pause-on-alarm | --no-pause-on-alarm] \
[--verify-spread <min>] [--verify-older-every <n>] [--verify-read-parts <n>] \
[--incremental <off|differential|incremental>] [--full-every <days>]
tilldev ark policies rm <id> # the policy's backups stay--cron sets the backup schedule and --verify-cron the restore-verification cadence, both cron in UTC; a schedule that can never fire is rejected. The --keep-* flags define GFS retention; an all-zero retention is rejected. On update, only the flags you pass change: a single --keep-* leaves the other buckets as they are. See Schedules and upkeep for missed slots and schedule errors.
--verify-spread, --verify-older-every and --verify-read-parts set how scheduled verification covers everything you keep: the minutes after its slot a verification starts within (default 60), how often one proves an older backup instead of the newest (every 4th run), and how many parts a restic repository’s data is read in, one per run (30). See Verification at scale.
--incremental has the base backups of a WAL-G or pgBackRest source build on earlier ones: differential stores what changed since the last full backup, incremental what changed since the backup before it, and off (the default) takes a full one every time. --full-every takes a full one at least that many days apart (default 7). See Incremental and differential base backups.
--rpo is the recovery point objective in seconds: how much recent data a restore may lose. Left out, it defaults to the schedule’s longest gap plus time for the backup to run (300 for a source that streams WAL). An objective the schedule can’t meet is refused, and 0 sets none. ark sources ls shows each source’s recovery point against it. --rto is the restore time objective, judged against how long the latest restore took, verifications included.
| Placement flag | Meaning |
|---|---|
--min-copies | Copies per backup, 1 to 16 (default 1). Each lands on a different provider or region. |
--allow-same-provider | Copies may sit on the source’s own provider. By default they may not; --different-provider restores that. |
--allow-same-region | Copies may sit in the source’s own region. By default they may not; --different-region restores that. |
--residency | Copies only go to targets in this residency zone; global (the default) allows any. |
--separate-credentials | No two copies of a backup share a target credential domain, so one leaked storage key reaches one copy at most. --shared-credentials turns it off. |
When the active targets can’t hold as many copies as the rule asks, backups still run with the copies they can place, and the policy says so: in policies ls and get, on the Policies page, and as placement_problem in the API. On update, placement flags you leave out keep their values.
--pause-on-alarm pauses the source when its ransomware canary raises an alarm, so no more backups of encrypted data are taken. The alarmed backup is quarantined either way, with every backup built on it.
To keep every policy in one file next to your code, use a policy file. <file> may be - for stdin, YAML or JSON, and a problem is shown at its line and column. -o on plan saves the plan for apply to apply exactly as reviewed; --detailed-exitcode exits 2 when there are changes and 0 when there are none. Without a saved plan, apply shows the plan and asks, or needs --yes when it can’t ask.
tilldev ark export [-o <file>] [--force] # every policy as a policy file (YAML)
tilldev ark plan <file> [--prune] [-o <plan.json>] [--detailed-exitcode]
# what the file would change; changes nothing
tilldev ark apply <file | plan.json> [--prune] [--yes]
# every change or none, each in the audit logBackups, verify, restore
tilldev ark backups ls [--source <id>] [--epoch <n>] # --epoch: one run across sources; why any failed, under the table
# KIND: what each backup builds on, and why one ran full
tilldev ark backups run --source <id> [--policy <id>] # one run per target
# Restore-verification (non-destructive) + history:
tilldev ark verify <backup-id>
tilldev ark restores [--backup <id>] # PROVED: which backup, and the part of its repository read
# A real, destructive restore is four-eyes gated and destination-allowlisted:
tilldev ark restore <backup-id> --into <source-id> \
--approved-by <second-approver-uuid> --approval-ref <ticket>
# Point in time (walg or pgbackrest sources): the newest base backup before --at, plus WAL.
tilldev ark restore --source <id> --at 2026-09-28T14:05:00Z --into <source-id> \
--approved-by <second-approver-uuid> --approval-ref <ticket>
tilldev ark wal [--source <id>] # each archive's engine, how far it reaches, any gaps
# A run's stores together, with their references checked:
tilldev ark runs [--limit <n>] [--before <run>]
tilldev ark runs show <run> # consistent or why not, and the point
tilldev ark runs verify <run>
tilldev ark runs restore <run> --into <source>=<destination>[,…] \
--approved-by <second-approver-uuid> --approval-ref <ticket>ark restore is destructive. It requires --into (an allowlisted registered source in your org), plus --approved-by (a different owner/admin’s user id) and --approval-ref. The server re-verifies the approver. ark verify is non-destructive and needs none of this.--at takes an ISO 8601 time; give its zone (Z or +02:00), or it is read as this machine’s local time. With a backup id, --at recovers that WAL-G or pgBackRest backup forward to the moment. See point-in-time recovery for what it needs.
ark runs restore is gated the same way, with one destination per store in --into, by name or id. Stores you leave out aren’t restored. See runs for what each store comes back to and how references are checked.
Failover & replicas
tilldev ark failover status --source <id>
tilldev ark failover promote --source <id> [--allow-data-loss] \
--approved-by <uuid> --approval-ref <ticket>
tilldev ark failover failback --source <id> \
--approved-by <uuid> --approval-ref <ticket>
tilldev ark failover finish --source <id>
tilldev ark failover repoint --source <id> # admin: DNS records back at the primary
tilldev ark failover reseed --source <id> --node <provider:region> [--discard-writes] \
--approved-by <uuid> --approval-ref <ticket>
tilldev ark failover auto --source <id> [--arm|--disarm] [--quorum <n>] [--confirm-window <sec>] \
[--autonomy|--no-autonomy] [--approved-by <uuid> --approval-ref <ticket>]
tilldev ark failover autonomy --source <id> # in force or not, the grant, each agent's report
tilldev ark failover autonomy release --source <id> \
--approved-by <uuid> --approval-ref <ticket> # the grant's agent is gone for good
tilldev ark replicaspromote, failback and reseed are four-eyes gated. promote refuses a replica whose lag isn’t measured and in bounds unless you pass --allow-data-loss; failback has the agent promote the old primary back once it replicates from the new one and has caught up, and finish keeps the new primary instead. reseed rebuilds a replica or old primary from the current primary, replacing its data; --discard-writes confirms an old primary’s unreplicated writes may be deleted. auto shows or arms automatic failover; arming and loosening need an approver. status shows the current failover plus each replica’s lag, whether it can be promoted or reseeded, recent reseeds, synchronous replication, the three durability tiers, and where the source’s DNS records point when the agent moves clients. repoint asks the agent to point records that drifted outside a failover back at the node taking writes. The full model is on the Failover page.
Ransomware canary & masked clones
# Ransomware canary — entropy/dedup anomalies vs. each source's baseline:
tilldev ark canary # list open + past canary events
tilldev ark canary ack <event-id> # triage: acknowledged | confirm | dismiss
# Masked clones of a PostgreSQL backup, served by your own agent until they expire:
tilldev ark clone ls
tilldev ark clone create <backup-id> [--profile standard|strict|none] \
[--ttl <minutes>] [--name <n>]
tilldev ark clone show <clone-id> # where it runs, what was masked, what differs
tilldev ark clone connect <clone-id> [--cert <path>] # saves its certificate, prints a verify-full URL
tilldev ark clone expire <clone-id>
# A raw (unmasked, real-PII) clone is four-eyes gated, like a real restore:
tilldev ark clone create <backup-id> --profile none \
--approved-by <second-approver-uuid> --approval-ref <ticket>ark canary lists entropy/dedup anomalies the canary raised against each source’s baseline, with the sample-versus-baseline evidence; triage them with ack / confirm / dismiss. Dismissing a false positive releases the backup from quarantine, with the backups built on it, and tells you if the source is still paused. ark clone create asks an agent to build a masked clone of a PostgreSQL backup; show lists every masked column and what was done to it, and connect saves the clone’s certificate and prints its URL. A --profile none unmasked clone requires four-eyes approval.
Status, signals and alerts
tilldev ark alerts route --project <pulse project> \
(--slack <url> | --webhook <url> | --email <addr> | --pagerduty <key> | --teams <url>) \
[--kinds failover.*,backup.failed,…] [--sources <name|id>,…] \
[--min-severity critical|high|medium|low|info] [--quiet-minutes <n>] [--name <text>]
# Page on-call for failovers and missed recovery points:
tilldev ark alerts route --project mobile --pagerduty <routing-key> --kinds 'failover.*,rpo.*'
tilldev ark status [--stack <slug>] # each source against its objectives, open alarms, failover steps
tilldev ark alarms # alarms open now, worst first
tilldev ark signals [--source <name|id>] [--kinds <k>,…] [--min-severity <s>] \
[--since 6h|7d|<time>] [--before <time>] [--limit <n>] [--cursor <c>]ark alerts route makes a TillPulse alert rule that sends TillArk signals to the destinations you name, from high up unless you set --min-severity. --kinds takes kinds or families such as failover.*; --sources takes source names or ids. An alarm that clears goes wherever its opening went. List and switch off rules with tilldev pulse alerts rules.
ark status shows each source’s recovery point and last measured recovery time against its objectives, its restores of the last 30 days, under COVERED how many kept backups a verification has proven and how much of its repository’s data the reads cover, the replica a failover would promote, and any failover of the last 7 days, then the open alarms. --stack narrows it to one stack’s sources. ark signals lists what TillArk recorded, newest first, for up to 90 days; it pages for you up to --limit (1,000 at most) and prints a --cursor to carry on from.
Tokens and the audit trail
tilldev ark tokens ls
tilldev ark tokens create [--name <n>] [--scope read|operator|admin] [--can-approve] [--expires <days>]
tilldev ark tokens revoke <token-id>
tilldev ark audit [--limit <n>] [--before <entry>]
tilldev ark audit verify [--full] [--trust <fingerprint>]tokens create mints a tark_ control-plane token on a read < operator < admin scope, with an optional --can-approve capability and --expires. The secret is shown once and stored sha256-hash-only. These are the tokens the agent and machine clients authenticate with.
ark audit lists the audit log newest first, each entry with its number, and ends with the state of its hash chain; --before pages back. ark audit verify checks every entry and signed checkpoint on your machine and remembers what it verified, so later runs check only what is new and that nothing before it changed. It exits 1 when the log doesn’t verify. See Security for what it catches.
See the Quickstart for the commands in sequence, the Agent for the self-hosted runner, or the overview.