TILLARK · MASKED CLONES

Last night's data, safely on a dev box.

Teams need realistic data to build and test against, and reach for a copy of production: that is how real customer data ends up in staging, on laptops and in screenshots. TillArk already holds a recent, verified backup, so it can give you a masked clone instead: the same schema and volume, with personal data replaced, gone when you are done.

01How it works

Built and served on your own host

A clone is built by one of your agents, from a logical backup of a PostgreSQL source. The agent masks rows as it copies them into a new PostgreSQL server on its host, then serves that server until the clone expires. The rows never pass through TillArk; it keeps only the clone’s connection details, encrypted under your organisation’s key.

text
tilldev ark clone create <backup-id> --profile standard --ttl 240
        │
        ├─▶ an agent that hosts clones takes the job
        ├─▶ checks the backup's signed manifest and restores it into a private sandbox
        ├─▶ reads the schema, samples values and plans every column's masking
        ├─▶ creates a new PostgreSQL server and copies each row through the masker
        │      (unmasked rows never reach the clone; the sandbox is then deleted)
        ├─▶ checks the result: no sampled original survives, keys still join
        └─▶ serves the clone over TLS with its own certificate
              └─ tilldev ark clone connect <clone-id>
                 deleted when it expires

A clone never writes to one of your sources. Only a succeeded or verified backup can be cloned, and not one the ransomware canary has quarantined.

02Setup

Set up a clone host

Pick an agent host with the PostgreSQL server installed, 12 or newer and at least as new as the sources you clone, and set:

bash
TILLARK_CLONE_LISTEN=0.0.0.0          # the address clones listen on
TILLARK_CLONE_HOST=clones.internal    # the name your team connects to

Restart the agent. TillArk → Agents then shows it as hosting clones, and clones can be created. Open the clone port range (default 55400-55499) to the machines that use clones and no further. The agent guide covers the ports, how many clones a host serves and where they are kept.

03Masking

The right change for each column

Each column is classified by its name and by its values: a column whose sampled values are almost all email addresses is treated as email, whatever it is called. Then:

Column holdsBecomesWhy
email · phone · namea realistic stand-inThe same value always becomes the same stand-in, so lookups and joins still work. Emails stay unique.
national ID · M-Pesa referencea keyed pseudonymKeeps the format (digits stay digits, the length holds) and never maps two values to one, so unique and foreign keys hold.
card · password · tokenremovedNever fabricated: the value becomes a placeholder such as [card], and a password or token becomes NULL where the column allows it.
address · birth date · locationremovedReplaced by a placeholder; under the strict profile, NULL where the column allows it.
free text · JSONscrubbedEmails, phone numbers, IDs, cards and tokens inside the text are replaced using the SDK’s on-device scrub rules. JSON keeps its structure.
everything elseunchangedRow IDs, amounts, timestamps and flags stay as they are, so the data stays realistic.

Columns joined by a foreign key are masked alike, including case-insensitive ones, so a join returns the rows it did in production. The masking key is random for each clone and never stored, so no one can reverse a clone’s values or match them against another clone’s.

text
standard   realistic stand-ins; free text scrubbed               ← the default
strict     also removes national IDs, addresses, birth dates and IP addresses
none       no masking: REAL data                                  ← admin and four-eyes only
04Connect

Connect over verified TLS

When a clone is ready, Connect on its card in TillArk → Clones, or the CLI, gives you its credentials and its server certificate:

bash
$ tilldev ark clone connect 0f3c9a2e-…
✓ Saved the clone's certificate to /home/you/tillark-clone-0f3c9a2e.crt
postgresql://clone:…@clones.internal:55412/ledger?sslmode=verify-full&sslrootcert=/home/you/tillark-clone-0f3c9a2e.crt

$ psql "postgresql://clone:…@clones.internal:55412/ledger?sslmode=verify-full&sslrootcert=…"
  • The URL uses sslmode=verify-full with the saved certificate, so it connects only to that clone. The clone refuses connections without TLS.
  • The clone’s user owns its database and can change it freely, but is not a superuser.
  • Every connect is recorded in the audit log as clone.connect. The credentials of an unmasked clone need admin scope.
05Fidelity

What a clone keeps

Tables and partitions, rows, indexes, primary, unique and foreign keys, constraints, triggers, views, materialized views (refreshed from the masked rows), sequences at their positions, row-level security policies, and the extensions the host has. Where a clone must differ, it says so: its card and tilldev ark clone show list each difference, for example:

  • an extension the clone host does not have;
  • large objects, which are not copied because they can’t be masked;
  • a check constraint the masked rows no longer meet, kept for new rows only.

show also lists every masked column and what was done to it, so a reviewer can confirm what the clone holds.

06Guardrails

Expiring, scoped and audited

  • Expiring. Every clone has a lifetime, 1 hour by default and at most 24. Its host deletes it when it expires or when someone expires it early.
  • Checked before it is served. A clone is ready only when its rows loaded, no sampled original value survived masking, and its server accepts nothing but TLS.
  • Unmasked clones are gated. A --profile none clone holds real data. It needs admin scope and a second owner or admin to approve it, and is marked as real data wherever it appears.
Supported sources
Clones are built from logical backups of PostgreSQL sources. Other source types, and PostgreSQL backups taken with WAL-G or pgBackRest, can be verified and restored but not cloned.

All clone commands are in the CLI reference. Back to the TillArk overview.