Skip to content

Back up and restore the Hub

What you'll have at the end: encrypted database backups written on a schedule to durable storage, a manual backup you have watched succeed, and a restore procedure you have actually tested — before the day you need it.

Prerequisites: a running Hub you can log in to — see Hub → Install procedure. On a bare (non-Docker) install, the Hub host also needs the PostgreSQL client tools (pg_dump and psql); the packaged Docker image already contains them.

Time: about 10 minutes for setup and the first backup; a restore test on a spare Hub adds perhaps 15 more.

All of this lives in one place: Settings → Data and recovery → Backups (https://hub.example.internal/settings).

What a backup contains — and what it doesn't

A Cyphers backup is a complete dump of the Hub's PostgreSQL database (pg_dump), compressed and encrypted, written as a single <name>.sql.enc file. That covers everything the Hub stores in the database: endpoints, certificates and observations, renewal history, scores, audit log, settings — and the Hub's CA private keys, which are persisted to the database (encrypted under the CA passphrase) as well as to disk, so a restored database can re-provision the CA key files on its own.

It does not contain files that live outside PostgreSQL:

  • The CA passphrase (CYPHERS_CA_KEY_PASSPHRASE) and the rest of the Hub's environment file (.env: DATABASE_URL, CYPHERS_CA_URL, ports, NATS settings). Without the CA passphrase, the CA keys inside a restored database cannot be decrypted — keep a copy of the .env file (or at minimum its secrets) in your secret store.
  • The backup passphrase itself. It is stored only encrypted; see the warning in Step 1.
  • Scout-held material. Every Scout's private key, client certificate and delivered endpoint certificates live in the Scout's own data directory on each endpoint, never on the Hub — see Scout → Install procedure. A lost Scout is re-enrolled, not restored from a Hub backup.

Step 1 — Set the backup directory and passphrase

On Settings → Data and recovery → Backups, fill in the form and click Save backup settings:

  • Backup directory — where backup files are written, as a path on the Hub host. The default is ./backups, which sits next to the Hub itself — fine for a first test, but a backup on the same disk as the database dies with it. Point it at durable, off-box storage (a mounted network share, a separate volume that is itself backed up):
/mnt/backup/cyphers
  • Passphrase — the key every backup file is encrypted with. The field is write-only: it is never shown again, and leaving it blank on a later save keeps the current one.

You should now see the Schedule status line change from "No passphrase is set, so no backup can be written." to a schedule message, and the Back up now button become clickable.

The passphrase is required to restore — store it outside the Hub

A backup file is useless without the passphrase it was encrypted with, and the Hub stores that passphrase only in its own database — inside the very thing you are backing up. Put it in your password manager or secret store now, alongside the CA passphrase. Also: changing the passphrase later strands every backup encrypted under the old one, so if you ever rotate it, keep the old passphrase until its backups have aged out.

Step 2 — Enable the schedule

In the same form, tick Run backups on a schedule, set Every (hours) (default 24) and Backups retained (default 7 — how many completed backups the retention prune keeps, oldest deleted first after each successful backup), and click Save backup settings again.

You should now see the Schedule line read "Scheduled every 24 hours" (or your interval). The scheduler runs inside the Hub process and checks every minute whether the interval has elapsed since the last completed backup — there is no external cron to configure.

Retention only prunes what the schedule wrote

The keep-N rule applies to backups recorded in the Hub's history. Other .sql.enc files in the directory (for example labelled pre-upgrade dumps) are age-swept by the daily retention job instead, and files in any other format are never deleted — only reported in the log.

Step 3 — Run a first backup and confirm it landed

Click Back up now.

You should now see, within a few moments, the Last backup status line show a fresh timestamp instead of "No backup has completed yet." — and the file on disk on the Hub host:

$ ls -l /mnt/backup/cyphers
-rw-r--r-- 1 cyphers cyphers 2469888 Aug 22 14:03 cyphers_backup_20260822_140301.sql.enc

If the attempt fails, the panel shows "Last attempt failed:" with the reason, and the action reports "Backup failed; see backup history for the reason". The most common first-run causes are a backup directory the Hub process cannot write, or missing pg_dump on a bare install.

One-shot backups from the CLI

cyphers-hub backup --label pre-upgrade writes a pre-upgrade-<timestamp>.sql.enc into the same directory, encrypted with the same passphrase. It exists for pre-deploy rollback dumps — see Fleet upgrades — and deliberately does not reset the schedule clock.

Step 4 — Test a restore on a spare Hub (do this periodically)

An untested backup is a hope, not a backup. On a schedule of your choosing (quarterly is a reasonable floor), prove the whole chain — file, passphrase, restore — on a spare or staging Hub, never your production one:

  1. Stand up a scratch Hub against a scratch database (Hub → Install procedure).
  2. In its Settings → Data and recovery → Backups, save the same passphrase your production backups were encrypted with. The restore decrypts with the passphrase configured on the Hub doing the restoring — this step is also your proof that the passphrase in your secret store is the right one.
  3. Copy a recent .sql.enc file onto the spare Hub's host.
  4. Follow Step 5 below on the spare Hub.

You should now see, after the restore, your production data in the spare Hub's dashboard — endpoints on Manage, history under Activity — and http://hub.example.internal:8443/readyz answering with all checks ok.

Restoring without any running Hub

For disaster recovery there is also a database-free path: CYPHERS_BACKUP_PASS=<passphrase> cyphers-hub backup-decrypt-file --input <file>.sql.enc --output dump.sql.gz decrypts and verifies the envelope to a plain gzipped SQL dump you can feed to psql yourself. It needs only the binary and the backup passphrase.

Step 5 — Restore

Restoring replaces the entire database

A restore replaces this Hub's entire database with the contents of the chosen file. Everything recorded after that backup was taken — observations, renewals, audit entries, settings changes — is gone. There is no partial or selective restore. On a production Hub, this is a last resort, not an undo button.

Open Restore from a backup at the bottom of the Backups panel, and:

  1. Enter the full path of the backup file (on the Hub host) in Backup file path.
  2. Type RESTORE in the Type RESTORE field.
  3. Click Restore.

The Hub decrypts and integrity-verifies the file before anything touches the database — a wrong passphrase, a truncated file, or a non-backup file is rejected with "Restore failed; the Hub is unchanged" and the database is left as it was. Only a file that decrypts and validates cleanly is then applied.

You should now see the dashboard reflecting the restored data, and /readyz reporting every check ok.

Where to next