--- type: concept tags: [parking, durability, backup, recovery, security, crypto] sources: [] updated: 2026-06-29 --- # Backup & Disaster Recovery The appliance's [[sqlite]] DB **is** the signed [[append-only-event-chain]] — the whole revenue/audit history. A disk failure or a stolen/destroyed PC currently means **total loss** (this is [[open-questions]] #5). This page is the settled design for an on-site, admin-driven backup that survives **total hardware loss** and restores to a fresh appliance with the signed chain still verifying. (Designed 2026-06-29.) ## The recovery scenario it must satisfy The driving scenario (the one that forces every decision below): **the PC is gone** — stolen or destroyed. Its SSD is LUKS-encrypted and **TPM-sealed**, so the disk is unrecoverable *by design* (a stolen disk won't unlock off its own TPM — see [[disk-os-hardening]], [[tpm]]). We do **not** want the dead disk; we want to stand up a **new PC**, restore the backup, and continue signing the **same** chain. For that to work, recovery must depend on **(a)** the backup file and **(b) two keys held out-of-band** — never on the dead machine. ## Key custody — the load-bearing decision This is the part the whole plan rests on, and it interacts with the secure-element question ([[open-questions]] #6). Three **independent** keys, three custodians: | Key | Lives | Recoverable after PC loss? | Job | | --- | --- | --- | --- | | **`EVENT_SIGNING_KEY`** | [[fleet-deployment-komodo\|Komodo]] secret (`park_buzi_event_signing_key`), escrowed offsite | **Yes — by design** | Signs + verifies the ledger chain | | **`park_buzi_backup_key`** *(new)* | Komodo secret, escrowed offsite, **separate** from the signing key | **Yes** | Encrypts/decrypts the backup file | | **LUKS / TPM disk key** | The appliance's TPM only | **No — deliberately** | At-rest protection of the powered-off SSD | - **The signing key is decoupled from the TPM** — kept an *extractable software HMAC secret* ([[append-only-event-chain]], `signer.ts`), held in Komodo and escrowed by the operator. This is a **conscious trade**: a truly non-extractable TPM-sealed signing key (the #6 upgrade) would make the ledger unforgeable even against a host-root attacker — but it would also make the **old ledger permanently unverifiable after total hardware loss** (the sealed key dies with the machine; `buildVerifier(keyId)` would return `undefined` forever). You cannot have *both* "key can never be extracted" *and* "I can rescue the key after the machine dies" — they are the same property from two sides. Against the [[threat-model|primary adversary]] (the **booth operator**, who has a UI login, not host root) an escrowed software key is already tamper-evident, so the recoverable design is chosen **today**; revisiting #6 means re-accepting the unverifiable-after-loss cost. See [[tpm]] "TPM vs. ATECC608", [[fleet-deployment-komodo]] (the "EVENT_SIGNING_KEY-in-Core is a fraud-root blast radius" caveat is the same trade). - **Backup key is separate from the signing key** even though Komodo holds both — so they can be managed independently. Rationale: (1) the **signing key must almost never rotate** (every rotation fractures the chain into a new `keyId` segment — old events stay pinned to the old key forever), whereas the **backup key may want routine rotation** (a USB went home, a target was decommissioned); coupling them drags the cheap op into the expensive one. (2) The backup key **travels to every backup destination** (USB, NAS, SFTP); the signing key should travel *nowhere* but Komodo → process memory — sharing one key means every backup target conceptually exposes the signing key. (3) Keeping them separate keeps the **#6 TPM-migration door open** without re-wiring backups. Decided 2026-06-29 (the "one fewer secret to escrow" simplicity of a shared key is real, but weakest here because Komodo already holds both). > **The keys are never inside the backup they unlock.** A key can't decrypt the file it's locked in. > Recovery = backup file **+** both escrowed keys, supplied out-of-band. The runbook must say this > plainly so nobody "helpfully" stores the keys next to the backups. ## What a backup contains **Full SQLite DB, snapshots included** — one self-contained, restore-to-identical-appliance file (ledger + sessions + config + subscriptions + the [[entry-exit-points|snapshot]] BLOBs). Chosen for completeness over size. > **Size caveat (interacts with [[open-questions]] #10).** Snapshot BLOBs **dominate** DB size and > bloat *every* backup. They are unsigned, advisory, and already disk-pressure-pruned > ([[entry-exit-points]]). A future **"exclude snapshots" toggle** (ledger/sessions/config only — much > smaller, signed chain still fully preserved) is the obvious knob if backup size becomes a problem; the > default is the complete picture. The backup is produced via SQLite **online-backup / `VACUUM INTO`** (a consistent snapshot of the live WAL-mode DB — **never a raw file copy**, which can capture a torn WAL), then encrypted with `park_buzi_backup_key`. **Acceptance test:** a restored copy must still pass `verifyChain` — the signed chain is the thing being protected, so an unverifiable restore is a failed backup. ## Triggers - **Manual** — an admin-only **"Back up now"** button runs immediately to the configured target. - **Periodic** — an **in-process daily timer** (same pattern as the snapshot-retention prune, [[entry-exit-points]] / `snapshot-retention.ts`): runs only if the configured target is reachable/mounted; surfaces last-success / last-error in the UI. No OS cron — it lives inside the Fastify process, works inside the [[container-deployment|Docker container]], and is configured in one place. ([[offline-first]]: the periodic path must tolerate a missing/unmounted target without failing the app.) ## Destinations (admin-configurable) All three supported in the first cut; the manual button and the periodic timer share them: - **Local / USB / SATA disk** — a mounted path on an attached disk. Simplest, fully offline, matches the air-gapped appliance. The strong first target. - **Network drive (SMB/NFS)** — a mounted share on the isolated LAN (a site NAS). Still local-network, no internet ([[network-isolation]]). - **SFTP** — push to an SFTP endpoint, useful for an offsite copy. **FTP is excluded** (plaintext credentials + data); SFTP is the safe equivalent. ## Retention at the destination **Keep last N + thinned dailies** (e.g. last 7 daily / last 4 weekly) — bounded disk use, and it survives the "a bad/partial run clobbered the only good copy" failure. (A single rolling overwrite-latest file was rejected for exactly that reason.) ## Threat-model fit — restore is the dangerous half Writing a backup is benign; **restore is operator-adversary surface** ([[threat-model]]). A restored DB *replaces* the live signed chain — so a malicious restore is a way to swap in a doctored history. Therefore: - **Restore is NOT a booth button.** It is an **admin-only, out-of-band runbook action** (new appliance, deliberate provisioning step), not something reachable from the operator console. - The backup **target configuration** and the **"Back up now"** action are admin-gated. - Backups do **not** weaken the chain's tamper-evidence: a restored chain is re-verified with the escrowed `EVENT_SIGNING_KEY`; a tampered restore fails `verifyChain` just as a tampered live DB would. The backup is a **durability** control, not an integrity one — integrity stays with the signed chain + [[reconciliation]]. ## As-built (2026-06-29) — engine + local/mounted target The first slice is **built and tested**: the backup **engine + a local/mounted target + the daily timer + the manual route**. What landed: - **`apps/server/src/backup.ts`** — the engine. Consistent online copy via better-sqlite3's native `.backup()` (a transactionally-consistent snapshot of the live WAL DB — **not** a raw file copy), then **AES-256-GCM** encryption with a **scrypt-derived** key from `BACKUP_KEY`. Self-describing header (`magic | version | salt | iv | … | authTag`) so a restore tool needs only the key + the file — **zero new dependencies** (Node `crypto`). The plaintext intermediate is written to **scratch** (not the removable/network target) and **wiped in a `finally`**, success or fail. Retention = **keep-last-N + one-per-day-within-N-days** (`pruneOldBackups`). Tested: round-trip decrypts to a **byte-identical, queryable DB**; a flipped byte or wrong key **fails GCM auth**; short key rejected; scratch plaintext always removed. - **`backup-service.ts`** — the **target directory is admin-chosen** (`site_config.backup_target_dir`, migration 0016) and read **fresh each run**, so changing it in the UI takes effect with no restart. Only the **encryption key stays an env/Komodo secret** (`BACKUP_KEY`) — a key must never live in the DB it backs up. Retention knobs (`BACKUP_KEEP_LAST`, `BACKUP_KEEP_DAILY_DAYS`) stay env. The service **serializes** concurrent runs (single in-flight guard) and records last-success / last-error; `status()` exposes `targetDir` + `keyPresent` so the UI distinguishes "no target" from "no key". - **`routes/backup.ts`** — `GET /api/backup/status` (`backup:read`); `PUT /api/backup/config` to set/ clear the target (`backup:update`); `POST /api/backup/test` to probe a candidate path server-side — exists / is-a-dir / writable (`backup:update`); `POST /api/backup/run` (`backup:create`), a clean **409 `backup_not_configured`** when target+key aren't both set. New `backup` permission resource (`backup:read/update/create`) in `@parking/shared`. **No restore route** — out-of-band by design. - **`apps/web/src/BackupSettings.tsx`** — a Setup → **Backup** tab (gated `backup:read`): an editable **target-path field** with a **Test target** probe (localized ok/missing/not-a-dir/not-writable), **Save**, the status panel (config state, last-run size/pruned/error, a distinct amber **missing BACKUP_KEY** warning), a **Back up now** button, and the restore-is-out-of-band note. Full i18n (sq + en). - **`server.ts`** — an **unref'd daily timer** (`backupService.runScheduled`), a **no-op until configured**, and **deliberately NOT run at startup** (a just-power-cut booth shouldn't write to a possibly-unmounted disk; the daily cadence + the manual button cover it). - Env documented in `apps/server/.env.example` (with the escrow + separate-key notes). **SMB/NFS already work** — they're just a mounted path the admin enters as the target. **Deferred to follow-up slices:** an **SFTP** target and a **restore runbook / CLI**. ## Status Design settled 2026-06-29; **engine + admin-configured local/mounted target + admin UI BUILT 2026-06-29** (SFTP + restore tooling pending). The target directory is **admin-chosen in the UI** (`site_config`, migration 0016), not an env var — the on-site admin picks where backups land; only `BACKUP_KEY` stays a server secret. Resolves the *design* half of [[open-questions]] #5 and the first build slices; records the key-custody stance that bears on #6 (signing stays decoupled from the TPM) and #10 (snapshots bloat backups → future exclude toggle). See [[append-only-event-chain]], [[disk-os-hardening]], [[tpm]], [[fleet-deployment-komodo]], [[reconciliation]].