audit 3: a restore is ended by its Resync, and half an archive is refused
test / test (push) Canceled after 0s
test / test (push) Canceled after 0s
The restore manifest was consumed as soon as its version was checked, before store.Open and the Resync; a start that failed in between left a data directory the next start took for an ordinary one (M1). openData now runs CheckRestore, store.Open, the Resync, and only then ConsumeRestore. A truncated archive had nothing to refuse it: the manifest is the first entry, so a download cut short or a tar stopped by a damaged block was a restore on half a database. Every archive now ends with data/contents.json, the size and SHA-256 of each data/ file (deploy files are the operator's to edit), and CheckRestore compares /data with it before the database is opened, then renames it contents.checked.json so a start after a failed one does not compare again unless the manifest changed. Tests: a start whose Resync fails, then a start that resyncs and consumes; an archive cut in the database, before the deploy files and in its last entry (refused, database untouched, no -wal); a damaged key named; the record's names confined to data/; the domain import of a cut plain export, a cut and an altered .spde (refused, nothing created). e2e on the test server: full_backup_restore now also sees contents.json in the archive and gone after the first start, through the real entrypoint and daemons (all pass, 762 s); go vet + go test of 24 packages pass there. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -960,7 +960,8 @@ single-connection trade-off that follows from it.
|
||||
| `postfix/dmarc_relay_domains` | Domains accepted for DMARC report delivery |
|
||||
| `postfix/queue/` | Postfix transit mail (deferred/active); survives container recreate |
|
||||
| `log/mail.log` | Postfix delivery log + rotated copies (excluded from backups) |
|
||||
| `manifest.json` | Backup version stamp (consumed on restore) |
|
||||
| `manifest.json` | Backup version stamp; removed by the first start after a restore once its Resync is done |
|
||||
| `contents.json` | A backup's last entry: the size and SHA-256 of each file under `data/`, compared by the first start after a restore before it opens the database (renamed `contents.checked.json` once compared, removed with the manifest) |
|
||||
|
||||
Not in `/data`: TLS certificates for the panel (reverse-proxy mount) — though
|
||||
full backups also archive the operator's `./certs` PEM files when present.
|
||||
|
||||
@@ -1074,11 +1074,24 @@ those after it move up one patch number each.
|
||||
|
||||
### Stage 3 — persistence and restore (M1–M5, L1–L4, L7) → `2.5.3`
|
||||
|
||||
- [ ] **Consume the manifest after the resync succeeds** (`CheckRestore` reports;
|
||||
- [x] **Consume the manifest after the resync succeeds** (`CheckRestore` reports;
|
||||
`run` deletes after `resyncAfterRestore` returns nil). Test: Resync fails →
|
||||
manifest kept → second boot resyncs. And a corrupted or truncated archive
|
||||
is refused on restore and on import with its message, nothing half
|
||||
written (a row of review-2026-08-followups, placed here 2026-10-14). — **Opus**
|
||||
*Done 2026-10-15:* `cmd/panel` `openData` runs CheckRestore, `store.Open`,
|
||||
the Resync and then `backup.ConsumeRestore`. A truncated archive had
|
||||
nothing to refuse it - the manifest is the first entry, so half an
|
||||
archive was a restore on half a database: every archive now ends with
|
||||
`data/contents.json` (size and SHA-256 of each `data/` file; the deploy
|
||||
files are the operator's to edit), CheckRestore compares `/data` with it
|
||||
before the database is opened and renames it `contents.checked.json`, so
|
||||
a start after a failed one (which had the database open) does not compare
|
||||
again unless the manifest changed. Tests: the start that fails, a cut in
|
||||
the database / before the deploy files / in the last entry (database
|
||||
untouched, no `-wal`), a damaged key named; the domain import of a cut
|
||||
plain export, a cut and an altered `.spde` (each refused, nothing
|
||||
created). The CLI's `-decrypt -o` is step 3.
|
||||
- [ ] **Indexes**: `idx_send_log_domain_created_at (domain_id, created_at)` —
|
||||
on the id column stage 2 adds, replacing `idx_send_log_domain`;
|
||||
partial `idx_send_log_queued (created_at) WHERE status = 'queued'`;
|
||||
|
||||
+17
-1
@@ -809,7 +809,9 @@ blocked. On that same first boot the panel also runs one **Resync** — OpenDKIM
|
||||
tables and Postfix's sender map are re-derived from SQLite (inbound relay maps
|
||||
when `INBOUND_RELAY_ENABLE=true`; DMARC maps when `DMARC_REPORTS_ENABLE=true`)
|
||||
and both daemons are reloaded, healing any drift between the extracted files and
|
||||
the database. The *Reload configuration* button on Server › Health resyncs
|
||||
the database. The manifest is deleted only after that Resync is done: a first
|
||||
start that fails on the way leaves it, and the next start is a restore again
|
||||
and runs the Resync. The *Reload configuration* button on Server › Health resyncs
|
||||
OpenDKIM, the sender map, and inbound maps only — not DMARC maps. This is why the
|
||||
compose file pins a fixed tag rather than `:latest`: without a known version,
|
||||
there'd be no way to tell which image restoring a given backup actually requires
|
||||
@@ -857,6 +859,20 @@ tag](#fixed-image-tag) if it is not published), then `docker compose up -d`
|
||||
again — the manifest is still there because the failed boot never got to
|
||||
delete it.
|
||||
|
||||
An archive unpacked **in part** — a download cut short, or a `tar` that
|
||||
stopped at a damaged block (it says so and exits non-zero) — is refused at
|
||||
step 5 as well, before the database is opened:
|
||||
|
||||
```
|
||||
backup: the archive was not unpacked whole (data/contents.json, its last entry, is not there) - it was cut short or is damaged; nothing was opened. Unpack it again, from an intact copy, into an empty directory
|
||||
```
|
||||
|
||||
The last entry of every panel and CLI archive, `data/contents.json`, records
|
||||
each file under `data/` with its size and SHA-256, and the first start
|
||||
compares `/data` with it; a file that differs is named in the message.
|
||||
`docker-compose.yml`, `.env` and `certs/` are not compared — they are yours to
|
||||
edit before that start.
|
||||
|
||||
**Moving to a different host** is the same flow: create an empty project
|
||||
directory, unpack the backup there, edit `.env` (and `docker-compose.yml` if
|
||||
needed) for the new hostname or proxy, then `docker compose up -d`. The archive
|
||||
|
||||
+1
-1
@@ -42,7 +42,7 @@ in `git log` and [CHANGELOG.md](../CHANGELOG.md).
|
||||
| from-header-binding | Hold a message's `From:` header against the application that sends it | shipped `2.4.0` | 9/9 | [plans/from-header-binding.md](plans/from-header-binding.md) |
|
||||
| panel-redesign | Panel redesign (accepted mockups, design contract) | shipped `2.0.0` | 27/27 | [plans/panel-redesign.md](plans/panel-redesign.md) |
|
||||
| schema-squash | Squash SQLite migrations into a 2.x baseline | built in panel-redesign (stage 1) | — | [plans/panel-redesign.md](plans/panel-redesign.md) |
|
||||
| audit-2026-10-12 | Follow-ups of the 2026-10-12 full-tree code audit, seven stages | in progress (stage 0 shipped `2.4.2`, stages 1+2 `2.5.0`, 2b `2.5.1`, 2c `2.5.2`; next stage 3 → `2.5.3`) | 49/81 | [audit-2026-10-12.md](audit-2026-10-12.md) |
|
||||
| audit-2026-10-12 | Follow-ups of the 2026-10-12 full-tree code audit, seven stages | in progress (stage 0 shipped `2.4.2`, stages 1+2 `2.5.0`, 2b `2.5.1`, 2c `2.5.2`; next stage 3 → `2.5.3`) | 50/81 | [audit-2026-10-12.md](audit-2026-10-12.md) |
|
||||
|
||||
**Recommended order** (not binding; owner, 2026-10-09: feature order follows
|
||||
development convenience): **panel-redesign** shipped as `2.0.0` (2026-10-11) and
|
||||
|
||||
Reference in New Issue
Block a user