backup: no full backup holds the Postfix queue

Owner, 2026-10-11: the queue goes out of the command's archive too.

The commit that made the panel's backup work left the queue in the
archive of selfpost-backup, which runs as root and can read it. But that
copy was never the queue: the archive is made while the instance runs,
Postfix moves messages between the queue's directories all the time, and
a message on its way could be in the archive twice or not at all. One
rule now - QueueDir is left out of every archive - and
Params.Unprivileged is only about a certs/ file the panel may not open.

What is not carried over is mail still waiting for delivery; a restored
instance builds an empty queue (the e2e restore scenario starts one).
To keep the queue: stop the instance and tar the project directory, as
the guide's methods 3 and 4 already describe.

Guide: the description of a full backup, methods 1 and 2 and the
comparison table. Help follows the design commit before this one.

  go vet ./internal/backup/: ok
  go test ./internal/backup/ ./cmd/... ./internal/web/...: ok

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

CHANGELOG.md

@@ -16,6 +16,12 @@ Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); version
   as one parse failure and the reason is in the mail log, on the message's
   own delivery line (Server › System log, `dropped: …`). Only a failure of
   the database is still retried.
+- **A full backup no longer holds the Postfix queue**, from the panel or from
+  `selfpost-backup`. It is taken while the instance runs, and a copy of a
+  queue Postfix is working on can hold a message twice or not at all. Mail
+  still waiting for delivery is not carried over; a restored instance starts
+  with an empty queue. To keep the queue, stop the instance and `tar` the
+  project directory.
 - DMARC reports are kept 500 **for each domain** (and 90 days), not 500 for
   the whole server - see Security.
 
@@ -26,12 +32,12 @@ Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); version
   includes the queue, Postfix keeps the queue readable by its own user alone,
   and the panel is not that user ("open /data/postfix/queue/active:
   permission denied" in the log). `selfpost-backup` run with `docker exec`
-  was not affected - it runs as root. The panel's archive now leaves the queue
+  was not affected - it runs as root. The queue is no longer part of a backup
-  out, and likewise a file in `certs/` it may not open (usually the private
+  (see Changed), and the panel's archive leaves out a file in `certs/` it may
-  key, named in the log); the page, Help and the guide say so, and the guide
+  not open (usually the private key, named in the log); the page, Help and
-  says to put the key back before starting a restored instance. Everything is
+  the guide say so, and the guide says to put the key back before starting a
-  read through before the download starts, so a failure is a message on the
+  restored instance. Everything is read through before the download starts,
-  page and not a file cut short.
+  so a failure is a message on the page and not a file cut short.
 
 ### Security
 

docs/guide.md

@@ -542,10 +542,14 @@ journal-milter (level 2) is down. There is no per-IP bypass.
 ### Full backup and restore
 
 **Full backup** is a self-contained project archive: `data/` (SQLite, all
-domains' DKIM keys, all applications' SASL credentials, the Postfix queue,
+domains' DKIM keys, all applications' SASL credentials, `manifest.json` with
-`manifest.json` with the version that created it), plus `docker-compose.yml`,
+the version that created it), plus `docker-compose.yml`, `.env`, and `certs/`
-`.env`, and `certs/` from the operator directory next to `./data`. Delivery
+from the operator directory next to `./data`. Delivery logs under `data/log/`
-logs under `data/log/` are excluded. The base compose file mounts the project
+are excluded, and so is the Postfix queue: the archive is taken while the
+instance runs, and a copy of a queue Postfix is working on can hold a message
+twice or not at all. Mail still waiting for delivery is therefore not carried
+over, and a restored instance starts with an empty queue; to keep the queue,
+stop the instance and use `tar` (methods 3 and 4 below). The base compose file mounts the project
 directory read-only at `/selfpost-deploy` so the panel and CLI can read those
 deploy files — without that mount, *Full backup* refuses with an error.
 
@@ -556,23 +560,17 @@ instance keeps running, what goes into the archive, and what happens on restore.
 
 **1. Panel — while the container is running (no downtime).** *Server › Backup*
 → *Full backup*. *Encrypt with a password* is ticked; untick it for a plain
-archive. The archive of the CLI below, less two things the panel may not read:
+archive. The archive of the CLI below, less one thing the panel may not read:
-it runs as its own user, not as root. **The Postfix queue** is left out —
+it runs as its own user, not as root, and **a file in `certs/` it cannot open**
-Postfix keeps it readable by its own user alone — so mail that was waiting for
+is left out — usually the private key (the panel's log names it). Put the key
-delivery at that moment is not in the archive; the restored instance starts
-with an empty queue. **A file in `certs/` the panel cannot open** is left out
-too, which is usually the private key (the panel's log names it); put the key
 back into `certs/` before starting a restored instance. If anything else
 cannot be read, the panel says so and downloads nothing.
 
 **2. `selfpost-backup` CLI — while the container is running (no downtime).**
 The panel button and this command call the same code path: a consistent SQLite
 snapshot via `VACUUM INTO`, then a gzip tar of `data/` (minus diagnostics) plus
-deploy files. `docker exec` runs the command as root, so this archive is the
+deploy files. `docker exec` runs the command as root, so this archive has
-complete one: it has the Postfix queue and everything in `certs/`. The queue is
+everything in `certs/`, the private key included.
-copied while Postfix works on it, so a message on its way through may be in the
-archive twice or not at all; stop the service first (method 3) if the queue
-matters.
 
 ```sh
 docker exec <container> selfpost-backup > selfpost-backup.tar.gz
@@ -627,7 +625,7 @@ methods avoid it by stopping first.
 | Self-contained archive | Yes | Yes | No — deploy files separate |
 | SQLite consistency | `VACUUM INTO` snapshot | Safe when stopped | Safe when stopped |
 | `data/log/` | Excluded | Included (optional exclude) | Included (optional exclude) |
-| Postfix queue | CLI: yes, copied live. Panel: no | Yes | Yes |
+| Postfix queue | No | Yes | Yes |
 | `certs/` private key | CLI: yes. Panel: only if the panel may read it | Yes | No — deploy files separate |
 | `data/manifest.json` | Written (version stamp) | Absent | Absent |
 | Restore version check | Yes — must match image tag | No | No |

internal/backup/backup.go

@@ -1,16 +1,16 @@
 // Package backup implements SelfPost's full-server backup and the restore
 // version guard (architecture.md § Persistence). A full backup is a
 // gzip-compressed tar of the consolidated persistent state under /data — the
-// SQLite database (as a consistent snapshot), the per-domain DKIM keys, the
+// SQLite database (as a consistent snapshot), the per-domain DKIM keys and the
-// SASL database, and the Postfix queue — plus docker-compose.yml, .env, and
+// SASL database — plus docker-compose.yml, .env, and certs/ from the
-// certs/ from the operator's deploy directory, and a manifest recording the
+// operator's deploy directory, and a manifest recording the SelfPost version
-// SelfPost version that produced it. Postfix delivery logs under log/ are
+// that produced it. Postfix delivery logs under log/ are excluded
-// excluded (diagnostics, not state).
+// (diagnostics, not state), and so is the Postfix queue (see QueueDir).
 //
 // Restore is not a separate code path in the panel: a backup is extracted into
 // the operator's project directory (data/, docker-compose.yml, .env, certs/)
 // before first start. The archive carries everything needed to bring the
-// instance back — DKIM keys, sasldb2, sender map, queue, and deploy files —
+// instance back — DKIM keys, sasldb2, sender map, and deploy files —
 // so the operator only adjusts hostname or proxy settings on a new host. The
 // restore-specific steps the panel runs are CheckRestore, which refuses to
 // boot if the manifest's version does not match the running binary so
@@ -81,22 +81,23 @@ type Params struct {
 	DeployRoot string
 	OnWarn     func(string)
 	// Unprivileged says the archive is made by the panel, which runs as its
-	// own user and not as root. Two things under the roots are then not its to
+	// own user and not as root. A file under certs/ it may not open - usually
-	// read, and are left out instead of failing the whole archive:
+	// the private key, root and Postfix's group only - is then left out
-	//
+	// instead of failing the whole archive, and OnWarn names it. The
-	//   - the Postfix queue (QueueDir). Postfix keeps every queue file and
+	// selfpost-backup command, run as root, leaves this unset and takes it.
-	//     directory readable by its own user alone, and that cannot be
-	//     configured. What is lost is mail not yet delivered at that moment;
-	//     a start without a queue directory builds an empty one.
-	//   - a file under certs/ the panel may not open, which is usually the
-	//     private key (root and Postfix's group only). OnWarn names it.
-	//
-	// The selfpost-backup command, run as root, leaves this unset and takes
-	// both.
 	Unprivileged bool
 }
 
-// QueueDir is the Postfix queue, relative to the data directory.
+// QueueDir is the Postfix queue, relative to the data directory. No backup
+// holds it (owner, 2026-10-11). An archive is made while the instance runs,
+// and Postfix moves messages between the queue's directories all the time: a
+// copy taken then can hold a message twice or not at all, so what it would
+// bring back is not the queue. Nor could the panel read it - Postfix keeps
+// every queue file readable by its own user alone, which is why the panel's
+// backup failed outright from 1.3.0 to 2.0.0. What is not carried over is mail
+// still waiting for delivery; a start without a queue directory builds an
+// empty one. Whoever needs the queue stops the instance and archives the
+// project directory (guide § Taking a full backup).
 const QueueDir = "postfix/queue"
 
 // excludedFromArchive lists the data-directory entries a backup never carries.
@@ -203,11 +204,8 @@ func validateDeployRoot(deployRoot string) error {
 // left out of the archive. A directory left out is not descended into.
 type skipFunc func(path, rel string, d fs.DirEntry) bool
 
-// skipData leaves the Postfix queue out of an unprivileged archive.
+// skipData leaves the Postfix queue out of every archive.
 func (p Params) skipData() skipFunc {
-	if !p.Unprivileged {
-		return nil
-	}
 	return func(_, rel string, _ fs.DirEntry) bool {
 		return filepath.ToSlash(rel) == QueueDir
 	}

internal/backup/backup_test.go

@@ -112,7 +112,6 @@ func TestCreateIncludesStateExcludesTransient(t *testing.T) {
 		DataArchivePrefix + "opendkim/keys/example.com/selfpost.private",
 		DataArchivePrefix + "sasl/sasldb2",
 		DataArchivePrefix + "postfix/sender_login_maps",
-		DataArchivePrefix + "postfix/queue/deferred/sample",
 		ComposeFileName,
 		EnvFileName,
 		CertsDirName + "/fullchain.pem",
@@ -129,6 +128,7 @@ func TestCreateIncludesStateExcludesTransient(t *testing.T) {
 		DataArchivePrefix + "selfpost.db-shm",
 		DataArchivePrefix + "log/mail.log",
 		DataArchivePrefix + "log/mail.log.1",
+		DataArchivePrefix + "postfix/queue/deferred/sample",
 	} {
 		if _, ok := files[name]; ok {
 			t.Errorf("archive should not contain %s", name)
@@ -236,8 +236,8 @@ func TestCheckRestoreWrongFormatRejected(t *testing.T) {
 	}
 }
 
-// The panel's archive: no Postfix queue, everything else as before, and Check
+// The panel's archive: like any other it has no Postfix queue, the rest is
-// agrees that it can be made.
+// there, and Check agrees that it can be made.
 func TestCreateUnprivilegedLeavesTheQueueOut(t *testing.T) {
 	dataDir, dbPath, deployRoot := seedDataDir(t)
 	p := Params{DataDir: dataDir, DBPath: dbPath, Version: "1.2.3", DeployRoot: deployRoot, Unprivileged: true}

internal/web/view/help.go

@@ -190,8 +190,8 @@ var serverTopics = []HelpTopic{
 	{ID: HelpBackup, Title: "Backup and moving a domain", Body: []Text{
 		Rich("Full backup downloads one archive of the whole instance: ", Code("data/"), " (the database, DKIM keys and application credentials), ",
 			Code("docker-compose.yml"), ", ", Code(".env"), " and ", Code("certs/"),
-			". The panel cannot read the Postfix queue, nor usually the private key in ", Code("certs/"),
+			". The Postfix queue is not in it: mail still waiting for delivery is not carried over. Nor, usually, is the private key in ", Code("certs/"),
-			", and leaves them out; ", Code("selfpost-backup"), " run in the container takes both. It holds secrets, so leave Encrypt with a password ticked: the file is sealed as ", Code(".spbk"),
+			", which the panel cannot read; ", Code("selfpost-backup"), " run in the container takes the key. It holds secrets, so leave Encrypt with a password ticked: the file is sealed as ", Code(".spbk"),
 			", the password is not stored, and without it nothing can be recovered. To restore, unpack it into an empty project directory (decrypt a ",
 			Code(".spbk"), " first with ", Code("selfpost-backup -decrypt"),
 			") and start the same SelfPost version that made it; the panel refuses data from another version. The reverse-proxy configuration and ",