mix/selfpost
1
0
Fork 0
Code Issues Pull Requests Actions 8 Packages Projects Releases Wiki Activity

backup: the panel's archive is the command's, private key included

Browse Source 
Owner, 2026-10-11: the private key is needed, and a backup from the
panel and one from the console have to be the same.

The commit that made the panel's backup work skipped a file under
certs/ the panel could not open - on the test server the private key,
which the certificate hook leaves to root and Postfix's group. That gave
an archive with a certificate and no key, and two kinds of archive.

- Params.Unprivileged is gone. Nothing is left out on account of who
  makes the archive; the queue is out for everyone (previous commit).
- When the panel cannot read a file under certs/, backup.Check says so
  before the download and the panel refuses with what to do on the host:
  chgrp 998 certs/privkey.pem && chmod 640. backup.UnreadableCert tells
  that case from any other failure.
- 998 is the container's selfpost group, which holds the panel and
  Postfix. Its id was whatever groupadd gave; the Dockerfile now fixes
  it at the value it has always had, because an operator addresses it
  by number from the host.
- Guide: method 1 is the same archive as method 2, with the permission
  the key needs and a note for whatever renews the certificate. Help and
  the box follow the design commit before this one.
- e2e: the restore scenario first makes the key unreadable and expects
  the refusal naming the file and the way out, then takes the backup and
  expects the key in it; nothing is put back by hand before the restart.

  go vet ./internal/backup/ ./internal/web/handlers/: ok
  go test ./internal/backup/ ./cmd/... ./internal/web/...: ok
  make e2e on the test server: ok

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
mix
2026-10-11 20:43:10 +03:00
co-authored by Claude Opus 5.5
parent 75ebbf6a57
commit dd0fefdcb7
10 changed files with 138 additions and 93 deletions
Download Patch File Download Diff File Expand all files Collapse all files
internal/backup/backup.go
+28 -44
View File
@@ -80,12 +80,6 @@ type Params struct {
Version string
DeployRoot string
OnWarn func(string)
// Unprivileged says the archive is made by the panel, which runs as its
// own user and not as root. A file under certs/ it may not open - usually
// the private key, root and Postfix's group only - is then left out
// instead of failing the whole archive, and OnWarn names it. The
// selfpost-backup command, run as root, leaves this unset and takes it.
Unprivileged bool
}
// QueueDir is the Postfix queue, relative to the data directory. No backup
@@ -164,11 +158,11 @@ func Create(w io.Writer, p Params) error {
return err
}
if err := addTree(tw, p.DataDir, DataArchivePrefix, excludedFromArchive, p.skipData()); err != nil {
if err := addTree(tw, p.DataDir, DataArchivePrefix, excludedFromArchive, skipQueue); err != nil {
return err
}
if err := addDeployFiles(tw, p.DeployRoot, p.OnWarn, p.skipCert()); err != nil {
if err := addDeployFiles(tw, p.DeployRoot, p.OnWarn); err != nil {
return err
}
@@ -204,36 +198,9 @@ func validateDeployRoot(deployRoot string) error {
// left out of the archive. A directory left out is not descended into.
type skipFunc func(path, rel string, d fs.DirEntry) bool
// skipData leaves the Postfix queue out of every archive.
func (p Params) skipData() skipFunc {
return func(_, rel string, _ fs.DirEntry) bool {
return filepath.ToSlash(rel) == QueueDir
}
}
// skipCert leaves a file under certs/ that cannot be opened out of an
// unprivileged archive, and says so through OnWarn.
func (p Params) skipCert() skipFunc {
if !p.Unprivileged {
return nil
}
return func(path, rel string, d fs.DirEntry) bool {
if d.IsDir() {
return false
}
f, err := os.Open(path)
if err == nil {
f.Close()
return false
}
if !errors.Is(err, fs.ErrPermission) {
return false
}
if p.OnWarn != nil {
p.OnWarn(CertsDirName + "/" + filepath.ToSlash(rel) + " is not readable by the panel and is left out of the backup")
}
return true
}
// skipQueue leaves the Postfix queue out of every archive.
func skipQueue(_, rel string, _ fs.DirEntry) bool {
return filepath.ToSlash(rel) == QueueDir
}
// Check reads through everything Create would put into the archive, without
@@ -241,6 +208,10 @@ func (p Params) skipCert() skipFunc {
// streams its archive to the browser, so once Create has begun a failure can
// only cut the download short; this is run first, while a refusal can still be
// shown as a page.
//
// The panel is not root, and nothing is left out on that account: its archive
// is the one selfpost-backup makes, or it is not made. What it most likely
// cannot read is the private key under certs/ - see UnreadableCert.
func Check(p Params) error {
if p.DataDir == "" || p.DBPath == "" {
return fmt.Errorf("backup: DataDir and DBPath are required")
@@ -248,15 +219,28 @@ func Check(p Params) error {
if err := ValidateDeployRoot(p.DeployRoot); err != nil {
return err
}
quiet := p
quiet.OnWarn = nil
if err := addTree(nil, p.DataDir, DataArchivePrefix, excludedFromArchive, quiet.skipData()); err != nil {
if err := addTree(nil, p.DataDir, DataArchivePrefix, excludedFromArchive, skipQueue); err != nil {
return err
}
return addDeployFiles(nil, p.DeployRoot, nil, quiet.skipCert())
return addDeployFiles(nil, p.DeployRoot, nil)
}
func addDeployFiles(tw *tar.Writer, deployRoot string, onWarn func(string), skip skipFunc) error {
// UnreadableCert reports whether err, from Check or Create, is a file under
// the deploy root's certs/ that the caller is not allowed to read, and names
// it relative to the deploy root ("certs/privkey.pem").
func UnreadableCert(err error, deployRoot string) (name string, ok bool) {
var pathErr *fs.PathError
if !errors.As(err, &pathErr) || !errors.Is(err, fs.ErrPermission) {
return "", false
}
rel, relErr := filepath.Rel(filepath.Join(deployRoot, CertsDirName), pathErr.Path)
if relErr != nil || rel == ".." || strings.HasPrefix(filepath.ToSlash(rel), "../") {
return "", false
}
return CertsDirName + "/" + filepath.ToSlash(rel), true
}
func addDeployFiles(tw *tar.Writer, deployRoot string, onWarn func(string)) error {
for _, name := range []string{ComposeFileName, EnvFileName} {
src := filepath.Join(deployRoot, name)
info, err := os.Stat(src)
@@ -281,7 +265,7 @@ func addDeployFiles(tw *tar.Writer, deployRoot string, onWarn func(string), skip
}
return fmt.Errorf("backup: stat %s: %w", CertsDirName, err)
}
return addTree(tw, certsDir, CertsDirName+"/", nil, skip)
return addTree(tw, certsDir, CertsDirName+"/", nil, nil)
}
// addTree walks root and adds every regular file (and directory, to preserve
internal/backup/backup_test.go
+29 -4
View File
@@ -5,7 +5,9 @@ import (
"bytes"
"compress/gzip"
"encoding/json"
"fmt"
"io"
"io/fs"
"os"
"path/filepath"
"strings"
@@ -236,11 +238,11 @@ func TestCheckRestoreWrongFormatRejected(t *testing.T) {
}
}
// The panel's archive: like any other it has no Postfix queue, the rest is
// there, and Check agrees that it can be made.
func TestCreateUnprivilegedLeavesTheQueueOut(t *testing.T) {
// No archive holds the Postfix queue; the rest is there, the private key
// included, and Check agrees that it can be made.
func TestCreateLeavesTheQueueOut(t *testing.T) {
dataDir, dbPath, deployRoot := seedDataDir(t)
p := Params{DataDir: dataDir, DBPath: dbPath, Version: "1.2.3", DeployRoot: deployRoot, Unprivileged: true}
p := Params{DataDir: dataDir, DBPath: dbPath, Version: "1.2.3", DeployRoot: deployRoot}
if err := Check(p); err != nil {
t.Fatalf("Check: %v", err)
@@ -262,6 +264,7 @@ func TestCreateUnprivilegedLeavesTheQueueOut(t *testing.T) {
DataArchivePrefix + "postfix/sender_login_maps",
ComposeFileName, EnvFileName,
CertsDirName + "/fullchain.pem",
CertsDirName + "/privkey.pem",
} {
if _, ok := files[name]; !ok {
t.Errorf("archive missing %s", name)
@@ -269,6 +272,28 @@ func TestCreateUnprivilegedLeavesTheQueueOut(t *testing.T) {
}
}
// A file under certs/ that may not be read is told apart from any other
// failure, so the panel can say what to do about it.
func TestUnreadableCert(t *testing.T) {
root := filepath.Join("srv", "deploy")
denied := func(path string) error {
return fmt.Errorf("backup: open %s: %w", path, &fs.PathError{Op: "open", Path: path, Err: fs.ErrPermission})
}
if name, ok := UnreadableCert(denied(filepath.Join(root, "certs", "privkey.pem")), root); !ok || name != "certs/privkey.pem" {
t.Errorf("certs/privkey.pem: got %q, %v", name, ok)
}
if name, ok := UnreadableCert(denied(filepath.Join(root, ".env")), root); ok {
t.Errorf(".env was taken for a certificate file: %q", name)
}
if name, ok := UnreadableCert(denied(filepath.Join("data", "postfix", "x")), root); ok {
t.Errorf("a data file was taken for a certificate file: %q", name)
}
missing := &fs.PathError{Op: "open", Path: filepath.Join(root, "certs", "privkey.pem"), Err: fs.ErrNotExist}
if _, ok := UnreadableCert(missing, root); ok {
t.Error("a missing file was taken for one that may not be read")
}
}
// Check finds what Create would stumble over, before anything is written.
func TestCheckReportsWhatCannotBeRead(t *testing.T) {
dataDir, dbPath, deployRoot := seedDataDir(t)
internal/web/handlers/handlers_backup.go
+19 -10
View File
@@ -70,14 +70,11 @@ func (h *Handlers) HandleBackup(w http.ResponseWriter, r *http.Request) {
h.renderBackupPageWith(w, r, http.StatusBadRequest, "", pwErr)
return
}
// The panel is not root: its archive leaves out the Postfix queue and a
// certificate file it may not open (backup.Params.Unprivileged).
params := backup.Params{
DataDir: h.cfg.DataDir,
DBPath: h.cfg.DBPath,
Version: h.cfg.Version,
DeployRoot: h.cfg.DeployRoot,
Unprivileged: true,
DataDir: h.cfg.DataDir,
DBPath: h.cfg.DBPath,
Version: h.cfg.Version,
DeployRoot: h.cfg.DeployRoot,
OnWarn: func(msg string) {
logf("panel: full backup: %s", msg)
},
@@ -90,11 +87,11 @@ func (h *Handlers) HandleBackup(w http.ResponseWriter, r *http.Request) {
// Everything is read through once before the first byte goes out. Past the
// headers a failure can only cut the download short - which is what the
// button did from 1.3.0 to 2.0.0, with a 200 and ten bytes, because the
// queue was not the panel's to read.
// queue was not the panel's to read. Nothing is skipped to make the archive
// possible: it is the one selfpost-backup makes, or it is refused.
if err := backup.Check(params); err != nil {
logf("panel: full backup: %v", err)
h.renderBackupPageWith(w, r, http.StatusInternalServerError, "",
"Could not read everything a backup holds. Nothing was downloaded; the reason is in the panel's log.")
h.renderBackupPageWith(w, r, http.StatusInternalServerError, "", unreadableBackupErr(err, h.cfg.DeployRoot))
return
}
@@ -333,6 +330,18 @@ func decryptErrorMessage(err error) string {
}
}
// unreadableBackupErr phrases what backup.Check found. The usual case is the
// private key: the panel runs as its own user, and a key readable by root
// alone is not its to read. The way out is on the host, so the message says
// what to do there (guide § Taking a full backup).
func unreadableBackupErr(err error, deployRoot string) string {
if name, ok := backup.UnreadableCert(err, deployRoot); ok {
return "The panel is not allowed to read " + name + ", and a backup without it would not bring the server back. " +
"On the host, give the file to group 998 and make it group-readable (chgrp 998, chmod 640), then try again. Nothing was downloaded."
}
return "Could not read everything a backup holds. Nothing was downloaded; the reason is in the panel's log."
}
// deployBackupErr phrases a pre-flight backup failure for the operator.
func deployBackupErr(err error) string {
if strings.Contains(err.Error(), "DeployRoot") || strings.Contains(err.Error(), "deploy root") {
internal/web/view/help.go
+2 -1
View File
@@ -193,7 +193,8 @@ var serverTopics = []HelpTopic{
". The Postfix queue is not in it. The backup is taken while the server is running, and Postfix keeps moving messages through the queue, "+
"so a copy could hold a message twice or miss it; mail still waiting for delivery is therefore not carried over, and a restored server "+
"starts with an empty queue. To keep the queue, stop the container and archive the project directory instead. The private key in ", Code("certs/"),
" is usually missing too, because the panel is not allowed to read it; ", Code("selfpost-backup"), " run in the container takes the key. It holds secrets, so leave Encrypt with a password ticked: the file is sealed as ", Code(".spbk"),
" has to be readable by the panel: give it to group 998 on the host (", Code("chgrp 998"), ", ", Code("chmod 640"),
"). Until then the panel refuses to make a backup and names the file, rather than hand over an archive without the key. It holds secrets, so leave Encrypt with a password ticked: the file is sealed as ", Code(".spbk"),
", the password is not stored, and without it nothing can be recovered. To restore, unpack it into an empty project directory (decrypt a ",
Code(".spbk"), " first with ", Code("selfpost-backup -decrypt"),
") and start the same SelfPost version that made it; the panel refuses data from another version. The reverse-proxy configuration and ",
internal/web/view/templates/backup.html
+1 -1
View File
@@ -13,7 +13,7 @@
<div class="column is-6">
{{template "box_open" .Full}}
{{template "body_open"}}
<p class="mb-3"><code>data/</code> (database, DKIM keys, credentials), <code>docker-compose.yml</code>, <code>.env</code> and <code>certs/</code> in one archive. The Postfix queue is left out: it cannot be copied reliably while the server is running. So is a certificate file the panel may not read. Restore into an empty project directory and start the <strong>same SelfPost version</strong>. The reverse-proxy vhost is not included.</p>
<p class="mb-3"><code>data/</code> (database, DKIM keys, credentials), <code>docker-compose.yml</code>, <code>.env</code> and <code>certs/</code> in one archive. The Postfix queue is left out: it cannot be copied reliably while the server is running. Restore into an empty project directory and start the <strong>same SelfPost version</strong>. The reverse-proxy vhost is not included.</p>
<form class="sp-form" method="post" action="{{.FullAction}}">
<div class="field"><label class="checkbox"><input type="checkbox" name="encrypt" value="1"{{if .Encrypt}} checked{{end}} data-encrypt-toggle> Encrypt with a password (<code>.spbk</code>)</label></div>
<div data-encrypt-fields>