From 6a7d010868b9a901f70b2cef863c499dbb7a17a5 Mon Sep 17 00:00:00 2001 From: Mikhail Yenuchenko Date: Thu, 6 Aug 2026 16:43:28 +0300 Subject: [PATCH] =?UTF-8?q?feat:=20optional=20password=20encryption=20for?= =?UTF-8?q?=20backup=20and=20domain=20export=20(code-review.md=20=C2=A7=20?= =?UTF-8?q?Phase=201.5)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Both secret-bearing downloads can now be sealed with a password. Unticked, the forms produce exactly the files they did before. - internal/secretfile: envelope format — magic/type/scrypt params/salt/nonce prefix header, then 64 KiB AES-256-GCM chunks each authenticated with the header, its counter and an end-of-stream flag, so truncation, reordering and tampering fail to open instead of restoring a plausible prefix. Streams both ways, so a full backup never sits in memory. - Panel: "Encrypt with a password" checkbox on the full-backup and domain-export forms (shared partial, toggled from panel.js — no inline script); domain import detects an encrypted export by magic bytes, not by extension, and asks for the password. - selfpost-backup: writes .spbk when given a password and converts one back with -decrypt, which a restore needs. The password comes from SELFPOST_BACKUP_PASSWORD or -password-file, never argv. - Docs: README, security.md (+ accepted risk: encryption stays opt-in), architecture.md, progress.md, CHANGELOG. Verified locally: panel-encrypted archive decrypts through the CLI and unpacks; wrong password and password mismatch are refused; UI checked in a browser. Co-Authored-By: Claude Opus 5 --- CHANGELOG.md | 18 + README.md | 38 +- cmd/selfpost-backup/main.go | 137 +++++- cmd/selfpost-backup/main_test.go | 139 +++++++ docs/architecture.md | 9 + docs/code-review.md | 17 +- docs/progress.md | 3 +- docs/security.md | 25 ++ internal/secretfile/secretfile.go | 458 +++++++++++++++++++++ internal/secretfile/secretfile_test.go | 162 ++++++++ internal/web/handlers_apps.go | 4 + internal/web/handlers_backup.go | 188 ++++++++- internal/web/handlers_backup_test.go | 127 ++++++ internal/web/static/panel.css | 19 + internal/web/static/panel.js | 31 ++ internal/web/templates.go | 8 +- internal/web/templates/backup.html | 19 +- internal/web/templates/domain_detail.html | 7 +- internal/web/templates/encrypt_fields.html | 23 ++ internal/web/validate.go | 6 + 20 files changed, 1402 insertions(+), 36 deletions(-) create mode 100644 cmd/selfpost-backup/main_test.go create mode 100644 internal/secretfile/secretfile.go create mode 100644 internal/secretfile/secretfile_test.go create mode 100644 internal/web/handlers_backup_test.go create mode 100644 internal/web/templates/encrypt_fields.html diff --git a/CHANGELOG.md b/CHANGELOG.md index 3082730..cd56291 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -28,6 +28,24 @@ Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); version ### Added +- Optional password encryption for the two secret-bearing downloads (plan + phase 1.5, `docs/code-review.md`): an *Encrypt with a password* checkbox on + the full-backup and domain-export forms writes a `.spbk` / `.spde` envelope + instead of the plain `.tar.gz` / `.json` — scrypt key derivation and + AES-256-GCM over 64 KiB chunks, each authenticated with the header, its + counter and an end-of-stream flag, so a truncated or altered file refuses to + open (`internal/secretfile`). Unticked, both downloads are byte-for-byte what + they were. +- Domain import accepts an encrypted export: the envelope is detected by its + magic bytes, and a password field appears next to the file picker + (`internal/web/handlers_backup.go`, `templates/encrypt_fields.html`). +- `selfpost-backup` writes encrypted archives and reads them back: + `-decrypt` (with `-i`/`-o`) turns a `.spbk` into the plain `.tar.gz` a + restore unpacks. The password comes from `SELFPOST_BACKUP_PASSWORD` or + `-password-file`, never from argv. +- docs: README *Encrypting a backup or export*; `docs/security.md` § + *Резервная копия и экспорт домена* + accepted risk (encryption is opt-in); + `docs/architecture.md` persistence § envelope summary. - docs: `docs/roadmap.md` v1.x tail — retire `implementation-plan.md` in the release commit (move to `docs/archive/`, retarget its references in README, docs, Makefile, release workflow and the e2e test comment). diff --git a/README.md b/README.md index 2dc37c9..d3481e9 100644 --- a/README.md +++ b/README.md @@ -296,7 +296,43 @@ Two related but distinct operations — spec 7.5: Both files are **secrets** — they contain the admin password hash (full backup) or working application credentials (domain export) in the clear or in directly reversible form. Treat them like any other credential material: -encrypt at rest, restrict who can read them, don't email them around. +restrict who can read them, don't email them around — and encrypt them, which +SelfPost can do for you. + +### Encrypting a backup or export + +Both download forms carry an **Encrypt with a password** checkbox. Ticked, the +file that comes down is an encrypted envelope instead of the plain archive: + +| Artefact | Plain | Encrypted | +|----------|-------|-----------| +| Full backup | `.tar.gz` | `.spbk` | +| Domain export | `.json` | `.spde` | + +The key is derived from the password with scrypt and the contents are sealed +with AES-256-GCM, in chunks, so a truncated or altered file fails to open rather +than restoring quietly. **SelfPost does not store the password** — lose it and +the file is unrecoverable, which is the entire point. + +*Import a domain* takes an encrypted export directly: tick **The file is +encrypted** and give the password. + +A full backup has to be turned back into a plain archive before it can be +unpacked into `/data`, which the CLI does with the same password: + +```sh +docker exec -i selfpost-backup -decrypt < backup.spbk > backup.tar.gz +``` + +The CLI also *writes* encrypted backups for scripted/cron use. The password +comes from `SELFPOST_BACKUP_PASSWORD` or `-password-file ` (first line), +never from a command-line argument, which would be visible in the process list: + +```sh +docker exec -e SELFPOST_BACKUP_PASSWORD="$PW" selfpost-backup > backup.spbk +``` + +With no password set, the CLI keeps writing the plain `.tar.gz` it always has. ## Published ports diff --git a/cmd/selfpost-backup/main.go b/cmd/selfpost-backup/main.go index a84434e..c90858b 100644 --- a/cmd/selfpost-backup/main.go +++ b/cmd/selfpost-backup/main.go @@ -9,21 +9,39 @@ // Use -o to write to a file instead. The resulting archive contains DKIM private // keys, the admin password hash and SASL credentials — treat it as a secret // (spec 7.5.A). +// +// Given a password (SELFPOST_BACKUP_PASSWORD or -password-file, never an +// argument, which would show up in the process list) the archive is written as +// an encrypted .spbk envelope instead. Turn one back into a plain .tar.gz with +// the same password: +// +// docker exec -i selfpost-backup -decrypt < backup.spbk > backup.tar.gz package main import ( "flag" "fmt" + "io" "os" "path/filepath" + "strings" "codeberg.org/mix/selfpost/internal/backup" "codeberg.org/mix/selfpost/internal/buildinfo" + "codeberg.org/mix/selfpost/internal/secretfile" ) +// passwordEnv names the environment variable holding the encryption password. +// A password must never be a command-line argument: the process list is +// readable by every process in the container. +const passwordEnv = "SELFPOST_BACKUP_PASSWORD" + func main() { showVersion := flag.Bool("version", false, "print version and exit") - out := flag.String("o", "", "write the archive to this file instead of stdout") + out := flag.String("o", "", "write the output to this file instead of stdout") + in := flag.String("i", "", "read the encrypted archive from this file instead of stdin (-decrypt only)") + decrypt := flag.Bool("decrypt", false, "decrypt an encrypted backup (.spbk) back to a plain .tar.gz") + pwFile := flag.String("password-file", "", "read the encryption password from this file (first line); "+passwordEnv+" is used when unset") flag.Parse() if *showVersion { @@ -31,40 +49,137 @@ func main() { return } - if err := run(*out); err != nil { + password, err := readPassword(*pwFile) + if err == nil { + if *decrypt { + err = runDecrypt(*in, *out, password) + } else { + err = run(*out, password) + } + } + if err != nil { fmt.Fprintf(os.Stderr, "selfpost-backup: %v\n", err) os.Exit(1) } } -func run(outPath string) error { +// run writes a backup, encrypting it when a password was supplied. +func run(outPath, password string) error { dataDir := envDefault("SELFPOST_DATA_DIR", "/data") dbPath := envDefault("SELFPOST_DB_PATH", filepath.Join(dataDir, "selfpost.db")) - w := os.Stdout - if outPath != "" { - // Backups are secret; create them owner-only. - f, err := os.OpenFile(outPath, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, 0o600) + w, closeOut, err := openOutput(outPath) + if err != nil { + return err + } + defer closeOut() + + sink := w + var env *secretfile.Writer + if password != "" { + env, err = secretfile.NewWriter(w, secretfile.TypeFullBackup, password) if err != nil { return err } - defer f.Close() - w = f + sink = env } - if err := backup.Create(w, backup.Params{ + if err := backup.Create(sink, backup.Params{ DataDir: dataDir, DBPath: dbPath, Version: buildinfo.Version, }); err != nil { return err } + if env != nil { + if err := env.Close(); err != nil { + return err + } + } if outPath != "" { - fmt.Fprintf(os.Stderr, "selfpost-backup: wrote %s (SelfPost %s)\n", outPath, buildinfo.Version) + kind := "plain" + if password != "" { + kind = "encrypted" + } + fmt.Fprintf(os.Stderr, "selfpost-backup: wrote %s (%s, SelfPost %s)\n", outPath, kind, buildinfo.Version) } return nil } +// runDecrypt turns a .spbk envelope back into the plain gzip tar, so an +// encrypted backup can be extracted with ordinary tools during a restore. +func runDecrypt(inPath, outPath, password string) error { + if password == "" { + return fmt.Errorf("-decrypt needs the password (set %s or use -password-file)", passwordEnv) + } + + r := io.Reader(os.Stdin) + if inPath != "" { + f, err := os.Open(inPath) + if err != nil { + return err + } + defer f.Close() + r = f + } + + env, err := secretfile.NewReader(r, password) + if err != nil { + return err + } + if env.Type() != secretfile.TypeFullBackup { + return fmt.Errorf("that file is a %s, not a full backup", env.Type()) + } + + w, closeOut, err := openOutput(outPath) + if err != nil { + return err + } + defer closeOut() + + if _, err := io.Copy(w, env); err != nil { + return err + } + if outPath != "" { + fmt.Fprintf(os.Stderr, "selfpost-backup: wrote %s (decrypted)\n", outPath) + } + return nil +} + +// openOutput returns stdout, or a freshly created owner-only file: both plain +// and encrypted backups are secret enough to keep off other users' eyes. +func openOutput(outPath string) (io.Writer, func(), error) { + if outPath == "" { + return os.Stdout, func() {}, nil + } + f, err := os.OpenFile(outPath, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, 0o600) + if err != nil { + return nil, nil, err + } + return f, func() { _ = f.Close() }, nil +} + +// readPassword takes the password from the given file (first line) or, when no +// file is named, from the environment. An empty result means "no encryption". +func readPassword(pwFile string) (string, error) { + if pwFile == "" { + return os.Getenv(passwordEnv), nil + } + data, err := os.ReadFile(pwFile) + if err != nil { + return "", fmt.Errorf("read password file: %w", err) + } + // A password file is usually written with a trailing newline; take the first + // line and strip the line ending, but keep any other whitespace, which may + // well be part of the password. + line, _, _ := strings.Cut(string(data), "\n") + line = strings.TrimSuffix(line, "\r") + if line == "" { + return "", fmt.Errorf("password file %s is empty", pwFile) + } + return line, nil +} + func envDefault(key, def string) string { if v := os.Getenv(key); v != "" { return v diff --git a/cmd/selfpost-backup/main_test.go b/cmd/selfpost-backup/main_test.go new file mode 100644 index 0000000..bd2780b --- /dev/null +++ b/cmd/selfpost-backup/main_test.go @@ -0,0 +1,139 @@ +package main + +import ( + "archive/tar" + "compress/gzip" + "io" + "os" + "path/filepath" + "strings" + "testing" + + "codeberg.org/mix/selfpost/internal/store" +) + +// seedDataDir builds the minimum /data tree a backup can be taken from. +func seedDataDir(t *testing.T) string { + t.Helper() + dataDir := t.TempDir() + st, err := store.Open(filepath.Join(dataDir, "selfpost.db")) + if err != nil { + t.Fatalf("open store: %v", err) + } + if _, err := st.AddDomain("example.com", "selfpost"); err != nil { + t.Fatalf("add domain: %v", err) + } + if err := st.Close(); err != nil { + t.Fatalf("close store: %v", err) + } + t.Setenv("SELFPOST_DATA_DIR", dataDir) + t.Setenv("SELFPOST_DB_PATH", filepath.Join(dataDir, "selfpost.db")) + return dataDir +} + +// An encrypted backup is only worth having if the container it came from can +// hand it back as an ordinary archive during a restore, so the two halves of +// the CLI are tested as the one round trip an operator actually performs. +func TestEncryptedBackupRoundTrip(t *testing.T) { + seedDataDir(t) + dir := t.TempDir() + encrypted := filepath.Join(dir, "backup.spbk") + plain := filepath.Join(dir, "backup.tar.gz") + const password = "a long enough password" + + if err := run(encrypted, password); err != nil { + t.Fatalf("create encrypted backup: %v", err) + } + head, err := os.ReadFile(encrypted) + if err != nil { + t.Fatalf("read backup: %v", err) + } + if !strings.HasPrefix(string(head), "SELFPOST") { + t.Fatalf("encrypted backup does not start with the envelope magic") + } + + if err := runDecrypt(encrypted, plain, "the wrong password"); err == nil { + t.Fatal("decryption with the wrong password succeeded") + } + if err := runDecrypt(encrypted, plain, password); err != nil { + t.Fatalf("decrypt: %v", err) + } + + // What comes out must be the same gzip tar the plain path produces. + f, err := os.Open(plain) + if err != nil { + t.Fatalf("open decrypted archive: %v", err) + } + defer f.Close() + gz, err := gzip.NewReader(f) + if err != nil { + t.Fatalf("gzip: %v", err) + } + names := map[string]bool{} + tr := tar.NewReader(gz) + for { + hdr, err := tr.Next() + if err == io.EOF { + break + } + if err != nil { + t.Fatalf("tar: %v", err) + } + names[hdr.Name] = true + } + for _, want := range []string{"manifest.json", "selfpost.db"} { + if !names[want] { + t.Errorf("decrypted archive has no %s (entries: %v)", want, names) + } + } +} + +// Without a password the CLI keeps producing the plain archive that existing +// backup scripts consume. +func TestUnencryptedBackupStaysPlain(t *testing.T) { + seedDataDir(t) + out := filepath.Join(t.TempDir(), "backup.tar.gz") + if err := run(out, ""); err != nil { + t.Fatalf("create backup: %v", err) + } + f, err := os.Open(out) + if err != nil { + t.Fatalf("open archive: %v", err) + } + defer f.Close() + if _, err := gzip.NewReader(f); err != nil { + t.Fatalf("plain backup is not a gzip archive: %v", err) + } +} + +// Decrypting needs a password, and it must come from a file or the environment +// — never an argument, which the process list would expose. +func TestReadPassword(t *testing.T) { + dir := t.TempDir() + pwFile := filepath.Join(dir, "pw") + if err := os.WriteFile(pwFile, []byte("from the file\nignored second line\n"), 0o600); err != nil { + t.Fatalf("write password file: %v", err) + } + + t.Setenv(passwordEnv, "from the environment") + got, err := readPassword("") + if err != nil || got != "from the environment" { + t.Errorf("readPassword(\"\") = %q, %v", got, err) + } + got, err = readPassword(pwFile) + if err != nil || got != "from the file" { + t.Errorf("readPassword(file) = %q, %v", got, err) + } + + if err := os.WriteFile(pwFile, nil, 0o600); err != nil { + t.Fatalf("truncate password file: %v", err) + } + if _, err := readPassword(pwFile); err == nil { + t.Error("an empty password file was accepted") + } + + os.Unsetenv(passwordEnv) + if err := runDecrypt("", "", ""); err == nil { + t.Error("-decrypt without a password was accepted") + } +} diff --git a/docs/architecture.md b/docs/architecture.md index 69f723e..7a9c6ad 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -176,6 +176,15 @@ Not in `/data`: TLS certificates (reverse-proxy mount), Postfix queue `/data` tree; version check on restore. Stopped-container `tar` of `./data` is safe (see README). +**Optional encryption** of the two secret-bearing downloads +([internal/secretfile](../internal/secretfile/secretfile.go)): password → +scrypt → AES-256-GCM over 64 KiB chunks, each authenticated with the header, +its counter and an end-of-stream flag (so truncation and reordering fail to +open). Full backup `.tar.gz` → `.spbk`, domain export `.json` → `.spde`; the +plain forms remain the default. Domain import detects the envelope by magic +bytes; an encrypted full backup is converted back with `selfpost-backup +-decrypt` before restore. + --- ## Security (summary) diff --git a/docs/code-review.md b/docs/code-review.md index acbec9a..62e27f7 100644 --- a/docs/code-review.md +++ b/docs/code-review.md @@ -219,7 +219,7 @@ E2E покрывает: bootstrap, SMTP AUTH, DKIM, send-log lifecycle, negative | CSRF without tokens | POST без Origin/Sec-Fetch-Site пропускается | [security.md](security.md) | | Fail-open L2 rate limit | DB error → mail проходит | [`internal/milter/ratelimit.go`](../internal/milter/ratelimit.go) | | Shallow SPF check | Не следует `include:`/`redirect=` | README, `internal/dnscheck/spf.go` | -| Plaintext backup/export at rest | DKIM-ключи, SASL, пароли приложений в cleartext `.tar.gz`/`.json` | **Mitigation:** R13 (optional encryption) | +| Plaintext backup/export at rest | DKIM-ключи, SASL, пароли приложений в cleartext `.tar.gz`/`.json` | **Закрыто:** R13 — опциональное шифрование (`.spbk`/`.spde`); открытый вариант остаётся умолчанием, риск переформулирован в [security.md](security.md) | **Не риск (решение оператора):** «Session resurrection from backup» — снято из [security.md](security.md). @@ -234,7 +234,7 @@ E2E покрывает: bootstrap, SMTP AUTH, DKIM, send-log lifecycle, negative | # | Действие | Приоритет | Модель | |---|----------|-----------|--------| | L1 | **Предрелизный security review** (§ D) — обязательный гейт | **P0** | **Fable** | -| L2 | **Шифрование бэкапа и экспорта домена** (R13) — optional, checkbox + password | P1 | **Opus** + Sonnet | +| L2 | **Шифрование бэкапа и экспорта домена** (R13) — optional, checkbox + password — **выполнено** | P1 | **Opus** + Sonnet | | L3 | Send-log gap mitigation — опционально | P2 | Opus | | L4 | Transaction wrap для rate limit count+insert — опционально | P3 | Opus | @@ -352,7 +352,7 @@ Go `html/template` + HTMX polling + [`panel.css`](../internal/web/static/panel.c | R6 | GUI: visibility-aware HTMX polling | Sonnet | | R7 | CONTRIBUTING.md | Sonnet | | R8 | ADR для CSRF policy | Sonnet | -| R13 | Шифрование бэкапа и экспорта домена (checkbox + password) | **Opus** + Sonnet | +| R13 | Шифрование бэкапа и экспорта домена (checkbox + password) — **выполнено** | **Opus** + Sonnet | ### v2.x (roadmap, не начинать без согласования) @@ -384,7 +384,16 @@ Go `html/template` + HTMX polling + [`panel.css`](../internal/web/static/panel.c 5. Sonnet: bump compose image tag + Codeberg URLs (в том же release commit) 6. Git tag vX.Y.Z -### Фаза 1.5 — Шифрование резервных копий (P1, v1.x) +### Фаза 1.5 — Шифрование резервных копий (P1, v1.x) — **выполнено 2026-08-06** + +Реализовано как спланировано: `internal/secretfile` (E1) → `selfpost-backup` ++ панель (E2) → экспорт/импорт домена (E3) → UI-чекбокс (E4) → docs (E5). +Отличия от плана: конверт потоковый (64 KiB чанки AES-256-GCM с AAD +`header+counter+last`), а не одноблочный, иначе полный бэкап пришлось бы +держать в памяти целиком; манифест остался внутри tar, то есть внутри +шифротекста, как и планировалось; в CLI добавлен режим `-decrypt` — без него +зашифрованный бэкап нечем распаковать при restore. Детали — +[progress.md](progress.md), CHANGELOG `[Unreleased]`. **Проблема:** полный бэкап и экспорт домена содержат DKIM-ключи, SASL-креды и plaintext-пароли приложений; сейчас `.tar.gz` / `.json` без шифрования. diff --git a/docs/progress.md b/docs/progress.md index 85baab2..7d3321b 100644 --- a/docs/progress.md +++ b/docs/progress.md @@ -51,7 +51,8 @@ - **Рецензирование кодовой базы** (2026-08-05): [code-review.md](code-review.md) — 10 разделов (архитектура, качество, docs, GUI, legacy, риски), приоритизированный план реализации и маршрутизация моделей. Критичных багов не найдено; блокер релиза — § D ниже. - **§ D выполнен (2026-08-06):** предрелизная ревизия безопасности моделью Fable — диф от аудита v1.0 (Фаза 11, `bd64e80`) до HEAD + полный проход по чек-листу [security.md](security.md) (бывшее ТЗ 7.6). Эксплуатируемых находок нет; одна правка defence-in-depth (`--` перед логином в argv `saslpasswd2`, `internal/app/sasl.go` + тест). Принятые риски не пополнились. Детали — [implementation-plan.md](implementation-plan.md) § D и CHANGELOG `[Unreleased]/Security`. Локально `go vet`/`go test ./internal/app/...` чистые; падения `internal/domain` (`TestWriteLoadPrivateKeyRoundtrip`, `TestRenderTables`) и `internal/logtail` (`TestFollowTailsAndRotates`) — Windows-специфика (права файлов/`\` в путях/rename открытого файла), на Linux CI зелено. - **Фаза 1 выполнена (2026-08-06)** ([code-review.md](code-review.md) § Фаза 1 — doc/code hygiene, P1): cleanup ~30 stale «Phase N» комментариев в коде и shell-скриптах; исправлен stale-комментарий в `handlers_domains.go`; ADR CSRF (Origin vs токены) добавлен в [security.md](security.md); known-limitations по log-tailer уже был в [architecture.md](architecture.md) § Log tailer — отдельного действия не потребовалось; `docs/logo` в [roadmap.md](roadmap.md) закрыт (каталога нет, критерию соответствует); `gofmt -l` добавлен в CI (`.github/workflows/test.yml`). `gofmt`/`go vet`/`go test ./...` чистые в обоих модулях (dev-server). -- **Дальше:** релизный гейт (Фаза 0) закрыт по существу — e2e C.4 и ревизия § D пройдены; остаются только шаги, которые делаются в момент резки версии (бамп тега образа + Codeberg URL, git tag) по явной команде пользователя. Остальные пункты из [code-review.md](code-review.md) — polish (фазы 2–3, 1.5). +- **Фаза 1.5 выполнена (2026-08-06)** ([code-review.md](code-review.md) § Фаза 1.5 — шифрование резервных копий, P1): новый пакет `internal/secretfile` — конверт `magic SELFPOST1 | type | scrypt-параметры | salt | nonce-prefix` + поток 64 KiB чанков AES-256-GCM, каждый с AAD `header+counter+last`, поэтому обрезка, перестановка и подмена не открываются (стриминг в обе стороны — полный бэкап не держится в памяти). Панель: чекбокс «Encrypt with a password» в форме полного бэкапа и экспорта домена (общий партиал `templates/encrypt_fields.html`, показ/очистка полей — `panel.js`, без inline-скриптов), импорт домена принимает `.spde` (шифрование определяется по magic, не по расширению) с полем пароля. CLI `selfpost-backup`: пишет `.spbk` при заданном пароле и умеет `-decrypt` (иначе зашифрованный бэкап нечем распаковать при restore); пароль — только `SELFPOST_BACKUP_PASSWORD` / `-password-file`, никогда argv. Умолчание не изменилось: галочка снята — прежние `.tar.gz` / `.json` байт в байт. Тесты: round-trip по размерам (0, границы чанка, несколько чанков), неверный пароль, обрезка, перестановка чанков, порча байта, чужие KDF-параметры; валидация формы пароля; round-trip CLI create→decrypt→tar. Docs: README § *Encrypting a backup or export*, [security.md](security.md) § «Резервная копия и экспорт домена» + принятый риск (шифрование опционально), [architecture.md](architecture.md) § Persistence. `gofmt`/`go vet`/`go test ./...` чистые (кроме известных Windows-падений `internal/domain`, `internal/logtail`). E2E-сценарий не добавлялся: в `test/e2e/` бэкапа не было и раньше, а прогнать новый тест локально нечем (нет Docker) — кандидат при следующем прогоне на dev-сервере. +- **Дальше:** релизный гейт (Фаза 0) закрыт по существу — e2e C.4 и ревизия § D пройдены; остаются только шаги, которые делаются в момент резки версии (бамп тега образа + Codeberg URL, git tag) по явной команде пользователя. Остальные пункты из [code-review.md](code-review.md) — polish (фазы 2–3). - **Принятые риски** — [security.md](security.md). **Опционально v1.x / 2.x** — [roadmap.md](roadmap.md) (хвост документации, send-log gaps, Фаза O1+, роль администратора домена). - **Прод:** `selfpost.example.com`, реальный Let's Encrypt сертификат, живой e2e (DKIM/SPF pass). Контейнер там всё ещё на образе v1.0 — Фаза 14 в него не выкатывалась. При апгрейде: админа один раз разлогинит (сменилось имя cookie), а от reverse-proxy требуется передача исходного `Host` (Apache-фрагмент из `deploy/` это делает). diff --git a/docs/security.md b/docs/security.md index 6c4d653..295c4dd 100644 --- a/docs/security.md +++ b/docs/security.md @@ -70,6 +70,25 @@ Hardening сверх обязательного (security-заголовки, п - `TRUSTED_PROXY_CIDR` — только явно доверенные прокси для `X-Forwarded-For` при rate-limit логина; пусто = XFF игнорируется. +### Резервная копия и экспорт домена + +- Оба файла — секреты: полный бэкап несёт DKIM-ключи, `sasldb2` и хеш пароля + админа; экспорт домена — DKIM-ключ и **рабочие** пароли приложений открытым + текстом (иначе перенос без пересоздания кредов невозможен). +- Оба скачивания можно зашифровать паролем (чекбокс в форме): scrypt + (N=2¹⁵, r=8, p=1) → AES-256-GCM, поток из 64 KiB чанков, каждый + аутентифицирован заголовком, номером и флагом конца потока — обрезанный или + подменённый файл не открывается вместо тихого восстановления «хвоста». + Формат и обёртка: [internal/secretfile](../internal/secretfile/secretfile.go). +- Расширения: `.spbk` (полный бэкап), `.spde` (экспорт домена); незашифрованные + остаются `.tar.gz` / `.json`. Импорт домена определяет шифрование по magic + файла, а не по расширению. +- Пароль нигде не сохраняется: восстановить файл без него нельзя. Пароль в CLI — + только через `SELFPOST_BACKUP_PASSWORD` или `-password-file`, никогда + аргументом (список процессов читается любым процессом контейнера). +- Минимальная длина пароля — как у пароля администратора (12): файл лежит + offline и подбирается без ограничений по времени. + --- ## Принятые риски @@ -93,6 +112,12 @@ Hardening сверх обязательного (security-заголовки, п панели, отправит запрос сам — против этого работают автоэкранирование `html/template` и CSP, поэтому шаблоны не должны содержать inline-скриптов и inline-стилей. +- **Шифрование бэкапа и экспорта — опция, а не умолчание.** Галочка снята — + файл скачивается открытым, как в 1.0. Иначе оператор, у которого нет места + для хранения пароля, потерял бы возможность сделать бэкап вообще, а + безвозвратно нерасшифровываемый архив хуже незашифрованного: пароль SelfPost + не хранит. Триггером сделать шифрование обязательным считать появление + второго администратора (тогда «кто скачал» перестаёт быть одним человеком). - **Send-log может навсегда остаться `queued` после рестарта панели или пересоздания контейнера.** Log-tailer стартует с конца `mail.log`; файл не в `/data` и теряется при recreate. Rename-ротация это не лечит. См. [architecture.md](architecture.md) diff --git a/internal/secretfile/secretfile.go b/internal/secretfile/secretfile.go new file mode 100644 index 0000000..e8c46c6 --- /dev/null +++ b/internal/secretfile/secretfile.go @@ -0,0 +1,458 @@ +// Package secretfile wraps SelfPost's secret-bearing downloads — the full +// server backup and the single-domain export — in a password-encrypted +// envelope. Both artefacts carry DKIM private keys, SASL credentials and +// application passwords in the clear, so an operator who stores them outside +// the server (the whole point of a backup) has to protect them by hand. The +// envelope makes that optional-but-easy: tick a box, give a password, and the +// file that leaves the panel is useless without it. +// +// Layout — a fixed header followed by a sequence of independently +// authenticated chunks, so both writing and reading stream and a multi-megabyte +// backup never has to sit in memory: +// +// magic 9 "SELFPOST1" (the trailing digit is the envelope version) +// type 1 PayloadType: what the plaintext is +// kdf 1 KDF identifier (1 = scrypt) +// logN 1 scrypt cost, log2(N) +// r 4 scrypt block size (big-endian) +// p 4 scrypt parallelisation (big-endian) +// salt 16 scrypt salt +// prefix 8 nonce prefix +// chunks … repeated: length (4, big-endian) + AES-256-GCM ciphertext +// +// The KDF parameters travel in the header so a file encrypted today still opens +// after the cost is raised. Every chunk is sealed with the nonce prefix plus its +// own counter, and takes the whole header, that counter and an end-of-stream +// flag as additional data: a chunk cannot be reordered, swapped between files, +// or dropped from the end without decryption failing. +package secretfile + +import ( + "crypto/aes" + "crypto/cipher" + "crypto/rand" + "encoding/binary" + "errors" + "fmt" + "io" + + "golang.org/x/crypto/scrypt" +) + +// PayloadType identifies what a decrypted envelope contains, so opening a file +// can report "that is a domain export, not a backup" instead of handing an +// unexpected payload to a parser. +type PayloadType byte + +const ( + // TypeFullBackup is the gzip-compressed tar of a full server backup. + TypeFullBackup PayloadType = 1 + // TypeDomainExport is the JSON document of a single-domain export. + TypeDomainExport PayloadType = 2 +) + +// String names a payload type for operator-facing messages. +func (t PayloadType) String() string { + switch t { + case TypeFullBackup: + return "full backup" + case TypeDomainExport: + return "domain export" + default: + return fmt.Sprintf("unknown type %d", byte(t)) + } +} + +// File name suffixes for the two encrypted artefacts: SelfPost BacKup and +// SelfPost Domain Export. They exist so an operator can tell an encrypted file +// from a plain .tar.gz/.json at a glance; nothing reads them back — the magic +// bytes decide. +const ( + ExtBackup = ".spbk" + ExtDomainExport = ".spde" +) + +// magic prefixes every envelope. MagicLen bytes are enough to tell an encrypted +// file from a plain one, which is what an import handler needs to decide whether +// to ask for a password. +var magic = []byte("SELFPOST1") + +// MagicLen is the number of leading bytes HasMagic inspects. +const MagicLen = 9 + +const ( + kdfScrypt = 1 + + saltLen = 16 + noncePrefixLen = 8 + counterLen = 4 + headerLen = MagicLen + 1 + 1 + 1 + 4 + 4 + saltLen + noncePrefixLen + + // chunkSize is the plaintext carried by one sealed chunk. 64 KiB keeps the + // per-chunk overhead negligible while bounding the buffer a reader must + // allocate for a hostile length field. + chunkSize = 64 * 1024 + tagLen = 16 + keyLen = 32 // AES-256 +) + +// Default scrypt parameters: 32 MiB and roughly a tenth of a second per attempt +// on ordinary hardware. The panel derives a key at most once per download, so +// the cost is invisible to the operator and expensive for anyone brute-forcing a +// stolen backup. +const ( + defaultLogN = 15 + defaultR = 8 + defaultP = 1 + + // maxLogN caps what a file may ask for, so a hostile header cannot make the + // reader allocate its way to an out-of-memory kill before it ever fails + // authentication. + maxLogN = 20 + maxR = 32 + maxP = 16 +) + +var ( + // ErrNotEncrypted reports a file that is not a SelfPost envelope at all — + // most often a plain .json export handed to the encrypted path. + ErrNotEncrypted = errors.New("secretfile: not an encrypted SelfPost file") + // ErrWrongPassword reports a failed authentication: the password is wrong, + // or the file has been altered. The two are indistinguishable by design. + ErrWrongPassword = errors.New("secretfile: wrong password or corrupted file") + // ErrCorrupt reports structural damage — a truncated or malformed envelope. + ErrCorrupt = errors.New("secretfile: corrupted file") +) + +// HasMagic reports whether b begins with the envelope magic. b may be shorter +// than MagicLen (a short file simply is not an envelope). +func HasMagic(b []byte) bool { + if len(b) < MagicLen { + return false + } + for i, c := range magic { + if b[i] != c { + return false + } + } + return true +} + +// header is the parsed fixed prefix, kept alongside its raw bytes because those +// bytes are the additional data every chunk is authenticated with. +type header struct { + raw []byte + typ PayloadType + logN uint8 + r, p uint32 + salt []byte + noncePrefix []byte +} + +func (h *header) marshal() []byte { + buf := make([]byte, 0, headerLen) + buf = append(buf, magic...) + buf = append(buf, byte(h.typ), kdfScrypt, h.logN) + buf = binary.BigEndian.AppendUint32(buf, h.r) + buf = binary.BigEndian.AppendUint32(buf, h.p) + buf = append(buf, h.salt...) + buf = append(buf, h.noncePrefix...) + return buf +} + +func parseHeader(buf []byte) (*header, error) { + if !HasMagic(buf) { + return nil, ErrNotEncrypted + } + if len(buf) != headerLen { + return nil, ErrCorrupt + } + h := &header{raw: buf} + i := MagicLen + h.typ = PayloadType(buf[i]) + if buf[i+1] != kdfScrypt { + return nil, fmt.Errorf("%w: unsupported key derivation %d", ErrCorrupt, buf[i+1]) + } + h.logN = buf[i+2] + h.r = binary.BigEndian.Uint32(buf[i+3 : i+7]) + h.p = binary.BigEndian.Uint32(buf[i+7 : i+11]) + i += 11 + h.salt = buf[i : i+saltLen] + h.noncePrefix = buf[i+saltLen : i+saltLen+noncePrefixLen] + + // Reject absurd work factors before spending any memory on them. + if h.logN < 1 || h.logN > maxLogN || h.r < 1 || h.r > maxR || h.p < 1 || h.p > maxP { + return nil, fmt.Errorf("%w: unreasonable key-derivation parameters", ErrCorrupt) + } + return h, nil +} + +// deriveKey runs scrypt with the header's parameters. +func (h *header) deriveKey(password string) ([]byte, error) { + key, err := scrypt.Key([]byte(password), h.salt, 1< 0 { + n := chunkSize - len(e.buf) + if n > len(p) { + n = len(p) + } + e.buf = append(e.buf, p[:n]...) + p = p[n:] + written += n + if len(e.buf) == chunkSize { + if err := e.flush(false); err != nil { + return written, err + } + } + } + return written, nil +} + +// flush seals the buffered plaintext as one chunk. last marks the end of the +// stream inside the authenticated data. +func (e *Writer) flush(last bool) error { + e.sealed = e.aead.Seal(e.sealed[:0], nonce(e.hdr.noncePrefix, e.counter), e.buf, chunkAAD(e.hdr.raw, e.counter, last)) + var length [4]byte + binary.BigEndian.PutUint32(length[:], uint32(len(e.sealed))) + if _, err := e.w.Write(length[:]); err != nil { + e.err = fmt.Errorf("secretfile: write chunk: %w", err) + return e.err + } + if _, err := e.w.Write(e.sealed); err != nil { + e.err = fmt.Errorf("secretfile: write chunk: %w", err) + return e.err + } + e.buf = e.buf[:0] + e.counter++ + return nil +} + +// Close seals whatever is buffered as the final chunk. It does not close the +// underlying writer. +func (e *Writer) Close() error { + if e.err != nil { + return e.err + } + if e.closed { + return nil + } + e.closed = true + // Always emitted, even for empty input: the end-of-stream chunk is what + // proves the file was not truncated. + return e.flush(true) +} + +// Reader decrypts an envelope. Chunks are verified as they are read, so a +// truncated or tampered file surfaces as a read error rather than as short but +// plausible plaintext. +type Reader struct { + r io.Reader + aead cipher.AEAD + hdr *header + plain []byte // decrypted, not yet handed to the caller + sealed []byte // reusable ciphertext scratch + counter uint32 + done bool + err error +} + +// NewReader reads and verifies the envelope header, derives the key and returns +// a Reader over the plaintext. It fails with ErrNotEncrypted for a file that is +// not an envelope; a wrong password is only detected on the first Read, when +// the first chunk fails authentication. +func NewReader(r io.Reader, password string) (*Reader, error) { + raw := make([]byte, headerLen) + if _, err := io.ReadFull(r, raw); err != nil { + if errors.Is(err, io.EOF) || errors.Is(err, io.ErrUnexpectedEOF) { + // Too short to be an envelope: if what we did read is not even the + // magic, say so — that is the common "plain file" case. + if !HasMagic(raw) { + return nil, ErrNotEncrypted + } + return nil, ErrCorrupt + } + return nil, fmt.Errorf("secretfile: read header: %w", err) + } + h, err := parseHeader(raw) + if err != nil { + return nil, err + } + key, err := h.deriveKey(password) + if err != nil { + return nil, err + } + aead, err := newAEAD(key) + if err != nil { + return nil, err + } + return &Reader{r: r, aead: aead, hdr: h}, nil +} + +// Type reports what the envelope claims to contain. The claim is authenticated +// (the header is additional data for every chunk), so it is trustworthy as soon +// as the first Read succeeds. +func (d *Reader) Type() PayloadType { return d.hdr.typ } + +// Read returns decrypted plaintext, verifying one chunk at a time. +func (d *Reader) Read(p []byte) (int, error) { + for len(d.plain) == 0 { + if d.err != nil { + return 0, d.err + } + if d.done { + return 0, io.EOF + } + if err := d.next(); err != nil { + d.err = err + return 0, err + } + } + n := copy(p, d.plain) + d.plain = d.plain[n:] + return n, nil +} + +// next reads, verifies and decrypts the following chunk, setting done when the +// chunk carries the end-of-stream flag. +func (d *Reader) next() error { + var length [4]byte + if _, err := io.ReadFull(d.r, length[:]); err != nil { + if errors.Is(err, io.EOF) || errors.Is(err, io.ErrUnexpectedEOF) { + // The stream ended without a chunk marked last: truncated. + return ErrCorrupt + } + return fmt.Errorf("secretfile: read chunk: %w", err) + } + n := binary.BigEndian.Uint32(length[:]) + if n < tagLen || n > chunkSize+tagLen { + return ErrCorrupt + } + if cap(d.sealed) < int(n) { + d.sealed = make([]byte, n) + } + d.sealed = d.sealed[:n] + if _, err := io.ReadFull(d.r, d.sealed); err != nil { + if errors.Is(err, io.EOF) || errors.Is(err, io.ErrUnexpectedEOF) { + return ErrCorrupt + } + return fmt.Errorf("secretfile: read chunk: %w", err) + } + + // A chunk authenticates under exactly one of the two end-of-stream flags; + // trying the ordinary one first keeps the common case to a single open. + nc := nonce(d.hdr.noncePrefix, d.counter) + plain, err := d.aead.Open(nil, nc, d.sealed, chunkAAD(d.hdr.raw, d.counter, false)) + if err != nil { + plain, err = d.aead.Open(nil, nc, d.sealed, chunkAAD(d.hdr.raw, d.counter, true)) + if err != nil { + return ErrWrongPassword + } + d.done = true + } + d.plain = plain + d.counter++ + return nil +} diff --git a/internal/secretfile/secretfile_test.go b/internal/secretfile/secretfile_test.go new file mode 100644 index 0000000..19720b6 --- /dev/null +++ b/internal/secretfile/secretfile_test.go @@ -0,0 +1,162 @@ +package secretfile + +import ( + "bytes" + "errors" + "io" + "testing" +) + +// encrypt is the round-trip helper: seal data and return the envelope bytes. +func encrypt(t *testing.T, typ PayloadType, password string, data []byte) []byte { + t.Helper() + var buf bytes.Buffer + w, err := NewWriter(&buf, typ, password) + if err != nil { + t.Fatalf("NewWriter: %v", err) + } + if _, err := w.Write(data); err != nil { + t.Fatalf("Write: %v", err) + } + if err := w.Close(); err != nil { + t.Fatalf("Close: %v", err) + } + return buf.Bytes() +} + +func decrypt(t *testing.T, enc []byte, password string) ([]byte, PayloadType, error) { + t.Helper() + r, err := NewReader(bytes.NewReader(enc), password) + if err != nil { + return nil, 0, err + } + out, err := io.ReadAll(r) + return out, r.Type(), err +} + +func TestRoundTripSizes(t *testing.T) { + // Empty, sub-chunk, exactly one chunk, and several chunks with a partial + // tail: the boundaries where chunk framing tends to break. + sizes := []int{0, 1, 1000, chunkSize - 1, chunkSize, chunkSize + 1, 3*chunkSize + 77} + for _, n := range sizes { + data := make([]byte, n) + for i := range data { + data[i] = byte(i * 7) + } + enc := encrypt(t, TypeFullBackup, "correct horse battery staple", data) + got, typ, err := decrypt(t, enc, "correct horse battery staple") + if err != nil { + t.Fatalf("size %d: decrypt: %v", n, err) + } + if typ != TypeFullBackup { + t.Errorf("size %d: type = %v, want full backup", n, typ) + } + if !bytes.Equal(got, data) { + t.Errorf("size %d: plaintext mismatch (%d bytes back)", n, len(got)) + } + } +} + +func TestCiphertextDoesNotLeakPlaintext(t *testing.T) { + secret := []byte("-----BEGIN RSA PRIVATE KEY-----\nMIIEow...\n") + enc := encrypt(t, TypeDomainExport, "a very long password", secret) + if bytes.Contains(enc, secret) { + t.Fatal("plaintext found verbatim in the envelope") + } + if !HasMagic(enc) { + t.Fatal("envelope does not start with the magic") + } +} + +func TestWrongPassword(t *testing.T) { + enc := encrypt(t, TypeDomainExport, "the right password", []byte("secret payload")) + _, _, err := decrypt(t, enc, "the wrong password") + if !errors.Is(err, ErrWrongPassword) { + t.Fatalf("err = %v, want ErrWrongPassword", err) + } +} + +func TestNotEncrypted(t *testing.T) { + for _, plain := range [][]byte{[]byte(`{"format":"selfpost-domain-export"}`), {}, []byte("SELF")} { + if _, err := NewReader(bytes.NewReader(plain), "pw"); !errors.Is(err, ErrNotEncrypted) { + t.Errorf("NewReader(%q) = %v, want ErrNotEncrypted", plain, err) + } + if HasMagic(plain) { + t.Errorf("HasMagic(%q) = true", plain) + } + } +} + +func TestTruncationDetected(t *testing.T) { + // A backup cut short mid-transfer must fail loudly rather than restore a + // plausible-looking prefix. + data := bytes.Repeat([]byte("payload"), 20000) // spans several chunks + enc := encrypt(t, TypeFullBackup, "password password", data) + + // Drop the final chunk entirely: what remains is a sequence of valid, + // correctly authenticated chunks, none of which is marked last. + var chunks []int + for off := headerLen; off < len(enc); { + n := int(enc[off])<<24 | int(enc[off+1])<<16 | int(enc[off+2])<<8 | int(enc[off+3]) + chunks = append(chunks, off) + off += 4 + n + } + if len(chunks) < 2 { + t.Fatalf("expected several chunks, got %d", len(chunks)) + } + truncated := enc[:chunks[len(chunks)-1]] + r, err := NewReader(bytes.NewReader(truncated), "password password") + if err != nil { + t.Fatalf("NewReader (truncated): %v", err) + } + if _, err := io.ReadAll(r); !errors.Is(err, ErrCorrupt) { + t.Fatalf("read truncated = %v, want ErrCorrupt", err) + } +} + +func TestTamperDetected(t *testing.T) { + enc := encrypt(t, TypeFullBackup, "password password", []byte("some archive bytes")) + for _, off := range []int{MagicLen /* type byte */, headerLen + 6 /* ciphertext */} { + bad := bytes.Clone(enc) + bad[off] ^= 0xff + if _, _, err := decrypt(t, bad, "password password"); err == nil { + t.Errorf("flipping byte %d was accepted", off) + } + } +} + +func TestChunkReorderRejected(t *testing.T) { + // Two full chunks plus a tail, so swapping the first two is possible without + // changing any length. + data := make([]byte, 2*chunkSize+10) + for i := range data { + data[i] = byte(i) + } + enc := encrypt(t, TypeFullBackup, "password password", data) + + const framed = 4 + chunkSize + tagLen + first := headerLen + second := headerLen + framed + swapped := bytes.Clone(enc) + copy(swapped[first:first+framed], enc[second:second+framed]) + copy(swapped[second:second+framed], enc[first:first+framed]) + + if _, _, err := decrypt(t, swapped, "password password"); !errors.Is(err, ErrWrongPassword) { + t.Fatalf("reordered chunks = %v, want authentication failure", err) + } +} + +func TestEmptyPasswordRejected(t *testing.T) { + if _, err := NewWriter(io.Discard, TypeFullBackup, ""); err == nil { + t.Fatal("empty password accepted") + } +} + +func TestUnreasonableKDFParamsRejected(t *testing.T) { + enc := encrypt(t, TypeFullBackup, "password password", []byte("x")) + bad := bytes.Clone(enc) + bad[MagicLen+2] = 40 // logN far beyond maxLogN + if _, err := NewReader(bytes.NewReader(bad), "password password"); !errors.Is(err, ErrCorrupt) { + t.Fatalf("err = %v, want ErrCorrupt", err) + } +} diff --git a/internal/web/handlers_apps.go b/internal/web/handlers_apps.go index d32e6e7..8d0e67e 100644 --- a/internal/web/handlers_apps.go +++ b/internal/web/handlers_apps.go @@ -32,6 +32,8 @@ type detailView struct { // RateLimitErr surfaces a validation error from a domain- or application-level // rate-limit form (spec 7.4) as a page banner. RateLimitErr string + // ExportErr surfaces a rejected encryption password from the export card. + ExportErr string } // appRateLimitView pairs an application with its differentiated rate-limit @@ -131,6 +133,8 @@ func (s *Server) renderDomainDetail(w http.ResponseWriter, r *http.Request, stat "Wildcard": store.AddressModeWildcard, "List": store.AddressModeList, "RateLimitErr": view.RateLimitErr, + "ExportErr": view.ExportErr, + "MinPwLen": minSecretFilePasswordLen, "DomainHasRL": domainRLok && domainRL.Active(), "DomainRLIPs": strings.Join(domainRL.AllowedIPs, "\n"), "DomainRLMax": intOrBlank(domainRL.MaxMessages), diff --git a/internal/web/handlers_backup.go b/internal/web/handlers_backup.go index acb16fe..f3a19a8 100644 --- a/internal/web/handlers_backup.go +++ b/internal/web/handlers_backup.go @@ -1,20 +1,25 @@ package web import ( + "bytes" "encoding/json" "errors" "fmt" + "io" "net/http" "time" "codeberg.org/mix/selfpost/internal/backup" "codeberg.org/mix/selfpost/internal/domain" + "codeberg.org/mix/selfpost/internal/secretfile" "codeberg.org/mix/selfpost/internal/store" ) // maxImportBytes caps a domain-import upload. A domain export is a small JSON // document (a DKIM key and a handful of credentials); this leaves generous head -// room while refusing anything large enough to be an abuse attempt. +// room while refusing anything large enough to be an abuse attempt. An +// encrypted export adds only a header and per-chunk tags, so the same ceiling +// covers both forms. const maxImportBytes = 1 << 20 // 1 MiB // handleBackupPage renders the backup/migration screen: the full-server backup @@ -27,11 +32,21 @@ func (s *Server) handleBackupPage(w http.ResponseWriter, r *http.Request) { // renderBackupPage draws the page; importErr surfaces a failed domain import // (spec 7.5.B) next to the form that produced it. func (s *Server) renderBackupPage(w http.ResponseWriter, r *http.Request, status int, importErr string) { + s.renderBackupPageWith(w, r, status, importErr, "") +} + +// renderBackupPageWith is renderBackupPage with the second of the page's two +// error slots: backupErr belongs to the full-backup card (a rejected encryption +// password), importErr to the import card, so neither message appears under the +// wrong form. +func (s *Server) renderBackupPageWith(w http.ResponseWriter, r *http.Request, status int, importErr, backupErr string) { s.render(w, status, "backup", map[string]any{ "Title": "SelfPost — backup", "User": currentUser(r), "Active": "backup", "ImportErr": importErr, + "BackupErr": backupErr, + "MinPwLen": minSecretFilePasswordLen, }) } @@ -39,36 +54,84 @@ func (s *Server) renderBackupPage(w http.ResponseWriter, r *http.Request, status // authenticated admin action (this handler sits behind the auth middleware). The // archive carries DKIM private keys, the admin password hash and SASL // credentials, so it is served with no-store and as an attachment to discourage -// caching of secret material. +// caching of secret material. When the operator ticks "encrypt with a +// password", the archive is wrapped in a .spbk envelope on the way out, so the +// file that lands on their disk — wherever it is copied afterwards — is useless +// without the password. func (s *Server) handleBackup(w http.ResponseWriter, r *http.Request) { - filename := fmt.Sprintf("selfpost-backup-%s.tar.gz", time.Now().UTC().Format("20060102-150405")) + password, pwErr := secretFilePassword(r) + if pwErr != "" { + s.renderBackupPageWith(w, r, http.StatusBadRequest, "", pwErr) + return + } - w.Header().Set("Content-Type", "application/gzip") + stamp := time.Now().UTC().Format("20060102-150405") + filename := fmt.Sprintf("selfpost-backup-%s.tar.gz", stamp) + contentType := "application/gzip" + if password != "" { + filename = fmt.Sprintf("selfpost-backup-%s%s", stamp, secretfile.ExtBackup) + contentType = "application/octet-stream" + } + + w.Header().Set("Content-Type", contentType) w.Header().Set("Content-Disposition", fmt.Sprintf("attachment; filename=%q", filename)) w.Header().Set("Cache-Control", "no-store") - if err := backup.Create(w, backup.Params{ + // Everything below streams: past this point headers (and possibly some + // bytes) are already on the wire, so a failure cannot switch to a clean + // error page. Log it and let the truncated download fail loudly on the + // client side — for an encrypted archive that is a missing end-of-stream + // chunk, which decryption refuses outright. + sink := io.Writer(w) + var env *secretfile.Writer + if password != "" { + var err error + // The only failures here are key derivation (which happens before + // anything is written) and writing the envelope header, which fails only + // if the client is already gone. + env, err = secretfile.NewWriter(w, secretfile.TypeFullBackup, password) + if err != nil { + logf("panel: full backup: encrypt: %v", err) + http.Error(w, "backup failed", http.StatusInternalServerError) + return + } + sink = env + } + + if err := backup.Create(sink, backup.Params{ DataDir: s.cfg.DataDir, DBPath: s.cfg.DBPath, Version: s.cfg.Version, }); err != nil { - // Headers (and possibly some bytes) may already be on the wire, so we - // cannot switch to a clean error page; log it and let the truncated - // download fail loudly on the client side. logf("panel: full backup failed: %v", err) return } + if env != nil { + if err := env.Close(); err != nil { + logf("panel: full backup failed: %v", err) + } + } } // handleExportDomain streams a single-domain export as a secret download (spec // 7.5.B). Like the full backup it is POST-only (state is not changed, but the // response contains the domain's DKIM private key and application passwords, so -// it must not be prefetchable or cached). +// it must not be prefetchable or cached). Like the full backup it can be +// encrypted with a password, in which case the download is a .spde envelope +// instead of plain JSON. func (s *Server) handleExportDomain(w http.ResponseWriter, r *http.Request) { d, ok := s.lookupDomain(w, r) if !ok { return } + password, pwErr := secretFilePassword(r) + if pwErr != "" { + s.renderDomainDetail(w, r, http.StatusBadRequest, d, detailView{ + FormMode: store.AddressModeWildcard, + ExportErr: pwErr, + }) + return + } exp, err := s.domains.Export(d.ID) if err != nil { logf("panel: export domain %d: %v", d.ID, err) @@ -83,7 +146,29 @@ func (s *Server) handleExportDomain(w http.ResponseWriter, r *http.Request) { } filename := fmt.Sprintf("selfpost-domain-%s.json", d.Name) - w.Header().Set("Content-Type", "application/json") + contentType := "application/json" + if password != "" { + // An export is small, so it is sealed in memory: the response is only + // started once the ciphertext is complete and nothing can half-fail. + var buf bytes.Buffer + env, err := secretfile.NewWriter(&buf, secretfile.TypeDomainExport, password) + if err == nil { + _, err = env.Write(body) + } + if err == nil { + err = env.Close() + } + if err != nil { + logf("panel: export domain %d: encrypt: %v", d.ID, err) + http.Error(w, "export failed", http.StatusInternalServerError) + return + } + body = buf.Bytes() + filename = fmt.Sprintf("selfpost-domain-%s%s", d.Name, secretfile.ExtDomainExport) + contentType = "application/octet-stream" + } + + w.Header().Set("Content-Type", contentType) w.Header().Set("Content-Disposition", fmt.Sprintf("attachment; filename=%q", filename)) w.Header().Set("Cache-Control", "no-store") _, _ = w.Write(body) @@ -108,8 +193,49 @@ func (s *Server) handleImportDomain(w http.ResponseWriter, r *http.Request) { } defer file.Close() + // An encrypted export announces itself with the envelope magic, so the file + // decides which path it takes; the password field is only consulted when the + // file actually needs it, and a password typed for a plain file is a plain + // mistake worth reporting. + head := make([]byte, secretfile.MagicLen) + n, err := io.ReadFull(file, head) + if err != nil && !errors.Is(err, io.EOF) && !errors.Is(err, io.ErrUnexpectedEOF) { + s.renderBackupPage(w, r, http.StatusBadRequest, "Could not read the uploaded file.") + return + } + source := io.MultiReader(bytes.NewReader(head[:n]), file) + password := r.PostFormValue("import_password") + + if secretfile.HasMagic(head[:n]) { + if password == "" { + s.renderBackupPage(w, r, http.StatusBadRequest, "That file is encrypted — enter the password it was exported with.") + return + } + env, err := secretfile.NewReader(source, password) + if err != nil { + s.renderBackupPage(w, r, http.StatusBadRequest, decryptErrorMessage(err)) + return + } + if env.Type() != secretfile.TypeDomainExport { + s.renderBackupPage(w, r, http.StatusBadRequest, "That file is an encrypted "+env.Type().String()+", not a domain export.") + return + } + // Read the whole plaintext first: authentication of the last chunk is + // what proves the file is intact, and a streaming JSON decoder could + // accept a truncated document before ever reaching it. + plain, err := io.ReadAll(env) + if err != nil { + s.renderBackupPage(w, r, http.StatusBadRequest, decryptErrorMessage(err)) + return + } + source = bytes.NewReader(plain) + } else if password != "" { + s.renderBackupPage(w, r, http.StatusBadRequest, "That file is not encrypted — leave the password empty.") + return + } + var exp domain.DomainExport - dec := json.NewDecoder(file) + dec := json.NewDecoder(source) dec.DisallowUnknownFields() if err := dec.Decode(&exp); err != nil { s.renderBackupPage(w, r, http.StatusBadRequest, "That file is not a valid SelfPost domain export.") @@ -134,6 +260,46 @@ func (s *Server) handleImportDomain(w http.ResponseWriter, r *http.Request) { http.Redirect(w, r, fmt.Sprintf("/domains/%d?imported=1", d.ID), http.StatusSeeOther) } +// secretFilePassword reads the "encrypt this download with a password" controls +// shared by the full-backup and domain-export forms. It returns the password to +// encrypt with — empty when the box is not ticked, which keeps the plain +// .tar.gz/.json behaviour of earlier versions — or a message to show above the +// form. The confirmation field is checked here rather than in the browser +// because a typo in an encryption password is unrecoverable: the archive would +// be sealed with a secret the operator does not know. +func secretFilePassword(r *http.Request) (password, errMsg string) { + if err := r.ParseForm(); err != nil { + return "", "Invalid form submission." + } + if r.PostFormValue("encrypt") == "" { + return "", "" + } + password = r.PostFormValue("password") + if len([]rune(password)) < minSecretFilePasswordLen { + return "", fmt.Sprintf("The encryption password must be at least %d characters.", minSecretFilePasswordLen) + } + if password != r.PostFormValue("password_confirm") { + return "", "The two passwords do not match." + } + return password, "" +} + +// decryptErrorMessage phrases an envelope failure for the operator. A wrong +// password and a damaged file are deliberately indistinguishable to the code, +// so the message names both possibilities. +func decryptErrorMessage(err error) string { + switch { + case errors.Is(err, secretfile.ErrWrongPassword): + return "Wrong password, or the file has been altered since it was exported." + case errors.Is(err, secretfile.ErrCorrupt): + return "That file is damaged or incomplete." + case errors.Is(err, secretfile.ErrNotEncrypted): + return "That file is not a SelfPost export." + default: + return "Could not decrypt the file." + } +} + // importErrorMessage maps a domain-import failure (already logged by the caller) // to an HTTP status and a user-facing message. Duplicate domain/login are called // out specifically; other failures — validation errors describing what is wrong diff --git a/internal/web/handlers_backup_test.go b/internal/web/handlers_backup_test.go new file mode 100644 index 0000000..f6bbd7b --- /dev/null +++ b/internal/web/handlers_backup_test.go @@ -0,0 +1,127 @@ +package web + +import ( + "errors" + "fmt" + "net/http" + "net/http/httptest" + "net/url" + "strings" + "testing" + + "codeberg.org/mix/selfpost/internal/secretfile" +) + +// postForm builds the kind of request the backup and export forms submit. +func postForm(values url.Values) *http.Request { + r := httptest.NewRequest(http.MethodPost, "/backup", strings.NewReader(values.Encode())) + r.Header.Set("Content-Type", "application/x-www-form-urlencoded") + return r +} + +// The encryption password is only ever typed once into a file nobody can +// recover without it, so every way of getting it wrong has to be caught before +// the archive is sealed — and leaving the box unticked has to keep producing +// the plain archive earlier versions produced. +func TestSecretFilePassword(t *testing.T) { + long := strings.Repeat("x", minSecretFilePasswordLen) + short := strings.Repeat("x", minSecretFilePasswordLen-1) + + tests := []struct { + name string + form url.Values + wantPass string + wantErr bool + }{ + { + name: "unticked box means no encryption", + form: url.Values{"password": {long}, "password_confirm": {long}}, + wantPass: "", + }, + { + name: "ticked with a matching password", + form: url.Values{"encrypt": {"1"}, "password": {long}, "password_confirm": {long}}, + wantPass: long, + }, + { + name: "mistyped confirmation", + form: url.Values{"encrypt": {"1"}, "password": {long}, "password_confirm": {long + "!"}}, + wantErr: true, + }, + { + name: "too short", + form: url.Values{"encrypt": {"1"}, "password": {short}, "password_confirm": {short}}, + wantErr: true, + }, + { + name: "ticked but empty", + form: url.Values{"encrypt": {"1"}}, + wantErr: true, + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + pass, errMsg := secretFilePassword(postForm(tt.form)) + if tt.wantErr { + if errMsg == "" { + t.Fatalf("password %q accepted, want a rejection", pass) + } + if pass != "" { + t.Errorf("a rejected form still yielded password %q", pass) + } + return + } + if errMsg != "" { + t.Fatalf("unexpected rejection: %s", errMsg) + } + if pass != tt.wantPass { + t.Errorf("password = %q, want %q", pass, tt.wantPass) + } + }) + } +} + +// The messages the import form shows must distinguish the operator's likely +// mistakes; a wrong password and a tampered file stay deliberately merged. +func TestDecryptErrorMessage(t *testing.T) { + tests := []struct { + err error + want string + }{ + {secretfile.ErrWrongPassword, "Wrong password"}, + {fmt.Errorf("read: %w", secretfile.ErrCorrupt), "damaged"}, + {secretfile.ErrNotEncrypted, "not a SelfPost export"}, + {errors.New("something else"), "Could not decrypt"}, + } + for _, tt := range tests { + if got := decryptErrorMessage(tt.err); !strings.Contains(got, tt.want) { + t.Errorf("decryptErrorMessage(%v) = %q, want it to mention %q", tt.err, got, tt.want) + } + } +} + +// The encryption controls are shared markup pulled into two pages; a page that +// forgets to include the partial (or the data it needs) loses the option +// silently, since the plain download still works. +func TestBackupPageOffersEncryption(t *testing.T) { + tmpl, err := loadTemplates() + if err != nil { + t.Fatalf("loadTemplates: %v", err) + } + s := &Server{tmpl: tmpl, cfg: Config{Version: "test"}} + rec := httptest.NewRecorder() + s.renderBackupPageWith(rec, httptest.NewRequest(http.MethodGet, "/backup", nil), + http.StatusOK, "", "The two passwords do not match.") + + body := rec.Body.String() + for _, want := range []string{ + `name="encrypt"`, `name="password"`, `name="password_confirm"`, + `name="import_password"`, "data-encrypt-toggle", "data-encrypt-fields", + fmt.Sprintf("at least %d characters", minSecretFilePasswordLen), + "The two passwords do not match.", + } { + if !strings.Contains(body, want) { + t.Errorf("backup page is missing %q", want) + } + } +} diff --git a/internal/web/static/panel.css b/internal/web/static/panel.css index faf2491..faf651d 100644 --- a/internal/web/static/panel.css +++ b/internal/web/static/panel.css @@ -249,3 +249,22 @@ button.copy { flex: none; margin-top: 0.3rem; } } .actions button.danger:hover, .actions a.danger:hover, .nav button.danger:hover { background: #3d1a18 !important; } } + +/* The optional "encrypt this download" block on the backup, export and import + forms. Its label is the one checkbox in the panel, so it opts out of the + block-level label rule above and sits on one line with its box; the fields it + reveals are indented under it to read as its consequence rather than as three + more fields of the form. panel.js hides the inner block until the box is + ticked (and empties it when unticked); without JavaScript everything stays + visible, which the server handles identically. */ +.encrypt { margin-top: 1.2rem; } +.encrypt label.check { + display: flex; align-items: center; gap: 0.5rem; margin: 0; font-weight: 600; +} +.encrypt label.check input { width: auto; margin: 0; } +.encrypt-fields { + margin-left: 1.6rem; padding-left: 0.9rem; border-left: 2px solid #e2e5e9; +} +.encrypt-fields label { margin-top: 0.7rem; } +.encrypt-fields .muted { margin: 0.5rem 0 0; font-size: 0.85rem; } +@media (prefers-color-scheme: dark) { .encrypt-fields { border-color: #2b3138 !important; } } diff --git a/internal/web/static/panel.js b/internal/web/static/panel.js index b0bdcda..dc2bbbb 100644 --- a/internal/web/static/panel.js +++ b/internal/web/static/panel.js @@ -72,7 +72,38 @@ }); } + // --- Encryption password fields shown only when asked for -------------- + // The backup, export and import forms carry an optional password block. It + // is hidden until the checkbox next to it is ticked, and cleared when it is + // unticked, so a password typed and then abandoned is never submitted. With + // JavaScript blocked the block stays visible and the forms behave exactly as + // the server reads them: the checkbox alone decides whether encryption + // happens. + function syncEncryptFields(box) { + var form = box.closest("form"); + var fields = form && form.querySelector("[data-encrypt-fields]"); + if (!fields) { + return; + } + fields.hidden = !box.checked; + if (!box.checked) { + fields.querySelectorAll("input").forEach(function (input) { + input.value = ""; + }); + } + } + + function initEncryptFields(root) { + root.querySelectorAll("input[data-encrypt-toggle]").forEach(function (box) { + syncEncryptFields(box); + box.addEventListener("change", function () { + syncEncryptFields(box); + }); + }); + } + document.addEventListener("DOMContentLoaded", function () { initAddressFields(document); + initEncryptFields(document); }); })(); diff --git a/internal/web/templates.go b/internal/web/templates.go index b3f43d5..03aab27 100644 --- a/internal/web/templates.go +++ b/internal/web/templates.go @@ -21,14 +21,16 @@ type templates struct { // pageFiles maps a logical page name to its template files. Every page // composes with layout.html; pages that embed a polling fragment (spec 7.1) // list that fragment's file too, so the same {{define}} block renders both -// the initial page and the fragment's own refresh responses identically. +// the initial page and the fragment's own refresh responses identically. Pages +// sharing a block of markup (the encryption fields on the two secret downloads) +// list that partial the same way. var pageFiles = map[string][]string{ "setup": {"templates/setup.html"}, "login": {"templates/login.html"}, "dashboard": {"templates/dashboard.html"}, "account": {"templates/account.html"}, - "backup": {"templates/backup.html"}, - "domain_detail": {"templates/domain_detail.html"}, + "backup": {"templates/backup.html", "templates/encrypt_fields.html"}, + "domain_detail": {"templates/domain_detail.html", "templates/encrypt_fields.html"}, "domain_delete": {"templates/domain_delete.html"}, "deliveries": {"templates/deliveries.html", "templates/deliveries_rows.html"}, "mail_queue": {"templates/mail_queue.html", "templates/mail_queue_body.html"}, diff --git a/internal/web/templates/backup.html b/internal/web/templates/backup.html index bebecb2..60e6f31 100644 --- a/internal/web/templates/backup.html +++ b/internal/web/templates/backup.html @@ -10,8 +10,11 @@ start. TLS certificates and the mail queue are not included.

The backup file is a secret (it contains private keys and credentials). Store and transfer it securely and delete it - once the restore succeeds.

-
+ once the restore succeeds. Encrypting it below is the simplest way to do that: + the download is then a .spbk file that only the password opens.

+ {{if .BackupErr}}

{{.BackupErr}}

{{end}} + + {{template "encryptfields" .}}
@@ -25,7 +28,17 @@ {{if .ImportErr}}

{{.ImportErr}}

{{end}}
- + +
+ +
+ + +
+
diff --git a/internal/web/templates/domain_detail.html b/internal/web/templates/domain_detail.html index 879102a..c1bb454 100644 --- a/internal/web/templates/domain_detail.html +++ b/internal/web/templates/domain_detail.html @@ -283,8 +283,11 @@ import the DNS record stays the same, so no DNS change is needed.

The export file is a secret — it contains the private DKIM key and application passwords. Transfer it securely and delete it - after the import.

-
+ after the import, or encrypt it below and move a .spde file + instead; the import form asks for the password.

+ {{if .ExportErr}}

{{.ExportErr}}

{{end}} + + {{template "encryptfields" .}}
diff --git a/internal/web/templates/encrypt_fields.html b/internal/web/templates/encrypt_fields.html new file mode 100644 index 0000000..3340201 --- /dev/null +++ b/internal/web/templates/encrypt_fields.html @@ -0,0 +1,23 @@ +{{/* Password fields shared by the full-backup and domain-export forms. The + checkbox is the switch: unticked, the download keeps its historic plain + form (.tar.gz / .json); ticked, the file is sealed in a password-encrypted + envelope (.spbk / .spde). The fields start hidden and are revealed by + panel.js — with JavaScript blocked they are simply always visible, and the + server still decides from the checkbox alone. */}} +{{define "encryptfields"}} +
+ +
+ + + + +

Keep this password: without it the file cannot be opened, + and SelfPost does not store it anywhere.

+
+
+{{end}} diff --git a/internal/web/validate.go b/internal/web/validate.go index c720d93..3ed4dc7 100644 --- a/internal/web/validate.go +++ b/internal/web/validate.go @@ -15,6 +15,12 @@ const ( maxUsernameLen = 64 ) +// minSecretFilePasswordLen is the floor for the password protecting an +// encrypted backup or domain export. Such a file is offline and can be attacked +// at leisure, so the floor matches the administrator password's rather than the +// weaker "any password is better than none". +const minSecretFilePasswordLen = minAdminPasswordLen + // validateUsername enforces a strict server-side whitelist (spec 7.6.2): // letters, digits, dot, dash, underscore. Client validation is never trusted. func validateUsername(u string) error {