feat: optional password encryption for backup and domain export (code-review.md § Phase 1.5)

Both secret-bearing downloads can now be sealed with a password. Unticked, the
forms produce exactly the files they did before.

- internal/secretfile: envelope format — magic/type/scrypt params/salt/nonce
  prefix header, then 64 KiB AES-256-GCM chunks each authenticated with the
  header, its counter and an end-of-stream flag, so truncation, reordering and
  tampering fail to open instead of restoring a plausible prefix. Streams both
  ways, so a full backup never sits in memory.
- Panel: "Encrypt with a password" checkbox on the full-backup and
  domain-export forms (shared partial, toggled from panel.js — no inline
  script); domain import detects an encrypted export by magic bytes, not by
  extension, and asks for the password.
- selfpost-backup: writes .spbk when given a password and converts one back
  with -decrypt, which a restore needs. The password comes from
  SELFPOST_BACKUP_PASSWORD or -password-file, never argv.
- Docs: README, security.md (+ accepted risk: encryption stays opt-in),
  architecture.md, progress.md, CHANGELOG.

Verified locally: panel-encrypted archive decrypts through the CLI and unpacks;
wrong password and password mismatch are refused; UI checked in a browser.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-06 16:43:28 +03:00
parent 670982fb3e
commit 6a7d010868
20 changed files with 1402 additions and 36 deletions
+458
View File
@@ -0,0 +1,458 @@
// Package secretfile wraps SelfPost's secret-bearing downloads — the full
// server backup and the single-domain export — in a password-encrypted
// envelope. Both artefacts carry DKIM private keys, SASL credentials and
// application passwords in the clear, so an operator who stores them outside
// the server (the whole point of a backup) has to protect them by hand. The
// envelope makes that optional-but-easy: tick a box, give a password, and the
// file that leaves the panel is useless without it.
//
// Layout — a fixed header followed by a sequence of independently
// authenticated chunks, so both writing and reading stream and a multi-megabyte
// backup never has to sit in memory:
//
// magic 9 "SELFPOST1" (the trailing digit is the envelope version)
// type 1 PayloadType: what the plaintext is
// kdf 1 KDF identifier (1 = scrypt)
// logN 1 scrypt cost, log2(N)
// r 4 scrypt block size (big-endian)
// p 4 scrypt parallelisation (big-endian)
// salt 16 scrypt salt
// prefix 8 nonce prefix
// chunks … repeated: length (4, big-endian) + AES-256-GCM ciphertext
//
// The KDF parameters travel in the header so a file encrypted today still opens
// after the cost is raised. Every chunk is sealed with the nonce prefix plus its
// own counter, and takes the whole header, that counter and an end-of-stream
// flag as additional data: a chunk cannot be reordered, swapped between files,
// or dropped from the end without decryption failing.
package secretfile
import (
"crypto/aes"
"crypto/cipher"
"crypto/rand"
"encoding/binary"
"errors"
"fmt"
"io"
"golang.org/x/crypto/scrypt"
)
// PayloadType identifies what a decrypted envelope contains, so opening a file
// can report "that is a domain export, not a backup" instead of handing an
// unexpected payload to a parser.
type PayloadType byte
const (
// TypeFullBackup is the gzip-compressed tar of a full server backup.
TypeFullBackup PayloadType = 1
// TypeDomainExport is the JSON document of a single-domain export.
TypeDomainExport PayloadType = 2
)
// String names a payload type for operator-facing messages.
func (t PayloadType) String() string {
switch t {
case TypeFullBackup:
return "full backup"
case TypeDomainExport:
return "domain export"
default:
return fmt.Sprintf("unknown type %d", byte(t))
}
}
// File name suffixes for the two encrypted artefacts: SelfPost BacKup and
// SelfPost Domain Export. They exist so an operator can tell an encrypted file
// from a plain .tar.gz/.json at a glance; nothing reads them back — the magic
// bytes decide.
const (
ExtBackup = ".spbk"
ExtDomainExport = ".spde"
)
// magic prefixes every envelope. MagicLen bytes are enough to tell an encrypted
// file from a plain one, which is what an import handler needs to decide whether
// to ask for a password.
var magic = []byte("SELFPOST1")
// MagicLen is the number of leading bytes HasMagic inspects.
const MagicLen = 9
const (
kdfScrypt = 1
saltLen = 16
noncePrefixLen = 8
counterLen = 4
headerLen = MagicLen + 1 + 1 + 1 + 4 + 4 + saltLen + noncePrefixLen
// chunkSize is the plaintext carried by one sealed chunk. 64 KiB keeps the
// per-chunk overhead negligible while bounding the buffer a reader must
// allocate for a hostile length field.
chunkSize = 64 * 1024
tagLen = 16
keyLen = 32 // AES-256
)
// Default scrypt parameters: 32 MiB and roughly a tenth of a second per attempt
// on ordinary hardware. The panel derives a key at most once per download, so
// the cost is invisible to the operator and expensive for anyone brute-forcing a
// stolen backup.
const (
defaultLogN = 15
defaultR = 8
defaultP = 1
// maxLogN caps what a file may ask for, so a hostile header cannot make the
// reader allocate its way to an out-of-memory kill before it ever fails
// authentication.
maxLogN = 20
maxR = 32
maxP = 16
)
var (
// ErrNotEncrypted reports a file that is not a SelfPost envelope at all —
// most often a plain .json export handed to the encrypted path.
ErrNotEncrypted = errors.New("secretfile: not an encrypted SelfPost file")
// ErrWrongPassword reports a failed authentication: the password is wrong,
// or the file has been altered. The two are indistinguishable by design.
ErrWrongPassword = errors.New("secretfile: wrong password or corrupted file")
// ErrCorrupt reports structural damage — a truncated or malformed envelope.
ErrCorrupt = errors.New("secretfile: corrupted file")
)
// HasMagic reports whether b begins with the envelope magic. b may be shorter
// than MagicLen (a short file simply is not an envelope).
func HasMagic(b []byte) bool {
if len(b) < MagicLen {
return false
}
for i, c := range magic {
if b[i] != c {
return false
}
}
return true
}
// header is the parsed fixed prefix, kept alongside its raw bytes because those
// bytes are the additional data every chunk is authenticated with.
type header struct {
raw []byte
typ PayloadType
logN uint8
r, p uint32
salt []byte
noncePrefix []byte
}
func (h *header) marshal() []byte {
buf := make([]byte, 0, headerLen)
buf = append(buf, magic...)
buf = append(buf, byte(h.typ), kdfScrypt, h.logN)
buf = binary.BigEndian.AppendUint32(buf, h.r)
buf = binary.BigEndian.AppendUint32(buf, h.p)
buf = append(buf, h.salt...)
buf = append(buf, h.noncePrefix...)
return buf
}
func parseHeader(buf []byte) (*header, error) {
if !HasMagic(buf) {
return nil, ErrNotEncrypted
}
if len(buf) != headerLen {
return nil, ErrCorrupt
}
h := &header{raw: buf}
i := MagicLen
h.typ = PayloadType(buf[i])
if buf[i+1] != kdfScrypt {
return nil, fmt.Errorf("%w: unsupported key derivation %d", ErrCorrupt, buf[i+1])
}
h.logN = buf[i+2]
h.r = binary.BigEndian.Uint32(buf[i+3 : i+7])
h.p = binary.BigEndian.Uint32(buf[i+7 : i+11])
i += 11
h.salt = buf[i : i+saltLen]
h.noncePrefix = buf[i+saltLen : i+saltLen+noncePrefixLen]
// Reject absurd work factors before spending any memory on them.
if h.logN < 1 || h.logN > maxLogN || h.r < 1 || h.r > maxR || h.p < 1 || h.p > maxP {
return nil, fmt.Errorf("%w: unreasonable key-derivation parameters", ErrCorrupt)
}
return h, nil
}
// deriveKey runs scrypt with the header's parameters.
func (h *header) deriveKey(password string) ([]byte, error) {
key, err := scrypt.Key([]byte(password), h.salt, 1<<h.logN, int(h.r), int(h.p), keyLen)
if err != nil {
return nil, fmt.Errorf("secretfile: derive key: %w", err)
}
return key, nil
}
func newAEAD(key []byte) (cipher.AEAD, error) {
block, err := aes.NewCipher(key)
if err != nil {
return nil, fmt.Errorf("secretfile: cipher: %w", err)
}
aead, err := cipher.NewGCM(block)
if err != nil {
return nil, fmt.Errorf("secretfile: gcm: %w", err)
}
return aead, nil
}
// nonce composes the per-chunk nonce: the file's random prefix followed by the
// chunk counter, so no two chunks in a file — and, with overwhelming
// probability, no two chunks across files — share one.
func nonce(prefix []byte, counter uint32) []byte {
n := make([]byte, 0, noncePrefixLen+counterLen)
n = append(n, prefix...)
return binary.BigEndian.AppendUint32(n, counter)
}
// chunkAAD binds a chunk to its file, its position and its end-of-stream flag.
func chunkAAD(hdr []byte, counter uint32, last bool) []byte {
aad := make([]byte, 0, len(hdr)+counterLen+1)
aad = append(aad, hdr...)
aad = binary.BigEndian.AppendUint32(aad, counter)
if last {
return append(aad, 1)
}
return append(aad, 0)
}
// Writer encrypts a stream into w. Callers must Close it: the final chunk (and
// with it the end-of-stream marker that makes truncation detectable) is only
// written then.
type Writer struct {
w io.Writer
aead cipher.AEAD
hdr *header
buf []byte // pending plaintext, at most chunkSize
sealed []byte // reusable ciphertext scratch
counter uint32
closed bool
err error
}
// NewWriter derives a key from password and writes the envelope header to w.
// Deriving the key is deliberately slow (scrypt), so call this once per file.
func NewWriter(w io.Writer, typ PayloadType, password string) (*Writer, error) {
if password == "" {
return nil, errors.New("secretfile: empty password")
}
h := &header{
typ: typ,
logN: defaultLogN,
r: defaultR,
p: defaultP,
salt: make([]byte, saltLen),
noncePrefix: make([]byte, noncePrefixLen),
}
if _, err := rand.Read(h.salt); err != nil {
return nil, fmt.Errorf("secretfile: salt: %w", err)
}
if _, err := rand.Read(h.noncePrefix); err != nil {
return nil, fmt.Errorf("secretfile: nonce: %w", err)
}
h.raw = h.marshal()
key, err := h.deriveKey(password)
if err != nil {
return nil, err
}
aead, err := newAEAD(key)
if err != nil {
return nil, err
}
if _, err := w.Write(h.raw); err != nil {
return nil, fmt.Errorf("secretfile: write header: %w", err)
}
return &Writer{
w: w,
aead: aead,
hdr: h,
buf: make([]byte, 0, chunkSize),
sealed: make([]byte, 0, chunkSize+tagLen),
}, nil
}
// Write buffers plaintext, sealing and emitting a chunk whenever a full one has
// accumulated.
func (e *Writer) Write(p []byte) (int, error) {
if e.err != nil {
return 0, e.err
}
if e.closed {
return 0, errors.New("secretfile: write after close")
}
written := 0
for len(p) > 0 {
n := chunkSize - len(e.buf)
if n > len(p) {
n = len(p)
}
e.buf = append(e.buf, p[:n]...)
p = p[n:]
written += n
if len(e.buf) == chunkSize {
if err := e.flush(false); err != nil {
return written, err
}
}
}
return written, nil
}
// flush seals the buffered plaintext as one chunk. last marks the end of the
// stream inside the authenticated data.
func (e *Writer) flush(last bool) error {
e.sealed = e.aead.Seal(e.sealed[:0], nonce(e.hdr.noncePrefix, e.counter), e.buf, chunkAAD(e.hdr.raw, e.counter, last))
var length [4]byte
binary.BigEndian.PutUint32(length[:], uint32(len(e.sealed)))
if _, err := e.w.Write(length[:]); err != nil {
e.err = fmt.Errorf("secretfile: write chunk: %w", err)
return e.err
}
if _, err := e.w.Write(e.sealed); err != nil {
e.err = fmt.Errorf("secretfile: write chunk: %w", err)
return e.err
}
e.buf = e.buf[:0]
e.counter++
return nil
}
// Close seals whatever is buffered as the final chunk. It does not close the
// underlying writer.
func (e *Writer) Close() error {
if e.err != nil {
return e.err
}
if e.closed {
return nil
}
e.closed = true
// Always emitted, even for empty input: the end-of-stream chunk is what
// proves the file was not truncated.
return e.flush(true)
}
// Reader decrypts an envelope. Chunks are verified as they are read, so a
// truncated or tampered file surfaces as a read error rather than as short but
// plausible plaintext.
type Reader struct {
r io.Reader
aead cipher.AEAD
hdr *header
plain []byte // decrypted, not yet handed to the caller
sealed []byte // reusable ciphertext scratch
counter uint32
done bool
err error
}
// NewReader reads and verifies the envelope header, derives the key and returns
// a Reader over the plaintext. It fails with ErrNotEncrypted for a file that is
// not an envelope; a wrong password is only detected on the first Read, when
// the first chunk fails authentication.
func NewReader(r io.Reader, password string) (*Reader, error) {
raw := make([]byte, headerLen)
if _, err := io.ReadFull(r, raw); err != nil {
if errors.Is(err, io.EOF) || errors.Is(err, io.ErrUnexpectedEOF) {
// Too short to be an envelope: if what we did read is not even the
// magic, say so — that is the common "plain file" case.
if !HasMagic(raw) {
return nil, ErrNotEncrypted
}
return nil, ErrCorrupt
}
return nil, fmt.Errorf("secretfile: read header: %w", err)
}
h, err := parseHeader(raw)
if err != nil {
return nil, err
}
key, err := h.deriveKey(password)
if err != nil {
return nil, err
}
aead, err := newAEAD(key)
if err != nil {
return nil, err
}
return &Reader{r: r, aead: aead, hdr: h}, nil
}
// Type reports what the envelope claims to contain. The claim is authenticated
// (the header is additional data for every chunk), so it is trustworthy as soon
// as the first Read succeeds.
func (d *Reader) Type() PayloadType { return d.hdr.typ }
// Read returns decrypted plaintext, verifying one chunk at a time.
func (d *Reader) Read(p []byte) (int, error) {
for len(d.plain) == 0 {
if d.err != nil {
return 0, d.err
}
if d.done {
return 0, io.EOF
}
if err := d.next(); err != nil {
d.err = err
return 0, err
}
}
n := copy(p, d.plain)
d.plain = d.plain[n:]
return n, nil
}
// next reads, verifies and decrypts the following chunk, setting done when the
// chunk carries the end-of-stream flag.
func (d *Reader) next() error {
var length [4]byte
if _, err := io.ReadFull(d.r, length[:]); err != nil {
if errors.Is(err, io.EOF) || errors.Is(err, io.ErrUnexpectedEOF) {
// The stream ended without a chunk marked last: truncated.
return ErrCorrupt
}
return fmt.Errorf("secretfile: read chunk: %w", err)
}
n := binary.BigEndian.Uint32(length[:])
if n < tagLen || n > chunkSize+tagLen {
return ErrCorrupt
}
if cap(d.sealed) < int(n) {
d.sealed = make([]byte, n)
}
d.sealed = d.sealed[:n]
if _, err := io.ReadFull(d.r, d.sealed); err != nil {
if errors.Is(err, io.EOF) || errors.Is(err, io.ErrUnexpectedEOF) {
return ErrCorrupt
}
return fmt.Errorf("secretfile: read chunk: %w", err)
}
// A chunk authenticates under exactly one of the two end-of-stream flags;
// trying the ordinary one first keeps the common case to a single open.
nc := nonce(d.hdr.noncePrefix, d.counter)
plain, err := d.aead.Open(nil, nc, d.sealed, chunkAAD(d.hdr.raw, d.counter, false))
if err != nil {
plain, err = d.aead.Open(nil, nc, d.sealed, chunkAAD(d.hdr.raw, d.counter, true))
if err != nil {
return ErrWrongPassword
}
d.done = true
}
d.plain = plain
d.counter++
return nil
}
+162
View File
@@ -0,0 +1,162 @@
package secretfile
import (
"bytes"
"errors"
"io"
"testing"
)
// encrypt is the round-trip helper: seal data and return the envelope bytes.
func encrypt(t *testing.T, typ PayloadType, password string, data []byte) []byte {
t.Helper()
var buf bytes.Buffer
w, err := NewWriter(&buf, typ, password)
if err != nil {
t.Fatalf("NewWriter: %v", err)
}
if _, err := w.Write(data); err != nil {
t.Fatalf("Write: %v", err)
}
if err := w.Close(); err != nil {
t.Fatalf("Close: %v", err)
}
return buf.Bytes()
}
func decrypt(t *testing.T, enc []byte, password string) ([]byte, PayloadType, error) {
t.Helper()
r, err := NewReader(bytes.NewReader(enc), password)
if err != nil {
return nil, 0, err
}
out, err := io.ReadAll(r)
return out, r.Type(), err
}
func TestRoundTripSizes(t *testing.T) {
// Empty, sub-chunk, exactly one chunk, and several chunks with a partial
// tail: the boundaries where chunk framing tends to break.
sizes := []int{0, 1, 1000, chunkSize - 1, chunkSize, chunkSize + 1, 3*chunkSize + 77}
for _, n := range sizes {
data := make([]byte, n)
for i := range data {
data[i] = byte(i * 7)
}
enc := encrypt(t, TypeFullBackup, "correct horse battery staple", data)
got, typ, err := decrypt(t, enc, "correct horse battery staple")
if err != nil {
t.Fatalf("size %d: decrypt: %v", n, err)
}
if typ != TypeFullBackup {
t.Errorf("size %d: type = %v, want full backup", n, typ)
}
if !bytes.Equal(got, data) {
t.Errorf("size %d: plaintext mismatch (%d bytes back)", n, len(got))
}
}
}
func TestCiphertextDoesNotLeakPlaintext(t *testing.T) {
secret := []byte("-----BEGIN RSA PRIVATE KEY-----\nMIIEow...\n")
enc := encrypt(t, TypeDomainExport, "a very long password", secret)
if bytes.Contains(enc, secret) {
t.Fatal("plaintext found verbatim in the envelope")
}
if !HasMagic(enc) {
t.Fatal("envelope does not start with the magic")
}
}
func TestWrongPassword(t *testing.T) {
enc := encrypt(t, TypeDomainExport, "the right password", []byte("secret payload"))
_, _, err := decrypt(t, enc, "the wrong password")
if !errors.Is(err, ErrWrongPassword) {
t.Fatalf("err = %v, want ErrWrongPassword", err)
}
}
func TestNotEncrypted(t *testing.T) {
for _, plain := range [][]byte{[]byte(`{"format":"selfpost-domain-export"}`), {}, []byte("SELF")} {
if _, err := NewReader(bytes.NewReader(plain), "pw"); !errors.Is(err, ErrNotEncrypted) {
t.Errorf("NewReader(%q) = %v, want ErrNotEncrypted", plain, err)
}
if HasMagic(plain) {
t.Errorf("HasMagic(%q) = true", plain)
}
}
}
func TestTruncationDetected(t *testing.T) {
// A backup cut short mid-transfer must fail loudly rather than restore a
// plausible-looking prefix.
data := bytes.Repeat([]byte("payload"), 20000) // spans several chunks
enc := encrypt(t, TypeFullBackup, "password password", data)
// Drop the final chunk entirely: what remains is a sequence of valid,
// correctly authenticated chunks, none of which is marked last.
var chunks []int
for off := headerLen; off < len(enc); {
n := int(enc[off])<<24 | int(enc[off+1])<<16 | int(enc[off+2])<<8 | int(enc[off+3])
chunks = append(chunks, off)
off += 4 + n
}
if len(chunks) < 2 {
t.Fatalf("expected several chunks, got %d", len(chunks))
}
truncated := enc[:chunks[len(chunks)-1]]
r, err := NewReader(bytes.NewReader(truncated), "password password")
if err != nil {
t.Fatalf("NewReader (truncated): %v", err)
}
if _, err := io.ReadAll(r); !errors.Is(err, ErrCorrupt) {
t.Fatalf("read truncated = %v, want ErrCorrupt", err)
}
}
func TestTamperDetected(t *testing.T) {
enc := encrypt(t, TypeFullBackup, "password password", []byte("some archive bytes"))
for _, off := range []int{MagicLen /* type byte */, headerLen + 6 /* ciphertext */} {
bad := bytes.Clone(enc)
bad[off] ^= 0xff
if _, _, err := decrypt(t, bad, "password password"); err == nil {
t.Errorf("flipping byte %d was accepted", off)
}
}
}
func TestChunkReorderRejected(t *testing.T) {
// Two full chunks plus a tail, so swapping the first two is possible without
// changing any length.
data := make([]byte, 2*chunkSize+10)
for i := range data {
data[i] = byte(i)
}
enc := encrypt(t, TypeFullBackup, "password password", data)
const framed = 4 + chunkSize + tagLen
first := headerLen
second := headerLen + framed
swapped := bytes.Clone(enc)
copy(swapped[first:first+framed], enc[second:second+framed])
copy(swapped[second:second+framed], enc[first:first+framed])
if _, _, err := decrypt(t, swapped, "password password"); !errors.Is(err, ErrWrongPassword) {
t.Fatalf("reordered chunks = %v, want authentication failure", err)
}
}
func TestEmptyPasswordRejected(t *testing.T) {
if _, err := NewWriter(io.Discard, TypeFullBackup, ""); err == nil {
t.Fatal("empty password accepted")
}
}
func TestUnreasonableKDFParamsRejected(t *testing.T) {
enc := encrypt(t, TypeFullBackup, "password password", []byte("x"))
bad := bytes.Clone(enc)
bad[MagicLen+2] = 40 // logN far beyond maxLogN
if _, err := NewReader(bytes.NewReader(bad), "password password"); !errors.Is(err, ErrCorrupt) {
t.Fatalf("err = %v, want ErrCorrupt", err)
}
}
+4
View File
@@ -32,6 +32,8 @@ type detailView struct {
// RateLimitErr surfaces a validation error from a domain- or application-level
// rate-limit form (spec 7.4) as a page banner.
RateLimitErr string
// ExportErr surfaces a rejected encryption password from the export card.
ExportErr string
}
// appRateLimitView pairs an application with its differentiated rate-limit
@@ -131,6 +133,8 @@ func (s *Server) renderDomainDetail(w http.ResponseWriter, r *http.Request, stat
"Wildcard": store.AddressModeWildcard,
"List": store.AddressModeList,
"RateLimitErr": view.RateLimitErr,
"ExportErr": view.ExportErr,
"MinPwLen": minSecretFilePasswordLen,
"DomainHasRL": domainRLok && domainRL.Active(),
"DomainRLIPs": strings.Join(domainRL.AllowedIPs, "\n"),
"DomainRLMax": intOrBlank(domainRL.MaxMessages),
+177 -11
View File
@@ -1,20 +1,25 @@
package web
import (
"bytes"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"time"
"codeberg.org/mix/selfpost/internal/backup"
"codeberg.org/mix/selfpost/internal/domain"
"codeberg.org/mix/selfpost/internal/secretfile"
"codeberg.org/mix/selfpost/internal/store"
)
// maxImportBytes caps a domain-import upload. A domain export is a small JSON
// document (a DKIM key and a handful of credentials); this leaves generous head
// room while refusing anything large enough to be an abuse attempt.
// room while refusing anything large enough to be an abuse attempt. An
// encrypted export adds only a header and per-chunk tags, so the same ceiling
// covers both forms.
const maxImportBytes = 1 << 20 // 1 MiB
// handleBackupPage renders the backup/migration screen: the full-server backup
@@ -27,11 +32,21 @@ func (s *Server) handleBackupPage(w http.ResponseWriter, r *http.Request) {
// renderBackupPage draws the page; importErr surfaces a failed domain import
// (spec 7.5.B) next to the form that produced it.
func (s *Server) renderBackupPage(w http.ResponseWriter, r *http.Request, status int, importErr string) {
s.renderBackupPageWith(w, r, status, importErr, "")
}
// renderBackupPageWith is renderBackupPage with the second of the page's two
// error slots: backupErr belongs to the full-backup card (a rejected encryption
// password), importErr to the import card, so neither message appears under the
// wrong form.
func (s *Server) renderBackupPageWith(w http.ResponseWriter, r *http.Request, status int, importErr, backupErr string) {
s.render(w, status, "backup", map[string]any{
"Title": "SelfPost — backup",
"User": currentUser(r),
"Active": "backup",
"ImportErr": importErr,
"BackupErr": backupErr,
"MinPwLen": minSecretFilePasswordLen,
})
}
@@ -39,36 +54,84 @@ func (s *Server) renderBackupPage(w http.ResponseWriter, r *http.Request, status
// authenticated admin action (this handler sits behind the auth middleware). The
// archive carries DKIM private keys, the admin password hash and SASL
// credentials, so it is served with no-store and as an attachment to discourage
// caching of secret material.
// caching of secret material. When the operator ticks "encrypt with a
// password", the archive is wrapped in a .spbk envelope on the way out, so the
// file that lands on their disk — wherever it is copied afterwards — is useless
// without the password.
func (s *Server) handleBackup(w http.ResponseWriter, r *http.Request) {
filename := fmt.Sprintf("selfpost-backup-%s.tar.gz", time.Now().UTC().Format("20060102-150405"))
password, pwErr := secretFilePassword(r)
if pwErr != "" {
s.renderBackupPageWith(w, r, http.StatusBadRequest, "", pwErr)
return
}
w.Header().Set("Content-Type", "application/gzip")
stamp := time.Now().UTC().Format("20060102-150405")
filename := fmt.Sprintf("selfpost-backup-%s.tar.gz", stamp)
contentType := "application/gzip"
if password != "" {
filename = fmt.Sprintf("selfpost-backup-%s%s", stamp, secretfile.ExtBackup)
contentType = "application/octet-stream"
}
w.Header().Set("Content-Type", contentType)
w.Header().Set("Content-Disposition", fmt.Sprintf("attachment; filename=%q", filename))
w.Header().Set("Cache-Control", "no-store")
if err := backup.Create(w, backup.Params{
// Everything below streams: past this point headers (and possibly some
// bytes) are already on the wire, so a failure cannot switch to a clean
// error page. Log it and let the truncated download fail loudly on the
// client side — for an encrypted archive that is a missing end-of-stream
// chunk, which decryption refuses outright.
sink := io.Writer(w)
var env *secretfile.Writer
if password != "" {
var err error
// The only failures here are key derivation (which happens before
// anything is written) and writing the envelope header, which fails only
// if the client is already gone.
env, err = secretfile.NewWriter(w, secretfile.TypeFullBackup, password)
if err != nil {
logf("panel: full backup: encrypt: %v", err)
http.Error(w, "backup failed", http.StatusInternalServerError)
return
}
sink = env
}
if err := backup.Create(sink, backup.Params{
DataDir: s.cfg.DataDir,
DBPath: s.cfg.DBPath,
Version: s.cfg.Version,
}); err != nil {
// Headers (and possibly some bytes) may already be on the wire, so we
// cannot switch to a clean error page; log it and let the truncated
// download fail loudly on the client side.
logf("panel: full backup failed: %v", err)
return
}
if env != nil {
if err := env.Close(); err != nil {
logf("panel: full backup failed: %v", err)
}
}
}
// handleExportDomain streams a single-domain export as a secret download (spec
// 7.5.B). Like the full backup it is POST-only (state is not changed, but the
// response contains the domain's DKIM private key and application passwords, so
// it must not be prefetchable or cached).
// it must not be prefetchable or cached). Like the full backup it can be
// encrypted with a password, in which case the download is a .spde envelope
// instead of plain JSON.
func (s *Server) handleExportDomain(w http.ResponseWriter, r *http.Request) {
d, ok := s.lookupDomain(w, r)
if !ok {
return
}
password, pwErr := secretFilePassword(r)
if pwErr != "" {
s.renderDomainDetail(w, r, http.StatusBadRequest, d, detailView{
FormMode: store.AddressModeWildcard,
ExportErr: pwErr,
})
return
}
exp, err := s.domains.Export(d.ID)
if err != nil {
logf("panel: export domain %d: %v", d.ID, err)
@@ -83,7 +146,29 @@ func (s *Server) handleExportDomain(w http.ResponseWriter, r *http.Request) {
}
filename := fmt.Sprintf("selfpost-domain-%s.json", d.Name)
w.Header().Set("Content-Type", "application/json")
contentType := "application/json"
if password != "" {
// An export is small, so it is sealed in memory: the response is only
// started once the ciphertext is complete and nothing can half-fail.
var buf bytes.Buffer
env, err := secretfile.NewWriter(&buf, secretfile.TypeDomainExport, password)
if err == nil {
_, err = env.Write(body)
}
if err == nil {
err = env.Close()
}
if err != nil {
logf("panel: export domain %d: encrypt: %v", d.ID, err)
http.Error(w, "export failed", http.StatusInternalServerError)
return
}
body = buf.Bytes()
filename = fmt.Sprintf("selfpost-domain-%s%s", d.Name, secretfile.ExtDomainExport)
contentType = "application/octet-stream"
}
w.Header().Set("Content-Type", contentType)
w.Header().Set("Content-Disposition", fmt.Sprintf("attachment; filename=%q", filename))
w.Header().Set("Cache-Control", "no-store")
_, _ = w.Write(body)
@@ -108,8 +193,49 @@ func (s *Server) handleImportDomain(w http.ResponseWriter, r *http.Request) {
}
defer file.Close()
// An encrypted export announces itself with the envelope magic, so the file
// decides which path it takes; the password field is only consulted when the
// file actually needs it, and a password typed for a plain file is a plain
// mistake worth reporting.
head := make([]byte, secretfile.MagicLen)
n, err := io.ReadFull(file, head)
if err != nil && !errors.Is(err, io.EOF) && !errors.Is(err, io.ErrUnexpectedEOF) {
s.renderBackupPage(w, r, http.StatusBadRequest, "Could not read the uploaded file.")
return
}
source := io.MultiReader(bytes.NewReader(head[:n]), file)
password := r.PostFormValue("import_password")
if secretfile.HasMagic(head[:n]) {
if password == "" {
s.renderBackupPage(w, r, http.StatusBadRequest, "That file is encrypted — enter the password it was exported with.")
return
}
env, err := secretfile.NewReader(source, password)
if err != nil {
s.renderBackupPage(w, r, http.StatusBadRequest, decryptErrorMessage(err))
return
}
if env.Type() != secretfile.TypeDomainExport {
s.renderBackupPage(w, r, http.StatusBadRequest, "That file is an encrypted "+env.Type().String()+", not a domain export.")
return
}
// Read the whole plaintext first: authentication of the last chunk is
// what proves the file is intact, and a streaming JSON decoder could
// accept a truncated document before ever reaching it.
plain, err := io.ReadAll(env)
if err != nil {
s.renderBackupPage(w, r, http.StatusBadRequest, decryptErrorMessage(err))
return
}
source = bytes.NewReader(plain)
} else if password != "" {
s.renderBackupPage(w, r, http.StatusBadRequest, "That file is not encrypted — leave the password empty.")
return
}
var exp domain.DomainExport
dec := json.NewDecoder(file)
dec := json.NewDecoder(source)
dec.DisallowUnknownFields()
if err := dec.Decode(&exp); err != nil {
s.renderBackupPage(w, r, http.StatusBadRequest, "That file is not a valid SelfPost domain export.")
@@ -134,6 +260,46 @@ func (s *Server) handleImportDomain(w http.ResponseWriter, r *http.Request) {
http.Redirect(w, r, fmt.Sprintf("/domains/%d?imported=1", d.ID), http.StatusSeeOther)
}
// secretFilePassword reads the "encrypt this download with a password" controls
// shared by the full-backup and domain-export forms. It returns the password to
// encrypt with — empty when the box is not ticked, which keeps the plain
// .tar.gz/.json behaviour of earlier versions — or a message to show above the
// form. The confirmation field is checked here rather than in the browser
// because a typo in an encryption password is unrecoverable: the archive would
// be sealed with a secret the operator does not know.
func secretFilePassword(r *http.Request) (password, errMsg string) {
if err := r.ParseForm(); err != nil {
return "", "Invalid form submission."
}
if r.PostFormValue("encrypt") == "" {
return "", ""
}
password = r.PostFormValue("password")
if len([]rune(password)) < minSecretFilePasswordLen {
return "", fmt.Sprintf("The encryption password must be at least %d characters.", minSecretFilePasswordLen)
}
if password != r.PostFormValue("password_confirm") {
return "", "The two passwords do not match."
}
return password, ""
}
// decryptErrorMessage phrases an envelope failure for the operator. A wrong
// password and a damaged file are deliberately indistinguishable to the code,
// so the message names both possibilities.
func decryptErrorMessage(err error) string {
switch {
case errors.Is(err, secretfile.ErrWrongPassword):
return "Wrong password, or the file has been altered since it was exported."
case errors.Is(err, secretfile.ErrCorrupt):
return "That file is damaged or incomplete."
case errors.Is(err, secretfile.ErrNotEncrypted):
return "That file is not a SelfPost export."
default:
return "Could not decrypt the file."
}
}
// importErrorMessage maps a domain-import failure (already logged by the caller)
// to an HTTP status and a user-facing message. Duplicate domain/login are called
// out specifically; other failures — validation errors describing what is wrong
+127
View File
@@ -0,0 +1,127 @@
package web
import (
"errors"
"fmt"
"net/http"
"net/http/httptest"
"net/url"
"strings"
"testing"
"codeberg.org/mix/selfpost/internal/secretfile"
)
// postForm builds the kind of request the backup and export forms submit.
func postForm(values url.Values) *http.Request {
r := httptest.NewRequest(http.MethodPost, "/backup", strings.NewReader(values.Encode()))
r.Header.Set("Content-Type", "application/x-www-form-urlencoded")
return r
}
// The encryption password is only ever typed once into a file nobody can
// recover without it, so every way of getting it wrong has to be caught before
// the archive is sealed — and leaving the box unticked has to keep producing
// the plain archive earlier versions produced.
func TestSecretFilePassword(t *testing.T) {
long := strings.Repeat("x", minSecretFilePasswordLen)
short := strings.Repeat("x", minSecretFilePasswordLen-1)
tests := []struct {
name string
form url.Values
wantPass string
wantErr bool
}{
{
name: "unticked box means no encryption",
form: url.Values{"password": {long}, "password_confirm": {long}},
wantPass: "",
},
{
name: "ticked with a matching password",
form: url.Values{"encrypt": {"1"}, "password": {long}, "password_confirm": {long}},
wantPass: long,
},
{
name: "mistyped confirmation",
form: url.Values{"encrypt": {"1"}, "password": {long}, "password_confirm": {long + "!"}},
wantErr: true,
},
{
name: "too short",
form: url.Values{"encrypt": {"1"}, "password": {short}, "password_confirm": {short}},
wantErr: true,
},
{
name: "ticked but empty",
form: url.Values{"encrypt": {"1"}},
wantErr: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
pass, errMsg := secretFilePassword(postForm(tt.form))
if tt.wantErr {
if errMsg == "" {
t.Fatalf("password %q accepted, want a rejection", pass)
}
if pass != "" {
t.Errorf("a rejected form still yielded password %q", pass)
}
return
}
if errMsg != "" {
t.Fatalf("unexpected rejection: %s", errMsg)
}
if pass != tt.wantPass {
t.Errorf("password = %q, want %q", pass, tt.wantPass)
}
})
}
}
// The messages the import form shows must distinguish the operator's likely
// mistakes; a wrong password and a tampered file stay deliberately merged.
func TestDecryptErrorMessage(t *testing.T) {
tests := []struct {
err error
want string
}{
{secretfile.ErrWrongPassword, "Wrong password"},
{fmt.Errorf("read: %w", secretfile.ErrCorrupt), "damaged"},
{secretfile.ErrNotEncrypted, "not a SelfPost export"},
{errors.New("something else"), "Could not decrypt"},
}
for _, tt := range tests {
if got := decryptErrorMessage(tt.err); !strings.Contains(got, tt.want) {
t.Errorf("decryptErrorMessage(%v) = %q, want it to mention %q", tt.err, got, tt.want)
}
}
}
// The encryption controls are shared markup pulled into two pages; a page that
// forgets to include the partial (or the data it needs) loses the option
// silently, since the plain download still works.
func TestBackupPageOffersEncryption(t *testing.T) {
tmpl, err := loadTemplates()
if err != nil {
t.Fatalf("loadTemplates: %v", err)
}
s := &Server{tmpl: tmpl, cfg: Config{Version: "test"}}
rec := httptest.NewRecorder()
s.renderBackupPageWith(rec, httptest.NewRequest(http.MethodGet, "/backup", nil),
http.StatusOK, "", "The two passwords do not match.")
body := rec.Body.String()
for _, want := range []string{
`name="encrypt"`, `name="password"`, `name="password_confirm"`,
`name="import_password"`, "data-encrypt-toggle", "data-encrypt-fields",
fmt.Sprintf("at least %d characters", minSecretFilePasswordLen),
"The two passwords do not match.",
} {
if !strings.Contains(body, want) {
t.Errorf("backup page is missing %q", want)
}
}
}
+19
View File
@@ -249,3 +249,22 @@ button.copy { flex: none; margin-top: 0.3rem; }
}
.actions button.danger:hover, .actions a.danger:hover, .nav button.danger:hover { background: #3d1a18 !important; }
}
/* The optional "encrypt this download" block on the backup, export and import
forms. Its label is the one checkbox in the panel, so it opts out of the
block-level label rule above and sits on one line with its box; the fields it
reveals are indented under it to read as its consequence rather than as three
more fields of the form. panel.js hides the inner block until the box is
ticked (and empties it when unticked); without JavaScript everything stays
visible, which the server handles identically. */
.encrypt { margin-top: 1.2rem; }
.encrypt label.check {
display: flex; align-items: center; gap: 0.5rem; margin: 0; font-weight: 600;
}
.encrypt label.check input { width: auto; margin: 0; }
.encrypt-fields {
margin-left: 1.6rem; padding-left: 0.9rem; border-left: 2px solid #e2e5e9;
}
.encrypt-fields label { margin-top: 0.7rem; }
.encrypt-fields .muted { margin: 0.5rem 0 0; font-size: 0.85rem; }
@media (prefers-color-scheme: dark) { .encrypt-fields { border-color: #2b3138 !important; } }
+31
View File
@@ -72,7 +72,38 @@
});
}
// --- Encryption password fields shown only when asked for --------------
// The backup, export and import forms carry an optional password block. It
// is hidden until the checkbox next to it is ticked, and cleared when it is
// unticked, so a password typed and then abandoned is never submitted. With
// JavaScript blocked the block stays visible and the forms behave exactly as
// the server reads them: the checkbox alone decides whether encryption
// happens.
function syncEncryptFields(box) {
var form = box.closest("form");
var fields = form && form.querySelector("[data-encrypt-fields]");
if (!fields) {
return;
}
fields.hidden = !box.checked;
if (!box.checked) {
fields.querySelectorAll("input").forEach(function (input) {
input.value = "";
});
}
}
function initEncryptFields(root) {
root.querySelectorAll("input[data-encrypt-toggle]").forEach(function (box) {
syncEncryptFields(box);
box.addEventListener("change", function () {
syncEncryptFields(box);
});
});
}
document.addEventListener("DOMContentLoaded", function () {
initAddressFields(document);
initEncryptFields(document);
});
})();
+5 -3
View File
@@ -21,14 +21,16 @@ type templates struct {
// pageFiles maps a logical page name to its template files. Every page
// composes with layout.html; pages that embed a polling fragment (spec 7.1)
// list that fragment's file too, so the same {{define}} block renders both
// the initial page and the fragment's own refresh responses identically.
// the initial page and the fragment's own refresh responses identically. Pages
// sharing a block of markup (the encryption fields on the two secret downloads)
// list that partial the same way.
var pageFiles = map[string][]string{
"setup": {"templates/setup.html"},
"login": {"templates/login.html"},
"dashboard": {"templates/dashboard.html"},
"account": {"templates/account.html"},
"backup": {"templates/backup.html"},
"domain_detail": {"templates/domain_detail.html"},
"backup": {"templates/backup.html", "templates/encrypt_fields.html"},
"domain_detail": {"templates/domain_detail.html", "templates/encrypt_fields.html"},
"domain_delete": {"templates/domain_delete.html"},
"deliveries": {"templates/deliveries.html", "templates/deliveries_rows.html"},
"mail_queue": {"templates/mail_queue.html", "templates/mail_queue_body.html"},
+16 -3
View File
@@ -10,8 +10,11 @@
start. TLS certificates and the mail queue are not included.</p>
<p class="muted"><strong>The backup file is a secret</strong> (it contains
private keys and credentials). Store and transfer it securely and delete it
once the restore succeeds.</p>
<form class="inline" method="post" action="/backup">
once the restore succeeds. Encrypting it below is the simplest way to do that:
the download is then a <code>.spbk</code> file that only the password opens.</p>
{{if .BackupErr}}<p class="error">{{.BackupErr}}</p>{{end}}
<form method="post" action="/backup">
{{template "encryptfields" .}}
<button type="submit">Download full backup</button>
</form>
</div>
@@ -25,7 +28,17 @@
{{if .ImportErr}}<p class="error">{{.ImportErr}}</p>{{end}}
<form method="post" action="/domains/import" enctype="multipart/form-data">
<label for="importfile">Domain export file</label>
<input id="importfile" name="file" type="file" accept=".json,application/json" required>
<input id="importfile" name="file" type="file" accept=".json,.spde,application/json" required>
<div class="encrypt">
<label class="check">
<input type="checkbox" data-encrypt-toggle>
<span>The file is encrypted (<code>.spde</code>)</span>
</label>
<div class="encrypt-fields" data-encrypt-fields>
<label for="importpw">Password</label>
<input id="importpw" name="import_password" type="password" autocomplete="off">
</div>
</div>
<button type="submit">Import domain</button>
</form>
</div>
+5 -2
View File
@@ -283,8 +283,11 @@
import the DNS record stays the same, so no DNS change is needed.</p>
<p class="muted"><strong>The export file is a secret</strong> — it contains the
private DKIM key and application passwords. Transfer it securely and delete it
after the import.</p>
<form class="inline" method="post" action="/domains/{{.Domain.ID}}/export">
after the import, or encrypt it below and move a <code>.spde</code> file
instead; the import form asks for the password.</p>
{{if .ExportErr}}<p class="error">{{.ExportErr}}</p>{{end}}
<form method="post" action="/domains/{{.Domain.ID}}/export">
{{template "encryptfields" .}}
<button type="submit">Export domain</button>
</form>
</div>
@@ -0,0 +1,23 @@
{{/* Password fields shared by the full-backup and domain-export forms. The
checkbox is the switch: unticked, the download keeps its historic plain
form (.tar.gz / .json); ticked, the file is sealed in a password-encrypted
envelope (.spbk / .spde). The fields start hidden and are revealed by
panel.js — with JavaScript blocked they are simply always visible, and the
server still decides from the checkbox alone. */}}
{{define "encryptfields"}}
<div class="encrypt">
<label class="check">
<input type="checkbox" name="encrypt" value="1" data-encrypt-toggle>
<span>Encrypt with a password</span>
</label>
<div class="encrypt-fields" data-encrypt-fields>
<label for="encpw">Password</label>
<input id="encpw" name="password" type="password" autocomplete="new-password"
minlength="{{.MinPwLen}}" placeholder="at least {{.MinPwLen}} characters">
<label for="encpw2">Repeat password</label>
<input id="encpw2" name="password_confirm" type="password" autocomplete="new-password">
<p class="muted">Keep this password: without it the file cannot be opened,
and SelfPost does not store it anywhere.</p>
</div>
</div>
{{end}}
+6
View File
@@ -15,6 +15,12 @@ const (
maxUsernameLen = 64
)
// minSecretFilePasswordLen is the floor for the password protecting an
// encrypted backup or domain export. Such a file is offline and can be attacked
// at leisure, so the floor matches the administrator password's rather than the
// weaker "any password is better than none".
const minSecretFilePasswordLen = minAdminPasswordLen
// validateUsername enforces a strict server-side whitelist (spec 7.6.2):
// letters, digits, dot, dash, underscore. Client validation is never trusted.
func validateUsername(u string) error {