From 2cfd239d16b19d738661c1f28ce711e598e9ed3c Mon Sep 17 00:00:00 2001 From: Mix Date: Fri, 16 Oct 2026 01:56:56 +0300 Subject: [PATCH] docs: panel-dkim-signing stage 3 - the documents without OpenDKIM MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit architecture § OpenDKIM becomes § DKIM signing (keys, the selector table and when it is rewritten, the two sockets, what is signed and the limits, the old tree); the process table, start order, liveness, the mail-path diagram, § Sender check, Persistence and the restore Resync follow the code. security.md: the panel signs with the login's key, the keys are the panel's alone, group selfpost and CAP_KILL as they are now; OpenDKIM's accepted risks go; new ones: what sendmail(1) submits is signed as the server, Sender: is signed but not judged (the owner's decision), and go-milter keeps every header per session, so the signer's 1 MiB refusal does not bound the library's copy. guide: Health, the container log's dkim-signer lines, backups, the long-line limit, and upgrading to 2.8.0 - with the rollback to 2.7.x: its start reads the queue as its own postfix (uid 101) before it hands the queue over, so the hand-over by name is run in the 2.7.x image first (measured on a copy of the test server's queue: postsuper failed before, passed after). development, README, SECURITY, product, the assumptions register's header: what left the image. entrypoint.sh: a comment, the rollback note as the guide gives it. Plan stage 3 lines ticked, roadmap 12/14. Written by four agents, one document each, against a brief of what the code does; reviewed here. Co-Authored-By: Claude Opus 5.5 --- README.md | 6 +- SECURITY.md | 3 +- build/entrypoint.sh | 4 +- docs/architecture.md | 224 ++++++++++++++++++-------- docs/assumptions-2026-10-14.md | 8 + docs/development.md | 66 +++++--- docs/guide.md | 131 ++++++++++++--- docs/plans/panel-dkim-signing.md | 25 ++- docs/product.md | 4 +- docs/roadmap.md | 2 +- docs/security.md | 266 ++++++++++++++++++++++++------- 11 files changed, 549 insertions(+), 190 deletions(-) diff --git a/README.md b/README.md index b7f797d..133cab6 100644 --- a/README.md +++ b/README.md @@ -6,9 +6,9 @@ Self-hosted SMTP relay with a web panel — one Docker image, no provider. -Postfix, OpenDKIM, and a small Go panel run together under `supervisord`; you -configure domains, DKIM keys, and SASL applications once, then point your apps -at the SMTP endpoint. +Postfix and a small Go panel, which also signs outgoing mail with DKIM, run +together under `supervisord`; you configure domains, DKIM keys, and SASL +applications once, then point your apps at the SMTP endpoint. SelfPost sends mail straight to the internet from **your own IP**, with per-domain DKIM signing. No mailboxes or webmail. An optional inbound relay diff --git a/SECURITY.md b/SECURITY.md index e5dd6aa..bde9bce 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -70,7 +70,8 @@ These are the operator's responsibility or accepted trade-offs, documented in - Deliverability outcomes: mail rejected or filtered by a receiving provider is a policy decision of that provider, not a defect - Denial of service through sheer volume against a single-tenant relay -- Vulnerabilities in upstream Postfix, OpenDKIM, or the base image — report +- Vulnerabilities in upstream Postfix, the Go libraries SelfPost builds on, + or the base image — report those upstream; if SelfPost's configuration makes an upstream issue exploitable when it otherwise would not be, that *is* in scope diff --git a/build/entrypoint.sh b/build/entrypoint.sh index 47c6365..67663c7 100644 --- a/build/entrypoint.sh +++ b/build/entrypoint.sh @@ -120,8 +120,8 @@ find /data -mindepth 1 -maxdepth 1 ! -name log ! -name postfix ! -name opendkim # panel would otherwise make the server's own name a new key, and its # published record would no longer match. The selector table is written by # the panel before its signer serves (cmd/panel/signer.go). After a rollback -# to 2.7.x, a domain added there has its key only in the old tree: remove -# /data/dkim before the next start of 2.8.x to copy again. +# to 2.7.x, a domain added there has its key only in the old tree: its +# directory is copied into /data/dkim/keys by hand (guide § Upgrading). # (util-linux 2.41, setpriv(1): --reuid, --regid, --clear-groups; cp(1) # -P: never follow a symbolic link in the source; chown -R and find never # follow one either. GNU chmod keeps a directory's set-group-ID bit under a diff --git a/docs/architecture.md b/docs/architecture.md index eac612f..d87a3d2 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -26,15 +26,17 @@ Managed programs ([build/supervisord.conf](../build/supervisord.conf)): | Program | User | Priority | Role | |---|---|---|---| -| `syslog-relay` | panel | 50 | The container's `/dev/log`: passes what OpenDKIM logs through syslog(3) to the container's log (`panel -syslog-relay`) | -| `opendkim` | root → `opendkim` (`UserID` in opendkim.conf) | 100 | DKIM signing milter | -| `panel` | panel | 200 | HTTP UI + sender check + DKIM signer + journal-milter + log-tailer + rate-limit recalc; with the inbound relay on also the inbound journal milter | -| `postfix` | root (wrapper) | 300 | MTA — started only after the milter sockets exist (OpenDKIM's, the sender check's, the DKIM signer's two, the journal milter's and, with the inbound relay on, the inbound journal milter's) | +| `panel` | panel | 200 | HTTP UI + sender check + DKIM signer + journal-milter + log-tailer + rate-limit recalc + notifier; with the inbound relay on also the inbound journal milter | +| `postfix` | root (wrapper) | 300 | MTA — started only after the milter sockets exist (the sender check's, the DKIM signer's two, the journal milter's and, with the inbound relay on, the inbound journal milter's) | | `postfix-reload` | root | — | On-demand `postfix reload` (autostart off) | | `cert-reload` | root | 400 | Daily: re-copies a renewed TLS certificate for Postfix and reloads it | | `logrotate` | root | 400 | Periodic `mail.log` rotation | -Start order: syslog-relay → OpenDKIM → panel (opens the sender check's, the DKIM signer's and the journal-milter's sockets) → Postfix wrapper +Until `2.8.0` two more programs ran before the panel: `opendkim`, the DKIM +signing milter, and `syslog-relay`, the container's `/dev/log` for what +OpenDKIM logged; the panel signs now (§ DKIM signing). + +Start order: panel (opens the sender check's, the DKIM signer's two and the journal-milter's sockets) → Postfix wrapper polls unix sockets (timeout `MILTER_WAIT_TIMEOUT`, default 30s) then runs `postfix start-fg` and stays its parent: on TERM it runs `postfix stop`, so a stop of the program or of the container stops Postfix as Postfix means it to @@ -55,14 +57,15 @@ new one: the entrypoint clears what the last run left in `/run` (the milter sockets, the sender check's alive mark). Besides a stop, supervisord itself dying, or the entrypoint refusing a setting, nothing else exits the container. `logrotate`'s FATAL (a config logrotate would ignore), -`postfix-reload`'s (reported to the page that asked), `cert-reload`'s and -`syslog-relay`'s stay on Server › Health instead. +`postfix-reload`'s (reported to the page that asked) and `cert-reload`'s +stay on Server › Health instead. The panel runs `postfix-reload` through supervisord's XML-RPC interface and waits for its exit status (`internal/supervisor`, `RunOnce`); the program rests EXITED after a run. -**Liveness:** `GET /healthz` (unauthenticated) returns 200 when opendkim, -panel, and postfix are RUNNING; Docker `HEALTHCHECK` uses the same probe (its +**Liveness:** `GET /healthz` (unauthenticated) returns 200 when panel - whose +roles include the DKIM signer - and postfix are RUNNING +(`internal/health`, `Liveness`); Docker `HEALTHCHECK` uses the same probe (its 90-second start period covers postfix's STARTING). --- @@ -73,7 +76,7 @@ panel, and postfix are RUNNING; Docker `HEALTHCHECK` uses the same probe (its Client ──TLS+SASL──► Postfix (465 smtps, optional 587 submission) │ ├─► sender check (From: against the login, tempfail on failure) - ├─► OpenDKIM milter (sign, tempfail on failure) + ├─► DKIM signer (the login's domain, tempfail on failure) ├─► journal-milter (send log + L2 rate limits, fail-open) └─► outbound MX delivery (port 25 client) @@ -96,7 +99,7 @@ smtp/inet` removes Debian's stock smtpd). Outbound delivery still uses the - **587/submission** — only when `SUBMISSION_ENABLE=true`; STARTTLS with `smtpd_tls_security_level=encrypt`. - **25/smtp inet** — when `INBOUND_RELAY_ENABLE=true` and/or - `DMARC_REPORTS_ENABLE=true`. No SASL, no OpenDKIM, no journal-milter — + `DMARC_REPORTS_ENABLE=true`. No SASL, no sender check, no signing, no journal-milter — `smtpd_milters` is always overridden on this service, because main.cf's submission chain would otherwise let a signing outage defer relayed mail and would file inbound mail into the outbound send log under the sender's @@ -216,24 +219,104 @@ smtp/inet` removes Debian's stock smtpd). Outbound delivery still uses the the certificate and key belong together it renews the copy (each file renamed into place) and reloads Postfix. -### OpenDKIM +### DKIM signing -Per-domain keys under `/data/opendkim/keys`; `KeyTable` / `SigningTable` maintained -by the panel. Socket `/run/opendkim/opendkim.sock`. +The panel signs outgoing mail itself, in a role of its own, `dkim-signer` +([cmd/panel/signer.go](../cmd/panel/signer.go), +[internal/milter/sign.go](../internal/milter/sign.go)), since `2.8.0`; until +then OpenDKIM did, in a program of its own +([plans/panel-dkim-signing.md](plans/panel-dkim-signing.md)). -The server's own name (`SELFPOST_HOSTNAME`) has a key and a line in both -tables too, without being a row of the registry -(`domain.Service.SetServerName`): the panel's notifications are sent as -`postmaster@` that name and signed with it. No application can use it as an -envelope sender - the sender map is built from the registry - and since -`2.4.0` none can have a message signed as the server by writing it into -`From:` either: the sender check holds the header against the login (§ Sender -check). A sending domain of the same name shares its key and selector with -the server; deleting that domain leaves the server signed under the default -selector, which is the same key only if that was the domain's. At its start -the notifier makes sure the tables have the line and name the selector the -panel shows (`EnsureServerSigned`): tables are otherwise rebuilt only when a -domain is added or deleted. +- **Keys** — `/data/dkim/keys//.private`, PKCS#1 PEM, one + for each sending domain and one for the server's own name + (`internal/domain`, `Keys`; `DKIM_DIR`, default `/data/dkim`). A key is + made when a domain is added and an existing one is reused, never + overwritten: that would invalidate the record already published + (`EnsureKey`). A domain export carries the key and an import writes it. A + deleted domain's key directory is removed once the selector table no + longer names it; one a deletion had to leave, Reload configuration removes + (`Service.Resync`). +- **The selector table** — `/data/dkim/selectors`, a line ` ` + for every name the server signs, sorted, written whole from the database + and renamed into place (`Keys.Rebuild`). It is rewritten when a domain is + added, deleted or imported, by Server › Health › Reload configuration, + after a restore, and at every start of the signer before it serves + (`WriteSelectors`): Postfix waits for the signer's sockets, so no message + meets a table from before - one a rewrite cut short left, the first start + of `2.8.0` without one, or one naming the old selector after a changed + `DKIM_SELECTOR_DEFAULT`. A rewrite at start that fails is tried again every + 10 seconds; meanwhile the signer defers what it finds no key for, and the + pages stay up. Each role builds a domain service of its own, so the + writers are serialised by a mutex (`tableMu`): a writer that read the + registry before another's domain was added cannot write after it. +- **The signer reads files, not the database.** The panel's one SQLite + connection is shared with the pages, and a long query would hold up mail. + It reads the table for every message (a few lines; a stamp of time and + size would miss a rewrite of the same size within one tick of the file + system's clock) and a key again when its file's time or size changes; a key + the table no longer names is forgotten. Nothing is reloaded. +- **The panel's alone.** All of `/data/dkim` is `panel:panel`, directories + `0700`, files `0600`, set by the entrypoint at every start: no group, no + other program reads it. +- **Two sockets, both strict** (`default_action=tempfail`, so while the role + does not answer mail waits, never goes unsigned): + `DKIM_SIGN_SOCKET` (`/run/selfpost/sign.sock`) for what applications send + on 465/587, signed with the key of the domain the SASL login belongs to - + `{auth_authen}`, then `Bindings.DomainOf`, the `sender_login_maps` file + Postfix and the sender check read; `DKIM_SIGN_LOCAL_SOCKET` + (`/run/selfpost/sign-local.sock`) for what the panel itself submits with + `sendmail(1)` (`non_smtpd_milters`), signed with the key of the server's + name. The socket a message comes through, not the absence of a login, + decides that the server's key may sign it. +- **The server's own name** (`SELFPOST_HOSTNAME`) has a key and a line in the + table without being a row of the registry (`domain.Service.SetServerName`): + the panel's notifications are sent as `postmaster@` that name and signed + with it. The name is kept in lower case for DKIM, as the signer finds keys + by the name in lower case; the SASL realm, which defaults to the hostname, + stays as written. No application can use it as an envelope sender - the + sender map is built from the registry - and since `2.4.0` none can have a + message signed as the server by writing it into `From:` either: the sender + check holds the header against the login (§ Sender check). A sending + domain of the same name shares its key and selector with the server; + deleting that domain leaves the server signed under the default selector, + which is the same key only if that was the domain's. +- **What is signed** — `relaxed/simple`, `rsa-sha256`, with + `github.com/emersion/go-msgauth` v0.7.0 (`dkim`). `h=`: every header of + the message whose name is in RFC 6376 § 5.4.1's list as OpenDKIM's + `dkim_should_signhdrs[]` held it, and `Sender` and `List-Unsubscribe-Post` + besides (`signHeaders`), each occurrence, in message order; then, once + more, each of `Subject`, `Date`, `To`, `Cc`, `Reply-To` and `Sender` the + message has (`overSign` - only those present: Postfix's cleanup adds a + missing `Date:` after the milters, out of their sight, and a name in `h=` + the message lacked would break the signature over the one added); then `From` once more, last (`signLines`). Each header is hashed as + Postfix will deliver it: the value with its leading space (the signer asks + for `SMFIP_HDR_LEADSPC`), the lines of a folded one joined with CRLF. The + signature is inserted at the top, above Postfix's own `Received:`, as + `DKIM-Signature: ...`. `disable_mime_output_conversion=yes` + ([build/postfix-config.sh](../build/postfix-config.sh)): an 8-bit message + is delivered as it was signed, not converted to 7-bit for a server that + does not announce 8BITMIME. +- **Limits.** What cannot be signed - no line in the table, no key file, a + login with no domain, a read error - is deferred. A header section over + 1 MiB is refused, `552 5.3.4`: the signer holds it whole until the body + starts. A body line over 998 bytes is logged (`dkim-signer: ...`) + and the message goes on: Postfix's SMTP client breaks such a line on + delivery (`smtp_line_length_limit`), and the signature will not verify. A + message has an hour from the end of its headers to its end - go-milter + v0.4.1 tells a session nothing of a dropped connection - after which it is + deferred, never signed over part of its body. +- **Logs** go to the panel's stdout, the container's log, as `dkim-signer:` + lines; nothing is written to `mail.log` (nor did OpenDKIM write there). +- **The old tree,** `/data/opendkim` - the keys, `KeyTable` and + `SigningTable` until `2.8.0`, shared with OpenDKIM through group + `selfpost`. While `/data/dkim` holds no key and the old tree does, the + entrypoint copies `/data/opendkim/keys` to `/data/dkim/keys` as the panel + user (`setpriv`), into `/data/dkim.new`, renamed when whole; a copy that + fails stops the start - the server's name would otherwise get a new key + and its published record would no longer match. Keys, selectors and DNS + records do not change. The old tree is left for a rollback to a 2.7.x + image until the next release, made the panel's alone at every start like + the new one, and left out of backups (§ Persistence). ### Panel binary ([cmd/panel](../cmd/panel)) @@ -261,9 +344,8 @@ milter, § Postfix above, port 25): with the key of the domain the SASL login belongs to) and `DKIM_SIGN_LOCAL_SOCKET` (the panel's own mail, `non_smtpd_milters`, signed with the key of the server's name); after the sender check and - **strict**. `relaxed/simple`, RFC 6376 § 5.4.1's headers and `From` once - more, go-msgauth/dkim; keys and selectors from OpenDKIM's KeyTable while - both sign (`internal/milter/sign.go`). + **strict**. Keys and the selector table from `/data/dkim`, no database: + § DKIM signing above. 4. **journal-milter** — unix socket `JOURNAL_MILTER_SOCKET`; records From/To/ Subject/SASL user at DATA; enforces level-2 rate limits; **fail-open** (`default_action=accept`) so milter failure does not stop mail. Domain @@ -287,12 +369,13 @@ milter, § Postfix above, port 25): 7. **notifier** — writes to the panel's users when something needs them (§ Notifications below). -Milter chain in Postfix: the sender check (tempfail), the panel's DKIM -signer (tempfail), OpenDKIM (tempfail), then the journal (accept on -failure); until OpenDKIM leaves the image both sign every message -(docs/plans/panel-dkim-signing.md, stage 1). Mail submitted locally with -`sendmail(1)` — the notifier's — passes the signer's own socket and OpenDKIM -(`non_smtpd_milters`): it is signed, it is the panel's own and so needs no +Milter chain in Postfix ([build/postfix-config.sh](../build/postfix-config.sh), +`smtpd_milters`): the sender check (tempfail), the DKIM signer (tempfail), +then the journal (accept on failure) - the check first, since the signature +is held to the `From:` domain and a message the check could not judge must +not be signed. Until `2.8.0` OpenDKIM signed in the signer's place. Mail +submitted locally with `sendmail(1)` — the notifier's — passes the signer's +own socket alone (`non_smtpd_milters`, tempfail): it is signed, it is the panel's own and so needs no check of whose it is, and it is not an application's mail to be journalled or limited. Bounces Postfix generates pass no milter. @@ -302,10 +385,13 @@ generates pass no milter. [internal/milter/sender.go](../internal/milter/sender.go) holds a message's `From:` header against the application that sends it ([plans/from-header-binding.md](plans/from-header-binding.md)). Postfix holds -the envelope sender against the SASL login, and OpenDKIM chooses the signing -key by the header; until `2.4.0` nothing connected the two, and an -application could have mail signed as another domain of the server or as the -server itself. +the envelope sender against the SASL login, and the signer signs with the key +of the login's domain (§ DKIM signing): the signature's `d=` is that domain, +and a receiver holds it to the domain of `From:` (DMARC, RFC 7489 § 3.1.1). +Without the check an application could send under the server's signature as +another domain of the server or as the server itself. The check came in +`2.4.0`, when OpenDKIM still signed and chose its key by the header itself; +the hole it closed was the same. - **The rule is the envelope's.** The check reads the file Postfix reads for `smtpd_sender_login_maps` and makes the same lookup: the address itself is @@ -324,20 +410,20 @@ server itself. `src/global/mail_addr_find.c`, `mail_addr_find_opt()`). A key written twice makes Postfix refuse the whole table, and the check defers then too. - **How the header is read.** Exactly one `From:` field (none, or more than - one, is refused), read with OpenDKIM's own parser: libopendkim's - `dkim_mail_parse()`, ported line for line from Debian's opendkim - 2.11.0~beta2 (`internal/milter/mailparse.go`) and tested against the C on - two hundred thousand lines. The check asks Postfix for header values with - their leading space, as OpenDKIM does (`SMFIP_HDR_LEADSPC`), so both read the - same bytes. The address it returns is the one judged - the one OpenDKIM - signs for - whatever a display name, a comment or a quoted local part - holds; a parse failure, no address, or a second one after it is refused, - and so is an address over 256 bytes, which OpenDKIM would match its - `SigningTable` against cut short (`dkimf_apply_signtable()`, - `MAXADDRESS`). Encoded words are not decoded, by either. The line must also - be one address in RFC 5322's grammar (`net/mail`), at the domain OpenDKIM - signs for, so that a mail program shows the address that is signed; that - address, its local part unquoted, is what the bindings are asked about. + one, is refused). The line, its folds joined, must be one address in RFC + 5322's grammar as Go's `net/mail` reads it - a mail program's reading, the + address it shows; a display name may say anything (an encoded one in a + charset Go does not convert is passed through unread). A line a lenient + program could show as another address - `ceo@bank.com `, + `` - is refused. That address, its local part + unquoted as Postfix holds the envelope's, is what the bindings are asked + about, folded to lower case in ASCII only: a Unicode fold would make the + Kelvin sign a `k` and let a look-alike pass. The check asks Postfix for + header values with their leading space (`SMFIP_HDR_LEADSPC`), as the + signer does, so both read the same bytes. Until `2.8.0` the line was also + read with a port of OpenDKIM's own parser, so that the address judged was + the one OpenDKIM signed for, and an address over 256 bytes was refused; + both went with OpenDKIM. - **The verdict** comes at the end of the headers: `550 5.7.1`, permanent, naming the line, as a reply code of the milter's own. What the client wrote is reduced to printable ASCII and cut before it is said back, logged or @@ -925,19 +1011,19 @@ the cookie and the restored row's `expires_at` has not passed. ## Code layers Multi-store writes that must land in more than one place (SQLite row, -`sasldb2` entry, Postfix map, OpenDKIM table) go through a service, which is +`sasldb2` entry, Postfix map, DKIM key or selector table) go through a service, which is also where the rollback of a partial failure lives. Handlers may call `store` directly for single-table reads and simple writes (sessions, users, send-log queries); the first-run setup-token file is read and written in `web` itself. The adapters below the services are the only code that knows -about Postfix, OpenDKIM, DNS or the log file, which is what makes them +about Postfix, the milter protocol, DNS or the log file, which is what makes them substitutable in tests — `milter.Store`, `app.SenderMaps` and `logtail.StatusStore` are the seams the unit tests replace with fakes. ```mermaid flowchart TB subgraph cmd ["cmd — composition root"] - panel["panel: HTTP + journal-milter + log-tailer"] + panel["panel: HTTP + sender check + DKIM signer + journal-milter + log-tailer"] backupcli["selfpost-backup CLI"] end subgraph web ["internal/web — HTTP surface"] @@ -995,7 +1081,7 @@ flowchart TB dmarcSvc --> postfix ``` -The roles inside the `panel` process (HTTP server, sender check, +The roles inside the `panel` process (HTTP server, sender check, DKIM signer, journal-milter, log-tailer, rate-limit recalc, notifier) share one binary and one SQLite handle on purpose — see [Panel binary](#panel-binary-cmdpanel) for why, and *Persistence* below for the @@ -1009,7 +1095,8 @@ single-connection trade-off that follows from it. |---|---| | `selfpost.db` | SQLite panel state (users, sessions, domains, apps, send log, L2 limits, log-tailer offset, inbound relay, DMARC reports) — see [Database](#database) and [schema-migrations.md](schema-migrations.md) | | `setup-token` | First-run setup token file | -| `opendkim/` | DKIM keys + tables | +| `dkim/` | DKIM keys (`keys//.private`) and the selector table (`selectors`) the signer reads; the panel's alone, `0700` / `0600` — § DKIM signing | +| `opendkim/` | The key tree OpenDKIM read until `2.8.0` (keys, `KeyTable`, `SigningTable`): copied to `dkim/` once, then kept only for a rollback to 2.7.x until the next release; the panel's alone, excluded from backups | | `sasl/sasldb2` | Application SASL credentials | | `postfix/sender_login_maps` | Login → From binding | | `postfix/relay_domains` | Inbound domains accepted on port 25 | @@ -1019,7 +1106,7 @@ single-connection trade-off that follows from it. | `postfix/dmarc_recipients` | Allow-listed DMARC aggregate report addresses | | `postfix/dmarc_transport` | Pipe transport for report ingest | | `postfix/dmarc_relay_domains` | Domains accepted for DMARC report delivery | -| `postfix/queue/` | Postfix transit mail (deferred/active); survives container recreate | +| `postfix/queue/` | Postfix transit mail (deferred/active); survives container recreate. Handed to `postfix` / `postdrop` by name at every start, before `postfix check` (`own_queue` in `build/entrypoint.sh`): the package accounts' numbers moved in `2.8.0` (postfix 101 → 100, postdrop 103 → 102) | | `log/mail.log` | Postfix delivery log + rotated copies (excluded from backups) | | `manifest.json` | Backup version stamp; removed by the first start after a restore once its Resync is done | | `contents.json` | A backup's last entry: the size and SHA-256 of each file under `data/`, compared by the first start after a restore before it opens the database (renamed `contents.checked.json` once compared, removed with the manifest) | @@ -1089,15 +1176,18 @@ the stored log lines follow; `postfix reload` in `postrotate` — see § Log tailer above). **Restore:** panel button or `selfpost-backup` CLI — self-contained archive: -`data/` (SQLite snapshot + tree minus `log/`, the setup token and any `tls/` -under `/data`), `docker-compose.yml`, `.env`, and `certs/` when present; +`data/` (SQLite snapshot + tree minus `log/`, the setup token, the old +`opendkim/` tree and any `tls/` under `/data`), `docker-compose.yml`, `.env`, +and `certs/` when present; version check on restore. Requires the project directory mounted read-only at `SELFPOST_DEPLOY_ROOT` (`/selfpost-deploy` in the default compose file). On the -first successful boot after restore, the panel runs one **Resync** — OpenDKIM's -tables, Postfix's sender map, inbound relay maps and the filter lists (when -`INBOUND_RELAY_ENABLE=true`), and DMARC maps (when `DMARC_REPORTS_ENABLE=true`) -are re-derived from SQLite and both daemons are reloaded, so drift between the -extracted archive and the database is healed before mail flows. Manual +first successful boot after restore, the panel runs one **Resync** +(`mailStack.Resync`, `cmd/panel/httpserver.go`) — the DKIM selector table +(and the keys of deleted domains removed), Postfix's sender map, inbound relay +maps and the filter lists (when `INBOUND_RELAY_ENABLE=true`), and DMARC maps +(when `DMARC_REPORTS_ENABLE=true`) are re-derived from SQLite and Postfix is +reloaded, so drift between the extracted archive and the database is healed +before mail flows; the signer reads the new table without a reload. Manual `POST /server/health/reload` (the *Reload configuration* button on Server › Health) writes the same files in the same order (`HandleReload`; before `2.5.4` it left out the filter lists and the DMARC maps), and on a failure diff --git a/docs/assumptions-2026-10-14.md b/docs/assumptions-2026-10-14.md index 22609e5..eee6b0c 100644 --- a/docs/assumptions-2026-10-14.md +++ b/docs/assumptions-2026-10-14.md @@ -46,6 +46,14 @@ the trixie sources. This register is left as it was made: its line numbers and versions are those of the bookworm sources above, which are now kept under `dev/upstream-bookworm/` (where it says `dev/upstream/`). +With panel-dkim-signing (`2.8.0`) OpenDKIM left the image: the panel signs +outgoing mail itself (`internal/milter/sign.go`), and `build/opendkim.conf`, +the syslog relay (`internal/syslogrelay`, `/dev/log`), the KeyTable and +SigningTable, the USR1 reload through supervisord and the port of +OpenDKIM's From: parser (`internal/milter/mailparse.go`) went with it. The +rows about OpenDKIM, the syslog relay and those pieces describe code that +is gone; they are left as they were, like the rest of this register. + Counts at the inventory, by the justification the tree had: | Part | Entries | CITED | MEASURED | GUESSED | WRONG? | diff --git a/docs/development.md b/docs/development.md index bc13831..b3ac8f6 100644 --- a/docs/development.md +++ b/docs/development.md @@ -155,16 +155,16 @@ commit rule but **no** version cut until the next product stage ships. | **Build** | [Makefile](../Makefile): primary targets `vet`, `test`, `build`, `e2e` (also `all`, `clean`) | | **Container** | Docker + Compose v2 on the dev host and in CI | | **Image (build stage)** | `golang:1.26-trixie` (bookworm until `2.7.0`) — [build/Dockerfile](../build/Dockerfile). The tag floats to the newest 1.26 patch release, and most standard-library security fixes (`net/http`, `html/template`, …) arrive only that way: rebuild the image after each Go patch release, no edit needed. Dependencies are raised by hand (`go get …@latest`, `go mod tidy`), at least at each audit | -| **Image (runtime)** | `debian:trixie-slim` (bookworm until `2.7.0`) + Postfix, OpenDKIM, supervisord, SASL, logrotate | +| **Image (runtime)** | `debian:trixie-slim` (bookworm until `2.7.0`) + Postfix, supervisord, SASL, logrotate (OpenDKIM until `2.8.0`; the panel signs since) | | **CI** | GitHub Actions — [.github/workflows/](../.github/workflows/) | | **Image registry** | `ghcr.io/mixeme/selfpost` | **Repository layout** (brief; process details in [architecture.md](architecture.md)): -- `cmd/panel` — HTTP panel + journal-milter + log-tailer +- `cmd/panel` — HTTP panel + milters (sender check, DKIM signer, journal) + log-tailer + notifier - `cmd/selfpost-backup` — backup CLI (`docker exec … selfpost-backup`) - `internal/` — domain logic, store, web, health -- `build/` — Dockerfile, supervisord, Postfix/OpenDKIM, entrypoint +- `build/` — Dockerfile, supervisord, Postfix, entrypoint - `deploy/` — `docker-compose.yml`, proxy examples, `.env.example` - `test/e2e/` — **separate Go module**; container integration tests @@ -183,8 +183,9 @@ dependencies must be permissive or GPL-family (see | Package | Version | Repository | License | |---|---|---|---| | `github.com/emersion/go-milter` | v0.4.1 | | BSD-2-Clause | -| `golang.org/x/crypto` | v0.54.0 | | BSD-3-Clause | -| `modernc.org/sqlite` | v1.53.0 | (mirror: ) | BSD-3-Clause | +| `github.com/emersion/go-msgauth` | v0.7.0 (`dkim`, the signer) | | MIT | +| `golang.org/x/crypto` | v0.58.0 | | BSD-3-Clause | +| `modernc.org/sqlite` | v1.60.1 | (mirror: ) | BSD-3-Clause | Transitive dependencies — `go mod graph` / `go.sum`; all indirect packages in the tree are AGPL-3.0-compatible. @@ -203,14 +204,16 @@ the tree are AGPL-3.0-compatible. | Package | Version | Repository | License | |---|---|---|---| -| `github.com/emersion/go-msgauth` | v0.6.8 | | BSD-2-Clause | +| `github.com/emersion/go-msgauth` | v0.6.8 | | MIT | The test module is not part of the main `go build` graph and is not shipped in -the image. +the image. The e2e sink image also installs dkimpy (`python3-dkim`, Debian +trixie) as the verifier that is not the signer's library +([test/e2e/sink/Dockerfile](../test/e2e/sink/Dockerfile)). ### Debian packages in the runtime image -Postfix, OpenDKIM, `supervisord`, `sasl2-bin`, `logrotate`, and others come +Postfix, `supervisord`, `sasl2-bin`, `logrotate`, and others come from Debian trixie repositories; licenses are in each package's `copyright` file on . The image also ships [LICENSE](../LICENSE), [NOTICE](../NOTICE), and the IBM @@ -222,9 +225,9 @@ the OFL text at `/static/OFL.txt`. The owner's rule (2026-10-14): what another program does is read in its documentation or its source - never guessed. SelfPost reads Postfix's log -lines and reply codes, feeds OpenDKIM's tables, parses `postqueue`, -`postconf`, `supervisorctl` and `db_dump` output, talks the milter protocol -through go-milter, relies on SQLite's locking, on logrotate's and Docker's +lines and reply codes, parses `postqueue`, `postconf`, `supervisorctl` and +`db_dump` output, talks the milter protocol through go-milter, signs with +go-msgauth over the headers Postfix hands a milter, relies on SQLite's locking, on logrotate's and Docker's behaviour. Every block of code that does so carries a comment that says what it relies on and where that is written, for the version the image runs (the Debian trixie package, not upstream's head): @@ -236,11 +239,14 @@ what it relies on and where that is written, for the version the image runs ``` The Debian sources of the image's versions are fetched with `apt-get source` -inside a `debian:trixie` container (`postfix`, `opendkim`, `cyrus-sasl2`, -`supervisor`, `logrotate`, and `sendmail` for libmilter); `dev/upstream/` -(not in git) is where they are kept for reading, and `dev/upstream-bookworm/` -holds the bookworm ones the citations named before `2.7.0`. Go modules are -read in the module cache at the version `go.mod` pins. +inside a `debian:trixie` container (`postfix`, `cyrus-sasl2`, `supervisor`, +`logrotate`); `dev/upstream/` (not in git) is where they are kept for +reading, and `dev/upstream-bookworm/` holds the bookworm ones the citations +named before `2.7.0`. `opendkim` and `sendmail` (for libmilter) are no longer +the image's since `2.8.0`; they stay in `dev/upstream/` only as what the +removed code and some historical comments cite (the signer's header list, +`internal/milter/sign.go`, names OpenDKIM's). Go modules are read in the +module cache at the version `go.mod` pins. The image takes `debian:trixie-slim` and installs without pinning (pinned Debian versions leave the mirrors), so a rebuild after a point or security @@ -480,9 +486,29 @@ L1/L2 limits, the panel down or hung mid-message (Postfix defers the mail rather than letting it through: `panel_down_defers_mail`, `panel_hung_mid_message_is_counted`), bad `SELFPOST_HOSTNAME`, session survives `docker restart`); startup checks that supervisord actually brought up -OpenDKIM, the panel, and Postfix (`checkSupervisorProcesses`), plus logrotate -config-mode and forced-rotation checks (`checkLogrotateConfigMode`, -`checkLogrotateRotation` — [test/e2e/logrotate_check.go](../test/e2e/logrotate_check.go)). +the panel, Postfix, cert-reload and logrotate (`checkSupervisorProcesses`), +that `/data/dkim` is the panel's alone, directories 0700 and files 0600 +(`checkDKIMTree`), and that the image's Debian packages are the cited ones +(`checkCitedPackageVersions`), plus logrotate config-mode and forced-rotation +checks (`checkLogrotateConfigMode`, `checkLogrotateRotation` — +[test/e2e/logrotate_check.go](../test/e2e/logrotate_check.go)). + +DKIM has an oracle that is not the signer's library: on every positive send +and on a notification, `checkPanelSignature` +([test/e2e/mail_helpers.go](../test/e2e/mail_helpers.go)) requires exactly +one signature, the panel's (`relaxed/simple`, `rsa-sha256`, `From` signed and +over-signed), and has dkimpy (`dkimverify` in the sink) verify it against the +stand's CoreDNS; go-msgauth's verifier (`verifyDKIM`) checks the published +record as a receiver would. `signature_over_awkward_headers` +([test/e2e/dkim_shapes_test.go](../test/e2e/dkim_shapes_test.go)) sends the +header shapes Postfix hands a milter in its own way - folded, tab-led, no +space after the colon, two `To:` - and an 8-bit body, and dkimpy verifies +what was delivered. `TestStartsOnA27Data` +([test/e2e/upgrade_test.go](../test/e2e/upgrade_test.go)) starts the image on +a `/data` a 2.7.x image left: the key under `/data/opendkim` is copied to +`/data/dkim` unchanged and the selector table names it, the old tree becomes +the panel's alone, and a queue owned by other uids is handed to Postfix's +users by name. Waits poll with a timeout (`waitFor`); a fixed sleep is the exception and says why where it stands (`sender_check_test.go`: a downtime worth a second or two, and 35 s to show a row is not written again). @@ -528,7 +554,7 @@ prepare (version from release tag or workflow_dispatch input; checkout vX.Y.Z) → GitHub Packages API → drop untagged versions no tagged manifest references ``` -Native per-arch matrix (no QEMU): running the full Postfix/OpenDKIM stack under +Native per-arch matrix (no QEMU): running the full Postfix stack under emulation for e2e is impractical. E2e first, then push — the registry receives the bytes that passed the gate. Each release leaves three tags in GHCR: `X.Y.Z` (the multi-arch manifest operators pin) and `X.Y.Z-amd64` / diff --git a/docs/guide.md b/docs/guide.md index db67dd1..edba29a 100644 --- a/docs/guide.md +++ b/docs/guide.md @@ -451,9 +451,9 @@ assumptions stay here in `docs/guide.md`. **Overview** (`/overview`, global role) shows the stamp — the worst of its checks — and one card per check: machine (processor and memory), supervised -processes (Postfix, OpenDKIM, panel), TLS certificate, queue, milter sockets -and reverse DNS. A check the panel could not run shows as a warning, never as -fine. Below the cards are the outbound domain list and, with the inbound relay +processes (Postfix, the panel - whose roles include the DKIM signer - and +their helpers), TLS certificate, queue, milter sockets and reverse DNS. A +check the panel could not run shows as a warning, never as fine. Below the cards are the outbound domain list and, with the inbound relay on, the inbound one. Each card leads to its table on **Server › Health** (`/server/health`) — the queue card to the Outbound queue — and Health has the readings, in the order of what stops mail first: @@ -466,7 +466,12 @@ readings, in the order of what stops mail first: memory is a warning, because both delay or kill the mail path; throughput is only reported. - **Processes**, **TLS certificate** (validity and expiry) and **Milter - sockets** (OpenDKIM, the sender check and the journal-milter). + sockets**: the DKIM signer's two - `dkim-signer · applications` for what + applications send on 465/587, `dkim-signer · notifications` for the + panel's own mail - the sender check, the send log (the journal-milter) + and, with one configured, the spam filter. Postfix attaches the signer + strictly, so either of its rows down means the mail that goes through it + waits. - **Sender check** — what the strict check of the `From:` line has cost today, in the last 30 days and in the last 12 months: for how long the panel did not answer while Postfix was running, how many connections @@ -487,10 +492,12 @@ readings, in the order of what stops mail first: reverse DNS), see [Server-level DNS](#server-level-dns-ptrrdns); use *Re-check DNS* after changing DNS. - **Configuration** — the **Reload configuration** button writes again, from - the database, every file the panel keeps for the mail path - OpenDKIM's - tables, the Postfix sender map, the inbound relay maps and the filter lists - (with `INBOUND_RELAY_ENABLE=true`), the DMARC report maps (with - `DMARC_REPORTS_ENABLE=true`) - and has OpenDKIM and Postfix re-read them: + the database, every file the panel keeps for the mail path - the DKIM + selector table (`/data/dkim/selectors`), the Postfix sender map, the + inbound relay maps and the filter lists (with `INBOUND_RELAY_ENABLE=true`), + the DMARC report maps (with `DMARC_REPORTS_ENABLE=true`) - removes the keys + of deleted domains, and has Postfix re-read them; the signer needs no + reload, it reads the table for every message. This is what the first start after a restore does (see [Restore](#restore)). Use it if the daemons drifted from what the panel shows after manual edits under `/data`. A step that fails is named on the page, and the panel's log says @@ -513,6 +520,16 @@ readings, in the order of what stops mail first: six hours. It lives in the data volume, so it survives a container recreate along with the rest of the state — `./data/log/` on the host — but it is *not* included in backups: it is diagnostics, not state. +- **The container log** (`docker compose logs selfpost`) — what the panel + itself says, its milters among it; none of it is in `mail.log`. A message + the DKIM signer cannot sign is not sent unsigned, it waits: Postfix + answers an application with a temporary error and the application sends + it again later. Why is a line of the container log that begins `dkim-signer:` and + ends `- deferring`, e.g. `dkim-signer: no signing key for example.com - + deferring` (the selector table has no line for the domain: *Reload + configuration* on Server › Health writes it again from the database) or + `dkim-signer: no sending domain for login "shop" - deferring`. The sender + check's lines begin `sender-check:`. ### Account @@ -650,11 +667,52 @@ published, then `docker compose up -d`. The backup version check requires the ru to match the version that created a full backup — see [Fixed image tag](#fixed-image-tag). +**To 2.8.0** there is nothing to do. From 2.8.0 the panel signs outgoing +mail itself, in a milter of its own, and OpenDKIM is no longer in the image; +the keys, the selectors and the DNS records stay as they are. The keys move +from `data/opendkim/keys` to `data/dkim/keys`: on the first start the +entrypoint copies them, as the panel's user, into `data/dkim.new`, renamed +into place when the copy is whole, and says so in the container log +(`entrypoint: the DKIM keys were copied from /data/opendkim to /data/dkim; +/data/opendkim is left for a rollback`). A copy that fails stops the start - +the panel would otherwise give the server's own name a new key, and its +published record would no longer match. `data/dkim` is the panel's alone +(directories 0700, files 0600); so is `data/opendkim`, which is left as it +was, for a rollback, until the next release, and which a full backup leaves +out. The selector table, `data/dkim/selectors`, is written from the database +at every start, before the signer serves. + +**Rolling back to 2.7.x** after that: the 2.7.x image signs with OpenDKIM +from `data/opendkim`, which holds the keys as they were at the upgrade. A +domain added under 2.8.x has no key there, and 2.7.x sends its mail +unsigned; one deleted and added again under 2.8.x has its old key there, +which no longer matches its published record. Postfix's queue under +`data/postfix/queue` belongs to Postfix's users, which have other numbers +in the two images (postfix 100 in 2.8.0, 101 in 2.7.x); a 2.7.x start +reads the queue before it hands it over and stops at `postfix check` +("scan_dir_push: open directory defer: Permission denied"). With the +instance stopped, hand the queue over in the 2.7.x image itself, by name, +before its first start (the image is the one you roll back to): + +```bash +docker run --rm -v ./data:/data --entrypoint sh ghcr.io/mixeme/selfpost:2.7.0 -c 'chown -R postfix:postfix /data/postfix/queue && chgrp postdrop /data/postfix/queue/public /data/postfix/queue/maildrop && chown root /data/postfix/queue /data/postfix/queue/pid' +``` + +(Measured on the test server, 2026-10-16, on a copy of its queue: 2.7.x's +`postsuper` failed before and passed after.) Going forward again needs +nothing for the queue - 2.8.0 hands it over itself. A domain added under +2.7.x has its key only in `data/opendkim/keys//`, and the +entrypoint copies only while `data/dkim` holds no key at all: with the +instance stopped, copy that directory into `data/dkim/keys/` (the start +makes it the panel's). Removing `data/dkim` instead has the old tree copied +again whole and loses the keys of domains added under 2.8.x. + ### Container health The image declares a Docker `HEALTHCHECK` that probes `GET /healthz` on port -8080 (unauthenticated). It returns `200 ok` when OpenDKIM, the panel, and -Postfix are all `RUNNING` under supervisord; otherwise `503 unhealthy`. This +8080 (unauthenticated). It returns `200 ok` when the panel - whose roles +include the DKIM signer - and Postfix are both `RUNNING` under supervisord; +otherwise `503 unhealthy`. This catches a dead mail path that would still leave the HTTP server up, but it does **not** verify TLS certificates, DNS records, or end-to-end delivery — use the authenticated [Overview and Health](#overview-and-health) pages for @@ -708,10 +766,12 @@ as themselves, the host's IPv6 address and `127.0.0.1` as `172.18.0.1`. ### Full backup and restore **Full backup** is a self-contained project archive: `data/` (SQLite, all -domains' DKIM keys, all applications' SASL credentials, `manifest.json` with -the version that created it), plus `docker-compose.yml`, `.env`, and `certs/` -from the operator directory next to `./data`. Delivery logs under `data/log/` -are excluded, and so is the Postfix queue: the archive is taken while the +domains' DKIM keys under `data/dkim/`, all applications' SASL credentials, +`manifest.json` with the version that created it), plus `docker-compose.yml`, +`.env`, and `certs/` from the operator directory next to `./data`. Delivery +logs under `data/log/` are excluded, so is `data/opendkim/` (the key tree an +installation upgraded from 2.7.x keeps for a rollback, see +[Upgrading](#upgrading)), and so is the Postfix queue: the archive is taken while the instance runs, and a copy of a queue Postfix is working on can hold a message twice or not at all. Mail still waiting for delivery is therefore not carried over, and a restored instance starts with an empty queue; to keep the queue, @@ -828,10 +888,10 @@ container of the **exact same image version** that created the backup — SelfPo refuses to start otherwise and tells you which tag to use. On the first successful start after restore, `data/manifest.json` from the archive is **deleted** — it guards only that one boot, so a later in-place upgrade is not -blocked. On that same first boot the panel also runs one **Resync** — OpenDKIM's -tables and Postfix's sender map are re-derived from SQLite (inbound relay maps +blocked. On that same first boot the panel also runs one **Resync** — the DKIM +selector table and Postfix's sender map are re-derived from SQLite (inbound relay maps when `INBOUND_RELAY_ENABLE=true`; DMARC maps when `DMARC_REPORTS_ENABLE=true`) -and both daemons are reloaded, healing any drift between the extracted files and +and Postfix is reloaded, healing any drift between the extracted files and the database. The manifest is deleted only after that Resync is done: a first start that fails on the way leaves it, and the next start is a restore again and runs the Resync. The *Reload configuration* button on Server › Health runs the same Resync. @@ -1142,7 +1202,7 @@ inbound mail is being refused with a temporary error (`tempfail`) or going throu (`accept`). **Inbound mail is not in the Outbound log.** Port 25 runs its own milter chain: -the optional anti-spam milter, and nothing else. OpenDKIM does not sign relayed +the optional anti-spam milter, and nothing else. The DKIM signer does not sign relayed mail (and a signing outage therefore cannot hold it up), and the journal-milter does not file it — the [Outbound log](#outbound-log) stays a record of what *you* sent. @@ -1259,7 +1319,7 @@ listed for it. Since `2.4.0` that rule covers both senders a message has: - the **envelope sender** (`MAIL FROM`), which Postfix checks; - the **`From:` line** of the message - the one a mail program shows and the - one the DKIM signature is chosen by - which the panel checks. + one DMARC holds the DKIM signature's domain to - which the panel checks. An address with an extension counts as the address: an application listed as `invoices@…` may also send as `invoices+march@…`, in the envelope and in @@ -1272,13 +1332,15 @@ Postfix asks who owns the address before it asks who owns the domain. A message whose `From:` is anything else is refused with `550 5.7.1` and a text naming the line: another of your domains, the server's own name, an address at a domain that is not on this server, a line with two addresses, -or no `From:` at all. The line is read the way the signer, OpenDKIM, reads -it - the panel runs OpenDKIM's own parser - so only its address counts: what a -display name or a comment around it says is not judged, and a line with more -than one address is refused. A display name with a comma or brackets has to -be quoted (`"Shop, Inc." `), as the mail standard asks and mail -libraries do; unquoted, OpenDKIM may take a word of the name for the -address. The refusal +or no `From:` at all. The line must be one address as the mail standard +(RFC 5322) writes it - the address a mail program shows - and only that +address counts: what a display name or a comment around it says is not +judged, and a line with more than one address is refused. A display name +with a comma or brackets has to be quoted (`"Shop, Inc." `), as +the standard asks and mail libraries do; unquoted, the line does not read +as one address and is refused. The address is held to the application's +list the way Postfix holds the envelope sender, a quoted local part as the +same address unquoted. The refusal is permanent, so the application does not send the message again; it is a row of the [Outbound log](#outbound-log) tagged `wrong From`, and the row's page shows what the line said. @@ -1299,6 +1361,23 @@ its mail sends it again later, one that sends directly gets an error to retry; nothing is accepted unchecked. **Server › Health** counts how often that happens. +Since `2.8.0` the check also guards the signature: the panel signs an +application's message with the key of the domain its login belongs to, and +DMARC counts that signature only when its domain aligns with the domain of +`From:`. Without the check an application could send under this server's +signature as another of its domains, or as the server itself. + +**Lines of more than 998 bytes.** The signer signs the message as the +application handed it over. Postfix's SMTP client breaks a body line longer +than 998 bytes when it delivers the message (`smtp_line_length_limit`), +after the signature was made, and the receiver's check of the signature +then fails. Such a message is still sent; the container log names its +domain and login (`dkim-signer: example.com (login "shop"): a body line +longer than 998 bytes - ...`). Mail libraries keep lines short with +quoted-printable or base64; an application that writes its own body has to +as well. A message whose header section is over 1 MiB is refused, +`552 5.3.4`. + *Export domain* writes a single-domain file; *Import a domain* on *Server › Backup* reads one back in (**global role only**) — see [Exporting and importing a single domain](#exporting-and-importing-a-single-domain). diff --git a/docs/plans/panel-dkim-signing.md b/docs/plans/panel-dkim-signing.md index 1d4c882..bbf3000 100644 --- a/docs/plans/panel-dkim-signing.md +++ b/docs/plans/panel-dkim-signing.md @@ -199,12 +199,21 @@ libmilter1.0.1. ### Stage 3 - documents, the canary, the cut -- [ ] architecture § OpenDKIM -> § DKIM signing; security.md (the +- [x] architecture § OpenDKIM -> § DKIM signing; security.md (the mail-path bullets, the accepted risks that were OpenDKIM's); guide; development's cited-package table (opendkim and libopendkim11 leave); the assumptions register's header notes what left; help (design commit first) and Health's label (Q3, mockup first). -- [ ] Reviews: technical and security. +- [x] Reviews: technical and security. + +*Done (2026-10-16):* help and Health went with stage 2 (design `005922b`); +the reviews ran on stages 1-2 and their findings are fixed in `c2f63d2`, +with the first rollout's incident (the queue's owner numbers); the +documents in the commit after it. Open: holding `Sender:` to the login +(security review F1, the owner's decision), and go-milter v0.4.1 keeping +every header in a map of its own per session (security.md, accepted risks) +- a header section the signer refuses past 1 MiB is still held by the +library, on 465/587 and on port 25 alike, as before 2.8.0. - [ ] On the test server: a day of the owner's mail; one message from an application through 465 and one notification to a real mailbox, the headers read (dkim=pass with the panel's `d=`). @@ -222,7 +231,11 @@ libmilter1.0.1. from the old path, which the move leaves as a copy until the next release - the keys as they were at the upgrade: a domain added under 2.8.x is not in that tree, and 2.7.x sends its mail unsigned; one re-added under 2.8.x - has another key there. Going forward again after a rollback, a domain - added under 2.7.x has its key only in the old tree: remove `/data/dkim` - before the start, and the entrypoint copies again (it copies while - `/data/dkim` holds no key). + has another key there. 2.7.x reads the queue as its own postfix (uid + 101) before it hands it over, and 2.8.0 left it to uid 100: the guide + (§ Upgrading) gives the hand-over by name to run in the 2.7.x image first, + measured on a copy of the test server's queue. Going forward again after + a rollback, a domain added under 2.7.x has its key only in the old tree: + copy its directory into `/data/dkim/keys` (removing `/data/dkim` instead + copies the old tree again and loses the keys of domains added under + 2.8.x). diff --git a/docs/product.md b/docs/product.md index 3f96dfa..831a3d9 100644 --- a/docs/product.md +++ b/docs/product.md @@ -41,8 +41,8 @@ decision: are not a current target but not ruled out forever. 2. **Send from your own IP (DIY)** — no intermediate relay provider. Requires outbound port 25, static IP, and configurable PTR/rDNS from the operator. -3. **Single container** — Postfix, OpenDKIM, and the panel run under one - `supervisord` inside one image. +3. **Single container** — Postfix and the panel, which signs outgoing mail + with DKIM itself, run under one `supervisord` inside one image. 4. **Panel is internet-facing** — mandatory security requirements in [security.md](security.md). diff --git a/docs/roadmap.md b/docs/roadmap.md index 38c7c42..d3d961c 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -44,7 +44,7 @@ in `git log` and [CHANGELOG.md](../CHANGELOG.md). | own-mta-builds | Analysis: build Postfix and OpenDKIM ourselves, patched, instead of Debian's packages | analysed 2026-10-15: no own builds; three rows proposed instead | — | [own-mta-builds-2026-10-15.md](own-mta-builds-2026-10-15.md) | | port25-bare-newline | The inbound relay's port 25 refuses bare newlines in DATA (`smtpd_forbid_bare_newline=normalize`) | shipped `2.6.1` | — | — | | base-trixie | The image's base from Debian bookworm (LTS since 2026-07-12) to trixie: Postfix 3.10 with the 2026 security batches | shipped `2.7.0` | — | — | -| panel-dkim-signing | DKIM signing in the panel's own milter; OpenDKIM, the syslog relay and the parser port removed | agreed (2026-10-15; stages 1-2 done) | 10/14 | [plans/panel-dkim-signing.md](plans/panel-dkim-signing.md) | +| panel-dkim-signing | DKIM signing in the panel's own milter; OpenDKIM, the syslog relay and the parser port removed | agreed (2026-10-15; stages 1-2 done, documents and reviews) | 12/14 | [plans/panel-dkim-signing.md](plans/panel-dkim-signing.md) | | phone-desktop-mode | A phone browser in "desktop site" mode gets the phone navigation | agreed (2026-10-13: after stages 1 and 2 of audit-2026-10-12) | — | — | | from-header-binding | Hold a message's `From:` header against the application that sends it | shipped `2.4.0` | 9/9 | [plans/from-header-binding.md](plans/from-header-binding.md) | | panel-redesign | Panel redesign (accepted mockups, design contract) | shipped `2.0.0` | 27/27 | [plans/panel-redesign.md](plans/panel-redesign.md) | diff --git a/docs/security.md b/docs/security.md index b115a47..1f472ca 100644 --- a/docs/security.md +++ b/docs/security.md @@ -52,7 +52,18 @@ under Accepted risks (the allow list keyed on an address anyone can claim, the list files readable by anyone). A third it named - with the filter down, a sender's own verdict headers were recorded as the filter's - was closed the same day at the owner's suggestion: the filter signs each message and a -verdict is read only next to its signature. +verdict is read only next to its signature. The 2026-10-16 security review of +panel-dkim-signing (`2.8.0`: the panel's own DKIM signer, OpenDKIM removed) +found no path by which an application gets another tenant's key or the +server's, or a valid signature over a `From:` a mail program reads as another +domain; no Critical or High. The signer held a message's whole header section, +bounded only by `message_size_limit` (now 1 MiB, refused above it); the old +key tree stayed readable by group `selfpost` and went into every backup (now +the panel's alone and left out); a hostname with capitals deferred every +notification (the server's name is kept in lower case for DKIM); only `From` +was over-signed (now the shown headers too) - all fixed. `Sender:` is now +signed but not held to the login, which waits for the owner; that and what the +header cap leaves are under Accepted risks. (2) **Accepted risks** — deliberate departures beyond the mandatory, recorded so the decision is not lost. @@ -211,8 +222,24 @@ operator can act on it. - The panel process is **not root** (`user=panel` in supervisord); path access is granted through the `selfpost` group with minimal permissions. The supervisord control socket, which can stop or restart any program, is the - `supervisor` group's, whose one member is the panel; OpenDKIM and Postfix - share `selfpost` for files and do not reach it (`2.6.0`). + `supervisor` group's, whose one member is the panel; Postfix shares + `selfpost` for files and does not reach it (`2.6.0`; OpenDKIM shared + `selfpost` too until `2.8.0`). +- **Group `selfpost` (gid 10998) has two members, `panel` and `postfix`.** + Through it Postfix reads what the panel writes - `sasldb2` and the maps + under `/data/postfix` (2750, files 0640) - and connects to the panel's + milter sockets in `/run/selfpost` (2750, sockets 0660); the panel reads + `mail.log`, which `postfix` writes, through it the other way + ([build/Dockerfile](../build/Dockerfile), `build/entrypoint.sh`). The DKIM + keys are not in it (§ Mail path). +- **The container keeps `KILL`** (`deploy/docker-compose.yml`, `cap_add`; + every other capability not listed there is dropped). Without it root + cannot signal a process of another uid (kill(2), capabilities(7); measured + on the test server, 2026-10-16), and two root processes must: supervisord + stops and restarts the panel, and Postfix's master ends its children, + which run as `postfix`. Until `2.8.0` it also carried OpenDKIM's `USR1` + reload through supervisord, which went with OpenDKIM; the plan had taken + that for its only use, wrongly. ### Mail path (security-relevant) @@ -226,50 +253,93 @@ operator can act on it. application sent from. What a complaint needs stays with the operator - the queue id in the header, the panel's log and smtpd's `client=` line in mail.log. Port 25 keeps the full header, as RFC 5321 requires of an MTA. -- **An application signs only as itself (`2.4.0`).** OpenDKIM chooses the - key by the `From:` header and Postfix holds only the envelope sender - against the login, so until `2.4.0` an application of one sending domain - could write another registered domain - or, since `2.3.0`, the server's - own name - into `From:` and have the message signed with that name's key - (found by the security review of panel-notifications, 2026-10-12). The - panel's sender check now holds the header to the table the envelope is - held to and refuses anything else with `550 5.7.1`, a domain the server - does not sign for included (the owner's rule: the header may say what the - envelope may, nothing more). It is the first milter of the chain and - strict: a message it could not judge is deferred, never passed on to the - signer. Since `2.5.2` it reads the header with OpenDKIM's own parser, - `dkim_mail_parse()` ported line for line from the opendkim the image runs - (`internal/milter/mailparse.go`), and judges the address that parser - returns - the address OpenDKIM signs for - whatever a display name, a - comment or a quoted local part holds; one `From:` field and one address in - it stay required, and an address longer than the 256 bytes OpenDKIM - matches its `SigningTable` against is refused, since cut there it could end - in another domain's name (opendkim.c, `dkimf_apply_signtable()`, - `MAXADDRESS`). That the port is OpenDKIM's parser is tested against the C - itself, two hundred thousand lines and every result alike; that the signed - domain follows, by the e2e: it sends every `From:` line the check passes (a - list shared with the unit test, - `internal/milter/testdata/from-lines-passed.txt`) and wants it signed for - the application's domain and no other, and every - line of a second list naming another sender of the server in a quoted - string, a comment, a quoted pair, capitals or an encoded word - (`from-lines-other.txt`) must be refused or arrive signed for the - application's domain, never for that sender. What this does not promise is - what a display name looks like to a person: a name of another sender in - it passes, signed with the application's own key - a forged display name is - as old as mail, and no key of another's is involved. What the client wrote - reaches the reply, the log and the send log only as printable ASCII, cut, - and a refused message is one row of the log whatever its recipient count. - Mail the panel submits itself does not pass the check: only root, - `postfix` and the panel can submit locally. Architecture § Sender check. +- **The panel signs, with the key of the login's domain (`2.8.0`).** The + DKIM signer is a role of the panel process (`internal/milter/sign.go`, + `cmd/panel/signer.go`) on two sockets of its own, both attached strictly + (`default_action=tempfail`, `build/postfix-config.sh`). `sign.sock`, + second in `smtpd_milters` after the sender check, signs what applications + send on 465/587 with the key of the domain the SASL login belongs to: + `{auth_authen}`, looked up in the `sender_login_maps` file Postfix itself + reads (`Bindings.DomainOf`). `sign-local.sock`, the whole of + `non_smtpd_milters`, signs what is submitted with sendmail(1) with the key + of the server's own name. Nothing in a message chooses the key, and the + socket a message comes through - not a missing login - decides that the + server's key may sign it. What cannot be signed (a login with no domain, a + name with no key, a read error) is deferred, never sent unsigned. Port 25 + has no signer. `relaxed/simple`, rsa-sha256; `h=` names every header of + RFC 6376 § 5.4.1's list the message has, and `Sender` and + `List-Unsubscribe-Post`, then once more each of the shown ones present + (Subject, Date, To, Cc, Reply-To, Sender) and `From` last, so one of those + added after signing breaks the signature. Until `2.8.0` OpenDKIM signed - a + 2018 beta with three 2026 CVEs open and nobody upstream to fix them + ([own-mta-builds](own-mta-builds-2026-10-15.md) § 1) - and chose the key + by the `From:` header; it left the image, and with it the syslog relay and + `/dev/log`, the port of its `From:` parser, its tables and their reload by + a signal across uids. Architecture § DKIM signing. +- **The DKIM keys are the panel's alone (`2.8.0`).** The key tree and the + selector table the signer reads, `/data/dkim/keys//.private` + and `/data/dkim/selectors` (`internal/domain`, `Keys`), are owned by + `panel`, directories 0700 and files 0600 - no group, no other user; the + signer reads them and never the database. The entrypoint sets these modes + at every start (e2e `checkDKIMTree` holds the tree to them). Until `2.8.0` + the keys were under `/data/opendkim`, 2750/0640 in group `selfpost` so + that OpenDKIM could read them - and so could every `postfix` process. That + tree is kept for a rollback to 2.7.x until the next release; since `2.8.0` + the entrypoint makes it the panel's alone as well (0700/0600, panel:panel) + at every start, copies its keys as the panel user (`setpriv`) while + `/data/dkim` holds none, and a backup leaves it out (§ Backup and domain + export). A root step of the entrypoint follows no symbolic link the panel + could plant in either tree (`drop_links`; `cp -RP`; `chown -R` and `find` + do not follow one). +- **An application signs only as itself (`2.4.0`).** The signature's `d=` is + the login's domain, and a receiver holds it to the domain of the `From:` + line (DMARC, RFC 7489 § 3.1.1): without a check of that line an + application could still send under the server's signature as another + domain of the server, or as the server itself. The panel's sender check + holds the header to the table the envelope is held to and refuses anything + else with `550 5.7.1`, a domain the server does not sign for included (the + owner's rule: the header may say what the envelope may, nothing more). It + is the first milter of the chain and strict: a message it could not judge + is deferred, and one it refuses never reaches the signer. The `From:` + line must be one field holding one address in RFC 5322's grammar as Go's + `net/mail` reads it - a mail program's reading; a line a lenient program + could show as another address (`ceo@bank.com `, + ``) is refused - and that address, its local + part unquoted as Postfix unquotes the envelope's, is held against the + application's bindings; its domain is compared in ASCII lower case only, + so a look-alike such as the Kelvin sign for `k` is no letter of a + registered domain (`internal/milter/sender.go`, `judge`, `asciiLower`). + The unit test reads a list of lines naming another sender of the server in a quoted string, a + comment, a quoted pair, capitals or an encoded word + (`internal/milter/testdata/from-lines-other.txt`) and wants each refused + or read as the application's own address; the e2e sends every line of a + second list the check passes (`from-lines-passed.txt`, shared with the + unit test) and wants it signed for the application's domain and no other. + What this does not promise is what a display name looks like to a person: + a name of another sender in it passes, signed with the application's own + key - a forged display name is as old as mail, and no key of another's is + involved. Nor does it look at `Sender:` (Accepted risks). What the client + wrote reaches the reply, the log and the send log only as printable ASCII, + cut, and a refused message is one row of the log whatever its recipient + count. Until `2.4.0` nothing held the header: OpenDKIM chose the key by it + and Postfix holds only the envelope sender against the login, so an + application of one sending domain could write another registered domain - + or, since `2.3.0`, the server's own name - into `From:` and have the + message signed with that name's key (security review of + panel-notifications, 2026-10-12). From `2.5.2` to `2.8.0` the check read + the line with OpenDKIM's own parser, ported line for line, and refused an + address over the 256 bytes OpenDKIM matched its `SigningTable` against; + with the key chosen by the login both went. Mail the panel submits itself + does not pass the check: only root, `postfix` and the panel can submit + locally (Accepted risks). Architecture § Sender check. - **Inbound relay (optional, `INBOUND_RELAY_ENABLE`)** — port 25 is not an open relay either: SASL is off; `smtpd_relay_restrictions` / `smtpd_recipient_restrictions` are `reject_unauth_destination` and `reject_unlisted_recipient`; maps list only configured domains and recipients. Domains with no upstream host are omitted from the maps so mail is never accepted with nowhere to send it. Prefer recipient mode `list` to - refuse unknown addresses at RCPT (no backscatter). OpenDKIM is not attached - on inbound. An optional antispam milter is inbound-only; default action is + refuse unknown addresses at RCPT (no backscatter). Nothing signs on port + 25. An optional antispam milter is inbound-only; default action is fail-open (`accept`). Since `2.6.1` the listener ends DATA only at the standard `.` (`smtpd_forbid_bare_newline=normalize`, the 2023 SMTP-smuggling defence, which Postfix 3.7 left off by default): a @@ -335,7 +405,12 @@ operator can act on it. panel users' password hashes, `docker-compose.yml`, `.env`, and the TLS private key from `certs/` when present; a domain export carries the DKIM key and **working** application passwords in the clear (otherwise a transfer - without recreating credentials would be impossible). + without recreating credentials would be impossible). The keys a backup + carries are those of `/data/dkim`; `/data/opendkim`, the tree 2.7.x read + and kept for a rollback, is left out (`internal/backup`, + `excludedFromArchive`, `2.8.0`): an archive restores into its own version + alone, and a second copy of every key - one that outlives a deleted + domain's - has no place in it. - Links under `certs/` (and `certs` itself as a link) are followed, in the container's view: the archive carries whatever regular file they lead to that the panel's user may read. `certs/` is the operator's, mounted @@ -526,11 +601,12 @@ deferred item from the roadmap. instead be given the files through a group it shares with the panel. - ~~**An application can learn which domains are senders of this server.**~~ Struck in `2.5.2` (audit 2026-10-12, stage 2c; L8 closed): the sender - check reads a `From:` line with OpenDKIM's own parser, ported - (`internal/milter/mailparse.go`), and decides by the address OpenDKIM signs - for; what a display name, a comment or a quoted local part names no longer - matters, so refused or passed says nothing about other domains of the - server. + check read a `From:` line with OpenDKIM's own parser, ported, and decided + by the address OpenDKIM signed for; what a display name, a comment or a + quoted local part names no longer mattered, so refused or passed said + nothing about other domains of the server. Since `2.8.0` the check judges + the one address `net/mail` reads and the signer takes the login's key, so + the same holds without the port. - ~~**A row of the inbound log belongs to a domain by name.**~~ Struck in `2.5.0` (audit 2026-10-12, L12): both logs now hold a domain by id, see § Roles. @@ -581,12 +657,32 @@ deferred item from the roadmap. a dozen times a day at most by someone who flaps the zone; a domain's limits once a day; and no more than 60 messages leave in any hour whatever the rules arrive at. -- **The sender check stops mail while the panel does not answer.** It is a - role of the panel process, attached with `default_action=tempfail`, so a - panel that is down or slower than the milter timeouts (15 s to connect - and for a command, 30 s for the verdict at the end of the headers) defers - every submission: availability of sending now depends on the panel, - which it did not before `2.4.0`. Chosen by the owner over the alternative, +- **Whatever is submitted with sendmail(1) is signed as the server.** The + signer's local socket, the whole of `non_smtpd_milters`, signs every + message submitted locally with the key of the server's own name, whatever + its `From:` says, and no sender check stands on that path (`2.8.0`; under + OpenDKIM a local message was signed for its `From:` domain). Who can submit + is what bounds it: root, `postfix`, and the panel through `postdrop` (the + item above) - so any code running as `panel`, the panel process and + `dmarc-ingest`, which pipe(8) runs as `panel`, can have a message signed as + the server; the same code can read every key anyway, as it could in 2.7. + Mail from the Internet never reaches the socket: smtpd runs its own milter + list, and one whose list is empty (port 25 with DMARC reports on and the + inbound relay off, `-o smtpd_milters=`) turns `non_smtpd_milters` off for + what it receives + (Postfix 3.10.13, src/smtpd/smtpd.c, `setup_milters()`, "Safety: disable + non_smtpd_milters when not sending our own mail filter list"), and the + applications on 465/587 have no code in the container. Trigger to revisit: + anything else able to run as `panel` or `postfix` in the container, or a + local submitter that is not the panel. +- **The sender check and the signer stop mail while the panel does not + answer.** Both are roles of the panel process, attached with + `default_action=tempfail`, so a panel that is down or slower than the + milter timeouts (15 s to connect and for a command, 30 s for the verdict + at the end of the headers) defers every submission and its own + notifications: availability of sending now depends on the panel, which it + did not before `2.4.0` (until `2.8.0` the signer was OpenDKIM, attached as + strictly, a process of its own). Chosen by the owner over the alternative, a check that is absent whenever the panel is - that one could be switched off by whoever can make the panel slow. Supervisord starts a panel that died again at once; one that dies within a second of starting is tried @@ -607,19 +703,65 @@ deferred item from the roadmap. of the restart defer mail instead of passing it unjournalled (technical review, 2026-10-12). Trigger to revisit: anything untrusted able to open a Unix socket in the container. -- **The container's `/dev/log` takes datagrams from any user in it** (since - `2.6.0`, for OpenDKIM's log): what is written there reaches the container - log, each line marked `syslog: `, control characters written out, at most - 200 lines a second (the rest counted and dropped). A user who can write it - can add lines to `docker logs` under that mark and no other; the panel - reads only `mail.log`. logrotate's state file is under `/data/log`, the - `postfix` user's directory: that user could stop rotation through it - - as through the log itself, which it owns. Trigger to revisit: anything - untrusted able to run in the container. +- ~~**The container's `/dev/log` takes datagrams from any user in it.**~~ + Struck in `2.8.0`: the syslog relay that read it from `2.6.0` existed for + OpenDKIM's log, and left the image with OpenDKIM, as did the `/dev/log` + link; the signer logs to the panel's own output (the container log). +- **logrotate's state file is under `/data/log`,** the `postfix` user's + directory: that user could stop rotation through it - as through the log + itself, which it owns. Trigger to revisit: anything untrusted able to run + in the container. - **When the hostname is itself a sending domain,** the server's mail is signed with that domain's key. Whoever may export the domain has the key, and its applications may send as `postmaster@` by right. The guide says so; a hostname that is nobody's sending domain avoids it. +- **`Sender:` is signed but not held to the login** (open: the owner's + decision). Since `2.8.0` the signer signs `Sender:` and `Resent-From:` + and over-signs `Sender:`, so neither can be added or changed after the + message leaves without breaking the signature; the sender check judges + `From:` alone (`senderSession.Header` keeps `From` and `Subject`). An + application of `mine.com` can write `From: billing@mine.com` and + `Sender: ceo@bank.com`: the check passes it, the signature is + `d=mine.com`, DMARC aligns on `From:` and passes, and a mail program that + shows `Sender:` beside `From:` - Outlook as "on behalf of", by the + review's account, not tried on this build - shows the other address. No + key of another's is involved, as with a display name; until `2.8.0` the + header was neither checked nor signed. The fix the security review of + 2026-10-16 (F1) gives, and recommends: the check also reads `Sender:` and + `Resent-From:` and refuses one whose single address the application may + not send as, under the rule `From:` is held to - a few lines in + `internal/milter/sender.go` and their tests. It changes what is refused + (an application whose mail program writes a `Sender:` of another domain + today would be refused), hence the owner's decision; this item goes when + it is taken. +- **A header section is bounded at 1 MiB by the signer, not by the milter + library.** The signer counts a message's headers as Postfix hands them + over and refuses a section of more than 1 MiB with `552 5.3.4` + (`internal/milter/sign.go`, `headerLimit`; `2.8.0`): it keeps no more + than that and drops its list once the block it signs is built, so its own + memory per message is a few mebibytes at most. No mail program writes such + a section, and the refusal is permanent, so a sender is told rather than + retried. What the cap does not bound: go-milter v0.4.1 adds every header + of a message to a map of its own in each milter it is sent to (session.go, + `CodeHeader`), which none of SelfPost's milters reads - the sender check, + the signer and the journal milter on 465/587, the inbound journal milter + on port 25 - and that copy is bounded only by the port's message size + limit (Postfix's default 10 240 000 bytes on 465/587; + `INBOUND_MESSAGE_SIZE_LIMIT` or `DMARC_MESSAGE_SIZE_LIMIT` on port 25, + from strangers), since a refusal at the end of the headers comes after + they are all in. A message of nothing but headers still costs the panel + about its size once per milter while it is in the milters, times the + messages Postfix runs at once (as many as its cleanup processes, + `default_process_limit`, 100), and `deploy/docker-compose.yml` sets no + memory limit, so the host's out-of-memory killer chooses what goes; a + panel killed is started again, and submissions wait meanwhile. The + library's copy is older than `2.8.0`: every milter of the panel has had + it. The fix, priced: a go-milter patched not to keep the map (vendored; + SelfPost reads headers only through `Header`), or a memory limit in the + compose file, which makes an exhausted panel a restart of the panel alone + rather than a victim the host picks. Trigger to revisit: the inbound relay + on an address strangers send to in volume, or a panel seen near the + host's memory. - **A message's page can show another message's log lines after a queue-id collision.** Postfix's short queue ids repeat: the id is the microsecond of arrival and the queue file's inode, and inodes come round within seconds.