diff --git a/CHANGELOG.md b/CHANGELOG.md index f4b5d7d..381a6b5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -377,7 +377,7 @@ Nothing for an operator to do. ### Added - **Inbound › Log shows what the upstream answered** (inbound-upstream-result, - [the plan](docs/plans/inbound-upstream-result.md)): an *Upstream* column + the plan, since removed): an *Upstream* column beside the decision - *queued*, *held*, *sent*, *deferred* (a greylisting, an upstream that cannot be reached), *bounced* (refused, or expired in the queue) - opening the last reply, the host that gave it and when. Read from @@ -456,7 +456,7 @@ schema 11. ### Added - **The panel signs outgoing mail with DKIM itself** (panel-dkim-signing, - [the plan](docs/plans/panel-dkim-signing.md)): a milter of the panel's on + the plan, since removed): a milter of the panel's on two sockets, one for applications' mail - signed with the key of the domain the login belongs to - and one for the panel's own, signed with the server's name. `relaxed/simple`, RFC 6376 § 5.4.1's headers and `From` @@ -643,7 +643,7 @@ No migration; schema 11. The listener now sets `normalize` - the setting Postfix recommends and 3.9 made the default: DATA ends only at the standard `.`, a bare newline elsewhere is read as a line end. Found while reading for - [own-mta-builds](docs/own-mta-builds-2026-10-15.md) (§ 5); `test/e2e` + own-mta-builds (§ 5, since removed); `test/e2e` sends the sequence and finds one message at the upstream with all of it inside. Only installations with `INBOUND_RELAY_ENABLE=true` or `DMARC_REPORTS_ENABLE=true` have the listener. @@ -1359,7 +1359,7 @@ Documentation only; no product code changes. held to - any address of the application's domain, or the addresses of its list, with or without a `+extension` - and refuses anything else with `550 5.7.1`, a domain this server does not sign for included - ([plans/from-header-binding.md](docs/plans/from-header-binding.md)). + (plans/from-header-binding.md, since removed). - **The check is strict.** It is the first milter of the chain, on a socket of its own (`SENDER_CHECK_SOCKET`), attached with `default_action=tempfail`: while the panel does not answer, Postfix defers @@ -1413,7 +1413,7 @@ Documentation only; no product code changes. a user's own switch, `users.notify`, on for everyone; `notify_state`, what the panel has already written to its users about. The instance switch is a settings key, on unless turned off - ([plans/panel-notifications.md](docs/plans/panel-notifications.md)). + (plans/panel-notifications.md, since removed). - **The panel writes to its users when something needs them.** Four things, each said once: a sending domain's DKIM, SPF or DMARC record stops matching (after two hourly checks in a row, and once more when it matches again); a @@ -1463,7 +1463,7 @@ Documentation only; no product code changes. The table that will hold what Postfix wrote about a message for as long as its send-log row lives, with the store methods to write a line, to write it once, to read the lines of a row and to prune - ([plans/delivery-log-storage.md](docs/plans/delivery-log-storage.md)). + (plans/delivery-log-storage.md, since removed). - **The log tailer keeps what Postfix writes about queued messages.** A `mail.log` line headed by a queue id is stored with the line's own time, one transaction per pass over the log; after an unclean stop a line read twice @@ -2063,7 +2063,7 @@ screens. Nothing in `internal/web` changes; the design stays `candidate` scrolling at 1280px and up. Documented on the components page and in the plan (§ Type and tables). - Panel redesign, header comparison sheet - ([docs/assets/panel-redesign/header/](docs/assets/panel-redesign/header/index.html)): + (`docs/assets/panel-redesign/header/`, since removed): at nav size the stamp dissolves into the brick bar and the perforated edge was drawn inverted (round bumps instead of round holes). Three replacements beside the current header — **A** wordmark with the stamp's real hole-to-tooth @@ -2137,7 +2137,7 @@ screens. Nothing in `internal/web` changes; the design stays `candidate` grammar (`measure` / `fill` plus Web Awesome layout CSS), no JS component library, new mockups rather than `docs/assets/panel-ui/`. Status: candidate. - Roadmap item and plan for `/preflight` installation check page - ([docs/plans/preflight.md](docs/plans/preflight.md)) — instance-level + (`docs/plans/preflight.md`, since removed) — instance-level infrastructure verification (rDNS, TLS, ports, HELO banner, proxy headers, DKIM socket, test email). Status: candidate. @@ -2311,7 +2311,7 @@ antispam and quarantine. Upgrading from 1.9.0 is a tag bump; no migration. ### Added -- [docs/plans/inbound-antispam-panel.md](docs/plans/inbound-antispam-panel.md) — +- `docs/plans/inbound-antispam-panel.md` (since removed) — agreed plan for inbound antispam journal (inbound-journal milter + mail.log rejects) and instance-wide allow/deny lists synced to rspamd maps; target `1.10.0`; Composer → Opus → Fable workflow. @@ -2511,7 +2511,7 @@ tag bump; no schema migration. self-contained backups and persistent queue. - docs: HTML mockups for a full panel UI refresh - ([docs/assets/panel-ui/](docs/assets/panel-ui/index.html)) — current screens, + (`docs/assets/panel-ui/`, since removed) — current screens, agreed and candidate roadmap surfaces (queue-retries, inbound-relay, dmarc-reports, panel-docs), a hybrid width (ops pages use the window, forms keep a reading measure), and a 390px emergency layout. The stamp and brick diff --git a/README.md b/README.md index 45fd48a..763ba93 100644 --- a/README.md +++ b/README.md @@ -44,7 +44,7 @@ send log and DNS checks in the panel, encrypted backups. | [Architecture](docs/architecture.md) | As-built technical design | | [Security design](docs/security.md) | Mandatory requirements, accepted risks, the CSRF ADR | | [Development](docs/development.md) | Building, testing, docs rules, model routing, commits | -| [Roadmap](docs/roadmap.md) | Open work (1.x+) — direction, not commitments | +| [Roadmap](docs/roadmap.md) | Open work — direction, not commitments | | [CHANGELOG](CHANGELOG.md) | Release history | Found a vulnerability? Do not open an issue — [SECURITY.md](SECURITY.md) has diff --git a/build/Dockerfile b/build/Dockerfile index 5056805..f06bb2c 100644 --- a/build/Dockerfile +++ b/build/Dockerfile @@ -57,7 +57,7 @@ ENV DEBIAN_FRONTEND=noninteractive # test say whether it does. # No DKIM package is installed: the panel signs itself (internal/milter, # sign.go) and makes the keys in Go (internal/domain/dkim.go); OpenDKIM left -# the image in 2.8.0 (docs/plans/panel-dkim-signing.md). +# the image in 2.8.0 (docs/architecture.md § DKIM signing). # systemd-standalone-sysusers is named so that cron-daemon-common, which # logrotate brings (logrotate 3.22.0-1, debian/control: "Depends: cron | # anacron | cron-daemon | systemd-sysv"), creates its group with it rather diff --git a/build/entrypoint.sh b/build/entrypoint.sh index dc3c3c9..4e03067 100644 --- a/build/entrypoint.sh +++ b/build/entrypoint.sh @@ -111,7 +111,7 @@ find /data -mindepth 1 -maxdepth 1 ! -name log ! -name postfix ! -name sasl ! -n # serves (cmd/panel/signer.go). Until 2.8.0 the keys were under # /data/opendkim, shared with OpenDKIM; 2.8.x copied them to /data/dkim at its # first start and kept the old tree for a rollback to 2.7.x until the next -# release, this one (docs/plans/panel-dkim-signing.md, Q2). It is removed +# release, this one (CHANGELOG [2.9.0]). It is removed # once /data/dkim holds a key. A /data that never ran 2.8.x - keys under the # old tree only - stops the start instead: the panel would make the server's # own name a new key, and its published record would no longer match; 2.8.x diff --git a/build/postfix-config.sh b/build/postfix-config.sh index e1dd3e9..8fb13ea 100644 --- a/build/postfix-config.sh +++ b/build/postfix-config.sh @@ -426,7 +426,7 @@ postconf -e \ # deferred (docs/security.md). # # The panel's DKIM signer follows, strict too: unreachable, it defers rather -# than send unsigned (docs/plans/panel-dkim-signing.md). It signs with the +# than send unsigned (docs/architecture.md § DKIM signing). It signs with the # key of the login's domain. The journal-milter is monitoring only and is # fail-open (default_action=accept): its failure must never block the relay. # Per-milter settings use Postfix 3.0+ brace syntax. @@ -756,7 +756,7 @@ if [ "${INBOUND_RELAY_ENABLE}" = "true" ] || [ "${DMARC_REPORTS_ENABLE}" = "true # only here (above), so no client is spared. The submission ports take # the default, normalize as well since this Postfix: an application whose # SMTP library ends DATA with a bare . is no longer read to its - # end there (own-mta-builds, § 5: their smuggled transaction would still + # end there (docs/security.md § Mail path: their smuggled transaction would still # be the authenticated session's, so they were left to the default). postconf -P \ "smtp/inet/smtpd_forbid_bare_newline=normalize" \ diff --git a/cmd/panel/sendercheck.go b/cmd/panel/sendercheck.go index 652f687..69015e3 100644 --- a/cmd/panel/sendercheck.go +++ b/cmd/panel/sendercheck.go @@ -24,7 +24,7 @@ const aliveEvery = 2 * time.Second // try again later"), so while this role is not serving, mail is deferred: // what that costs is counted here and // by the log tailer, and shown on Server > Health -// (docs/plans/from-header-binding.md § Statistics). +// (docs/architecture.md § Sender check). func serveSenderCheck(ctx context.Context, cfg config, st *store.Store) error { ln, err := listenMilter(cfg.senderSocket) if err != nil { diff --git a/docs/architecture.md b/docs/architecture.md index f3cf08b..75af945 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -177,9 +177,15 @@ smtp/inet` removes Debian's stock smtpd). Outbound delivery still uses the off the journal on each view of Filter lists - the rows a list decided (the deny rule's reply, the allow rule's symbol, which the journal milter keeps ahead of its six-symbol cap) matched against the entries as rspamd matches - them. When that stops being good enough is written in - [plans/inbound-antispam-panel.md](plans/inbound-antispam-panel.md) § Filter - list usage. + them. So a count reaches back as far as the journal does, a row counts for + every fitting entry of the deciding list, and a count is of messages, + roughly. A counter kept with each entry (hits and last hit on + `inbound_spam_list`, a migration) is worth it when any one of these holds + (owner, 2026-10-12): the journal is cut by its 10 000-row cap rather than by + age (the tailer's prune keeps removing rows while the oldest row of + Inbound › Log is younger than the retention); the retention is set short, + towards its 7 days; counts are wanted that outlive the journal; or the page + is slow - past a few hundred entries, measure it. **Inbound relay** (`INBOUND_RELAY_ENABLE=true`): accepts only `relay_domains` + `relay_recipient_maps` (`reject_unauth_destination`, `reject_unlisted_recipient`). Maps under `/data/postfix/` (`relay_domains`, @@ -235,8 +241,7 @@ smtp/inet` removes Debian's stock smtpd). Outbound delivery still uses the The panel signs outgoing mail itself, in a role of its own, `dkim-signer` ([cmd/panel/signer.go](../cmd/panel/signer.go), [internal/milter/sign.go](../internal/milter/sign.go)), since `2.8.0`; until -then OpenDKIM did, in a program of its own -([plans/panel-dkim-signing.md](plans/panel-dkim-signing.md)). +then OpenDKIM did, in a program of its own (CHANGELOG `[2.8.0]`). - **Keys** — `/data/dkim/keys//.private`, PKCS#1 PEM, one for each sending domain and one for the server's own name @@ -353,7 +358,7 @@ milter, § Postfix above, port 25): (`300s` / `4000s` / `5d` / `0`) with a muted note on the box. 2. **sender check** — unix socket `SENDER_CHECK_SOCKET`, first in the chain and **strict** (`default_action=tempfail`): § Sender check below. -3. **DKIM signer** (`2.8.0`, [plans/panel-dkim-signing.md](plans/panel-dkim-signing.md)) +3. **DKIM signer** (`2.8.0`, § DKIM signing) — unix sockets `DKIM_SIGN_SOCKET` (applications' mail on 465/587, signed with the key of the domain the SASL login belongs to) and `DKIM_SIGN_LOCAL_SOCKET` (the panel's own mail, `non_smtpd_milters`, @@ -397,8 +402,8 @@ generates pass no milter. ### Sender check [internal/milter/sender.go](../internal/milter/sender.go) holds a message's -`From:` header against the application that sends it -([plans/from-header-binding.md](plans/from-header-binding.md)). Postfix holds +`From:` header against the application that sends it (since `2.4.0`; +`Sender:` too since `2.8.0`). Postfix holds the envelope sender against the SASL login, and the signer signs with the key of the login's domain (§ DKIM signing): the signature's `d=` is that domain, and a receiver holds it to the domain of `From:` (DMARC, RFC 7489 § 3.1.1). @@ -504,10 +509,15 @@ the hole it closed was the same. [internal/notify](../internal/notify/notify.go) reports four things to the users of the panel: a sending domain's DNS record that stopped matching, a level-2 rate limit that refused mail, the TLS certificate about to expire, a -message more than three days in the queue. The rules of when a message goes -out are the owner's and are written in -[plans/panel-notifications.md](plans/panel-notifications.md) § When a message -is sent; the operator's view is guide § Notifications. +message more than three days in the queue. The operator's view is guide +§ Notifications. When a message goes out is the owner's rule (2026-10-12): + +| Subject | Sent | "Cleared" message | +|---|---|---| +| DNS mismatch | when it has held for an hour (two checks, below); a resolver that did not answer is not a mismatch | the same way back | +| Certificate | 14 days before expiry, and 3 days before if it is still the same certificate | none | +| Level-2 limit refused mail | at the first refusal for a sending domain, then at most once a day for it | none | +| Queue older than three days | when such a message appears | one, when none is left | - **Readings.** Once an hour, starting three minutes after the process does: every sending domain's records through a DNS checker of the notifier's own, @@ -1179,7 +1189,7 @@ full backups also archive the operator's `./certs` PEM files when present. ### Database The schema is an embedded baseline, `internal/store/migrations/0001_init.sql`, -and the migrations after it (`0002` to `0012` today; the next is `0013`), +and the migrations after it (`0002` to `0014` today; the next is `0015`), applied by `store.Open` in file order; `PRAGMA user_version` counts the files applied and `PRAGMA application_id` is `0x53503230` ("SP20"). There is no 1.x migration @@ -1198,7 +1208,11 @@ The tables that carry the panel's model: the user's own address, edited on Account and set by the global role on the user form; `all_domains` / `all_inbound_domains` widen a *domain* user to every domain of that direction, including ones added later (with a flag set - the assignment rows are ignored). + the assignment rows are ignored); `email_confirmed_at` is when `email` was + confirmed, `''` while it is not (v13). +- **`user_links`** — the one-time links that confirm an address and reset a + password (v13): the token's SHA-256, `purpose`, the address it went to, its + expiry; one live link per user and purpose. - **`user_domains`** and **`user_inbound_domains`** — the outbound and inbound domains assigned to a *domain* user. Separate tables: the same name on both is two assignments, and a sending domain gives no inbound access or the other diff --git a/docs/assumptions-2026-10-14.md b/docs/assumptions-2026-10-14.md index eee6b0c..d60c0a7 100644 --- a/docs/assumptions-2026-10-14.md +++ b/docs/assumptions-2026-10-14.md @@ -1,5 +1,12 @@ # What SelfPost assumes of other programs - the register of 2026-10-14 +> **Closed 2026-10-14** (stage 2c of [audit-2026-10-12](audit-2026-10-12.md), +> shipped `2.5.2`): every entry answered, the citations written into the +> code, what the sources contradicted fixed. The file stays because the code +> names its entries (`assumptions C-V5`, `A-43`); its `file:line` references +> are of the tree at `0f984ff`, not of today's, and OpenDKIM, named +> throughout, left the image in `2.8.0`. + The heuristics review of audit-2026-10-12, stage 2c. The owner's rule (development.md § "Code that relies on another component's behaviour"): what Postfix, OpenDKIM, Cyrus SASL, rspamd, supervisord, logrotate, Docker, diff --git a/docs/audit-2026-10-12.md b/docs/audit-2026-10-12.md index e2a5c09..4b339bc 100644 --- a/docs/audit-2026-10-12.md +++ b/docs/audit-2026-10-12.md @@ -1,5 +1,10 @@ # Code audit — 2026-10-12 +> **Closed 2026-10-15.** Every stage shipped (`2.4.2` to `2.6.0`; stage 6, +> documents and tests, without a cut). The file stays because the code cites +> its finding ids (`H4`, `M2`, `L9`...); its `file:line` references are of the +> tree at `8f226bd`, not of today's. + **What this is.** A full-tree audit of the code and infrastructure at `8f226bd` (release `2.4.0`): leftovers, unfinished work, defects, performance, doc drift and test gaps — explicit ones (there are no `TODO`/`FIXME` markers anywhere in @@ -1570,7 +1575,7 @@ tested (the count and the spared are). sleeps said where they are); roadmap.md's closing paragraph. Already right by then: the `.alive` mark, `dmarc-ingest` as a second writer, the chain head, M1's wording, the version stamp. -- [ ] roadmap.md: done ahead on 2026-10-13/14 with the owner - +- [x] roadmap.md: done ahead on 2026-10-13/14 with the owner - `template-data-typing` struck, `structured-logging` dropped, `review-2026-08-followups` closed with each row placed in a stage or dropped with its reason. What is left: check that every *Test gaps* entry diff --git a/docs/development.md b/docs/development.md index 7223b62..4467c4a 100644 --- a/docs/development.md +++ b/docs/development.md @@ -1,7 +1,7 @@ # SelfPost — development **What this file is.** How to build, test, document, and ship changes. Open -work after 1.0 (1.x+) lives in [roadmap.md](roadmap.md) and linked +work lives in [roadmap.md](roadmap.md) and linked [plans/](plans/). Product boundaries: [product.md](product.md). As-built layout: [architecture.md](architecture.md). @@ -90,7 +90,7 @@ checklist changes; update the numerator when steps are checked off. ### Roadmap track: commit and version -For the current 1.x+ roadmap queue ([roadmap.md](roadmap.md) index), stricter +For the roadmap queue ([roadmap.md](roadmap.md) index), stricter than the default «meaningful step» rule in [§ Commits and release build](#commits-and-release-build): **After each checklist step** @@ -106,58 +106,10 @@ than the default «meaningful step» rule in [§ Commits and release build](#com [deploy/docker-compose.yml](../deploy/docker-compose.yml). 2. Git tag `vX.Y.Z` and publish the GitHub Release only on explicit request (see [§ Release image](#release-image)). - -**Planned version cuts** (from pin `2.15.0`; adjust if semver changes mid-track): - -| Stage | ID | Cut | -|---|---|---| -| 1 | queue-retries | `1.3.1` (PATCH) | -| 2 | inbound-relay | `1.4.0` | -| 3 | send-log-retention | `1.5.0` | -| 4 | domain-stats-auto-ratelimit | `1.6.0` | -| 5 | dmarc-reports | `1.7.0` | -| 6 | panel-docs | `1.8.0` | -| 7 | application auth IPs | `1.9.0` | -| 8 | port-25 Postfix startup fix | `1.9.1` (PATCH) | -| 9 | 2026-08-19 code review pass | `1.9.2` (PATCH) | -| 10 | panel-redesign prep (docs only, owner-requested cut) | `1.9.3` (PATCH) | -| 11 | panel-redesign plan: component kit, stages, reorder (docs only, owner-requested cut) | `1.9.4` (PATCH) | -| 12 | GHCR release fix: keep per-arch images, sweep orphaned versions (CI only) | `1.9.5` (PATCH) | -| 13 | panel-redesign | `2.0.0` (MAJOR, no compatibility) | -| 14 | DMARC ingest drops and bounds, full backup from the panel, no queue in a backup | `2.0.1` (PATCH) | -| 15 | inbound-antispam-panel | `2.1.0` (was `1.10.0`, reordered 2026-10-09) | -| 16 | delivery-log-storage | `2.2.0` | -| 17 | panel-notifications | `2.3.0` | -| 18 | from-header-binding | `2.4.0` | -| 19 | code audit 2026-10-12 and its plan (docs only, owner-requested cut) | `2.4.1` (PATCH) | -| 20 | audit-2026-10-12 stage 0: release hygiene | `2.4.2` (PATCH) | -| 21 | audit-2026-10-12 stages 1 and 2 | `2.5.0` | -| 22 | audit-2026-10-12 stage 2b: the log tailer | `2.5.1` | -| 23 | audit-2026-10-12 stage 2c: the sender check reads From: as OpenDKIM does | `2.5.2` | -| 24 | audit-2026-10-12 stage 3: persistence and restore | `2.5.3` | -| 25 | audit-2026-10-12 stage 4: panel behaviour | `2.5.4` | -| 26 | audit-2026-10-12 stage 5: image and Postfix hardening | `2.6.0` | -| 27 | port25-bare-newline | `2.6.1` (PATCH) | -| 28 | base-trixie | `2.7.0` | -| 29 | panel-dkim-signing | `2.8.0` | -| 30 | dns-record-collapse, settings-save-row, notification-mail-recheck; `/data/opendkim` removed | `2.9.0` | -| 31 | log panes wrap on a phone | `2.9.1` (PATCH) | -| 32 | inbound-upstream-result | `2.10.0` | -| 33 | long, non-repeating queue ids | `2.10.1` (PATCH) | -| 34 | phone-desktop-mode | `2.10.2` (PATCH) | -| 35 | backup-reauth | `2.10.3` (PATCH) | -| 36 | origin-strict | `2.10.4` (PATCH) | -| 37 | health-setup-checks | `2.11.0` | -| 38 | test-email | `2.12.0` | -| 39 | local-time | `2.13.0` | -| 40 | docs-audit | `2.13.1` | -| 41 | password-reset | `2.14.0` | -| 42 | overview-head | `2.14.1` | -| 43 | dmarc-day, health-sockets | `2.14.2` | -| 44 | label-tag | `2.14.3` | -| 45 | security-audit-2026-10-17, stages 0-6 | `2.14.4` | -| 46 | etrn-silent | `2.14.5` | -| 47 | server-dmarc-hint, upstream-tls-verify | `2.15.0` | +3. The item leaves [roadmap.md](roadmap.md) and its plan file is deleted; what + it did is in the CHANGELOG, the rest in git history. A report whose + finding ids the code cites (the audits, the assumptions register) stays, + marked closed. Docs-only prep (checklists in plans, no product code) uses the same per-step commit rule but **no** version cut until the next product stage ships. @@ -308,7 +260,8 @@ question, the comment says so and what was measured instead - "measured on the e2e stand, 2026-10-14: ..." - and a measurement is never written as if it were read. A test fixture copied from a real log says where and when it was seen; one composed by hand says that it was composed. The heuristics -review of 2026-10-14 (audit-2026-10-12, stage 2c) brought the code that +review of 2026-10-14 ([audit-2026-10-12](audit-2026-10-12.md), stage 2c; the +[assumptions register](assumptions-2026-10-14.md)) brought the code that existed then up to this rule. --- @@ -623,7 +576,8 @@ There is no `docs/archive/` directory. | Development process (this file) | [development.md](development.md) | | Security requirements and accepted risks | [security.md](security.md) | | SQLite schema and migrations | [schema-migrations.md](schema-migrations.md) | -| Roadmap (1.x+) | [roadmap.md](roadmap.md) | +| Roadmap | [roadmap.md](roadmap.md) | +| Closed audits, cited by finding id from the code | [audit-2026-10-12.md](audit-2026-10-12.md), [assumptions-2026-10-14.md](assumptions-2026-10-14.md), [security-audit-2026-10-17.md](security-audit-2026-10-17.md) | | Active design plans | [plans/](plans/) | | Release history | [CHANGELOG.md](../CHANGELOG.md) | @@ -640,8 +594,8 @@ There is no `docs/archive/` directory. | [deploy/.env.example](../deploy/.env.example) | Public env template; full reference in [guide.md](guide.md) | | [CHANGELOG.md](../CHANGELOG.md) | Keep a Changelog | -Out of scope for v1.x: `CONTRIBUTING.md`, man pages, a separate docs site -(candidates in [roadmap.md](roadmap.md)). +Out of scope: `CONTRIBUTING.md` (dropped, with its condition, in +[roadmap.md](roadmap.md)), man pages, a separate docs site. ### Maintaining documentation diff --git a/docs/own-mta-builds-2026-10-15.md b/docs/own-mta-builds-2026-10-15.md deleted file mode 100644 index 33e55d2..0000000 --- a/docs/own-mta-builds-2026-10-15.md +++ /dev/null @@ -1,276 +0,0 @@ -# Own builds of Postfix and OpenDKIM - the analysis of 2026-10-15 - -**Question (owner, 2026-10-15; roadmap [own-mta-builds](roadmap.md#own-mta-builds)):** -is it worth patching Postfix and OpenDKIM and shipping our own builds, to -optimise something in SelfPost or to lift a limit we work around? - -**Answer:** no to carrying patched builds of either - nothing we work around -needs a patch, and a patch set makes one maintainer the security team of -two MTAs. The question does point at two real problems, each with a cheaper -answer: - -1. **The image's base is out of date.** `debian:bookworm-slim` left regular - security support on 2026-07-12; its Postfix `3.7.11` is code of mid-2024, - and the three Postfix security batches of 2026 (June, August, September) - carry no CVE ids, so nothing will bring them to bookworm's LTS. The - answer is Debian trixie (`postfix 3.10.13`, the August batch included), - not a build of our own: proposed as roadmap row - [base-trixie](roadmap.md#base-trixie). -2. **OpenDKIM has no upstream left.** The package is a beta of 2018; three - CVEs of 2026 are open in every Debian suite with "the vendor did not - respond"; Debian's bug on them has had no answer since 2026-10-03. Every - limit we have worked around is OpenDKIM's. A patched OpenDKIM of our own - would make us its maintainer; signing in the panel's own milter, which - already runs in the mail path, removes the program and most of what was - built around it: proposed as roadmap row - [panel-dkim-signing](roadmap.md#panel-dkim-signing). - -A side finding of the reading: port 25 of the inbound relay accepts bare -newlines - Postfix 3.7's default for the 2023 SMTP-smuggling parameter - -one line fixes it: [port25-bare-newline](roadmap.md#port25-bare-newline). - -Everything below is the evidence and the prices. Versions are those of the -image (`docs/development.md`, the cited-package table); upstream source is -`dev/upstream/`; web sources are linked where a fact came from one. - ---- - -## 1. What the image runs, and who maintains it now - -| Component | In the image (bookworm) | In Debian trixie | Upstream | -|---|---|---|---| -| Debian | 12 bookworm, LTS since 2026-07-12, until 2028-06-30 ([debian.org](https://www.debian.org/News/2026/20260712.en.html)) | 13 trixie, stable since 2025-08-09 | — | -| Postfix | `3.7.11-0+deb12u1` (unchanged since the image was pinned; [tracker](https://tracker.debian.org/pkg/postfix)) | `3.10.13-0+deb13u1` (trixie-security) | `3.11.7`, stable branches 3.8–3.11 ([3.11.7](https://postfix.cs.utah.edu/announcements/postfix-3.11.7.html)) | -| OpenDKIM | `2.11.0~beta2-8+deb12u1` | `2.11.0~beta2-9.1` (same code; a gcc-14 build fix, a path move, one verifier-side header patch) | `2.11.0-Beta2`, 2018; no release since ([releases](https://github.com/trusteddomainproject/OpenDKIM/releases), [issue #422](https://github.com/trusteddomainproject/OpenDKIM/issues/422), unanswered) | -| libmilter | `8.17.1.9-2+deb12u2` | `8.18.1-6` | — | -| Cyrus SASL | `2.1.28+dfsg-10` | `2.1.28+dfsg1-9` (same upstream) | — | -| supervisor | `4.2.5-1` | `4.2.5-3` (same upstream) | — | -| logrotate | `3.21.0-1` | `3.22.0-1` | — | -| Go build stage | `golang:1.26-bookworm` | `golang:1.26-trixie` exists (Docker Hub, 2026-10-10) | — | - -**Postfix in bookworm is 2024 code with the 2026 fixes missing.** Upstream -issued three security batches this year, all without CVE ids: - -- June 2026: unbounded reads of SMTP commands, responses and BDAT data - ("large SMTP inputs"); a TLSA-parsing crash (DANE only - not us). -- August 2026 ([3.11.6](https://openwall.com/lists/oss-security/2026/08/20/5)): - sixteen items, among them a remote memory exhaustion by many small BDAT - commands (smtpd, since 3.4), a remote smtpd crash by DATA without MAIL - FROM after a BDAT error, a policy bypass after an - `smtpd_end_of_data_restrictions` reject (we set none), and the Milter - left in "accept" state by the same bug. -- September 2026 ([3.11.7](https://postfix.cs.utah.edu/announcements/postfix-3.11.7.html)): - two SMTP-smuggling cases behind `smtpd_proxy_filter` (we use none), a - crash under `strict_smtputf8` (not set), a multi-line response over-read - reachable by a malicious Milter, a `postqueue` use-after-free. - -Legacy releases 3.7.22 and 3.7.23 exist upstream but, as the announcements -say, "do not include the patches for out-of-support Postfix versions" of -June. Debian's tracker lists one 2026 CVE for bookworm's Postfix -([CVE-2026-43964](https://security-tracker.debian.org/tracker/CVE-2026-43964), -a one-byte over-read, `no-dsa`) and no bookworm upload at all in 2026; the -LTS team works from CVE ids, and these batches have none. The exposed -listeners are ours: 465 and 587 to the Internet, 25 when the inbound relay -is on. - -**OpenDKIM has three open CVEs and nobody to fix them.** Debian bug -[#1149875](https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1149875) -(2026-10-03, severity important, no reply from maintainer or security team -as of 2026-10-15) covers -[CVE-2026-100888](https://security-tracker.debian.org/tracker/CVE-2026-100888) -(`dkim_canon_selecthdrs`, out-of-bounds write), -[CVE-2026-100889](https://security-tracker.debian.org/tracker/CVE-2026-100889) -(`dkim_qp_decode`, off-by-one) and -[CVE-2026-101277](https://security-tracker.debian.org/tracker/CVE-2026-101277) -(`dkim_process_set`, the tag tokenizer); all three "vendor was contacted -early but did not respond in any way". Read in `dev/upstream/opendkim-2.11.0~beta2` -for our configuration (`Mode s`, attached to 465/587 and to the panel's -local submission only; never to port 25): - -- `dkim_qp_decode` is called from `dkim_sig_domainok()`, - `dkim_sig_getreportinfo()` and `dkim_sig_getidentity()` - (`libopendkim/dkim.c:1718, 7712, 7732, 7801`) - verifier-side functions, - reached from `dkim_eoh_verify()`. Not on our path. -- `dkim_process_set` runs on every `DKIM-Signature:` header of a message - **in either mode**: `dkim_header()` (`dkim.c:6447`, the call at `:6625`) - has no mode gate. On our path that header comes only from an authenticated - application; the exposure is a tenant's, not the Internet's. -- `dkim_canon_selecthdrs` runs when signing as well as when verifying - (`dkim_canon_runheaders()`, `dkim-canon.c:1194, 1280`). The write past - the array is `lhdrs[shcnt] = NULL` (`dkim-canon.c:1052`) once every - header of the message has been matched and a further name follows. When - verifying, the names are the signature's `h=`, the attacker's. When - signing, OpenDKIM composes the list itself - the headers present that - are in `SignHeaders` (by default RFC 6376 § 5.4's SHOULD list, - `dkim_should_signhdrs[]`, `dkim.c:223`) plus `OversignHeaders`, our - `From` - so the condition is a message whose every header is in that - list. Whether Postfix's own `Received:` keeps it out of reach I did not - run; it is one ASan run with `miltertest` on the stand to know, and the - answer changes only how soon § 4's row is wanted, not whether. - -A patch for each exists in the researchers' write-ups (for `dkim_qp_decode`: -[weitongli.com](https://weitongli.com/share/opendkim-qp-off-by-one.html)). -Nobody upstream will take them; Debian may yet carry them - the bug is -twelve days old - but the project behind the package is gone, and the -next finding meets the same silence. - -## 2. The limits met so far, and what would lift each - -| Limit (where it is worked around) | Own patched build | Debian trixie | Signing in the panel | -|---|---|---|---| -| OpenDKIM logs through syslog(3) only (`internal/syslogrelay`, a supervisord program, `/dev/log` in the entrypoint; audit L30) | yes, a stderr patch | no | yes - the program goes, and the relay with it (nothing else in the container uses syslog(3)) | -| The panel reads Postfix's state from `mail.log`, `postconf -h -x`, `postqueue -p`/`-j` | no: these *are* Postfix's interfaces; a patch would invent a private one to maintain | partly: 3.11 adds `postconf -j` (JSON) - not in trixie's 3.10 | no (unchanged) | -| The sender check reads `From:` the way OpenDKIM does: `dkim_mail_parse()` ported (`internal/milter/mailparse.go`, 327 lines), the 256-byte `MAXADDRESS` refusal, two diff/probe scripts against the C | no: the port exists because two programs parse the line; a patch does not merge them (OpenDKIM's `SenderMacro` could pick the key by the envelope instead - it changes nothing, since the check already pins `From:` to the login's senders) | no | yes: one parser, ours, and the key chosen by the login's domain, not by `From:` - the class "signed with another tenant's key" is gone by construction; `From:` still has to be the application's (DMARC alignment), with `net/mail` | -| The image held to bookworm's versions (Postfix 3.7.11, OpenDKIM 2.11.0~beta2) | yes, at the price in § 3 | yes for Postfix (3.10.13, security-tracked to 2028 and beyond in trixie LTS); OpenDKIM's version is the same everywhere | OpenDKIM's version stops mattering | -| A socket file outlives a crashed OpenDKIM (`health/socket.go`, assumptions #38) | yes, with a patch | no | yes - no second milter process | -| Keys group-readable, `RequireSafeKeys no`, the `selfpost` group between panel and OpenDKIM | no | no | yes - the panel reads its own keys; the group stays for Postfix's sasldb and maps | -| `SIGUSR1` through supervisord to reload the tables, which is what needs `CAP_KILL` in the compose file (cross-uid kill) | no | no | yes - no tables, no signal; `CAP_KILL` can probably go (to verify on the stand: supervisord then signals only root's wrapper and the panel's own uid) | -| `smtpd_forbid_bare_newline` defaults to `no` on Postfix < 3.9 (`src/smtpd/smtpd.c:807`); our port 25 does not set it | yes | yes - the default is `normalize` since 3.9 | no (it is Postfix's) - see § 5 | - -Nothing in the Postfix column needs a patch. The limits are OpenDKIM's. - -## 3. The options, priced - -Prices are estimates in stages of the kind the 2026-10-12 audit used (a -stage: write, tests, two reviews, e2e, rollout, commit). Nothing here was -measured on the stand; where a number matters it says so. - -### A. Patched builds of both, carried by us - **not recommended** - -- *Gives:* the stderr patch for OpenDKIM, the three CVE fixes, any option - compiled in or out, the newest Postfix on the day we rebuild. -- *Costs, once:* a build stage per program (`apt-get source`, quilt, a - patch series, `dpkg-buildpackage`), its build-dependencies, the e2e - `checkCitedPackageVersions` rewritten for our version strings, every - citation of `3.7.11` (331 lines in 65 files) and `2.11.0~beta2` (55 in - 14) re-pointed at our tree. Image build time is not measured; from the - size of the sources it is minutes, not seconds, on every rebuild. -- *Costs, for ever:* Postfix issued seven releases in 2026; each is ours - to notice, rebase the series on, rebuild and roll out, with no Debian - security team between the announcement and the image. For OpenDKIM there - is no upstream to rebase on at all: the series *is* the program, and - the next CVE is ours to find and fix. One maintainer, two MTAs. -- *Why not:* § 2 shows no Postfix limit a patch lifts, and every OpenDKIM - limit lifted by option E without taking OpenDKIM on. - -### B. Unpatched upstream build where the distribution's version is the limit - **not recommended** - -- *Gives:* Postfix 3.11.7 today instead of trixie's 3.10.13 - the - September batch two to four weeks earlier (trixie-security carried 3.10.11 - → 3.10.12 → 3.10.13 between July and August; 3.10.14 is upstream now). - For OpenDKIM there is no newer upstream to build. -- *Costs:* A's build stage and watching duty for Postfix, minus the patch - series; Debian's own patches (paths, SASL and LMDB build options) - become ours to reproduce. -- *Why not:* the gap to trixie-security is weeks, the duty is permanent. - -### C. A patch sent upstream instead of carried - **nothing to send** - -- Postfix: no code change is needed (§ 2); Wietse Venema's project is - responsive, but we have no patch for it. -- OpenDKIM: the project does not answer CVE reporters, the Debian - maintainer or its own issue tracker. A patch sent there is a patch - carried - option A. - -### D. The base image to Debian trixie - **recommended, next code row** → [base-trixie](roadmap.md#base-trixie) - -- *Gives:* Postfix 3.10.13 with the June and August batches and the - September one when trixie-security ships 3.10.14; Debian's security team - until 2028, then trixie LTS; `smtpd_forbid_bare_newline=normalize` by - default; `smtpd_hide_client_session` (3.10: a `Received:` header without - the application's address - a product question of its own, not decided - here); OpenSSL 3.5; libmilter 8.18.1. -- *Costs:* one stage, one MINOR release. The work is the re-reading: the - 331 citation lines against the 3.10.13 tree (most will hold word for - word; `src/` paths are stable), with attention to what 3.8–3.10 changed - on our paths - reject logging now carries `sasl_method`, `sasl_username`, - `sasl_sender` (3.10, 20241023: `refusals.go`, `inbound.go` regexes), the - queue id or `NOQUEUE` on an abnormal connection end and cleanup's - `canceled` line (3.10: new line shapes for the log tailer), `postfix(1)` - logging to stderr when it is not a terminal (3.8: the wrapper's output - under supervisord), DNS results capped at 100 records and stray CR/LF - replaced by default (3.9), the removed parameters (3.9: - `permit_naked_ip_address`, `reject_maps_rbl`, `check_relay_domains` - we - set none). Then: `dev/upstream` refetched from trixie, the cited-package - table and `checkCitedPackageVersions`, the e2e sink image, the build - stage tag, `docs/assumptions-2026-10-14.md`'s header, the guide's - "what the image contains". OpenDKIM's citations do not change (same - upstream code). -- *Risk:* a log line we parse changing shape unnoticed - the e2e suite - and the fixtures labelled "seen" are the net; a day on the test server - before the cut. - -### E. DKIM signing in the panel's own milter; OpenDKIM removed - **recommended, after D** → [panel-dkim-signing](roadmap.md#panel-dkim-signing) - -- *Gives:* every OpenDKIM row of § 2; about 1,500 lines gone (the relay - and its tests, the parser port and its tests, the probe scripts, the - table renderer and its reload, `opendkim.conf`), two programs fewer - under supervisord, two packages fewer in the image, no CVE watch on a - dead project, probably `CAP_KILL` dropped. Keys, selectors and the - published DNS records are unchanged: the panel already generates the - keys (`internal/domain/dkim.go`, PKCS#1 PEM) - nothing for an operator - to do. -- *How:* `github.com/emersion/go-msgauth/dkim` (v0.7.0, 2025-04, MIT; the - author of the `go-milter` we run; the e2e suite already verifies with it). - Read for this analysis: `SignOptions` gives `relaxed`/`simple` per side, - `HeaderKeys` with over-signing as RFC 6376 § 5.4.2 wants it (a name - listed once more than present is kept in `h=` and nothing is hashed for - it, `sign.go`), last-instance-first header picking (`header.go`), RSA and - Ed25519, and a streaming `NewSigner` (the body is hashed as it passes, - not buffered). The milter: headers and body chunks in, `InsertHeader(0, - "DKIM-Signature", …)` at end of message - the shape of go-msgauth's own - `cmd/dkim-milter`, some 450 lines. The key: by the login's domain on - 465/587 (`Bindings.DomainOf`, which the sender check has), by the - server's name for the panel's local submission (`non_smtpd_milters` - pointed at the same socket - strict, as OpenDKIM is today). The header - set: `dkim_should_signhdrs[]`, `From` over-signed once; canonicalization - `relaxed/simple`, as now. -- *Costs:* two to three stages, one MINOR release: the signer and its - tests; the removal (code, image, compose, supervisord, entrypoint, - Health's liveness set, Resync's steps, docs: architecture § OpenDKIM, - security, guide, development's table, assumptions § 3); e2e with an - **independent verifier as the oracle** - go-msgauth must not check its - own work: `opendkim-testmsg` in the sink image or rspamd's verifier, the - way `mailparse-diff.sh` holds the port to the C today - and a message to - a real mailbox before the cut. -- *Risk:* a signing defect fails DMARC for every tenant at once; the - oracle and the canary are the answer. go-msgauth is a small library with - a quiet release pace - vendorable if it ever matters. -- *Why after D:* D's exposure is remote and unauthenticated (465/587), - E's is a tenant's; D is one stage, E is three; neither waits on the - other (E re-reads no Postfix citation, D no OpenDKIM one). - -## 4. Recommendation - -In this order, each a roadmap row of its own, none of them an own build: - -1. **[port25-bare-newline](roadmap.md#port25-bare-newline)** - one line in - `build/postfix-config.sh`, a PATCH release, now (§ 5). -2. **[base-trixie](roadmap.md#base-trixie)** - option D, one stage, MINOR. -3. **[panel-dkim-signing](roadmap.md#panel-dkim-signing)** - option E, two - to three stages, MINOR. - -Options A, B and C are declined with the arguments above; what would bring -A back is a limit of Postfix's that its interfaces cannot answer, and none -is known. - -## 5. Side finding: bare newlines on port 25 - -Postfix 3.7.11 has the 2023 SMTP-smuggling parameter -(`smtpd_forbid_bare_newline`, `src/global/mail_params.h:4249`) with the -default `no` "for Postfix < 3.9" (`src/smtpd/smtpd.c:807`); its -`RELEASE_NOTES` recommend `normalize`, which 3.9 made the default. The -inbound relay's listener (`build/postfix-config.sh`, the `smtp/inet` -block) does not set it, so with `INBOUND_RELAY_ENABLE=true` the relay is -the receiving side the smuggling research described: a bare `.` -inside DATA ends the message early and what follows is read as the next -transaction. Ports 465 and 587 are not the case that matters - the -smuggled transaction is still the authenticated session's, under its -sender bindings and milters. The fix is -`smtp/inet/smtpd_forbid_bare_newline=normalize` with an e2e check on port -25 (a bare-LF DATA, one message delivered); a PATCH release. Trixie's 3.10 -would set it by default, but the relay should not wait for that row. - -A smaller one: `opendkim-tools` is installed in the image and used by -nothing at run time (the key is generated in Go; the probe script runs on -the server from source). It goes with option E, or with D if E waits. diff --git a/docs/plans/delivery-log-storage.md b/docs/plans/delivery-log-storage.md deleted file mode 100644 index 1667d6f..0000000 --- a/docs/plans/delivery-log-storage.md +++ /dev/null @@ -1,185 +0,0 @@ -# Plan: delivery-log-storage - -**Status:** shipped in `2.2.0` (2026-10-12); agreed the same day (owner: option 1 of the roadmap item, "store the lines") -**Date:** 2026-10-12 -**Version:** `2.2.0` MINOR (one DDL migration; a 2.1.0 data directory boots unchanged). - ---- - -## Goal - -A message's page (`/outbound/log/{id}`, mockup `out-message`) shows what -Postfix wrote about the message for as long as its row is in the send log. -Today the *Delivery log* box is filled by a search of the current `mail.log` -on every page view, so it is empty for anything sent before the last daily -rotation, while the row itself lives for ninety days. - -## Context (as-built) - -- The log tailer ([internal/logtail](../../internal/logtail/logtail.go)) - already reads every `mail.log` line once, from a persisted offset, to - advance delivery statuses. -- `logtail.QueueLines` searches the last 4 MB of the current file for one - queue id; `Handlers.deliveryLog` calls it per page view. -- A message with several recipients is several send-log rows under one queue - id; the page of each shows every line of the message, as the search does - today. - -## Decisions - -### What is stored - -A `mail.log` line whose head is a queue id - after the timestamp, the host -name and `program[pid]:` - goes into a new table with that id, the line's own -time (`lineTime`, the same reading the inbound journal uses) and the line as -written, cut to 4096 bytes on a character boundary. The head is matched from -the start of the line: the tail is the far end's words. `NOQUEUE` and -`warning` are not queue ids - the pattern takes Postfix's short form -(hexadecimal) and its long form (no vowels) and nothing else. - -The tailer does not ask whether the id belongs to a send-log row before it -writes. The first lines of a message (`client=`, `message-id=`) are logged -before the journal milter writes the row at end-of-message, so such a check -would drop exactly them. - -**Mail that came in on port 25 is left out** (security review, 2026-10-12). A -queue id that `postfix/inbound/smtpd` opened is a stranger's message; it never -gets a send-log row, and what a stranger can make Postfix log - his own -`message-id`, up to the cut - is not stored to be pruned later. The tailer -remembers such ids (4096 at most) until qmgr's `removed` or until another -listener opens a message under the id. What is still stored without a row is -little: the bounces Postfix generates, and port-25 mail met in mid-flight -after a restart. - -Lines are written in one transaction per pass over the log, 500 at most -(technical review): a commit per line is a disk sync per line on the one -connection the journal milter writes through. - -### Which lines are a row's - -A line is shown on a row's page when the queue id is the same **and** the -line's time lies between one hour before the row was written and the queue -lifetime (the larger of `maximal_queue_lifetime` and `bounce_queue_lifetime`, -as `postconf` reports them) plus a day after it. Postfix reuses a short queue -id once the message has left the queue - the id is the microsecond of arrival -and the inode of the queue file - so a row that lives ninety days can meet a -later message under its id. The window does not rule that out, it limits it -to the days the row's own message could still be in the queue; this is the -bound the roadmap item asked for. The search of the current file that the -page did before had the same exposure for a day. `enable_long_queue_ids` -would end it and is a change to the mail path, not part of this item. -*It was made in `2.10.1` (owner, 2026-10-16).* - -### Retention - -Lines live as long as a send-log row with their queue id does. One rule covers -both ends: a line older than an hour whose queue id has no send-log row is -deleted. That removes the lines of a row the retention sweep has just deleted, -and the lines of messages that never had a row. It runs at start and after -every send-log sweep, six-hourly. It asks the send log once per queue id; at a -million lines that is still a scan of the table, of the order of a second, -during which the one connection is taken. - -There is no cap of its own: with port-25 mail left out, the table is bounded -by what authenticated applications send, which level 1 and level 2 limit. - -### After an unclean stop - -The tailer saves its place every few seconds, so after a crash it reads some -lines twice. While it is catching up at start a line is written only if the -same line is not there already - the rule the inbound journal follows. - -### What is not kept - -- Lines the tailer never reads are not stored: the tail of a file that was - rotated away while the panel was down, as with delivery statuses - (architecture.md § Log tailer). -- A message in the queue across the upgrade to 2.2.0 has only its later lines - stored, and the page then shows those alone: the search of the file runs - only for a row with nothing stored. The only installation is the owner's - test server; no merge of the two sources is built for one moment of it. - -### The page - -The page reads the table: the latest 200 lines of the row, oldest first. When -the table has none, it still searches the current `mail.log` as before - that -is what shows a message accepted before 2.2.0 while its lines are in today's -file. Lines in rotated files are not imported: the only installation is the -owner's test server. - -Nothing in the page's layout changes; the `out-message` mockup already draws -the box as a `log_pane`. Two texts change, mockup first: the *Outbound log* -help topic (it says an old message may show no lines) and the note shown when -there are none. - -### Size - -Six to ten lines of about 200 bytes per message: 1-2 KB next to a row of a few -hundred bytes. The table becomes the larger part of what the send log costs in -`/data` and in a backup; the retention setting bounds both together. - -## Schema (migration `0003_send_log_lines.sql`) - -```sql -CREATE TABLE send_log_lines ( - id INTEGER PRIMARY KEY AUTOINCREMENT, - queue_id TEXT NOT NULL, - logged_at TEXT NOT NULL, - line TEXT NOT NULL -); -CREATE INDEX idx_send_log_lines_queue_id ON send_log_lines (queue_id, logged_at); -``` - -No foreign key: a line is written before its row exists, and one queue id is -several rows. - -## Scope - -In: the table, the tailer writing and pruning it, the page reading it, the two -texts, docs. - -Out: the inbound journal (it decides for itself, roadmap § delivery-log-storage); -importing rotated files; `enable_long_queue_ids`; any change to `mail.log` -rotation or to the Server › Mail log page. - -## Security - -- The lines carry the far end's text (replies, HELO names). They are shown - today from the file through the same escaping; storing them changes where - they are read from, not what is shown or to whom. A domain administrator - sees the lines of messages of their own domains, as now. -- They now enter the database and so a full backup, which `log/` does not: - submitting clients' addresses and SASL logins, message ids, receivers' - replies. [security.md](../security.md) says so. -- A line is matched from its head, so a reply cannot make itself a line of - another message; Postfix does not log raw line breaks, and `postlogd` is the - file's only writer. -- A stranger on port 25 does not write to the table (above). - -## Done when - -A message sent before the last rotation shows its lines on its page; the lines -disappear with the row; the page's wording no longer blames rotation for an -ordinary case. - -## Model routing (this plan) - -| Role | Model | -|------|-------| -| All implementation (code, tests, docs) | **Opus** | -| Technical review (reviewer ≠ author: a separate read-only agent) | **Opus** | -| Security review | **Fable** | - -## Implementation checklist - -Target version cut: **`2.2.0`** (MINOR). One commit per step; -[development.md](../development.md) § Plan checklists. - -- [x] Migration `0003_send_log_lines.sql`, store methods (insert, insert once, lines of a row, prune) and their tests — **Opus** -- [x] Log tailer: store queue-id lines, write once while catching up, prune with the send-log sweep — **Opus** (hourly as first built; six-hourly after the reviews) -- [x] The two texts, mockup first: the *Outbound log* help topic and the empty note (`design:` commit) — **Opus** (the help topic is in the mockups; the empty note is not drawn there - the `out-message` mockup shows a message with lines - and changes with the page) -- [x] Message page reads the table, falls back to the current `mail.log`; help text in `help.go` — **Opus** -- [x] [architecture.md](../architecture.md), [guide.md](../guide.md), [schema-migrations.md](../schema-migrations.md) — **Opus** (schema-migrations.md went in with the migration) -- [x] `go vet`, `go test`, e2e: a message's page still lists its lines after a forced rotation — **Opus** (on the test server: vet ok, 22 packages ok, e2e ok with the new step `delivery_log_survives_rotation`) -- [x] Technical review — **Opus** (2026-10-12; no Critical or High. Fixed: one transaction per pass instead of a commit per line, the prune asks the send log once per queue id and runs six-hourly, the cut keeps characters whole, tests for what the comments claimed. Left as found and written down under *What is not kept*: lines the tailer never read, and a message in the queue across the upgrade) -- [x] Security review — **Fable** (2026-10-12; no Critical or High. Fixed: port-25 mail is not stored, so a stranger cannot grow the table; [security.md](../security.md) states what a backup now carries. The window against queue-id reuse is described as the limit it is) diff --git a/docs/plans/from-header-binding.md b/docs/plans/from-header-binding.md deleted file mode 100644 index ba5819d..0000000 --- a/docs/plans/from-header-binding.md +++ /dev/null @@ -1,260 +0,0 @@ -# Plan: from-header-binding - -**Status:** shipped in `2.4.0` (2026-10-12); agreed the same day - the design questions were closed with the owner one at a time and he took it into work ("Да") -**Date:** 2026-10-12 -**Version:** `2.4.0` MINOR (one migration; it refuses mail that was accepted before). - ---- - -## Goal - -An application sends only as itself - in the `From:` line a mail program -shows and OpenDKIM chooses the key by, not only in the envelope. - -Until now Postfix held the envelope sender against the SASL login and nothing -held the header. An application of one sending domain could write another -registered domain, or the server's own name, into `From:` and have the message -signed with that name's key (security review of panel-notifications, -2026-10-12). - -## Decisions (owner, 2026-10-12) - -### The rule - -**`From:` is held to what the envelope is held to.** An application set to -any address of its domain may write any address of that domain; one with a -list of addresses, only those. Anything else is refused: another registered -domain, the server's name, an address at a domain this server knows nothing -of, no `From:` at all. - -I proposed the narrower rule - refuse only a `From:` that names a domain this -server signs for, and let a stranger's address through unsigned as before - -and, for an application with a list, to check the domain alone. The owner -chose the wider one both times. What it stops working: a form that puts its -visitor's address into `From:`, and an application with a list whose envelope -address and header address differ, until the second is added to the list. - -When the hostname is itself a sending domain its applications write -`…@` by right, as the guide already says. - -### Where it is checked, and what happens when it cannot be - -**The panel checks, and the check is strict:** while the panel does not -answer, Postfix replies with a temporary error and accepts nothing unchecked. - -I first proposed the opposite (fail-open, as the journal and the level-2 limit -are) and then a small process of its own. The owner's objection to the first: -a check that is absent while the panel is down does not close the hole, and -the panel only has to be slow - 15 seconds - for Postfix to go on without it. -His answer to the second: not yet; measure first. - -So the check is a role of the panel process on a socket of its own, attached -with `default_action=tempfail`. The journal milter keeps its socket and stays -fail-open: a slow database still does not defer mail. The check reads no -database - it reads the file of sender bindings Postfix itself reads for the -envelope, so the two cannot disagree - and moving it into a process of its own -later is a change of where it is started. - -### Statistics, to decide whether the check needs its own process - -Per day, kept a year: - -- **for how long the panel did not answer while Postfix was running.** The - panel marks itself alive every few seconds; at start the gap since the last - mark is its downtime, counted when Postfix was not restarted within it (a - rollout or a reboot stops mail for another reason); -- **how many messages were deferred because the check did not answer**, read - from `mail.log`. This one also sees a panel that is running and not - answering, which the mark does not. - -Shown on **Server › Health** in a card of its own, *Sender check*: today, the -last 30 days, the last 12 months. The card warns when a message was deferred -in the last 24 hours; downtime that deferred nothing warns of nothing. - -### What a refusal looks like - -- **To the application:** `550 5.7.1`, permanent, naming the address. A - level-2 limit answers 4xx because the message will pass in an hour; a wrong - `From:` will not, and a 4xx would have the application send it again for - days. -- **In the Outbound log:** a row with the reason in the status tag - - `wrong From`, beside `rate-limited` and the tag of a refused client address. - The built panel wrote `rejected` for all of them; it follows the mockup now. - (The mockup's status filter is not built - the log filters by domain and - application - so there is nothing to keep in step with the tags.) -- **On the message's page:** a fact *From: line* with the address written in - the header, and a History step that says which address was refused, for - which application, what that application may write, and that it was told - `550 5.7.1`. - -### Health, re-laid - -The owner on the first picture of the new card: too many holes on the screen, -the numbers should follow importance, and the boxes should read left to right -and top to bottom - with stacked columns the third box was below the fold and -the fourth was not. - -- Order: 01 Processes, 02 Milter sockets, 03 Sender check, 04 TLS - certificate, 05 Hostname and reverse DNS, 06 Notification mail, 07 Machine, - 08 Configuration. -- Layout `cards` (new, on the components page): two stacks, odd numbers on the - left and even on the right, each closed up, so a short box leaves no gap - beside a tall one and the eye still goes 01, 02, 03. On a phone the boxes - come in the order of their numbers. - -## Settled while building - -- **The header is read twice.** One `From:` field holding one address, by - Go's parser; then every `@name` of the raw value is held against OpenDKIM's - own `SigningTable`. The first build took the names for the second look from - the sender bindings, and the e2e stand let a line through that named a - domain with a key and no application yet: a domain is a signing name from - the moment it is added, not from its first application. -- **What Postfix writes when the check does not answer** (seen on the stand): - a warning naming the socket, then `milter-reject: CONNECT … 451 4.7.1 - Service unavailable` and the same for `EHLO`; the client is told `454 4.3.0 - Try again later` at `AUTH` and leaves. No `MAIL` is ever offered, so what - is counted is the connection turned away, once. The journal milter's - level-2 refusal is written in the same words, which is why a reject line - counts only after the warning. -- **When Postfix started** is asked of supervisord (`supervisorctl status`, - the uptime of `postfix`). -- **The reply** is a milter reply code, `550 5.7.1 From: - `; what - the client wrote is reduced to printable ASCII and cut first. -- **The Delivery log box stays on a refused message's page**, with the one - sentence it already had for a message that never reached the queue; the - picture shown to the owner had left the box out. - -## To settle while building (as first written) - -- How the header is read so that the check and OpenDKIM cannot disagree: one - `From:` field, one address, and a cruder second look at the raw value. -- What Postfix writes in `mail.log` when a strict milter does not answer, and - how a line is told to be this milter's and not OpenDKIM's. -- How the panel learns when Postfix started. -- Schema: the reason of a refusal and the header address on a send-log row; - the table of days. -- That a refusal for `From:` is not taken for a refusal under a limit by the - notifier. - -## Implementation checklist - -- [x] Mockups: Health card and layout, the log row, help topic; accepted by the owner 2026-10-12 (`design:` commit) -- [x] Migration and store: refusal reason and header address, days of the sender check (`0006_sender_check.sql`) -- [x] The check: a role of the panel on its own socket, bindings read from Postfix's file; Postfix attaches it strictly; start order -- [x] Statistics: the alive mark and downtime, deferred mail from `mail.log` -- [x] Panel: Health (card, sockets row, layout), Outbound log tags, the message's page, help -- [x] guide.md, architecture.md, security.md (the accepted risk goes), schema-migrations.md -- [x] `go vet`, `go test`, e2e: refused for another domain's `From:`, for the hostname's and for an address off the list; accepted for its own (on the test server: vet ok, 23 packages ok, e2e ok - `sender_check_holds_the_from_line`, `panel_down_defers_mail`) -- [x] Technical review — **Opus** (2026-10-12; one Critical, fixed: a folded `From:` line - what a mail library makes of a long display name - was refused for good. Also fixed: a display name in a charset Go does not convert was refused; the statistics ran before the check served, and a hung `supervisorctl` would have kept it from serving; the count of deferred mail missed a check that stops answering in the middle of a message, and is now taken at Postfix's warning; a card whose figures could not be read said ok; a killed container's stale socket satisfied the wrapper's wait; outside quoted strings the line may now hold one `@` only. Three were first left as found and fixed the same evening at the owner's question: downtime is cut at the UTC midnights it crossed; the row of a message with the null envelope sender takes the domain the login is bound to; a log read again after an unclean stop does not count again what it counted before) -- [x] Security review — **Fable** (2026-10-12; no Critical or High. The same folded line, as its one Medium. Fixed: another sender's name in an encoded display name passed the second look; a client could arm the deferred counter by writing the socket's name where Postfix quotes it in a warning; a refused message wrote a row per recipient, a thousand at most - now one. Recorded: what a display name looks like to a person is not promised. The review also found the check stricter than Postfix for an address with an extension (`invoices+x@`, allowed in the envelope and refused in the header); the owner asked for it to work, and the check now makes Postfix's second try, with `recipient_delimiter` set explicitly. Confirmed from the code: no path to the signer around the check, and every failure of the check defers) - -## Not done - -Written at the owner's word when he had the version cut ("Запиши все -недоделки в план", 2026-10-12). Nothing here was promised and dropped in -silence; each line says what is missing and what it would take. - -### Decided to wait - -- **A process of its own for the check.** The owner: measure first. Server › - Health now has the numbers; the decision is his when they have had time to - say something. Moving the role out is a change of where it is started - - a subcommand of the panel binary and a `[program:]` in supervisord - and - the downtime figure then needs a new meaning. - -### Shown to the owner one way and built another - -- **The Delivery log box is on a refused message's page.** The picture he - approved had left it out; the built page keeps it, with one sentence. And - the sentence is less than the truth for a `From:` refusal: cleanup did - write a line about it under a queue id (`milter-reject: END-OF-MESSAGE ... - 5.7.1 From: ...`), but the check answers before the queue id reaches it, so - the row keeps none and the page cannot show that line. **Decided - 2026-10-13 (owner):** the box is required - he had overlooked its absence - from the picture, it was never a decision to drop it. The tailer attaches - the line to the row, for all three kinds of refusal; a step of - [audit-2026-10-12](../audit-2026-10-12.md) stage 2b. -- **The Outbound log has no status filter.** The mockup draws one; the built - log filters by domain and application. With three kinds of refusal now - told apart by tag, a filter by status and reason would be worth having. - **Decided 2026-10-13 (owner):** a filter by status, without the split by - reason, and a filter by day as the Inbound log has; a step of [audit-2026-10-12](../audit-2026-10-12.md) stage 4. -- **"Messages deferred" counts connections Postfix turned away.** Postfix - refuses from the connection on, before a message is offered, so a monitor - that only connects during an outage is counted. The guide says so; the - card's column head and the help topic still say "messages". **Decided - 2026-10-13 (owner):** the column is renamed *Connections turned away*; a - step of [audit-2026-10-12](../audit-2026-10-12.md) stage 2b. - -### Known and small - -- **"Today" on the card is the UTC day**, and the card does not say so. - **Decided 2026-10-13 (owner):** not a label - the local day; it became the - roadmap item `local-time`, for the whole panel. -- **A deferral in the very second of the last one counted before an unclean - stop is passed over** at the restart: `mail.log` stamps whole seconds. - Chosen over counting a stretch of the log twice. **Decided 2026-10-14 - (owner):** counted exactly - the counter keeps a place in the log, saved - with the count; a step of [audit-2026-10-12](../audit-2026-10-12.md) - stage 2b. -- **Deferrals are lost when `mail.log` is rotated while the panel is down**: - the tailer starts on the new file. **Decided 2026-10-13 (owner):** the - tailer reads the rotated file to its end first - not the deferrals alone, - everything the panel takes from the log; a step of - [audit-2026-10-12](../audit-2026-10-12.md) stage 2b. -- **A start at which `supervisorctl` fails counts no downtime**; it is - logged. **Decided 2026-10-13 (owner):** the question is repeated, and - without an answer the downtime is counted from the alive mark and the day - marked as estimated; a step of - [audit-2026-10-12](../audit-2026-10-12.md) stage 2b. -- **Logins are compared as written**, Postfix compares them without regard - to case. The panel writes logins one way, so nothing differs today. -- **The phone order of the `cards` layout is written for four boxes to a - stack** and takes every child of a stack for a box (said at the partial). - The rule the technical review suggested instead - the order from the - box's own number - changes the component and would be drawn first. - **Decided 2026-10-13 (owner):** that rule is built; a step of - [audit-2026-10-12](../audit-2026-10-12.md) stage 4. -- **Health's socket check looks at the file, not at who listens**: a socket - nobody answers on would read "ok". The start now removes stale sockets, - which was the one way to get such a file. **Decided 2026-10-13 (owner):** - the check connects to the socket; a step of - [audit-2026-10-12](../audit-2026-10-12.md) stage 4. -- **The kit page's table of status tags does not list the new ones** - (`wrong From`, `wrong client`, `rate-limited`), and `rate-limited` is grey - beside two red tags - as the mockup has it, not as a decision. -- **Refusal rows still in the buffer when the panel stops are not written**; - the refusals themselves were answered. **Decided 2026-10-13 (owner):** - the tailer writes such a row from the refusal's line in `mail.log`; a step - of [audit-2026-10-12](../audit-2026-10-12.md) stage 2b. - -### Accepted, in security.md - -- **The panel's downtime is mail's.** The price of the strict check. -- **What a display name looks like to a person is not promised.** A name of - another sender written without an `@` the scan can find passes, signed - with the application's own key. -- **A malformed packet on the check's socket ends the panel** (the milter - library panics on an empty one). The socket is reachable only inside the - container, by the group Postfix is in; the journal milter always had this, - and what is new is that the price is deferred mail. - -### Not verified - -- **OpenDKIM's reading of a `From:` line** was reasoned about, not run: its - parser is not in this repository. The two rules that do not depend on it - - no other signing name after any `@`, one `@` outside quotes - are what the - claim rests on. A test that feeds the stand's OpenDKIM the lines the check - passes and reads which key it signs with would settle it. **Decided - 2026-10-13 (owner):** that test is built; a step of - [audit-2026-10-12](../audit-2026-10-12.md) stage 2. -- **The pages were not looked at in a browser on the deployed panel**: I - cannot sign in there. They were rendered from the fixtures and - screenshotted, the polling and the phone order read from the code and - exercised by e2e only as far as the Health numbers. -- **No refusal has been seen on the test server itself**, only on the e2e - stand: there is no application there I can send as. -- **A check that stops answering in the middle of a message** is counted - from a warning composed from Postfix's source; the stand only produced the - case of nobody listening. diff --git a/docs/plans/inbound-antispam-panel.md b/docs/plans/inbound-antispam-panel.md deleted file mode 100644 index 358a223..0000000 --- a/docs/plans/inbound-antispam-panel.md +++ /dev/null @@ -1,325 +0,0 @@ -# Plan: inbound-antispam-panel - -**Status:** shipped in `2.1.0` (2026-10-12) -**Date:** 2026-08-19 -**Version:** `2.1.0` MINOR (opt-in; no change to outbound-only or inbound-without-filter paths). Was `1.10.0`; on 2026-10-09 the owner put the panel redesign first, so this ships on the 2.0 baseline and its screens are built from the redesign's component kit ([panel-redesign.md](panel-redesign.md) § Component kit). - ---- - -## Goal - -Give the operator a **panel view of inbound anti-spam decisions** and **editable -allow/deny lists** that tune filter behaviour — without SelfPost shipping or -starting an anti-spam engine ([product.md](../product.md) out-of-scope list -unchanged). - -Typical journal row: - -| When | From | To | Subject | Decision | -|------|------|----|---------|----------| -| 2026-08-19 09:14 | `spammer@evil.example` | `user@inbound.example` | Cheap pills | reject (rspamd: BAYES_SPAM +0.95) | - -Lists let the operator correct false positives/negatives (e.g. always allow -`billing@vendor.example`, block `192.0.2.0/24`). - -## Context (as-built) - -- Inbound relay ships in `[1.4.0]` (`INBOUND_RELAY_ENABLE`); optional - `INBOUND_ANTISPAM_MILTER` attaches an **external** filter (documented rspamd - sidecar in [deploy/antispam/docker-compose.antispam.yml](../../deploy/antispam/docker-compose.antispam.yml)). -- The inbound smtpd has **no journal-milter** today — only outbound 465/587 - writes `send_log` ([architecture.md](../architecture.md)). -- Inbound UI and handlers are **global-administrator only** today - (`requireInbound` → `requireGlobal`). - -## Quarantine - -**Not in this plan.** Held-mail review/release is an idea kept outside the roadmap, -[inbound-quarantine](inbound-quarantine.md). The journal may still -record a `quarantine` **decision label** when the external filter reports it, -without SelfPost storing the message. - -**DMARC `p=quarantine`** remains unrelated (outbound policy for receivers). - -## Decisions (closed 2026-08-19) - -### Capture mechanism - -**Inbound journal-milter in `panel` only** — no rspamd history API polling in v1. -A separate unix socket (`INBOUND_JOURNAL_MILTER_SOCKET`, default -`/run/selfpost/inbound-journal.sock`) on the inbound smtpd only; fail-open -(`default_action=accept`). - -**Milter order on port 25** (when antispam is enabled): - -```text -smtpd_milters = { antispam, inbound-journal } -``` - -Antispam runs first; inbound-journal runs at end-of-message on the **accept** -path and reads rspamd-added headers (`X-Rspamd-Action`, and `X-Spamd-Result` -for the score and symbols - rspamd has no `X-Rspamd-Score`; the sidecar adds -both through `deploy/antispam/rspamd/milter_headers.conf`). As built they are -read only next to rspamd's signature of the message, `X-SelfPost-Filter` -(owner, 2026-10-12: "А rspamd не может дописывать какой-то заголовок, по -которому мы будем знать, что это заголовки нашего спам фильтра?") - an HMAC -of the queue id, since a fixed mark could be written by a sender as easily as -the verdict itself. - -**Antispam rejects** do not reach end-of-message on the journal milter. v1 also -tails `mail.log` for inbound `postfix/smtpd` **`milter-reject`** / **`reject`** -lines and inserts journal rows (client IP, envelope from/to when logged, reason -text, subject empty if rejected before DATA). DMARC-ingest pipe traffic is out -of scope for this tailer. - -### Journal without `INBOUND_ANTISPAM_MILTER` - -Journal stays **on** whenever `INBOUND_RELAY_ENABLE=true` (not tied to the -antispam hook). Without an external filter: - -- **accept** rows — message relayed (engine `selfpost`, detail empty); -- **reject** / **tempfail** rows — from the mail.log tailer (Postfix policy, - unknown recipient, a sender domain that does not exist. As built, a refusal - whose log line names no recipient - the size limit answered at MAIL FROM - - and the per-client rate limit, which Postfix logs as a warning, make no row). - -Panel copy explains that rspamd symbols appear only when the antispam milter is -configured. - -### RBAC - -**Global administrator only** for journal and lists — same as inbound relay. No -domain-admin access in v1. - -*As built (2026-10-11):* the lists are the global role's; the **log follows -the delegation of Inbound** that 2.0 introduced after this was written - a -user with the `domain` role sees the rows of the inbound domains assigned to -them and nothing else, without the *Allow / Deny sender* links. That is what -the accepted `in-log` mockup draws (no "global role" in its head). The owner -was told and can turn it back into a global-only page. - -**Inbound › Log accepted by the owner on 2026-10-12** ("Этот экран -утверждаем"), from screenshots of the page built from its fixture, after his -changes to it: the sender column is headed *From*; *Why* is not a column but -a sheet over the page, opened by the decision tag, holding a centred table of -who decided, the verdict and symbols or the reply, and the subject whole -(kept on the sheet at his word: the table clips it); a message Postfix itself -refused offers no *Deny sender*. Inbound › Filter lists was redrawn at his word -the same day - kind in a column of its own, the note on the entry's sheet, -matches counted (§ Filter list usage) - and **accepted by him on 2026-10-12** -("Да"), after a deny entry he added in the panel refused a test message with -the list's reply and the journal recorded it. - -### Retention - -Mirror send-log retention: - -- Settings key `inbound_spam_log_retention_days` on `/server/settings` (global admin). -- Default **90** days; validation **7–365**; seed from env - `INBOUND_SPAM_LOG_RETENTION_DAYS` when the setting is missing. -- Prune every **6 hours** (same cadence as send-log retention). -- Hard cap **10 000** rows after age prune (drop oldest) — busier than DMARC - reports, lighter than unbounded growth. - -### Allow/deny lists - -| Topic | Decision | -|-------|----------| -| Entry types | sender address, sender domain (`@domain` suffix), client IP / CIDR | -| Precedence | **deny overrides allow** | -| Scope | instance-wide (not per inbound domain in v1) | -| Where applied | **rspamd map files** under `/data/antispam/`, one per list and entry type (`allow_ip.map`, `allow_from.map`, `allow_domain.map`, `deny_…`) - an IP map and a sender map are different map types to rspamd; documented compose fragment mounts them into the sidecar. Deny is a prefilter reject, allow a weight of -100 (as built: two prefilters do not give "deny overrides allow") | -| SelfPost pre-check | **out of v1** — lists live in rspamd only | -| Edit access | global administrator only | -| When inactive | lists editable in the panel but UI warns that sync applies only when `INBOUND_ANTISPAM_MILTER` is set; `postfix reload` / rspamd reload documented in guide | - -List change → atomic map rewrite (same pattern as Postfix maps). No reload: -rspamd rereads a changed map file by itself (about ten seconds on 3.11). - -### Filter list usage (owner, 2026-10-12) - -Each entry shows how often it decided a message (*Matched*, or *never*) and, -on its sheet, when it last did - so that entries nothing matches any more can -be removed. **The count is read off the journal on every view of the page; -nothing is stored for it.** A journal row counts for an entry when the list -decided the message - a rejected row whose reply is the deny rule's message, -an accepted row whose verdict carries the allow rule's symbol -(`SELFPOST_ALLOW_IP` / `_FROM` / `_DOMAIN`, which the journal milter always -keeps) - and the entry fits the row's sender or client address the way rspamd -matches it (`inbound.Lists.Usage`). - -What follows from counting this way, and is accepted: - -- the count reaches back as far as the journal does: its retention (90 days by - default), which is what the sheet says; -- a row counts for every fitting entry of the deciding list, since the journal - does not say which of two the filter met first; -- a message refused at end-of-message is one row whatever its recipient count, - so a count is of messages, roughly. - -**When to move to a counter kept with each entry** (a migration adding hits -and last-hit to `inbound_spam_list`, bumped where the journal row is written; -the owner asked for the condition to be written down). Any one of: - -1. **The journal is cut by its 10 000-row cap and not by age.** Then it holds - fewer days than its retention, the sheet's "in the last 90 days" is no - longer true, and *never* stops being evidence that an entry is stale. The - sign: the log tailer's prune line keeps reporting rows removed while the - oldest row of Inbound › Log is younger than the retention. -2. **The retention is set short** (towards its minimum of 7 days) for reasons - of its own, privacy or disk. A week without a match does not make an entry - stale. -3. **Counts are wanted that outlive the journal** - since the entry was added. -4. **The page is slow.** A view reads every journal row a list decided and - tries it against every entry. With the cap that is at most 10 000 rows - times the number of entries; past a few hundred entries, measure it. - -Until then no migration is spent on it. - -### Activation - -| Condition | Behaviour | -|-----------|-----------| -| `INBOUND_RELAY_ENABLE=false` | no inbound journal milter, no UI, no tables written | -| inbound on, antispam empty | journal + UI; lists visible but marked inactive for filter sync | -| inbound on + antispam set | full journal (symbols on accept) + list sync | - -No separate feature flag beyond inbound relay + existing antispam env vars. - -### UI - -The journal is **Inbound › Log** and list management is a new **Inbound › -Filter lists** page (global-only: the lists are instance-wide) — see -[panel-redesign.md](panel-redesign.md) § Room for roadmap items. The journal -table has filters (decision, domain, date range). Mockups are drawn first, in -`docs/assets/panel-redesign/panel/src/` under that plan's design contract — -not in the superseded `docs/assets/panel-ui/` (removed 2026-10-16). - -## Schema (migration `0002_inbound_spam_log.sql`, the first after the 2.0 baseline) - -**`inbound_spam_log`** - -| Column | Type | Notes | -|--------|------|-------| -| `id` | INTEGER PK | | -| `inbound_domain` | TEXT NOT NULL | recipient domain (from `relay_domains`) | -| `client_ip` | TEXT NOT NULL | | -| `from_addr` | TEXT NOT NULL | envelope from | -| `to_addr` | TEXT NOT NULL | envelope to (one row per recipient) | -| `subject` | TEXT NOT NULL | truncated with `mailhdr.SubjectMaxRunes` (200) | -| `decision` | TEXT NOT NULL | `accept`, `reject`, `tempfail`, `quarantine` | -| `engine` | TEXT NOT NULL | `rspamd`, `postfix`, `selfpost` | -| `detail` | TEXT NOT NULL | symbol/score summary or log-line reason; may be empty | -| `created_at` | TEXT NOT NULL | RFC3339 UTC | - -Index on `(created_at)`, `(inbound_domain, created_at)`. - -**`inbound_spam_list`** - -| Column | Type | Notes | -|--------|------|-------| -| `id` | INTEGER PK | | -| `list_type` | TEXT NOT NULL | `allow` or `deny` | -| `entry_type` | TEXT NOT NULL | `address`, `domain`, `ip` | -| `value` | TEXT NOT NULL | normalized entry | -| `note` | TEXT NOT NULL | operator comment; default `''` | -| `created_at` | TEXT NOT NULL | | - -Unique on `(list_type, entry_type, value)`. - -## Scope - -**In:** - -- Schema, store CRUD, retention loop, mail.log tailer extension for inbound - rejects. -- Inbound journal-milter (fail-open) + Postfix `postfix-config.sh` wiring. -- rspamd map sync under `/data/antispam/`. -- Panel journal + list UI (global admin). -- Full backup includes `/data/antispam/` and new SQLite tables. -- Tests; [guide.md](../guide.md); [architecture.md](../architecture.md). - -**Out:** - -- Shipping rspamd, ClamAV, or any filter binary inside the SelfPost image. -- Outbound spam filtering. -- [inbound-quarantine.md](inbound-quarantine.md) storage/release. -- rspamd rule editing, Bayes training UI, antivirus. -- MIME replay from the panel (metadata only). -- Per-domain lists; domain-admin RBAC. -- rspamd history API polling. - -## Architecture - -```mermaid -flowchart LR - internet["Internet :25"] --> inbound["Postfix inbound smtpd"] - inbound --> antispam["External antispam milter optional"] - antispam --> journalM["inbound-journal milter fail-open"] - journalM --> upstream["smtp upstream transport"] - journalM --> sqlite["inbound_spam_log"] - mailog["mail.log tailer"] --> sqlite - listsUI["Panel allow/deny lists"] --> maps["/data/antispam/*.map"] - maps --> antispam - sqlite --> journalUI["Panel inbound spam journal"] -``` - -## Security - -- Same validation whitelists as inbound relay ([security.md](../security.md)). -- Journal rows contain PII — retention, and access by reach: the global role - reads all of it, a `domain` user the rows of assigned inbound domains (as - built; see § RBAC). -- Inbound journal milter fail-open must not block mail (backup-MX role). -- List map writes injection-safe; deny wins over allow. - -## Done when - -- With inbound relay + antispam sidecar, panel shows accept rows with symbols - and reject rows from mail.log tailing. -- Operator can add/remove list entries; a test message reflects allow/deny. -- With antispam hook off, journal still shows Postfix-level decisions; list UI - shows inactive sync notice. -- `go vet`, `go test`, image build green; guide and architecture updated. - -## Risks - -- rspamd header/symbol format drift — thin parser, pin sidecar tag in compose. -- Journal volume — retention + 10k cap. -- mail.log parse brittleness — unit tests on sample lines; inbound smtpd only. - -## Dependencies - -- Inbound relay, shipped in `[1.4.0]` — as built in - [architecture.md](../architecture.md) § Mail path. Its plan file was removed - once it shipped. - -## Model routing (this plan) - -| Role | Model | -|------|-------| -| All implementation (code, tests, docs) | **Composer** | -| Technical review (reviewer ≠ author) | **Opus** | -| Security review | **Fable** | - -Composer ships each step; Opus reviews the accumulated diff once implementation -steps are done (fix loop: Composer addresses Opus findings, Opus re-checks). -Fable runs after Opus sign-off. - -## Implementation checklist - -Target version cut: **`2.1.0`** (MINOR), after `2.0.0`. One commit per step; -[development.md](../development.md) § Plan checklists. - -- [x] Agree journal fields, retention, and RBAC (this plan § Decisions) — **Composer** -- [x] Migration `0002_inbound_spam_log.sql` on the 2.0 baseline — **Composer** -- [x] Inbound journal-milter + Postfix wiring — **Composer** -- [x] mail.log tailer: inbound reject rows — **Composer** -- [x] List CRUD + validation + atomic rspamd map sync — **Composer** -- [x] Panel: Inbound › Log and Inbound › Filter lists from the kit partials, mockups `in-log` / `in-filter-lists` (add client IP and engine columns mockup-first), evidence pairs — **Composer** -- [x] Backup includes `/data/antispam/` — **Composer** (it always did: a backup walks all of `/data`; now held by a test, and the guide says the `antispam/` fragment directory is not in it) -- [x] Unit + handler tests — **Composer** (written with each step: store, milter, tailer, lists and their usage, view fixtures held to the outlines, handlers for both roles) -- [x] [guide.md](../guide.md), [architecture.md](../architecture.md) — **Composer** (kept up with each step; [security.md](../security.md) follows the security review) -- [x] `go vet`, `go test`, e2e if applicable — **Composer** (on the test server for every step; last on `7192c38`: vet ok, 22 packages ok, e2e ok, which relays, refuses, reads the log page and edits a list through the panel) -- [x] Technical review of inbound antispam changes — **Opus** (2026-10-12; findings fixed: client address of a connection's later messages, the lists' writer under a lock, the tailer's queue-id memory, hold rows written twice, the cap by time; and, outside the feature, the level-2 limit and client-address list skipped after a connection's first message) -- [x] Security review inbound antispam path — **Fable** (2026-10-12; no Critical or High; HELO-forged journal rows and the six-hourly row cap fixed; a sender's own verdict while the filter is down closed by the filter's signature, the owner's idea; accepted risks recorded in [security.md](../security.md)) diff --git a/docs/plans/inbound-quarantine.md b/docs/plans/inbound-quarantine.md index 3b29123..dd2fbde 100644 --- a/docs/plans/inbound-quarantine.md +++ b/docs/plans/inbound-quarantine.md @@ -1,8 +1,8 @@ # Plan: inbound-quarantine **Status:** an idea to think about, not on the roadmap (owner, 2026-10-12) — design TBD; needs explicit agreement before any code -**Version:** TBD (`1.x` MINOR if opt-in and backward-compatible; may warrant `2.x` -if it changes inbound storage or backup contract). +**Version:** TBD (a 2.x MINOR if opt-in; a major only if it breaks the +inbound storage or backup contract). --- @@ -17,8 +17,8 @@ so we can compare approaches before committing. ## Why it is separate from inbound-antispam-panel -[inbound-antispam-panel.md](inbound-antispam-panel.md) covers **metadata** -(journal + lists). Quarantine implies **storing message bodies** (or a durable +inbound-antispam-panel (shipped `2.1.0`; [architecture.md](../architecture.md) +§ Mail path) covers **metadata** (journal + lists). Quarantine implies **storing message bodies** (or a durable reference to them) under `/data`, retention, release workflow, and a wider attack/backup surface — closer to "mini-mailbox" than pure relay. @@ -68,9 +68,7 @@ to the upstream, 14 days, headers only in the panel); it does not settle them. - Inbound relay, shipped in `[1.4.0]` — as built in [architecture.md](../architecture.md) § Mail path. -- Sensible to decide **after** or **alongside** - [inbound-antispam-panel.md](inbound-antispam-panel.md) (journal/lists), but - not blocked on it for design discussion. +- Builds on inbound-antispam-panel (journal and lists, shipped `2.1.0`). ## Done when (placeholder) diff --git a/docs/plans/inbound-upstream-result.md b/docs/plans/inbound-upstream-result.md deleted file mode 100644 index f0aef6e..0000000 --- a/docs/plans/inbound-upstream-result.md +++ /dev/null @@ -1,100 +0,0 @@ -# inbound-upstream-result - -**Status:** shipped `2.10.0` (2026-10-16). - -## Problem - -The owner, 2026-10-16: "Нам не хватает в журнале для Inbound ответа -upstream. Не понятна судьба письма: принято, отклонено, хост не доступен и -пр." - -Inbound › Log shows what port 25 decided - accept, reject, tempfail - and -nothing after it. Its help says so ("Accept says what this server did, not -what the upstream did with the message afterwards"). The answer is in -`mail.log`: Postfix's `smtp` client logs every attempt to hand a message to -its upstream as `: to=<…>, relay=[ip]:port, …, dsn=…, status=sent -(250 …)` - or `deferred (451 4.7.1 Greylisted …)`, `deferred (connect to …: -Connection refused)` with `relay=none`, `bounced (… said: 550 …)` (Postfix -3.10.13, `src/global/log_adhoc.c`; `relay=none` when there was no session, -`src/smtp/smtp_trouble.c:273, 421`). The tailer reads those lines for the -outbound log and, finding no send-log row for a port-25 queue id, drops -them; the journal row does not even hold the queue id. - -## Decision - -The owner, asked whether the result goes in the row or on a page of its own -per message: "Пока только результат. Отдельная страница в план в случае -появления потребности." So: the row carries the result and the last reply; -no stored `mail.log` lines for port-25 mail (delivery-log-storage left them -out on purpose: a stranger sets the rate). A page per inbound message, with -every line, is for later, if the result is not enough. - -## Design - -- **Schema, migration 0012:** `inbound_spam_log` gains `queue_id`, - `upstream_status` (`''`, `queued`, `sent`, `deferred`, `bounced`), - `upstream_relay`, `upstream_reply` and `upstream_at`; an index on - `queue_id` for rows that have one. Rows written before have none and show - "—". -- **The milter** writes an accepted row with the queue id it already reads - at end-of-message (`{i}`, for the filter's signature) and `queued`. The - row is in the database before the message is queued, so before any - delivery line. -- **The tailer:** a delivery line whose queue id has no send-log row is - offered to the journal: the rows of that queue id and recipient take the - status, the relay (empty for `none`), the reply (printable, at most 255 - bytes) and the line's time. One update per attempt, no new row; the - journal's caps stand. qmgr's `status=expired, returned to sender` makes a - queued or deferred row `bounced`, its reply saying it expired after the - last one. -- **The page:** an Upstream column after Decision (mockup `5348330`): a - plain `queued` tag, or a tag opening an "Upstream" sheet - host, reply, - when, and what follows for deferred and bounced. Refused rows "—". -- **Docs:** help (word for word with the mockup), guide § Inbound log, - architecture (journal and tailer), security (what is stored from the - upstream and at what rate), CHANGELOG. - -## Reviews (2026-10-16) - -A technical and a security review of the tree before the commit; what each -finding became: - -- *A quoted RCPT TO* (both): the milter kept `"owner"@x` as given, every line - says `owner@x` - the row never matched. Now `recipientAsLogged`, as the - outbound milter does. -- *A forged line through a quoted local part* (security, medium): `to=<…>` - was read up to its first `>`; a local part in quotes may hold `>`, `,`, - `status=`. Port 25 refuses the one that would reach another row (an `@` in - the local part is a routed address: `src/trivial-rewrite/resolve.c:467`, - `src/smtpd/smtpd_check.c:1724`, `allow_untrusted_routing=no`), and the - pattern now reads addresses by Postfix's quoting all the same - (`loggedAddress`). -- *Held mail* (technical): a message the filter had held showed `queued`, - "not tried yet", for as long as it stayed held. Now `held`, a status of its - own (the CHECK of 0012, not shipped yet). -- *Rows that never close, and queue ids that repeat* (both): before writing - a message's rows the milter detaches older rows of its queue id; cleanup's - `removed (canceled)` detaches rows of a message never queued; an empty - `{i}` writes no result. -- *A bounce of null-sender mail* (technical): "The sender was sent a - bounce" is wrong there - Postfix does not bounce a bounce. The sheet says - so. -- Accepted, said in security.md: a row whose insert waited past the milter's - 30 s timeout can miss its first line and stay `queued`. Not of this row: - an old send-log row of a reused queue id and the same recipient takes the - line first (sendlog.go, as before). - -## Steps - -- [x] Migration 0012, store: write the queue id, update by queue id and - recipient, read the result back. -- [x] Milter: the queue id and `queued` on accepted rows. -- [x] Tailer: delivery and expiry lines of port-25 mail to the journal. -- [x] View, template, help; tests (store, milter, tailer, view). -- [x] e2e: a relayed message's row shows `sent` with the sink's `250`; an - upstream that refuses connections shows `deferred` with the reason. -- [x] Docs. *Done (2026-10-16), with the reviews' findings above.* -- [x] Roll out, read the server's journal; cut `2.10.0`. *Done (2026-10-16): on the - test server as 2.9.1-post.d515493, a message to selfpost@dev.mixdep.ru through - port 25 - its row sent, mc-ent.mixfed.ru's "250 2.0.0 Ok: queued as - 04F0D1023EF" on it.* diff --git a/docs/plans/panel-dkim-signing.md b/docs/plans/panel-dkim-signing.md deleted file mode 100644 index 635bada..0000000 --- a/docs/plans/panel-dkim-signing.md +++ /dev/null @@ -1,257 +0,0 @@ -# Plan: panel-dkim-signing - -**Status:** shipped `2.8.0` (2026-10-16); agreed 2026-10-15 - the row is agreed (owner, 2026-10-15, the -order port25-bare-newline -> base-trixie -> panel-dkim-signing); three -questions below were settled one at a time the same day. -**Date:** 2026-10-15 -**Version:** `2.8.0` MINOR (after base-trixie, shipped `2.7.0`). No migration; -nothing for an operator to do. - ---- - -## Goal - -The panel signs outgoing mail itself, in its own milter, and OpenDKIM -leaves the image. The analysis ([own-mta-builds](../own-mta-builds-2026-10-15.md) -§ 3 E) and the roadmap section give the reasons: OpenDKIM is a 2018 beta -with three open 2026 CVEs and nobody upstream; every OpenDKIM limit -SelfPost works around goes with it (the syslog relay, the From: parser -port and its 256-byte refusal, the tables reloaded by a cross-uid signal -and `CAP_KILL`, keys shared through a group). - -What stays the same for an operator: the keys, the selectors, the DNS -records, the panel's pages (one label on Health excepted, Q3). - -## What the signer does - -- **Library:** `github.com/emersion/go-msgauth/dkim` v0.7.0 (MIT; the - author of the go-milter v0.4.1 we run; the e2e suite verifies with its - v0.6.8 today). Read for this plan, `dkim/sign.go`: `NewSigner` hashes the - body as it is written (an `io.Pipe` into a goroutine), so a message is - not buffered; `HeaderKeys` is written to `h=` in the order given, each - name picked last instance first (`header.go`, `newHeaderPicker`), and a - name listed once more than it occurs is kept in `h=` with nothing hashed - for it - the over-signing RFC 6376 § 5.4.2 describes; RSA-SHA256; the - signature is the caller's to insert. -- **What is signed:** `relaxed/simple`, as `build/opendkim.conf` signs - today. `h=`: every header of the message whose name is in OpenDKIM's - `dkim_should_signhdrs[]` (`libopendkim/dkim.c:223-246`: from, reply-to, - subject, date, to, cc, the five resent-*, in-reply-to, references, the - seven list-*), each occurrence, in message order, plus `From` once more - (`OversignHeaders From` today). Absent names are not listed, as - OpenDKIM does not list them. -- **Which key:** on 465/587 the key of the domain the SASL login belongs - to - `{auth_authen}`, then `Bindings.DomainOf(login)`, the table the - sender check already reads (`internal/milter/sender.go:262-269`). For the - panel's own notifications (`sendmail(1)`, `non_smtpd_milters`) the key of - the server's name, as `EnsureServerSigned` provides it today. The sender - check has already held `From:` to that same domain, so the signature's - `d=` is the `From:` domain DMARC aligns on. -- **What Postfix hands a milter,** read in Postfix 3.10.13, - `src/milter/milter8.c:2329-2400`, `milter8_header()`: Postfix's own - `Received:` is not shown (and an inserted header at index 0 lands above - it, "relative to the message as delivered"); a multi-line header comes - as text separated by `\n`; one space after the colon is eaten unless the - milter asks for `SMFIP_HDR_LEADSPC` (`OptHeaderLeadingSpace`, which the - sender check already asks for); body records end in `\r\n` - (`milter8_body()`, :2470-2500). The signer rebuilds each header as - `Name:` + value with `\n` made `\r\n` - go-msgauth's own - `cmd/dkim-milter` writes `name + ": " + value + "\r\n"` and would sign a - folded header wrongly. A unit test signs folded, tab-led and - space-free headers and an independent verifier checks them. -- **When it cannot sign** (no key file, a login with no domain, a read - error): temporary failure, `default_action=tempfail`, as OpenDKIM is - attached today. A message is never sent unsigned by accident. -- **8-bit mail:** `disable_mime_output_conversion=yes`. Postfix converts an - 8-bit message to 7-bit for a server that does not announce 8BITMIME - (default `no`, `src/global/mail_params.h:3088`), which breaks any body - hash - OpenDKIM's today as well. Not set until now; set with this row. - -## Decisions settled before this plan - -From the roadmap section and the analysis, agreed with the row: the -library; `relaxed/simple`; the header set and `From` over-signed; the key -by the login's domain and by the server's name for local mail; strict; -the removals below; keys, selectors and records unchanged; the e2e oracle -an independent verifier; a message to a real mailbox before the cut. - -Decided here (technical, the owner may overrule): - -- **The oracle is dkimpy** (`python3-dkim` 1.1.8-2 in trixie, - `/usr/bin/dkimverify`, seen in `debian:trixie` 2026-10-15) - a separate - implementation in another language, maintained, packaged by Debian; not - OpenDKIM's `opendkim-testmsg` (the code this row removes) and not - go-msgauth (the signer's own library). It runs in the e2e sink's - network against the stand's CoreDNS. -- **A cross-check before OpenDKIM goes:** on the stand of stage 1 both - sign the same messages for a while (OpenDKIM after the panel in the - chain) and the two signatures' `bh=` and `h=` are compared on every - e2e message; they must agree before stage 2 removes OpenDKIM. -- **The From: parser port goes.** It exists so that the sender check - judges the address OpenDKIM chooses the key by. With the key chosen by - the login, the check keeps `net/mail` (one RFC 5322 address at an - allowed domain) and the 256-byte refusal goes; the refusal texts that - name OpenDKIM are rewritten. - -## Open questions (to the owner, one at a time) - -- **Q1. Where the signer runs.** (a) Its own role in the panel process on - two sockets of its own: `sign.sock` for 465/587 (the key by the login) - and `sign-local.sock` for local submission (the server's key only), in - the chain where OpenDKIM is now: sender check, signer, journal. (b) - Inside the sender check's milter: one socket fewer, but the check would - receive every body, and local mail - which has no login - would need the - same milter to tell "no login" from "local", deciding a key by the - absence of something. **Recommended: (a)** - the socket a message comes - through, not a missing login, decides that the server's key may sign - it; the sender check stays a header-only milter; and moving the signer - into a process of its own later is a change of where it is started. - **Decided (owner, 2026-10-15): (a).** -- **Q2. The key directory's name.** `/data/opendkim/keys//.private` - stays as it is, or becomes `/data/dkim/...` (the entrypoint moves the - tree once; KeyTable and SigningTable are deleted). A backup restores - only into the image of its own version, so no older archive meets the - new name. **Recommended: rename** - the cost is the entrypoint's move, - the paths in the code and the backup tests; a directory named after a - program the image no longer has would mislead whoever reads `/data`. - **Decided (owner, 2026-10-15): rename to `/data/dkim`.** -- **Q3. Health.** The Overview's socket check and the process list name - OpenDKIM and the syslog relay. **Recommended:** the socket row becomes - "DKIM signer" (the panel's `sign.sock`, checked the same way), the two - programs leave the process list. A mockup first, as for any page - change. - **Settled 2026-10-15:** shown to the owner as a picture; it follows from - the card's own rule (a row per milter socket Postfix needs, and the - signer is strict), so it is not his choice - the two rows as drawn. - -## Stages - -### Stage 1 - the signer, beside OpenDKIM (no release) - done `14de9a0` - -- [x] `internal/milter/sign.go`: the session (Connect, MailFrom with - `{auth_authen}`, Header, Headers, BodyChunk, Body), `OptHeaderLeadingSpace`, - `InsertHeader(0, "DKIM-Signature", ...)` (the server's `Actions` gain - add-header), the key read per message from the key file (a small cache - keyed by path and mtime), the selector from the store as `Record` reads - it. -- [x] Unit tests: sign and verify with go-msgauth; folded and tab-led - headers; a header absent and present twice; an empty body; a body - without a final newline; a login with no domain and a missing key file - answer tempfail; the server's socket never signs with a domain's key. -- [x] `cmd/panel`: the role, its two sockets (`listenMilter`, group - selfpost, 0660), the wrapper waits for them. -- [x] `build/postfix-config.sh`: `smtpd_milters` = sender, sign, OpenDKIM, - journal (OpenDKIM kept for the cross-check); `non_smtpd_milters` = - sign-local, OpenDKIM; `disable_mime_output_conversion=yes`. -- [x] e2e: dkimpy in the stand; every positive send and the notification - verified by dkimpy against the panel's signature; the panel's and - OpenDKIM's `bh=`/`h=` compared. - -*Done (2026-10-15, `14de9a0`):* e2e exit 0 on the server; on an -application's message and on a notification the panel's and OpenDKIM's -signatures agree on `h=`, `bh=`, `c=`, `s=`, `a=` and dkimpy verifies both. -Two findings while writing it: a folded header reaches a milter -` -`-separated (the signer rebuilds CRLF), and go-milter calls no Abort on a -dropped connection (a one-hour watchdog closes the signer). - -### Stage 2 - OpenDKIM removed (no release) - done - -- [x] Chain: OpenDKIM out of both milter lists. -- [x] Go: `internal/domain/opendkim.go` reduced to key storage (tables, - `Rebuild`, `assertConfigSafe`, the USR1 reload go), `internal/supervisor` - (its only caller is the reload), `internal/syslogrelay`, - `internal/milter/mailparse.go` and its tests and testdata, the - sender-check texts; Resync's "OpenDKIM's tables" step; Health's - liveness set and socket row (Q3); `OPENDKIM_SOCKET`/`OPENDKIM_DIR`. -- [x] Image: the `opendkim` package, `opendkim.conf`, the two supervisord - programs, the `/dev/log` link and `/run/opendkim`, `usermod -aG selfpost - opendkim`, crashexit's spared list; `deploy/docker-compose.yml`'s `KILL` - capability (its only use was the reload) - **kept, see below**. -- [x] Keys: `/data/opendkim/keys` -> `/data/dkim/keys` (Q2): the entrypoint - copies the tree once and leaves the old one for a rollback to 2.7.x until - the next release; KeyTable and SigningTable go with the new layout. - *2.9.0:* the entrypoint removes the old tree once `/data/dkim` holds a - key, and stops on a `/data` with keys under the old tree only; the copy - and the backup exclusion are gone. -- [x] Tests: `test/upstream/*.sh`, e2e's OpenDKIM log and process checks, - the 256-byte test rewritten for the new rule. - -*Done (2026-10-16):* `internal/domain/keys.go` (`Keys`, `NewKeys`, -`DKIM_DIR`) keeps the keys and writes a selector table, -`/data/dkim/selectors`, a line ` ` each, which the signer -reads instead of the KeyTable: the signer reads no database, as the sender -check reads none - the panel's one SQLite connection is shared with the -pages, and a long query would hold up mail. The table is written before the -signer serves (`EnsureServerSigned`, moved there from the notifier), so the -first start of 2.8.0 needs nothing of the operator. Keys and table are -0600, directories 0700, owned by the panel; the entrypoint copies the old -keys as the panel user (`setpriv`), into `/data/dkim.new`, renamed when -whole, and stops the start if the copy fails - else the server's name would -get a new key. Three findings: `KILL` stays - measured on the test server, -root without it cannot signal a process of another uid, and supervisord -stops the panel and Postfix's master its children that way (the plan's -"only use" was wrong); the sender check's `Allowed` took the domain through -`strings.ToLower`, which makes the Kelvin sign a "k" - the comparison with -OpenDKIM's reading had hidden it, it now folds ASCII only; the Milter -sockets card told the spam filter by its row having a note, which the -signer's rows have too. The e2e drops the OpenDKIM-only cases -(`testNeverSignedForAnother`) and checks the one signature, the tree's -modes and the cited packages without opendkim, libopendkim11 and -libmilter1.0.1. - -### Stage 3 - documents, the canary, the cut - -- [x] architecture § OpenDKIM -> § DKIM signing; security.md (the - mail-path bullets, the accepted risks that were OpenDKIM's); guide; - development's cited-package table (opendkim and libopendkim11 leave); - the assumptions register's header notes what left; help (design commit - first) and Health's label (Q3, mockup first). -- [x] Reviews: technical and security. - -*Done (2026-10-16):* help and Health went with stage 2 (design `005922b`); -the reviews ran on stages 1-2 and their findings are fixed in `c2f63d2`, -with the first rollout's incident (the queue's owner numbers); the -documents in the commit after it. `Sender:` is held to the login like -`From:` (security review F1; the owner's decision, 2026-10-16). go-milter -v0.4.1 kept every header in a map of its own per session, on 465/587 and on -port 25 alike: the panel builds it patched not to -(`third_party/go-milter/PATCHES.md`; the owner's decision, 2026-10-16). -- [x] On the test server: a day of the owner's mail; one message from an - application through 465 and one notification to a real mailbox, the - headers read (dkim=pass with the panel's `d=`). -- [x] Cut `2.8.0`. - -*Done (2026-10-16):* the owner's message through 465 (`dev/send-check.ps1`) -arrived at Gmail with `dkim=pass header.i=@carrel.mixfed.ru -header.s=selfpost`, `spf=pass`, `dmarc=pass`, and at Yandex with -`dkim=pass` and `X-Yandex-Spam: 1`; its `h=` over-signs the shown headers. -His `dev/from-check.ps1` sent thirteen `From:`/`Sender:` cases: four -accepted and delivered, nine refused as expected, each a send-log row of -reason `from` or `sender`. The notification to a real mailbox was not -sent - nothing raised one; its signature is verified by dkimpy on every e2e -run, and the server name's key, unchanged by the move (digests compared), -matched its published record on 2026-10-15. The day of mail was cut short -at the owner's word ("Выпускай", 2026-10-16), the test server having run -the stage's code since 2026-10-15 22:48 UTC. Cut as `2.8.0`, no tag. - -## Risks - -- **A signing defect fails DMARC for every tenant at once.** The answer is - the oracle (another implementation), the cross-check against OpenDKIM on - the same messages before it goes, and the canary before the cut. -- **go-msgauth is small and released slowly.** It can be vendored if it - ever matters; the signing path is about 300 lines of it. -- **Rollback:** until stage 2 ships, the chain change is one line; after - the cut, the previous image (2.7.x) still has OpenDKIM and reads the keys - from the old path, which the move leaves as a copy until the next release - - the keys as they were at the upgrade: a domain added under 2.8.x is not - in that tree, and 2.7.x sends its mail unsigned; one re-added under 2.8.x - has another key there. 2.7.x reads the queue as its own postfix (uid - 101) before it hands it over, and 2.8.0 left it to uid 100: the guide - (§ Upgrading) gives the hand-over by name to run in the 2.7.x image first, - measured on a copy of the test server's queue. Going forward again after - a rollback, a domain added under 2.7.x has its key only in the old tree: - copy its directory into `/data/dkim/keys` (removing `/data/dkim` instead - copies the old tree again and loses the keys of domains added under - 2.8.x). diff --git a/docs/plans/panel-notifications.md b/docs/plans/panel-notifications.md deleted file mode 100644 index 907c21a..0000000 --- a/docs/plans/panel-notifications.md +++ /dev/null @@ -1,119 +0,0 @@ -# Plan: panel-notifications - -**Status:** shipped in `2.3.0` (2026-10-12); agreed the same day - the design questions were closed with the owner and he took it into work ("Делаем") -**Date:** 2026-10-12 -**Version:** `2.3.0` MINOR (one migration; opt-out by a switch). - ---- - -## Goal - -The panel writes to its administrators when something needs them, instead of -waiting to be opened. - -## Decisions (owner, 2026-10-12) - -### Events of the first version - -Four, each of them something the panel can already tell and a person has to -act on: - -| Event | Who is told | -|---|---| -| A sending domain's DNS record stopped matching (SPF, DKIM, DMARC) | global users, and `domain` users the domain is assigned to | -| A level-2 rate limit refused mail | the same, for the domain of the limit | -| The TLS certificate is about to expire | global users | -| A message has been in the queue for more than **three days** | global users | - -Three days, not one (owner: "one day is not that much"): Postfix keeps a -message five, so two are left to act before it bounces. The threshold is a -constant, not a setting. - -Left for later: a process or milter socket down (a relay in that state often -cannot send), an inbound upstream unreachable, DMARC failures appearing -(noisy). Mail held in quarantine is not an event: quarantine is an idea -outside the roadmap. - -### Sender and path - -From `postmaster@`, submitted locally. Not from one of the -sending domains (owner): with several domains the choice is arbitrary, and a -message about the certificate has no business coming from a domain. - -These messages have to arrive (owner: "we need the best deliverability for -them"), so the server's own name gets what a sending domain has: - -- a **DKIM key for the hostname**, generated by the panel, with which - notifications are signed - no domain has to be chosen for it. An SPF record - alone would pass DMARC, but not after the user's mailbox forwards the - message on, and large receivers treat unsigned mail more strictly; -- an **SPF record** for the hostname, `v=spf1 a -all`. - -The records themselves are under **Server › Settings** (owner: "the DNS -records go to Settings, Health has only the status"), in a box of their own -drawn with the record component of a domain's DNS page: host, value to -publish, copy buttons, state. **Server › Health** has a card of its own, -*Notification mail*, with the state of each record and a link to Settings. -A DMARC record for the hostname is not asked for: the name inherits its -parent domain's policy, and with SPF and DKIM aligned it passes any. -Without the records notifications are still sent, and the card warns. - -Notifications are the panel's own mail: they are not rows of the Outbound log -and count against no level-2 limit. - -A relay that cannot send cannot say so; none of the four events is "the relay -is down", and that is why. - -### Who chooses what - -- One switch for the instance under **Server › Settings**. -- One checkbox per user under **Account**, "send me notifications" - on when - the profile has an e-mail. No choice per event: four events, none of them - noise. Per-event choice comes when there are more and some are. -- A user without an e-mail gets nothing. - -### When a message is sent - -A background check once an hour for DNS, certificate and queue. - -| Event | Sent | "Cleared" message | -|---|---|---| -| DNS mismatch | after two checks in a row found it (it has held for an hour). A resolver that did not answer is not a mismatch | after two good checks in a row | -| Certificate | 14 days before expiry, and again 3 days before if it is still the same certificate | none | -| Level-2 limit refused mail | at the first refusal of a sending domain's applications, then at most once a day for that domain (one subject per domain, as built: what the refused row says of its domain is the client's own word, so the registry is asked) | none | -| Queue older than three days | when such a message appears | one, when none is left | - -Event state is kept in the database, so a restart announces nothing a second -time. - -## To settle while building - -- How the panel submits: `sendmail` as the unprivileged panel user, and what - that needs of the image. -- Whether a notification carries a link to the panel - there is no setting - for the panel's own address. -- Where the refusal of a level-2 limit is noticed: in the journal milter as - it refuses, or from the `rejected` rows of the send log on the hourly check. -- Schema: the user's checkbox, the instance switch, the event state table. - -## Screens - -Not drawn. The Notifications box was taken out of the `account` and -`settings` mockups on 2026-10-10 (owner: 2.0 ships no box for a feature that -is not built). Drawing them again - one checkbox on Account, one switch on -Server › Settings, the SPF check on Server › Health - is the first step, shown -to the owner before any code ([panel-redesign.md](panel-redesign.md) -§ Enforcement). - -## Implementation checklist - -- [x] Mockups: Account checkbox, Settings switch and records, Health status card, help topic; accepted by the owner 2026-10-12 (`design:` commit) -- [x] Migration and store: user flag, instance switch, event state -- [x] Sending: local submission from `postmaster@`, message text (`sendmail` as the panel user; signed through `non_smtpd_milters` with a key of the server's own name; no link in the text - the panel does not know its own address) -- [x] Background checker: DNS, certificate, queue; the rules of *When a message is sent* -- [x] Level-2 refusal event (from the `rejected` rows of the send log, looked at every minute) -- [x] Panel: Account, Settings, Health card, help -- [x] guide.md, architecture.md, security.md, schema-migrations.md -- [x] `go vet`, `go test`, e2e: a level-2 refusal reaches a mailbox as a signed notification (on the test server: vet ok, 23 packages ok, e2e ok; "once" and "clears" are held by the unit tests of `internal/notify`) -- [x] Technical review — **Opus** (2026-10-12; no Critical. Fixed: refusals counted by the registered domain and only under a limit, with an index of their own; account e-mails may be at public mail hosts; "said" means sent; DNS checks must be 45 minutes apart; the notifier's own mail is not a queue subject and a cut or unreadable listing is no listing; the tables must name the selector the panel shows; an unanswered `a`/`mx` lookup is not an SPF mismatch; a pass stops between domains at shutdown; days are rounded as a person counts. Left as found: a recipient whose copy failed while another's went out is not retried (since 2.5.4 it is: audit 2026-10-12, stage 4); a certificate that lives under 14 days is reported at each renewal) -- [x] Security review — **Fable** (2026-10-12; no Critical. Fixed: the limit notification was built on the domain a client writes in `MAIL FROM` - a mail cannon and a way to write to another tenant's users; a ceiling of 60 messages an hour; [security.md](../security.md) corrected on who can submit locally and on what the account e-mail now is. Recorded as accepted, with a roadmap candidate: an application can have a message signed as the server by writing its name into `From:` - the header is not held against the login, for sending domains either) diff --git a/docs/plans/panel-redesign.md b/docs/plans/panel-redesign.md index 9b8b32e..1b7e7bd 100644 --- a/docs/plans/panel-redesign.md +++ b/docs/plans/panel-redesign.md @@ -1,15 +1,16 @@ # Plan: panel-redesign -**Status:** shipped in `2.0.0` (2026-10-11) -**Date:** 2026-09-21 (replaces the 2026-09-08 givens) -**Version:** `2.0.0` MAJOR — a breaking release by the owner's decision (§ No compatibility). +**Status:** the design contract of the panel; built and shipped in `2.0.0` +(2026-10-11), a breaking release by the owner's decision (§ No compatibility). +Its implementation checklist (27/27) is in the git history. +**Date:** 2026-09-21 (replaces the 2026-09-08 givens) --- ## What this file is -The design contract for the control panel and the plan to implement it. The -design itself is drawn and accepted: +The design contract for the control panel. The design itself is drawn and +accepted: | Artefact | Role | |---|---| @@ -72,9 +73,9 @@ and the migration chain is deleted to zero.** The redesign is `2.0.0`. - **Schema.** `internal/store/migrations/` is emptied and replaced by one baseline, `0001_init.sql`, equal to the 2.0 schema (including the new assignment table below). No historical `admin` table, no create-then-drop. - This is [schema-squash](../roadmap.md#schema-squash) with its trigger now - decided and without the upgrade gate that item planned: 2.0 starts from an - empty data directory. + This was the roadmap item schema-squash, with its trigger decided and + without the upgrade gate that item planned: 2.0 starts from an empty data + directory. - **Docs.** `schema-migrations.md` shrinks to the 2.0 baseline; the 1.x chain stays in git history. @@ -145,8 +146,7 @@ shown another user's login name, and three explicit choices are simpler. - The 2.0 baseline has `users.email`: the user's own address, edited on **Account**, set by the global administrator on the user form and shown in the users list. It is what the panel writes to — event notifications - ([panel-notifications](../roadmap.md#panel-notifications)) and a reset link - ([password-reset](../roadmap.md#password-reset)) — and what the fill button + (`2.3.0`) and a reset link (`2.14.0`) — and what the fill button offers. **Account** is Profile and Password; Server › Settings keeps nothing of DMARC. - Baseline: `domains.dmarc_rua` alone (`''` = no reports). The hosted choice is @@ -226,38 +226,24 @@ shell, and its menu entry is hidden if the page does not exist. ## Room for roadmap items -The mockups draw today's features, the four rows above, and the screens of -the open roadmap items — none of them needed a new component or layout. A -mockup of a feature is its target look, not a decision to build it: each item -keeps its own status and plan. From here on a feature's screens are drawn +The mockups drew the 2.0 pages and the screens of the roadmap items of the +time; none of them needed a new component or layout. A mockup of a feature is +its target look, not a decision to build it. A feature's screens are drawn mockup-first in this system (`panel/src/`, `build.py`, `check.py --write`) as -the first step of that feature's plan — not invented during implementation. +the first step of that feature's work, not invented during implementation. -| Roadmap item | Where it lands | Built from | Drawn | -|---|---|---|---| -| inbound-antispam-panel (agreed, `2.1.0`) | journal = **Inbound › Log** (add client IP and engine to the mockup's columns; `quarantine` joins the decisions); lists = new **Inbound › Filter lists**, global-only because the lists are instance-wide | list layout: add form + two tables, `status_tag`, filter form in `box_head`; the log links *Allow / Deny sender* to the lists | `in-log`, `in-filter-lists` | -| inbound-quarantine (an idea, off the roadmap since 2026-10-12) | new **Inbound › Quarantine**, list + detail with *Release* / *Discard* | list layout; detail like `out-message` with `facts`, `log_pane` for headers only, Release beside a `sp-danger-zone` Discard | `in-quarantine`, `in-quarantine-message` — the open questions of its plan (where mail lives, what release means, retention, RBAC) stay open; the mockup assumes SelfPost-held mail, release to upstream, 14 days, delegated like the rest of Inbound | -| preflight (candidate) | new **Server › Preflight**, next to Health: Health is what runs now, Preflight is the deeper on-demand installation check with a test e-mail form | `postmark` verdict, a table of checks with `status_tag` and a *What to do* column, one form box for the test e-mail | `preflight` | -| panel-notifications (candidate) | one checkbox per user on **Account** (no choice per event, owner 2026-10-12); one switch under **Server › Settings** | a checkbox in a box, `help` | **no** — drawn once, taken out of `account` and `settings` on 2026-10-10 (owner): 2.0 ships no box for a feature that is not built; redrawn as the first step of its plan | -| password-reset (candidate) | a link on sign-in, two signed-out screens (request, set new password) | the signed-out layout | **no** | -| delivery-log-storage (candidate) | the *Delivery log* box of **Outbound › Log › message**, filled from a table instead of a grep of today's `mail.log` | `log_pane`, unchanged | `out-message` | -| csrf-tokens | no screen; every form gains a hidden field — forms are rendered through one helper so the token cannot be forgotten | — | n/a | -| template-data-typing | no screen; the component partials are the natural first users of typed data | — | n/a | -| structured-logging, contributing, review follow-ups | none | — | n/a | +Drawn and not built (`deferredScreens` in the guard tests): + +| Screen | Item | State | +|---|---|---| +| `in-queue` | [queue-table](../roadmap.md#queue-table): the queue as a table from `postqueue -j`, and an Inbound › Queue | undecided; Outbound › Queue shows postqueue's listing in a log pane (`outlineExempt`) | +| `in-quarantine`, `in-quarantine-message` | inbound-quarantine, an idea off the roadmap ([its notes](inbound-quarantine.md)) | its open questions stay open; the mockup assumes SelfPost-held mail, release to upstream, 14 days, delegated like the rest of Inbound | +| `preflight` | preflight as a page of its own, dropped 2026-10-13 ([roadmap](../roadmap.md#dropped-with-the-condition-for-return)); its checks went to Server › Health, the test e-mail to its own form | a leftover of a dropped item | The sibling strip under the navbar holds five or six entries per group without wrapping; Inbound would reach five (Domains · Log · Queue · Quarantine · Filter lists), Server six. Past that a group is split rather than squeezed. -Ordering with 2.0 (owner, 2026-10-09: feature order is not fixed, take -whatever suits development): **the redesign ships first as `2.0.0`**, and -inbound-antispam-panel follows as `2.1.0`, built from the component kit on -the 2.0 baseline. The other way round would have styled two screens twice and -written a 1.x migration only to dissolve it into the baseline. The antispam -back end (journal milter, log tailer, list CRUD, rspamd map sync) touches -nothing under `internal/web/view` and may be built alongside stages 0–1 below -if time allows; its two screens wait for the kit. - ## Three layers 1. **Bulma, as shipped.** Vendored `bulma.min.css`, never edited. Only the @@ -485,60 +471,3 @@ reviewed. - Dark scheme and the narrow-screen shell: Bulma ships both; they are drawn and agreed as a `design:` change before anyone implements them. - Weakening CSP; a JS bundler; a front-end framework. - -## Implementation checklist -Roadmap status: **agreed** (owner, 2026-10-09). Stages are ordered for -development, not by feature priority (owner, 2026-10-09). Stage 1 touches no -file under `internal/web/view` and may run alongside stage 0; stage 2 starts -only when both are done. Every stage-2 step ends with the evidence of -§ Evidence for "done". - -**Decided** - -- [x] Owner decision recorded here: keep today's paths or rename with redirects — **owner** (2026-09-21: rename, § Routes) -- [x] Owner decision: no compatibility, migrations to zero, `2.0.0` — **owner** (2026-09-21, § No compatibility) -- [x] Owner decision: inbound is delegated per user, separately from outbound — **owner** (2026-09-21, § Who sees what) -- [x] Owner decision: the redesign ships before inbound-antispam-panel; feature order follows development convenience — **owner** (2026-10-09, § Room for roadmap items) -- [x] Decision inside 2.0: **Server › Health** is built with the restyle (the tables that leave Overview need a home, § Routes); **queue as a table** and the **inbound queue** stay roadmap decisions — both queue pages ship old content in the new shell, the Inbound › Queue entry hidden — (2026-10-09) - -**Stage 0 — foundation and component kit** (§ Component kit) - -- [x] Vendor Bulma 1.0.4 and the icon subset (MIT) under `/static`, NOTICE and development.md rows, pinned checksum — **Haiku** -- [x] Guard tests from § Enforcement and `TestRoutesFollowNavigation`, red against today's panel where expected, with the `legacyPages` ratchet listing every page; `design-first` CI step including the ratchet and the history-blob check (§ Against resurrected pages) — **Opus** -- [x] `panel.css` from `panel/theme.css` + `shared/brand.css`; `components.html` partials with typed inputs in `components.go`; `status_tag`, `copy_field`, `wbr_at` — **Sonnet** -- [x] `layout.html`: navbar, perforated edge, sibling strip, user menu, footer, visibility flags; the signed-out shell — **Sonnet** -- [x] `GET /server/components` behind the global role rendering every partial from fixtures; kit acceptance (§ Component kit) with evidence — **Sonnet**, reviewed by **Opus** - -**Stage 1 — data and routes** (no template work; parallel with stage 0) - -- [x] Schema from zero: single `0001_init.sql` baseline with `users.email`, `user_inbound_domains`, the two `all_*` flags; store tests; `schema-migrations.md` rewritten — **Opus** -- [x] Inbound delegation: `requireInboundDomain`, lists and log filtered in the query, two-list user form data, per-route 404 tests for both roles including another tenant's id — **Opus** -- [x] Routes renamed per § Routes: new paths, one fragment name, old paths removed, subtree `requireGlobal()` for `/server/`; links in the old templates retargeted so the panel works on every commit — **Opus** -- [x] The application form as one POST; `out-app-created` as the response with `Cache-Control: no-store` — **Opus** -- [x] Server › Health route and handler (tables and *Reload configuration* from `/status`); Overview handler reduced to verdicts — **Opus** - -**Stage 2 — pages, in groups** (each step: partial calls only, page off the ratchet list, evidence pairs) - -- [x] Signed-out pages: login, setup — **Sonnet** -- [x] Overview, Health, and the Account / Settings split — **Sonnet** -- [x] Outbound: domains, domain, domain settings, application form, shown-once password, delete — **Sonnet** -- [x] Outbound: log, message, DMARC hub / domain / report — **Sonnet** -- [x] Inbound: domains, domain with the *Spam filter* box, delete — **Sonnet** -- [x] Server: system log, backup, users, user form, delete; Help as `help_topic` partials, the drawer removed — **Sonnet** -- [x] Queues: old content inside the new shell (outbound), Inbound › Queue hidden — **Sonnet** -- [x] Remove the old `panel.css`, the drawer code in `panel.js` and every class nothing references; `legacyPages` empty, vocabulary test green with no allow-list — **Sonnet** - -**Stage 3 — reviews, docs, 2.0.0** - -- [x] Technical review of the whole against the mockups, evidence pairs complete — **Opus** -- [x] Security review of the route rename, subtree authorization and inbound delegation — **Fable** -- [x] guide.md screenshots and wording for the new navigation; architecture.md § Panel HTTP surface and § Persistence; security.md roles; *Removed* in the CHANGELOG with one line that 2.0 starts from an empty data directory; strike the three UI rows of review-2026-08-followups that the redesign retires — **Sonnet** -- [x] `go vet`, `go test ./...`, e2e suite; version cut `2.0.0` — **Haiku** - -**After 2.0** — features land in the kit, each from its own plan, mockup first -(§ Room for roadmap items): inbound-antispam-panel `2.1.0`; then, in whatever -order suits development, preflight, delivery-log-storage, panel-notifications, -password-reset (two screens to draw first), queue as a table with the inbound -queue, csrf-tokens -(one hidden field in the form partial once security.md is settled). -inbound-quarantine is an idea outside the roadmap. diff --git a/docs/product.md b/docs/product.md index c111161..7e7f68f 100644 --- a/docs/product.md +++ b/docs/product.md @@ -1,6 +1,6 @@ # SelfPost — product boundaries -**What this file is.** Stable product definition for SelfPost v1.0: purpose, +**What this file is.** Stable product definition of SelfPost: purpose, deployment assumptions, explicit out-of-scope items, and the multi-domain model. User-facing overview in [README.md](../README.md); install and operations in [guide.md](guide.md); as-built technical detail in diff --git a/docs/roadmap.md b/docs/roadmap.md index 25476eb..85e8052 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -1,10 +1,12 @@ -# Roadmap: open work (1.x+) +# Roadmap: open work -**Status:** a working tracker of extensions to the v1.0 boundary, each taken up -only after explicit agreement ([product.md](product.md), -[.cursor/rules/agent-rules.mdc](../.cursor/rules/agent-rules.mdc)). Detailed -design lives in [plans/](plans/). Items marked `candidate` need an OK before -any code is written. +**Status:** a working tracker of extensions to the product boundary +([product.md](product.md), +[.cursor/rules/agent-rules.mdc](../.cursor/rules/agent-rules.mdc)), each taken +up only after explicit agreement. Items marked `candidate` need an OK before +any code is written; detailed design, when an item needs it, lives in +[plans/](plans/). A shipped item leaves this file: what it did is in +[CHANGELOG.md](../CHANGELOG.md), its section and plan stay in git history. **Reading this from outside the project:** nothing here is a commitment or a release promise. There are no dates, the order is a recommendation rather than @@ -13,14 +15,12 @@ What the project *will not* do is a separate question, answered in [product.md](product.md) — an item's absence from this file does not mean it is planned but unlisted. -**Versioning:** SemVer MINOR in the **1.x+** line by default (`1.1.0`…), as long -as defaults and migrations stay compatible with `1.0.0`. A major `2.x` only for -an explicit break. One such break, when 2.x is cut for any reason, is -[schema-squash](#schema-squash) — replacing the 1.x SQLite migration chain -with a baseline. That item does not by itself justify a major. +**Versioning:** SemVer in the 2.x line — MINOR for a feature or a migration, +PATCH for a fix. A major only for an explicit break; `2.0.0` was one +([schema-migrations.md](schema-migrations.md) § 2.0 starts from an empty data +directory). -**Process:** [development.md](development.md). The history of closed phases is -in `git log` and [CHANGELOG.md](../CHANGELOG.md). +**Process:** [development.md](development.md). --- @@ -28,65 +28,9 @@ in `git log` and [CHANGELOG.md](../CHANGELOG.md). | ID | Topic | Status | Progress | Plan | |---|---|---|---|---| -| security-audit-2026-10-17 | Follow-ups of the 2026-10-17 security audit: A2 (high), seven mediums, the lows | stages 0-6 shipped `2.14.4` (2026-10-17); stage H (the host, S1-S3, S5) the owner's | 27/29 | [security-audit-2026-10-17.md](security-audit-2026-10-17.md) | -| upstream-tls-verify | Inbound: offer "Verified" TLS to the upstream (Postfix `secure`, the upstream's name checked), beside Required, which checks no certificate (audit M7) | shipped `2.15.0` | — | — | -| server-dmarc-hint | Server › Settings lists a DMARC record for the server's own name beside its DKIM and SPF (audit S3, product side) | shipped `2.15.0` | — | — | -| inbound-upstream-result | Inbound › Log shows what the upstream answered: sent, deferred, bounced, with its last reply | shipped `2.10.0` | 7/7 | [plans/inbound-upstream-result.md](plans/inbound-upstream-result.md) | -| health-setup-checks | Server › Health also checks the certificate's name, Postfix's greeting and the trusted proxy (was preflight) | shipped `2.11.0` | — | — | -| test-email | Send a test message from the panel (was preflight) | shipped `2.12.0` | — | — | -| inbound-antispam-panel | Inbound antispam journal + allow/deny lists | shipped `2.1.0` | 12/12 | [plans/inbound-antispam-panel.md](plans/inbound-antispam-panel.md) | -| origin-strict | Refuse a state-changing request that carries neither `Sec-Fetch-Site` nor `Origin` (was csrf-tokens) | shipped `2.10.4` | — | — | -| backup-reauth | Ask for the account password before a backup or a domain export is downloaded | shipped `2.10.3` | — | — | -| review-2026-08-followups | Remaining findings of the 2026-08-19 code review | closed 2026-10-14 (rows placed in [audit-2026-10-12](audit-2026-10-12.md)) | — | — | -| panel-notifications | E-mail notifications about important events | shipped `2.3.0` | 10/10 | [plans/panel-notifications.md](plans/panel-notifications.md) | -| delivery-log-storage | Keep a message's `mail.log` lines with its send-log row | shipped `2.2.0` | 8/8 | [plans/delivery-log-storage.md](plans/delivery-log-storage.md) | -| password-reset | Password reset by e-mail | shipped `2.14.0` | — | — | -| local-time | The panel shows time, and counts its days, in a time zone other than UTC | shipped `2.13.0` | — | — | -| notification-mail-recheck | The Notification mail records on Server › Settings can be checked again from where they are shown | shipped `2.9.0` | — | — | -| settings-save-row | Server › Settings: the Save button no longer stands alone under a long right column | shipped `2.9.0` | — | — | -| dns-record-collapse | A DNS record that is right is shown folded: its name and its tag, the values on request | shipped `2.9.0` | — | — | -| go-milter-upstream | Prepare an issue and a PR for go-milter: its header map outlives a message and is unbounded | prepared 2026-10-16, the owner posts | — | [third_party/go-milter/PATCHES.md](../third_party/go-milter/PATCHES.md) | -| own-mta-builds | Analysis: build Postfix and OpenDKIM ourselves, patched, instead of Debian's packages | analysed 2026-10-15: no own builds; three rows proposed instead | — | [own-mta-builds-2026-10-15.md](own-mta-builds-2026-10-15.md) | -| port25-bare-newline | The inbound relay's port 25 refuses bare newlines in DATA (`smtpd_forbid_bare_newline=normalize`) | shipped `2.6.1` | — | — | -| base-trixie | The image's base from Debian bookworm (LTS since 2026-07-12) to trixie: Postfix 3.10 with the 2026 security batches | shipped `2.7.0` | — | — | -| panel-dkim-signing | DKIM signing in the panel's own milter; OpenDKIM, the syslog relay and the parser port removed | shipped `2.8.0` | 14/14 | [plans/panel-dkim-signing.md](plans/panel-dkim-signing.md) | -| phone-desktop-mode | A phone browser in "desktop site" mode gets the phone navigation | shipped `2.10.2` | — | — | -| from-header-binding | Hold a message's `From:` header against the application that sends it | shipped `2.4.0` | 9/9 | [plans/from-header-binding.md](plans/from-header-binding.md) | -| panel-redesign | Panel redesign (accepted mockups, design contract) | shipped `2.0.0` | 27/27 | [plans/panel-redesign.md](plans/panel-redesign.md) | -| schema-squash | Squash SQLite migrations into a 2.x baseline | built in panel-redesign (stage 1) | — | [plans/panel-redesign.md](plans/panel-redesign.md) | -| audit-2026-10-12 | Follow-ups of the 2026-10-12 full-tree code audit, seven stages | done 2026-10-15 (stage 0 shipped `2.4.2`, stages 1+2 `2.5.0`, 2b `2.5.1`, 2c `2.5.2`, 3 `2.5.3`, 4 `2.5.4`, 5 `2.6.0`; stage 6 docs and tests, no cut) | 81/81 | [audit-2026-10-12.md](audit-2026-10-12.md) | - -**Recommended order** (not binding; owner, 2026-10-09: feature order follows -development convenience): **panel-redesign** shipped as `2.0.0` (2026-10-11) and -**inbound-antispam-panel** as `2.1.0` (2026-10-12), built in the new component -kit — styling its two screens once instead of twice. **delivery-log-storage** -shipped as `2.2.0` the same day, **panel-notifications** as `2.3.0` and -**from-header-binding** as `2.4.0`. Stage 0 of -[audit-2026-10-12](audit-2026-10-12.md) shipped as `2.4.2` (2026-10-14), its -stages 1 and 2 as `2.5.0` the same day, stage 2b as `2.5.1`, stage 2c as -`2.5.2` (2026-10-15), and stages 3, 4 and 5 as `2.5.3`, `2.5.4` and -`2.6.0` the same day; stage 6 (documents and tests, no cut) closed the -audit. Of the three rows from [own-mta-builds](#own-mta-builds), -**port25-bare-newline** shipped as `2.6.1` the same day and -**base-trixie** as `2.7.0`, and **panel-dkim-signing** as `2.8.0` (2026-10-16). **dns-record-collapse**, **settings-save-row** and **notification-mail-recheck** shipped together as `2.9.0` the same day, with `/data/opendkim` removed as panel-dkim-signing planned, then `2.9.1` (log panes on a phone) and **inbound-upstream-result** as `2.10.0`. The follow-ups of the [security audit of 2026-10-17](security-audit-2026-10-17.md) shipped together as `2.14.4`, stages 0-6 in one cut; its stage H is the owner's host. What is agreed after it is marked so in the index; every other -remaining item is a candidate. panel-docs shipped in -[CHANGELOG.md](../CHANGELOG.md) `[1.8.0]`; dmarc-reports in `[1.7.0]` -(its ingest path first ran in `2.0.0` and was reviewed on 2026-10-11, see -[security.md](security.md)); domain-stats-auto-ratelimit in -`[1.6.0]`; send-log-retention in `[1.5.0]`; inbound-relay in `[1.4.0]`; -queue-retries in `[1.3.1]`; the 2026-08-13 full-tree review follow-ups are in -`[1.3.0]`. Candidates need explicit agreement before they join the queue. - -The 2026-08-19 full-tree code review is closed except for one item, listed -above and not urgent: [origin-strict](#origin-strict) (its "add CSRF tokens", -decided otherwise on 2026-10-13). Of its other three, -template-data-typing is done (the last untyped map, the health sampler's, went -in [audit-2026-10-12](audit-2026-10-12.md) stage 4, `2.5.4`), -structured-logging is [dropped](#dropped-with-the-condition-for-return) -(2026-10-13) and [review-2026-08-followups](#review-2026-08-followups) is -closed (2026-10-14). What that review produced is in -[CHANGELOG.md](../CHANGELOG.md) `[1.9.2]`; its own document was removed once -these four items carried everything still open, and stays in git history. +| security-audit-2026-10-17 | Stage H of the 2026-10-17 security audit: the test server's host (S1, S2, S5) and its DNS (S3) | stages 0-6 shipped `2.14.4`, the product side of S3 `2.15.0`; stage H the owner's | 27/29 | [security-audit-2026-10-17.md](security-audit-2026-10-17.md) § Stage H | +| go-milter-upstream | An issue and a PR for go-milter: its header map outlives a message and is unbounded | prepared 2026-10-16, the owner posts | — | [third_party/go-milter/PATCHES.md](../third_party/go-milter/PATCHES.md) | +| queue-table | Outbound › Queue as the table the mockup draws, from `postqueue -j`; an Inbound › Queue (`in-queue`) | candidate, undecided since 2026-10-09 | — | — | After a context reset, pick an item marked `agreed` or `in progress`, then work the **Implementation checklist** in its linked plan. The `Progress` column above @@ -95,646 +39,76 @@ is `done/total` checklist steps in that plan ([development.md](development.md) --- -## panel-docs +## security-audit-2026-10-17 -**Status:** done — shipped in `[1.8.0]` (2026-08-18). The implementation plan -file was removed as shipped cleanup; release details remain in -[CHANGELOG.md](../CHANGELOG.md). - -**Goal:** built-in operator documentation in the panel — short pages (or a -help drawer) that explain what each Status check and other controls mean, -without sending the operator out to `docs/guide.md`. - -**Boundary:** in-panel help only; not a second copy of the full operator guide. -Seed content includes the Status blurbs removed from the cards in favour of a -denser layout — Machine (kernel counters / rate window), TLS certificate -(port 465, reverse-proxy mount), Hostname / reverse DNS (forward-confirmed -reverse DNS, PTR at the hosting provider), and similar notes for other panel -surfaces as they lose inline commentary. - -**Done when:** an operator can open help from the panel for those topics; the -removed Status blurbs are preserved there (or equivalent); no requirement to -read the git tree for day-to-day meaning of a card. - -**Dependencies / risks:** copy ownership and translation; keeping help in sync -when checks change; not bloating every page with a second column of prose. - -**Version:** shipped `1.8.0`. +The audit's stages 0-6 shipped together as `2.14.4`, the server's DMARC hint +(the product side of S3) and Verified TLS to the upstream (M7) as `2.15.0`. +What is left is stage H, the owner's host and DNS, about an hour of his time +and no product code: S1, S2 and S5, whose commands are in +`dev/security-audit-2026-10-17-host.md` (not published), and S3's records - +`_dmarc.selfpost.mixfed.ru TXT "v=DMARC1; p=reject; rua=mailto:"` and `mixfed.ru CAA 0 issue "letsencrypt.org"`. The report's +§ Stage H has the "done when" of each. --- -## inbound-antispam-panel +## go-milter-upstream -**Goal:** panel view of inbound anti-spam decisions (when, from, to, subject, -action) plus editable allow/deny lists that tune the external filter — without -shipping rspamd or another engine inside the image. +**Problem (owner, 2026-10-16):** the panel builds go-milter v0.4.1 patched +(`third_party/go-milter`, its `PATCHES.md`): upstream adds every header of +a session to a map it hands to `Headers` and empties only on ABORT. Two +things follow, for every user of the library: the map of a second message +on one connection still holds the first message's headers (SelfPost met it +as audit 2026-10-12, L9, and keeps its own headers since), and the map is +bounded by no limit of its own - a message of nothing but headers, up to +the MTA's message size limit, is held whole in each milter. Upstream had no +issue about it on 2026-10-16; its last commit was 2026-04-04. -**Boundary:** metadata journal and list sync only; SelfPost does not run the -filter ([product.md](product.md)). Quarantine is not on the roadmap -([Ideas](#ideas-not-on-the-roadmap)). Design closed in -[plans/inbound-antispam-panel.md](plans/inbound-antispam-panel.md) § Decisions. +**Goal:** an issue and a PR, prepared here and posted by the owner from his +GitHub account (nothing is sent from this side): -**Done when:** with inbound relay enabled, the operator sees journal rows -(accepts via inbound-journal milter, rejects via mail.log tailer) and can -maintain instance-wide allow/deny lists synced to rspamd maps when the antispam -hook is set. +- the issue: both effects, each with a reproduction through the library's + own client (two messages on one connection; the map's size at EOH); +- the PR, small and compatible: the map emptied when a message starts + (`CodeMail`), as it is on ABORT - this fixes the first effect and bounds + the second to one message; +- in the issue, as a proposal: an option of `Server` not to keep the map, + for a milter that reads headers through `Header` alone - what SelfPost's + patch does unconditionally, which upstream cannot take as it is (it would + break backends that read the map). -**Dependencies / risks:** inbound relay `[1.4.0]`; rspamd map/header coupling; -journal PII; global-admin-only RBAC. +**Done when:** the texts and the branch are ready for the owner to post. -**Version:** shipped `2.1.0` (2026-10-12), MINOR (was `1.10.0`; reordered after the redesign on 2026-10-09). +*Prepared (2026-10-16):* in `dev/upstream-pr` (not in git): `ISSUE.md`, +`PR.md`, a README with the steps, and a clone of go-milter on branch +`reset-headers-per-message` - one commit over `master` (0277691) that +empties the map on SMFIC_MAIL, with `TestMilterClient_HeadersPerMessage`, +which fails on `master` (the second message sees both `From` values) +and passes with it. Read for the issue: Postfix carries every message of +one SMTP session over the one milter connection smtpd opened (Postfix +3.10.13, src/milter/milter8.c:2723, LOCAL_SEND_FD of the milter socket to +cleanup; src/smtpd/smtpd.c:3475, milter_send()). +When upstream releases a version that does not keep the map (or offers the +option), `third_party/go-milter` goes and `go.mod`'s `replace` with it. --- -## health-setup-checks - -**Decided by the owner on 2026-10-13**, in place of the former candidate -*preflight* (a page of its own, Server › Preflight, planned in August before -Server › Health existed). Of that plan's eight checks Health already makes -three (reverse DNS, certificate validity, the DKIM socket); five were missing. -Three of the five go into Health, one becomes [test-email](#test-email), one -is not built ([Dropped](#dropped-with-the-condition-for-return)). No second -page of checks. - -**Goal:** three more lines on Server › Health, in the cards that are there: - -- **The certificate is for this server**: its names include - `SELFPOST_HOSTNAME` (TLS certificate card). Today a certificate for another - name is "valid". -- **Postfix greets with the hostname**: the 220 line on port 25 names - `SELFPOST_HOSTNAME`, not `localhost` or a container id (Hostname card). -- **The trusted proxy**: the address the panel sees this request from, and - whether it took it from `X-Forwarded-For`. With `TRUSTED_PROXY_CIDR` wrong - or empty behind a proxy the header is ignored - ([internal/web/auth/auth.go](../internal/web/auth/auth.go)), every client - has the proxy's address and the sign-in limiter is shared by all of them - - silently. - -**Boundary:** lines in existing cards, each with what to do when it fails; -no new card unless the mockup shows the lines do not fit. Mockup-first -([plans/panel-redesign.md](plans/panel-redesign.md) § Enforcement). The -mockup `preflight` stays drawn and unbuilt. - -**Done when:** each line is shown wrong on the e2e stand or in a test with a -wrong certificate name, a wrong greeting and an untrusted proxy, and right on -the test server. - -**Done (2026-10-16):** mockups `b2dcefc`, `cc33890`, `0feaa55` (the owner: -the greeting on a row of its own, the name check a tag beside *Names*). -The certificate: `health.Certificate.ForHost` - DNS names as a TLS client -matches them, the CN only without any, as OpenSSL and so a sending Postfix -still does - fails a certificate for another name. The greeting: -`postconf -x -h smtpd_banner`, read once at the start (main.cf is not -rewritten after), is what smtpd writes after `220` -(src/smtpd/smtpd.c:5979); a first word other than the hostname warns. The -proxy: `auth.Module.Seen` - an `X-Forwarded-For` left unread because the -peer is not trusted warns, with the `/32` (or `/128`) to set. Wrong cases in -unit tests, right ones in e2e (`testFormPostWithTheBrowsersOrigin`). - -**Version:** `2.x` MINOR; `agreed`. - ---- - -## test-email - -**Decided by the owner on 2026-10-13**, taken out of the former *preflight*. - -**Goal:** a global administrator sends a test message from the panel to an -address they type, and sees whether Postfix took it; the message then appears -in Outbound › Log like any other. Today nobody can send through the server -without setting up an application first. - -**Boundary:** global role only; from `postmaster@SELFPOST_HOSTNAME`, the -sender notifications use, or from a sending domain the administrator picks - -settled at the mockup. It does not follow the message further than the log -does. Where the form lives (Health, or a sending domain's page) is settled at -the mockup; mockup-first. - -**Done when:** the form sends, the row appears in the log, the form is -rate-limited and refuses a recipient list, and it passes a security review - -a form that sends mail is a new surface. - -**Done (2026-10-17):** from `postmaster@` the hostname (the owner, -2026-10-16: "Да, адрес годится"), card 09 on Health in the static stack -(mockups `e92769f`, `e3b2501`, `8f0ccf9`). The row is written before the -hand-over and learns its queue id from cleanup's Message-ID line -(`logtail.AwaitMessageID`); the browser lands on it. One address -(`validate.AccountEmail`), ten an hour from the send log, global role. - -**Version:** `2.x` MINOR; `agreed`. - ---- - -## origin-strict - -**Decided by the owner on 2026-10-13**, in place of the former candidate -*csrf-tokens* (per-session tokens on every form, recorded by the 2026-08-19 -review): no tokens; the origin check is made strict instead. - -**Goal:** `originAllowed` -([internal/web/security.go](../internal/web/security.go)) refuses a -state-changing request that carries neither `Sec-Fetch-Site` nor `Origin`. -Today such a request is let through, which leaves a client that sends neither -header open to a request forged by any site -([security.md](security.md) § Accepted risks). - -**Boundary:** one branch of one function, its test, and the documents. No -template changes, no hidden fields. Before the change, find what posts to the -panel without a browser (the e2e stand, scripts) and make it send `Origin`. - -**Done when:** a `POST` with neither header is refused with a test that proves -it, the e2e stand passes, the accepted risk about header-less requests is -struck from [security.md](security.md) and its ADR states the strict rule. - -**Done (2026-10-16):** `originAllowed` returns false in the "neither -header" branch; the unit test's case is refused, e2e posts with neither -header and gets 403, and the e2e suite's client sends `Origin` on every -form it posts. The accepted risk is struck from security.md and the ADR's -decision says refused; architecture and the guide say so. - -**Dependencies / risks:** a browser that sends neither header can no longer -save anything in the panel - accepted, the panel's users pick their own -browsers. Tokens stay a recorded alternative, not a task: the conditions under -which they are built are in [security.md](security.md), ADR § *CSRF via origin -checking* (**Revisit if**). - -**Version:** `2.x` PATCH; `agreed`. - ---- - -## backup-reauth - -**Agreed with the owner on 2026-10-13**, out of -[review-2026-08-followups](#review-2026-08-followups). - -**Problem:** Server › Backup hands the full backup and a domain export to any -live session of a global administrator -([handlers_backup.go](../internal/web/handlers/handlers_backup.go)). The -archive holds the whole database - panel password hashes, application (SASL) -passwords, DKIM keys - so a stolen session cookie, or a panel left open, is a -quiet copy of the server. *Encrypt with a password* does not help: the thief -chooses that password. - -**Goal:** both downloads ask for the current account password, checked the -way a password change checks it and under the same limiter. - -**Boundary:** one field in each of the two forms (mockup-first); import is -not touched; the `selfpost-backup` CLI is not touched - it runs inside the -container. - -**Done when:** a download without the password or with a wrong one is refused -with a test that proves it, attempts are limited, and -[security.md](security.md) § Backup says so. - -**Done (2026-10-16):** "Your account password" in both forms (mockup -`1888dce`), checked by `reauth` in `internal/web/handlers` with bcrypt and -`AllowPasswordChange` (ten in fifteen minutes per user, shared with the -Account form); a wrong or missing one is 401 on the same page, logged, and -nothing is sent. Handler and e2e tests; security.md § Backup and domain -export, guide § Full backup and restore, help. - -**Version:** `2.x` MINOR; `agreed`. - ---- - -## review-2026-08-followups - -**Goal:** the leftovers of the 2026-08-19 full-tree code review that are too -small to be roadmap items of their own. Everything still open from that review -is in the table below; the rest of it shipped in -[CHANGELOG.md](../CHANGELOG.md) `[1.9.2]`. - -| Area | Item | -|---|---| -| Security | ~~Re-authenticate (ask for the current password) before a backup download~~ — struck: agreed on 2026-10-13 as an item of its own, [backup-reauth](#backup-reauth) | -| Tests | ~~The `internal/dmarc` service layer (`service.go`, `addresses.go`) has no tests~~ — struck: a step of audit-2026-10-12 stage 4 (DMARC reports tested where they come in) | -| Tests | ~~No direct tests for the apps / domains / users / dmarc handlers; `authz_test` covers them only indirectly~~ — struck: users with stage 2 (`submitUserUpdate`), DMARC with stage 4; apps and domains get theirs when a step changes them - a test without a change finds nothing the e2e does not | -| Tests | ~~Nothing covers error page templates, a corrupted backup archive, malformed or oversized DMARC XML, delivery-log pagination edges, concurrent store operations, or zero-traffic `store` stats~~ — struck: the corrupted archive goes to stage 3, DMARC XML to stage 4, paging edges to stage 4 (`parsePage`), concurrency of the last global to stage 2; error templates, other concurrency and zero-traffic stats are dropped (owner, 2026-10-14) - code no step changes, and a test kept for its own sake finds nothing | -| Edge case | ~~`internal/secretfile` accepts an empty password at library level~~ — struck: audit-2026-10-12 stage 3 (M5) | -| Edge case | ~~The log fingerprint in `logtail/offset.go` can collide across a rotation — include the inode~~ — struck: agreed on 2026-10-13 as part of a step of [audit-2026-10-12](audit-2026-10-12.md) stage 2b (a log rotated while the panel was down is read to its end) | -| Edge case | ~~`parsePage` does not clamp to `lastPage`~~ — struck: audit-2026-10-12 stage 4 | -| Edge case | ~~`dnscheck/resolver.go` fans out four parallel queries where a fallback chain would do~~ — struck: finding M14 of [audit-2026-10-12](audit-2026-10-12.md) | -| UI | ~~`.pair` (DMARC templates) and `.split` (every other page) are two names for one layout~~ — struck: retired by [panel-redesign](#panel-redesign); both stylesheets are gone and every page is laid out by the component kit | -| UI | ~~Two `back_link` elements in a row in `dmarc_domain.html`~~ — struck: retired by [panel-redesign](#panel-redesign); the page is rebuilt on the kit with one breadcrumb in its head | -| UI | ~~`initShowWhen` is not re-run after an htmx swap~~ — struck: the show/hide rules listen on the document and are applied again on `htmx:load` | - -**Boundary:** no feature work and no behaviour change an operator would notice, -apart from the backup re-auth prompt. Anything here that grows past an hour -becomes its own roadmap item instead. - -**Done when:** each row is either implemented or struck with a written reason. -**Closed 2026-10-14** with the owner: every row is struck above, placed in a -stage of [audit-2026-10-12](audit-2026-10-12.md) or dropped with its reason. - -**Dependencies / risks:** One finding of that review was **declined** and is -deliberately absent: `//go:build linux` for the `/proc` reader in -[internal/health/machine.go](../internal/health/machine.go). That code is -parameterised by root directory, calls nothing platform-specific, already -degrades gracefully when `/proc` is missing (`TestMachineSamplerWithoutProc`), -and tagging it would only delete cross-platform test coverage. - -**Version:** no bearing on semver; `candidate`. - ---- - -## delivery-log-storage - -**Problem (owner, 2026-10-09):** the *Delivery log* box on a message's page -(`/deliveries/{id}`, mockup `out-message`) is usually empty. The lines are not -stored anywhere: `logtail.QueueLines` -([internal/logtail/logtail.go](../internal/logtail/logtail.go)) greps the -**current** `mail.log` on every page view, within its last 4 MB, and never -opens the rotated files. Rotation is daily -([build/logrotate-mail.conf](../build/logrotate-mail.conf)), so a message -older than the last rotation — in practice anything not sent today — shows -"Nothing for this queue id in the current mail log" while its send-log row -lives on for ninety days. The fourteen rotated files on disk hold the lines -and are never read. - -**Goal:** a message's page shows what Postfix wrote about it for as long as -the message is in the send log. - -**Options:** - -1. **Store the lines** (recommended). The log tailer already reads every - `mail.log` line once to update delivery status; a line whose queue id - belongs to a send-log row is also appended to a new table - (`send_log_lines`: row id, written at, text), five to ten short lines per - message. Retention follows the row — the existing send-log sweep deletes - them together — and the page reads the table instead of the file. Lines - written while the panel was down are caught up the way status updates - are, from the persisted read offset. The grep stays only as a fallback - for rows older than the table. -2. **Grep the rotated files too.** `QueueLines` walks `mail.log.1`, - `mail.log.2.gz`, … until it finds the id. No schema, but each view - decompresses up to fourteen files, and past fourteen days the answer is - still empty against a ninety-day row. - -**Boundary:** outbound send-log rows only; the inbound journal -([inbound-antispam-panel](#inbound-antispam-panel)) decides for itself -whether it wants the same. No change to what the box looks like — the -`out-message` mockup already draws it as a `log_pane`. - -**Done when:** a message sent before the last rotation shows its lines on its -page; the lines disappear with the row; the page wording no longer blames -rotation for an ordinary case. - -**Dependencies / risks:** a queue id is reused by Postfix after a message -leaves the queue, so matching must be bounded to the row's lifetime (accepted -time to final status plus the queue lifetime), not to the id alone; the table -grows with the send log (small — a few hundred bytes per message) and is -covered by the same retention. - -**Version:** shipped `2.2.0` (2026-10-12), MINOR; option 1. Decisions and -checklist: [plans/delivery-log-storage.md](plans/delivery-log-storage.md). - ---- - -## panel-notifications - -**Goal:** the panel tells its administrators when something needs them instead -of waiting to be opened. First version, four events: a sending domain's DNS -record stops matching, a level-2 rate limit refuses mail, the TLS certificate -is about to expire, a message has been in the queue for more than three days. - -**Boundary:** e-mail only, from `postmaster@`, submitted locally - -not from one of the sending domains. One checkbox per user under **Account** -(no choice per event), one instance switch under **Server › Settings**; a -domain administrator hears only about assigned domains. No streams, no -digests, no webhooks, no chat integrations. Not in the first version: a -process or milter down, an inbound upstream unreachable, DMARC failures. - -**Done when:** each of the four events reaches the account e-mail of every -user who has notifications on and may see its subject, by the rules of the -plan (a "cleared" message for DNS and the queue only); a user without an -e-mail gets nothing; the switch turns it all off. - -**Dependencies / risks:** the account e-mail (2.0 baseline, see -[plans/panel-redesign.md](plans/panel-redesign.md) § Account e-mail); a relay -that cannot send cannot report that it cannot send - none of the four events -is "the relay is down"; mail from the bare hostname needs an SPF record for -it to be believed, which Server › Health will check; event state must survive -restarts so a reboot does not re-announce everything. The screens are not -drawn: the Notifications box was taken out of `account` and `settings` on -2026-10-10 and is redrawn as the first step. - -**Version:** shipped `2.3.0` (2026-10-12), MINOR - the design questions were closed -with the owner on 2026-10-12 -([plans/panel-notifications.md](plans/panel-notifications.md)) and he took it into -work the same day. - ---- - -## password-reset - -**Goal:** a user who forgot their password gets a one-time link at their -account e-mail instead of asking the global administrator to set a new one. - -**Boundary:** a link on the sign-in page, a short-lived single-use token, the -same answer whether or not the username exists. The only global administrator -without a working e-mail still recovers from the container (today's path). - -**The e-mail must be confirmed (owner, 2026-10-13).** A reset link goes only -to an address its owner has confirmed; an account whose e-mail is not -confirmed has no reset, and recovers as today (the global administrator sets -a password, or the container). The panel does not confirm addresses today - -`users.email` is whatever was typed - so the item builds that first: saving -an e-mail under Account mails a one-time link to it, opening the link marks -the address confirmed (a migration: the mark on `users`), changing the -address clears the mark, and Account shows which it is. An address typed for -a user by the global administrator is unconfirmed until that user opens the -link. To settle at the mockup: whether notifications (2.3.0) keep going to an -unconfirmed address. - -**Done when:** the flow works end to end, tokens expire and cannot be reused, -requests are rate-limited, an account with an unconfirmed e-mail gets no -link and the sign-in page answers the same for it, and it passes a security -review. - -**Dependencies / risks:** the account e-mail; an account-takeover surface where -there was none — security review is part of the item, not an afterthought. -Not drawn yet. Changing the account e-mail must then require the current -password (2.0 security review), because the e-mail becomes where a reset link -goes and today Account saves it without the password. - -**Version:** `2.x` MINOR; `agreed` (owner, 2026-10-13: domain administrators -to come need it - they are not the owner and should not have to ask him for -a new password). Mockup-first; changing the account e-mail asks for the -current password in the same cut. - -**Notifications to an unconfirmed address (owner, 2026-10-17):** "Без -подтверждения слать" - they go to it confirmed or not; only a reset link -needs it confirmed. Every address saved before the release is unconfirmed, -and cutting them off would have stopped every warning at once. - -**Done (2026-10-17):** design 2f5f5b2, route guard b7bbb2c, panel guard -0774da2. Migration 0013 (`users.email_confirmed_at`, `user_links`). Links -are 256 bits, kept as SHA-256, confirmation 24 h and reset 30 min, taken -once (`DELETE … RETURNING`), built on `SELFPOST_HOSTNAME` (never `Host`); -the request answers the same for every name and mails after answering; -limits per client address and per account; a link's GET page has a button -(mail scanners open links). A reset ends every session. Changing the -e-mail on Account asks for the password. The global role's user form sends -a typed address its link. e2e `password_reset_by_e_mail` runs it through -the sink. - ---- - -## local-time - -**Agreed with the owner on 2026-10-13.** It came from one cell: *Today* on -Health's Sender check card is the UTC day and does not say so. Offered a -label ("Today, UTC") or the local day, he chose the local day - which is a -decision about the whole panel, not one cell. - -**Today:** every time the panel shows is UTC and says so (*Time, UTC* in the -logs, "UTC" after a date), and everything counted by day - the sender -check's days, the delivery-log measurement, the statistics behind the -charts, retention - is counted by the UTC day. - -**Goal:** the panel shows time in a chosen time zone, and "today" is that -zone's day. - -**Where the zone comes from (owner, 2026-10-13):** one setting of the -instance, *Time zone* under Server › Settings, UTC by default; it holds for -every user and for notification mail. Not the browser's zone: only a script -could read it, and the panel works without JavaScript. Not a setting of each -user yet - **that is added (the same list under Account, "as the server" by -default) when** a user of the panel lives in another zone than the one set -for the instance, which the first domain administrator who is not the owner -may bring. - -**What is written beside a time (owner, 2026-10-13):** nothing. The "UTC" -labels go - the column is *Time*, a date is a date - and the zone is stated -in one place, the setting. His reason: there is nobody to send a line of the -panel to yet. **The offset is written where "UTC" is today ("Time, UTC+3", -"2026-09-21 16:40 UTC+3") when** times of the panel start to be read by -someone who does not know the instance's zone: a second user of the panel, -or lines copied out of it to another person. - -**A trap this leaves, known in advance (owner asked for it to be written -down, 2026-10-13):** Server › System log, and the *Delivery log* box on a -message's page, show the lines of `mail.log` as Postfix wrote them, each with -its own stamp ("Oct 13 14:01:52"). Those stamps are the container's time - -UTC unless the container is given a `TZ` - and they are not rewritten. Once -the rest of the panel is in the instance's zone and no time carries a label, -a message that the Outbound log puts at 17:01 has lines stamped 14:01 on its -own page, with nothing on the screen to say why. The plan must close this -before the item ships, one way or another: rewrite the stamp when a line is -shown, say on those two screens which clock the lines use, or run the -container in the instance's zone (which moves `mail.log` itself and what the -tailer reads). The guide's section on time must say the same thing as the -screens. - -**How a day is counted (owner, 2026-10-13):** the tables that keep a row -per day - `sender_check_days`, and `delivery_log_days` of -[audit-2026-10-12](audit-2026-10-12.md) stage 1 - key the row by the date in -the instance's zone at the moment of counting, from the release on. Rows -counted before it stay UTC days. Changing the zone later leaves the rows -already written as days of the old zone, a shift of a few hours in a -statistic; accepted - the zone is set once, and rows per hour (which would -survive a change) cost a migration for it. The charts and statistics group -the log by hour already and need nothing. - -**Open, to settle at the mockups:** the trap above; which of the screens -that show a date show the year. - -**Boundary:** what the panel shows and how it groups by day. `mail.log`, -the database's stored instants and the e2e stay as they are. - -**Version:** `2.x` MINOR; `agreed`; the plan is written when it is taken up. - -**Where the zone comes from, changed (owner, 2026-10-17):** not a setting -in the panel but the container's own zone, `TZ` in `.env`; Settings only -shows it ("Да" to "пояс в `.env`, на Settings только показ?"). His reason -was the trap above: a container cannot change its zone without a restart, -and only the container's zone moves Postfix's stamps with the rest. So the -trap is closed by the third way - the container runs in the zone, Postfix -stamps `mail.log` in it (`TZ` is in Postfix's `import_environment`), and the -tailer reads the stamps in the same zone, as it always did (`lineTime`, -`time.Local`). - -**Done (2026-10-17):** design c205106 (no "UTC" on any screen, Settings box -05 *Time zone*, read-only, the help topic). The panel formats every time in -`time.Local`; the day tables key their rows by the local date, the sender -check cuts downtime at local midnights, the log filters' days run from local -midnight; stored instants stay UTC. Notification mail and the test message -write the zone's abbreviation after a time (they are read away from the -panel). The year question settled as the mockups have it. `tzdata` is named -in the image. Tests run in UTC (a `TestMain` in each package that formats -or counts days) and in Moscow where the zone is the point. - -**The one exception (owner, 2026-10-17):** "Показ времени отчёта по UTC -принимаем. Это не проблема. Пояснения добавь." The day a DMARC report -covers stays the reporter's, in UTC (RFC 7489 counts its date range in UTC): -the lists and a report's title name it, the report's page gives its window -in the panel's zone with a line naming that day, and Settings and Help say -it is the exception (2.14.2). - ---- - -## phone-desktop-mode - -**Problem (owner, 2026-10-11):** the panel opened on a phone with the -browser's "desktop site" switched on does not become the desktop panel. -Seen on Server › Health at `selfpost.mixfed.ru`: the page body is laid out in -columns (Machine and Processes side by side, three cards in the next row), -while the navigation is the phone one - not a bar but a stack with every -section open (Outbound, Inbound, Server, the user menu), filling the first -screen before the page starts. The content also sits flush against the left -edge of the screen. - -**Goal:** find out why, then decide what the panel should be in that mode - -the desktop panel, or a phone panel that is at least consistent. - -**Where to look first (a guess, not checked):** "desktop site" on Android -lays a page out about 980 px wide. Bulma turns the navigation into a bar at -1024 px and puts columns side by side from 769 px, so 980 px falls between -the two: desktop columns under a touch navigation. `panel.css` has its own -breakpoints at 768, 1023 and 1407 px. Check the width the phone really -reports, why every dropdown is open rather than folded behind the burger, -and where the left padding went. - -**Boundary:** the navigation and page gutter between 769 and 1023 px, which -is also a tablet in portrait and a narrow desktop window - the same defect is -expected there. A design change is mockup-first -([plans/panel-redesign.md](plans/panel-redesign.md) § Enforcement). - -**Done when:** the cause is written down here, and the panel at that width -has one coherent layout, checked on a phone in "desktop site" mode and in a -980 px window. - -**Cause (2026-10-16):** not the mode - every width under 1024 px. The bar's -menu was always open (`navbar-menu is-active`, and no burger to close it), -and Bulma lays a navbar menu out below 1024 px as a block with every -dropdown open: on any phone the sections and all their pages filled the -first screen. "Desktop site" makes the page about 980 px wide, where Bulma -already puts columns side by side (from 769 px) but not yet the bar - the -mix in the screenshot. And a Bulma `.container` has side margins only from -1024 px, so below it the page met the screen's edge. - -**Done (2026-10-16):** the owner chose "Вариант 1" - a menu behind a button -- with "сворачивание разделов". Below 1024 px the bar's menu is hidden and -*Menu* at the right of the brand opens a panel over the page: Overview, each -section a fold with the current one open, the user's last (Account, Help, -Sign out); `
` all through, no script. The page and the strip of -sibling pages have a 1rem gutter. From 1024 px nothing changed. Mockup -`e0e30e2`. - -**Version:** `2.x` PATCH or MINOR; `agreed` (owner, 2026-10-13), taken up -after stages 1 and 2 of [audit-2026-10-12](audit-2026-10-12.md) - mail shown -wrong and the tenant boundary come before layout. - ---- - -## from-header-binding - -**Problem (security review of panel-notifications, 2026-10-12):** Postfix -holds the envelope sender against the SASL login -(`reject_sender_login_mismatch`), and OpenDKIM chooses the signing key by the -`From:` header. Nothing holds the header against the login. An application -of one sending domain can therefore write another registered domain - or, -since 2.3.0, the server's own name - into `From:` and have the message signed -with that name's key: mail that passes DMARC as a domain the application has -no right to, or that looks like a notification of the panel. - -**Agreed with the owner on 2026-10-12**, built and reviewed the same day -([plans/from-header-binding.md](plans/from-header-binding.md); what is not -done is listed there under *Not done*). The rule he -chose is wider than the one first written here: `From:` is held to exactly -what the envelope is held to - any address of the application's domain, or -the addresses of its list - and anything else is refused, a stranger's domain -included. The check is a role of the panel on a strict socket of its own: -while the panel does not answer, mail is deferred, not accepted unchecked. -Server › Health counts what that costs, to decide later whether the check -needs a process of its own. - -**Done when:** the e2e stand shows an application refused for another -tenant's `From:`, for the hostname's and for an address off its list, and -accepted for its own. - -**Version:** shipped `2.4.0` (2026-10-12), MINOR. What it costs - the -panel's downtime is now mail's - is in [security.md](security.md) § Accepted -risks. - ---- - -## panel-redesign - -**Goal:** a panel an operator opens a few times a year and still finds their -way in — the important things in view, details one or two clicks away, one -visual language, no page tuning its own width. The design is drawn and -accepted: 33 screens and a components page under -[docs/assets/panel-redesign/](assets/panel-redesign/index.html) (Bulma base, -postmark / numbered heads / perforated edge, icons). Contract and checklist: -[plans/panel-redesign.md](plans/panel-redesign.md). - -**Boundary:** templates, one `panel.css`, shared partials, vendored Bulma and -icons. Navigation regroups by mail direction (Overview · Outbound · Inbound · -Server) and paths are renamed to match. **No compatibility is kept** (owner, -2026-09-21): no redirects, the migration chain is replaced by one baseline -and 2.0 starts from an empty data directory — the only live installation is -the owner's. Inbound domains become delegable to a -domain administrator, separately from outbound. Server › Health is built -with the restyle (decided 2026-10-09). Three things the mockups show keep -their own decision and are not smuggled in: queue as a table -(`postqueue -j`), inbound queue, inbound log (rides on -inbound-antispam-panel) — until then both queues ship the old content in the -new shell. No CSP change, no JavaScript widgets, no second framework. - -**Done when:** every page renders with the component skeleton recorded in -`panel/outlines.json`, the guard tests in the plan are green with no -allow-list, and each page has an evidence screenshot accepted against its -mockup. - -**Dependencies / risks:** an implementer substituting local patches or a look -of their own — answered by guard tests that land before the templates, a -`design-first` CI step and evidence-based acceptance (plan § Enforcement). -Competes with feature work; Bulma is ~680 kB minified, embedded once. - -**Version:** shipped `2.0.0` (2026-10-11), MAJOR. - ---- - -## schema-squash - -**Goal:** when 2.x is cut, stop shipping the 1.x migration chain in the binary -and replace it with one baseline equal to the schema at the then-current head. -Fresh 2.x data directories no longer create-then-drop the historical `admin` -table. Current chain, legacy notes, and gate thresholds: -[schema-migrations.md](schema-migrations.md) (update that file when migrations -ship; adjust the squash gate to its head at cut time). - -**Boundary:** 1.x keeps the full chain so a `1.0.0` data directory still boots. -Do not delete, rename, or reorder migration files while MINOR compatibility with -`1.0.0` holds. Git history keeps the old files either way; only the embedded -set in the 2.x image changes. - -Restore remains a separate lock: the backup manifest version must match the -running binary ([architecture.md](architecture.md) § Persistence). It does not -replace the squash upgrade gate. - -**Done when:** 2.x embeds a single baseline (plus any 2.x-only migrations after -it); the gate in [schema-migrations.md](schema-migrations.md) is tested; the -operator guide says a 2.x image will not open an unfinished 1.x database. - -**Trigger decided 2026-09-21:** [panel-redesign](#panel-redesign) is the 2.0 -cut, and the owner dropped the upgrade gate with it — the chain is deleted to -zero and 2.0 starts from an empty data directory. The -**Boundary** and **Done when** above are superseded on that point by -[plans/panel-redesign.md](plans/panel-redesign.md) § No compatibility; this -item ships as a step of that plan, not on its own. - -**Built 2026-10-09** (panel-redesign stage 1): `internal/store/migrations/` -holds one baseline, `0001_init.sql`. The gate that survives is a refusal, not -an upgrade: `store.Open` rejects a database that was not created by 2.x -(`ErrForeignSchema`, tested), so a 1.x file is never half-read — the "missed -gate" risk below. See [schema-migrations.md](schema-migrations.md). - -**Dependencies / risks:** a decided 2.x cut (another breaking change, or an -explicit major). Squashing the current short chain is not a reason to cut 2.x on -its own. A missed gate leaves a mid-chain 1.x database silently stuck. -**Version:** `2.x` major only; not a 1.x item. +## queue-table + +**Problem:** the 2.0 mockups draw Outbound › Queue as a table and an +Inbound › Queue (`in-queue`); the redesign kept both out as decisions of +their own (owner, 2026-10-09). Outbound › Queue shows `postqueue -p`'s own +listing in a log pane, and the mockup's `in-queue` is drawn but not built +(`deferredScreens`, `outlineExempt` in the guard tests). + +**Goal:** the outbound queue as a table from `postqueue -j` - a row per +message with its sender, recipients, age and last reason - and, if the owner +wants it, the same for mail port 25 holds for an upstream. + +**Recommendation:** the outbound table first, read-only - about 4 h with its +tests, since the notifier already reads `postqueue -j` for each message's +arrival time; the inbound queue and any action on a row (retry, delete) only +if the owner asks for them, each drawn mockup-first. --- @@ -780,11 +154,12 @@ without that it is not offered again. line "last connection from outside on each port", read from `mail.log`, and a command in the help to run from one's own machine. **Returns if:** the owner accepts an outside probe, or asks for one of the two substitutes. -- **preflight as a page of its own** (dropped 2026-10-13): see - [health-setup-checks](#health-setup-checks). Its plan, +- **preflight as a page of its own** (dropped 2026-10-13): its installation + checks went to Server › Health (health-setup-checks, `2.11.0`) and its test + e-mail to a form of its own (test-email, `2.12.0`). Its plan, `plans/preflight.md`, was removed and stays in git history. - **csrf-tokens** (decided 2026-10-13) - session-bound tokens on every form. - Replaced by [origin-strict](#origin-strict); the conditions for building + Replaced by origin-strict (`2.10.4`); the conditions for building tokens after all are in [security.md](security.md), ADR § *CSRF via origin checking*. @@ -803,237 +178,3 @@ joins the index only when the owner says so. [product.md](product.md). Notes and open questions: [plans/inbound-quarantine.md](plans/inbound-quarantine.md); the screens `in-quarantine` and `in-quarantine-message` are drawn and stay unbuilt. - -## dns-record-collapse - -**Problem (owner, 2026-10-15):** a DNS record block - the DKIM, SPF and -DMARC records on an outbound domain's page, the server's records on -Server › Settings, the MX of an inbound domain - shows the host, the type -and the value to publish whether or not the record is already right. When -DNS holds the record as it should be, the values are of no use on the page: -they only push the records that need work further down. - -**Goal:** a record whose check is *ok* is drawn folded - its name and its -tag, one line - and opens on request to show the host, type and value (to -copy them again, or to compare). A record that is missing, wrong or not -checked yet stays open, as it is now, with *In DNS now* where it differs. - -**Notes:** the block is the one `dns_record` partial (components.html), so -the change is made once; it is a design change, mockup first (the kit's -dns_record section and the four pages that draw records), wording -included. No JavaScript needed for the folding itself: `
` opens -and closes without it under the panel's CSP. Whether a fold remembers being -opened across the polls of a page, and whether *Re-check DNS* folds a record -that has just turned right, are for the mockup. - -**Done (2026-10-16):** a record whose tag is *published* and that has no -*In DNS now* is a `
` whose summary is the head: name, tag and -*Show values* (the eye of the icon subset - it has no chevron), the -values behind it. The head's note is hidden while folded, or a long one -(the server's SPF in a half column) wraps the fold to two lines; it shows -when opened. The two questions answered themselves: no page that draws -records polls them, and *Re-check DNS* posts and draws the page again, so a -fold is never kept across a refresh and a record that has just turned -right comes back folded. - -## notification-mail-recheck - -**Problem (owner, 2026-10-15):** Server › Settings shows the DNS records of -the server's own name in its *Notification mail* box (DKIM and SPF, with -their tags), and nothing on that page says how to check them again after -publishing. The way exists elsewhere: *Re-check DNS* on Server › Health -re-runs the server's checks, these records' included -(`HandleHealthRecheck` → `serverMail(true)`), and Health has a Notification -mail card of its own - but from Settings it cannot be found. - -**Goal:** the records can be checked again from where they are shown - a -*Re-check DNS* action in the box's head, or at least the box saying where -the check is and how long a result is kept. Design change, mockup first; -decide together with [dns-record-collapse](#dns-record-collapse) (a record -that turns right on the re-check folds). - -**Done (2026-10-16, the owner: "Да, бери notification-mail-recheck"):** -*Re-check DNS* in the page head of Settings - where an outbound domain's -page and Health put the same action; a box head holds a note, a tag, a -link or a filter form - posting to `POST /server/settings/dns-recheck` -(global role), which re-runs Health's lookups (the server's address and -`serverMail(true)`) and returns to Settings with "DNS re-checked."; the -Notification mail box's end slot reads "The server's own name · checked -3 min ago". A record that has turned right comes back folded. - -## own-mta-builds - -**Question (owner, 2026-10-15):** is it worth patching Postfix and OpenDKIM -and shipping our own builds of them, to optimise something in SelfPost or to -lift a limit we work around? Weigh it, for and against; nothing is agreed -beyond the analysis. - -**For the analysis** (starting points, none of them weighed yet): - -- *Limits met so far* - OpenDKIM logs through syslog(3) only, hence the - syslog relay (audit 2026-10-12, L30); the panel reads Postfix's state from - `mail.log` lines and `postconf`/`postqueue` output; the sender check reads - From: the way OpenDKIM does rather than asking it; the image is held to - what Debian bookworm packages (Postfix 3.7.11, OpenDKIM 2.11.0~beta2, a - beta that upstream has not followed with a release). -- *For own builds* - such limits lifted at the source; newer upstream - versions than the distribution's; options compiled in or out. -- *Against* - security fixes become ours to follow and rebuild (Debian's - security team does it now); longer and less reproducible image builds; a - patch set to carry across upstream versions by one maintainer; the docs' - habit of citing upstream source by version would have to cite ours. -- *Middle ways to compare* - an unpatched upstream build only where the - distribution's version is the limit; a patch sent upstream instead of - carried; replacing OpenDKIM with signing in the panel's own milter (Go), - which the panel already runs for the sender check. - -**Outcome expected:** a recommendation with costs - which limits a build of -our own would remove, what each costs to keep, and whether a middle way -gets the same for less. - -**Outcome (2026-10-15):** [own-mta-builds-2026-10-15.md](own-mta-builds-2026-10-15.md). -No own builds: no limit of Postfix's needs a patch (the panel reads it -through the interfaces it offers), and every limit worked around is -OpenDKIM's - a project with no release since 2018, three CVEs of 2026 open -in every Debian suite and no one answering. What the question pointed at -is answered by three rows instead, in this order: -[port25-bare-newline](#port25-bare-newline) (one line, now), -[base-trixie](#base-trixie) (bookworm is in LTS since 2026-07-12 and its -Postfix 3.7.11 lacks the three 2026 security batches, which carry no CVE -ids and so will not come), and -[panel-dkim-signing](#panel-dkim-signing) (signing in the panel's own -milter; OpenDKIM, the syslog relay and the parser port go). The owner -agreed to the three and their order on 2026-10-15. - -## port25-bare-newline - -**Finding (own-mta-builds, 2026-10-15):** Postfix 3.7.11 has the 2023 -SMTP-smuggling parameter `smtpd_forbid_bare_newline` with the default `no` -(`src/smtpd/smtpd.c:807`, "Postfix < 3.9: no"; 3.9 made `normalize` the -default and the 3.7 `RELEASE_NOTES` recommend it). The inbound relay's -`smtp/inet` block in `build/postfix-config.sh` does not set it, so with -`INBOUND_RELAY_ENABLE=true` a bare `.` inside DATA ends the -message early and what follows is read as the next transaction. Ports 465 -and 587 are not the case that matters: there the smuggled transaction is -still the authenticated session's, under its bindings and milters. - -**Goal:** `smtp/inet/smtpd_forbid_bare_newline=normalize`, an e2e check on -port 25 (a bare-LF DATA, one message delivered, nothing smuggled), a -PATCH release. [base-trixie](#base-trixie) would bring the default, but -the relay should not wait for it. - -## base-trixie - -**Problem (own-mta-builds, 2026-10-15):** the image's base, -`debian:bookworm-slim`, left regular security support on 2026-07-12 (LTS -until 2028-06-30). Its `postfix 3.7.11-0+deb12u1` is code of mid-2024; -upstream's three security batches of 2026 (June: unbounded SMTP inputs; -August: BDAT memory exhaustion, a DATA crash, a policy bypass; September: -smuggling behind a proxy filter, a Milter-reachable over-read) carry no -CVE ids, and the LTS team works from CVE ids - no bookworm upload in 2026. -Trixie's `postfix 3.10.13-0+deb13u1` has the August batch; 3.10.14 with -September's is upstream and trixie-security follows within weeks. - -**Goal:** `debian:trixie-slim` and `golang:1.26-trixie`; every citation of -`3.7.11` (331 lines in 65 files) re-read against the 3.10.13 tree, with -attention to 3.10's reject logging (`sasl_method`, `sasl_username`, -`sasl_sender` on the line), the queue id on an abnormal connection end, -cleanup's `canceled` line, 3.8's `postfix(1)` stderr logging under -supervisord; `dev/upstream` refetched from trixie; the cited-package table -and `checkCitedPackageVersions`; the e2e sink image; `opendkim-tools` -dropped from the image (used by nothing at run time). OpenDKIM's signing -and From:-parsing code is the same in trixie (its new Debian patches touch -the verifier and librbl), its citations stand. One stage, a MINOR release. Also -gained: `smtpd_hide_client_session` (3.10); the owner decided on -2026-10-15 to turn it on for 465 and 587 in the same release, so the -`Received:` header no longer shows the application's address. - -## panel-dkim-signing - -**Problem (own-mta-builds, 2026-10-15):** OpenDKIM is a beta of 2018 with -no upstream answering (CVE-2026-100888, -100889, -101277 open in every -Debian suite, "vendor did not respond"; Debian bug #1149875 unanswered). -Every limit SelfPost works around is its: it logs through syslog(3) only -(the relay), it chooses the key by `From:` with its own parser (the port, -the 256-byte refusal, the probe scripts), its socket file outlives a -crash, its tables are reloaded by a cross-uid signal (`CAP_KILL`), its -keys must be shared through a group. - -**Goal:** the panel's own milter signs - `github.com/emersion/go-msgauth/dkim` -(the e2e suite's verifier already; `relaxed/simple`, `From` over-signed, -RFC 6376 § 5.4's header set, the body hashed as it streams), the key by -the login's domain on 465/587 and by the server's name for the panel's -local submission (`non_smtpd_milters` on the same socket, strict). -Removed: the `opendkim` and `opendkim-tools` packages, `opendkim.conf`, -two supervisord programs (OpenDKIM, the syslog relay), the table renderer -and its reload, the parser port and its scripts, Health's OpenDKIM -entries, probably `CAP_KILL` - about 1,500 lines. Keys, selectors and -published DNS records unchanged; nothing for an operator to do. The e2e -oracle must be an independent verifier (`opendkim-testmsg` in the sink -image, or rspamd), not the signing library, and a message to a real -mailbox precedes the cut. Two to three stages, a MINOR release; after -[base-trixie](#base-trixie) (remote exposure first; the two touch -disjoint citations). - -## settings-save-row - -**Problem (owner, 2026-10-15, a screenshot of `selfpost.mixfed.ru`):** on -Server › Settings the *Save settings* button sits under the two columns, at -the left, after a long gap: the left column (retention, sending rate limits) -is short, the right one (the notifications switch, then *Notification mail* -with its records open) is long. The button saves the retention fields and -the switch; the records under the switch are not fields, and they are what -pushes the button down. - -**Goal:** the button reads as the save of the fields it saves. Either the -gap goes by itself when [dns-record-collapse](#dns-record-collapse) folds -the records that are right, or the page changes - the records out of the -form's columns, or the button nearer its fields. Decide with the mockup, -after dns-record-collapse, on a page whose records are missing as well as -on one whose records are right. - -**Done (2026-10-16):** shown both layouts in both states, the owner chose -"Вариант 2: форма в левой колонке, кнопка под своими полями." Folding alone -closed the gap only while the records are right. The form is now the left -column - 01 Log retention, 02 Notifications, *Save settings* - and the -right column, outside it, holds 03 Notification mail and 04 Sending rate -limits. - -## go-milter-upstream - -**Problem (owner, 2026-10-16):** the panel builds go-milter v0.4.1 patched -(`third_party/go-milter`, its `PATCHES.md`): upstream adds every header of -a session to a map it hands to `Headers` and empties only on ABORT. Two -things follow, for every user of the library: the map of a second message -on one connection still holds the first message's headers (SelfPost met it -as audit 2026-10-12, L9, and keeps its own headers since), and the map is -bounded by no limit of its own - a message of nothing but headers, up to -the MTA's message size limit, is held whole in each milter. Upstream had no -issue about it on 2026-10-16; its last commit was 2026-04-04. - -**Goal:** an issue and a PR, prepared here and posted by the owner from his -GitHub account (nothing is sent from this side): - -- the issue: both effects, each with a reproduction through the library's - own client (two messages on one connection; the map's size at EOH); -- the PR, small and compatible: the map emptied when a message starts - (`CodeMail`), as it is on ABORT - this fixes the first effect and bounds - the second to one message; -- in the issue, as a proposal: an option of `Server` not to keep the map, - for a milter that reads headers through `Header` alone - what SelfPost's - patch does unconditionally, which upstream cannot take as it is (it would - break backends that read the map). - -**Done when:** the texts and the branch are ready for the owner to post. - -*Prepared (2026-10-16):* in `dev/upstream-pr` (not in git): `ISSUE.md`, -`PR.md`, a README with the steps, and a clone of go-milter on branch -`reset-headers-per-message` - one commit over `master` (0277691) that -empties the map on SMFIC_MAIL, with `TestMilterClient_HeadersPerMessage`, -which fails on `master` (the second message sees both `From` values) -and passes with it. Read for the issue: Postfix carries every message of -one SMTP session over the one milter connection smtpd opened (Postfix -3.10.13, src/milter/milter8.c:2723, LOCAL_SEND_FD of the milter socket to -cleanup; src/smtpd/smtpd.c:3475, milter_send()). -When upstream releases a version that does not keep the map (or offers the -option), `third_party/go-milter` goes and `go.mod`'s `replace` with it. diff --git a/docs/schema-migrations.md b/docs/schema-migrations.md index d17213d..ce47c44 100644 --- a/docs/schema-migrations.md +++ b/docs/schema-migrations.md @@ -17,8 +17,8 @@ When you add or review a migration, update this file in the same change. | Field | Value | |---|---| -| Chain head | `user_version = 13` | -| Files | `0001_init.sql` (the 2.0 baseline), `0002_inbound_spam_log.sql`, `0003_send_log_lines.sql`, `0004_notifications.sql`, `0005_send_log_rejected_index.sql`, `0006_sender_check.sql`, `0007_delivery_log_days.sql`, `0008_tenant_boundary.sql`, `0009_refusal_line.sql`, `0010_send_log_indexes.sql`, `0011_notify_addressees.sql`, `0012_inbound_upstream.sql` | +| Chain head | `user_version = 14` | +| Files | `0001_init.sql` (the 2.0 baseline), `0002_inbound_spam_log.sql`, `0003_send_log_lines.sql`, `0004_notifications.sql`, `0005_send_log_rejected_index.sql`, `0006_sender_check.sql`, `0007_delivery_log_days.sql`, `0008_tenant_boundary.sql`, `0009_refusal_line.sql`, `0010_send_log_indexes.sql`, `0011_notify_addressees.sql`, `0012_inbound_upstream.sql`, `0013_email_links.sql`, `0014_upstream_tls_secure.sql` | | File mark | `PRAGMA application_id = 0x53503230` ("SP20") | | Database file | `/data/selfpost.db` (bind mount) | | Compatibility | 2.x MINOR releases must boot a `2.0.0` data directory. **Nothing is carried over from 1.x.** | @@ -91,6 +91,7 @@ audit (`2.5.0`), `0009_refusal_line.sql` (`2.5.1`) and | 11 | `0011_notify_addressees.sql` | 2.5.4 | DDL | `notify_addressees (key, address, told_at)`: who a subject's notification is owed to and who was handed it, so a copy that failed or was held back by the hourly ceiling goes out at a later check and a subject's end goes to those told (audit 2026-10-12, stage 4); deleted with its `notify_state` row | | 12 | `0012_inbound_upstream.sql` | 2.10.0 | DDL | `inbound_spam_log` gains `queue_id`, `upstream_status` (`''`, `queued`, `held`, `sent`, `deferred`, `bounced`; CHECK), `upstream_relay`, `upstream_reply` and `upstream_at`, and the partial index `idx_inbound_spam_log_queue_id` on rows that have a queue id: what the upstream answered about mail accepted on port 25, put on the row by the log tailer (inbound-upstream-result). Rows from before have none | | 13 | `0013_email_links.sql` | 2.14.0 | DDL | `users.email_confirmed_at` (`''` for every address saved before it: all unconfirmed) and `user_links`, the one-time links that confirm an address and reset a password, with `idx_user_links_user` (password-reset; owner, 2026-10-13 and 2026-10-17) | +| 14 | `0014_upstream_tls_secure.sql` | 2.15.0 | DDL + data | `inbound_transports.tls_mode` may be `secure` (Verified: Postfix's level that checks the upstream's certificate against its name; upstream-tls-verify, security audit 2026-10-17, M7). SQLite cannot alter a CHECK, so the table is made anew with the wider one and its rows copied | **Kind:** *DDL* — schema only; *data* — `INSERT`/`UPDATE` that must stay correct for operators upgrading from older 2.x images. @@ -113,7 +114,7 @@ for operators upgrading from older 2.x images. | `send_log` | Delivery journal. `reason` says why a row was refused (`limit`, `client`, `from`, and `sender` since 2.8.0) or that the sweep closed it (`reconcile`, `reconcile-contradicted`), `header_from` what the From: or Sender: line of a message refused for it said (v6), `refusal_line`, `client_ip` and `recovered` (v9), and `domain_id` (v8: the row belongs to a domain by id, `ON DELETE SET NULL`) | | `logtail_state` | Log-tailer read position | | `inbound_domains` | Inbound relay domains | -| `inbound_transports` | Upstream host / port / TLS per inbound domain | +| `inbound_transports` | Upstream host / port / TLS per inbound domain; `tls_mode` is `may`, `encrypt`, `secure` (v14) or `none` | | `inbound_recipients` | Allow-list when `recipient_mode = list` | | `user_inbound_domains` | Inbound domains assigned to a `domain` user — granted separately from the outbound ones | | `dmarc_reports` | Parsed aggregate report summaries | diff --git a/docs/security-audit-2026-10-17.md b/docs/security-audit-2026-10-17.md index 97b08db..b4d3a24 100644 --- a/docs/security-audit-2026-10-17.md +++ b/docs/security-audit-2026-10-17.md @@ -1,5 +1,11 @@ # Security audit — 2026-10-17 +> **Stages 0-6 shipped in `2.14.4` (2026-10-17)**, the product side of S3 +> and the fix of M7 in `2.15.0` (2026-10-18). Stage H, the owner's host and +> DNS, is open and tracked in [roadmap.md](roadmap.md#security-audit-2026-10-17). +> The file stays because the code cites its finding ids (`A2`, `M7`...); its +> `file:line` references are of the tree at `44bf09d`. + **What this is.** A full security audit of SelfPost at `44bf09d` (release `2.14.3`): the code tree, the image and its process model, the deployment fragments, CI/CD and the supply chain, the dependencies, and the live diff --git a/docs/security.md b/docs/security.md index 3ba945e..638b56f 100644 --- a/docs/security.md +++ b/docs/security.md @@ -365,8 +365,8 @@ operator can act on it. MIME boundary and the signature still verified), then once more each of the shown ones present (Subject, Date, To, Cc, Reply-To, Sender) and `From` last, so one of those added after signing breaks the signature. Until `2.8.0` OpenDKIM signed - a - 2018 beta with three 2026 CVEs open and nobody upstream to fix them - ([own-mta-builds](own-mta-builds-2026-10-15.md) § 1) - and chose the key + 2018 beta with three 2026 CVEs open and nobody upstream to fix them - and + chose the key by the `From:` header; it left the image, and with it the syslog relay and `/dev/log`, the port of its `From:` parser, its tables and their reload by a signal across uids. Architecture § DKIM signing. @@ -445,8 +445,7 @@ operator can act on it. and later; `src/global/mail_params.h`, `DEF_SMTPD_FORBID_BARE_LF`), so they end DATA the same way; the port-25 setting stays explicit. Before `2.7.0` a smuggled transaction on a submission port was still the - authenticated session's, under its sender bindings and milters - ([own-mta-builds](own-mta-builds-2026-10-15.md) § 5). + authenticated session's, under its sender bindings and milters. - **The panel's milters judge every message of a connection.** Postfix introduces a client to a milter once per connection, and the milter library replaces the milter's session after any reply that is not "continue". Until @@ -651,7 +650,7 @@ deferred item from the roadmap. templates must contain no inline scripts and no inline styles. Decided by the owner on 2026-10-13: no tokens; the roadmap item that asked for them is dropped and the origin check is made strict instead - ([origin-strict](roadmap.md#origin-strict)). This bullet and the ADR below + (origin-strict, `2.10.4`). This bullet and the ADR below are the single record of the decision, and the ADR's **Revisit if** says when tokens are built after all. - **Destructive-action confirmation (`data-confirm`) is JavaScript-only.** @@ -872,7 +871,8 @@ deferred item from the roadmap. seconds or the whole instance; Server › Health counts the downtime and the mail deferred for it. Trigger to revisit: those numbers not staying at nothing - then the check moves into a process of its own - ([plans/from-header-binding.md](plans/from-header-binding.md)). + (a subcommand of the panel binary and a `[program:]` of supervisord; the + downtime figure then needs a new meaning). - **A malformed packet on a milter socket of the panel ends the panel.** The milter library indexes into an empty packet and nobody recovers the panic, so the process exits and supervisord starts it again. The sockets are @@ -956,21 +956,13 @@ deferred item from the roadmap. uses with another mode is refused and told that mode - not the domain - so the form tells whether some domain forwards to a given host and port. Accepted with the rest of this item. -- **Account's profile (username, e-mail) is saved without the password.** - Saving the profile does not ask for the current password, as changing the - password does. Since 2.3.0 the e-mail is where notifications go, so a stolen - session can turn them to another address: the thief then learns what they - say - which domain's DNS is wrong, which application is refused, when the - certificate expires, sender addresses of old queued mail - and the user - stops hearing it. Nothing in a notification gives access. A `domain` user - can likewise point signed mail of this server at a third party's address, - a dozen messages a day per assigned domain at most. The profile form also - says when a username is taken, so a signed-in user can tell which - usernames exist; signing in does not (`2.5.4`). It - becomes a requirement to ask for the current password when changing - `users.email` as soon as [password-reset](roadmap.md#password-reset) mails a - link to it — from then on whoever can change the address could take the - account over. +- **Account's username is saved without the password.** Changing the + e-mail on Account asks for the current password since `2.14.0`, on the + password change's limit: it is where a reset link goes, so whoever could + change it could take the account over. The username and the notification + switch are saved without it. The profile form says when a username is + taken, so a signed-in user can tell which usernames exist; signing in does + not (`2.5.4`). ## ADR: CSRF via origin checking, without tokens @@ -1023,7 +1015,7 @@ external-site case this ADR covers. **Decision of 2026-10-13 (owner).** Asked to choose between this ADR, tokens, and a strict origin check, the owner chose the ADR with the strict check -([origin-strict](roadmap.md#origin-strict)) and no tokens. Session-bound tokens +(origin-strict, `2.10.4`) and no tokens. Session-bound tokens are the answer to return to - a hidden field in every state-changing form and htmx path, checked against the session, on top of the origin check - when either condition above comes true, and also when: a browser the panel must diff --git a/internal/inbound/lists.go b/internal/inbound/lists.go index 2b08110..8bcdf10 100644 --- a/internal/inbound/lists.go +++ b/internal/inbound/lists.go @@ -340,9 +340,9 @@ type Usage struct { // counts for every entry of the deciding list that fits it: the journal does // not say which of two fitting entries the filter met first. // -// A counter kept with each entry would not depend on the journal; the plan -// (docs/plans/inbound-antispam-panel.md, Filter list usage) says when that is -// worth a migration. +// A counter kept with each entry would not depend on the journal; +// docs/architecture.md (the antispam filter lists) says when that is worth a +// migration. func (l *Lists) Usage() (map[int64]Usage, error) { entries, err := l.store.ListSpamListEntries() if err != nil { diff --git a/internal/logtail/sendercheck.go b/internal/logtail/sendercheck.go index cd62580..8efa3e5 100644 --- a/internal/logtail/sendercheck.go +++ b/internal/logtail/sendercheck.go @@ -8,7 +8,7 @@ import ( ) // The sender check is attached to Postfix strictly: while the panel does not -// answer on its socket, mail is deferred (docs/plans/from-header-binding.md). +// answer on its socket, mail is deferred (docs/architecture.md § Sender check). // How many submissions that happened to is read here, from what Postfix // writes about it, and shown on Server > Health. It is the figure that also // sees a panel which is running and not answering. diff --git a/internal/milter/sender.go b/internal/milter/sender.go index bebe217..b6c6e51 100644 --- a/internal/milter/sender.go +++ b/internal/milter/sender.go @@ -22,7 +22,7 @@ import ( // The sender check: a message's From: line is held against the application // that sends it, under the rule Postfix holds the envelope sender to -// (docs/plans/from-header-binding.md). The signer signs with the key of the +// (docs/architecture.md § Sender check). The signer signs with the key of the // domain the login belongs to (sign.go), and a receiver holds that signature // to the domain of this line (DMARC, RFC 7489 § 3.1.1), so without the check // an application could send under the server's signature as another domain diff --git a/internal/milter/sign.go b/internal/milter/sign.go index a3aa9eb..84684b0 100644 --- a/internal/milter/sign.go +++ b/internal/milter/sign.go @@ -1,6 +1,6 @@ package milter -// The DKIM signer (docs/plans/panel-dkim-signing.md): the panel's own milter +// The DKIM signer (docs/architecture.md § DKIM signing): the panel's own milter // that signs what applications send on 465/587 with the key of the domain // their login belongs to, and what the panel itself sends with the key of // the server's name. Each of the two is a socket of its own, so the socket a diff --git a/internal/notify/notify.go b/internal/notify/notify.go index 25a34c1..cbf6e90 100644 --- a/internal/notify/notify.go +++ b/internal/notify/notify.go @@ -5,7 +5,7 @@ // // Each subject has a row of state in the database, so a thing is said once and // a restart says nothing again. The rules of when a message goes out are the -// owner's (docs/plans/panel-notifications.md § When a message is sent) and +// owner's (docs/architecture.md § Notifications) and // live in this file; what the messages say is in text.go, how they leave in // mailer.go. package notify diff --git a/internal/web/handlers/handlers_notifications_test.go b/internal/web/handlers/handlers_notifications_test.go index f40d45a..48faf81 100644 --- a/internal/web/handlers/handlers_notifications_test.go +++ b/internal/web/handlers/handlers_notifications_test.go @@ -13,7 +13,7 @@ import ( "github.com/mixeme/selfpost/internal/web/auth" ) -// The panel pages of notifications (plans/panel-notifications.md): the user's +// The panel pages of notifications (docs/architecture.md § Notifications): the user's // checkbox on Account, the instance's switch and the DNS records of the server's // own name on Server › Settings, and the state of those records on Server › // Health. What is sent and when is internal/notify's and tested there. diff --git a/test/e2e/mail_helpers.go b/test/e2e/mail_helpers.go index c58d767..003b76b 100644 --- a/test/e2e/mail_helpers.go +++ b/test/e2e/mail_helpers.go @@ -88,8 +88,9 @@ func verifyDKIM(raw []byte, domain string) error { return nil } -// checkPanelSignature is the oracle of docs/plans/panel-dkim-signing.md: the -// message carries one DKIM signature, the panel's, for domain - signed as +// checkPanelSignature is the oracle of the panel's DKIM signer +// (docs/architecture.md § DKIM signing): the message carries one DKIM +// signature, the panel's, for domain - signed as // OpenDKIM signed until 2.8.0 (relaxed/simple, rsa-sha256, From over-signed) // - and dkimpy (python3-dkim in the sink image), an implementation that is // not go-msgauth, the panel's library, verifies it. Stage 1 compared it with diff --git a/test/e2e/port25_config_check.go b/test/e2e/port25_config_check.go index b9bbdd9..c7fd33e 100644 --- a/test/e2e/port25_config_check.go +++ b/test/e2e/port25_config_check.go @@ -54,7 +54,7 @@ var port25Cases = []port25Case{ // The SMTP-smuggling defence. Postfix 3.10.13 has it by default // (src/global/mail_params.h, DEF_SMTPD_FORBID_BARE_LF "normalize"); // set explicitly since 2.6.1, when the image ran Postfix 3.7.11, - // whose default was "no" (own-mta-builds, § 5), and kept so the + // whose default was "no" (docs/security.md § Mail path), and kept so the // listener does not depend on the default. testInboundBareNewline // drives it. {param: "smtpd_forbid_bare_newline", contains: "normalize", wantExact: true}, diff --git a/test/e2e/sink/Dockerfile b/test/e2e/sink/Dockerfile index 4b10c0d..5a93702 100644 --- a/test/e2e/sink/Dockerfile +++ b/test/e2e/sink/Dockerfile @@ -4,7 +4,7 @@ # the panel's signer added) straight off disk. smtp-sink ships in the postfix # package itself. python3-dkim (dkimpy 1.1.8-2 in trixie, /usr/bin/dkimverify) # is the independent DKIM verifier the suite holds the panel's signer to -# (docs/plans/panel-dkim-signing.md); it looks the keys up through the +# (docs/architecture.md § DKIM signing); it looks the keys up through the # stand's CoreDNS (compose.override.yml, sink's dns). FROM debian:trixie-slim