Owner, 2026-10-18: "Надо почистить документы от устаревшего и сделанного." - Deleted, shipped: plans/delivery-log-storage, from-header-binding, inbound-antispam-panel, inbound-upstream-result, panel-dkim-signing, panel-notifications, and the own-mta-builds analysis (its three rows shipped as 2.6.1, 2.7.0, 2.8.0). What outlived them moved first: when a filter-list counter is worth a migration, and the notifications' "when" table, into architecture.md; the sender check's own-process trigger into security.md. Code comments that cited them point at architecture.md or security.md; CHANGELOG keeps their names, unlinked. - Kept, marked closed: audit-2026-10-12, assumptions-2026-10-14 and security-audit-2026-10-17 - the code cites their finding ids in 50-84 files. Their file:line refs are of the tree each audited. - roadmap.md: open work only - stage H of the security audit (the owner's), go-milter-upstream (the owner posts), and queue-table, which the redesign left undecided and only its own shipped section still named; Dropped and Ideas as they were. 2.x versioning instead of 1.x. - development.md: the 47-row planned-version-cuts table (history, in the CHANGELOG) gone; a stage's last step now removes its roadmap item and plan. Docs map lists the closed audits. - panel-redesign.md: the done checklist out; Room for roadmap items says which drawn screens are unbuilt and why. - Stale facts fixed: schema-migrations.md and architecture.md at schema 14 (0013's users columns and user_links, 0014's secure TLS mode); security.md's accepted risk "profile saved without the password" - the e-mail asks for it since 2.14.0. Tests: go build, go vet; go test of cmd and the touched internal packages pass except logtail's three Windows-bound rotation tests (rename of an open file; CI is Linux). Only comments changed in Go. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Claude Opus 5.5
+10
-10
@@ -377,7 +377,7 @@ Nothing for an operator to do.
|
||||
### Added
|
||||
|
||||
- **Inbound › Log shows what the upstream answered** (inbound-upstream-result,
|
||||
[the plan](docs/plans/inbound-upstream-result.md)): an *Upstream* column
|
||||
the plan, since removed): an *Upstream* column
|
||||
beside the decision - *queued*, *held*, *sent*, *deferred* (a greylisting, an
|
||||
upstream that cannot be reached), *bounced* (refused, or expired in the
|
||||
queue) - opening the last reply, the host that gave it and when. Read from
|
||||
@@ -456,7 +456,7 @@ schema 11.
|
||||
### Added
|
||||
|
||||
- **The panel signs outgoing mail with DKIM itself** (panel-dkim-signing,
|
||||
[the plan](docs/plans/panel-dkim-signing.md)): a milter of the panel's on
|
||||
the plan, since removed): a milter of the panel's on
|
||||
two sockets, one for applications' mail - signed with the key of the
|
||||
domain the login belongs to - and one for the panel's own, signed with the
|
||||
server's name. `relaxed/simple`, RFC 6376 § 5.4.1's headers and `From`
|
||||
@@ -643,7 +643,7 @@ No migration; schema 11.
|
||||
The listener now sets `normalize` - the setting Postfix recommends and 3.9
|
||||
made the default: DATA ends only at the standard `<CR><LF>.<CR><LF>`, a
|
||||
bare newline elsewhere is read as a line end. Found while reading for
|
||||
[own-mta-builds](docs/own-mta-builds-2026-10-15.md) (§ 5); `test/e2e`
|
||||
own-mta-builds (§ 5, since removed); `test/e2e`
|
||||
sends the sequence and finds one message at the upstream with all of it
|
||||
inside. Only installations with `INBOUND_RELAY_ENABLE=true` or
|
||||
`DMARC_REPORTS_ENABLE=true` have the listener.
|
||||
@@ -1359,7 +1359,7 @@ Documentation only; no product code changes.
|
||||
held to - any address of the application's domain, or the addresses of its
|
||||
list, with or without a `+extension` - and refuses anything else with
|
||||
`550 5.7.1`, a domain this server does not sign for included
|
||||
([plans/from-header-binding.md](docs/plans/from-header-binding.md)).
|
||||
(plans/from-header-binding.md, since removed).
|
||||
- **The check is strict.** It is the first milter of the chain, on a socket
|
||||
of its own (`SENDER_CHECK_SOCKET`), attached with
|
||||
`default_action=tempfail`: while the panel does not answer, Postfix defers
|
||||
@@ -1413,7 +1413,7 @@ Documentation only; no product code changes.
|
||||
a user's own switch, `users.notify`, on for everyone; `notify_state`, what
|
||||
the panel has already written to its users about. The instance switch is a
|
||||
settings key, on unless turned off
|
||||
([plans/panel-notifications.md](docs/plans/panel-notifications.md)).
|
||||
(plans/panel-notifications.md, since removed).
|
||||
- **The panel writes to its users when something needs them.** Four things,
|
||||
each said once: a sending domain's DKIM, SPF or DMARC record stops matching
|
||||
(after two hourly checks in a row, and once more when it matches again); a
|
||||
@@ -1463,7 +1463,7 @@ Documentation only; no product code changes.
|
||||
The table that will hold what Postfix wrote about a message for as long as
|
||||
its send-log row lives, with the store methods to write a line, to write it
|
||||
once, to read the lines of a row and to prune
|
||||
([plans/delivery-log-storage.md](docs/plans/delivery-log-storage.md)).
|
||||
(plans/delivery-log-storage.md, since removed).
|
||||
- **The log tailer keeps what Postfix writes about queued messages.** A
|
||||
`mail.log` line headed by a queue id is stored with the line's own time, one
|
||||
transaction per pass over the log; after an unclean stop a line read twice
|
||||
@@ -2063,7 +2063,7 @@ screens. Nothing in `internal/web` changes; the design stays `candidate`
|
||||
scrolling at 1280px and up. Documented on the components page and in the
|
||||
plan (§ Type and tables).
|
||||
- Panel redesign, header comparison sheet
|
||||
([docs/assets/panel-redesign/header/](docs/assets/panel-redesign/header/index.html)):
|
||||
(`docs/assets/panel-redesign/header/`, since removed):
|
||||
at nav size the stamp dissolves into the brick bar and the perforated edge
|
||||
was drawn inverted (round bumps instead of round holes). Three replacements
|
||||
beside the current header — **A** wordmark with the stamp's real hole-to-tooth
|
||||
@@ -2137,7 +2137,7 @@ screens. Nothing in `internal/web` changes; the design stays `candidate`
|
||||
grammar (`measure` / `fill` plus Web Awesome layout CSS), no JS component
|
||||
library, new mockups rather than `docs/assets/panel-ui/`. Status: candidate.
|
||||
- Roadmap item and plan for `/preflight` installation check page
|
||||
([docs/plans/preflight.md](docs/plans/preflight.md)) — instance-level
|
||||
(`docs/plans/preflight.md`, since removed) — instance-level
|
||||
infrastructure verification (rDNS, TLS, ports, HELO banner, proxy headers,
|
||||
DKIM socket, test email). Status: candidate.
|
||||
|
||||
@@ -2311,7 +2311,7 @@ antispam and quarantine. Upgrading from 1.9.0 is a tag bump; no migration.
|
||||
|
||||
### Added
|
||||
|
||||
- [docs/plans/inbound-antispam-panel.md](docs/plans/inbound-antispam-panel.md) —
|
||||
- `docs/plans/inbound-antispam-panel.md` (since removed) —
|
||||
agreed plan for inbound antispam journal (inbound-journal milter + mail.log
|
||||
rejects) and instance-wide allow/deny lists synced to rspamd maps; target
|
||||
`1.10.0`; Composer → Opus → Fable workflow.
|
||||
@@ -2511,7 +2511,7 @@ tag bump; no schema migration.
|
||||
self-contained backups and persistent queue.
|
||||
|
||||
- docs: HTML mockups for a full panel UI refresh
|
||||
([docs/assets/panel-ui/](docs/assets/panel-ui/index.html)) — current screens,
|
||||
(`docs/assets/panel-ui/`, since removed) — current screens,
|
||||
agreed and candidate roadmap surfaces (queue-retries, inbound-relay,
|
||||
dmarc-reports, panel-docs), a hybrid width (ops pages use the window, forms
|
||||
keep a reading measure), and a 390px emergency layout. The stamp and brick
|
||||
|
||||
@@ -44,7 +44,7 @@ send log and DNS checks in the panel, encrypted backups.
|
||||
| [Architecture](docs/architecture.md) | As-built technical design |
|
||||
| [Security design](docs/security.md) | Mandatory requirements, accepted risks, the CSRF ADR |
|
||||
| [Development](docs/development.md) | Building, testing, docs rules, model routing, commits |
|
||||
| [Roadmap](docs/roadmap.md) | Open work (1.x+) — direction, not commitments |
|
||||
| [Roadmap](docs/roadmap.md) | Open work — direction, not commitments |
|
||||
| [CHANGELOG](CHANGELOG.md) | Release history |
|
||||
|
||||
Found a vulnerability? Do not open an issue — [SECURITY.md](SECURITY.md) has
|
||||
|
||||
+1
-1
@@ -57,7 +57,7 @@ ENV DEBIAN_FRONTEND=noninteractive
|
||||
# test say whether it does.
|
||||
# No DKIM package is installed: the panel signs itself (internal/milter,
|
||||
# sign.go) and makes the keys in Go (internal/domain/dkim.go); OpenDKIM left
|
||||
# the image in 2.8.0 (docs/plans/panel-dkim-signing.md).
|
||||
# the image in 2.8.0 (docs/architecture.md § DKIM signing).
|
||||
# systemd-standalone-sysusers is named so that cron-daemon-common, which
|
||||
# logrotate brings (logrotate 3.22.0-1, debian/control: "Depends: cron |
|
||||
# anacron | cron-daemon | systemd-sysv"), creates its group with it rather
|
||||
|
||||
+1
-1
@@ -111,7 +111,7 @@ find /data -mindepth 1 -maxdepth 1 ! -name log ! -name postfix ! -name sasl ! -n
|
||||
# serves (cmd/panel/signer.go). Until 2.8.0 the keys were under
|
||||
# /data/opendkim, shared with OpenDKIM; 2.8.x copied them to /data/dkim at its
|
||||
# first start and kept the old tree for a rollback to 2.7.x until the next
|
||||
# release, this one (docs/plans/panel-dkim-signing.md, Q2). It is removed
|
||||
# release, this one (CHANGELOG [2.9.0]). It is removed
|
||||
# once /data/dkim holds a key. A /data that never ran 2.8.x - keys under the
|
||||
# old tree only - stops the start instead: the panel would make the server's
|
||||
# own name a new key, and its published record would no longer match; 2.8.x
|
||||
|
||||
@@ -426,7 +426,7 @@ postconf -e \
|
||||
# deferred (docs/security.md).
|
||||
#
|
||||
# The panel's DKIM signer follows, strict too: unreachable, it defers rather
|
||||
# than send unsigned (docs/plans/panel-dkim-signing.md). It signs with the
|
||||
# than send unsigned (docs/architecture.md § DKIM signing). It signs with the
|
||||
# key of the login's domain. The journal-milter is monitoring only and is
|
||||
# fail-open (default_action=accept): its failure must never block the relay.
|
||||
# Per-milter settings use Postfix 3.0+ brace syntax.
|
||||
@@ -756,7 +756,7 @@ if [ "${INBOUND_RELAY_ENABLE}" = "true" ] || [ "${DMARC_REPORTS_ENABLE}" = "true
|
||||
# only here (above), so no client is spared. The submission ports take
|
||||
# the default, normalize as well since this Postfix: an application whose
|
||||
# SMTP library ends DATA with a bare <LF>.<LF> is no longer read to its
|
||||
# end there (own-mta-builds, § 5: their smuggled transaction would still
|
||||
# end there (docs/security.md § Mail path: their smuggled transaction would still
|
||||
# be the authenticated session's, so they were left to the default).
|
||||
postconf -P \
|
||||
"smtp/inet/smtpd_forbid_bare_newline=normalize" \
|
||||
|
||||
@@ -24,7 +24,7 @@ const aliveEvery = 2 * time.Second
|
||||
// try again later"), so while this role is not serving, mail is deferred:
|
||||
// what that costs is counted here and
|
||||
// by the log tailer, and shown on Server > Health
|
||||
// (docs/plans/from-header-binding.md § Statistics).
|
||||
// (docs/architecture.md § Sender check).
|
||||
func serveSenderCheck(ctx context.Context, cfg config, st *store.Store) error {
|
||||
ln, err := listenMilter(cfg.senderSocket)
|
||||
if err != nil {
|
||||
|
||||
+28
-14
@@ -177,9 +177,15 @@ smtp/inet` removes Debian's stock smtpd). Outbound delivery still uses the
|
||||
off the journal on each view of Filter lists - the rows a list decided (the
|
||||
deny rule's reply, the allow rule's symbol, which the journal milter keeps
|
||||
ahead of its six-symbol cap) matched against the entries as rspamd matches
|
||||
them. When that stops being good enough is written in
|
||||
[plans/inbound-antispam-panel.md](plans/inbound-antispam-panel.md) § Filter
|
||||
list usage.
|
||||
them. So a count reaches back as far as the journal does, a row counts for
|
||||
every fitting entry of the deciding list, and a count is of messages,
|
||||
roughly. A counter kept with each entry (hits and last hit on
|
||||
`inbound_spam_list`, a migration) is worth it when any one of these holds
|
||||
(owner, 2026-10-12): the journal is cut by its 10 000-row cap rather than by
|
||||
age (the tailer's prune keeps removing rows while the oldest row of
|
||||
Inbound › Log is younger than the retention); the retention is set short,
|
||||
towards its 7 days; counts are wanted that outlive the journal; or the page
|
||||
is slow - past a few hundred entries, measure it.
|
||||
**Inbound relay** (`INBOUND_RELAY_ENABLE=true`): accepts only
|
||||
`relay_domains` + `relay_recipient_maps` (`reject_unauth_destination`,
|
||||
`reject_unlisted_recipient`). Maps under `/data/postfix/` (`relay_domains`,
|
||||
@@ -235,8 +241,7 @@ smtp/inet` removes Debian's stock smtpd). Outbound delivery still uses the
|
||||
The panel signs outgoing mail itself, in a role of its own, `dkim-signer`
|
||||
([cmd/panel/signer.go](../cmd/panel/signer.go),
|
||||
[internal/milter/sign.go](../internal/milter/sign.go)), since `2.8.0`; until
|
||||
then OpenDKIM did, in a program of its own
|
||||
([plans/panel-dkim-signing.md](plans/panel-dkim-signing.md)).
|
||||
then OpenDKIM did, in a program of its own (CHANGELOG `[2.8.0]`).
|
||||
|
||||
- **Keys** — `/data/dkim/keys/<name>/<selector>.private`, PKCS#1 PEM, one
|
||||
for each sending domain and one for the server's own name
|
||||
@@ -353,7 +358,7 @@ milter, § Postfix above, port 25):
|
||||
(`300s` / `4000s` / `5d` / `0`) with a muted note on the box.
|
||||
2. **sender check** — unix socket `SENDER_CHECK_SOCKET`, first in the chain
|
||||
and **strict** (`default_action=tempfail`): § Sender check below.
|
||||
3. **DKIM signer** (`2.8.0`, [plans/panel-dkim-signing.md](plans/panel-dkim-signing.md))
|
||||
3. **DKIM signer** (`2.8.0`, § DKIM signing)
|
||||
— unix sockets `DKIM_SIGN_SOCKET` (applications' mail on 465/587, signed
|
||||
with the key of the domain the SASL login belongs to) and
|
||||
`DKIM_SIGN_LOCAL_SOCKET` (the panel's own mail, `non_smtpd_milters`,
|
||||
@@ -397,8 +402,8 @@ generates pass no milter.
|
||||
### Sender check
|
||||
|
||||
[internal/milter/sender.go](../internal/milter/sender.go) holds a message's
|
||||
`From:` header against the application that sends it
|
||||
([plans/from-header-binding.md](plans/from-header-binding.md)). Postfix holds
|
||||
`From:` header against the application that sends it (since `2.4.0`;
|
||||
`Sender:` too since `2.8.0`). Postfix holds
|
||||
the envelope sender against the SASL login, and the signer signs with the key
|
||||
of the login's domain (§ DKIM signing): the signature's `d=` is that domain,
|
||||
and a receiver holds it to the domain of `From:` (DMARC, RFC 7489 § 3.1.1).
|
||||
@@ -504,10 +509,15 @@ the hole it closed was the same.
|
||||
[internal/notify](../internal/notify/notify.go) reports four things to the
|
||||
users of the panel: a sending domain's DNS record that stopped matching, a
|
||||
level-2 rate limit that refused mail, the TLS certificate about to expire, a
|
||||
message more than three days in the queue. The rules of when a message goes
|
||||
out are the owner's and are written in
|
||||
[plans/panel-notifications.md](plans/panel-notifications.md) § When a message
|
||||
is sent; the operator's view is guide § Notifications.
|
||||
message more than three days in the queue. The operator's view is guide
|
||||
§ Notifications. When a message goes out is the owner's rule (2026-10-12):
|
||||
|
||||
| Subject | Sent | "Cleared" message |
|
||||
|---|---|---|
|
||||
| DNS mismatch | when it has held for an hour (two checks, below); a resolver that did not answer is not a mismatch | the same way back |
|
||||
| Certificate | 14 days before expiry, and 3 days before if it is still the same certificate | none |
|
||||
| Level-2 limit refused mail | at the first refusal for a sending domain, then at most once a day for it | none |
|
||||
| Queue older than three days | when such a message appears | one, when none is left |
|
||||
|
||||
- **Readings.** Once an hour, starting three minutes after the process does:
|
||||
every sending domain's records through a DNS checker of the notifier's own,
|
||||
@@ -1179,7 +1189,7 @@ full backups also archive the operator's `./certs` PEM files when present.
|
||||
### Database
|
||||
|
||||
The schema is an embedded baseline, `internal/store/migrations/0001_init.sql`,
|
||||
and the migrations after it (`0002` to `0012` today; the next is `0013`),
|
||||
and the migrations after it (`0002` to `0014` today; the next is `0015`),
|
||||
applied by `store.Open` in file order; `PRAGMA user_version` counts the files
|
||||
applied and
|
||||
`PRAGMA application_id` is `0x53503230` ("SP20"). There is no 1.x migration
|
||||
@@ -1198,7 +1208,11 @@ The tables that carry the panel's model:
|
||||
the user's own address, edited on Account and set by the global role on the
|
||||
user form; `all_domains` / `all_inbound_domains` widen a *domain* user to
|
||||
every domain of that direction, including ones added later (with a flag set
|
||||
the assignment rows are ignored).
|
||||
the assignment rows are ignored); `email_confirmed_at` is when `email` was
|
||||
confirmed, `''` while it is not (v13).
|
||||
- **`user_links`** — the one-time links that confirm an address and reset a
|
||||
password (v13): the token's SHA-256, `purpose`, the address it went to, its
|
||||
expiry; one live link per user and purpose.
|
||||
- **`user_domains`** and **`user_inbound_domains`** — the outbound and inbound
|
||||
domains assigned to a *domain* user. Separate tables: the same name on both
|
||||
is two assignments, and a sending domain gives no inbound access or the other
|
||||
|
||||
@@ -1,5 +1,12 @@
|
||||
# What SelfPost assumes of other programs - the register of 2026-10-14
|
||||
|
||||
> **Closed 2026-10-14** (stage 2c of [audit-2026-10-12](audit-2026-10-12.md),
|
||||
> shipped `2.5.2`): every entry answered, the citations written into the
|
||||
> code, what the sources contradicted fixed. The file stays because the code
|
||||
> names its entries (`assumptions C-V5`, `A-43`); its `file:line` references
|
||||
> are of the tree at `0f984ff`, not of today's, and OpenDKIM, named
|
||||
> throughout, left the image in `2.8.0`.
|
||||
|
||||
The heuristics review of audit-2026-10-12, stage 2c. The owner's rule
|
||||
(development.md § "Code that relies on another component's behaviour"):
|
||||
what Postfix, OpenDKIM, Cyrus SASL, rspamd, supervisord, logrotate, Docker,
|
||||
|
||||
@@ -1,5 +1,10 @@
|
||||
# Code audit — 2026-10-12
|
||||
|
||||
> **Closed 2026-10-15.** Every stage shipped (`2.4.2` to `2.6.0`; stage 6,
|
||||
> documents and tests, without a cut). The file stays because the code cites
|
||||
> its finding ids (`H4`, `M2`, `L9`...); its `file:line` references are of the
|
||||
> tree at `8f226bd`, not of today's.
|
||||
|
||||
**What this is.** A full-tree audit of the code and infrastructure at `8f226bd`
|
||||
(release `2.4.0`): leftovers, unfinished work, defects, performance, doc drift
|
||||
and test gaps — explicit ones (there are no `TODO`/`FIXME` markers anywhere in
|
||||
@@ -1570,7 +1575,7 @@ tested (the count and the spared are).
|
||||
sleeps said where they are); roadmap.md's closing paragraph. Already right
|
||||
by then: the `.alive` mark, `dmarc-ingest` as a second writer, the chain
|
||||
head, M1's wording, the version stamp.
|
||||
- [ ] roadmap.md: done ahead on 2026-10-13/14 with the owner -
|
||||
- [x] roadmap.md: done ahead on 2026-10-13/14 with the owner -
|
||||
`template-data-typing` struck, `structured-logging` dropped,
|
||||
`review-2026-08-followups` closed with each row placed in a stage or
|
||||
dropped with its reason. What is left: check that every *Test gaps* entry
|
||||
|
||||
+12
-58
@@ -1,7 +1,7 @@
|
||||
# SelfPost — development
|
||||
|
||||
**What this file is.** How to build, test, document, and ship changes. Open
|
||||
work after 1.0 (1.x+) lives in [roadmap.md](roadmap.md) and linked
|
||||
work lives in [roadmap.md](roadmap.md) and linked
|
||||
[plans/](plans/). Product boundaries: [product.md](product.md). As-built layout:
|
||||
[architecture.md](architecture.md).
|
||||
|
||||
@@ -90,7 +90,7 @@ checklist changes; update the numerator when steps are checked off.
|
||||
|
||||
### Roadmap track: commit and version
|
||||
|
||||
For the current 1.x+ roadmap queue ([roadmap.md](roadmap.md) index), stricter
|
||||
For the roadmap queue ([roadmap.md](roadmap.md) index), stricter
|
||||
than the default «meaningful step» rule in [§ Commits and release build](#commits-and-release-build):
|
||||
|
||||
**After each checklist step**
|
||||
@@ -106,58 +106,10 @@ than the default «meaningful step» rule in [§ Commits and release build](#com
|
||||
[deploy/docker-compose.yml](../deploy/docker-compose.yml).
|
||||
2. Git tag `vX.Y.Z` and publish the GitHub Release only on explicit request (see
|
||||
[§ Release image](#release-image)).
|
||||
|
||||
**Planned version cuts** (from pin `2.15.0`; adjust if semver changes mid-track):
|
||||
|
||||
| Stage | ID | Cut |
|
||||
|---|---|---|
|
||||
| 1 | queue-retries | `1.3.1` (PATCH) |
|
||||
| 2 | inbound-relay | `1.4.0` |
|
||||
| 3 | send-log-retention | `1.5.0` |
|
||||
| 4 | domain-stats-auto-ratelimit | `1.6.0` |
|
||||
| 5 | dmarc-reports | `1.7.0` |
|
||||
| 6 | panel-docs | `1.8.0` |
|
||||
| 7 | application auth IPs | `1.9.0` |
|
||||
| 8 | port-25 Postfix startup fix | `1.9.1` (PATCH) |
|
||||
| 9 | 2026-08-19 code review pass | `1.9.2` (PATCH) |
|
||||
| 10 | panel-redesign prep (docs only, owner-requested cut) | `1.9.3` (PATCH) |
|
||||
| 11 | panel-redesign plan: component kit, stages, reorder (docs only, owner-requested cut) | `1.9.4` (PATCH) |
|
||||
| 12 | GHCR release fix: keep per-arch images, sweep orphaned versions (CI only) | `1.9.5` (PATCH) |
|
||||
| 13 | panel-redesign | `2.0.0` (MAJOR, no compatibility) |
|
||||
| 14 | DMARC ingest drops and bounds, full backup from the panel, no queue in a backup | `2.0.1` (PATCH) |
|
||||
| 15 | inbound-antispam-panel | `2.1.0` (was `1.10.0`, reordered 2026-10-09) |
|
||||
| 16 | delivery-log-storage | `2.2.0` |
|
||||
| 17 | panel-notifications | `2.3.0` |
|
||||
| 18 | from-header-binding | `2.4.0` |
|
||||
| 19 | code audit 2026-10-12 and its plan (docs only, owner-requested cut) | `2.4.1` (PATCH) |
|
||||
| 20 | audit-2026-10-12 stage 0: release hygiene | `2.4.2` (PATCH) |
|
||||
| 21 | audit-2026-10-12 stages 1 and 2 | `2.5.0` |
|
||||
| 22 | audit-2026-10-12 stage 2b: the log tailer | `2.5.1` |
|
||||
| 23 | audit-2026-10-12 stage 2c: the sender check reads From: as OpenDKIM does | `2.5.2` |
|
||||
| 24 | audit-2026-10-12 stage 3: persistence and restore | `2.5.3` |
|
||||
| 25 | audit-2026-10-12 stage 4: panel behaviour | `2.5.4` |
|
||||
| 26 | audit-2026-10-12 stage 5: image and Postfix hardening | `2.6.0` |
|
||||
| 27 | port25-bare-newline | `2.6.1` (PATCH) |
|
||||
| 28 | base-trixie | `2.7.0` |
|
||||
| 29 | panel-dkim-signing | `2.8.0` |
|
||||
| 30 | dns-record-collapse, settings-save-row, notification-mail-recheck; `/data/opendkim` removed | `2.9.0` |
|
||||
| 31 | log panes wrap on a phone | `2.9.1` (PATCH) |
|
||||
| 32 | inbound-upstream-result | `2.10.0` |
|
||||
| 33 | long, non-repeating queue ids | `2.10.1` (PATCH) |
|
||||
| 34 | phone-desktop-mode | `2.10.2` (PATCH) |
|
||||
| 35 | backup-reauth | `2.10.3` (PATCH) |
|
||||
| 36 | origin-strict | `2.10.4` (PATCH) |
|
||||
| 37 | health-setup-checks | `2.11.0` |
|
||||
| 38 | test-email | `2.12.0` |
|
||||
| 39 | local-time | `2.13.0` |
|
||||
| 40 | docs-audit | `2.13.1` |
|
||||
| 41 | password-reset | `2.14.0` |
|
||||
| 42 | overview-head | `2.14.1` |
|
||||
| 43 | dmarc-day, health-sockets | `2.14.2` |
|
||||
| 44 | label-tag | `2.14.3` |
|
||||
| 45 | security-audit-2026-10-17, stages 0-6 | `2.14.4` |
|
||||
| 46 | etrn-silent | `2.14.5` |
|
||||
| 47 | server-dmarc-hint, upstream-tls-verify | `2.15.0` |
|
||||
3. The item leaves [roadmap.md](roadmap.md) and its plan file is deleted; what
|
||||
it did is in the CHANGELOG, the rest in git history. A report whose
|
||||
finding ids the code cites (the audits, the assumptions register) stays,
|
||||
marked closed.
|
||||
|
||||
Docs-only prep (checklists in plans, no product code) uses the same per-step
|
||||
commit rule but **no** version cut until the next product stage ships.
|
||||
@@ -308,7 +260,8 @@ question, the comment says so and what was measured instead - "measured on
|
||||
the e2e stand, 2026-10-14: ..." - and a measurement is never written as if
|
||||
it were read. A test fixture copied from a real log says where and when it
|
||||
was seen; one composed by hand says that it was composed. The heuristics
|
||||
review of 2026-10-14 (audit-2026-10-12, stage 2c) brought the code that
|
||||
review of 2026-10-14 ([audit-2026-10-12](audit-2026-10-12.md), stage 2c; the
|
||||
[assumptions register](assumptions-2026-10-14.md)) brought the code that
|
||||
existed then up to this rule.
|
||||
|
||||
---
|
||||
@@ -623,7 +576,8 @@ There is no `docs/archive/` directory.
|
||||
| Development process (this file) | [development.md](development.md) |
|
||||
| Security requirements and accepted risks | [security.md](security.md) |
|
||||
| SQLite schema and migrations | [schema-migrations.md](schema-migrations.md) |
|
||||
| Roadmap (1.x+) | [roadmap.md](roadmap.md) |
|
||||
| Roadmap | [roadmap.md](roadmap.md) |
|
||||
| Closed audits, cited by finding id from the code | [audit-2026-10-12.md](audit-2026-10-12.md), [assumptions-2026-10-14.md](assumptions-2026-10-14.md), [security-audit-2026-10-17.md](security-audit-2026-10-17.md) |
|
||||
| Active design plans | [plans/](plans/) |
|
||||
| Release history | [CHANGELOG.md](../CHANGELOG.md) |
|
||||
|
||||
@@ -640,8 +594,8 @@ There is no `docs/archive/` directory.
|
||||
| [deploy/.env.example](../deploy/.env.example) | Public env template; full reference in [guide.md](guide.md) |
|
||||
| [CHANGELOG.md](../CHANGELOG.md) | Keep a Changelog |
|
||||
|
||||
Out of scope for v1.x: `CONTRIBUTING.md`, man pages, a separate docs site
|
||||
(candidates in [roadmap.md](roadmap.md)).
|
||||
Out of scope: `CONTRIBUTING.md` (dropped, with its condition, in
|
||||
[roadmap.md](roadmap.md)), man pages, a separate docs site.
|
||||
|
||||
### Maintaining documentation
|
||||
|
||||
|
||||
@@ -1,276 +0,0 @@
|
||||
# Own builds of Postfix and OpenDKIM - the analysis of 2026-10-15
|
||||
|
||||
**Question (owner, 2026-10-15; roadmap [own-mta-builds](roadmap.md#own-mta-builds)):**
|
||||
is it worth patching Postfix and OpenDKIM and shipping our own builds, to
|
||||
optimise something in SelfPost or to lift a limit we work around?
|
||||
|
||||
**Answer:** no to carrying patched builds of either - nothing we work around
|
||||
needs a patch, and a patch set makes one maintainer the security team of
|
||||
two MTAs. The question does point at two real problems, each with a cheaper
|
||||
answer:
|
||||
|
||||
1. **The image's base is out of date.** `debian:bookworm-slim` left regular
|
||||
security support on 2026-07-12; its Postfix `3.7.11` is code of mid-2024,
|
||||
and the three Postfix security batches of 2026 (June, August, September)
|
||||
carry no CVE ids, so nothing will bring them to bookworm's LTS. The
|
||||
answer is Debian trixie (`postfix 3.10.13`, the August batch included),
|
||||
not a build of our own: proposed as roadmap row
|
||||
[base-trixie](roadmap.md#base-trixie).
|
||||
2. **OpenDKIM has no upstream left.** The package is a beta of 2018; three
|
||||
CVEs of 2026 are open in every Debian suite with "the vendor did not
|
||||
respond"; Debian's bug on them has had no answer since 2026-10-03. Every
|
||||
limit we have worked around is OpenDKIM's. A patched OpenDKIM of our own
|
||||
would make us its maintainer; signing in the panel's own milter, which
|
||||
already runs in the mail path, removes the program and most of what was
|
||||
built around it: proposed as roadmap row
|
||||
[panel-dkim-signing](roadmap.md#panel-dkim-signing).
|
||||
|
||||
A side finding of the reading: port 25 of the inbound relay accepts bare
|
||||
newlines - Postfix 3.7's default for the 2023 SMTP-smuggling parameter -
|
||||
one line fixes it: [port25-bare-newline](roadmap.md#port25-bare-newline).
|
||||
|
||||
Everything below is the evidence and the prices. Versions are those of the
|
||||
image (`docs/development.md`, the cited-package table); upstream source is
|
||||
`dev/upstream/`; web sources are linked where a fact came from one.
|
||||
|
||||
---
|
||||
|
||||
## 1. What the image runs, and who maintains it now
|
||||
|
||||
| Component | In the image (bookworm) | In Debian trixie | Upstream |
|
||||
|---|---|---|---|
|
||||
| Debian | 12 bookworm, LTS since 2026-07-12, until 2028-06-30 ([debian.org](https://www.debian.org/News/2026/20260712.en.html)) | 13 trixie, stable since 2025-08-09 | — |
|
||||
| Postfix | `3.7.11-0+deb12u1` (unchanged since the image was pinned; [tracker](https://tracker.debian.org/pkg/postfix)) | `3.10.13-0+deb13u1` (trixie-security) | `3.11.7`, stable branches 3.8–3.11 ([3.11.7](https://postfix.cs.utah.edu/announcements/postfix-3.11.7.html)) |
|
||||
| OpenDKIM | `2.11.0~beta2-8+deb12u1` | `2.11.0~beta2-9.1` (same code; a gcc-14 build fix, a path move, one verifier-side header patch) | `2.11.0-Beta2`, 2018; no release since ([releases](https://github.com/trusteddomainproject/OpenDKIM/releases), [issue #422](https://github.com/trusteddomainproject/OpenDKIM/issues/422), unanswered) |
|
||||
| libmilter | `8.17.1.9-2+deb12u2` | `8.18.1-6` | — |
|
||||
| Cyrus SASL | `2.1.28+dfsg-10` | `2.1.28+dfsg1-9` (same upstream) | — |
|
||||
| supervisor | `4.2.5-1` | `4.2.5-3` (same upstream) | — |
|
||||
| logrotate | `3.21.0-1` | `3.22.0-1` | — |
|
||||
| Go build stage | `golang:1.26-bookworm` | `golang:1.26-trixie` exists (Docker Hub, 2026-10-10) | — |
|
||||
|
||||
**Postfix in bookworm is 2024 code with the 2026 fixes missing.** Upstream
|
||||
issued three security batches this year, all without CVE ids:
|
||||
|
||||
- June 2026: unbounded reads of SMTP commands, responses and BDAT data
|
||||
("large SMTP inputs"); a TLSA-parsing crash (DANE only - not us).
|
||||
- August 2026 ([3.11.6](https://openwall.com/lists/oss-security/2026/08/20/5)):
|
||||
sixteen items, among them a remote memory exhaustion by many small BDAT
|
||||
commands (smtpd, since 3.4), a remote smtpd crash by DATA without MAIL
|
||||
FROM after a BDAT error, a policy bypass after an
|
||||
`smtpd_end_of_data_restrictions` reject (we set none), and the Milter
|
||||
left in "accept" state by the same bug.
|
||||
- September 2026 ([3.11.7](https://postfix.cs.utah.edu/announcements/postfix-3.11.7.html)):
|
||||
two SMTP-smuggling cases behind `smtpd_proxy_filter` (we use none), a
|
||||
crash under `strict_smtputf8` (not set), a multi-line response over-read
|
||||
reachable by a malicious Milter, a `postqueue` use-after-free.
|
||||
|
||||
Legacy releases 3.7.22 and 3.7.23 exist upstream but, as the announcements
|
||||
say, "do not include the patches for out-of-support Postfix versions" of
|
||||
June. Debian's tracker lists one 2026 CVE for bookworm's Postfix
|
||||
([CVE-2026-43964](https://security-tracker.debian.org/tracker/CVE-2026-43964),
|
||||
a one-byte over-read, `no-dsa`) and no bookworm upload at all in 2026; the
|
||||
LTS team works from CVE ids, and these batches have none. The exposed
|
||||
listeners are ours: 465 and 587 to the Internet, 25 when the inbound relay
|
||||
is on.
|
||||
|
||||
**OpenDKIM has three open CVEs and nobody to fix them.** Debian bug
|
||||
[#1149875](https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1149875)
|
||||
(2026-10-03, severity important, no reply from maintainer or security team
|
||||
as of 2026-10-15) covers
|
||||
[CVE-2026-100888](https://security-tracker.debian.org/tracker/CVE-2026-100888)
|
||||
(`dkim_canon_selecthdrs`, out-of-bounds write),
|
||||
[CVE-2026-100889](https://security-tracker.debian.org/tracker/CVE-2026-100889)
|
||||
(`dkim_qp_decode`, off-by-one) and
|
||||
[CVE-2026-101277](https://security-tracker.debian.org/tracker/CVE-2026-101277)
|
||||
(`dkim_process_set`, the tag tokenizer); all three "vendor was contacted
|
||||
early but did not respond in any way". Read in `dev/upstream/opendkim-2.11.0~beta2`
|
||||
for our configuration (`Mode s`, attached to 465/587 and to the panel's
|
||||
local submission only; never to port 25):
|
||||
|
||||
- `dkim_qp_decode` is called from `dkim_sig_domainok()`,
|
||||
`dkim_sig_getreportinfo()` and `dkim_sig_getidentity()`
|
||||
(`libopendkim/dkim.c:1718, 7712, 7732, 7801`) - verifier-side functions,
|
||||
reached from `dkim_eoh_verify()`. Not on our path.
|
||||
- `dkim_process_set` runs on every `DKIM-Signature:` header of a message
|
||||
**in either mode**: `dkim_header()` (`dkim.c:6447`, the call at `:6625`)
|
||||
has no mode gate. On our path that header comes only from an authenticated
|
||||
application; the exposure is a tenant's, not the Internet's.
|
||||
- `dkim_canon_selecthdrs` runs when signing as well as when verifying
|
||||
(`dkim_canon_runheaders()`, `dkim-canon.c:1194, 1280`). The write past
|
||||
the array is `lhdrs[shcnt] = NULL` (`dkim-canon.c:1052`) once every
|
||||
header of the message has been matched and a further name follows. When
|
||||
verifying, the names are the signature's `h=`, the attacker's. When
|
||||
signing, OpenDKIM composes the list itself - the headers present that
|
||||
are in `SignHeaders` (by default RFC 6376 § 5.4's SHOULD list,
|
||||
`dkim_should_signhdrs[]`, `dkim.c:223`) plus `OversignHeaders`, our
|
||||
`From` - so the condition is a message whose every header is in that
|
||||
list. Whether Postfix's own `Received:` keeps it out of reach I did not
|
||||
run; it is one ASan run with `miltertest` on the stand to know, and the
|
||||
answer changes only how soon § 4's row is wanted, not whether.
|
||||
|
||||
A patch for each exists in the researchers' write-ups (for `dkim_qp_decode`:
|
||||
[weitongli.com](https://weitongli.com/share/opendkim-qp-off-by-one.html)).
|
||||
Nobody upstream will take them; Debian may yet carry them - the bug is
|
||||
twelve days old - but the project behind the package is gone, and the
|
||||
next finding meets the same silence.
|
||||
|
||||
## 2. The limits met so far, and what would lift each
|
||||
|
||||
| Limit (where it is worked around) | Own patched build | Debian trixie | Signing in the panel |
|
||||
|---|---|---|---|
|
||||
| OpenDKIM logs through syslog(3) only (`internal/syslogrelay`, a supervisord program, `/dev/log` in the entrypoint; audit L30) | yes, a stderr patch | no | yes - the program goes, and the relay with it (nothing else in the container uses syslog(3)) |
|
||||
| The panel reads Postfix's state from `mail.log`, `postconf -h -x`, `postqueue -p`/`-j` | no: these *are* Postfix's interfaces; a patch would invent a private one to maintain | partly: 3.11 adds `postconf -j` (JSON) - not in trixie's 3.10 | no (unchanged) |
|
||||
| The sender check reads `From:` the way OpenDKIM does: `dkim_mail_parse()` ported (`internal/milter/mailparse.go`, 327 lines), the 256-byte `MAXADDRESS` refusal, two diff/probe scripts against the C | no: the port exists because two programs parse the line; a patch does not merge them (OpenDKIM's `SenderMacro` could pick the key by the envelope instead - it changes nothing, since the check already pins `From:` to the login's senders) | no | yes: one parser, ours, and the key chosen by the login's domain, not by `From:` - the class "signed with another tenant's key" is gone by construction; `From:` still has to be the application's (DMARC alignment), with `net/mail` |
|
||||
| The image held to bookworm's versions (Postfix 3.7.11, OpenDKIM 2.11.0~beta2) | yes, at the price in § 3 | yes for Postfix (3.10.13, security-tracked to 2028 and beyond in trixie LTS); OpenDKIM's version is the same everywhere | OpenDKIM's version stops mattering |
|
||||
| A socket file outlives a crashed OpenDKIM (`health/socket.go`, assumptions #38) | yes, with a patch | no | yes - no second milter process |
|
||||
| Keys group-readable, `RequireSafeKeys no`, the `selfpost` group between panel and OpenDKIM | no | no | yes - the panel reads its own keys; the group stays for Postfix's sasldb and maps |
|
||||
| `SIGUSR1` through supervisord to reload the tables, which is what needs `CAP_KILL` in the compose file (cross-uid kill) | no | no | yes - no tables, no signal; `CAP_KILL` can probably go (to verify on the stand: supervisord then signals only root's wrapper and the panel's own uid) |
|
||||
| `smtpd_forbid_bare_newline` defaults to `no` on Postfix < 3.9 (`src/smtpd/smtpd.c:807`); our port 25 does not set it | yes | yes - the default is `normalize` since 3.9 | no (it is Postfix's) - see § 5 |
|
||||
|
||||
Nothing in the Postfix column needs a patch. The limits are OpenDKIM's.
|
||||
|
||||
## 3. The options, priced
|
||||
|
||||
Prices are estimates in stages of the kind the 2026-10-12 audit used (a
|
||||
stage: write, tests, two reviews, e2e, rollout, commit). Nothing here was
|
||||
measured on the stand; where a number matters it says so.
|
||||
|
||||
### A. Patched builds of both, carried by us - **not recommended**
|
||||
|
||||
- *Gives:* the stderr patch for OpenDKIM, the three CVE fixes, any option
|
||||
compiled in or out, the newest Postfix on the day we rebuild.
|
||||
- *Costs, once:* a build stage per program (`apt-get source`, quilt, a
|
||||
patch series, `dpkg-buildpackage`), its build-dependencies, the e2e
|
||||
`checkCitedPackageVersions` rewritten for our version strings, every
|
||||
citation of `3.7.11` (331 lines in 65 files) and `2.11.0~beta2` (55 in
|
||||
14) re-pointed at our tree. Image build time is not measured; from the
|
||||
size of the sources it is minutes, not seconds, on every rebuild.
|
||||
- *Costs, for ever:* Postfix issued seven releases in 2026; each is ours
|
||||
to notice, rebase the series on, rebuild and roll out, with no Debian
|
||||
security team between the announcement and the image. For OpenDKIM there
|
||||
is no upstream to rebase on at all: the series *is* the program, and
|
||||
the next CVE is ours to find and fix. One maintainer, two MTAs.
|
||||
- *Why not:* § 2 shows no Postfix limit a patch lifts, and every OpenDKIM
|
||||
limit lifted by option E without taking OpenDKIM on.
|
||||
|
||||
### B. Unpatched upstream build where the distribution's version is the limit - **not recommended**
|
||||
|
||||
- *Gives:* Postfix 3.11.7 today instead of trixie's 3.10.13 - the
|
||||
September batch two to four weeks earlier (trixie-security carried 3.10.11
|
||||
→ 3.10.12 → 3.10.13 between July and August; 3.10.14 is upstream now).
|
||||
For OpenDKIM there is no newer upstream to build.
|
||||
- *Costs:* A's build stage and watching duty for Postfix, minus the patch
|
||||
series; Debian's own patches (paths, SASL and LMDB build options)
|
||||
become ours to reproduce.
|
||||
- *Why not:* the gap to trixie-security is weeks, the duty is permanent.
|
||||
|
||||
### C. A patch sent upstream instead of carried - **nothing to send**
|
||||
|
||||
- Postfix: no code change is needed (§ 2); Wietse Venema's project is
|
||||
responsive, but we have no patch for it.
|
||||
- OpenDKIM: the project does not answer CVE reporters, the Debian
|
||||
maintainer or its own issue tracker. A patch sent there is a patch
|
||||
carried - option A.
|
||||
|
||||
### D. The base image to Debian trixie - **recommended, next code row** → [base-trixie](roadmap.md#base-trixie)
|
||||
|
||||
- *Gives:* Postfix 3.10.13 with the June and August batches and the
|
||||
September one when trixie-security ships 3.10.14; Debian's security team
|
||||
until 2028, then trixie LTS; `smtpd_forbid_bare_newline=normalize` by
|
||||
default; `smtpd_hide_client_session` (3.10: a `Received:` header without
|
||||
the application's address - a product question of its own, not decided
|
||||
here); OpenSSL 3.5; libmilter 8.18.1.
|
||||
- *Costs:* one stage, one MINOR release. The work is the re-reading: the
|
||||
331 citation lines against the 3.10.13 tree (most will hold word for
|
||||
word; `src/` paths are stable), with attention to what 3.8–3.10 changed
|
||||
on our paths - reject logging now carries `sasl_method`, `sasl_username`,
|
||||
`sasl_sender` (3.10, 20241023: `refusals.go`, `inbound.go` regexes), the
|
||||
queue id or `NOQUEUE` on an abnormal connection end and cleanup's
|
||||
`canceled` line (3.10: new line shapes for the log tailer), `postfix(1)`
|
||||
logging to stderr when it is not a terminal (3.8: the wrapper's output
|
||||
under supervisord), DNS results capped at 100 records and stray CR/LF
|
||||
replaced by default (3.9), the removed parameters (3.9:
|
||||
`permit_naked_ip_address`, `reject_maps_rbl`, `check_relay_domains` - we
|
||||
set none). Then: `dev/upstream` refetched from trixie, the cited-package
|
||||
table and `checkCitedPackageVersions`, the e2e sink image, the build
|
||||
stage tag, `docs/assumptions-2026-10-14.md`'s header, the guide's
|
||||
"what the image contains". OpenDKIM's citations do not change (same
|
||||
upstream code).
|
||||
- *Risk:* a log line we parse changing shape unnoticed - the e2e suite
|
||||
and the fixtures labelled "seen" are the net; a day on the test server
|
||||
before the cut.
|
||||
|
||||
### E. DKIM signing in the panel's own milter; OpenDKIM removed - **recommended, after D** → [panel-dkim-signing](roadmap.md#panel-dkim-signing)
|
||||
|
||||
- *Gives:* every OpenDKIM row of § 2; about 1,500 lines gone (the relay
|
||||
and its tests, the parser port and its tests, the probe scripts, the
|
||||
table renderer and its reload, `opendkim.conf`), two programs fewer
|
||||
under supervisord, two packages fewer in the image, no CVE watch on a
|
||||
dead project, probably `CAP_KILL` dropped. Keys, selectors and the
|
||||
published DNS records are unchanged: the panel already generates the
|
||||
keys (`internal/domain/dkim.go`, PKCS#1 PEM) - nothing for an operator
|
||||
to do.
|
||||
- *How:* `github.com/emersion/go-msgauth/dkim` (v0.7.0, 2025-04, MIT; the
|
||||
author of the `go-milter` we run; the e2e suite already verifies with it).
|
||||
Read for this analysis: `SignOptions` gives `relaxed`/`simple` per side,
|
||||
`HeaderKeys` with over-signing as RFC 6376 § 5.4.2 wants it (a name
|
||||
listed once more than present is kept in `h=` and nothing is hashed for
|
||||
it, `sign.go`), last-instance-first header picking (`header.go`), RSA and
|
||||
Ed25519, and a streaming `NewSigner` (the body is hashed as it passes,
|
||||
not buffered). The milter: headers and body chunks in, `InsertHeader(0,
|
||||
"DKIM-Signature", …)` at end of message - the shape of go-msgauth's own
|
||||
`cmd/dkim-milter`, some 450 lines. The key: by the login's domain on
|
||||
465/587 (`Bindings.DomainOf`, which the sender check has), by the
|
||||
server's name for the panel's local submission (`non_smtpd_milters`
|
||||
pointed at the same socket - strict, as OpenDKIM is today). The header
|
||||
set: `dkim_should_signhdrs[]`, `From` over-signed once; canonicalization
|
||||
`relaxed/simple`, as now.
|
||||
- *Costs:* two to three stages, one MINOR release: the signer and its
|
||||
tests; the removal (code, image, compose, supervisord, entrypoint,
|
||||
Health's liveness set, Resync's steps, docs: architecture § OpenDKIM,
|
||||
security, guide, development's table, assumptions § 3); e2e with an
|
||||
**independent verifier as the oracle** - go-msgauth must not check its
|
||||
own work: `opendkim-testmsg` in the sink image or rspamd's verifier, the
|
||||
way `mailparse-diff.sh` holds the port to the C today - and a message to
|
||||
a real mailbox before the cut.
|
||||
- *Risk:* a signing defect fails DMARC for every tenant at once; the
|
||||
oracle and the canary are the answer. go-msgauth is a small library with
|
||||
a quiet release pace - vendorable if it ever matters.
|
||||
- *Why after D:* D's exposure is remote and unauthenticated (465/587),
|
||||
E's is a tenant's; D is one stage, E is three; neither waits on the
|
||||
other (E re-reads no Postfix citation, D no OpenDKIM one).
|
||||
|
||||
## 4. Recommendation
|
||||
|
||||
In this order, each a roadmap row of its own, none of them an own build:
|
||||
|
||||
1. **[port25-bare-newline](roadmap.md#port25-bare-newline)** - one line in
|
||||
`build/postfix-config.sh`, a PATCH release, now (§ 5).
|
||||
2. **[base-trixie](roadmap.md#base-trixie)** - option D, one stage, MINOR.
|
||||
3. **[panel-dkim-signing](roadmap.md#panel-dkim-signing)** - option E, two
|
||||
to three stages, MINOR.
|
||||
|
||||
Options A, B and C are declined with the arguments above; what would bring
|
||||
A back is a limit of Postfix's that its interfaces cannot answer, and none
|
||||
is known.
|
||||
|
||||
## 5. Side finding: bare newlines on port 25
|
||||
|
||||
Postfix 3.7.11 has the 2023 SMTP-smuggling parameter
|
||||
(`smtpd_forbid_bare_newline`, `src/global/mail_params.h:4249`) with the
|
||||
default `no` "for Postfix < 3.9" (`src/smtpd/smtpd.c:807`); its
|
||||
`RELEASE_NOTES` recommend `normalize`, which 3.9 made the default. The
|
||||
inbound relay's listener (`build/postfix-config.sh`, the `smtp/inet`
|
||||
block) does not set it, so with `INBOUND_RELAY_ENABLE=true` the relay is
|
||||
the receiving side the smuggling research described: a bare `<LF>.<LF>`
|
||||
inside DATA ends the message early and what follows is read as the next
|
||||
transaction. Ports 465 and 587 are not the case that matters - the
|
||||
smuggled transaction is still the authenticated session's, under its
|
||||
sender bindings and milters. The fix is
|
||||
`smtp/inet/smtpd_forbid_bare_newline=normalize` with an e2e check on port
|
||||
25 (a bare-LF DATA, one message delivered); a PATCH release. Trixie's 3.10
|
||||
would set it by default, but the relay should not wait for that row.
|
||||
|
||||
A smaller one: `opendkim-tools` is installed in the image and used by
|
||||
nothing at run time (the key is generated in Go; the probe script runs on
|
||||
the server from source). It goes with option E, or with D if E waits.
|
||||
@@ -1,185 +0,0 @@
|
||||
# Plan: delivery-log-storage
|
||||
|
||||
**Status:** shipped in `2.2.0` (2026-10-12); agreed the same day (owner: option 1 of the roadmap item, "store the lines")
|
||||
**Date:** 2026-10-12
|
||||
**Version:** `2.2.0` MINOR (one DDL migration; a 2.1.0 data directory boots unchanged).
|
||||
|
||||
---
|
||||
|
||||
## Goal
|
||||
|
||||
A message's page (`/outbound/log/{id}`, mockup `out-message`) shows what
|
||||
Postfix wrote about the message for as long as its row is in the send log.
|
||||
Today the *Delivery log* box is filled by a search of the current `mail.log`
|
||||
on every page view, so it is empty for anything sent before the last daily
|
||||
rotation, while the row itself lives for ninety days.
|
||||
|
||||
## Context (as-built)
|
||||
|
||||
- The log tailer ([internal/logtail](../../internal/logtail/logtail.go))
|
||||
already reads every `mail.log` line once, from a persisted offset, to
|
||||
advance delivery statuses.
|
||||
- `logtail.QueueLines` searches the last 4 MB of the current file for one
|
||||
queue id; `Handlers.deliveryLog` calls it per page view.
|
||||
- A message with several recipients is several send-log rows under one queue
|
||||
id; the page of each shows every line of the message, as the search does
|
||||
today.
|
||||
|
||||
## Decisions
|
||||
|
||||
### What is stored
|
||||
|
||||
A `mail.log` line whose head is a queue id - after the timestamp, the host
|
||||
name and `program[pid]:` - goes into a new table with that id, the line's own
|
||||
time (`lineTime`, the same reading the inbound journal uses) and the line as
|
||||
written, cut to 4096 bytes on a character boundary. The head is matched from
|
||||
the start of the line: the tail is the far end's words. `NOQUEUE` and
|
||||
`warning` are not queue ids - the pattern takes Postfix's short form
|
||||
(hexadecimal) and its long form (no vowels) and nothing else.
|
||||
|
||||
The tailer does not ask whether the id belongs to a send-log row before it
|
||||
writes. The first lines of a message (`client=`, `message-id=`) are logged
|
||||
before the journal milter writes the row at end-of-message, so such a check
|
||||
would drop exactly them.
|
||||
|
||||
**Mail that came in on port 25 is left out** (security review, 2026-10-12). A
|
||||
queue id that `postfix/inbound/smtpd` opened is a stranger's message; it never
|
||||
gets a send-log row, and what a stranger can make Postfix log - his own
|
||||
`message-id`, up to the cut - is not stored to be pruned later. The tailer
|
||||
remembers such ids (4096 at most) until qmgr's `removed` or until another
|
||||
listener opens a message under the id. What is still stored without a row is
|
||||
little: the bounces Postfix generates, and port-25 mail met in mid-flight
|
||||
after a restart.
|
||||
|
||||
Lines are written in one transaction per pass over the log, 500 at most
|
||||
(technical review): a commit per line is a disk sync per line on the one
|
||||
connection the journal milter writes through.
|
||||
|
||||
### Which lines are a row's
|
||||
|
||||
A line is shown on a row's page when the queue id is the same **and** the
|
||||
line's time lies between one hour before the row was written and the queue
|
||||
lifetime (the larger of `maximal_queue_lifetime` and `bounce_queue_lifetime`,
|
||||
as `postconf` reports them) plus a day after it. Postfix reuses a short queue
|
||||
id once the message has left the queue - the id is the microsecond of arrival
|
||||
and the inode of the queue file - so a row that lives ninety days can meet a
|
||||
later message under its id. The window does not rule that out, it limits it
|
||||
to the days the row's own message could still be in the queue; this is the
|
||||
bound the roadmap item asked for. The search of the current file that the
|
||||
page did before had the same exposure for a day. `enable_long_queue_ids`
|
||||
would end it and is a change to the mail path, not part of this item.
|
||||
*It was made in `2.10.1` (owner, 2026-10-16).*
|
||||
|
||||
### Retention
|
||||
|
||||
Lines live as long as a send-log row with their queue id does. One rule covers
|
||||
both ends: a line older than an hour whose queue id has no send-log row is
|
||||
deleted. That removes the lines of a row the retention sweep has just deleted,
|
||||
and the lines of messages that never had a row. It runs at start and after
|
||||
every send-log sweep, six-hourly. It asks the send log once per queue id; at a
|
||||
million lines that is still a scan of the table, of the order of a second,
|
||||
during which the one connection is taken.
|
||||
|
||||
There is no cap of its own: with port-25 mail left out, the table is bounded
|
||||
by what authenticated applications send, which level 1 and level 2 limit.
|
||||
|
||||
### After an unclean stop
|
||||
|
||||
The tailer saves its place every few seconds, so after a crash it reads some
|
||||
lines twice. While it is catching up at start a line is written only if the
|
||||
same line is not there already - the rule the inbound journal follows.
|
||||
|
||||
### What is not kept
|
||||
|
||||
- Lines the tailer never reads are not stored: the tail of a file that was
|
||||
rotated away while the panel was down, as with delivery statuses
|
||||
(architecture.md § Log tailer).
|
||||
- A message in the queue across the upgrade to 2.2.0 has only its later lines
|
||||
stored, and the page then shows those alone: the search of the file runs
|
||||
only for a row with nothing stored. The only installation is the owner's
|
||||
test server; no merge of the two sources is built for one moment of it.
|
||||
|
||||
### The page
|
||||
|
||||
The page reads the table: the latest 200 lines of the row, oldest first. When
|
||||
the table has none, it still searches the current `mail.log` as before - that
|
||||
is what shows a message accepted before 2.2.0 while its lines are in today's
|
||||
file. Lines in rotated files are not imported: the only installation is the
|
||||
owner's test server.
|
||||
|
||||
Nothing in the page's layout changes; the `out-message` mockup already draws
|
||||
the box as a `log_pane`. Two texts change, mockup first: the *Outbound log*
|
||||
help topic (it says an old message may show no lines) and the note shown when
|
||||
there are none.
|
||||
|
||||
### Size
|
||||
|
||||
Six to ten lines of about 200 bytes per message: 1-2 KB next to a row of a few
|
||||
hundred bytes. The table becomes the larger part of what the send log costs in
|
||||
`/data` and in a backup; the retention setting bounds both together.
|
||||
|
||||
## Schema (migration `0003_send_log_lines.sql`)
|
||||
|
||||
```sql
|
||||
CREATE TABLE send_log_lines (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
queue_id TEXT NOT NULL,
|
||||
logged_at TEXT NOT NULL,
|
||||
line TEXT NOT NULL
|
||||
);
|
||||
CREATE INDEX idx_send_log_lines_queue_id ON send_log_lines (queue_id, logged_at);
|
||||
```
|
||||
|
||||
No foreign key: a line is written before its row exists, and one queue id is
|
||||
several rows.
|
||||
|
||||
## Scope
|
||||
|
||||
In: the table, the tailer writing and pruning it, the page reading it, the two
|
||||
texts, docs.
|
||||
|
||||
Out: the inbound journal (it decides for itself, roadmap § delivery-log-storage);
|
||||
importing rotated files; `enable_long_queue_ids`; any change to `mail.log`
|
||||
rotation or to the Server › Mail log page.
|
||||
|
||||
## Security
|
||||
|
||||
- The lines carry the far end's text (replies, HELO names). They are shown
|
||||
today from the file through the same escaping; storing them changes where
|
||||
they are read from, not what is shown or to whom. A domain administrator
|
||||
sees the lines of messages of their own domains, as now.
|
||||
- They now enter the database and so a full backup, which `log/` does not:
|
||||
submitting clients' addresses and SASL logins, message ids, receivers'
|
||||
replies. [security.md](../security.md) says so.
|
||||
- A line is matched from its head, so a reply cannot make itself a line of
|
||||
another message; Postfix does not log raw line breaks, and `postlogd` is the
|
||||
file's only writer.
|
||||
- A stranger on port 25 does not write to the table (above).
|
||||
|
||||
## Done when
|
||||
|
||||
A message sent before the last rotation shows its lines on its page; the lines
|
||||
disappear with the row; the page's wording no longer blames rotation for an
|
||||
ordinary case.
|
||||
|
||||
## Model routing (this plan)
|
||||
|
||||
| Role | Model |
|
||||
|------|-------|
|
||||
| All implementation (code, tests, docs) | **Opus** |
|
||||
| Technical review (reviewer ≠ author: a separate read-only agent) | **Opus** |
|
||||
| Security review | **Fable** |
|
||||
|
||||
## Implementation checklist
|
||||
|
||||
Target version cut: **`2.2.0`** (MINOR). One commit per step;
|
||||
[development.md](../development.md) § Plan checklists.
|
||||
|
||||
- [x] Migration `0003_send_log_lines.sql`, store methods (insert, insert once, lines of a row, prune) and their tests — **Opus**
|
||||
- [x] Log tailer: store queue-id lines, write once while catching up, prune with the send-log sweep — **Opus** (hourly as first built; six-hourly after the reviews)
|
||||
- [x] The two texts, mockup first: the *Outbound log* help topic and the empty note (`design:` commit) — **Opus** (the help topic is in the mockups; the empty note is not drawn there - the `out-message` mockup shows a message with lines - and changes with the page)
|
||||
- [x] Message page reads the table, falls back to the current `mail.log`; help text in `help.go` — **Opus**
|
||||
- [x] [architecture.md](../architecture.md), [guide.md](../guide.md), [schema-migrations.md](../schema-migrations.md) — **Opus** (schema-migrations.md went in with the migration)
|
||||
- [x] `go vet`, `go test`, e2e: a message's page still lists its lines after a forced rotation — **Opus** (on the test server: vet ok, 22 packages ok, e2e ok with the new step `delivery_log_survives_rotation`)
|
||||
- [x] Technical review — **Opus** (2026-10-12; no Critical or High. Fixed: one transaction per pass instead of a commit per line, the prune asks the send log once per queue id and runs six-hourly, the cut keeps characters whole, tests for what the comments claimed. Left as found and written down under *What is not kept*: lines the tailer never read, and a message in the queue across the upgrade)
|
||||
- [x] Security review — **Fable** (2026-10-12; no Critical or High. Fixed: port-25 mail is not stored, so a stranger cannot grow the table; [security.md](../security.md) states what a backup now carries. The window against queue-id reuse is described as the limit it is)
|
||||
@@ -1,260 +0,0 @@
|
||||
# Plan: from-header-binding
|
||||
|
||||
**Status:** shipped in `2.4.0` (2026-10-12); agreed the same day - the design questions were closed with the owner one at a time and he took it into work ("Да")
|
||||
**Date:** 2026-10-12
|
||||
**Version:** `2.4.0` MINOR (one migration; it refuses mail that was accepted before).
|
||||
|
||||
---
|
||||
|
||||
## Goal
|
||||
|
||||
An application sends only as itself - in the `From:` line a mail program
|
||||
shows and OpenDKIM chooses the key by, not only in the envelope.
|
||||
|
||||
Until now Postfix held the envelope sender against the SASL login and nothing
|
||||
held the header. An application of one sending domain could write another
|
||||
registered domain, or the server's own name, into `From:` and have the message
|
||||
signed with that name's key (security review of panel-notifications,
|
||||
2026-10-12).
|
||||
|
||||
## Decisions (owner, 2026-10-12)
|
||||
|
||||
### The rule
|
||||
|
||||
**`From:` is held to what the envelope is held to.** An application set to
|
||||
any address of its domain may write any address of that domain; one with a
|
||||
list of addresses, only those. Anything else is refused: another registered
|
||||
domain, the server's name, an address at a domain this server knows nothing
|
||||
of, no `From:` at all.
|
||||
|
||||
I proposed the narrower rule - refuse only a `From:` that names a domain this
|
||||
server signs for, and let a stranger's address through unsigned as before -
|
||||
and, for an application with a list, to check the domain alone. The owner
|
||||
chose the wider one both times. What it stops working: a form that puts its
|
||||
visitor's address into `From:`, and an application with a list whose envelope
|
||||
address and header address differ, until the second is added to the list.
|
||||
|
||||
When the hostname is itself a sending domain its applications write
|
||||
`…@<hostname>` by right, as the guide already says.
|
||||
|
||||
### Where it is checked, and what happens when it cannot be
|
||||
|
||||
**The panel checks, and the check is strict:** while the panel does not
|
||||
answer, Postfix replies with a temporary error and accepts nothing unchecked.
|
||||
|
||||
I first proposed the opposite (fail-open, as the journal and the level-2 limit
|
||||
are) and then a small process of its own. The owner's objection to the first:
|
||||
a check that is absent while the panel is down does not close the hole, and
|
||||
the panel only has to be slow - 15 seconds - for Postfix to go on without it.
|
||||
His answer to the second: not yet; measure first.
|
||||
|
||||
So the check is a role of the panel process on a socket of its own, attached
|
||||
with `default_action=tempfail`. The journal milter keeps its socket and stays
|
||||
fail-open: a slow database still does not defer mail. The check reads no
|
||||
database - it reads the file of sender bindings Postfix itself reads for the
|
||||
envelope, so the two cannot disagree - and moving it into a process of its own
|
||||
later is a change of where it is started.
|
||||
|
||||
### Statistics, to decide whether the check needs its own process
|
||||
|
||||
Per day, kept a year:
|
||||
|
||||
- **for how long the panel did not answer while Postfix was running.** The
|
||||
panel marks itself alive every few seconds; at start the gap since the last
|
||||
mark is its downtime, counted when Postfix was not restarted within it (a
|
||||
rollout or a reboot stops mail for another reason);
|
||||
- **how many messages were deferred because the check did not answer**, read
|
||||
from `mail.log`. This one also sees a panel that is running and not
|
||||
answering, which the mark does not.
|
||||
|
||||
Shown on **Server › Health** in a card of its own, *Sender check*: today, the
|
||||
last 30 days, the last 12 months. The card warns when a message was deferred
|
||||
in the last 24 hours; downtime that deferred nothing warns of nothing.
|
||||
|
||||
### What a refusal looks like
|
||||
|
||||
- **To the application:** `550 5.7.1`, permanent, naming the address. A
|
||||
level-2 limit answers 4xx because the message will pass in an hour; a wrong
|
||||
`From:` will not, and a 4xx would have the application send it again for
|
||||
days.
|
||||
- **In the Outbound log:** a row with the reason in the status tag -
|
||||
`wrong From`, beside `rate-limited` and the tag of a refused client address.
|
||||
The built panel wrote `rejected` for all of them; it follows the mockup now.
|
||||
(The mockup's status filter is not built - the log filters by domain and
|
||||
application - so there is nothing to keep in step with the tags.)
|
||||
- **On the message's page:** a fact *From: line* with the address written in
|
||||
the header, and a History step that says which address was refused, for
|
||||
which application, what that application may write, and that it was told
|
||||
`550 5.7.1`.
|
||||
|
||||
### Health, re-laid
|
||||
|
||||
The owner on the first picture of the new card: too many holes on the screen,
|
||||
the numbers should follow importance, and the boxes should read left to right
|
||||
and top to bottom - with stacked columns the third box was below the fold and
|
||||
the fourth was not.
|
||||
|
||||
- Order: 01 Processes, 02 Milter sockets, 03 Sender check, 04 TLS
|
||||
certificate, 05 Hostname and reverse DNS, 06 Notification mail, 07 Machine,
|
||||
08 Configuration.
|
||||
- Layout `cards` (new, on the components page): two stacks, odd numbers on the
|
||||
left and even on the right, each closed up, so a short box leaves no gap
|
||||
beside a tall one and the eye still goes 01, 02, 03. On a phone the boxes
|
||||
come in the order of their numbers.
|
||||
|
||||
## Settled while building
|
||||
|
||||
- **The header is read twice.** One `From:` field holding one address, by
|
||||
Go's parser; then every `@name` of the raw value is held against OpenDKIM's
|
||||
own `SigningTable`. The first build took the names for the second look from
|
||||
the sender bindings, and the e2e stand let a line through that named a
|
||||
domain with a key and no application yet: a domain is a signing name from
|
||||
the moment it is added, not from its first application.
|
||||
- **What Postfix writes when the check does not answer** (seen on the stand):
|
||||
a warning naming the socket, then `milter-reject: CONNECT … 451 4.7.1
|
||||
Service unavailable` and the same for `EHLO`; the client is told `454 4.3.0
|
||||
Try again later` at `AUTH` and leaves. No `MAIL` is ever offered, so what
|
||||
is counted is the connection turned away, once. The journal milter's
|
||||
level-2 refusal is written in the same words, which is why a reject line
|
||||
counts only after the warning.
|
||||
- **When Postfix started** is asked of supervisord (`supervisorctl status`,
|
||||
the uptime of `postfix`).
|
||||
- **The reply** is a milter reply code, `550 5.7.1 From: <line> - <why>`; what
|
||||
the client wrote is reduced to printable ASCII and cut first.
|
||||
- **The Delivery log box stays on a refused message's page**, with the one
|
||||
sentence it already had for a message that never reached the queue; the
|
||||
picture shown to the owner had left the box out.
|
||||
|
||||
## To settle while building (as first written)
|
||||
|
||||
- How the header is read so that the check and OpenDKIM cannot disagree: one
|
||||
`From:` field, one address, and a cruder second look at the raw value.
|
||||
- What Postfix writes in `mail.log` when a strict milter does not answer, and
|
||||
how a line is told to be this milter's and not OpenDKIM's.
|
||||
- How the panel learns when Postfix started.
|
||||
- Schema: the reason of a refusal and the header address on a send-log row;
|
||||
the table of days.
|
||||
- That a refusal for `From:` is not taken for a refusal under a limit by the
|
||||
notifier.
|
||||
|
||||
## Implementation checklist
|
||||
|
||||
- [x] Mockups: Health card and layout, the log row, help topic; accepted by the owner 2026-10-12 (`design:` commit)
|
||||
- [x] Migration and store: refusal reason and header address, days of the sender check (`0006_sender_check.sql`)
|
||||
- [x] The check: a role of the panel on its own socket, bindings read from Postfix's file; Postfix attaches it strictly; start order
|
||||
- [x] Statistics: the alive mark and downtime, deferred mail from `mail.log`
|
||||
- [x] Panel: Health (card, sockets row, layout), Outbound log tags, the message's page, help
|
||||
- [x] guide.md, architecture.md, security.md (the accepted risk goes), schema-migrations.md
|
||||
- [x] `go vet`, `go test`, e2e: refused for another domain's `From:`, for the hostname's and for an address off the list; accepted for its own (on the test server: vet ok, 23 packages ok, e2e ok - `sender_check_holds_the_from_line`, `panel_down_defers_mail`)
|
||||
- [x] Technical review — **Opus** (2026-10-12; one Critical, fixed: a folded `From:` line - what a mail library makes of a long display name - was refused for good. Also fixed: a display name in a charset Go does not convert was refused; the statistics ran before the check served, and a hung `supervisorctl` would have kept it from serving; the count of deferred mail missed a check that stops answering in the middle of a message, and is now taken at Postfix's warning; a card whose figures could not be read said ok; a killed container's stale socket satisfied the wrapper's wait; outside quoted strings the line may now hold one `@` only. Three were first left as found and fixed the same evening at the owner's question: downtime is cut at the UTC midnights it crossed; the row of a message with the null envelope sender takes the domain the login is bound to; a log read again after an unclean stop does not count again what it counted before)
|
||||
- [x] Security review — **Fable** (2026-10-12; no Critical or High. The same folded line, as its one Medium. Fixed: another sender's name in an encoded display name passed the second look; a client could arm the deferred counter by writing the socket's name where Postfix quotes it in a warning; a refused message wrote a row per recipient, a thousand at most - now one. Recorded: what a display name looks like to a person is not promised. The review also found the check stricter than Postfix for an address with an extension (`invoices+x@`, allowed in the envelope and refused in the header); the owner asked for it to work, and the check now makes Postfix's second try, with `recipient_delimiter` set explicitly. Confirmed from the code: no path to the signer around the check, and every failure of the check defers)
|
||||
|
||||
## Not done
|
||||
|
||||
Written at the owner's word when he had the version cut ("Запиши все
|
||||
недоделки в план", 2026-10-12). Nothing here was promised and dropped in
|
||||
silence; each line says what is missing and what it would take.
|
||||
|
||||
### Decided to wait
|
||||
|
||||
- **A process of its own for the check.** The owner: measure first. Server ›
|
||||
Health now has the numbers; the decision is his when they have had time to
|
||||
say something. Moving the role out is a change of where it is started -
|
||||
a subcommand of the panel binary and a `[program:]` in supervisord - and
|
||||
the downtime figure then needs a new meaning.
|
||||
|
||||
### Shown to the owner one way and built another
|
||||
|
||||
- **The Delivery log box is on a refused message's page.** The picture he
|
||||
approved had left it out; the built page keeps it, with one sentence. And
|
||||
the sentence is less than the truth for a `From:` refusal: cleanup did
|
||||
write a line about it under a queue id (`milter-reject: END-OF-MESSAGE ...
|
||||
5.7.1 From: ...`), but the check answers before the queue id reaches it, so
|
||||
the row keeps none and the page cannot show that line. **Decided
|
||||
2026-10-13 (owner):** the box is required - he had overlooked its absence
|
||||
from the picture, it was never a decision to drop it. The tailer attaches
|
||||
the line to the row, for all three kinds of refusal; a step of
|
||||
[audit-2026-10-12](../audit-2026-10-12.md) stage 2b.
|
||||
- **The Outbound log has no status filter.** The mockup draws one; the built
|
||||
log filters by domain and application. With three kinds of refusal now
|
||||
told apart by tag, a filter by status and reason would be worth having.
|
||||
**Decided 2026-10-13 (owner):** a filter by status, without the split by
|
||||
reason, and a filter by day as the Inbound log has; a step of [audit-2026-10-12](../audit-2026-10-12.md) stage 4.
|
||||
- **"Messages deferred" counts connections Postfix turned away.** Postfix
|
||||
refuses from the connection on, before a message is offered, so a monitor
|
||||
that only connects during an outage is counted. The guide says so; the
|
||||
card's column head and the help topic still say "messages". **Decided
|
||||
2026-10-13 (owner):** the column is renamed *Connections turned away*; a
|
||||
step of [audit-2026-10-12](../audit-2026-10-12.md) stage 2b.
|
||||
|
||||
### Known and small
|
||||
|
||||
- **"Today" on the card is the UTC day**, and the card does not say so.
|
||||
**Decided 2026-10-13 (owner):** not a label - the local day; it became the
|
||||
roadmap item `local-time`, for the whole panel.
|
||||
- **A deferral in the very second of the last one counted before an unclean
|
||||
stop is passed over** at the restart: `mail.log` stamps whole seconds.
|
||||
Chosen over counting a stretch of the log twice. **Decided 2026-10-14
|
||||
(owner):** counted exactly - the counter keeps a place in the log, saved
|
||||
with the count; a step of [audit-2026-10-12](../audit-2026-10-12.md)
|
||||
stage 2b.
|
||||
- **Deferrals are lost when `mail.log` is rotated while the panel is down**:
|
||||
the tailer starts on the new file. **Decided 2026-10-13 (owner):** the
|
||||
tailer reads the rotated file to its end first - not the deferrals alone,
|
||||
everything the panel takes from the log; a step of
|
||||
[audit-2026-10-12](../audit-2026-10-12.md) stage 2b.
|
||||
- **A start at which `supervisorctl` fails counts no downtime**; it is
|
||||
logged. **Decided 2026-10-13 (owner):** the question is repeated, and
|
||||
without an answer the downtime is counted from the alive mark and the day
|
||||
marked as estimated; a step of
|
||||
[audit-2026-10-12](../audit-2026-10-12.md) stage 2b.
|
||||
- **Logins are compared as written**, Postfix compares them without regard
|
||||
to case. The panel writes logins one way, so nothing differs today.
|
||||
- **The phone order of the `cards` layout is written for four boxes to a
|
||||
stack** and takes every child of a stack for a box (said at the partial).
|
||||
The rule the technical review suggested instead - the order from the
|
||||
box's own number - changes the component and would be drawn first.
|
||||
**Decided 2026-10-13 (owner):** that rule is built; a step of
|
||||
[audit-2026-10-12](../audit-2026-10-12.md) stage 4.
|
||||
- **Health's socket check looks at the file, not at who listens**: a socket
|
||||
nobody answers on would read "ok". The start now removes stale sockets,
|
||||
which was the one way to get such a file. **Decided 2026-10-13 (owner):**
|
||||
the check connects to the socket; a step of
|
||||
[audit-2026-10-12](../audit-2026-10-12.md) stage 4.
|
||||
- **The kit page's table of status tags does not list the new ones**
|
||||
(`wrong From`, `wrong client`, `rate-limited`), and `rate-limited` is grey
|
||||
beside two red tags - as the mockup has it, not as a decision.
|
||||
- **Refusal rows still in the buffer when the panel stops are not written**;
|
||||
the refusals themselves were answered. **Decided 2026-10-13 (owner):**
|
||||
the tailer writes such a row from the refusal's line in `mail.log`; a step
|
||||
of [audit-2026-10-12](../audit-2026-10-12.md) stage 2b.
|
||||
|
||||
### Accepted, in security.md
|
||||
|
||||
- **The panel's downtime is mail's.** The price of the strict check.
|
||||
- **What a display name looks like to a person is not promised.** A name of
|
||||
another sender written without an `@` the scan can find passes, signed
|
||||
with the application's own key.
|
||||
- **A malformed packet on the check's socket ends the panel** (the milter
|
||||
library panics on an empty one). The socket is reachable only inside the
|
||||
container, by the group Postfix is in; the journal milter always had this,
|
||||
and what is new is that the price is deferred mail.
|
||||
|
||||
### Not verified
|
||||
|
||||
- **OpenDKIM's reading of a `From:` line** was reasoned about, not run: its
|
||||
parser is not in this repository. The two rules that do not depend on it -
|
||||
no other signing name after any `@`, one `@` outside quotes - are what the
|
||||
claim rests on. A test that feeds the stand's OpenDKIM the lines the check
|
||||
passes and reads which key it signs with would settle it. **Decided
|
||||
2026-10-13 (owner):** that test is built; a step of
|
||||
[audit-2026-10-12](../audit-2026-10-12.md) stage 2.
|
||||
- **The pages were not looked at in a browser on the deployed panel**: I
|
||||
cannot sign in there. They were rendered from the fixtures and
|
||||
screenshotted, the polling and the phone order read from the code and
|
||||
exercised by e2e only as far as the Health numbers.
|
||||
- **No refusal has been seen on the test server itself**, only on the e2e
|
||||
stand: there is no application there I can send as.
|
||||
- **A check that stops answering in the middle of a message** is counted
|
||||
from a warning composed from Postfix's source; the stand only produced the
|
||||
case of nobody listening.
|
||||
@@ -1,325 +0,0 @@
|
||||
# Plan: inbound-antispam-panel
|
||||
|
||||
**Status:** shipped in `2.1.0` (2026-10-12)
|
||||
**Date:** 2026-08-19
|
||||
**Version:** `2.1.0` MINOR (opt-in; no change to outbound-only or inbound-without-filter paths). Was `1.10.0`; on 2026-10-09 the owner put the panel redesign first, so this ships on the 2.0 baseline and its screens are built from the redesign's component kit ([panel-redesign.md](panel-redesign.md) § Component kit).
|
||||
|
||||
---
|
||||
|
||||
## Goal
|
||||
|
||||
Give the operator a **panel view of inbound anti-spam decisions** and **editable
|
||||
allow/deny lists** that tune filter behaviour — without SelfPost shipping or
|
||||
starting an anti-spam engine ([product.md](../product.md) out-of-scope list
|
||||
unchanged).
|
||||
|
||||
Typical journal row:
|
||||
|
||||
| When | From | To | Subject | Decision |
|
||||
|------|------|----|---------|----------|
|
||||
| 2026-08-19 09:14 | `spammer@evil.example` | `user@inbound.example` | Cheap pills | reject (rspamd: BAYES_SPAM +0.95) |
|
||||
|
||||
Lists let the operator correct false positives/negatives (e.g. always allow
|
||||
`billing@vendor.example`, block `192.0.2.0/24`).
|
||||
|
||||
## Context (as-built)
|
||||
|
||||
- Inbound relay ships in `[1.4.0]` (`INBOUND_RELAY_ENABLE`); optional
|
||||
`INBOUND_ANTISPAM_MILTER` attaches an **external** filter (documented rspamd
|
||||
sidecar in [deploy/antispam/docker-compose.antispam.yml](../../deploy/antispam/docker-compose.antispam.yml)).
|
||||
- The inbound smtpd has **no journal-milter** today — only outbound 465/587
|
||||
writes `send_log` ([architecture.md](../architecture.md)).
|
||||
- Inbound UI and handlers are **global-administrator only** today
|
||||
(`requireInbound` → `requireGlobal`).
|
||||
|
||||
## Quarantine
|
||||
|
||||
**Not in this plan.** Held-mail review/release is an idea kept outside the roadmap,
|
||||
[inbound-quarantine](inbound-quarantine.md). The journal may still
|
||||
record a `quarantine` **decision label** when the external filter reports it,
|
||||
without SelfPost storing the message.
|
||||
|
||||
**DMARC `p=quarantine`** remains unrelated (outbound policy for receivers).
|
||||
|
||||
## Decisions (closed 2026-08-19)
|
||||
|
||||
### Capture mechanism
|
||||
|
||||
**Inbound journal-milter in `panel` only** — no rspamd history API polling in v1.
|
||||
A separate unix socket (`INBOUND_JOURNAL_MILTER_SOCKET`, default
|
||||
`/run/selfpost/inbound-journal.sock`) on the inbound smtpd only; fail-open
|
||||
(`default_action=accept`).
|
||||
|
||||
**Milter order on port 25** (when antispam is enabled):
|
||||
|
||||
```text
|
||||
smtpd_milters = { antispam, inbound-journal }
|
||||
```
|
||||
|
||||
Antispam runs first; inbound-journal runs at end-of-message on the **accept**
|
||||
path and reads rspamd-added headers (`X-Rspamd-Action`, and `X-Spamd-Result`
|
||||
for the score and symbols - rspamd has no `X-Rspamd-Score`; the sidecar adds
|
||||
both through `deploy/antispam/rspamd/milter_headers.conf`). As built they are
|
||||
read only next to rspamd's signature of the message, `X-SelfPost-Filter`
|
||||
(owner, 2026-10-12: "А rspamd не может дописывать какой-то заголовок, по
|
||||
которому мы будем знать, что это заголовки нашего спам фильтра?") - an HMAC
|
||||
of the queue id, since a fixed mark could be written by a sender as easily as
|
||||
the verdict itself.
|
||||
|
||||
**Antispam rejects** do not reach end-of-message on the journal milter. v1 also
|
||||
tails `mail.log` for inbound `postfix/smtpd` **`milter-reject`** / **`reject`**
|
||||
lines and inserts journal rows (client IP, envelope from/to when logged, reason
|
||||
text, subject empty if rejected before DATA). DMARC-ingest pipe traffic is out
|
||||
of scope for this tailer.
|
||||
|
||||
### Journal without `INBOUND_ANTISPAM_MILTER`
|
||||
|
||||
Journal stays **on** whenever `INBOUND_RELAY_ENABLE=true` (not tied to the
|
||||
antispam hook). Without an external filter:
|
||||
|
||||
- **accept** rows — message relayed (engine `selfpost`, detail empty);
|
||||
- **reject** / **tempfail** rows — from the mail.log tailer (Postfix policy,
|
||||
unknown recipient, a sender domain that does not exist. As built, a refusal
|
||||
whose log line names no recipient - the size limit answered at MAIL FROM -
|
||||
and the per-client rate limit, which Postfix logs as a warning, make no row).
|
||||
|
||||
Panel copy explains that rspamd symbols appear only when the antispam milter is
|
||||
configured.
|
||||
|
||||
### RBAC
|
||||
|
||||
**Global administrator only** for journal and lists — same as inbound relay. No
|
||||
domain-admin access in v1.
|
||||
|
||||
*As built (2026-10-11):* the lists are the global role's; the **log follows
|
||||
the delegation of Inbound** that 2.0 introduced after this was written - a
|
||||
user with the `domain` role sees the rows of the inbound domains assigned to
|
||||
them and nothing else, without the *Allow / Deny sender* links. That is what
|
||||
the accepted `in-log` mockup draws (no "global role" in its head). The owner
|
||||
was told and can turn it back into a global-only page.
|
||||
|
||||
**Inbound › Log accepted by the owner on 2026-10-12** ("Этот экран
|
||||
утверждаем"), from screenshots of the page built from its fixture, after his
|
||||
changes to it: the sender column is headed *From*; *Why* is not a column but
|
||||
a sheet over the page, opened by the decision tag, holding a centred table of
|
||||
who decided, the verdict and symbols or the reply, and the subject whole
|
||||
(kept on the sheet at his word: the table clips it); a message Postfix itself
|
||||
refused offers no *Deny sender*. Inbound › Filter lists was redrawn at his word
|
||||
the same day - kind in a column of its own, the note on the entry's sheet,
|
||||
matches counted (§ Filter list usage) - and **accepted by him on 2026-10-12**
|
||||
("Да"), after a deny entry he added in the panel refused a test message with
|
||||
the list's reply and the journal recorded it.
|
||||
|
||||
### Retention
|
||||
|
||||
Mirror send-log retention:
|
||||
|
||||
- Settings key `inbound_spam_log_retention_days` on `/server/settings` (global admin).
|
||||
- Default **90** days; validation **7–365**; seed from env
|
||||
`INBOUND_SPAM_LOG_RETENTION_DAYS` when the setting is missing.
|
||||
- Prune every **6 hours** (same cadence as send-log retention).
|
||||
- Hard cap **10 000** rows after age prune (drop oldest) — busier than DMARC
|
||||
reports, lighter than unbounded growth.
|
||||
|
||||
### Allow/deny lists
|
||||
|
||||
| Topic | Decision |
|
||||
|-------|----------|
|
||||
| Entry types | sender address, sender domain (`@domain` suffix), client IP / CIDR |
|
||||
| Precedence | **deny overrides allow** |
|
||||
| Scope | instance-wide (not per inbound domain in v1) |
|
||||
| Where applied | **rspamd map files** under `/data/antispam/`, one per list and entry type (`allow_ip.map`, `allow_from.map`, `allow_domain.map`, `deny_…`) - an IP map and a sender map are different map types to rspamd; documented compose fragment mounts them into the sidecar. Deny is a prefilter reject, allow a weight of -100 (as built: two prefilters do not give "deny overrides allow") |
|
||||
| SelfPost pre-check | **out of v1** — lists live in rspamd only |
|
||||
| Edit access | global administrator only |
|
||||
| When inactive | lists editable in the panel but UI warns that sync applies only when `INBOUND_ANTISPAM_MILTER` is set; `postfix reload` / rspamd reload documented in guide |
|
||||
|
||||
List change → atomic map rewrite (same pattern as Postfix maps). No reload:
|
||||
rspamd rereads a changed map file by itself (about ten seconds on 3.11).
|
||||
|
||||
### Filter list usage (owner, 2026-10-12)
|
||||
|
||||
Each entry shows how often it decided a message (*Matched*, or *never*) and,
|
||||
on its sheet, when it last did - so that entries nothing matches any more can
|
||||
be removed. **The count is read off the journal on every view of the page;
|
||||
nothing is stored for it.** A journal row counts for an entry when the list
|
||||
decided the message - a rejected row whose reply is the deny rule's message,
|
||||
an accepted row whose verdict carries the allow rule's symbol
|
||||
(`SELFPOST_ALLOW_IP` / `_FROM` / `_DOMAIN`, which the journal milter always
|
||||
keeps) - and the entry fits the row's sender or client address the way rspamd
|
||||
matches it (`inbound.Lists.Usage`).
|
||||
|
||||
What follows from counting this way, and is accepted:
|
||||
|
||||
- the count reaches back as far as the journal does: its retention (90 days by
|
||||
default), which is what the sheet says;
|
||||
- a row counts for every fitting entry of the deciding list, since the journal
|
||||
does not say which of two the filter met first;
|
||||
- a message refused at end-of-message is one row whatever its recipient count,
|
||||
so a count is of messages, roughly.
|
||||
|
||||
**When to move to a counter kept with each entry** (a migration adding hits
|
||||
and last-hit to `inbound_spam_list`, bumped where the journal row is written;
|
||||
the owner asked for the condition to be written down). Any one of:
|
||||
|
||||
1. **The journal is cut by its 10 000-row cap and not by age.** Then it holds
|
||||
fewer days than its retention, the sheet's "in the last 90 days" is no
|
||||
longer true, and *never* stops being evidence that an entry is stale. The
|
||||
sign: the log tailer's prune line keeps reporting rows removed while the
|
||||
oldest row of Inbound › Log is younger than the retention.
|
||||
2. **The retention is set short** (towards its minimum of 7 days) for reasons
|
||||
of its own, privacy or disk. A week without a match does not make an entry
|
||||
stale.
|
||||
3. **Counts are wanted that outlive the journal** - since the entry was added.
|
||||
4. **The page is slow.** A view reads every journal row a list decided and
|
||||
tries it against every entry. With the cap that is at most 10 000 rows
|
||||
times the number of entries; past a few hundred entries, measure it.
|
||||
|
||||
Until then no migration is spent on it.
|
||||
|
||||
### Activation
|
||||
|
||||
| Condition | Behaviour |
|
||||
|-----------|-----------|
|
||||
| `INBOUND_RELAY_ENABLE=false` | no inbound journal milter, no UI, no tables written |
|
||||
| inbound on, antispam empty | journal + UI; lists visible but marked inactive for filter sync |
|
||||
| inbound on + antispam set | full journal (symbols on accept) + list sync |
|
||||
|
||||
No separate feature flag beyond inbound relay + existing antispam env vars.
|
||||
|
||||
### UI
|
||||
|
||||
The journal is **Inbound › Log** and list management is a new **Inbound ›
|
||||
Filter lists** page (global-only: the lists are instance-wide) — see
|
||||
[panel-redesign.md](panel-redesign.md) § Room for roadmap items. The journal
|
||||
table has filters (decision, domain, date range). Mockups are drawn first, in
|
||||
`docs/assets/panel-redesign/panel/src/` under that plan's design contract —
|
||||
not in the superseded `docs/assets/panel-ui/` (removed 2026-10-16).
|
||||
|
||||
## Schema (migration `0002_inbound_spam_log.sql`, the first after the 2.0 baseline)
|
||||
|
||||
**`inbound_spam_log`**
|
||||
|
||||
| Column | Type | Notes |
|
||||
|--------|------|-------|
|
||||
| `id` | INTEGER PK | |
|
||||
| `inbound_domain` | TEXT NOT NULL | recipient domain (from `relay_domains`) |
|
||||
| `client_ip` | TEXT NOT NULL | |
|
||||
| `from_addr` | TEXT NOT NULL | envelope from |
|
||||
| `to_addr` | TEXT NOT NULL | envelope to (one row per recipient) |
|
||||
| `subject` | TEXT NOT NULL | truncated with `mailhdr.SubjectMaxRunes` (200) |
|
||||
| `decision` | TEXT NOT NULL | `accept`, `reject`, `tempfail`, `quarantine` |
|
||||
| `engine` | TEXT NOT NULL | `rspamd`, `postfix`, `selfpost` |
|
||||
| `detail` | TEXT NOT NULL | symbol/score summary or log-line reason; may be empty |
|
||||
| `created_at` | TEXT NOT NULL | RFC3339 UTC |
|
||||
|
||||
Index on `(created_at)`, `(inbound_domain, created_at)`.
|
||||
|
||||
**`inbound_spam_list`**
|
||||
|
||||
| Column | Type | Notes |
|
||||
|--------|------|-------|
|
||||
| `id` | INTEGER PK | |
|
||||
| `list_type` | TEXT NOT NULL | `allow` or `deny` |
|
||||
| `entry_type` | TEXT NOT NULL | `address`, `domain`, `ip` |
|
||||
| `value` | TEXT NOT NULL | normalized entry |
|
||||
| `note` | TEXT NOT NULL | operator comment; default `''` |
|
||||
| `created_at` | TEXT NOT NULL | |
|
||||
|
||||
Unique on `(list_type, entry_type, value)`.
|
||||
|
||||
## Scope
|
||||
|
||||
**In:**
|
||||
|
||||
- Schema, store CRUD, retention loop, mail.log tailer extension for inbound
|
||||
rejects.
|
||||
- Inbound journal-milter (fail-open) + Postfix `postfix-config.sh` wiring.
|
||||
- rspamd map sync under `/data/antispam/`.
|
||||
- Panel journal + list UI (global admin).
|
||||
- Full backup includes `/data/antispam/` and new SQLite tables.
|
||||
- Tests; [guide.md](../guide.md); [architecture.md](../architecture.md).
|
||||
|
||||
**Out:**
|
||||
|
||||
- Shipping rspamd, ClamAV, or any filter binary inside the SelfPost image.
|
||||
- Outbound spam filtering.
|
||||
- [inbound-quarantine.md](inbound-quarantine.md) storage/release.
|
||||
- rspamd rule editing, Bayes training UI, antivirus.
|
||||
- MIME replay from the panel (metadata only).
|
||||
- Per-domain lists; domain-admin RBAC.
|
||||
- rspamd history API polling.
|
||||
|
||||
## Architecture
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
internet["Internet :25"] --> inbound["Postfix inbound smtpd"]
|
||||
inbound --> antispam["External antispam milter optional"]
|
||||
antispam --> journalM["inbound-journal milter fail-open"]
|
||||
journalM --> upstream["smtp upstream transport"]
|
||||
journalM --> sqlite["inbound_spam_log"]
|
||||
mailog["mail.log tailer"] --> sqlite
|
||||
listsUI["Panel allow/deny lists"] --> maps["/data/antispam/*.map"]
|
||||
maps --> antispam
|
||||
sqlite --> journalUI["Panel inbound spam journal"]
|
||||
```
|
||||
|
||||
## Security
|
||||
|
||||
- Same validation whitelists as inbound relay ([security.md](../security.md)).
|
||||
- Journal rows contain PII — retention, and access by reach: the global role
|
||||
reads all of it, a `domain` user the rows of assigned inbound domains (as
|
||||
built; see § RBAC).
|
||||
- Inbound journal milter fail-open must not block mail (backup-MX role).
|
||||
- List map writes injection-safe; deny wins over allow.
|
||||
|
||||
## Done when
|
||||
|
||||
- With inbound relay + antispam sidecar, panel shows accept rows with symbols
|
||||
and reject rows from mail.log tailing.
|
||||
- Operator can add/remove list entries; a test message reflects allow/deny.
|
||||
- With antispam hook off, journal still shows Postfix-level decisions; list UI
|
||||
shows inactive sync notice.
|
||||
- `go vet`, `go test`, image build green; guide and architecture updated.
|
||||
|
||||
## Risks
|
||||
|
||||
- rspamd header/symbol format drift — thin parser, pin sidecar tag in compose.
|
||||
- Journal volume — retention + 10k cap.
|
||||
- mail.log parse brittleness — unit tests on sample lines; inbound smtpd only.
|
||||
|
||||
## Dependencies
|
||||
|
||||
- Inbound relay, shipped in `[1.4.0]` — as built in
|
||||
[architecture.md](../architecture.md) § Mail path. Its plan file was removed
|
||||
once it shipped.
|
||||
|
||||
## Model routing (this plan)
|
||||
|
||||
| Role | Model |
|
||||
|------|-------|
|
||||
| All implementation (code, tests, docs) | **Composer** |
|
||||
| Technical review (reviewer ≠ author) | **Opus** |
|
||||
| Security review | **Fable** |
|
||||
|
||||
Composer ships each step; Opus reviews the accumulated diff once implementation
|
||||
steps are done (fix loop: Composer addresses Opus findings, Opus re-checks).
|
||||
Fable runs after Opus sign-off.
|
||||
|
||||
## Implementation checklist
|
||||
|
||||
Target version cut: **`2.1.0`** (MINOR), after `2.0.0`. One commit per step;
|
||||
[development.md](../development.md) § Plan checklists.
|
||||
|
||||
- [x] Agree journal fields, retention, and RBAC (this plan § Decisions) — **Composer**
|
||||
- [x] Migration `0002_inbound_spam_log.sql` on the 2.0 baseline — **Composer**
|
||||
- [x] Inbound journal-milter + Postfix wiring — **Composer**
|
||||
- [x] mail.log tailer: inbound reject rows — **Composer**
|
||||
- [x] List CRUD + validation + atomic rspamd map sync — **Composer**
|
||||
- [x] Panel: Inbound › Log and Inbound › Filter lists from the kit partials, mockups `in-log` / `in-filter-lists` (add client IP and engine columns mockup-first), evidence pairs — **Composer**
|
||||
- [x] Backup includes `/data/antispam/` — **Composer** (it always did: a backup walks all of `/data`; now held by a test, and the guide says the `antispam/` fragment directory is not in it)
|
||||
- [x] Unit + handler tests — **Composer** (written with each step: store, milter, tailer, lists and their usage, view fixtures held to the outlines, handlers for both roles)
|
||||
- [x] [guide.md](../guide.md), [architecture.md](../architecture.md) — **Composer** (kept up with each step; [security.md](../security.md) follows the security review)
|
||||
- [x] `go vet`, `go test`, e2e if applicable — **Composer** (on the test server for every step; last on `7192c38`: vet ok, 22 packages ok, e2e ok, which relays, refuses, reads the log page and edits a list through the panel)
|
||||
- [x] Technical review of inbound antispam changes — **Opus** (2026-10-12; findings fixed: client address of a connection's later messages, the lists' writer under a lock, the tailer's queue-id memory, hold rows written twice, the cap by time; and, outside the feature, the level-2 limit and client-address list skipped after a connection's first message)
|
||||
- [x] Security review inbound antispam path — **Fable** (2026-10-12; no Critical or High; HELO-forged journal rows and the six-hourly row cap fixed; a sender's own verdict while the filter is down closed by the filter's signature, the owner's idea; accepted risks recorded in [security.md](../security.md))
|
||||
@@ -1,8 +1,8 @@
|
||||
# Plan: inbound-quarantine
|
||||
|
||||
**Status:** an idea to think about, not on the roadmap (owner, 2026-10-12) — design TBD; needs explicit agreement before any code
|
||||
**Version:** TBD (`1.x` MINOR if opt-in and backward-compatible; may warrant `2.x`
|
||||
if it changes inbound storage or backup contract).
|
||||
**Version:** TBD (a 2.x MINOR if opt-in; a major only if it breaks the
|
||||
inbound storage or backup contract).
|
||||
|
||||
---
|
||||
|
||||
@@ -17,8 +17,8 @@ so we can compare approaches before committing.
|
||||
|
||||
## Why it is separate from inbound-antispam-panel
|
||||
|
||||
[inbound-antispam-panel.md](inbound-antispam-panel.md) covers **metadata**
|
||||
(journal + lists). Quarantine implies **storing message bodies** (or a durable
|
||||
inbound-antispam-panel (shipped `2.1.0`; [architecture.md](../architecture.md)
|
||||
§ Mail path) covers **metadata** (journal + lists). Quarantine implies **storing message bodies** (or a durable
|
||||
reference to them) under `/data`, retention, release workflow, and a wider
|
||||
attack/backup surface — closer to "mini-mailbox" than pure relay.
|
||||
|
||||
@@ -68,9 +68,7 @@ to the upstream, 14 days, headers only in the panel); it does not settle them.
|
||||
|
||||
- Inbound relay, shipped in `[1.4.0]` — as built in
|
||||
[architecture.md](../architecture.md) § Mail path.
|
||||
- Sensible to decide **after** or **alongside**
|
||||
[inbound-antispam-panel.md](inbound-antispam-panel.md) (journal/lists), but
|
||||
not blocked on it for design discussion.
|
||||
- Builds on inbound-antispam-panel (journal and lists, shipped `2.1.0`).
|
||||
|
||||
## Done when (placeholder)
|
||||
|
||||
|
||||
@@ -1,100 +0,0 @@
|
||||
# inbound-upstream-result
|
||||
|
||||
**Status:** shipped `2.10.0` (2026-10-16).
|
||||
|
||||
## Problem
|
||||
|
||||
The owner, 2026-10-16: "Нам не хватает в журнале для Inbound ответа
|
||||
upstream. Не понятна судьба письма: принято, отклонено, хост не доступен и
|
||||
пр."
|
||||
|
||||
Inbound › Log shows what port 25 decided - accept, reject, tempfail - and
|
||||
nothing after it. Its help says so ("Accept says what this server did, not
|
||||
what the upstream did with the message afterwards"). The answer is in
|
||||
`mail.log`: Postfix's `smtp` client logs every attempt to hand a message to
|
||||
its upstream as `<id>: to=<…>, relay=<host>[ip]:port, …, dsn=…, status=sent
|
||||
(250 …)` - or `deferred (451 4.7.1 Greylisted …)`, `deferred (connect to …:
|
||||
Connection refused)` with `relay=none`, `bounced (… said: 550 …)` (Postfix
|
||||
3.10.13, `src/global/log_adhoc.c`; `relay=none` when there was no session,
|
||||
`src/smtp/smtp_trouble.c:273, 421`). The tailer reads those lines for the
|
||||
outbound log and, finding no send-log row for a port-25 queue id, drops
|
||||
them; the journal row does not even hold the queue id.
|
||||
|
||||
## Decision
|
||||
|
||||
The owner, asked whether the result goes in the row or on a page of its own
|
||||
per message: "Пока только результат. Отдельная страница в план в случае
|
||||
появления потребности." So: the row carries the result and the last reply;
|
||||
no stored `mail.log` lines for port-25 mail (delivery-log-storage left them
|
||||
out on purpose: a stranger sets the rate). A page per inbound message, with
|
||||
every line, is for later, if the result is not enough.
|
||||
|
||||
## Design
|
||||
|
||||
- **Schema, migration 0012:** `inbound_spam_log` gains `queue_id`,
|
||||
`upstream_status` (`''`, `queued`, `sent`, `deferred`, `bounced`),
|
||||
`upstream_relay`, `upstream_reply` and `upstream_at`; an index on
|
||||
`queue_id` for rows that have one. Rows written before have none and show
|
||||
"—".
|
||||
- **The milter** writes an accepted row with the queue id it already reads
|
||||
at end-of-message (`{i}`, for the filter's signature) and `queued`. The
|
||||
row is in the database before the message is queued, so before any
|
||||
delivery line.
|
||||
- **The tailer:** a delivery line whose queue id has no send-log row is
|
||||
offered to the journal: the rows of that queue id and recipient take the
|
||||
status, the relay (empty for `none`), the reply (printable, at most 255
|
||||
bytes) and the line's time. One update per attempt, no new row; the
|
||||
journal's caps stand. qmgr's `status=expired, returned to sender` makes a
|
||||
queued or deferred row `bounced`, its reply saying it expired after the
|
||||
last one.
|
||||
- **The page:** an Upstream column after Decision (mockup `5348330`): a
|
||||
plain `queued` tag, or a tag opening an "Upstream" sheet - host, reply,
|
||||
when, and what follows for deferred and bounced. Refused rows "—".
|
||||
- **Docs:** help (word for word with the mockup), guide § Inbound log,
|
||||
architecture (journal and tailer), security (what is stored from the
|
||||
upstream and at what rate), CHANGELOG.
|
||||
|
||||
## Reviews (2026-10-16)
|
||||
|
||||
A technical and a security review of the tree before the commit; what each
|
||||
finding became:
|
||||
|
||||
- *A quoted RCPT TO* (both): the milter kept `"owner"@x` as given, every line
|
||||
says `owner@x` - the row never matched. Now `recipientAsLogged`, as the
|
||||
outbound milter does.
|
||||
- *A forged line through a quoted local part* (security, medium): `to=<…>`
|
||||
was read up to its first `>`; a local part in quotes may hold `>`, `,`,
|
||||
`status=`. Port 25 refuses the one that would reach another row (an `@` in
|
||||
the local part is a routed address: `src/trivial-rewrite/resolve.c:467`,
|
||||
`src/smtpd/smtpd_check.c:1724`, `allow_untrusted_routing=no`), and the
|
||||
pattern now reads addresses by Postfix's quoting all the same
|
||||
(`loggedAddress`).
|
||||
- *Held mail* (technical): a message the filter had held showed `queued`,
|
||||
"not tried yet", for as long as it stayed held. Now `held`, a status of its
|
||||
own (the CHECK of 0012, not shipped yet).
|
||||
- *Rows that never close, and queue ids that repeat* (both): before writing
|
||||
a message's rows the milter detaches older rows of its queue id; cleanup's
|
||||
`removed (canceled)` detaches rows of a message never queued; an empty
|
||||
`{i}` writes no result.
|
||||
- *A bounce of null-sender mail* (technical): "The sender was sent a
|
||||
bounce" is wrong there - Postfix does not bounce a bounce. The sheet says
|
||||
so.
|
||||
- Accepted, said in security.md: a row whose insert waited past the milter's
|
||||
30 s timeout can miss its first line and stay `queued`. Not of this row:
|
||||
an old send-log row of a reused queue id and the same recipient takes the
|
||||
line first (sendlog.go, as before).
|
||||
|
||||
## Steps
|
||||
|
||||
- [x] Migration 0012, store: write the queue id, update by queue id and
|
||||
recipient, read the result back.
|
||||
- [x] Milter: the queue id and `queued` on accepted rows.
|
||||
- [x] Tailer: delivery and expiry lines of port-25 mail to the journal.
|
||||
- [x] View, template, help; tests (store, milter, tailer, view).
|
||||
- [x] e2e: a relayed message's row shows `sent` with the sink's `250`; an
|
||||
upstream that refuses connections shows `deferred` with the reason.
|
||||
- [x] Docs. *Done (2026-10-16), with the reviews' findings above.*
|
||||
- [x] Roll out, read the server's journal; cut `2.10.0`. *Done (2026-10-16): on the
|
||||
test server as 2.9.1-post.d515493, a message to selfpost@dev.mixdep.ru through
|
||||
port 25 - its row sent, mc-ent.mixfed.ru's "250 2.0.0 Ok: queued as
|
||||
04F0D1023EF" on it.*
|
||||
@@ -1,257 +0,0 @@
|
||||
# Plan: panel-dkim-signing
|
||||
|
||||
**Status:** shipped `2.8.0` (2026-10-16); agreed 2026-10-15 - the row is agreed (owner, 2026-10-15, the
|
||||
order port25-bare-newline -> base-trixie -> panel-dkim-signing); three
|
||||
questions below were settled one at a time the same day.
|
||||
**Date:** 2026-10-15
|
||||
**Version:** `2.8.0` MINOR (after base-trixie, shipped `2.7.0`). No migration;
|
||||
nothing for an operator to do.
|
||||
|
||||
---
|
||||
|
||||
## Goal
|
||||
|
||||
The panel signs outgoing mail itself, in its own milter, and OpenDKIM
|
||||
leaves the image. The analysis ([own-mta-builds](../own-mta-builds-2026-10-15.md)
|
||||
§ 3 E) and the roadmap section give the reasons: OpenDKIM is a 2018 beta
|
||||
with three open 2026 CVEs and nobody upstream; every OpenDKIM limit
|
||||
SelfPost works around goes with it (the syslog relay, the From: parser
|
||||
port and its 256-byte refusal, the tables reloaded by a cross-uid signal
|
||||
and `CAP_KILL`, keys shared through a group).
|
||||
|
||||
What stays the same for an operator: the keys, the selectors, the DNS
|
||||
records, the panel's pages (one label on Health excepted, Q3).
|
||||
|
||||
## What the signer does
|
||||
|
||||
- **Library:** `github.com/emersion/go-msgauth/dkim` v0.7.0 (MIT; the
|
||||
author of the go-milter v0.4.1 we run; the e2e suite verifies with its
|
||||
v0.6.8 today). Read for this plan, `dkim/sign.go`: `NewSigner` hashes the
|
||||
body as it is written (an `io.Pipe` into a goroutine), so a message is
|
||||
not buffered; `HeaderKeys` is written to `h=` in the order given, each
|
||||
name picked last instance first (`header.go`, `newHeaderPicker`), and a
|
||||
name listed once more than it occurs is kept in `h=` with nothing hashed
|
||||
for it - the over-signing RFC 6376 § 5.4.2 describes; RSA-SHA256; the
|
||||
signature is the caller's to insert.
|
||||
- **What is signed:** `relaxed/simple`, as `build/opendkim.conf` signs
|
||||
today. `h=`: every header of the message whose name is in OpenDKIM's
|
||||
`dkim_should_signhdrs[]` (`libopendkim/dkim.c:223-246`: from, reply-to,
|
||||
subject, date, to, cc, the five resent-*, in-reply-to, references, the
|
||||
seven list-*), each occurrence, in message order, plus `From` once more
|
||||
(`OversignHeaders From` today). Absent names are not listed, as
|
||||
OpenDKIM does not list them.
|
||||
- **Which key:** on 465/587 the key of the domain the SASL login belongs
|
||||
to - `{auth_authen}`, then `Bindings.DomainOf(login)`, the table the
|
||||
sender check already reads (`internal/milter/sender.go:262-269`). For the
|
||||
panel's own notifications (`sendmail(1)`, `non_smtpd_milters`) the key of
|
||||
the server's name, as `EnsureServerSigned` provides it today. The sender
|
||||
check has already held `From:` to that same domain, so the signature's
|
||||
`d=` is the `From:` domain DMARC aligns on.
|
||||
- **What Postfix hands a milter,** read in Postfix 3.10.13,
|
||||
`src/milter/milter8.c:2329-2400`, `milter8_header()`: Postfix's own
|
||||
`Received:` is not shown (and an inserted header at index 0 lands above
|
||||
it, "relative to the message as delivered"); a multi-line header comes
|
||||
as text separated by `\n`; one space after the colon is eaten unless the
|
||||
milter asks for `SMFIP_HDR_LEADSPC` (`OptHeaderLeadingSpace`, which the
|
||||
sender check already asks for); body records end in `\r\n`
|
||||
(`milter8_body()`, :2470-2500). The signer rebuilds each header as
|
||||
`Name:` + value with `\n` made `\r\n` - go-msgauth's own
|
||||
`cmd/dkim-milter` writes `name + ": " + value + "\r\n"` and would sign a
|
||||
folded header wrongly. A unit test signs folded, tab-led and
|
||||
space-free headers and an independent verifier checks them.
|
||||
- **When it cannot sign** (no key file, a login with no domain, a read
|
||||
error): temporary failure, `default_action=tempfail`, as OpenDKIM is
|
||||
attached today. A message is never sent unsigned by accident.
|
||||
- **8-bit mail:** `disable_mime_output_conversion=yes`. Postfix converts an
|
||||
8-bit message to 7-bit for a server that does not announce 8BITMIME
|
||||
(default `no`, `src/global/mail_params.h:3088`), which breaks any body
|
||||
hash - OpenDKIM's today as well. Not set until now; set with this row.
|
||||
|
||||
## Decisions settled before this plan
|
||||
|
||||
From the roadmap section and the analysis, agreed with the row: the
|
||||
library; `relaxed/simple`; the header set and `From` over-signed; the key
|
||||
by the login's domain and by the server's name for local mail; strict;
|
||||
the removals below; keys, selectors and records unchanged; the e2e oracle
|
||||
an independent verifier; a message to a real mailbox before the cut.
|
||||
|
||||
Decided here (technical, the owner may overrule):
|
||||
|
||||
- **The oracle is dkimpy** (`python3-dkim` 1.1.8-2 in trixie,
|
||||
`/usr/bin/dkimverify`, seen in `debian:trixie` 2026-10-15) - a separate
|
||||
implementation in another language, maintained, packaged by Debian; not
|
||||
OpenDKIM's `opendkim-testmsg` (the code this row removes) and not
|
||||
go-msgauth (the signer's own library). It runs in the e2e sink's
|
||||
network against the stand's CoreDNS.
|
||||
- **A cross-check before OpenDKIM goes:** on the stand of stage 1 both
|
||||
sign the same messages for a while (OpenDKIM after the panel in the
|
||||
chain) and the two signatures' `bh=` and `h=` are compared on every
|
||||
e2e message; they must agree before stage 2 removes OpenDKIM.
|
||||
- **The From: parser port goes.** It exists so that the sender check
|
||||
judges the address OpenDKIM chooses the key by. With the key chosen by
|
||||
the login, the check keeps `net/mail` (one RFC 5322 address at an
|
||||
allowed domain) and the 256-byte refusal goes; the refusal texts that
|
||||
name OpenDKIM are rewritten.
|
||||
|
||||
## Open questions (to the owner, one at a time)
|
||||
|
||||
- **Q1. Where the signer runs.** (a) Its own role in the panel process on
|
||||
two sockets of its own: `sign.sock` for 465/587 (the key by the login)
|
||||
and `sign-local.sock` for local submission (the server's key only), in
|
||||
the chain where OpenDKIM is now: sender check, signer, journal. (b)
|
||||
Inside the sender check's milter: one socket fewer, but the check would
|
||||
receive every body, and local mail - which has no login - would need the
|
||||
same milter to tell "no login" from "local", deciding a key by the
|
||||
absence of something. **Recommended: (a)** - the socket a message comes
|
||||
through, not a missing login, decides that the server's key may sign
|
||||
it; the sender check stays a header-only milter; and moving the signer
|
||||
into a process of its own later is a change of where it is started.
|
||||
**Decided (owner, 2026-10-15): (a).**
|
||||
- **Q2. The key directory's name.** `/data/opendkim/keys/<domain>/<sel>.private`
|
||||
stays as it is, or becomes `/data/dkim/...` (the entrypoint moves the
|
||||
tree once; KeyTable and SigningTable are deleted). A backup restores
|
||||
only into the image of its own version, so no older archive meets the
|
||||
new name. **Recommended: rename** - the cost is the entrypoint's move,
|
||||
the paths in the code and the backup tests; a directory named after a
|
||||
program the image no longer has would mislead whoever reads `/data`.
|
||||
**Decided (owner, 2026-10-15): rename to `/data/dkim`.**
|
||||
- **Q3. Health.** The Overview's socket check and the process list name
|
||||
OpenDKIM and the syslog relay. **Recommended:** the socket row becomes
|
||||
"DKIM signer" (the panel's `sign.sock`, checked the same way), the two
|
||||
programs leave the process list. A mockup first, as for any page
|
||||
change.
|
||||
**Settled 2026-10-15:** shown to the owner as a picture; it follows from
|
||||
the card's own rule (a row per milter socket Postfix needs, and the
|
||||
signer is strict), so it is not his choice - the two rows as drawn.
|
||||
|
||||
## Stages
|
||||
|
||||
### Stage 1 - the signer, beside OpenDKIM (no release) - done `14de9a0`
|
||||
|
||||
- [x] `internal/milter/sign.go`: the session (Connect, MailFrom with
|
||||
`{auth_authen}`, Header, Headers, BodyChunk, Body), `OptHeaderLeadingSpace`,
|
||||
`InsertHeader(0, "DKIM-Signature", ...)` (the server's `Actions` gain
|
||||
add-header), the key read per message from the key file (a small cache
|
||||
keyed by path and mtime), the selector from the store as `Record` reads
|
||||
it.
|
||||
- [x] Unit tests: sign and verify with go-msgauth; folded and tab-led
|
||||
headers; a header absent and present twice; an empty body; a body
|
||||
without a final newline; a login with no domain and a missing key file
|
||||
answer tempfail; the server's socket never signs with a domain's key.
|
||||
- [x] `cmd/panel`: the role, its two sockets (`listenMilter`, group
|
||||
selfpost, 0660), the wrapper waits for them.
|
||||
- [x] `build/postfix-config.sh`: `smtpd_milters` = sender, sign, OpenDKIM,
|
||||
journal (OpenDKIM kept for the cross-check); `non_smtpd_milters` =
|
||||
sign-local, OpenDKIM; `disable_mime_output_conversion=yes`.
|
||||
- [x] e2e: dkimpy in the stand; every positive send and the notification
|
||||
verified by dkimpy against the panel's signature; the panel's and
|
||||
OpenDKIM's `bh=`/`h=` compared.
|
||||
|
||||
*Done (2026-10-15, `14de9a0`):* e2e exit 0 on the server; on an
|
||||
application's message and on a notification the panel's and OpenDKIM's
|
||||
signatures agree on `h=`, `bh=`, `c=`, `s=`, `a=` and dkimpy verifies both.
|
||||
Two findings while writing it: a folded header reaches a milter
|
||||
`
|
||||
`-separated (the signer rebuilds CRLF), and go-milter calls no Abort on a
|
||||
dropped connection (a one-hour watchdog closes the signer).
|
||||
|
||||
### Stage 2 - OpenDKIM removed (no release) - done
|
||||
|
||||
- [x] Chain: OpenDKIM out of both milter lists.
|
||||
- [x] Go: `internal/domain/opendkim.go` reduced to key storage (tables,
|
||||
`Rebuild`, `assertConfigSafe`, the USR1 reload go), `internal/supervisor`
|
||||
(its only caller is the reload), `internal/syslogrelay`,
|
||||
`internal/milter/mailparse.go` and its tests and testdata, the
|
||||
sender-check texts; Resync's "OpenDKIM's tables" step; Health's
|
||||
liveness set and socket row (Q3); `OPENDKIM_SOCKET`/`OPENDKIM_DIR`.
|
||||
- [x] Image: the `opendkim` package, `opendkim.conf`, the two supervisord
|
||||
programs, the `/dev/log` link and `/run/opendkim`, `usermod -aG selfpost
|
||||
opendkim`, crashexit's spared list; `deploy/docker-compose.yml`'s `KILL`
|
||||
capability (its only use was the reload) - **kept, see below**.
|
||||
- [x] Keys: `/data/opendkim/keys` -> `/data/dkim/keys` (Q2): the entrypoint
|
||||
copies the tree once and leaves the old one for a rollback to 2.7.x until
|
||||
the next release; KeyTable and SigningTable go with the new layout.
|
||||
*2.9.0:* the entrypoint removes the old tree once `/data/dkim` holds a
|
||||
key, and stops on a `/data` with keys under the old tree only; the copy
|
||||
and the backup exclusion are gone.
|
||||
- [x] Tests: `test/upstream/*.sh`, e2e's OpenDKIM log and process checks,
|
||||
the 256-byte test rewritten for the new rule.
|
||||
|
||||
*Done (2026-10-16):* `internal/domain/keys.go` (`Keys`, `NewKeys`,
|
||||
`DKIM_DIR`) keeps the keys and writes a selector table,
|
||||
`/data/dkim/selectors`, a line `<name> <selector>` each, which the signer
|
||||
reads instead of the KeyTable: the signer reads no database, as the sender
|
||||
check reads none - the panel's one SQLite connection is shared with the
|
||||
pages, and a long query would hold up mail. The table is written before the
|
||||
signer serves (`EnsureServerSigned`, moved there from the notifier), so the
|
||||
first start of 2.8.0 needs nothing of the operator. Keys and table are
|
||||
0600, directories 0700, owned by the panel; the entrypoint copies the old
|
||||
keys as the panel user (`setpriv`), into `/data/dkim.new`, renamed when
|
||||
whole, and stops the start if the copy fails - else the server's name would
|
||||
get a new key. Three findings: `KILL` stays - measured on the test server,
|
||||
root without it cannot signal a process of another uid, and supervisord
|
||||
stops the panel and Postfix's master its children that way (the plan's
|
||||
"only use" was wrong); the sender check's `Allowed` took the domain through
|
||||
`strings.ToLower`, which makes the Kelvin sign a "k" - the comparison with
|
||||
OpenDKIM's reading had hidden it, it now folds ASCII only; the Milter
|
||||
sockets card told the spam filter by its row having a note, which the
|
||||
signer's rows have too. The e2e drops the OpenDKIM-only cases
|
||||
(`testNeverSignedForAnother`) and checks the one signature, the tree's
|
||||
modes and the cited packages without opendkim, libopendkim11 and
|
||||
libmilter1.0.1.
|
||||
|
||||
### Stage 3 - documents, the canary, the cut
|
||||
|
||||
- [x] architecture § OpenDKIM -> § DKIM signing; security.md (the
|
||||
mail-path bullets, the accepted risks that were OpenDKIM's); guide;
|
||||
development's cited-package table (opendkim and libopendkim11 leave);
|
||||
the assumptions register's header notes what left; help (design commit
|
||||
first) and Health's label (Q3, mockup first).
|
||||
- [x] Reviews: technical and security.
|
||||
|
||||
*Done (2026-10-16):* help and Health went with stage 2 (design `005922b`);
|
||||
the reviews ran on stages 1-2 and their findings are fixed in `c2f63d2`,
|
||||
with the first rollout's incident (the queue's owner numbers); the
|
||||
documents in the commit after it. `Sender:` is held to the login like
|
||||
`From:` (security review F1; the owner's decision, 2026-10-16). go-milter
|
||||
v0.4.1 kept every header in a map of its own per session, on 465/587 and on
|
||||
port 25 alike: the panel builds it patched not to
|
||||
(`third_party/go-milter/PATCHES.md`; the owner's decision, 2026-10-16).
|
||||
- [x] On the test server: a day of the owner's mail; one message from an
|
||||
application through 465 and one notification to a real mailbox, the
|
||||
headers read (dkim=pass with the panel's `d=`).
|
||||
- [x] Cut `2.8.0`.
|
||||
|
||||
*Done (2026-10-16):* the owner's message through 465 (`dev/send-check.ps1`)
|
||||
arrived at Gmail with `dkim=pass header.i=@carrel.mixfed.ru
|
||||
header.s=selfpost`, `spf=pass`, `dmarc=pass`, and at Yandex with
|
||||
`dkim=pass` and `X-Yandex-Spam: 1`; its `h=` over-signs the shown headers.
|
||||
His `dev/from-check.ps1` sent thirteen `From:`/`Sender:` cases: four
|
||||
accepted and delivered, nine refused as expected, each a send-log row of
|
||||
reason `from` or `sender`. The notification to a real mailbox was not
|
||||
sent - nothing raised one; its signature is verified by dkimpy on every e2e
|
||||
run, and the server name's key, unchanged by the move (digests compared),
|
||||
matched its published record on 2026-10-15. The day of mail was cut short
|
||||
at the owner's word ("Выпускай", 2026-10-16), the test server having run
|
||||
the stage's code since 2026-10-15 22:48 UTC. Cut as `2.8.0`, no tag.
|
||||
|
||||
## Risks
|
||||
|
||||
- **A signing defect fails DMARC for every tenant at once.** The answer is
|
||||
the oracle (another implementation), the cross-check against OpenDKIM on
|
||||
the same messages before it goes, and the canary before the cut.
|
||||
- **go-msgauth is small and released slowly.** It can be vendored if it
|
||||
ever matters; the signing path is about 300 lines of it.
|
||||
- **Rollback:** until stage 2 ships, the chain change is one line; after
|
||||
the cut, the previous image (2.7.x) still has OpenDKIM and reads the keys
|
||||
from the old path, which the move leaves as a copy until the next release
|
||||
- the keys as they were at the upgrade: a domain added under 2.8.x is not
|
||||
in that tree, and 2.7.x sends its mail unsigned; one re-added under 2.8.x
|
||||
has another key there. 2.7.x reads the queue as its own postfix (uid
|
||||
101) before it hands it over, and 2.8.0 left it to uid 100: the guide
|
||||
(§ Upgrading) gives the hand-over by name to run in the 2.7.x image first,
|
||||
measured on a copy of the test server's queue. Going forward again after
|
||||
a rollback, a domain added under 2.7.x has its key only in the old tree:
|
||||
copy its directory into `/data/dkim/keys` (removing `/data/dkim` instead
|
||||
copies the old tree again and loses the keys of domains added under
|
||||
2.8.x).
|
||||
@@ -1,119 +0,0 @@
|
||||
# Plan: panel-notifications
|
||||
|
||||
**Status:** shipped in `2.3.0` (2026-10-12); agreed the same day - the design questions were closed with the owner and he took it into work ("Делаем")
|
||||
**Date:** 2026-10-12
|
||||
**Version:** `2.3.0` MINOR (one migration; opt-out by a switch).
|
||||
|
||||
---
|
||||
|
||||
## Goal
|
||||
|
||||
The panel writes to its administrators when something needs them, instead of
|
||||
waiting to be opened.
|
||||
|
||||
## Decisions (owner, 2026-10-12)
|
||||
|
||||
### Events of the first version
|
||||
|
||||
Four, each of them something the panel can already tell and a person has to
|
||||
act on:
|
||||
|
||||
| Event | Who is told |
|
||||
|---|---|
|
||||
| A sending domain's DNS record stopped matching (SPF, DKIM, DMARC) | global users, and `domain` users the domain is assigned to |
|
||||
| A level-2 rate limit refused mail | the same, for the domain of the limit |
|
||||
| The TLS certificate is about to expire | global users |
|
||||
| A message has been in the queue for more than **three days** | global users |
|
||||
|
||||
Three days, not one (owner: "one day is not that much"): Postfix keeps a
|
||||
message five, so two are left to act before it bounces. The threshold is a
|
||||
constant, not a setting.
|
||||
|
||||
Left for later: a process or milter socket down (a relay in that state often
|
||||
cannot send), an inbound upstream unreachable, DMARC failures appearing
|
||||
(noisy). Mail held in quarantine is not an event: quarantine is an idea
|
||||
outside the roadmap.
|
||||
|
||||
### Sender and path
|
||||
|
||||
From `postmaster@<SELFPOST_HOSTNAME>`, submitted locally. Not from one of the
|
||||
sending domains (owner): with several domains the choice is arbitrary, and a
|
||||
message about the certificate has no business coming from a domain.
|
||||
|
||||
These messages have to arrive (owner: "we need the best deliverability for
|
||||
them"), so the server's own name gets what a sending domain has:
|
||||
|
||||
- a **DKIM key for the hostname**, generated by the panel, with which
|
||||
notifications are signed - no domain has to be chosen for it. An SPF record
|
||||
alone would pass DMARC, but not after the user's mailbox forwards the
|
||||
message on, and large receivers treat unsigned mail more strictly;
|
||||
- an **SPF record** for the hostname, `v=spf1 a -all`.
|
||||
|
||||
The records themselves are under **Server › Settings** (owner: "the DNS
|
||||
records go to Settings, Health has only the status"), in a box of their own
|
||||
drawn with the record component of a domain's DNS page: host, value to
|
||||
publish, copy buttons, state. **Server › Health** has a card of its own,
|
||||
*Notification mail*, with the state of each record and a link to Settings.
|
||||
A DMARC record for the hostname is not asked for: the name inherits its
|
||||
parent domain's policy, and with SPF and DKIM aligned it passes any.
|
||||
Without the records notifications are still sent, and the card warns.
|
||||
|
||||
Notifications are the panel's own mail: they are not rows of the Outbound log
|
||||
and count against no level-2 limit.
|
||||
|
||||
A relay that cannot send cannot say so; none of the four events is "the relay
|
||||
is down", and that is why.
|
||||
|
||||
### Who chooses what
|
||||
|
||||
- One switch for the instance under **Server › Settings**.
|
||||
- One checkbox per user under **Account**, "send me notifications" - on when
|
||||
the profile has an e-mail. No choice per event: four events, none of them
|
||||
noise. Per-event choice comes when there are more and some are.
|
||||
- A user without an e-mail gets nothing.
|
||||
|
||||
### When a message is sent
|
||||
|
||||
A background check once an hour for DNS, certificate and queue.
|
||||
|
||||
| Event | Sent | "Cleared" message |
|
||||
|---|---|---|
|
||||
| DNS mismatch | after two checks in a row found it (it has held for an hour). A resolver that did not answer is not a mismatch | after two good checks in a row |
|
||||
| Certificate | 14 days before expiry, and again 3 days before if it is still the same certificate | none |
|
||||
| Level-2 limit refused mail | at the first refusal of a sending domain's applications, then at most once a day for that domain (one subject per domain, as built: what the refused row says of its domain is the client's own word, so the registry is asked) | none |
|
||||
| Queue older than three days | when such a message appears | one, when none is left |
|
||||
|
||||
Event state is kept in the database, so a restart announces nothing a second
|
||||
time.
|
||||
|
||||
## To settle while building
|
||||
|
||||
- How the panel submits: `sendmail` as the unprivileged panel user, and what
|
||||
that needs of the image.
|
||||
- Whether a notification carries a link to the panel - there is no setting
|
||||
for the panel's own address.
|
||||
- Where the refusal of a level-2 limit is noticed: in the journal milter as
|
||||
it refuses, or from the `rejected` rows of the send log on the hourly check.
|
||||
- Schema: the user's checkbox, the instance switch, the event state table.
|
||||
|
||||
## Screens
|
||||
|
||||
Not drawn. The Notifications box was taken out of the `account` and
|
||||
`settings` mockups on 2026-10-10 (owner: 2.0 ships no box for a feature that
|
||||
is not built). Drawing them again - one checkbox on Account, one switch on
|
||||
Server › Settings, the SPF check on Server › Health - is the first step, shown
|
||||
to the owner before any code ([panel-redesign.md](panel-redesign.md)
|
||||
§ Enforcement).
|
||||
|
||||
## Implementation checklist
|
||||
|
||||
- [x] Mockups: Account checkbox, Settings switch and records, Health status card, help topic; accepted by the owner 2026-10-12 (`design:` commit)
|
||||
- [x] Migration and store: user flag, instance switch, event state
|
||||
- [x] Sending: local submission from `postmaster@<hostname>`, message text (`sendmail` as the panel user; signed through `non_smtpd_milters` with a key of the server's own name; no link in the text - the panel does not know its own address)
|
||||
- [x] Background checker: DNS, certificate, queue; the rules of *When a message is sent*
|
||||
- [x] Level-2 refusal event (from the `rejected` rows of the send log, looked at every minute)
|
||||
- [x] Panel: Account, Settings, Health card, help
|
||||
- [x] guide.md, architecture.md, security.md, schema-migrations.md
|
||||
- [x] `go vet`, `go test`, e2e: a level-2 refusal reaches a mailbox as a signed notification (on the test server: vet ok, 23 packages ok, e2e ok; "once" and "clears" are held by the unit tests of `internal/notify`)
|
||||
- [x] Technical review — **Opus** (2026-10-12; no Critical. Fixed: refusals counted by the registered domain and only under a limit, with an index of their own; account e-mails may be at public mail hosts; "said" means sent; DNS checks must be 45 minutes apart; the notifier's own mail is not a queue subject and a cut or unreadable listing is no listing; the tables must name the selector the panel shows; an unanswered `a`/`mx` lookup is not an SPF mismatch; a pass stops between domains at shutdown; days are rounded as a person counts. Left as found: a recipient whose copy failed while another's went out is not retried (since 2.5.4 it is: audit 2026-10-12, stage 4); a certificate that lives under 14 days is reported at each renewal)
|
||||
- [x] Security review — **Fable** (2026-10-12; no Critical. Fixed: the limit notification was built on the domain a client writes in `MAIL FROM` - a mail cannon and a way to write to another tenant's users; a ceiling of 60 messages an hour; [security.md](../security.md) corrected on who can submit locally and on what the account e-mail now is. Recorded as accepted, with a roadmap candidate: an application can have a message signed as the server by writing its name into `From:` - the header is not held against the login, for sending domains either)
|
||||
@@ -1,15 +1,16 @@
|
||||
# Plan: panel-redesign
|
||||
|
||||
**Status:** shipped in `2.0.0` (2026-10-11)
|
||||
**Date:** 2026-09-21 (replaces the 2026-09-08 givens)
|
||||
**Version:** `2.0.0` MAJOR — a breaking release by the owner's decision (§ No compatibility).
|
||||
**Status:** the design contract of the panel; built and shipped in `2.0.0`
|
||||
(2026-10-11), a breaking release by the owner's decision (§ No compatibility).
|
||||
Its implementation checklist (27/27) is in the git history.
|
||||
**Date:** 2026-09-21 (replaces the 2026-09-08 givens)
|
||||
|
||||
---
|
||||
|
||||
## What this file is
|
||||
|
||||
The design contract for the control panel and the plan to implement it. The
|
||||
design itself is drawn and accepted:
|
||||
The design contract for the control panel. The design itself is drawn and
|
||||
accepted:
|
||||
|
||||
| Artefact | Role |
|
||||
|---|---|
|
||||
@@ -72,9 +73,9 @@ and the migration chain is deleted to zero.** The redesign is `2.0.0`.
|
||||
- **Schema.** `internal/store/migrations/` is emptied and replaced by one
|
||||
baseline, `0001_init.sql`, equal to the 2.0 schema (including the new
|
||||
assignment table below). No historical `admin` table, no create-then-drop.
|
||||
This is [schema-squash](../roadmap.md#schema-squash) with its trigger now
|
||||
decided and without the upgrade gate that item planned: 2.0 starts from an
|
||||
empty data directory.
|
||||
This was the roadmap item schema-squash, with its trigger decided and
|
||||
without the upgrade gate that item planned: 2.0 starts from an empty data
|
||||
directory.
|
||||
- **Docs.** `schema-migrations.md` shrinks to the 2.0 baseline; the 1.x chain
|
||||
stays in git history.
|
||||
|
||||
@@ -145,8 +146,7 @@ shown another user's login name, and three explicit choices are simpler.
|
||||
- The 2.0 baseline has `users.email`: the user's own address, edited on
|
||||
**Account**, set by the global administrator on the user form and shown in
|
||||
the users list. It is what the panel writes to — event notifications
|
||||
([panel-notifications](../roadmap.md#panel-notifications)) and a reset link
|
||||
([password-reset](../roadmap.md#password-reset)) — and what the fill button
|
||||
(`2.3.0`) and a reset link (`2.14.0`) — and what the fill button
|
||||
offers. **Account** is Profile and Password; Server › Settings keeps nothing
|
||||
of DMARC.
|
||||
- Baseline: `domains.dmarc_rua` alone (`''` = no reports). The hosted choice is
|
||||
@@ -226,38 +226,24 @@ shell, and its menu entry is hidden if the page does not exist.
|
||||
|
||||
## Room for roadmap items
|
||||
|
||||
The mockups draw today's features, the four rows above, and the screens of
|
||||
the open roadmap items — none of them needed a new component or layout. A
|
||||
mockup of a feature is its target look, not a decision to build it: each item
|
||||
keeps its own status and plan. From here on a feature's screens are drawn
|
||||
The mockups drew the 2.0 pages and the screens of the roadmap items of the
|
||||
time; none of them needed a new component or layout. A mockup of a feature is
|
||||
its target look, not a decision to build it. A feature's screens are drawn
|
||||
mockup-first in this system (`panel/src/`, `build.py`, `check.py --write`) as
|
||||
the first step of that feature's plan — not invented during implementation.
|
||||
the first step of that feature's work, not invented during implementation.
|
||||
|
||||
| Roadmap item | Where it lands | Built from | Drawn |
|
||||
|---|---|---|---|
|
||||
| inbound-antispam-panel (agreed, `2.1.0`) | journal = **Inbound › Log** (add client IP and engine to the mockup's columns; `quarantine` joins the decisions); lists = new **Inbound › Filter lists**, global-only because the lists are instance-wide | list layout: add form + two tables, `status_tag`, filter form in `box_head`; the log links *Allow / Deny sender* to the lists | `in-log`, `in-filter-lists` |
|
||||
| inbound-quarantine (an idea, off the roadmap since 2026-10-12) | new **Inbound › Quarantine**, list + detail with *Release* / *Discard* | list layout; detail like `out-message` with `facts`, `log_pane` for headers only, Release beside a `sp-danger-zone` Discard | `in-quarantine`, `in-quarantine-message` — the open questions of its plan (where mail lives, what release means, retention, RBAC) stay open; the mockup assumes SelfPost-held mail, release to upstream, 14 days, delegated like the rest of Inbound |
|
||||
| preflight (candidate) | new **Server › Preflight**, next to Health: Health is what runs now, Preflight is the deeper on-demand installation check with a test e-mail form | `postmark` verdict, a table of checks with `status_tag` and a *What to do* column, one form box for the test e-mail | `preflight` |
|
||||
| panel-notifications (candidate) | one checkbox per user on **Account** (no choice per event, owner 2026-10-12); one switch under **Server › Settings** | a checkbox in a box, `help` | **no** — drawn once, taken out of `account` and `settings` on 2026-10-10 (owner): 2.0 ships no box for a feature that is not built; redrawn as the first step of its plan |
|
||||
| password-reset (candidate) | a link on sign-in, two signed-out screens (request, set new password) | the signed-out layout | **no** |
|
||||
| delivery-log-storage (candidate) | the *Delivery log* box of **Outbound › Log › message**, filled from a table instead of a grep of today's `mail.log` | `log_pane`, unchanged | `out-message` |
|
||||
| csrf-tokens | no screen; every form gains a hidden field — forms are rendered through one helper so the token cannot be forgotten | — | n/a |
|
||||
| template-data-typing | no screen; the component partials are the natural first users of typed data | — | n/a |
|
||||
| structured-logging, contributing, review follow-ups | none | — | n/a |
|
||||
Drawn and not built (`deferredScreens` in the guard tests):
|
||||
|
||||
| Screen | Item | State |
|
||||
|---|---|---|
|
||||
| `in-queue` | [queue-table](../roadmap.md#queue-table): the queue as a table from `postqueue -j`, and an Inbound › Queue | undecided; Outbound › Queue shows postqueue's listing in a log pane (`outlineExempt`) |
|
||||
| `in-quarantine`, `in-quarantine-message` | inbound-quarantine, an idea off the roadmap ([its notes](inbound-quarantine.md)) | its open questions stay open; the mockup assumes SelfPost-held mail, release to upstream, 14 days, delegated like the rest of Inbound |
|
||||
| `preflight` | preflight as a page of its own, dropped 2026-10-13 ([roadmap](../roadmap.md#dropped-with-the-condition-for-return)); its checks went to Server › Health, the test e-mail to its own form | a leftover of a dropped item |
|
||||
|
||||
The sibling strip under the navbar holds five or six entries per group without
|
||||
wrapping; Inbound would reach five (Domains · Log · Queue · Quarantine ·
|
||||
Filter lists), Server six. Past that a group is split rather than squeezed.
|
||||
|
||||
Ordering with 2.0 (owner, 2026-10-09: feature order is not fixed, take
|
||||
whatever suits development): **the redesign ships first as `2.0.0`**, and
|
||||
inbound-antispam-panel follows as `2.1.0`, built from the component kit on
|
||||
the 2.0 baseline. The other way round would have styled two screens twice and
|
||||
written a 1.x migration only to dissolve it into the baseline. The antispam
|
||||
back end (journal milter, log tailer, list CRUD, rspamd map sync) touches
|
||||
nothing under `internal/web/view` and may be built alongside stages 0–1 below
|
||||
if time allows; its two screens wait for the kit.
|
||||
|
||||
## Three layers
|
||||
|
||||
1. **Bulma, as shipped.** Vendored `bulma.min.css`, never edited. Only the
|
||||
@@ -485,60 +471,3 @@ reviewed.
|
||||
- Dark scheme and the narrow-screen shell: Bulma ships both; they are drawn
|
||||
and agreed as a `design:` change before anyone implements them.
|
||||
- Weakening CSP; a JS bundler; a front-end framework.
|
||||
|
||||
## Implementation checklist
|
||||
Roadmap status: **agreed** (owner, 2026-10-09). Stages are ordered for
|
||||
development, not by feature priority (owner, 2026-10-09). Stage 1 touches no
|
||||
file under `internal/web/view` and may run alongside stage 0; stage 2 starts
|
||||
only when both are done. Every stage-2 step ends with the evidence of
|
||||
§ Evidence for "done".
|
||||
|
||||
**Decided**
|
||||
|
||||
- [x] Owner decision recorded here: keep today's paths or rename with redirects — **owner** (2026-09-21: rename, § Routes)
|
||||
- [x] Owner decision: no compatibility, migrations to zero, `2.0.0` — **owner** (2026-09-21, § No compatibility)
|
||||
- [x] Owner decision: inbound is delegated per user, separately from outbound — **owner** (2026-09-21, § Who sees what)
|
||||
- [x] Owner decision: the redesign ships before inbound-antispam-panel; feature order follows development convenience — **owner** (2026-10-09, § Room for roadmap items)
|
||||
- [x] Decision inside 2.0: **Server › Health** is built with the restyle (the tables that leave Overview need a home, § Routes); **queue as a table** and the **inbound queue** stay roadmap decisions — both queue pages ship old content in the new shell, the Inbound › Queue entry hidden — (2026-10-09)
|
||||
|
||||
**Stage 0 — foundation and component kit** (§ Component kit)
|
||||
|
||||
- [x] Vendor Bulma 1.0.4 and the icon subset (MIT) under `/static`, NOTICE and development.md rows, pinned checksum — **Haiku**
|
||||
- [x] Guard tests from § Enforcement and `TestRoutesFollowNavigation`, red against today's panel where expected, with the `legacyPages` ratchet listing every page; `design-first` CI step including the ratchet and the history-blob check (§ Against resurrected pages) — **Opus**
|
||||
- [x] `panel.css` from `panel/theme.css` + `shared/brand.css`; `components.html` partials with typed inputs in `components.go`; `status_tag`, `copy_field`, `wbr_at` — **Sonnet**
|
||||
- [x] `layout.html`: navbar, perforated edge, sibling strip, user menu, footer, visibility flags; the signed-out shell — **Sonnet**
|
||||
- [x] `GET /server/components` behind the global role rendering every partial from fixtures; kit acceptance (§ Component kit) with evidence — **Sonnet**, reviewed by **Opus**
|
||||
|
||||
**Stage 1 — data and routes** (no template work; parallel with stage 0)
|
||||
|
||||
- [x] Schema from zero: single `0001_init.sql` baseline with `users.email`, `user_inbound_domains`, the two `all_*` flags; store tests; `schema-migrations.md` rewritten — **Opus**
|
||||
- [x] Inbound delegation: `requireInboundDomain`, lists and log filtered in the query, two-list user form data, per-route 404 tests for both roles including another tenant's id — **Opus**
|
||||
- [x] Routes renamed per § Routes: new paths, one fragment name, old paths removed, subtree `requireGlobal()` for `/server/`; links in the old templates retargeted so the panel works on every commit — **Opus**
|
||||
- [x] The application form as one POST; `out-app-created` as the response with `Cache-Control: no-store` — **Opus**
|
||||
- [x] Server › Health route and handler (tables and *Reload configuration* from `/status`); Overview handler reduced to verdicts — **Opus**
|
||||
|
||||
**Stage 2 — pages, in groups** (each step: partial calls only, page off the ratchet list, evidence pairs)
|
||||
|
||||
- [x] Signed-out pages: login, setup — **Sonnet**
|
||||
- [x] Overview, Health, and the Account / Settings split — **Sonnet**
|
||||
- [x] Outbound: domains, domain, domain settings, application form, shown-once password, delete — **Sonnet**
|
||||
- [x] Outbound: log, message, DMARC hub / domain / report — **Sonnet**
|
||||
- [x] Inbound: domains, domain with the *Spam filter* box, delete — **Sonnet**
|
||||
- [x] Server: system log, backup, users, user form, delete; Help as `help_topic` partials, the drawer removed — **Sonnet**
|
||||
- [x] Queues: old content inside the new shell (outbound), Inbound › Queue hidden — **Sonnet**
|
||||
- [x] Remove the old `panel.css`, the drawer code in `panel.js` and every class nothing references; `legacyPages` empty, vocabulary test green with no allow-list — **Sonnet**
|
||||
|
||||
**Stage 3 — reviews, docs, 2.0.0**
|
||||
|
||||
- [x] Technical review of the whole against the mockups, evidence pairs complete — **Opus**
|
||||
- [x] Security review of the route rename, subtree authorization and inbound delegation — **Fable**
|
||||
- [x] guide.md screenshots and wording for the new navigation; architecture.md § Panel HTTP surface and § Persistence; security.md roles; *Removed* in the CHANGELOG with one line that 2.0 starts from an empty data directory; strike the three UI rows of review-2026-08-followups that the redesign retires — **Sonnet**
|
||||
- [x] `go vet`, `go test ./...`, e2e suite; version cut `2.0.0` — **Haiku**
|
||||
|
||||
**After 2.0** — features land in the kit, each from its own plan, mockup first
|
||||
(§ Room for roadmap items): inbound-antispam-panel `2.1.0`; then, in whatever
|
||||
order suits development, preflight, delivery-log-storage, panel-notifications,
|
||||
password-reset (two screens to draw first), queue as a table with the inbound
|
||||
queue, csrf-tokens
|
||||
(one hidden field in the form partial once security.md is settled).
|
||||
inbound-quarantine is an idea outside the roadmap.
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
# SelfPost — product boundaries
|
||||
|
||||
**What this file is.** Stable product definition for SelfPost v1.0: purpose,
|
||||
**What this file is.** Stable product definition of SelfPost: purpose,
|
||||
deployment assumptions, explicit out-of-scope items, and the multi-domain
|
||||
model. User-facing overview in [README.md](../README.md); install and operations
|
||||
in [guide.md](guide.md); as-built technical detail in
|
||||
|
||||
+78
-937
File diff suppressed because it is too large
Load Diff
@@ -17,8 +17,8 @@ When you add or review a migration, update this file in the same change.
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Chain head | `user_version = 13` |
|
||||
| Files | `0001_init.sql` (the 2.0 baseline), `0002_inbound_spam_log.sql`, `0003_send_log_lines.sql`, `0004_notifications.sql`, `0005_send_log_rejected_index.sql`, `0006_sender_check.sql`, `0007_delivery_log_days.sql`, `0008_tenant_boundary.sql`, `0009_refusal_line.sql`, `0010_send_log_indexes.sql`, `0011_notify_addressees.sql`, `0012_inbound_upstream.sql` |
|
||||
| Chain head | `user_version = 14` |
|
||||
| Files | `0001_init.sql` (the 2.0 baseline), `0002_inbound_spam_log.sql`, `0003_send_log_lines.sql`, `0004_notifications.sql`, `0005_send_log_rejected_index.sql`, `0006_sender_check.sql`, `0007_delivery_log_days.sql`, `0008_tenant_boundary.sql`, `0009_refusal_line.sql`, `0010_send_log_indexes.sql`, `0011_notify_addressees.sql`, `0012_inbound_upstream.sql`, `0013_email_links.sql`, `0014_upstream_tls_secure.sql` |
|
||||
| File mark | `PRAGMA application_id = 0x53503230` ("SP20") |
|
||||
| Database file | `/data/selfpost.db` (bind mount) |
|
||||
| Compatibility | 2.x MINOR releases must boot a `2.0.0` data directory. **Nothing is carried over from 1.x.** |
|
||||
@@ -91,6 +91,7 @@ audit (`2.5.0`), `0009_refusal_line.sql` (`2.5.1`) and
|
||||
| 11 | `0011_notify_addressees.sql` | 2.5.4 | DDL | `notify_addressees (key, address, told_at)`: who a subject's notification is owed to and who was handed it, so a copy that failed or was held back by the hourly ceiling goes out at a later check and a subject's end goes to those told (audit 2026-10-12, stage 4); deleted with its `notify_state` row |
|
||||
| 12 | `0012_inbound_upstream.sql` | 2.10.0 | DDL | `inbound_spam_log` gains `queue_id`, `upstream_status` (`''`, `queued`, `held`, `sent`, `deferred`, `bounced`; CHECK), `upstream_relay`, `upstream_reply` and `upstream_at`, and the partial index `idx_inbound_spam_log_queue_id` on rows that have a queue id: what the upstream answered about mail accepted on port 25, put on the row by the log tailer (inbound-upstream-result). Rows from before have none |
|
||||
| 13 | `0013_email_links.sql` | 2.14.0 | DDL | `users.email_confirmed_at` (`''` for every address saved before it: all unconfirmed) and `user_links`, the one-time links that confirm an address and reset a password, with `idx_user_links_user` (password-reset; owner, 2026-10-13 and 2026-10-17) |
|
||||
| 14 | `0014_upstream_tls_secure.sql` | 2.15.0 | DDL + data | `inbound_transports.tls_mode` may be `secure` (Verified: Postfix's level that checks the upstream's certificate against its name; upstream-tls-verify, security audit 2026-10-17, M7). SQLite cannot alter a CHECK, so the table is made anew with the wider one and its rows copied |
|
||||
|
||||
**Kind:** *DDL* — schema only; *data* — `INSERT`/`UPDATE` that must stay correct
|
||||
for operators upgrading from older 2.x images.
|
||||
@@ -113,7 +114,7 @@ for operators upgrading from older 2.x images.
|
||||
| `send_log` | Delivery journal. `reason` says why a row was refused (`limit`, `client`, `from`, and `sender` since 2.8.0) or that the sweep closed it (`reconcile`, `reconcile-contradicted`), `header_from` what the From: or Sender: line of a message refused for it said (v6), `refusal_line`, `client_ip` and `recovered` (v9), and `domain_id` (v8: the row belongs to a domain by id, `ON DELETE SET NULL`) |
|
||||
| `logtail_state` | Log-tailer read position |
|
||||
| `inbound_domains` | Inbound relay domains |
|
||||
| `inbound_transports` | Upstream host / port / TLS per inbound domain |
|
||||
| `inbound_transports` | Upstream host / port / TLS per inbound domain; `tls_mode` is `may`, `encrypt`, `secure` (v14) or `none` |
|
||||
| `inbound_recipients` | Allow-list when `recipient_mode = list` |
|
||||
| `user_inbound_domains` | Inbound domains assigned to a `domain` user — granted separately from the outbound ones |
|
||||
| `dmarc_reports` | Parsed aggregate report summaries |
|
||||
|
||||
@@ -1,5 +1,11 @@
|
||||
# Security audit — 2026-10-17
|
||||
|
||||
> **Stages 0-6 shipped in `2.14.4` (2026-10-17)**, the product side of S3
|
||||
> and the fix of M7 in `2.15.0` (2026-10-18). Stage H, the owner's host and
|
||||
> DNS, is open and tracked in [roadmap.md](roadmap.md#security-audit-2026-10-17).
|
||||
> The file stays because the code cites its finding ids (`A2`, `M7`...); its
|
||||
> `file:line` references are of the tree at `44bf09d`.
|
||||
|
||||
**What this is.** A full security audit of SelfPost at `44bf09d` (release
|
||||
`2.14.3`): the code tree, the image and its process model, the deployment
|
||||
fragments, CI/CD and the supply chain, the dependencies, and the live
|
||||
|
||||
+14
-22
@@ -365,8 +365,8 @@ operator can act on it.
|
||||
MIME boundary and the signature still verified), then once more each of the shown ones present
|
||||
(Subject, Date, To, Cc, Reply-To, Sender) and `From` last, so one of those
|
||||
added after signing breaks the signature. Until `2.8.0` OpenDKIM signed - a
|
||||
2018 beta with three 2026 CVEs open and nobody upstream to fix them
|
||||
([own-mta-builds](own-mta-builds-2026-10-15.md) § 1) - and chose the key
|
||||
2018 beta with three 2026 CVEs open and nobody upstream to fix them - and
|
||||
chose the key
|
||||
by the `From:` header; it left the image, and with it the syslog relay and
|
||||
`/dev/log`, the port of its `From:` parser, its tables and their reload by
|
||||
a signal across uids. Architecture § DKIM signing.
|
||||
@@ -445,8 +445,7 @@ operator can act on it.
|
||||
and later; `src/global/mail_params.h`, `DEF_SMTPD_FORBID_BARE_LF`), so
|
||||
they end DATA the same way; the port-25 setting stays explicit. Before
|
||||
`2.7.0` a smuggled transaction on a submission port was still the
|
||||
authenticated session's, under its sender bindings and milters
|
||||
([own-mta-builds](own-mta-builds-2026-10-15.md) § 5).
|
||||
authenticated session's, under its sender bindings and milters.
|
||||
- **The panel's milters judge every message of a connection.** Postfix
|
||||
introduces a client to a milter once per connection, and the milter library
|
||||
replaces the milter's session after any reply that is not "continue". Until
|
||||
@@ -651,7 +650,7 @@ deferred item from the roadmap.
|
||||
templates must contain no inline scripts and no inline styles. Decided by
|
||||
the owner on 2026-10-13: no tokens; the roadmap item that asked for them is
|
||||
dropped and the origin check is made strict instead
|
||||
([origin-strict](roadmap.md#origin-strict)). This bullet and the ADR below
|
||||
(origin-strict, `2.10.4`). This bullet and the ADR below
|
||||
are the single record of the decision, and the ADR's **Revisit if** says
|
||||
when tokens are built after all.
|
||||
- **Destructive-action confirmation (`data-confirm`) is JavaScript-only.**
|
||||
@@ -872,7 +871,8 @@ deferred item from the roadmap.
|
||||
seconds or the whole instance; Server › Health counts the
|
||||
downtime and the mail deferred for it. Trigger to revisit: those numbers
|
||||
not staying at nothing - then the check moves into a process of its own
|
||||
([plans/from-header-binding.md](plans/from-header-binding.md)).
|
||||
(a subcommand of the panel binary and a `[program:]` of supervisord; the
|
||||
downtime figure then needs a new meaning).
|
||||
- **A malformed packet on a milter socket of the panel ends the panel.** The
|
||||
milter library indexes into an empty packet and nobody recovers the panic,
|
||||
so the process exits and supervisord starts it again. The sockets are
|
||||
@@ -956,21 +956,13 @@ deferred item from the roadmap.
|
||||
uses with another mode is refused and told that mode - not the domain -
|
||||
so the form tells whether some domain forwards to a given host and port.
|
||||
Accepted with the rest of this item.
|
||||
- **Account's profile (username, e-mail) is saved without the password.**
|
||||
Saving the profile does not ask for the current password, as changing the
|
||||
password does. Since 2.3.0 the e-mail is where notifications go, so a stolen
|
||||
session can turn them to another address: the thief then learns what they
|
||||
say - which domain's DNS is wrong, which application is refused, when the
|
||||
certificate expires, sender addresses of old queued mail - and the user
|
||||
stops hearing it. Nothing in a notification gives access. A `domain` user
|
||||
can likewise point signed mail of this server at a third party's address,
|
||||
a dozen messages a day per assigned domain at most. The profile form also
|
||||
says when a username is taken, so a signed-in user can tell which
|
||||
usernames exist; signing in does not (`2.5.4`). It
|
||||
becomes a requirement to ask for the current password when changing
|
||||
`users.email` as soon as [password-reset](roadmap.md#password-reset) mails a
|
||||
link to it — from then on whoever can change the address could take the
|
||||
account over.
|
||||
- **Account's username is saved without the password.** Changing the
|
||||
e-mail on Account asks for the current password since `2.14.0`, on the
|
||||
password change's limit: it is where a reset link goes, so whoever could
|
||||
change it could take the account over. The username and the notification
|
||||
switch are saved without it. The profile form says when a username is
|
||||
taken, so a signed-in user can tell which usernames exist; signing in does
|
||||
not (`2.5.4`).
|
||||
|
||||
## ADR: CSRF via origin checking, without tokens
|
||||
|
||||
@@ -1023,7 +1015,7 @@ external-site case this ADR covers.
|
||||
|
||||
**Decision of 2026-10-13 (owner).** Asked to choose between this ADR, tokens,
|
||||
and a strict origin check, the owner chose the ADR with the strict check
|
||||
([origin-strict](roadmap.md#origin-strict)) and no tokens. Session-bound tokens
|
||||
(origin-strict, `2.10.4`) and no tokens. Session-bound tokens
|
||||
are the answer to return to - a hidden field in every state-changing form and
|
||||
htmx path, checked against the session, on top of the origin check - when
|
||||
either condition above comes true, and also when: a browser the panel must
|
||||
|
||||
@@ -340,9 +340,9 @@ type Usage struct {
|
||||
// counts for every entry of the deciding list that fits it: the journal does
|
||||
// not say which of two fitting entries the filter met first.
|
||||
//
|
||||
// A counter kept with each entry would not depend on the journal; the plan
|
||||
// (docs/plans/inbound-antispam-panel.md, Filter list usage) says when that is
|
||||
// worth a migration.
|
||||
// A counter kept with each entry would not depend on the journal;
|
||||
// docs/architecture.md (the antispam filter lists) says when that is worth a
|
||||
// migration.
|
||||
func (l *Lists) Usage() (map[int64]Usage, error) {
|
||||
entries, err := l.store.ListSpamListEntries()
|
||||
if err != nil {
|
||||
|
||||
@@ -8,7 +8,7 @@ import (
|
||||
)
|
||||
|
||||
// The sender check is attached to Postfix strictly: while the panel does not
|
||||
// answer on its socket, mail is deferred (docs/plans/from-header-binding.md).
|
||||
// answer on its socket, mail is deferred (docs/architecture.md § Sender check).
|
||||
// How many submissions that happened to is read here, from what Postfix
|
||||
// writes about it, and shown on Server > Health. It is the figure that also
|
||||
// sees a panel which is running and not answering.
|
||||
|
||||
@@ -22,7 +22,7 @@ import (
|
||||
|
||||
// The sender check: a message's From: line is held against the application
|
||||
// that sends it, under the rule Postfix holds the envelope sender to
|
||||
// (docs/plans/from-header-binding.md). The signer signs with the key of the
|
||||
// (docs/architecture.md § Sender check). The signer signs with the key of the
|
||||
// domain the login belongs to (sign.go), and a receiver holds that signature
|
||||
// to the domain of this line (DMARC, RFC 7489 § 3.1.1), so without the check
|
||||
// an application could send under the server's signature as another domain
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
package milter
|
||||
|
||||
// The DKIM signer (docs/plans/panel-dkim-signing.md): the panel's own milter
|
||||
// The DKIM signer (docs/architecture.md § DKIM signing): the panel's own milter
|
||||
// that signs what applications send on 465/587 with the key of the domain
|
||||
// their login belongs to, and what the panel itself sends with the key of
|
||||
// the server's name. Each of the two is a socket of its own, so the socket a
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
//
|
||||
// Each subject has a row of state in the database, so a thing is said once and
|
||||
// a restart says nothing again. The rules of when a message goes out are the
|
||||
// owner's (docs/plans/panel-notifications.md § When a message is sent) and
|
||||
// owner's (docs/architecture.md § Notifications) and
|
||||
// live in this file; what the messages say is in text.go, how they leave in
|
||||
// mailer.go.
|
||||
package notify
|
||||
|
||||
@@ -13,7 +13,7 @@ import (
|
||||
"github.com/mixeme/selfpost/internal/web/auth"
|
||||
)
|
||||
|
||||
// The panel pages of notifications (plans/panel-notifications.md): the user's
|
||||
// The panel pages of notifications (docs/architecture.md § Notifications): the user's
|
||||
// checkbox on Account, the instance's switch and the DNS records of the server's
|
||||
// own name on Server › Settings, and the state of those records on Server ›
|
||||
// Health. What is sent and when is internal/notify's and tested there.
|
||||
|
||||
@@ -88,8 +88,9 @@ func verifyDKIM(raw []byte, domain string) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
// checkPanelSignature is the oracle of docs/plans/panel-dkim-signing.md: the
|
||||
// message carries one DKIM signature, the panel's, for domain - signed as
|
||||
// checkPanelSignature is the oracle of the panel's DKIM signer
|
||||
// (docs/architecture.md § DKIM signing): the message carries one DKIM
|
||||
// signature, the panel's, for domain - signed as
|
||||
// OpenDKIM signed until 2.8.0 (relaxed/simple, rsa-sha256, From over-signed)
|
||||
// - and dkimpy (python3-dkim in the sink image), an implementation that is
|
||||
// not go-msgauth, the panel's library, verifies it. Stage 1 compared it with
|
||||
|
||||
@@ -54,7 +54,7 @@ var port25Cases = []port25Case{
|
||||
// The SMTP-smuggling defence. Postfix 3.10.13 has it by default
|
||||
// (src/global/mail_params.h, DEF_SMTPD_FORBID_BARE_LF "normalize");
|
||||
// set explicitly since 2.6.1, when the image ran Postfix 3.7.11,
|
||||
// whose default was "no" (own-mta-builds, § 5), and kept so the
|
||||
// whose default was "no" (docs/security.md § Mail path), and kept so the
|
||||
// listener does not depend on the default. testInboundBareNewline
|
||||
// drives it.
|
||||
{param: "smtpd_forbid_bare_newline", contains: "normalize", wantExact: true},
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
# the panel's signer added) straight off disk. smtp-sink ships in the postfix
|
||||
# package itself. python3-dkim (dkimpy 1.1.8-2 in trixie, /usr/bin/dkimverify)
|
||||
# is the independent DKIM verifier the suite holds the panel's signer to
|
||||
# (docs/plans/panel-dkim-signing.md); it looks the keys up through the
|
||||
# (docs/architecture.md § DKIM signing); it looks the keys up through the
|
||||
# stand's CoreDNS (compose.override.yml, sink's dns).
|
||||
FROM debian:trixie-slim
|
||||
|
||||
|
||||
Reference in New Issue
Block a user