# SelfPost — architecture (as-built) **Source of truth:** the code tree, not historical specs. Synchronise this file when env keys, routes, or mail-path behaviour change. Verification method: [development.md](development.md) § «Verifying docs against code». User install/operations: [README.md](../README.md), [guide.md](guide.md). Product boundaries: [product.md](product.md). --- ## Image and processes Single Debian slim image. `entrypoint.sh` (root) fixes `/data` ownership and milter socket directories, **requires** `SELFPOST_HOSTNAME` (FQDN with at least one dot; no scheme, port, or spaces — invalid or empty value → `exit 1` before Postfix config or supervisord), then execs `supervisord` as PID 1. The hostname is not a tunable default: it must match PTR/rDNS, TLS CN/SAN, and SASL realm together. Soft fallbacks (`localhost` in the panel vs container ID in Postfix) split realms and break SMTP AUTH with a silent `535` in clients while the panel looks healthy. Shipped `docker-compose.yml` already requires the variable; the entrypoint gate catches `docker run`, custom compose, and k8s. Managed programs ([build/supervisord.conf](../build/supervisord.conf)): | Program | User | Priority | Role | |---|---|---|---| | `opendkim` | root → `opendkim` (`UserID` in opendkim.conf) | 100 | DKIM signing milter | | `panel` | panel | 200 | HTTP UI + sender check + journal-milter + log-tailer + rate-limit recalc; with the inbound relay on also the inbound journal milter | | `postfix` | root (wrapper) | 300 | MTA — started only after the milter sockets exist (OpenDKIM's, the sender check's, the journal milter's and, with the inbound relay on, the inbound journal milter's) | | `postfix-reload` | root | — | On-demand `postfix reload` (autostart off) | | `cert-reload` | root | 400 | Daily: re-copies a renewed TLS certificate for Postfix and reloads it | | `logrotate` | root | 400 | Periodic `mail.log` rotation | Start order: OpenDKIM → panel (opens the sender check's and the journal-milter's sockets) → Postfix wrapper polls unix sockets (timeout `MILTER_WAIT_TIMEOUT`, default 30s) then `postfix start-fg`. `crashexit` event listener exits the container on any managed process FATAL so Docker `restart: unless-stopped` recreates a broken instance. **Liveness:** `GET /healthz` (unauthenticated) returns 200 when opendkim, panel, and postfix are RUNNING; Docker `HEALTHCHECK` uses the same probe. --- ## Mail path ``` Client ──TLS+SASL──► Postfix (465 smtps, optional 587 submission) │ ├─► sender check (From: against the login, tempfail on failure) ├─► OpenDKIM milter (sign, tempfail on failure) ├─► journal-milter (send log + L2 rate limits, fail-open) └─► outbound MX delivery (port 25 client) Internet ──► Postfix smtp inet :25 (when INBOUND_RELAY_ENABLE=true │ and/or DMARC_REPORTS_ENABLE=true) │ ├─► optional antispam milter (inbound relay only) ├─► DMARC aggregate ingest (pipe → dmarc-ingest) └─► smtp:[upstream]:port (inbound relay only; transport_maps; no local delivery) ``` The smtp/inet listener is **absent** when both flags are off (`postconf -MX smtp/inet` removes Debian's stock smtpd). Outbound delivery still uses the `smtp unix` client; it is not the same service. ### Postfix ([build/postfix-config.sh](../build/postfix-config.sh)) - **465/smtps** — implicit TLS, SASL required; primary listener. - **587/submission** — only when `SUBMISSION_ENABLE=true`; STARTTLS with `smtpd_tls_security_level=encrypt`. - **25/smtp inet** — when `INBOUND_RELAY_ENABLE=true` and/or `DMARC_REPORTS_ENABLE=true`. No SASL, no OpenDKIM, no journal-milter — `smtpd_milters` is always overridden on this service, because main.cf's submission chain would otherwise let a signing outage defer relayed mail and would file inbound mail into the outbound send log under the sender's domain. With the inbound relay on the chain is the optional antispam milter, then the panel's **inbound journal milter** (unix socket `INBOUND_JOURNAL_MILTER_SOCKET`, default `/run/selfpost/inbound-journal.sock`, `default_action=accept`); with DMARC ingest alone it is empty. Postfix hands a message to its milters one after another at end-of-message, each seeing what the one before added, so the journal milter sees only mail the filter let through and reads the filter's verdict headers (`X-Rspamd-Action`, `X-Spamd-Result`) off it. It writes one `inbound_spam_log` row per recipient at an inbound domain - a DMARC report address is not one - and never refuses anything ([internal/milter/inbound.go](../internal/milter/inbound.go)). The headers are read only while a filter is attached, and only next to the filter's signature: a third header, `X-SelfPost-Filter`, the HMAC-SHA256 of the message's queue id under the key in `/data/antispam/filter.key` (`inbound.FilterKey` makes it on first start; rspamd reads it through the mounted directory and signs in a custom routine of `milter_headers.conf`). With the filter down - it is attached fail-open - a sender's own verdict headers carry no signature and the row is that of mail nobody judged. Refused mail never reaches this milter. Refusals come from `mail.log`: this listener logs as `postfix/inbound` (`syslog_name`), and the log tailer turns a `reject:` line of `postfix/inbound/smtpd` (Postfix's own policy) and a `milter-reject` / `milter-hold` / `milter-discard` line of `cleanup` (the filter's verdict, counted only for a queue id that listener opened) into a row, dated by the line, when the recipient is at an inbound domain ([internal/logtail/inbound.go](../internal/logtail/inbound.go)). A line names one recipient, so a message refused at end-of-message is one row whatever its recipient count. The tailer saves its place every five seconds, so after an unclean stop it reads some lines again; the lines it reads on start are written only if the same row is not there already. The journal is pruned every six hours: rows older than the `inbound_spam_log_retention_days` setting, then down to 10 000. **Filter lists** (`inbound_spam_list`): allow and deny entries for the external filter - sender address, sender domain, client address or network - validated and normalised by `internal/inbound` (`lists.go`) and written as six map files under `/data/antispam/` (`ANTISPAM_DIR`), each atomically, on every change and on every start with the relay on. SelfPost never consults them. The sidecar mounts the directory and rspamd's multimap rules ([deploy/antispam/rspamd/multimap.conf](../deploy/antispam/rspamd/multimap.conf)) read them: a deny entry is a prefilter `reject`, an allow entry a symbol weighing -100. Allow is deliberately not a prefilter `accept`: with two prefilters rspamd lets the one that ran first decide, and deny then won or lost by rule order (seen on rspamd 3.11). A refusal by the deny list overrides any score, so deny wins whichever entries match. How often an entry decided a message is not stored: `Lists.Usage` reads it off the journal on each view of Filter lists - the rows a list decided (the deny rule's reply, the allow rule's symbol, which the journal milter keeps ahead of its six-symbol cap) matched against the entries as rspamd matches them. When that stops being good enough is written in [plans/inbound-antispam-panel.md](plans/inbound-antispam-panel.md) § Filter list usage. **Inbound relay** (`INBOUND_RELAY_ENABLE=true`): accepts only `relay_domains` + `relay_recipient_maps` (`reject_unauth_destination`, `reject_unlisted_recipient`). Maps under `/data/postfix/` (`relay_domains`, `transport`, `relay_recipients`, `tls_policy`), written atomically by [internal/postfix/inbound.go](../internal/postfix/inbound.go). Domains with an empty upstream host are omitted from the maps. Optional `INBOUND_ANTISPAM_MILTER` only when inbound relay is enabled (not on DMARC-only port 25); default `milter_default_action` is fail-open (`accept`). **DMARC ingest** (`DMARC_REPORTS_ENABLE=true`): allow-listed report addresses via `check_recipient_access` on `postfix/dmarc_recipients`; messages pipe to `dmarc-ingest` (symlink to `panel`). When both flags are on, one listener serves both paths; per-IP rate uses `INBOUND_RATE_LIMIT_MESSAGES_PER_IP` and message size uses the greater of `INBOUND_MESSAGE_SIZE_LIMIT` and `DMARC_MESSAGE_SIZE_LIMIT`. - **No open relay** — `permit_sasl_authenticated`, `reject_unauth_destination`; `smtpd_sender_login_maps` + `reject_sender_login_mismatch`. That holds the envelope sender; the `From:` header is held to the same table by the panel's sender check (§ Panel roles). - **Level-1 rate limit** — `smtpd_client_message_rate_limit` / `anvil_rate_time_unit` from `RATE_LIMIT_*` env vars; independent of milter. - **Chroot disabled** for all services (DNS/TLS inside container). - **TLS certs** — read-only mount at `TLS_CERT_FILE` / `TLS_KEY_FILE`, copied at start into `/etc/postfix/tls-internal` (Postfix wants the key root-owned and `0600`, which the mount cannot give), and Postfix reads the copy. `cert-reload` compares the mount with the copy daily; when they differ and the certificate and key belong together it renews the copy (each file renamed into place) and reloads Postfix. ### OpenDKIM Per-domain keys under `/data/opendkim/keys`; `KeyTable` / `SigningTable` maintained by the panel. Socket `/run/opendkim/opendkim.sock`. The server's own name (`SELFPOST_HOSTNAME`) has a key and a line in both tables too, without being a row of the registry (`domain.Service.SetServerName`): the panel's notifications are sent as `postmaster@` that name and signed with it. No application can use it as an envelope sender - the sender map is built from the registry - but the `From:` header is not held against the login, so an application can still have a message signed as the server ([security.md](security.md) § Accepted risks). A sending domain of the same name shares its key and selector with the server; deleting that domain leaves the server signed under the default selector, which is the same key only if that was the domain's. At its start the notifier makes sure the tables have the line and name the selector the panel shows (`EnsureServerSigned`): tables are otherwise rebuilt only when a domain is added or deleted. ### Panel binary ([cmd/panel](../cmd/panel)) One process, five roles (six with the inbound relay on): 1. **HTTP server** — `:8080` (`PANEL_HTTP_ADDR`); HTTPS terminated by reverse proxy only. On start it runs `postconf -h` once for the deferred-mail retry parameters (`queue_run_delay`, `minimal_backoff_time`, `maximal_backoff_time`, `maximal_queue_lifetime`, `bounce_queue_lifetime`, `delay_warning_time`) and caches the snapshot on the handlers config. The Outbound queue box and a delivery's `deferred` / `bounced` history print those numbers; they never call `postconf` per request. If `postconf` is missing, the panel logs a warning and uses Postfix 3.x compiled-in defaults (`300s` / `4000s` / `5d` / `0`) with a muted note on the box. 2. **sender check** — unix socket `SENDER_CHECK_SOCKET`, first in the chain and **strict** (`default_action=tempfail`): § Sender check below. 3. **journal-milter** — unix socket `JOURNAL_MILTER_SOCKET`; records From/To/ Subject/SASL user at DATA; enforces level-2 rate limits; **fail-open** (`default_action=accept`) so milter failure does not stop mail. Domain ceilings apply to every client IP; an application ceiling **overrides** the domain one for that login, higher or lower (guide § Rate limiting). Since 1.9.0 the client IP allow-list is a separate, restrictive control on the application (`applications.auth_ip_restrict`), checked here at `MAIL FROM` and fail-open like the rest of this milter — not an authentication boundary. The level-2 count is the stored send-log rows plus the messages this process has admitted but not yet written (`internal/milter/inflight.go`), so concurrent sessions cannot each spend the same last slot; a reservation is released at end-of-message, on ABORT, or after a 10-minute TTL. 4. **log-tailer** — follows `MAIL_LOG`, updates send-log delivery status by queue-id. Send-log `queued → sent` transitions depend on this goroutine alone (`UpdateStatus` is only called from [internal/logtail](../internal/logtail/logtail.go)). It also keeps the lines Postfix writes under a queue id, for the message's page (§ Log tailer, *Stored lines*), and counts the mail Postfix deferred because the sender check did not answer. 5. **rate-limit recalc** — every six hours (and on demand from Domain settings and the application form), recomputes level-2 **Auto** rate limits from send-log statistics. 6. **notifier** — writes to the panel's users when something needs them (§ Notifications below). Milter chain in Postfix: the sender check (tempfail), OpenDKIM (tempfail), then the journal (accept on failure). Mail submitted locally with `sendmail(1)` — the notifier's — passes OpenDKIM alone (`non_smtpd_milters`): it is signed, it is the panel's own and so needs no check of whose it is, and it is not an application's mail to be journalled or limited. Bounces Postfix generates pass no milter. ### Sender check [internal/milter/sender.go](../internal/milter/sender.go) holds a message's `From:` header against the application that sends it ([plans/from-header-binding.md](plans/from-header-binding.md)). Postfix holds the envelope sender against the SASL login, and OpenDKIM chooses the signing key by the header; until `2.4.0` nothing connected the two, and an application could have mail signed as another domain of the server or as the server itself. - **The rule is the envelope's.** The check reads the file Postfix reads for `smtpd_sender_login_maps` and makes the same lookup: the address itself is bound to the login, or `@domain` is. So an application set to any address of its domain may write any address of that domain, and one with a list only the listed ones; everything else is refused - another registered domain, the server's name, a domain the server knows nothing of. One table for both senders and one lookup: the address, the address without its `+extension` (`recipient_delimiter`, which `build/postfix-config.sh` sets for that reason), then the domain. Postfix compares logins without regard to case and the check as written, which is how the panel writes them. - **How the header is read.** Exactly one `From:` field (none, or more than one, is refused), parsed as an address list that must hold exactly one address. Then a cruder second look at the raw value: OpenDKIM finds the address with a parser of its own, and wherever the two could read a line differently - a display name, a comment, a quoted string - the other address would have to be spelled out in it. So no `@name` may appear there that is another name the server signs for; those names are read from OpenDKIM's own `SigningTable`; the display name is looked at once more as decoded, since a mail program shows `=?utf-8?q?director=40example.org?=` as an address though no `@` is written in it. And, needing no knowledge of that parser: outside quoted strings the line has exactly one `@`. A folded line is unfolded first, and a display name in a charset Go does not convert is passed through unread - only the address is judged. - **The verdict** comes at the end of the headers: `550 5.7.1`, permanent, naming the line, as a reply code of the milter's own. What the client wrote is reduced to printable ASCII and cut before it is said back, logged or stored. A message with no SASL login is refused as well; the submission ports do not let one this far. - **Strict, and therefore small.** Postfix attaches the socket with `default_action=tempfail`: while the panel does not answer on it within the milter timeouts, mail is deferred, not accepted unchecked. The role therefore reads no database. Its two inputs are files the panel rewrites by rename and the mail path itself goes by; they are read again when their time or size has changed, and a file that cannot be read defers mail - a table from before would let a deleted application go on sending. The send-log row of a refusal (`status=rejected`, `reason=from`, `header_from`) is handed to another goroutine through a buffer and is not waited for. It is one row for the message, naming the first recipient and how many more there were: a row per recipient would let one refused message write a thousand. - **What it costs is counted.** The role marks itself alive every two seconds (the socket's path with `.alive`, under `/run`, so a new container starts without one). At start the time since the mark is the check's downtime - counted from Postfix's own start if that was later, and not at all when supervisord says Postfix is not running, since mail was then stopped for another reason. Supervisord is asked for half a minute; when it does not answer, the time since the mark is taken as an upper bound and the day is marked estimated (`sender_check_days.estimated`), which the card shows as ≈. The log tailer counts the mail: Postfix writes a warning naming the socket and then, from the same smtpd process, `milter-reject: … 451 4.7.1 Service unavailable` from the connection on (the client is told `454 4.3.0` at `AUTH` and leaves), so what is counted is a connection turned away, once - a client brings a message when it connects. The reject line does not name the milter and the journal milter's level-2 refusal reads the same, hence the warning. A panel that is running and stops answering in the middle of a message (its socket accepts, nothing replies) is met by cleanup, not smtpd: past `DATA` it is cleanup that talks to the milters, and after the content timeout it writes `warning: milter unix:: can't read SMFIC_HEADER reply packet header` and refuses the message with `4.7.1` (seen on the e2e stand with the panel frozen). One cleanup process serves message after message, so each such warning is counted as a message. The two are counted apart (the owner, 2026-10-14): connections turned away in `deferred`, messages deferred mid-message in `messages` of `sender_check_days`, a row per UTC day kept a year, and Server › Health shows today, 30 days and 12 months in a column each. The counts also see a panel that is running and not answering, which the mark does not. Downtime is cut at the UTC midnights it crossed. A log read again after an unclean stop does not count again: the count and the place in the log it reached (the file's fingerprint and the offset past the line) are saved in one transaction, and while the tailer catches up a warning at or before that place in the same file is passed over - a second warning of the same second is not lost; in another file the time decides. - **Why it is not a process of its own.** Decided with the owner: measure first. The role has a socket of its own so that the journal stays fail-open, and moving it out later changes where it is started and nothing else. ### Notifications [internal/notify](../internal/notify/notify.go) reports four things to the users of the panel: a sending domain's DNS record that stopped matching, a level-2 rate limit that refused mail, the TLS certificate about to expire, a message more than three days in the queue. The rules of when a message goes out are the owner's and are written in [plans/panel-notifications.md](plans/panel-notifications.md) § When a message is sent; the operator's view is guide § Notifications. - **Readings.** Once an hour, starting three minutes after the process does: every sending domain's records through a DNS checker of the notifier's own, forced past the cache (`cmd/panel/notifier.go` builds the same query a domain's page does); the certificate file; `postqueue -j` for the arrival time of each queued message. Once a minute: the send log's `rejected` rows of the last day, through a partial index of their own. - **A refused row is not believed whole.** Its domain is the one in `MAIL FROM`, which the client typed and Postfix has not yet held against the login when the milter refuses; the same status is written when an application is refused for its client address or its `From:` line. Since `2.4.0` a row says which (`send_log.reason`) and only a limit's rows are taken; a row from before has no reason. So refusals are counted by the domain the login is registered under, only for logins that exist, and only while a level-2 limit is set on the login or its domain (`Store.LimitRefusalsSince`). One subject per sending domain, whichever of its applications are refused. - **State** is a row per subject in `notify_state` (`dns:`, `cert`, `queue`, `limit:`), which is what makes each thing be said once and a restart say nothing again. A DNS subject goes *pending* on the first bad check and *open*, with a message, on a later one at least 45 minutes after it - a count alone would let the last check before a restart and the first after it stand for "an hour apart"; back the same way. The certificate's row remembers which threshold was said for which expiry. A deleted domain's row goes with it. - **Said means sent.** A problem that arises while no user can be told, or while Postfix does not take the message, stays unsaid and is told at a later check. Its ending, with nobody left to tell, is simply over. If one recipient's copy fails and another's goes out, the subject is said. - **The notifier's own mail is not a subject.** A notification waiting for a mailbox that does not answer, or its bounce coming back to `postmaster@` - which receives no mail - is left out of the queue check. - **A ceiling** of 60 messages an hour stands over all of it. - **No information is not a finding.** A lookup that did not answer, a queue that cannot be listed, a certificate that cannot be read move no state. - **Recipients** come from the store at the moment of sending: global users for everything, domain users for subjects that are a domain's; only users with an e-mail and their own switch on. One message per recipient, so users do not see each other's addresses. - **The instance switch** (settings key `notifications`, on unless turned off) stops the checks themselves, not only the mail: no state moves while it is off. - **Sending** is `sendmail -i -f postmaster@ --
` as the unprivileged panel user, which is a member of `postdrop` and so can write the maildrop directory without the setgid bit `no-new-privileges` forbids. A message that could not be handed over leaves the state where it was and is tried at the next check. The text names the page to open and has no link: the panel does not know its own address. ### Log tailer and `mail.log` rotation `mail.log` lives at `/data/log/mail.log` — inside the persistent bind mount, so the delivery lines that resolve a `queued` send-log row are not lost when the container is recreated. `postlogd` writes it as user `postfix`; the panel reads it through the shared `selfpost` group (directory `2750 postfix:selfpost`, file `0640`, both normalised on every start by [build/entrypoint.sh](../build/entrypoint.sh)). The path is one default in two places, `maillog_file` in [build/postfix-config.sh](../build/postfix-config.sh) and `MAIL_LOG` in [cmd/panel/main.go](../cmd/panel/main.go). Backups exclude `log/`: it is diagnostic output, not state to restore. Rotation uses rename + `postfix reload` ([build/logrotate-mail.conf](../build/logrotate-mail.conf)), not `copytruncate` — the latter can drop `status=sent` lines and leave send-log rows stuck at `queued`. After rename, logrotate runs `create 0640 postfix selfpost` (postlogd recreates the file lazily on first write as mode `0600`, which the unprivileged panel user cannot read). `follow()` drains the old inode once more before switching descriptors; the panel treats a missing log file as an empty tail, not an error. That drain does not close the gap entirely: postlogd keeps writing to the renamed file until the reload reaches it, seconds later, and a line written there after the drain is not read. Keeping the old file open until it has stopped growing is the fix; it waits for the measurement below to say how much it matters. **Delivery lines are read at the head of the line.** The pattern that turns a line into a row's status (`deliveryRe`) is matched after the timestamp, the host name and `program[pid]:`, at the queue id - like the stored lines' and the inbound journal's. The tail of a line is the far end's words (a reply, a HELO name), and a pattern that could start there let a stranger's HELO name file a delivery (audit 2026-10-12, M12). **Read offset is persisted** (`logtail_state` table, in the baseline schema): the tailer stores its position plus a fingerprint of the log - the hash of its first 512 bytes and the file's device and inode, so two logs that begin alike are two files and a renamed file keeps its fingerprint - and on start resumes from it, parsing the tail written while the panel was down. If the fingerprint no longer matches, the log was rotated in the meantime: the tailer first opens `mail.log.1` (left plain by `delaycompress`), and if that is the file the saved place belongs to it reads it from there to its end, then the new file from its start - what Postfix wrote between the stop and the rotation is not lost. A `mail.log.1` that is another file (rotated twice) is logged with the time from which nothing was read. Re-parsing lines is harmless because `UpdateStatus` writes the same status onto the same row. Only a first-ever start, with nothing stored, begins at end-of-file, so installing the panel does not replay a pre-existing log. **A refusal's line, and a refusal the panel did not write.** A message refused for its `From:` line has a queue id (cleanup gives one before the headers are judged); the sender check writes it on the row, and the message's page shows the lines stored under it - only those of the minutes around the refusal, since Postfix frees the id at once. A message the journal milter refused at `MAIL FROM` has none: the tailer reads smtpd's `NOQUEUE: milter-reject: MAIL from [
]: …; from=<…>` - it names no login - and puts it on the newest refused row of that client address and sender within two minutes (`send_log.refusal_line`; the milter writes `send_log.client_ip`), retrying for 30 s when the row is not written yet; a line already on a row goes on no other. The sender check writes its row through a buffer after answering 550; a panel stopped in between wrote none. The tailer remembers each queue id's login from smtpd's `client=` line, and 30 s after cleanup's `milter-reject: END-OF-MESSAGE … 5.7.1 From:` line it writes the row from the line if no refused row of that id exists (`send_log.recovered`, shown on the page). It reads from its saved offset, so a kill is covered as well as a clean stop. **Queue reconcile** is the backstop for what the log cannot explain at all: a row still `queued` more than two minutes after it was accepted, whose queue id `postqueue -p` no longer lists, is marked `bounced` with the reason `reconcile` (swept every five minutes, [internal/logtail](../internal/logtail/logtail.go), [postfix.QueueIDs](../internal/postfix/queue.go)). Postfix having dropped the message means nothing more will ever be reported about it, so the row can only be closed on an assumption, and it is closed as a failure because a delivery the panel cannot evidence must not be shown as one. What keeps the sweep from guessing where it need not: it starts only after the tailer has read to end-of-file once (on a restart the log itself holds the answer); the two-minute grace covers messages merely in flight; a `postqueue` that cannot be read, or whose listing is over the 4 MiB cap, leaves every row untouched rather than closing them all or those past the cut; and the close, by the row's id, is conditional on the row still being `queued`, so a status the log wrote in the meantime stands. A delivery line updates only the newest row of its queue id and recipient: Postfix reuses a short queue id once the message has left the queue, and an old row must not be rewritten. `UpdateStatus`'s transaction starts with a write, so a commit by the `dmarc-ingest` process in between cannot fail it with `database is locked`. Now that the log survives the container, reaching this path should mean the lines are gone for good — rotated past fourteen files while the panel was down, or deleted — or that the tailer was not reading. Whether the sweep earns its place is measured, not argued (audit 2026-10-12, H3). `delivery_log_days` (migration `0007`) keeps, per UTC day for a year: `reconcile_closed`, the rows the sweep closed; `reconcile_missed`, those of them whose delivery line was in `mail.log` or `mail.log.1` when the sweep closed them - the tailer did not read a line that was there, which is what a lost line looks like (it is never read late); `reconcile_contradicted`, those a delivery line read later gave a status - the log's status replaces the sweep's and the row's reason becomes `reconcile-contradicted`, so a row counts once and can be found; `tailer_blind_seconds`, the time `follow()` could not read `mail.log` (the file gone, reopening it failing; stretches under three seconds - a rotation's tick between rename and create, which loses nothing - are not counted); and `queued_over_lifetime`, the rows still `queued` past the queue lifetime plus a maximal backoff, as the day's last sweep saw them - rows stuck although the sweep runs (mail on hold, or a queue that could not be listed). What removing the sweep would leave queued is closed − missed − contradicted: the rows nothing in the log ever resolved. No page shows the table: it is read off a copy of the database. The rule, written before the data: `reconcile_closed = 0` for a month → remove the sweep; closed, but neither missed nor contradicted → keep it; missed or contradicted → the tailer loses or delays lines - fix the tailer (the two deferred steps: the rotation window, pausing the sweep while blind) and keep the sweep; if what remains after that is never resolved by the log, mark such rows `lost` or remove the sweep. Decided after thirty days, or sooner if `reconcile_missed` or `reconcile_contradicted` is ever non-zero. **Stored lines** (`send_log_lines`, since 2.2.0; [internal/logtail/lines.go](../internal/logtail/lines.go)). A send-log row lives for the retention window, ninety days by default, and `mail.log` for a day before it is rotated; a message's page (`/outbound/log/{id}`) shows what Postfix wrote about it, so the tailer keeps those lines in the database as it reads them: - **What is kept:** a line headed by a queue id — after the timestamp, the host name and `program[pid]:` — with that id, the line's own time and the line as written, cut to 4096 bytes. The id is read from the head of the line only: the tail is the far end's text. `NOQUEUE` and `warning` stand in the same place and are not ids; the pattern accepts Postfix's short form (capital hexadecimal) and its long form (no vowels). - **Port-25 mail is left out.** A queue id opened by `postfix/inbound/smtpd` is a stranger's message and never gets a send-log row; none of its lines are stored, up to qmgr's `removed` or until another listener opens a message under the id (4096 such ids are remembered). What a stranger can make Postfix log is therefore not written to the database at his rate. - **No check for a row at write time.** A message's first lines (`client=`, `message-id=`) are logged before the journal milter writes its row at end-of-message, so the tailer cannot ask whether the id is in the send log. Lines that never get a row — bounces Postfix generates, port-25 mail met in mid-flight after a restart — are kept for a while. - **Written per pass, not per line:** the lines of one pass over the log go in one transaction (`InsertSendLogLines`), 500 at most, before the read offset is saved. A commit per line would be a disk sync per line on the connection the journal milter shares. - **Retention is one rule:** a line older than an hour whose queue id no send-log row carries is deleted (`PruneSendLogLines`), at start and after every six-hourly send-log sweep. That removes the lines of rows the sweep deleted and the lines that never had a row. The send log is asked once per queue id; the statement still walks the table, about a second per million lines, with the connection taken meanwhile. - **Which lines are a row's.** Postfix reuses a queue id once the message has left the queue, and a row outlives that by months. `SendLogLines` gives a row the lines of its queue id dated from one hour before the row was written to the queue lifetime (the larger of `maximal_queue_lifetime` and `bounce_queue_lifetime`) plus a day after it. That limits a chance reuse of the id to those days; it does not exclude it. A message with several recipients is several rows under one id, and each row's page shows all of the message's lines. - **Read twice, stored once.** The read offset is saved every few seconds, so after an unclean stop some lines are read again; while the tailer is catching up at start it writes a line only if the same line is not there. - **Not stored:** what the tailer never read — the tail of a file rotated away while the panel was down. - **Size:** six to ten lines of about 200 bytes per message — more than the row itself. The table is in the database and therefore in a full backup, which `log/` is not ([security.md](security.md) § Backup and domain export). **Two one-shot reads** sit beside the follow loop and are unrelated to it, both serving panel pages on request: `TailLines` (the last *n* lines, for `/server/log`) and `QueueLines` (the lines carrying one queue-id). `QueueLines` is what a message's page falls back to when no lines are stored for its row — a message accepted before 2.2.0. It scans a bounded tail of the current file — finding a message's lines means reading rather than seeking — and matches the id anchored on the character before it, since queue ids are hexadecimal runs and a shorter one is regularly the tail of a longer one. For such a message an empty result is the expected end state once the log has rotated, and the page reports it as such, not as a failure. --- ## Panel HTTP surface Canonical routes: [internal/web/web.go](../internal/web/web.go). The router is two flat `http.ServeMux`es built by `muxes()` — one pattern per route, method included (`GET /outbound/log/{id}`), nothing mounted as a sub-router, and building them only takes method values, so the route guard test can run it on a bare `Server` and ask which pattern answers a path: - **public** — `/healthz` (liveness), `/license` (the embedded `LICENSE`), `/static/`, `/setup/`, `/login`, `/logout`; - **authenticated** — everything else, behind `RequireAuth`: a request without a live session is redirected to `/login`. Both sit inside `secure()`, which sets the security headers and runs the origin check on every state-changing request (see below). ### Routes A path reads like the menu: `//`, an entity under its list, an action under the thing it changes. Every state change is a `POST`; a `GET` only reads (the delete confirmation at `GET …/delete` renders a form). | Group | Routes | |---|---| | Home | `GET /` → `/overview`, or for a user with the *domain* role to the first list they reach (`/outbound/domains`, or `/inbound/domains` when that is all) | | Overview | `GET /overview`, `GET /overview/fragment` | | Outbound › Domains | `GET, POST /outbound/domains` · `/outbound/domains/{id}` · `POST …/dns-recheck` · `GET, POST …/delete` · `GET …/settings` · `POST …/settings/reports`, `…/ratelimit`, `…/ratelimit/recalc`, `…/export` | | Outbound › Applications | `GET, POST /outbound/domains/{id}/applications/new` · `GET, POST …/applications/{aid}` · `POST …/applications/{aid}/password`, `…/{aid}/ratelimit/recalc`, `…/{aid}/delete` | | Outbound › Log, Queue | `GET /outbound/log` · `/outbound/log/fragment` · `/outbound/log/{id}` · `GET /outbound/queue` · `/outbound/queue/fragment` | | Outbound › DMARC reports | `GET /outbound/dmarc` · `…/dmarc/domains/{id}` · `…/dmarc/reports/{id}` — registered only with `DMARC_REPORTS_ENABLE=true` | | Inbound › Domains | `GET, POST /inbound/domains` · `/inbound/domains/{id}` · `POST …/dns-recheck`, `…/upstream`, `…/recipients` · `GET, POST …/delete` — registered only with `INBOUND_RELAY_ENABLE=true` | | Server | `GET /server/health` (+ `/fragment`) · `POST /server/health/recheck`, `/server/health/reload` · `GET /server/log` (+ `/fragment`) · `GET, POST /server/backup` · `POST /server/backup/import` · `/server/users`, `…/new`, `…/{uid}`, `…/{uid}/delete` · `GET, POST /server/settings` · `GET /server/components` | | Account | `GET /account` · `POST /account/profile`, `/account/password` | | User menu | `GET /help` · `POST /logout` (public mux) | An application is addressed under its domain, and an application id that does not belong to the domain in the path is a 404. The application form is one `POST`: sender, client IPs and rate limit are validated and saved together or not at all, also at creation. The page that shows a new password is the response to the `POST` that created or regenerated it; it has no `GET` path and carries `Cache-Control: no-store`. The pre-2.0 paths are not served and not redirected (`/status`, `/domains/…`, `/inbound/{id}/…`, `/deliveries`, `/mail-queue`, `/system-log`, `/backup`, `/users/…`, `/settings`, `/reload`, `/dmarc/…`, and `/account` as a redirect): each answers 404 once signed in. `TestRoutesFollowNavigation` ([internal/web/guard_routes_test.go](../internal/web/guard_routes_test.go)) holds the table: every path is registered with its methods, every menu entry points at a registered path, every old path answers 404, and no template contains an old path. ### Who may open what Authorization is by tree where the tree is the boundary, and by object where it is not. - **`/server/` is guarded once.** Routes under it are registered through `server()` in `web.go`, which wraps the handler in `globalOnly` (404 for anyone but the global role — the answer a missing page gets, so the panel does not confirm the page exists) and panics at start-up for a pattern outside `/server/`. The handlers also call `requireGlobal()` themselves. - **Overview and the Outbound queue** are global-only but not under `/server/`, so their handlers call `requireGlobal()`. - **Outbound is checked per domain.** `lookupDomain` resolves `{id}` and answers 404 for a missing domain and for one the principal may not reach alike ([handlers/handlers_domains.go](../internal/web/handlers/handlers_domains.go)); application routes go on to check the application belongs to that domain. Lists and the log are filtered by assignment in the query (`ListDomainsForUser`), not in the template. Adding and deleting a domain is `requireGlobal()`. - **Inbound is checked per inbound domain**, separately from outbound ([handlers/handlers_inbound.go](../internal/web/handlers/handlers_inbound.go)): `requireInbound` (the list), `requireInboundDomain` (one domain — page, upstream, recipients, DNS re-check) and `requireInboundGlobal` (add, delete). All three answer 404 alike. - **The DMARC hub** is open to anyone with outbound reach for the reports of their own domains; the *Ingest* box (server-wide statistics) is rendered for the global role only. - **Account and Help** are open to every signed-in user. - The principal ([web/auth/principal.go](../internal/web/auth/principal.go)) carries the role, the assigned outbound and inbound domain ids and the two *All* flags; `CanAccessDomain` and `CanAccessInboundDomain` are the only predicates. The reach, not the role, decides which menu groups exist. Routes marked global are tested per path for a user with the *domain* role, including another tenant's domain id ([handlers/handlers_delegation_test.go](../internal/web/handlers/handlers_delegation_test.go), [authz_test.go](../internal/web/handlers/authz_test.go)). ### Polled fragments A polled page answers its refresh at `/fragment` (`/overview/fragment`, `/outbound/log/fragment`, `/server/health/fragment`, …). The fragment is the same `{{define}}` block that renders the page's own box, so the first paint and a refresh cannot differ. HTMX refreshes them every 5 s while the operator is active on the page, every 30 s when the tab is visible but idle, and not at all when it is hidden — scheduled in `panel.js` via `data-poll`, not `hx-trigger="every …"`, because a trigger filter is evaluated with `new Function`, which the CSP forbids. Polling does not extend the session idle timeout (only non-`HX-Request` `GET`s and mutating requests count as activity). The Outbound queue's retry-policy box is outside the polled region: it is the start-up `postconf -h` snapshot (see [Panel binary](#panel-binary-cmdpanel)), not a live re-read. ### View engine and component kit Pages are rendered by [internal/web/view](../internal/web/view/view.go). There is one layout and one way to render. - **One layout.** Every page is parsed with `layout.html` (the shell: brick navbar with the wordmark, the perforated edge, the strip of sibling pages, user menu, footer) and `components.html` (the partials), then its own file. `layout_signed_out` is the same document without navigation, used for sign-in and setup; `Engine.Render` picks it when the page data names no user. The page's file defines `content`; `pageFiles` in `view.go` lists the files of each page, and fragments are listed in `fragmentFiles` and rendered by `RenderFragment` with no layout. - **Typed page data.** A page's data is a struct that embeds `Meta` (title, user, the viewer's reach, the menu group and page it sits in) and carries the page's boxes and rows as fields, built by a constructor in the `view` package (`NewOutDomain`, `NewHelp`, …) that the handler fills. The shell is derived from `Meta` and the feature flags (`SetInboundEnabled`, `SetDMARCEnabled`) in one place, `Engine.shell`, so which entries the menu has is decided once. No template reads a `map[string]any`; the one map left in the handlers is the health sampler's, which is turned into a typed `Health` before rendering. - **The component kit.** [components.html](../internal/web/view/templates/components.html) holds one `{{define}}` per partial (page head, postmark, box and its head and foot, flash, health card, DNS record, facts, side menu, timeline, log pane, empty state, confirm list, help topic, form rows) and is the only file that writes their `sp-` markup. [components.go](../internal/web/view/components.go) is the typed input of each (`Head`, `Box`, `Record`, `Fact`, `SideMenu`, …), so a page passes a struct, never a `dict`. Template functions: `status_tag` (a status → Bulma's tag classes, once), `wbr_at` (a break after the `@` in a table cell). A page that needs a component the kit lacks is a design change made in the mockups first ([plans/panel-redesign.md](plans/panel-redesign.md) § The contract). - **Help.** [help.go](../internal/web/view/help.go) is the Help page's data: topics grouped by section of the panel. A box's head links to its topic with `Box.Help` (`/help#`), and the page lists only the sections the viewer's menu shows. - **Assets.** [static/](../internal/web/view/static) is embedded (`//go:embed`): vendored `bulma.min.css` and a Tabler icon subset (checksummed), `panel.css` (the mockups' stylesheet, rule for rule), `htmx.min.js`, `panel.js`, the IBM Plex subset and the logo files. Served with a content-derived `ETag`, and `/static/` does not list its files. The kit page `GET /server/components` renders every partial in every state from fixtures (global role only). **Guard tests** hold the design in place; they run in `go test ./...` and CI. They read their rules from the accepted mockups ([docs/assets/panel-redesign/panel/](assets/panel-redesign/index.html)) rather than keep a copy: `TestPanelClassVocabulary` (only the Bulma subset and the `sp-` classes of `panel.css`; every `sp-` class defined, used and shown on the components page), `TestPanelCSSIsTheMockupStylesheet`, `TestPanelCSSContract` (no `!important`, no id or page selector, type floor, widths), `TestTemplatesUseComponents` (no hand-written component markup in a page), `TestPanelOutlinesMatchMockups` and `TestPanelPageStructure` (each rendered page has the component skeleton of its mockup), `TestRedesignedPagesLoadOnlyTheKit`, `TestVendoredAssetsArePinned`, `TestStaticHoldsOnlyTheKnownAssets`, `TestRoutesFollowNavigation`, `TestNoTemplateUsesInlineScriptOrStyle`. The ratchet list `internal/web/view/legacy_pages.txt` is empty and can only shrink; the CI step [design-first](../.github/scripts/design-first.sh) rejects a commit that changes a guard together with the code it judges, and a template that is a byte-for-byte return of one from history. The design contract and the process are in [plans/panel-redesign.md](plans/panel-redesign.md) § Enforcement. ### Security headers, CSP and scripts `secure()` ([internal/web/security.go](../internal/web/security.go)) puts these on every response: `Content-Security-Policy: default-src 'self'; object-src 'none'; base-uri 'none'; form-action 'self'; frame-ancestors 'none'`, `X-Content-Type-Options: nosniff`, `X-Frame-Options: DENY`, `Referrer-Policy: no-referrer`, and `Strict-Transport-Security` when the cookie is `Secure`. Because the policy has no exemptions, no template carries an inline `