# SelfPost — architecture (as-built) **Source of truth:** the code tree, not historical specs. Synchronise this file when env keys, routes, or mail-path behaviour change. Verification method: [development.md](development.md) § «Verifying docs against code». User install/operations: [README.md](../README.md), [guide.md](guide.md). Product boundaries: [product.md](product.md). --- ## Image and processes Single Debian slim image. `entrypoint.sh` (root) fixes `/data` ownership and milter socket directories, **requires** `SELFPOST_HOSTNAME` (FQDN with at least one dot; no scheme, port, or spaces — invalid or empty value → `exit 1` before Postfix config or supervisord), then execs `supervisord` as PID 1. The hostname is not a tunable default: it must match PTR/rDNS, TLS CN/SAN, and SASL realm together. Soft fallbacks (`localhost` in the panel vs container ID in Postfix) split realms and break SMTP AUTH with a silent `535` in clients while the panel looks healthy. Shipped `docker-compose.yml` already requires the variable; the entrypoint gate catches `docker run`, custom compose, and k8s. Managed programs ([build/supervisord.conf](../build/supervisord.conf)): | Program | User | Priority | Role | |---|---|---|---| | `opendkim` | root → `opendkim` (`UserID` in opendkim.conf) | 100 | DKIM signing milter | | `panel` | panel | 200 | HTTP UI + journal-milter + log-tailer + rate-limit recalc; with the inbound relay on also the inbound journal milter | | `postfix` | root (wrapper) | 300 | MTA — started only after both milter sockets exist | | `postfix-reload` | root | — | On-demand `postfix reload` (autostart off) | | `cert-reload` | root | 400 | Daily `postfix reload` for renewed TLS certs | | `logrotate` | root | 400 | Periodic `mail.log` rotation | Start order: OpenDKIM → panel (opens journal-milter socket) → Postfix wrapper polls unix sockets (timeout `MILTER_WAIT_TIMEOUT`, default 30s) then `postfix start-fg`. `crashexit` event listener exits the container on any managed process FATAL so Docker `restart: unless-stopped` recreates a broken instance. **Liveness:** `GET /healthz` (unauthenticated) returns 200 when opendkim, panel, and postfix are RUNNING; Docker `HEALTHCHECK` uses the same probe. --- ## Mail path ``` Client ──TLS+SASL──► Postfix (465 smtps, optional 587 submission) │ ├─► OpenDKIM milter (sign, tempfail on failure) ├─► journal-milter (send log + L2 rate limits, fail-open) └─► outbound MX delivery (port 25 client) Internet ──► Postfix smtp inet :25 (when INBOUND_RELAY_ENABLE=true │ and/or DMARC_REPORTS_ENABLE=true) │ ├─► optional antispam milter (inbound relay only) ├─► DMARC aggregate ingest (pipe → dmarc-ingest) └─► smtp:[upstream]:port (inbound relay only; transport_maps; no local delivery) ``` The smtp/inet listener is **absent** when both flags are off (`postconf -MX smtp/inet` removes Debian's stock smtpd). Outbound delivery still uses the `smtp unix` client; it is not the same service. ### Postfix ([build/postfix-config.sh](../build/postfix-config.sh)) - **465/smtps** — implicit TLS, SASL required; primary listener. - **587/submission** — only when `SUBMISSION_ENABLE=true`; STARTTLS with `smtpd_tls_security_level=encrypt`. - **25/smtp inet** — when `INBOUND_RELAY_ENABLE=true` and/or `DMARC_REPORTS_ENABLE=true`. No SASL, no OpenDKIM, no journal-milter — `smtpd_milters` is always overridden on this service, because main.cf's submission chain would otherwise let a signing outage defer relayed mail and would file inbound mail into the outbound send log under the sender's domain. With the inbound relay on the chain is the optional antispam milter, then the panel's **inbound journal milter** (unix socket `INBOUND_JOURNAL_MILTER_SOCKET`, default `/run/selfpost/inbound-journal.sock`, `default_action=accept`); with DMARC ingest alone it is empty. Postfix hands a message to its milters one after another at end-of-message, each seeing what the one before added, so the journal milter sees only mail the filter let through and reads the filter's verdict headers (`X-Rspamd-Action`, `X-Spamd-Result`) off it. It writes one `inbound_spam_log` row per recipient at an inbound domain - a DMARC report address is not one - and never refuses anything ([internal/milter/inbound.go](../internal/milter/inbound.go)). The headers are read only while a filter is attached; refused mail never reaches it. Refusals come from `mail.log`: this listener logs as `postfix/inbound` (`syslog_name`), and the log tailer turns a `reject:` line of `postfix/inbound/smtpd` (Postfix's own policy) and a `milter-reject` / `milter-hold` / `milter-discard` line of `cleanup` (the filter's verdict, counted only for a queue id that listener opened) into a row, dated by the line, when the recipient is at an inbound domain ([internal/logtail/inbound.go](../internal/logtail/inbound.go)). A line names one recipient, so a message refused at end-of-message is one row whatever its recipient count, and a line read twice (the tailer saves its place once a second; an unclean stop can lose that second) is two rows. The journal is pruned every six hours: rows older than the `inbound_spam_log_retention_days` setting, then down to 10 000. **Inbound relay** (`INBOUND_RELAY_ENABLE=true`): accepts only `relay_domains` + `relay_recipient_maps` (`reject_unauth_destination`, `reject_unlisted_recipient`). Maps under `/data/postfix/` (`relay_domains`, `transport`, `relay_recipients`, `tls_policy`), written atomically by [internal/postfix/inbound.go](../internal/postfix/inbound.go). Domains with an empty upstream host are omitted from the maps. Optional `INBOUND_ANTISPAM_MILTER` only when inbound relay is enabled (not on DMARC-only port 25); default `milter_default_action` is fail-open (`accept`). **DMARC ingest** (`DMARC_REPORTS_ENABLE=true`): allow-listed report addresses via `check_recipient_access` on `postfix/dmarc_recipients`; messages pipe to `dmarc-ingest` (symlink to `panel`). When both flags are on, one listener serves both paths; per-IP rate uses `INBOUND_RATE_LIMIT_MESSAGES_PER_IP` and message size uses the greater of `INBOUND_MESSAGE_SIZE_LIMIT` and `DMARC_MESSAGE_SIZE_LIMIT`. - **No open relay** — `permit_sasl_authenticated`, `reject_unauth_destination`; `smtpd_sender_login_maps` + `reject_sender_login_mismatch`. - **Level-1 rate limit** — `smtpd_client_message_rate_limit` / `anvil_rate_time_unit` from `RATE_LIMIT_*` env vars; independent of milter. - **Chroot disabled** for all services (DNS/TLS inside container). - **TLS certs** — read-only mount at `TLS_CERT_FILE` / `TLS_KEY_FILE`; daily reload via `cert-reload`. ### OpenDKIM Per-domain keys under `/data/opendkim/keys`; `KeyTable` / `SigningTable` maintained by the panel. Socket `/run/opendkim/opendkim.sock`. ### Panel binary ([cmd/panel](../cmd/panel)) One process, four roles: 1. **HTTP server** — `:8080` (`PANEL_HTTP_ADDR`); HTTPS terminated by reverse proxy only. On start it runs `postconf -h` once for the deferred-mail retry parameters (`queue_run_delay`, `minimal_backoff_time`, `maximal_backoff_time`, `maximal_queue_lifetime`, `bounce_queue_lifetime`, `delay_warning_time`) and caches the snapshot on the handlers config. The Outbound queue box and a delivery's `deferred` / `bounced` history print those numbers; they never call `postconf` per request. If `postconf` is missing, the panel logs a warning and uses Postfix 3.x compiled-in defaults (`300s` / `4000s` / `5d` / `0`) with a muted note on the box. 2. **journal-milter** — unix socket `JOURNAL_MILTER_SOCKET`; records From/To/ Subject/SASL user at DATA; enforces level-2 rate limits; **fail-open** (`default_action=accept`) so milter failure does not stop mail. Domain ceilings apply to every client IP; an application ceiling **overrides** the domain one for that login, higher or lower (guide § Rate limiting). Since 1.9.0 the client IP allow-list is a separate, restrictive control on the application (`applications.auth_ip_restrict`), checked here at `MAIL FROM` and fail-open like the rest of this milter — not an authentication boundary. The level-2 count is the stored send-log rows plus the messages this process has admitted but not yet written (`internal/milter/inflight.go`), so concurrent sessions cannot each spend the same last slot; a reservation is released at end-of-message, on ABORT, or after a 10-minute TTL. 3. **log-tailer** — follows `MAIL_LOG`, updates send-log delivery status by queue-id. Send-log `queued → sent` transitions depend on this goroutine alone (`UpdateStatus` is only called from [internal/logtail](../internal/logtail/logtail.go)). 4. **rate-limit recalc** — every six hours (and on demand from Domain settings and the application form), recomputes level-2 **Auto** rate limits from send-log statistics. Milter chain in Postfix: OpenDKIM (tempfail) then journal (accept on failure). ### Log tailer and `mail.log` rotation `mail.log` lives at `/data/log/mail.log` — inside the persistent bind mount, so the delivery lines that resolve a `queued` send-log row are not lost when the container is recreated. `postlogd` writes it as user `postfix`; the panel reads it through the shared `selfpost` group (directory `2750 postfix:selfpost`, file `0640`, both normalised on every start by [build/entrypoint.sh](../build/entrypoint.sh)). The path is one default in two places, `maillog_file` in [build/postfix-config.sh](../build/postfix-config.sh) and `MAIL_LOG` in [cmd/panel/main.go](../cmd/panel/main.go). Backups exclude `log/`: it is diagnostic output, not state to restore. Rotation uses rename + `postfix reload` ([build/logrotate-mail.conf](../build/logrotate-mail.conf)), not `copytruncate` — the latter can drop `status=sent` lines and leave send-log rows stuck at `queued`. After rename, logrotate runs `create 0640 postfix selfpost` (postlogd recreates the file lazily on first write as mode `0600`, which the unprivileged panel user cannot read). `follow()` drains the old inode once more before switching descriptors; the panel treats a missing log file as an empty tail, not an error. **Read offset is persisted** (`logtail_state` table, in the baseline schema): the tailer stores its position plus a fingerprint of the log's first 512 bytes, and on start resumes from it, parsing the tail written while the panel was down. If the fingerprint no longer matches (rotated or recreated in the meantime) it reads the current file from the start; re-parsing lines is harmless because `UpdateStatus` writes the same status onto the same row. Only a first-ever start, with nothing stored, begins at end-of-file, so installing the panel does not replay a pre-existing log. **Queue reconcile** is the backstop for what the log cannot explain at all: a row still `queued` more than two minutes after it was accepted, whose queue id `postqueue -p` no longer lists, is marked `bounced` (swept every five minutes, [internal/logtail](../internal/logtail/logtail.go), [postfix.QueueIDs](../internal/postfix/queue.go)). Postfix having dropped the message means nothing more will ever be reported about it, so the row can only be closed on an assumption, and it is closed as a failure because a delivery the panel cannot evidence must not be shown as one. Three things keep the sweep from guessing where it need not: it starts only after the tailer has read to end-of-file once (on a restart the log itself holds the answer), the two-minute grace covers messages merely in flight, and a `postqueue` that cannot be read leaves every row untouched rather than closing them all. Now that the log survives the container, reaching this path means the lines are gone for good — rotated past fourteen files while the panel was down, or deleted. **Two one-shot reads** sit beside the follow loop and are unrelated to it, both serving panel pages on request: `TailLines` (the last *n* lines, for `/server/log`) and `QueueLines` (the lines carrying one queue-id, for `/outbound/log/{id}`). `QueueLines` scans a bounded tail of the current file — finding a message's lines means reading rather than seeking — and matches the id anchored on the character before it, since queue ids are hexadecimal runs and a shorter one is regularly the tail of a longer one. Send-log rows outlive the log (retention 90 days, rotation 14 files), so an empty result is the expected end state for an older message and the page reports it as such, not as a failure. --- ## Panel HTTP surface Canonical routes: [internal/web/web.go](../internal/web/web.go). The router is two flat `http.ServeMux`es built by `muxes()` — one pattern per route, method included (`GET /outbound/log/{id}`), nothing mounted as a sub-router, and building them only takes method values, so the route guard test can run it on a bare `Server` and ask which pattern answers a path: - **public** — `/healthz` (liveness), `/license` (the embedded `LICENSE`), `/static/`, `/setup/`, `/login`, `/logout`; - **authenticated** — everything else, behind `RequireAuth`: a request without a live session is redirected to `/login`. Both sit inside `secure()`, which sets the security headers and runs the origin check on every state-changing request (see below). ### Routes A path reads like the menu: `//`, an entity under its list, an action under the thing it changes. Every state change is a `POST`; a `GET` only reads (the delete confirmation at `GET …/delete` renders a form). | Group | Routes | |---|---| | Home | `GET /` → `/overview`, or for a user with the *domain* role to the first list they reach (`/outbound/domains`, or `/inbound/domains` when that is all) | | Overview | `GET /overview`, `GET /overview/fragment` | | Outbound › Domains | `GET, POST /outbound/domains` · `/outbound/domains/{id}` · `POST …/dns-recheck` · `GET, POST …/delete` · `GET …/settings` · `POST …/settings/reports`, `…/ratelimit`, `…/ratelimit/recalc`, `…/export` | | Outbound › Applications | `GET, POST /outbound/domains/{id}/applications/new` · `GET, POST …/applications/{aid}` · `POST …/applications/{aid}/password`, `…/{aid}/ratelimit/recalc`, `…/{aid}/delete` | | Outbound › Log, Queue | `GET /outbound/log` · `/outbound/log/fragment` · `/outbound/log/{id}` · `GET /outbound/queue` · `/outbound/queue/fragment` | | Outbound › DMARC reports | `GET /outbound/dmarc` · `…/dmarc/domains/{id}` · `…/dmarc/reports/{id}` — registered only with `DMARC_REPORTS_ENABLE=true` | | Inbound › Domains | `GET, POST /inbound/domains` · `/inbound/domains/{id}` · `POST …/dns-recheck`, `…/upstream`, `…/recipients` · `GET, POST …/delete` — registered only with `INBOUND_RELAY_ENABLE=true` | | Server | `GET /server/health` (+ `/fragment`) · `POST /server/health/recheck`, `/server/health/reload` · `GET /server/log` (+ `/fragment`) · `GET, POST /server/backup` · `POST /server/backup/import` · `/server/users`, `…/new`, `…/{uid}`, `…/{uid}/delete` · `GET, POST /server/settings` · `GET /server/components` | | Account | `GET /account` · `POST /account/profile`, `/account/password` | | User menu | `GET /help` · `POST /logout` (public mux) | An application is addressed under its domain, and an application id that does not belong to the domain in the path is a 404. The application form is one `POST`: sender, client IPs and rate limit are validated and saved together or not at all, also at creation. The page that shows a new password is the response to the `POST` that created or regenerated it; it has no `GET` path and carries `Cache-Control: no-store`. The pre-2.0 paths are not served and not redirected (`/status`, `/domains/…`, `/inbound/{id}/…`, `/deliveries`, `/mail-queue`, `/system-log`, `/backup`, `/users/…`, `/settings`, `/reload`, `/dmarc/…`, and `/account` as a redirect): each answers 404 once signed in. `TestRoutesFollowNavigation` ([internal/web/guard_routes_test.go](../internal/web/guard_routes_test.go)) holds the table: every path is registered with its methods, every menu entry points at a registered path, every old path answers 404, and no template contains an old path. ### Who may open what Authorization is by tree where the tree is the boundary, and by object where it is not. - **`/server/` is guarded once.** Routes under it are registered through `server()` in `web.go`, which wraps the handler in `globalOnly` (404 for anyone but the global role — the answer a missing page gets, so the panel does not confirm the page exists) and panics at start-up for a pattern outside `/server/`. The handlers also call `requireGlobal()` themselves. - **Overview and the Outbound queue** are global-only but not under `/server/`, so their handlers call `requireGlobal()`. - **Outbound is checked per domain.** `lookupDomain` resolves `{id}` and answers 404 for a missing domain and for one the principal may not reach alike ([handlers/handlers_domains.go](../internal/web/handlers/handlers_domains.go)); application routes go on to check the application belongs to that domain. Lists and the log are filtered by assignment in the query (`ListDomainsForUser`), not in the template. Adding and deleting a domain is `requireGlobal()`. - **Inbound is checked per inbound domain**, separately from outbound ([handlers/handlers_inbound.go](../internal/web/handlers/handlers_inbound.go)): `requireInbound` (the list), `requireInboundDomain` (one domain — page, upstream, recipients, DNS re-check) and `requireInboundGlobal` (add, delete). All three answer 404 alike. - **The DMARC hub** is open to anyone with outbound reach for the reports of their own domains; the *Ingest* box (server-wide statistics) is rendered for the global role only. - **Account and Help** are open to every signed-in user. - The principal ([web/auth/principal.go](../internal/web/auth/principal.go)) carries the role, the assigned outbound and inbound domain ids and the two *All* flags; `CanAccessDomain` and `CanAccessInboundDomain` are the only predicates. The reach, not the role, decides which menu groups exist. Routes marked global are tested per path for a user with the *domain* role, including another tenant's domain id ([handlers/handlers_delegation_test.go](../internal/web/handlers/handlers_delegation_test.go), [authz_test.go](../internal/web/handlers/authz_test.go)). ### Polled fragments A polled page answers its refresh at `/fragment` (`/overview/fragment`, `/outbound/log/fragment`, `/server/health/fragment`, …). The fragment is the same `{{define}}` block that renders the page's own box, so the first paint and a refresh cannot differ. HTMX refreshes them every 5 s while the operator is active on the page, every 30 s when the tab is visible but idle, and not at all when it is hidden — scheduled in `panel.js` via `data-poll`, not `hx-trigger="every …"`, because a trigger filter is evaluated with `new Function`, which the CSP forbids. Polling does not extend the session idle timeout (only non-`HX-Request` `GET`s and mutating requests count as activity). The Outbound queue's retry-policy box is outside the polled region: it is the start-up `postconf -h` snapshot (see [Panel binary](#panel-binary-cmdpanel)), not a live re-read. ### View engine and component kit Pages are rendered by [internal/web/view](../internal/web/view/view.go). There is one layout and one way to render. - **One layout.** Every page is parsed with `layout.html` (the shell: brick navbar with the wordmark, the perforated edge, the strip of sibling pages, user menu, footer) and `components.html` (the partials), then its own file. `layout_signed_out` is the same document without navigation, used for sign-in and setup; `Engine.Render` picks it when the page data names no user. The page's file defines `content`; `pageFiles` in `view.go` lists the files of each page, and fragments are listed in `fragmentFiles` and rendered by `RenderFragment` with no layout. - **Typed page data.** A page's data is a struct that embeds `Meta` (title, user, the viewer's reach, the menu group and page it sits in) and carries the page's boxes and rows as fields, built by a constructor in the `view` package (`NewOutDomain`, `NewHelp`, …) that the handler fills. The shell is derived from `Meta` and the feature flags (`SetInboundEnabled`, `SetDMARCEnabled`) in one place, `Engine.shell`, so which entries the menu has is decided once. No template reads a `map[string]any`; the one map left in the handlers is the health sampler's, which is turned into a typed `Health` before rendering. - **The component kit.** [components.html](../internal/web/view/templates/components.html) holds one `{{define}}` per partial (page head, postmark, box and its head and foot, flash, health card, DNS record, facts, side menu, timeline, log pane, empty state, confirm list, help topic, form rows) and is the only file that writes their `sp-` markup. [components.go](../internal/web/view/components.go) is the typed input of each (`Head`, `Box`, `Record`, `Fact`, `SideMenu`, …), so a page passes a struct, never a `dict`. Template functions: `status_tag` (a status → Bulma's tag classes, once), `wbr_at` (a break after the `@` in a table cell). A page that needs a component the kit lacks is a design change made in the mockups first ([plans/panel-redesign.md](plans/panel-redesign.md) § The contract). - **Help.** [help.go](../internal/web/view/help.go) is the Help page's data: topics grouped by section of the panel. A box's head links to its topic with `Box.Help` (`/help#`), and the page lists only the sections the viewer's menu shows. - **Assets.** [static/](../internal/web/view/static) is embedded (`//go:embed`): vendored `bulma.min.css` and a Tabler icon subset (checksummed), `panel.css` (the mockups' stylesheet, rule for rule), `htmx.min.js`, `panel.js`, the IBM Plex subset and the logo files. Served with a content-derived `ETag`, and `/static/` does not list its files. The kit page `GET /server/components` renders every partial in every state from fixtures (global role only). **Guard tests** hold the design in place; they run in `go test ./...` and CI. They read their rules from the accepted mockups ([docs/assets/panel-redesign/panel/](assets/panel-redesign/index.html)) rather than keep a copy: `TestPanelClassVocabulary` (only the Bulma subset and the `sp-` classes of `panel.css`; every `sp-` class defined, used and shown on the components page), `TestPanelCSSIsTheMockupStylesheet`, `TestPanelCSSContract` (no `!important`, no id or page selector, type floor, widths), `TestTemplatesUseComponents` (no hand-written component markup in a page), `TestPanelOutlinesMatchMockups` and `TestPanelPageStructure` (each rendered page has the component skeleton of its mockup), `TestRedesignedPagesLoadOnlyTheKit`, `TestVendoredAssetsArePinned`, `TestStaticHoldsOnlyTheKnownAssets`, `TestRoutesFollowNavigation`, `TestNoTemplateUsesInlineScriptOrStyle`. The ratchet list `internal/web/view/legacy_pages.txt` is empty and can only shrink; the CI step [design-first](../.github/scripts/design-first.sh) rejects a commit that changes a guard together with the code it judges, and a template that is a byte-for-byte return of one from history. The design contract and the process are in [plans/panel-redesign.md](plans/panel-redesign.md) § Enforcement. ### Security headers, CSP and scripts `secure()` ([internal/web/security.go](../internal/web/security.go)) puts these on every response: `Content-Security-Policy: default-src 'self'; object-src 'none'; base-uri 'none'; form-action 'self'; frame-ancestors 'none'`, `X-Content-Type-Options: nosniff`, `X-Frame-Options: DENY`, `Referrer-Policy: no-referrer`, and `Strict-Transport-Security` when the cookie is `Secure`. Because the policy has no exemptions, no template carries an inline `