mix/selfpost
1
0
Fork 0
Code Issues Pull Requests Actions 21 Packages Projects Releases Wiki Activity

release: 1.9.2: Review pass: port-25 milters, DMARC validation, rate-limit fix

Browse Source 
Review pass over `1.4.0`…`1.9.1`: port 25 no longer shares the submission milter chain, DMARC reports are validated before they are stored, and the auto rate limit is correct for a non-default level-1 window. Schema migration `0010` (index only). Upgrading is a tag bump.
Added:
  - SQLite migration `0010_send_log_app_login_index.sql` —
  - DMARC aggregate reports are accepted as **zip** as well as gzip or plain XML
  - e2e: `checkPort25ConfigCombinations` regenerates the Postfix configuration
Changed:
  - README hero line and a **Positioning** block in `docs/product.md` record the
Fixed:
  - The setup account's password is hashed at bcrypt cost `12` like every other
  - Login/setup rate limiter: protecting blocked buckets from eviction is now
  - `postqueue -p` output is bounded to 4 MiB (the timeout alone left the size of
  - [schema-migrations.md](docs/schema-migrations.md): the schema section was
  - postqueue reader is now time-bounded: `postqueue -p` runs under a 5-second
  … (11 more in CHANGELOG)
Changed:
  - [roadmap.md](docs/roadmap.md) now carries everything the 2026-08-19 code
  - Extracted `internal/configsafe` — the empty-value and forbidden-character
  - Build, deploy and migration comments no longer cite the archived
  - Dropped the `parseDomainRateLimitForm` / `parseAppRateLimitForm` pass-through
  - Extracted shared helpers: `internal/atomicfile` (atomic temp-write+rename) and
  … (7 more in CHANGELOG)
Removed:
  - Unused legacy asset `docs/assets/icon.png` (superseded by the four brand SVGs
  - Plan documents for already-shipped features: `domain-stats-auto-ratelimit`,

Full detail: CHANGELOG.md (1.9.2).
This commit is contained in:
mix
2026-09-22 15:24:53 +03:00
parent 967d3d95de
commit 7ef7f75246
179 changed files with 11558 additions and 1738 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/plans/dmarc-reports.md
-79
View File
@@ -1,79 +0,0 @@
# Plan: dmarc-reports
**Status:** done — shipped in `[1.7.0]` (2026-08-18); security review (Fable)
of the ingest path pending.
---
## Goal
SelfPost **receives** DMARC aggregate reports on SMTP, parses them inside the
image, and **shows summaries in the panel** — pass/fail by source, hints when
`tighten p=` is reasonable. No external DMARC SaaS and no IMAP workflow for the
operator.
## Scope
**In:**
- Inbound SMTP for configured report addresses only (not a general backup-MX).
- gzip + XML aggregate parsing → SQLite summaries per sending domain.
- Panel page and/or per-domain section: recent reports, third-party senders,
delivery health of report ingestion.
- Reuse the `dmarc_report_email` setting (moved off the old `admin` table into
`settings` by migration `0005`) and `domains.dmarc_rua` for DNS templates;
when enabled, suggest a SelfPost-hosted report address.
**Out:**
- Forensic reports (`ruf=`).
- Full dashboards, APIs, email alerting.
- Mailboxes for people (IMAP/POP3/webmail).
## Architecture (sketch)
1. Receiving MTAs → SMTP to SelfPost (hub MX).
2. Postfix virtual alias or dedicated listener → panel ingest worker.
3. Parse XML → `dmarc_reports` table (domain, reporter, counts, date).
4. Panel reads SQLite; links from domain DNS card.
May share port-25 plumbing with [inbound-relay.md](inbound-relay.md) but must
remain a separate, opt-in feature that does not forward mail upstream.
## Done when
- Operator can point `rua=` at an address SelfPost accepts and see parsed
summaries in the panel within one reporting cycle.
- With the feature off, outbound-only behaviour is unchanged.
- Documented in [guide.md](../guide.md); migrations are backward-compatible.
## Risks
- Attack surface of accepting mail (mitigate: strict recipient allow-list).
- Report volume and retention (mitigate: caps + pruning).
## Implementation checklist
**Ingest path.** `DMARC_REPORTS_ENABLE=true` enables `smtp/inet` on 25 (shared
with inbound relay when both are on). Postfix `relay_domains` +
`transport_maps` route allow-listed recipients to a `dmarc-ingest` pipe
(`panel -dmarc-ingest`). `check_recipient_access` on `dmarc_recipients` is the
allow-list; no SASL, no local mailboxes.
**Schema.** Migration `0008_dmarc_reports.sql`: `dmarc_reports` (summary per
aggregate) + `dmarc_report_records` (per-source rows). Dedup on
`(reporter, report_id, domain)`.
**Retention.** Max 500 reports; drop older than 90 days; prune after each
ingest.
Target version cut: **`1.7.0`** (MINOR). One commit per step; code only after
roadmap status is **agreed**. Expand the sketch sections above before step 1
if still thin. See [development.md](../development.md) § Plan checklists.
- [x] Expand plan: ingest path, `dmarc_reports` schema, retention caps — **Sonnet**
- [x] Opt-in inbound SMTP for report addresses only (allow-list) — **Opus**
- [x] Worker: gzip/XML parse → SQLite — **Opus**
- [x] Panel: domain roll-up + parsed report (panel-ui mockups) — **Sonnet**
- [x] Tie-in `dmarc_report_email` / `domains.dmarc_rua` — **Sonnet**
- [x] Tests and [guide.md](../guide.md) — **Sonnet**
- [ ] Security review ingest path — **Fable**
- [x] `go vet`, `go test` on touched packages — **Haiku**
docs/plans/domain-stats-auto-ratelimit.md
-205
View File
@@ -1,205 +0,0 @@
# Plan: domain-stats-auto-ratelimit
**Status:** done — shipped in `[1.6.0]` (2026-08-18); security review (Fable) pending
**Date:** 2026-08-17
**Version:** `1.x` MINOR; migrations must stay compatible with `1.0.0`.
---
## Goal
Give the operator **30-day sending statistics** per domain and per application
(total volume, peak and average rate), and an optional **auto** level-2 rate
limit that sets `max_messages` from the average rate (avg × multiplier) over the
level-1 window.
## Scope
**In:**
- Rolling **30-day** stats on each domain page (domain aggregate + per-app rows):
message count, peak msg/h, average msg/h.
- Level-2 rate limit mode **manual** (today) or **auto** for domain and
application scopes.
- Auto formula: `max_messages = ceil(avg_hourly × multiplier)`, window =
level-1 window (`RATE_LIMIT_WINDOW_SECONDS`); capped at level 1.
- Background recalculation (e.g. every 6 h, alongside send-log prune); milter
reads stored `max_messages` / `window_seconds` only (no aggregates on the
hot path).
- Panel UI: stats card, manual/auto toggle, multiplier field, read-only computed
limit in auto mode, optional «Recalculate now».
- RBAC: domain-admin sees stats and may configure auto/manual for assigned
domains only; same authz as existing rate-limit handlers.
- Tests, [guide.md](../guide.md), [CHANGELOG.md](../../CHANGELOG.md); security
review (Fable) for rate-limit path changes.
**Out:**
- Changing level 1 (Postfix env) — auto only fills level 2 under the L1 cap.
- Automated IP warmup schedules ([guide.md](../guide.md) § IP warmup stays
operator-driven).
- Prometheus/Grafana, alerting, APIs.
- Counting level-1 refusals or `rejected` rows as sent volume.
- Per-client-IP analytics.
## Data source
All metrics come from SQLite `send_log`, same rules as
[`CountMessages`](../../internal/store/ratelimits.go):
- One message = one distinct `queue_id` (many recipients = one count).
- `status != rejected` (level-2 refusals never queued).
- Level-1 refusals are **not** in `send_log` — stats under-count refusals;
document in UI copy.
Retention today is env `SEND_LOG_RETENTION_DAYS` (default 90). Stats use the
last **30 days** of rows still present. If retention < 30 days (after
[send-log-retention](send-log-retention.md)), the stats window is
`min(30, retention)` with a warning.
### Metrics
| Metric | Definition |
|---|---|
| **total** | `COUNT(DISTINCT queue_id)` in the stats window |
| **peak rate** | maximum messages in any **hourly** bucket in that window (msg/h) |
| **avg rate** | `total / hours_in_window`, where `hours_in_window = min(720, age of domain/app in hours, retention hours)` |
Keys: `send_log.domain` (domain scope), `send_log.app_login` (application scope).
## Architecture
```mermaid
flowchart LR
milter["journal-milter"] --> sendLog["send_log"]
sendLog --> statsQuery["StatsQuery 30d"]
statsQuery --> domainPage["domain page stats card"]
statsQuery --> autoJob["auto recalc job"]
autoJob --> rateLimits["rate_limits"]
rateLimits --> milter
```
1. **[`internal/store/stats.go`](../../internal/store/stats.go)** — `SendStats`
with `Total`, `PeakPerHour`, `AvgPerHour`; `DomainSendStats(name, since)` /
`AppSendStats(login, since)`.
2. Hourly buckets: `strftime('%Y-%m-%d %H', created_at)` + `GROUP BY`; subquery
for peak; total via distinct `queue_id`.
3. Indexes `idx_send_log_domain` and `idx_send_log_created_at` exist; add
composite `(domain, created_at)` only if profiling shows need.
4. **Auto recalc** — panel goroutine (same interval as send-log prune): for each
`rate_limits` row with `mode = auto`, recompute `max_messages`, set
`auto_updated_at`. Milter unchanged except reading new columns via existing
`RateLimit` lookup.
### Auto rate limit
Extend [`RateLimit`](../../internal/store/ratelimits.go):
```go
type RateLimit struct {
// existing: Scope, RefID, AllowedIPs, MaxMessages, WindowSeconds
Mode string // "manual" | "auto"
AutoMultiplier float64 // default 2.5 when Mode == "auto"
AutoUpdatedAt time.Time
}
```
**Formula:**
```
avg_hourly = total_messages_in_window / hours_in_window
max_messages = ceil(avg_hourly * auto_multiplier)
window_seconds = L1 window (not editable in auto mode)
max_messages = min(max_messages, L1 max)
```
When `total == 0`: auto limit stays **inactive** (same as empty manual limit);
UI explains that traffic is required before auto can apply.
**Application overrides** ([`handlers_ratelimit.go`](../../internal/web/handlers/handlers_ratelimit.go)):
- Trusted IPs required.
- Auto app ceiling **strictly above** domain limit when domain limit is active.
- Ceiling ≤ L1.
**Fail-open:** store errors during recalc must not weaken enforcement of the
last successfully written limit; recalc failures are logged only.
### Migration (`0006_rate_limit_auto.sql`)
```sql
ALTER TABLE rate_limits ADD COLUMN mode TEXT NOT NULL DEFAULT 'manual'
CHECK (mode IN ('manual', 'auto'));
ALTER TABLE rate_limits ADD COLUMN auto_multiplier REAL;
ALTER TABLE rate_limits ADD COLUMN auto_updated_at TEXT;
```
Existing rows → `manual`.
### Domain export
Today rate limits are **not** exported. This plan adds them (including
`mode`, `auto_multiplier`) to domain transfer JSON — document as a boundary
change in [guide.md](../guide.md) § Export.
## Panel UI
- Domain page ([`domain_detail.html`](../../internal/web/view/templates/domain_detail.html)):
- **Sending statistics (30 days)** — total, peak msg/h, avg msg/h.
- Per-application stats in the app list.
- Rate limit: Manual / Auto, multiplier (e.g. 1.5–5.0, default 2.5), read-only
computed max/window in auto mode, «Recalculate now».
- Optional later: «30d» column on domain list (global admin only).
## Tests
- `internal/store/stats_test.go` — fixtures → total / peak / avg.
- `internal/store/ratelimits_test.go` — auto recalc, L1 cap, app > domain.
- Handler tests — auto form validation, multiplier bounds.
- Milter tests — enforced limit matches last recalculated values.
`go test` / `go vet` on touched packages.
## Done when
- Domain and app 30-day stats visible on the domain page; domain-admin scoping
enforced.
- Manual/auto toggle works for domain and app; auto recalc updates `rate_limits`
and milter enforces stored ceilings.
- Zero-traffic auto stays inactive with clear UI copy.
- [guide.md](../guide.md) and [CHANGELOG.md](../../CHANGELOG.md) updated;
security review passed.
## Risks
- Heavy aggregation on large `send_log` tables — mitigate with indexes or
nightly rollups (phase 2).
- Stats without level-1 visibility — mitigate with operator-facing caveat.
- Auto limit too tight after a spike — multiplier is operator-tuned; show peak
alongside avg in auto UI.
## Dependencies
- [`send-log-retention`](send-log-retention.md) is a separate roadmap item but
should land before or in parallel so operators can set retention ≥ 30 days
from the panel.
**Version:** `1.x` MINOR.
## Implementation checklist
Target version cut: **`1.6.0`** (MINOR). One commit per step; code only after
roadmap status is **agreed**. See [development.md](../development.md) § Plan
checklists.
- [x] Migration `0007_rate_limit_auto.sql` (`mode`, `auto_multiplier`, `auto_updated_at`) — **Opus**
- [x] `internal/store/stats.go`: total / peak / avg over 30 days — **Opus**
- [x] Auto recalc job (6h): `ceil(avg × multiplier)`, L1 cap, fail-open on error — **Opus**
- [x] Extend `RateLimit` + handler forms (manual/auto) — **Opus**
- [x] Domain page stats card + per-app stats (`domain_detail.html`) — **Sonnet**
- [x] Domain export JSON includes rate limits — **Opus**
- [x] Milter tests — enforced limit matches stored ceiling — **Opus**
- [x] Store and handler tests — **Sonnet**
- [x] [guide.md](../guide.md) — **Sonnet**
- [ ] Security review rate-limit path — **Fable** (pending post-release)
- [x] `go vet`, `go test` on touched packages — **Haiku**
docs/plans/inbound-antispam-panel.md
+11 -6
View File
@@ -121,11 +121,14 @@ No separate feature flag beyond inbound relay + existing antispam env vars.
### UI
New nav item **Inbound spam** (or subsection under **Inbound**) — journal table
with filters (decision, domain, date range), list management on the same page or
a tab. Mockup: add `docs/assets/panel-ui/inbound_spam.html` in the panel step.
The journal is **Inbound › Log** and list management is a new **Inbound ›
Filter lists** page (global-only: the lists are instance-wide) — see
[panel-redesign.md](panel-redesign.md) § Room for roadmap items. The journal
table has filters (decision, domain, date range). Mockups are drawn first, in
`docs/assets/panel-redesign/panel/src/` under that plan's design contract —
not in the superseded `docs/assets/panel-ui/`.
## Schema (migration `0010_inbound_spam_log.sql`)
## Schema (migration `0011_inbound_spam_log.sql`)
**`inbound_spam_log`**
@@ -218,7 +221,9 @@ flowchart LR
## Dependencies
- [inbound-relay.md](inbound-relay.md) (shipped).
- Inbound relay, shipped in `[1.4.0]` — as built in
[architecture.md](../architecture.md) § Mail path. Its plan file was removed
once it shipped.
## Model routing (this plan)
@@ -238,7 +243,7 @@ Target version cut: **`1.10.0`** (MINOR). One commit per step;
[development.md](../development.md) § Plan checklists.
- [x] Agree journal fields, retention, and RBAC (this plan § Decisions) — **Composer**
- [ ] Migration `0010_inbound_spam_log.sql` — **Composer**
- [ ] Migration `0011_inbound_spam_log.sql` — **Composer**
- [ ] Inbound journal-milter + Postfix wiring — **Composer**
- [ ] mail.log tailer: inbound reject rows — **Composer**
- [ ] List CRUD + validation + atomic rspamd map sync — **Composer**
docs/plans/inbound-quarantine.md
+13 -2
View File
@@ -22,6 +22,15 @@ so we can compare approaches before committing.
reference to them) under `/data`, retention, release workflow, and a wider
attack/backup surface — closer to "mini-mailbox" than pure relay.
## Mockup
A target look is drawn — list and detail —
[in-quarantine](../assets/panel-redesign/panel/in-quarantine.html),
[in-quarantine-message](../assets/panel-redesign/panel/in-quarantine-message.html)
— under the design contract of [panel-redesign.md](panel-redesign.md). It
assumes answers to the questions below (SelfPost holds the mail, release goes
to the upstream, 14 days, headers only in the panel); it does not settle them.
## Open questions (to decide before a checklist)
1. **Where mail lives**
@@ -43,7 +52,8 @@ attack/backup surface — closer to "mini-mailbox" than pure relay.
## Context (as-built)
- Inbound relay forwards or rejects; no local delivery ([inbound-relay.md](inbound-relay.md)).
- Inbound relay forwards or rejects; no local delivery
([architecture.md](../architecture.md) § Mail path).
- Optional `INBOUND_ANTISPAM_MILTER` — external rspamd may quarantine on its
own today, without panel integration.
- DMARC `p=quarantine` is unrelated (outbound policy for receivers).
@@ -56,7 +66,8 @@ attack/backup surface — closer to "mini-mailbox" than pure relay.
## Dependencies
- [inbound-relay.md](inbound-relay.md) (shipped).
- Inbound relay, shipped in `[1.4.0]` — as built in
[architecture.md](../architecture.md) § Mail path.
- Sensible to decide **after** or **alongside**
[inbound-antispam-panel.md](inbound-antispam-panel.md) (journal/lists), but
not blocked on it for design discussion.
docs/plans/inbound-relay.md
-189
View File
@@ -1,189 +0,0 @@
# Plan: inbound-relay (inbound relay)
**Status:** done — shipped in `[1.4.0]` (2026-08-17)
**Version:** 1.4.0 MINOR (flag off is compatible; not a 2.x break).
**Order:** after queue-retries `[1.3.1]`. One checklist step remains:
security review of the inbound path (Fable).
---
## Goal
The ability to accept mail on port 25 for explicitly configured domains and
forward it to a given upstream backend (a backup-MX / relay-forwarder role), as
a **module disabled by default** that changes neither the behaviour nor the
attack surface of the base outbound relay.
## What it is for (scenarios)
- **Backup-MX** — accept mail while the domain's primary mail server is
temporarily unreachable, and hand it over when it comes back.
- **A front for a server without a public IP** — the operator runs their own
mail server which, for whatever reason, **cannot accept mail from the
internet itself** (no static or public IP, behind NAT, a private address,
inbound port 25 blocked, and so on). SelfPost, with a public IP and a correct
PTR, acts as the domain's public entry node (the MX points at it) and
forwards mail to that internal or otherwise unreachable server.
## Scope boundary (critical — what this is NOT)
- **IT IS:** acceptance on 25 for domains from an explicit list, plus
forwarding (relay/forward) to an upstream (`relay_domains` +
`transport_maps` + `relay_recipient_maps`). Postfix here is a pure forwarder,
with no local delivery.
- **IT IS NOT (out of scope, [product.md](../product.md)):** local delivery to
mailboxes, IMAP/POP3, webmail, Dovecot. No mailboxes at all. SelfPost also
**neither implements nor bundles** an anti-spam or anti-virus engine
(rspamd/ClamAV) — but, unlike the earlier wording, it does **not** push
filtering onto the backend either (see the "Anti-spam" section below): it
provides an attachment point for an external filter.
## Why as an option / plugin
- Accepting on port 25 changes the threat model (open relay for inbound,
backscatter, spam ingress). So it is **off** by default behind the
`INBOUND_RELAY_ENABLE=false` env flag; turning it on is a deliberate step by
the operator.
- Isolation: separate SQLite tables, separate panel handlers and pages, a
separate branch of config generation. With the flag off, the inbound
listener, the tables and the UI are absent — the base outbound path is
byte-for-byte unchanged.
## What to do
- The `INBOUND_RELAY_ENABLE` env flag (default false); when `true`, generate
the inbound service and its config from panel state the same way the rest of
the config is generated (`postfix-config.sh`).
- **`master.cf`:** an inbound `smtp inet` on 25 for accepting from the internet
(today 25 is used only for outbound delivery). Separate from 465/587: on 25
SASL is **not** offered and sending outwards is **not** allowed — inbound
only, for `relay_domains`.
- **Anti-open-relay for inbound (mandatory):** the inbound smtpd's
`smtpd_relay_restrictions` / `smtpd_recipient_restrictions` accept mail
**only** for domains in `relay_domains` and **only** for known recipients
(`relay_recipient_maps`); everything else gets
`reject_unauth_destination` / `reject_unlisted_recipient`. An open relay, or
accepting "for anyone", is impossible.
- **Backscatter:** knowing the valid recipients is preferable (reject unknown
recipient at RCPT stage) so that bounces to non-existent addresses are never
generated.
- **The panel manages:** the list of inbound domains; for each one the upstream
destination (`host:port`, transport), an optional list of valid recipients,
and optional TLS to the upstream. Strict validation of domain, host and port
(whitelist), injection-safe writing of map files (as with
`sender_login_maps` in Phase 4), `os/exec` without a shell
([security.md](../security.md)).
- **Milters:** OpenDKIM is not needed on the inbound path (we do not sign
someone else's inbound mail). The journal-milter can optionally be reused for
an inbound journal (extra work), or the inbound path can go without it in the
first stage; fail-open behaviour is preserved.
- **Rate limit / size:** a coarse per-client-IP limit (`anvil`, as L1) and
`message_size_limit` on the inbound smtpd.
## Anti-spam (important, but optional)
This is a valuable option, but it is **not mandatory**: some operators will be
perfectly served by **blind forwarding without filtering** — when the backend
can filter on content itself, when the upstream is trusted, or when the volume
and risk are low. So the anti-spam hook is **off** by default (an empty
`INBOUND_ANTISPAM_MILTER`), and the inbound relay is fully functional without
it.
What matters is something else: where filtering is technically possible. With a
"blind" relay the destination backend sees **SelfPost's** address as the
connecting IP, not the original sender's, so everything on the backend that
depends on the origin IP breaks (DNSBL and reputation are checked against
SelfPost's IP; SPF returns fail, since SelfPost is not in the sending domain's
SPF). **The only point where the real client IP is still visible is the inbound
hop at SelfPost** — so for those who need filtering, it has to be *attachable
right here*, not delegated to a backend that has already lost the information.
The attachment design:
- **The anti-spam engine is a separate optional container** (rspamd or
similar), which the operator runs **only if this option is wanted** (the same
principle as the reverse proxy — a separate container outside the SelfPost
image). SelfPost **neither contains nor starts it** — the image and the "one
container, three processes" principle are unchanged, and
[product.md](../product.md)'s out-of-scope list is not violated (SelfPost
does not implement anti-spam).
- **SelfPost provides the attachment point:** a milter hook on the inbound
smtpd. The engine's address is set via env (for example,
`INBOUND_ANTISPAM_MILTER=inet:antispam:11332`, empty → the hook is off) and
is added to `smtpd_milters` for the **inbound path only** (not on 465/587).
Postfix passes the milter the real client IP, HELO and PTR — the filter sees
the true origin. `milter_default_action` for that milter is configurable
(fail-open vs tempfail); the default is to be decided during implementation.
- **A native backstop with no dependencies:** on that same inbound hop,
Postfix's own origin-IP facilities are available — `reject_rbl_client`
(DNSBL) and HELO/PTR checks — and they work even without an external
container. Plus preserving authentication results for downstream through ARC
or `Received`, where part of the filtering does remain on the backend.
- **docker-compose:** document an optional anti-spam sidecar fragment (like the
alternative reverse-proxy fragments) — the container comes up with the stack
only when the option is enabled.
- **Persistence:** new tables and map files under `/data` — they land in the
full backup automatically (Phase 9). Domain export/import can be extended
with the inbound configuration — optional, to be flagged.
- **DNS documentation:** an inbound domain needs an `MX` record pointing at the
server (unlike outbound, where no MX is required) — to be reflected in the
README's DNS section.
## Security
[security.md](../security.md): server-side input validation, escaped writes to
config files, `exec` without interpolation, no open relay, protection against
backscatter.
## Done when
With `INBOUND_RELAY_ENABLE=true` and a configured domain, mail arriving on port
25 for that domain is forwarded to the given upstream; mail for unconfigured
domains or recipients is rejected (not an open relay, no backscatter); with
`INBOUND_ANTISPAM_MILTER` set, inbound mail passes through the external filter
with the real origin IP (verified with a sidecar container), and with it empty
the hook stays out of the way; with `INBOUND_RELAY_ENABLE=false` the inbound
port, tables and UI are absent and the base outbound relay is unchanged;
`build`/`vet`/`test`/image green.
## Risks
- open relay / backscatter — removed by `relay_domains` +
`relay_recipient_maps` + `reject_unauth_destination`;
- the loss of the origin IP for filtering on the backend when forwarding —
removed by the anti-spam milter hook plus native DNSBL on the inbound hop,
where the origin IP is still visible;
- port 25 accepting mail widens the attack surface (off by default);
- semver: if the contract turns out incompatible (ports, backup, behaviour with
the flag off) a major `2.x` is possible; the decision comes after the
implementation.
**External deployment dependency:** the optional anti-spam container — outside
the SelfPost image, brought up by the operator when the option is enabled.
## Dependencies
A finished outbound path (already implemented). Agreement obtained — see the
status above.
## Implementation checklist
Target version cut: **`1.4.0`** (MINOR). One commit per step; see
[development.md](../development.md) § Plan checklists. UI reference:
[panel-ui inbound mockups](../assets/panel-ui/inbound.html).
- [x] Migration: inbound domain / recipient / transport tables under `/data` — **Opus**
- [x] `INBOUND_RELAY_ENABLE` (default false) in entrypoint + `postfix-config.sh` — **Opus**
- [x] `master.cf`: inbound `smtp inet` on 25; separate from 465/587 — **Opus**
- [x] Generate `relay_domains`, `transport_maps`, `relay_recipient_maps` (injection-safe) — **Opus**
- [x] `smtpd_relay_restrictions` / recipient maps — no open relay, no backscatter — **Opus**
- [x] `internal/store` CRUD + validation (domain, host, port) — **Opus**
- [x] Panel: list, add, domain detail, recipients, danger zone (mockups) — **Sonnet**
- [x] Rate limit + `message_size_limit` on inbound smtpd — **Opus**
- [x] Optional `INBOUND_ANTISPAM_MILTER` + compose fragment — **Opus**
- [x] DNS MX copy in README/guide; `.env.example` — **Sonnet**
- [x] Backup/export inbound config (per plan optional flag) — **Opus**
- [x] Unit + handler tests; image build and container smoke — **Opus**
- [x] [guide.md](../guide.md) and [security.md](../security.md) — **Sonnet**
- [ ] Security review inbound path — **Fable**
- [x] `go vet`, `go test`, e2e if applicable — **Haiku**
docs/plans/panel-docs.md
-54
View File
@@ -1,54 +0,0 @@
# Plan: panel-docs (in-panel operator documentation)
**Status:** agreed
**Date:** 2026-08-17
**Version:** `1.x` MINOR; `candidate` until explicitly agreed.
---
## Goal
Built-in operator documentation in the panel — short pages or a help drawer that
explain what each Status check and other controls mean, without sending the
operator out to [guide.md](../guide.md).
## Scope
**In:**
- Help drawer or short help pages (CSS-checkbox pattern from
[panel-ui mockups](../assets/panel-ui/system.html)).
- Seed content: Status blurbs removed from cards for a denser layout — Machine
(kernel counters / rate window), TLS certificate (port 465, reverse-proxy
mount), Hostname / reverse DNS (FCrDNS, PTR at the hosting provider), and
similar notes for other surfaces as inline commentary is removed.
- «?» entry points on domain cards (mockups).
**Out:**
- A second copy of the full operator guide.
- Translation workflow beyond English (same as the rest of the panel).
## Done when
An operator can open help from the panel for those topics; the removed Status
blurbs are preserved there (or equivalent); no requirement to read the git
tree for day-to-day meaning of a card.
## Risks
Copy ownership and keeping help in sync when checks change; not bloating every
page with a second column of prose.
## Implementation checklist
Target version cut: **`1.8.0`** (MINOR). One commit per step; code only after
roadmap status is **agreed**. See [development.md](../development.md) § Plan
checklists.
- [x] Help drawer / pages shell (CSS checkbox pattern from mockups) — **Sonnet**
- [x] Seed Status blurbs (Machine, TLS, PTR, …) — **Sonnet**
- [x] «?» entry points on domain cards — **Sonnet**
- [x] [guide.md](../guide.md) boundary: in-panel help vs full guide — **Sonnet**
- [x] Template tests — **Sonnet**
- [x] `go vet`, `go test` on touched packages — **Haiku**
docs/plans/panel-redesign.md
+409
View File
@@ -0,0 +1,409 @@
# Plan: panel-redesign
**Status:** candidate
**Date:** 2026-09-21 (replaces the 2026-09-08 givens)
**Version:** `2.0.0` MAJOR — a breaking release by the owner's decision (§ No compatibility).
---
## What this file is
The design contract for the control panel and the plan to implement it. The
design itself is drawn and accepted:
| Artefact | Role |
|---|---|
| [docs/assets/panel-redesign/panel/](../assets/panel-redesign/index.html) | 33 screens — **the** reference for what each page looks like |
| [panel/components.html](../assets/panel-redesign/panel/components.html) | every component in every state — the only parts a page may be built from |
| [panel/outlines.json](../assets/panel-redesign/panel/outlines.json) | the component skeleton of each screen, generated from the mockups |
| [panel/check.py](../assets/panel-redesign/panel/check.py) | the contract as a program; its rules are ported to Go tests below |
| [panel/audit.js](../assets/panel-redesign/panel/audit.js) | what only a browser can measure: text size, contrast, overflow, clipped values — pasted into the console, on the mockups or on the running panel |
When this file and a mockup disagree, the mockup wins and this file is fixed.
When an implementation and a mockup disagree, the implementation is wrong.
The 2026-09-08 givens ("no component library, Web Awesome layout CSS only") are
withdrawn. `docs/assets/panel-ui/` and the stage-1 directions `a/`, `b/`, `c/`
are history, not a brief.
## Goal
A panel an operator opens a few times a year and still finds their way in: the
important things in view, details one or two clicks away, one visual language
on every page, and no page that tunes its own width.
## Delivery constraints (unchanged)
| Constraint | Where |
|---|---|
| Go `html/template`, server-rendered; htmx 2.x for polled fragments | [internal/web/view/](../../internal/web/view/) |
| Assets embedded (`//go:embed`), no CDN at runtime | [view.go](../../internal/web/view/view.go) |
| CSP `default-src 'self'` — no inline script, no `style=""` | [security.go](../../internal/web/security.go) |
| State changes are native HTML POST; no script is the only way to submit | templates |
| New front-end is MIT-or-better and listed in NOTICE | [NOTICE](../../NOTICE) |
Bulma was chosen partly because it needs nothing from this table relaxed: no
JavaScript, no `data:` URIs.
## Information architecture
Navigation is grouped by mail direction. Page titles carry the group
(**Outbound domains**, **Inbound log**); menu entries do not repeat it.
| Group | Pages | Today |
|---|---|---|
| Overview | Overview | `/status` (verdicts only) |
| Outbound | Domains · Log · Queue · DMARC reports | `/domains`, `/deliveries`, `/mail-queue`, `/dmarc` |
| Inbound | Domains · Log · Queue | `/inbound`; Log and Queue are new |
| Server | Health · System log · Backup · Users · Settings | Health is new (the tables that left `/status`) |
| User menu | Account · Help · Sign out | Account is the user's half of `/settings`: credentials, e-mail, DMARC default |
A hidden page leaves no gap in the menu. The inbound and DMARC feature flags
stay as in [web.go](../../internal/web/web.go); who sees what changes — see
§ Who sees what.
## No compatibility
**Decided 2026-09-21 (owner): nothing is kept compatible across the redesign,
and the migration chain is deleted to zero.** The redesign is `2.0.0`.
- **URLs.** Old paths are not redirected; they are gone. Bookmarks break once.
- **Schema.** `internal/store/migrations/` is emptied and replaced by one
baseline, `0001_init.sql`, equal to the 2.0 schema (including the new
assignment table below). No historical `admin` table, no create-then-drop.
This is [schema-squash](../roadmap.md#schema-squash) with its trigger now
decided and without the upgrade gate that item planned: 2.0 starts from an
empty data directory.
- **Docs.** `schema-migrations.md` shrinks to the 2.0 baseline; the 1.x chain
stays in git history.
The only live installation is the owner's, and it is set up again on 2.0.
Nothing is built to ease that move, and nothing needs to be.
## Who sees what
**Decided 2026-09-21 (owner): a domain administrator manages inbound domains
too, and that is granted separately from outbound.** Today every inbound
handler is `requireGlobal()` and `user_domains` references outbound domains
only.
| | `global` | `domain` |
|---|---|---|
| Overview, Server | yes | no (404) |
| Outbound: domain, applications, settings, log, DMARC | all domains | assigned outbound domains |
| Inbound: domain (upstream, recipients), log | all domains | assigned inbound domains |
| Add / delete a domain, either direction | yes | no |
| Queues, either direction | yes | no, until the queue is a filterable table |
| Inbound › Quarantine | all domains | assigned inbound domains |
| Inbound › Filter lists, Server › Preflight | yes | no — the lists are instance-wide |
| Account, Help | yes | yes |
- New table `user_inbound_domains (user_id, inbound_domain_id)`, mirroring
`user_domains`; both live in the 2.0 baseline. The same name on both lists
is two assignments — the two kinds of domain stay separate entities.
- The user form has two lists (mockup `user.html`); the *domain* role needs at
least one assignment on either.
- **Each list has *All*, which includes domains added later** — a flag per
user and direction (`users.all_domains`, `users.all_inbound_domains`), not a
tick on every row: otherwise tomorrow's domain is silently unassigned. *All*
widens the list, not the role: adding and deleting domains, the queues and
Server stay with the global role. The form works without scripts — with
*All* ticked the server ignores the rows below it.
- **The role is a reach, not a rank.** Everyone in the panel is an
administrator, so the label is `global` or `domain` — in the users list, the
role select and the page kickers; never "admin".
- A group with nothing assigned is absent from that user's menu.
- **The spam filter is visible to whoever it affects.** Whether the instance
filters inbound mail is an environment setting, so a domain administrator
cannot look it up: every inbound domain page has a *Spam filter* box (on /
off, engine, what happens to suspicious mail, who keeps the lists), the
inbound domains list shows it in its head, and for the global administrator
the filter's socket is one of the milter sockets on Health and Overview.
- Authorization helpers come in pairs: `requireDomain(id)` and
`requireInboundDomain(id)`; lists and logs are filtered by assignment in the
query, not in the template.
## Account e-mail
**Decided 2026-09-21 (owner): a user's e-mail is theirs, not a DMARC field.**
Today the column is `users.dmarc_report_email` and the only place it shows is
the DMARC card, so it reads as if it existed for that alone.
- The 2.0 baseline has `users.email`. It is what the panel writes to: event
notifications ([panel-notifications](../roadmap.md#panel-notifications)) and
a reset link ([password-reset](../roadmap.md#password-reset)).
- **Account** edits it; the global administrator sees and sets it on the user
form and sees it in the users list.
- DMARC only *may* use it. **The default report address is a user's setting,
not the server's** (owner, 2026-09-21): it lives on **Account** with its
report-authorization record, as a choice — SelfPost hosted, *my account
e-mail*, another address, none. Historically it was the administrator's
profile field, later mirrored into an instance setting only the global user
could edit; Server › Settings keeps nothing of it.
- A domain can have several users, so "the default" must be someone's. A
domain's report address is *the default of a named user* (shown as
`admin's default — mix@example.org`), the hosted address, a custom one or
none. A new domain follows whoever created it; any user of the domain may
switch it to their own default. If that user is deleted the domain falls
to *none* and its DMARC check says so. Baseline: `users.dmarc_default_mode`,
`users.dmarc_default_address`, `domains.dmarc_rua_user_id`.
## Routes
**Decided 2026-09-21 (owner): paths are renamed to follow the new names and
structure.** A URL reads like the menu: `/<group>/<page>`, an entity under its
list, an action under the thing it changes.
| Screen (mockup) | New | Today |
|---|---|---|
| overview | `GET /overview` | `/status` |
| health | `GET /server/health` · `POST …/recheck` · `POST …/reload` | tables of `/status` · `/status/recheck` · `/reload` |
| preflight | `GET /server/preflight` · `POST …/preflight/run` · `POST …/preflight/test-email` | — (preflight; its plan says `/preflight`) |
| out-domains | `GET, POST /outbound/domains` | `/domains` |
| out-domain | `GET /outbound/domains/{id}` · `POST …/dns-recheck` | `/domains/{id}` |
| out-domain-settings | `GET /outbound/domains/{id}/settings` · `POST …/settings/reports` · `…/settings/ratelimit` · `…/settings/ratelimit/recalc` · `…/settings/export` | cards of `/domains/{id}` · `…/dmarc` · `…/ratelimit` · `…/ratelimit/recalc` · `…/export` |
| out-domain-delete | `GET, POST /outbound/domains/{id}/delete` | `/domains/{id}/delete` |
| out-app | `GET, POST /outbound/domains/{id}/applications/new` · `GET, POST …/applications/{aid}` · `POST …/{aid}/ratelimit/recalc` · `POST …/{aid}/delete` | `POST /domains/{id}/applications` · `POST /applications/{aid}/mode`, `/authips`, `/ratelimit` · `…/recalc` · `…/delete` |
| out-app-created | the response of `POST …/applications/new` and `POST …/{aid}/password`; no GET path, `Cache-Control: no-store` | a card on `/domains/{id}` after the POST |
| out-log · out-message | `GET /outbound/log` · `GET /outbound/log/{id}` | `/deliveries` · `/deliveries/{id}` |
| out-queue | `GET /outbound/queue` | `/mail-queue` |
| dmarc · dmarc-domain · dmarc-report | `GET /outbound/dmarc` · `…/dmarc/domains/{id}` · `…/dmarc/reports/{id}` | `/dmarc/…` |
| in-domains · in-domain | `GET, POST /inbound/domains` · `GET /inbound/domains/{id}` · `POST …/dns-recheck`, `…/upstream`, `…/recipients` | `/inbound` · `/inbound/{id}/…` |
| in-domain-delete | `GET, POST /inbound/domains/{id}/delete` | `/inbound/{id}/delete` |
| in-log · in-queue | `GET /inbound/log` · `GET /inbound/queue` | — (new, see below) |
| in-quarantine · in-quarantine-message | `GET /inbound/quarantine` · `GET …/quarantine/{id}` · `POST …/{id}/release` · `POST …/{id}/discard` | — (inbound-quarantine) |
| in-filter-lists | `GET, POST /inbound/filter-lists` · `POST …/filter-lists/{id}/delete` | — (inbound-antispam-panel) |
| system-log | `GET /server/log` | `/system-log` |
| backup | `GET, POST /server/backup` · `POST /server/backup/import` | `/backup` · `POST /domains/import` |
| users · user · user-delete | `/server/users` · `…/new` · `…/{uid}` · `…/{uid}/delete` | `/users/…` |
| settings | `GET, POST /server/settings` | the instance half of `/settings`: log retention, rate limits; plus the notifications switch |
| account | `GET /account` · `POST /account/profile`, `/password`, `/notifications`, `/dmarc` | `/settings`: credentials and the DMARC default (today `/account` redirects there) |
| help · components | `GET /help` · `GET /server/components` | `/help` · — |
Unchanged: `/healthz`, `/license`, `/static/`, `/setup/{token}`, `/login`,
`/logout`. `GET /` goes to `/overview`, or to `/outbound/domains` for a domain
administrator.
Rules that come with the rename:
- **One fragment name.** Every polled page answers `<page>/fragment`
(`/overview/fragment`, `/outbound/log/fragment`, …) — today it is
`fragment`, `rows` and `body`.
- **The application form is one POST.** The mockup has a single *Save
application*; sender, IP allow-list and rate limit are validated and saved
together, or not at all. The three separate POSTs go away.
- **No redirects.** Old paths, GET and POST alike, are removed (§ No
compatibility), including today's `/account` → `/settings` redirect. The
full list goes under *Removed* in the CHANGELOG.
- **Authorization can follow the tree — for `/server/` only.** Everything
under `/server/` is global-only, so `requireGlobal()` can guard that subtree
once instead of once per handler. `/outbound/` and `/inbound/` are checked
per domain (§ Who sees what). Both are security-relevant: they go through
the security review, and a per-route test asserts 404 for a domain
administrator on every path they must not reach — including another
tenant's domain id.
- `TestRoutesFollowNavigation` — every path in this table is registered with
the listed methods, every menu entry points at a registered path, every old
path answers 404, and no template contains an old path.
**Needs its own roadmap decision before it is built** (the mockups show the
target; none of this is smuggled in with the restyle):
- Queue as a table — parse `postqueue -j` instead of printing `postqueue -p`.
- Inbound queue — the same queue filtered by recipient domain.
- Inbound log — rides on `inbound-antispam-panel` (`1.10.0`).
- Server › Health — a route for the tables removed from Overview, and the
home of *Reload configuration* (today a card on `/status`).
Until a row above is agreed, its page ships in the old content inside the new
shell, and its menu entry is hidden if the page does not exist.
## Room for roadmap items
The mockups draw today's features, the four rows above, and the screens of
the open roadmap items — none of them needed a new component or layout. A
mockup of a feature is its target look, not a decision to build it: each item
keeps its own status and plan. From here on a feature's screens are drawn
mockup-first in this system (`panel/src/`, `build.py`, `check.py --write`) as
the first step of that feature's plan — not invented during implementation.
| Roadmap item | Where it lands | Built from | Drawn |
|---|---|---|---|
| inbound-antispam-panel (agreed, `1.10.0`) | journal = **Inbound › Log** (add client IP and engine to the mockup's columns; `quarantine` joins the decisions); lists = new **Inbound › Filter lists**, global-only because the lists are instance-wide | list layout: add form + two tables, `status_tag`, filter form in `box_head`; the log links *Allow / Deny sender* to the lists | `in-log`, `in-filter-lists` |
| inbound-quarantine (candidate) | new **Inbound › Quarantine**, list + detail with *Release* / *Discard* | list layout; detail like `out-message` with `facts`, `log_pane` for headers only, Release beside a `sp-danger-zone` Discard | `in-quarantine`, `in-quarantine-message` — the open questions of its plan (where mail lives, what release means, retention, RBAC) stay open; the mockup assumes SelfPost-held mail, release to upstream, 14 days, delegated like the rest of Inbound |
| preflight (candidate) | new **Server › Preflight**, next to Health: Health is what runs now, Preflight is the deeper on-demand installation check with a test e-mail form | `postmark` verdict, a table of checks with `status_tag` and a *What to do* column, one form box for the test e-mail | `preflight` |
| panel-notifications (candidate) | per-user event choice on **Account**; one switch under **Server › Settings** | checkboxes in a box, `help` | `account`, `settings` |
| password-reset (candidate) | a link on sign-in, two signed-out screens (request, set new password) | the signed-out layout | **no** |
| csrf-tokens | no screen; every form gains a hidden field — forms are rendered through one helper so the token cannot be forgotten | — | n/a |
| template-data-typing | no screen; the component partials are the natural first users of typed data | — | n/a |
| structured-logging, contributing, review follow-ups | none | — | n/a |
The sibling strip under the navbar holds five or six entries per group without
wrapping; Inbound would reach five (Domains · Log · Queue · Quarantine ·
Filter lists), Server six. Past that a group is split rather than squeezed.
Ordering with 2.0: inbound-antispam-panel ships first in `1.10.0` and adds its
migration to the 1.x chain as planned; the 2.0 baseline then folds it in.
## Three layers
1. **Bulma, as shipped.** Vendored `bulma.min.css`, never edited. Only the
classes listed in `check.py` (`BULMA`) are used; reaching for another is a
design change.
2. **Adapters** — where display logic lives once:
`status_tag` (status → `tag is-… is-light`) as a template function, and
`copy_field` (the read-only value with its Copy button).
3. **Own components** — everything with an `sp-` class, styled in one
`panel.css`, each rendered by one partial in `components.html`:
| Partial | What it is |
|---|---|
| `page_head` | path, `<h1>`, lead, actions; optionally a `postmark` |
| `postmark` | the round verdict stamp: ok / `sp-warn` / `sp-fail` |
| `box`, `box_head`, `box_foot` | numbered mono head with an end slot and an optional `sp-help` link to the section's Help topic (replaces the help drawer); variants `sp-danger-zone`, `sp-credential` |
| `flash` | Bulma `notification is-success / is-danger is-light` between `page_head` and the first box; a field's error stays in `help is-danger` |
| `health_card` | Overview only; ok / `sp-warn` / `sp-fail`; links to its detail |
| `dns_record` | one block per record: status, what to publish, "in DNS now" only on a mismatch |
| `facts` | label / value grid, `sp-big` for headline numbers |
| `side_menu` | Bulma `menu` with icons and count tags; sticky |
| `timeline`, `log_pane`, `empty_state`, `confirm_list`, `help_topic` | as on the components page |
| form rows | `sp-row` (wide + short), `sp-pair` (two equal) |
The shell belongs to `layout.html` alone: a brick navbar with hover menus
that carries the **wordmark, not the stamp** (at nav size the stamp dissolves
into the bar; it stays on sign-in, setup and the favicon); under it the
stamp's paper margin with the perforation along its outer edge — round holes,
flat teeth, the mark's ink hairline, one tile `perf-edge.svg`; then the
sibling-page strip; and the footer.
## Type and tables
- **Three sizes.** Body 16px; `--sp-fs-s` 14px for supporting text (labels,
`help`, notes, mono values, logs); `--sp-fs-xs` 13px for upper-case mono
micro-labels and tags only. Nothing a person has to read is under 13px.
Bulma's own small sizes are overridden (`help`, `tag`, `label`,
`menu-label`, the navbar dropdown) or not used (`is-small` on a control,
`is-size-7` — `sp-small` instead). `check.py` enforces the floor.
- **Contrast.** Text is at least 4.5:1 against the ground it actually sits on
(3:1 for large text). Two traps are closed in the theme: Bulma picks a solid
button's text colour from the fill's lightness, and the brick and the red
land on the wrong side — solid buttons are white on colour; and the muted
grey is `#5c6774`, not the 4.1:1 `#6b7684` of the old panel.
- **A table sits in a `table-container`**, so a wide one scrolls inside its
box instead of being cut off. Addresses in table cells get a `<wbr>` after
the `@` from one template function, so a column can shrink without breaking a
name mid-word. Screens fit without scrolling at 1280px and up; below that a
dense log scrolls inside its box.
## Layout rules
A page is exactly one of:
- **list** — `page_head`, then full-width boxes: the add form, then the table.
- **detail** — `page_head`, then `columns`: `is-3` `side_menu` + `is-9` boxes.
- **form** — `page_head`, then boxes in equal `columns`, one submit row below.
- **signed out** — the split brand / form screen.
Width comes from the layout, never from the page. A narrow form does not get a
`max-width`; it gets a column. **What stands beside it must be real** — the
form's own result, the facts it changes, a second form. A box written to fill
the space beside another is the same defect as a width tuned by hand, and the
first draft of Preflight had one. If nothing real belongs there, the form
shares its box with its result, or the page uses fewer columns.
## The contract
An implementer — human or model — **must not**:
1. Write an `sp-` class, a Bulma class outside the subset, or any other class
that the components page does not show.
2. Add a component, a state or a layout. If a screen seems to need one, **stop
and ask**; do not approximate it with ad-hoc markup.
3. Add a box whose only job is to fill space, or text that restates the page.
Every box answers a question the operator has on that page.
4. Put a selector in `panel.css` that names a page or an element id, uses
`!important`, sets a width outside the `WIDTH_OK` list, or a font size under
the 13px floor.
5. Write component markup by hand in a page template instead of calling the
partial.
6. Edit the vendored Bulma, or load a second CSS framework, icon set or any
JavaScript widget.
7. Change a mockup, `outlines.json`, `check.py` or a guard test in the same
commit as `internal/web`, or at all without the owner's explicit
instruction. **The guards are not the implementer's to adjust.** A failing
guard means the implementation is wrong, not the guard.
8. Report a step as done on the strength of a description. Done means the
evidence below exists.
**Changing the design** is allowed and has one route: mockup first. Edit the
fragment in `panel/src/`, the components page if a part changed, run
`build.py` and `check.py --write`, commit as `design: …` with the reason, get
it accepted — and only then touch `internal/web`.
## Enforcement
Guard tests live in `internal/web/view` and run in `go test ./...` and CI.
They land **before** any template is restyled (checklist step 2), written and
reviewed by models other than the one that implements the templates.
| Test | Fails when |
|---|---|
| `TestPanelClassVocabulary` | a template uses a class outside the Bulma subset + the `sp-` classes of `panel.css`; an `sp-` class is defined but unused, or used but absent from `components.html` |
| `TestPanelCSSContract` | `panel.css` has `!important`, an id or page-name selector, or a width outside `WIDTH_OK` |
| `TestTemplatesUseComponents` | a page template contains `sp-box-head`, `sp-postmark`, `sp-record`, `sp-facts`, … literally instead of `{{template "…"}}` |
| `TestPanelOutlinesMatchMockups` | a page rendered with fixtures has a different component skeleton than `outlines.json` — a different design cannot pass as this one |
| `TestVendoredBulmaChecksum` | `bulma.min.css` differs from the pinned SHA-256 |
| `TestNoTemplateUsesInlineScriptOrStyle` | (exists) inline `style` / script |
CI adds one step, `design-first`: a commit that touches
`internal/web/view/**` must not touch `docs/assets/panel-redesign/panel/**`
or the guard tests.
### Evidence for "done"
Every checklist step that restyles pages ends with, in the commit:
- `go test ./internal/web/...` output in the commit message body (pass);
- the output of `panel/audit.js` run against the restyled pages of the running
container, with an empty `problems` list;
- a screenshot of each restyled page at 1440×900, saved next to its mockup's
name under `docs/assets/panel-redesign/evidence/`, rendered from the running
container with the e2e fixtures — not from the mockup;
- a review by a model that did not write the step
([development.md](../development.md) § Model routing): mockup and screenshot
side by side, every difference either fixed or listed for the owner.
The owner accepts a step by looking at the pairs. A step without them is not
reviewed.
## Out of scope
- New features beyond the four rows listed under Information architecture.
- Dark scheme and the narrow-screen shell: Bulma ships both; they are drawn
and agreed as a `design:` change before anyone implements them.
- Weakening CSP; a JS bundler; a front-end framework.
## Implementation checklist
No code until the roadmap status is **agreed**.
- [x] Owner decision recorded here: keep today's paths or rename with redirects — **owner** (2026-09-21: rename, § Routes)
- [x] Owner decision: no compatibility, migrations to zero, `2.0.0` — **owner** (2026-09-21, § No compatibility)
- [x] Owner decision: inbound is delegated per user, separately from outbound — **owner** (2026-09-21, § Who sees what)
- [ ] Vendor Bulma 1.0.4 and the used Tabler icons (MIT) under `/static`, NOTICE and development.md rows, pinned checksum — **Haiku**
- [ ] Guard tests from the table above and `TestRoutesFollowNavigation`, red against today's panel where expected; `design-first` CI step — **Opus**
- [ ] Schema from zero: single `0001_init.sql` baseline with `user_inbound_domains`, store tests, `schema-migrations.md` rewritten — **Opus**
- [ ] Inbound delegation: `requireInboundDomain`, filtered lists and log, two-list user form, per-route 404 tests for both roles — **Opus**
- [ ] Routes renamed per § Routes: new paths, one fragment name, old paths removed, subtree `requireGlobal()` for `/server/` — **Opus**
- [ ] `panel.css` from `panel/theme.css`; `components.html` partials; `status_tag`; a `/components` page behind the global role rendering every partial — **Sonnet**
- [ ] `layout.html`: navbar, perforated edge, sibling strip, user menu, footer; visibility flags — **Sonnet**
- [ ] Signed-out pages: login, setup — **Sonnet**
- [ ] Overview (verdicts) and Account / Settings split — **Sonnet**
- [ ] Outbound: domains, domain, domain settings, application form as one POST, shown-once password, delete — **Sonnet**
- [ ] Outbound: log, message, DMARC hub / domain / report — **Sonnet**
- [ ] Inbound: domains, domain, delete — **Sonnet**
- [ ] Server: system log, backup, users, user form, delete; Help — **Sonnet**
- [ ] Remove the old `panel.css` rules and classes nothing references; vocabulary test green with no allow-list — **Sonnet**
- [ ] Technical review of the whole against the mockups, evidence pairs complete — **Opus**
- [ ] Security review of the route rename, subtree authorization and inbound delegation — **Fable**
- [ ] guide.md screenshots and wording for the new navigation; architecture.md § Panel HTTP surface and § Persistence; security.md roles; *Removed* in the CHANGELOG with one line that 2.0 starts from an empty data directory — **Sonnet**
- [ ] `go vet`, `go test ./...`, e2e suite — **Haiku**
docs/plans/preflight.md
+77
View File
@@ -0,0 +1,77 @@
# Plan: preflight (installation check page)
**Status:** candidate
**Date:** 2026-08-19
**Version:** `1.x` MINOR; `candidate` until explicitly agreed.
---
## Goal
A dedicated `/preflight` page (global-admin only) that runs infrastructure-level
checks and helps the administrator confirm the instance is fully configured and
ready to deliver mail. Each failed check shows an actionable recommendation
(what to do or where to look).
## Mockup
[preflight](../assets/panel-redesign/panel/preflight.html), under the design
contract of [panel-redesign.md](panel-redesign.md): **Server › Preflight**,
path `/server/preflight` once the routes are renamed.
## Scope
**In:**
- rDNS / PTR verification (reuse `dnscheck.Checker.checkServer`)
- TLS certificate validity + hostname match (CN/SAN contains `SELFPOST_HOSTNAME`)
- Port connectivity: 25, 465, and 587 (if `SUBMISSION_ENABLE`) — dial own
public IP from inside the container, expect a 220 banner
- HELO/EHLO banner — connect to localhost:25, verify Postfix announces
`SELFPOST_HOSTNAME` (not `localhost` or a container ID)
- Trusted proxy headers:
- If `TRUSTED_PROXY_CIDR` set: verify peer IP is in range and
`X-Forwarded-For` is present
- If `PANEL_COOKIE_SECURE=true`: verify `X-Forwarded-Proto: https`
- If `TRUSTED_PROXY_CIDR` empty: informational warning (panel exposed
directly)
- DKIM signing — OpenDKIM milter socket reachable (reuse `health.CheckSocket`)
- Send test email — admin enters recipient address, system sends via
localhost:25, shows queued/error result
**Out:**
- Per-domain DNS checks (SPF, DKIM TXT, DMARC) — those live on domain detail
pages.
- Delivery tracking of test email (only confirms local MTA accepted).
## Done when
An administrator can open `/preflight`, see a traffic-light summary of all
infrastructure prerequisites, read specific fix instructions for any failures,
and send a test email to verify end-to-end flow.
## Risks
- Port-connectivity check from inside the container may not reflect external
reachability if NAT hairpin is unsupported (document limitation).
- Test email requires at least one configured domain (or falls back to
`postmaster@SELFPOST_HOSTNAME`).
## Implementation checklist
Target version cut: TBD (MINOR). One commit per step; code only after roadmap
status is **agreed**. See [development.md](../development.md) § Plan checklists.
- [ ] Port-connectivity helper (`internal/health/port.go`) — dial own IP, read
SMTP banner — **Sonnet**
- [ ] HELO banner check (connect localhost:25, parse 220 line) — **Sonnet**
- [ ] Hostname-in-certificate check (extend or wrap `health.CheckCertificate`)
— **Sonnet**
- [ ] Handler `handlers_preflight.go` (assemble all checks, actionable
recommendations) — **Sonnet**
- [ ] Test-email handler (`POST /preflight/test-email`, `net/smtp` to
localhost:25) — **Sonnet**
- [ ] Template `preflight.html` (traffic-light cards + test-email form) —
**Sonnet**
- [ ] Routes in `web.go`, nav link — **Sonnet**
docs/plans/queue-retries.md
-156
View File
@@ -1,156 +0,0 @@
# Plan: queue-retries (Postfix retry policy in the panel)
**Status:** done — shipped in `[1.3.1]` (2026-08-17)
**Date:** 2026-08-13
**Version:** patch; no schema, no configuration surface.
**Order:** small panel item; does not wait on inbound-relay.
---
## Goal
Show the operator how Postfix retries deferred mail: first retry delay, later
backoff cap, and how long a message stays in the queue before it bounces. The
numbers come from this container's effective Postfix config, not from
hard-coded copy.
## Scope
**In:**
- A static «How delivery retries work» card on `/mail-queue` (global
administrator). Not inside the HTMX poll fragment — the snapshot is taken at
panel start.
- The same human-readable intervals in `/deliveries/{id}` history for
`deferred` and `bounced` (domain administrators never see Mail queue).
- Operator docs: [guide.md](../guide.md) Mail queue bullet;
[architecture.md](../architecture.md) notes the one-shot `postconf -h` at
panel start. [CHANGELOG.md](../../CHANGELOG.md) `### Added`.
**Out:**
- Changing Postfix retry parameters, or exposing them as panel settings
([product.md](../product.md): Postfix is used as-is).
- An `attempts` column on `send_log`, or «attempt 3 of N» — Postfix has no
attempt budget; it is time-based.
- Counting `status=deferred` lines in `mail.log` or reading `postcat`. Each
attempt is already on the delivery page's log table; the journal stores only
the last status.
- Duplicating the card on Status (summary + link to Mail queue already exist).
- Re-reading `postconf` on every HTTP request.
## Architecture
Acceptance is still synchronous SMTP. Delivery stays in Postfix's on-disk
queue. SelfPost does not enqueue, retry, or deliver.
```mermaid
flowchart LR
smtpAccept["SMTP 250 accepted"] --> postfixQueue["Postfix queue"]
postfixQueue -->|"deferred: backoff until queue lifetime"| retry["Retry MX"]
retry -->|sent| done["sent"]
retry -->|lifetime elapsed or 5xx| bounce["bounced"]
postfixQueue --> mailQueuePage["/mail-queue policy card"]
postfixQueue --> deliveryPage["/deliveries/id history copy"]
```
[build/postfix-config.sh](../../build/postfix-config.sh) does not set
`queue_run_delay`, `minimal_backoff_time`, `maximal_backoff_time`,
`maximal_queue_lifetime`, `bounce_queue_lifetime`, or `delay_warning_time`.
Debian/Postfix 3.x compiled-in defaults therefore apply unless the operator
overrides them (`postconf -e` inside the container).
### Loading the numbers
Once, when the HTTP role starts ([cmd/panel/httpserver.go](../../cmd/panel/httpserver.go),
after `postfix-config.sh` has run):
```
postconf -h queue_run_delay minimal_backoff_time maximal_backoff_time maximal_queue_lifetime bounce_queue_lifetime delay_warning_time
```
`postconf -h`, not a parse of `/etc/postfix/main.cf`: stock values are not
written to the file. `postconf` is the effective config, including a manual
override.
- Fixed argv, no user input — same pattern as
[postfix.Queue](../../internal/postfix/queue.go) /
[security.md](../security.md).
- Cache on `handlers.Config` (via `web.Config`). The HTMX fragment does not
call `postconf`.
- A live `postconf -e` is visible after the next panel (or container) restart.
While the process is up, the panel shows the start-up snapshot.
- Parse Postfix time units (`300s`, `5d`, `1h`, a bare number is seconds) in
`internal/postfix`. Format human strings (`5 minutes`, `5 days`,
`about 1 hour 7 minutes`) in one place so the Mail queue card and
`deliveryEvents` cannot drift.
- If `postconf` is missing (unit tests on Windows, binary outside the
container): log a warning, fall back to Postfix 3.x compiled-in defaults
(`300s` / `4000s` / `5d` / `0`), and put a muted note on the card. Tests
stub the lookup (as `queueIDs` in the log-tailer) or pass a fixture on
`Config`. Do not fail panel start.
Typical stock values, for orientation only — the UI prints whatever
`postconf` returned:
| Parameter | Stock | Meaning |
|---|---|---|
| `queue_run_delay` / `minimal_backoff_time` | `300s` | First retry and deferred-queue scan |
| `maximal_backoff_time` | `4000s` | Cap on the doubling gap (~1 h 7 min) |
| `maximal_queue_lifetime` | `5d` | Then bounce |
| `delay_warning_time` | `0` | No delay warning to the sender |
## Panel copy
Mail queue card facts: first retry; later retries (doubling, capped);
kept in queue; then bounced. Short prose: there is no fixed attempt count; a
`deferred` message stays in this listing until it is delivered or the queue
lifetime runs out.
`deliveryEvents(row, policy)`:
- `deferred`: retries, first after X, then with increasing gaps up to Y, for
up to Z.
- `bounced`: or Postfix gave up after Z in the queue.
## Tests
- Duration parser: `5d`, `300s`, `4000s`, `1h`, `0`, bare number.
- `/mail-queue` handler: card shows the fixture policy's human strings, not a
live `postconf`.
- Delivery page / `deliveryEvents`: `deferred` and `bounced` contain those
strings ([handlers_monitor_test.go](../../internal/web/handlers/handlers_monitor_test.go)).
- [templates_test.go](../../internal/web/view/templates_test.go): pass the new
fields if rendering `mail_queue` requires them.
`go test` / `go vet` on the touched packages.
## Done when
- `/mail-queue` states this Postfix's first retry, backoff cap, and queue
lifetime.
- A `deferred` / `bounced` delivery page uses the same intervals.
- A manual `postconf -e maximal_queue_lifetime=2d` followed by a panel restart
changes what the panel prints.
- Guide and architecture describe the snapshot; CHANGELOG has an Added entry.
## Risks
- Showing compiled-in fallbacks when `postconf` failed would mislead if the
operator had overridden them — mitigate with the muted note on the card.
- Inventing a max-attempt count would be false; the copy must stay time-based.
## Implementation checklist
Target version cut: **`1.3.1`** (PATCH). One commit per step; see
[development.md](../development.md) § Plan checklists.
- [x] `internal/postfix`: parse Postfix time units (`5d`, `300s`, bare seconds) + tests — **Opus**
- [x] `internal/postfix`: one-shot `postconf -h` (six keys), fallback + warn — **Opus**
- [x] Load policy at HTTP start in `cmd/panel/httpserver.go`; cache on handlers config — **Opus**
- [x] Human-readable duration formatter (shared by Mail queue card and delivery history) — **Sonnet**
- [x] «How delivery retries work» card on `/mail-queue` (outside HTMX fragment) — **Sonnet**
- [x] `deliveryEvents(row, policy)` — intervals in deferred/bounced copy — **Sonnet**
- [x] Handler and template tests (`handlers_monitor_test.go`, `templates_test.go`) — **Sonnet**
- [x] [guide.md](../guide.md) and [architecture.md](../architecture.md) — **Sonnet**
- [x] `go vet`, `go test` on touched packages — **Haiku**
docs/plans/send-log-retention.md
-139
View File
@@ -1,139 +0,0 @@
# Plan: send-log-retention
**Status:** done — shipped in `[1.5.0]` (2026-08-18)
**Date:** 2026-08-17
**Version:** `1.5.0` MINOR; no schema migration required (uses existing `settings` table).
---
## Goal
Let the **global administrator** change how long delivery journal rows
(`send_log`, `/deliveries`) are kept, from the panel — without editing `.env`.
## Context (as-built)
Retention **already exists**, but only via environment:
- `SEND_LOG_RETENTION_DAYS` (default **90**) in `.env` / Compose.
- [`cmd/panel/main.go`](../../cmd/panel/main.go) passes it to
[`logtail.Run`](../../internal/logtail/logtail.go).
- [`retentionLoop`](../../internal/logtail/logtail.go) prunes via
[`DeleteSendLogBefore`](../../internal/store/sendlog.go) every **6 hours**.
- No panel control; [`handlers_monitor.go`](../../internal/web/handlers/handlers_monitor.go)
hardcodes «ninety days» in copy.
- Migration `0001_init.sql` describes `settings` as the place for «retention
overrides», but no UI writes that key yet.
This plan moves the **effective** retention into SQLite `settings`, with env as
bootstrap only.
## Scope
**In:**
- Settings card on `/settings` (global administrator only): **Send log
retention (days)**.
- Key `send_log_retention_days` in [`settings`](../../internal/store/settings.go).
- Validation: integer range **7–365** (exact bounds fixed at implementation).
- On first use: if setting missing, seed from env
(`SEND_LOG_RETENTION_DAYS`, default 90) at panel start or first save.
- Log-tailer reads the setting **each prune cycle** (no container restart).
- Delivery pages and guide copy show the **current** retention, not a hardcoded
90.
- Tests; [guide.md](../guide.md); [CHANGELOG.md](../../CHANGELOG.md).
**Out:**
- Per-domain retention (instance-wide only).
- `mail.log` rotation (logrotate, 14 daily files — unchanged).
- Immediate prune on save when lowering retention (next 6 h cycle is enough;
optional «Prune now» not in v1).
- Domain-admin access to this setting.
## Architecture
```mermaid
flowchart LR
settingsPage["/settings form"] --> sqlite["settings.send_log_retention_days"]
env["SEND_LOG_RETENTION_DAYS bootstrap"] --> sqlite
sqlite --> retentionLoop["logtail retentionLoop"]
retentionLoop --> prune["DeleteSendLogBefore"]
```
1. **Read path** — `GetSendLogRetentionDays()`: settings value if valid, else env
default.
2. **Write path** — POST `/settings` (global admin): validate, `SetSetting`,
flash confirmation.
3. **Prune path** — change [`logtail.retentionLoop`](../../internal/logtail/logtail.go)
to accept `func() int` or `RetentionReader` that queries settings each cycle
(same 6 h ticker).
4. **Copy** — inject retention days into delivery list/detail templates and
remove hardcoded «ninety days».
`SEND_LOG_RETENTION_DAYS` remains documented in [guide.md](../guide.md) as the
**initial default** until changed in Settings.
## Relation to domain-stats-auto-ratelimit
[domain-stats-auto-ratelimit](domain-stats-auto-ratelimit.md) uses a **30-day**
stats window. Requires effective retention ≥ 30 for full accuracy. When
retention &lt; 30:
- Stats UI shows a warning and uses `min(30, retention)` as the window, or
- Settings validation warns when saving a value below 30 while stats/auto are
enabled (pick one at implementation; document in guide).
Recommended roadmap order: **send-log-retention** before or parallel with
domain-stats-auto-ratelimit.
## Panel UI
New card on [`settings.html`](../../internal/web/view/templates/settings.html)
(global admin block, near rate limits or under a «Deliveries» heading):
- Number input: retention days (7–365).
- Muted copy: rows older than this are deleted from `/deliveries`; main driver
of `/data` growth; does not affect `mail.log` rotation.
Domain administrators keep the narrow credentials-only settings page.
## Tests
- Save/load setting; reject out-of-range values.
- `retentionLoop` uses updated value without process restart (mock reader).
- Bootstrap: empty settings → env default used for prune.
- Template/delivery copy reflects configured days.
`go test` / `go vet` on touched packages.
## Done when
- Global admin can set retention on `/settings`; value persists in SQLite.
- Prune uses the panel value on the next cycle; env remains bootstrap default.
- Guide documents panel vs env; CHANGELOG entry added.
- Hardcoded «ninety days» removed from delivery UI.
## Risks
- Operator lowers retention while bookmarking old delivery URLs — existing
behaviour; copy already notes pruned rows are gone.
- Settings change without restart — must be tested so log-tailer never keeps a
stale int from panel start only.
**Version:** `1.x` MINOR.
## Implementation checklist
Target version cut: **`1.5.0`** (MINOR). One commit per step; code only after
roadmap status is **agreed**. See [development.md](../development.md) § Plan
checklists.
- [x] `GetSendLogRetentionDays` / `SetSetting` key `send_log_retention_days` (7–365) — **Opus**
- [x] Bootstrap from `SEND_LOG_RETENTION_DAYS` when settings empty — **Opus**
- [x] `logtail.retentionLoop`: read setting each prune cycle — **Opus**
- [x] Settings card on `/settings` (`settings.html`) — **Sonnet**
- [x] Remove hardcoded «ninety days» in handlers and templates — **Sonnet**
- [x] Tests: save/load, range, loop without restart — **Sonnet**
- [x] [guide.md](../guide.md) — **Sonnet**
- [x] `go vet`, `go test` on touched packages — **Haiku**