selfpost/docs/plans/inbound-antispam-panel.md

Plan: inbound-antispam-panel

Status: shipped in 2.1.0 (2026-10-12)
Date: 2026-08-19
Version: 2.1.0 MINOR (opt-in; no change to outbound-only or inbound-without-filter paths). Was 1.10.0; on 2026-10-09 the owner put the panel redesign first, so this ships on the 2.0 baseline and its screens are built from the redesign's component kit (panel-redesign.md § Component kit).

---

Goal

Give the operator a panel view of inbound anti-spam decisions and editable allow/deny lists that tune filter behaviour — without SelfPost shipping or starting an anti-spam engine (product.md out-of-scope list unchanged).

Typical journal row:

When	From	To	Subject	Decision
2026-08-19 09:14	spammer@evil.example	user@inbound.example	Cheap pills	reject (rspamd: BAYES_SPAM +0.95)

Lists let the operator correct false positives/negatives (e.g. always allow billing@vendor.example, block 192.0.2.0/24).

Context (as-built)

• Inbound relay ships in [1.4.0] (INBOUND_RELAY_ENABLE); optional INBOUND_ANTISPAM_MILTER attaches an external filter (documented rspamd sidecar in deploy/antispam/docker-compose.antispam.yml).
• The inbound smtpd has no journal-milter today — only outbound 465/587 writes send_log (architecture.md).
• Inbound UI and handlers are global-administrator only today (requireInbound → requireGlobal).

Quarantine

Not in this plan. Held-mail review/release is an idea kept outside the roadmap, inbound-quarantine. The journal may still record a quarantine decision label when the external filter reports it, without SelfPost storing the message.

DMARC p=quarantine remains unrelated (outbound policy for receivers).

Decisions (closed 2026-08-19)

Capture mechanism

Inbound journal-milter in panel only — no rspamd history API polling in v1. A separate unix socket (INBOUND_JOURNAL_MILTER_SOCKET, default /run/selfpost/inbound-journal.sock) on the inbound smtpd only; fail-open (default_action=accept).

Milter order on port 25 (when antispam is enabled):

smtpd_milters = { antispam, inbound-journal }

Antispam runs first; inbound-journal runs at end-of-message on the accept path and reads rspamd-added headers (X-Rspamd-Action, and X-Spamd-Result for the score and symbols - rspamd has no X-Rspamd-Score; the sidecar adds both through deploy/antispam/rspamd/milter_headers.conf). As built they are read only next to rspamd's signature of the message, X-SelfPost-Filter (owner, 2026-10-12: "А rspamd не может дописывать какой-то заголовок, по которому мы будем знать, что это заголовки нашего спам фильтра?") - an HMAC of the queue id, since a fixed mark could be written by a sender as easily as the verdict itself.

Antispam rejects do not reach end-of-message on the journal milter. v1 also tails mail.log for inbound postfix/smtpd milter-reject / reject lines and inserts journal rows (client IP, envelope from/to when logged, reason text, subject empty if rejected before DATA). DMARC-ingest pipe traffic is out of scope for this tailer.

Journal without INBOUND_ANTISPAM_MILTER

Journal stays on whenever INBOUND_RELAY_ENABLE=true (not tied to the antispam hook). Without an external filter:

• accept rows — message relayed (engine selfpost, detail empty);
• reject / tempfail rows — from the mail.log tailer (Postfix policy, unknown recipient, a sender domain that does not exist. As built, a refusal whose log line names no recipient - the size limit answered at MAIL FROM - and the per-client rate limit, which Postfix logs as a warning, make no row).

Panel copy explains that rspamd symbols appear only when the antispam milter is configured.

RBAC

Global administrator only for journal and lists — same as inbound relay. No domain-admin access in v1.

As built (2026-10-11): the lists are the global role's; the log follows the delegation of Inbound that 2.0 introduced after this was written - a user with the domain role sees the rows of the inbound domains assigned to them and nothing else, without the Allow / Deny sender links. That is what the accepted in-log mockup draws (no "global role" in its head). The owner was told and can turn it back into a global-only page.

Inbound › Log accepted by the owner on 2026-10-12 ("Этот экран утверждаем"), from screenshots of the page built from its fixture, after his changes to it: the sender column is headed From; Why is not a column but a sheet over the page, opened by the decision tag, holding a centred table of who decided, the verdict and symbols or the reply, and the subject whole (kept on the sheet at his word: the table clips it); a message Postfix itself refused offers no Deny sender. Inbound › Filter lists was redrawn at his word the same day - kind in a column of its own, the note on the entry's sheet, matches counted (§ Filter list usage) - and accepted by him on 2026-10-12 ("Да"), after a deny entry he added in the panel refused a test message with the list's reply and the journal recorded it.

Retention

Mirror send-log retention:

• Settings key inbound_spam_log_retention_days on /server/settings (global admin).
• Default 90 days; validation 7–365; seed from env INBOUND_SPAM_LOG_RETENTION_DAYS when the setting is missing.
• Prune every 6 hours (same cadence as send-log retention).
• Hard cap 10 000 rows after age prune (drop oldest) — busier than DMARC reports, lighter than unbounded growth.

Allow/deny lists

Topic	Decision
Entry types	sender address, sender domain (@domain suffix), client IP / CIDR
Precedence	deny overrides allow
Scope	instance-wide (not per inbound domain in v1)
Where applied	rspamd map files under /data/antispam/, one per list and entry type (allow_ip.map, allow_from.map, allow_domain.map, deny_…) - an IP map and a sender map are different map types to rspamd; documented compose fragment mounts them into the sidecar. Deny is a prefilter reject, allow a weight of -100 (as built: two prefilters do not give "deny overrides allow")
SelfPost pre-check	out of v1 — lists live in rspamd only
Edit access	global administrator only
When inactive	lists editable in the panel but UI warns that sync applies only when INBOUND_ANTISPAM_MILTER is set; postfix reload / rspamd reload documented in guide

List change → atomic map rewrite (same pattern as Postfix maps). No reload: rspamd rereads a changed map file by itself (about ten seconds on 3.11).

Filter list usage (owner, 2026-10-12)

Each entry shows how often it decided a message (Matched, or never) and, on its sheet, when it last did - so that entries nothing matches any more can be removed. The count is read off the journal on every view of the page; nothing is stored for it. A journal row counts for an entry when the list decided the message - a rejected row whose reply is the deny rule's message, an accepted row whose verdict carries the allow rule's symbol (SELFPOST_ALLOW_IP / _FROM / _DOMAIN, which the journal milter always keeps) - and the entry fits the row's sender or client address the way rspamd matches it (inbound.Lists.Usage).

What follows from counting this way, and is accepted:

• the count reaches back as far as the journal does: its retention (90 days by default), which is what the sheet says;
• a row counts for every fitting entry of the deciding list, since the journal does not say which of two the filter met first;
• a message refused at end-of-message is one row whatever its recipient count, so a count is of messages, roughly.

When to move to a counter kept with each entry (a migration adding hits and last-hit to inbound_spam_list, bumped where the journal row is written; the owner asked for the condition to be written down). Any one of:

1. The journal is cut by its 10 000-row cap and not by age. Then it holds fewer days than its retention, the sheet's "in the last 90 days" is no longer true, and never stops being evidence that an entry is stale. The sign: the log tailer's prune line keeps reporting rows removed while the oldest row of Inbound › Log is younger than the retention.
2. The retention is set short (towards its minimum of 7 days) for reasons of its own, privacy or disk. A week without a match does not make an entry stale.
3. Counts are wanted that outlive the journal - since the entry was added.
4. The page is slow. A view reads every journal row a list decided and tries it against every entry. With the cap that is at most 10 000 rows times the number of entries; past a few hundred entries, measure it.

Until then no migration is spent on it.

Activation

Condition	Behaviour
INBOUND_RELAY_ENABLE=false	no inbound journal milter, no UI, no tables written
inbound on, antispam empty	journal + UI; lists visible but marked inactive for filter sync
inbound on + antispam set	full journal (symbols on accept) + list sync

No separate feature flag beyond inbound relay + existing antispam env vars.

UI

The journal is Inbound › Log and list management is a new Inbound › Filter lists page (global-only: the lists are instance-wide) — see panel-redesign.md § Room for roadmap items. The journal table has filters (decision, domain, date range). Mockups are drawn first, in docs/assets/panel-redesign/panel/src/ under that plan's design contract — not in the superseded docs/assets/panel-ui/ (removed 2026-10-16).

Schema (migration 0002_inbound_spam_log.sql, the first after the 2.0 baseline)

inbound_spam_log

Column	Type	Notes
id	INTEGER PK
inbound_domain	TEXT NOT NULL	recipient domain (from relay_domains)
client_ip	TEXT NOT NULL
from_addr	TEXT NOT NULL	envelope from
to_addr	TEXT NOT NULL	envelope to (one row per recipient)
subject	TEXT NOT NULL	truncated with mailhdr.SubjectMaxRunes (200)
decision	TEXT NOT NULL	accept, reject, tempfail, quarantine
engine	TEXT NOT NULL	rspamd, postfix, selfpost
detail	TEXT NOT NULL	symbol/score summary or log-line reason; may be empty
created_at	TEXT NOT NULL	RFC3339 UTC

Index on (created_at), (inbound_domain, created_at).

inbound_spam_list

Column	Type	Notes
id	INTEGER PK
list_type	TEXT NOT NULL	allow or deny
entry_type	TEXT NOT NULL	address, domain, ip
value	TEXT NOT NULL	normalized entry
note	TEXT NOT NULL	operator comment; default ''
created_at	TEXT NOT NULL

Unique on (list_type, entry_type, value).

Scope

In:

• Schema, store CRUD, retention loop, mail.log tailer extension for inbound rejects.
• Inbound journal-milter (fail-open) + Postfix postfix-config.sh wiring.
• rspamd map sync under /data/antispam/.
• Panel journal + list UI (global admin).
• Full backup includes /data/antispam/ and new SQLite tables.
• Tests; guide.md; architecture.md.

Out:

• Shipping rspamd, ClamAV, or any filter binary inside the SelfPost image.
• Outbound spam filtering.
• inbound-quarantine.md storage/release.
• rspamd rule editing, Bayes training UI, antivirus.
• MIME replay from the panel (metadata only).
• Per-domain lists; domain-admin RBAC.
• rspamd history API polling.

Architecture

flowchart LR
  internet["Internet :25"] --> inbound["Postfix inbound smtpd"]
  inbound --> antispam["External antispam milter optional"]
  antispam --> journalM["inbound-journal milter fail-open"]
  journalM --> upstream["smtp upstream transport"]
  journalM --> sqlite["inbound_spam_log"]
  mailog["mail.log tailer"] --> sqlite
  listsUI["Panel allow/deny lists"] --> maps["/data/antispam/*.map"]
  maps --> antispam
  sqlite --> journalUI["Panel inbound spam journal"]

Security

• Same validation whitelists as inbound relay (security.md).
• Journal rows contain PII — retention, and access by reach: the global role reads all of it, a domain user the rows of assigned inbound domains (as built; see § RBAC).
• Inbound journal milter fail-open must not block mail (backup-MX role).
• List map writes injection-safe; deny wins over allow.

Done when

• With inbound relay + antispam sidecar, panel shows accept rows with symbols and reject rows from mail.log tailing.
• Operator can add/remove list entries; a test message reflects allow/deny.
• With antispam hook off, journal still shows Postfix-level decisions; list UI shows inactive sync notice.
• go vet, go test, image build green; guide and architecture updated.

Risks

• rspamd header/symbol format drift — thin parser, pin sidecar tag in compose.
• Journal volume — retention + 10k cap.
• mail.log parse brittleness — unit tests on sample lines; inbound smtpd only.

Dependencies

• Inbound relay, shipped in [1.4.0] — as built in architecture.md § Mail path. Its plan file was removed once it shipped.

Model routing (this plan)

Role	Model
All implementation (code, tests, docs)	Composer
Technical review (reviewer ≠ author)	Opus
Security review	Fable

Composer ships each step; Opus reviews the accumulated diff once implementation steps are done (fix loop: Composer addresses Opus findings, Opus re-checks). Fable runs after Opus sign-off.

Implementation checklist

Target version cut: 2.1.0 (MINOR), after 2.0.0. One commit per step; development.md § Plan checklists.

• Agree journal fields, retention, and RBAC (this plan § Decisions) — Composer
• Migration 0002_inbound_spam_log.sql on the 2.0 baseline — Composer
• Inbound journal-milter + Postfix wiring — Composer
• mail.log tailer: inbound reject rows — Composer
• List CRUD + validation + atomic rspamd map sync — Composer
• Panel: Inbound › Log and Inbound › Filter lists from the kit partials, mockups in-log / in-filter-lists (add client IP and engine columns mockup-first), evidence pairs — Composer
• Backup includes /data/antispam/ — Composer (it always did: a backup walks all of /data; now held by a test, and the guide says the antispam/ fragment directory is not in it)
• Unit + handler tests — Composer (written with each step: store, milter, tailer, lists and their usage, view fixtures held to the outlines, handlers for both roles)
• guide.md, architecture.md — Composer (kept up with each step; security.md follows the security review)
• go vet, go test, e2e if applicable — Composer (on the test server for every step; last on 7192c38: vet ok, 22 packages ok, e2e ok, which relays, refuses, reads the log page and edits a list through the panel)
• Technical review of inbound antispam changes — Opus (2026-10-12; findings fixed: client address of a connection's later messages, the lists' writer under a lock, the tailer's queue-id memory, hold rows written twice, the cap by time; and, outside the feature, the level-2 limit and client-address list skipped after a connection's first message)
• Security review inbound antispam path — Fable (2026-10-12; no Critical or High; HELO-forged journal rows and the six-hourly row cap fixed; a sender's own verdict while the filter is down closed by the filter's signature, the owner's idea; accepted risks recorded in security.md)