selfpost/docs/plans/inbound-antispam-panel.md

# Plan: inbound-antispam-panel

**Status:** agreed  
**Date:** 2026-08-19  
**Version:** `2.1.0` MINOR (opt-in; no change to outbound-only or inbound-without-filter paths). Was `1.10.0`; on 2026-10-09 the owner put the panel redesign first, so this ships on the 2.0 baseline and its screens are built from the redesign's component kit ([panel-redesign.md](panel-redesign.md) § Component kit).

---

## Goal

Give the operator a **panel view of inbound anti-spam decisions** and **editable
allow/deny lists** that tune filter behaviour — without SelfPost shipping or
starting an anti-spam engine ([product.md](../product.md) out-of-scope list
unchanged).

Typical journal row:

| When | From | To | Subject | Decision |
|------|------|----|---------|----------|
| 2026-08-19 09:14 | `spammer@evil.example` | `user@inbound.example` | Cheap pills | reject (rspamd: BAYES_SPAM +0.95) |

Lists let the operator correct false positives/negatives (e.g. always allow
`billing@vendor.example`, block `192.0.2.0/24`).

## Context (as-built)

- Inbound relay ships in `[1.4.0]` (`INBOUND_RELAY_ENABLE`); optional
  `INBOUND_ANTISPAM_MILTER` attaches an **external** filter (documented rspamd
  sidecar in [deploy/antispam/docker-compose.antispam.yml](../../deploy/antispam/docker-compose.antispam.yml)).
- The inbound smtpd has **no journal-milter** today — only outbound 465/587
  writes `send_log` ([architecture.md](../architecture.md)).
- Inbound UI and handlers are **global-administrator only** today
  (`requireInbound` → `requireGlobal`).

## Quarantine

**Not in this plan.** Held-mail review/release is tracked separately as roadmap
candidate [inbound-quarantine](inbound-quarantine.md). The journal may still
record a `quarantine` **decision label** when the external filter reports it,
without SelfPost storing the message.

**DMARC `p=quarantine`** remains unrelated (outbound policy for receivers).

## Decisions (closed 2026-08-19)

### Capture mechanism

**Inbound journal-milter in `panel` only** — no rspamd history API polling in v1.
A separate unix socket (`INBOUND_JOURNAL_MILTER_SOCKET`, default
`/run/selfpost/inbound-journal.sock`) on the inbound smtpd only; fail-open
(`default_action=accept`).

**Milter order on port 25** (when antispam is enabled):

```text
smtpd_milters = { antispam, inbound-journal }
```

Antispam runs first; inbound-journal runs at end-of-message on the **accept**
path and reads rspamd-added headers (`X-Rspamd-Action`, and `X-Spamd-Result`
for the score and symbols - rspamd has no `X-Rspamd-Score`; the sidecar adds
both through `deploy/antispam/rspamd/milter_headers.conf`).

**Antispam rejects** do not reach end-of-message on the journal milter. v1 also
tails `mail.log` for inbound `postfix/smtpd` **`milter-reject`** / **`reject`**
lines and inserts journal rows (client IP, envelope from/to when logged, reason
text, subject empty if rejected before DATA). DMARC-ingest pipe traffic is out
of scope for this tailer.

### Journal without `INBOUND_ANTISPAM_MILTER`

Journal stays **on** whenever `INBOUND_RELAY_ENABLE=true` (not tied to the
antispam hook). Without an external filter:

- **accept** rows — message relayed (engine `selfpost`, detail empty);
- **reject** / **tempfail** rows — from the mail.log tailer (Postfix policy,
  size limit, unknown recipient, etc.).

Panel copy explains that rspamd symbols appear only when the antispam milter is
configured.

### RBAC

**Global administrator only** for journal and lists — same as inbound relay. No
domain-admin access in v1.

*As built (2026-10-11):* the lists are the global role's; the **log follows
the delegation of Inbound** that 2.0 introduced after this was written - a
user with the `domain` role sees the rows of the inbound domains assigned to
them and nothing else, without the *Allow / Deny sender* links. That is what
the accepted `in-log` mockup draws (no "global role" in its head). The owner
was told and can turn it back into a global-only page.

**Inbound › Log accepted by the owner on 2026-10-12** ("Этот экран
утверждаем"), from screenshots of the page built from its fixture, after his
changes to it: the sender column is headed *From*; *Why* is not a column but
a sheet over the page, opened by the decision tag, holding a centred table of
who decided, the verdict and symbols or the reply, and the subject whole
(kept on the sheet at his word: the table clips it); a message Postfix itself
refused offers no *Deny sender*. Inbound › Filter lists was redrawn at his word
the same day - kind in a column of its own, the note on the entry's sheet,
matches counted (§ Filter list usage) - and **accepted by him on 2026-10-12**
("Да"), after a deny entry he added in the panel refused a test message with
the list's reply and the journal recorded it.

### Retention

Mirror send-log retention:

- Settings key `inbound_spam_log_retention_days` on `/server/settings` (global admin).
- Default **90** days; validation **7–365**; seed from env
  `INBOUND_SPAM_LOG_RETENTION_DAYS` when the setting is missing.
- Prune every **6 hours** (same cadence as send-log retention).
- Hard cap **10 000** rows after age prune (drop oldest) — busier than DMARC
  reports, lighter than unbounded growth.

### Allow/deny lists

| Topic | Decision |
|-------|----------|
| Entry types | sender address, sender domain (`@domain` suffix), client IP / CIDR |
| Precedence | **deny overrides allow** |
| Scope | instance-wide (not per inbound domain in v1) |
| Where applied | **rspamd map files** under `/data/antispam/`, one per list and entry type (`allow_ip.map`, `allow_from.map`, `allow_domain.map`, `deny_…`) - an IP map and a sender map are different map types to rspamd; documented compose fragment mounts them into the sidecar. Deny is a prefilter reject, allow a weight of -100 (as built: two prefilters do not give "deny overrides allow") |
| SelfPost pre-check | **out of v1** — lists live in rspamd only |
| Edit access | global administrator only |
| When inactive | lists editable in the panel but UI warns that sync applies only when `INBOUND_ANTISPAM_MILTER` is set; `postfix reload` / rspamd reload documented in guide |

List change → atomic map rewrite (same pattern as Postfix maps). No reload:
rspamd rereads a changed map file by itself (about ten seconds on 3.11).

### Filter list usage (owner, 2026-10-12)

Each entry shows how often it decided a message (*Matched*, or *never*) and,
on its sheet, when it last did - so that entries nothing matches any more can
be removed. **The count is read off the journal on every view of the page;
nothing is stored for it.** A journal row counts for an entry when the list
decided the message - a rejected row whose reply is the deny rule's message,
an accepted row whose verdict carries the allow rule's symbol
(`SELFPOST_ALLOW_IP` / `_FROM` / `_DOMAIN`, which the journal milter always
keeps) - and the entry fits the row's sender or client address the way rspamd
matches it (`inbound.Lists.Usage`).

What follows from counting this way, and is accepted:

- the count reaches back as far as the journal does: its retention (90 days by
  default), which is what the sheet says;
- a row counts for every fitting entry of the deciding list, since the journal
  does not say which of two the filter met first;
- a message refused at end-of-message is one row whatever its recipient count,
  so a count is of messages, roughly.

**When to move to a counter kept with each entry** (a migration adding hits
and last-hit to `inbound_spam_list`, bumped where the journal row is written;
the owner asked for the condition to be written down). Any one of:

1. **The journal is cut by its 10 000-row cap and not by age.** Then it holds
   fewer days than its retention, the sheet's "in the last 90 days" is no
   longer true, and *never* stops being evidence that an entry is stale. The
   sign: the log tailer's prune line keeps reporting rows removed while the
   oldest row of Inbound › Log is younger than the retention.
2. **The retention is set short** (towards its minimum of 7 days) for reasons
   of its own, privacy or disk. A week without a match does not make an entry
   stale.
3. **Counts are wanted that outlive the journal** - since the entry was added.
4. **The page is slow.** A view reads every journal row a list decided and
   tries it against every entry. With the cap that is at most 10 000 rows
   times the number of entries; past a few hundred entries, measure it.

Until then no migration is spent on it.

### Activation

| Condition | Behaviour |
|-----------|-----------|
| `INBOUND_RELAY_ENABLE=false` | no inbound journal milter, no UI, no tables written |
| inbound on, antispam empty | journal + UI; lists visible but marked inactive for filter sync |
| inbound on + antispam set | full journal (symbols on accept) + list sync |

No separate feature flag beyond inbound relay + existing antispam env vars.

### UI

The journal is **Inbound › Log** and list management is a new **Inbound ›
Filter lists** page (global-only: the lists are instance-wide) — see
[panel-redesign.md](panel-redesign.md) § Room for roadmap items. The journal
table has filters (decision, domain, date range). Mockups are drawn first, in
`docs/assets/panel-redesign/panel/src/` under that plan's design contract —
not in the superseded `docs/assets/panel-ui/`.

## Schema (migration `0002_inbound_spam_log.sql`, the first after the 2.0 baseline)

**`inbound_spam_log`**

| Column | Type | Notes |
|--------|------|-------|
| `id` | INTEGER PK | |
| `inbound_domain` | TEXT NOT NULL | recipient domain (from `relay_domains`) |
| `client_ip` | TEXT NOT NULL | |
| `from_addr` | TEXT NOT NULL | envelope from |
| `to_addr` | TEXT NOT NULL | envelope to (one row per recipient) |
| `subject` | TEXT NOT NULL | truncated with `mailhdr.SubjectMaxRunes` (200) |
| `decision` | TEXT NOT NULL | `accept`, `reject`, `tempfail`, `quarantine` |
| `engine` | TEXT NOT NULL | `rspamd`, `postfix`, `selfpost` |
| `detail` | TEXT NOT NULL | symbol/score summary or log-line reason; may be empty |
| `created_at` | TEXT NOT NULL | RFC3339 UTC |

Index on `(created_at)`, `(inbound_domain, created_at)`.

**`inbound_spam_list`**

| Column | Type | Notes |
|--------|------|-------|
| `id` | INTEGER PK | |
| `list_type` | TEXT NOT NULL | `allow` or `deny` |
| `entry_type` | TEXT NOT NULL | `address`, `domain`, `ip` |
| `value` | TEXT NOT NULL | normalized entry |
| `note` | TEXT NOT NULL | operator comment; default `''` |
| `created_at` | TEXT NOT NULL | |

Unique on `(list_type, entry_type, value)`.

## Scope

**In:**

- Schema, store CRUD, retention loop, mail.log tailer extension for inbound
  rejects.
- Inbound journal-milter (fail-open) + Postfix `postfix-config.sh` wiring.
- rspamd map sync under `/data/antispam/`.
- Panel journal + list UI (global admin).
- Full backup includes `/data/antispam/` and new SQLite tables.
- Tests; [guide.md](../guide.md); [architecture.md](../architecture.md).

**Out:**

- Shipping rspamd, ClamAV, or any filter binary inside the SelfPost image.
- Outbound spam filtering.
- [inbound-quarantine.md](inbound-quarantine.md) storage/release.
- rspamd rule editing, Bayes training UI, antivirus.
- MIME replay from the panel (metadata only).
- Per-domain lists; domain-admin RBAC.
- rspamd history API polling.

## Architecture

```mermaid
flowchart LR
  internet["Internet :25"] --> inbound["Postfix inbound smtpd"]
  inbound --> antispam["External antispam milter optional"]
  antispam --> journalM["inbound-journal milter fail-open"]
  journalM --> upstream["smtp upstream transport"]
  journalM --> sqlite["inbound_spam_log"]
  mailog["mail.log tailer"] --> sqlite
  listsUI["Panel allow/deny lists"] --> maps["/data/antispam/*.map"]
  maps --> antispam
  sqlite --> journalUI["Panel inbound spam journal"]
```

## Security

- Same validation whitelists as inbound relay ([security.md](../security.md)).
- Journal rows contain PII — retention and global-admin-only access.
- Inbound journal milter fail-open must not block mail (backup-MX role).
- List map writes injection-safe; deny wins over allow.

## Done when

- With inbound relay + antispam sidecar, panel shows accept rows with symbols
  and reject rows from mail.log tailing.
- Operator can add/remove list entries; a test message reflects allow/deny.
- With antispam hook off, journal still shows Postfix-level decisions; list UI
  shows inactive sync notice.
- `go vet`, `go test`, image build green; guide and architecture updated.

## Risks

- rspamd header/symbol format drift — thin parser, pin sidecar tag in compose.
- Journal volume — retention + 10k cap.
- mail.log parse brittleness — unit tests on sample lines; inbound smtpd only.

## Dependencies

- Inbound relay, shipped in `[1.4.0]` — as built in
  [architecture.md](../architecture.md) § Mail path. Its plan file was removed
  once it shipped.

## Model routing (this plan)

| Role | Model |
|------|-------|
| All implementation (code, tests, docs) | **Composer** |
| Technical review (reviewer ≠ author) | **Opus** |
| Security review | **Fable** |

Composer ships each step; Opus reviews the accumulated diff once implementation
steps are done (fix loop: Composer addresses Opus findings, Opus re-checks).
Fable runs after Opus sign-off.

## Implementation checklist

Target version cut: **`2.1.0`** (MINOR), after `2.0.0`. One commit per step;
[development.md](../development.md) § Plan checklists.

- [x] Agree journal fields, retention, and RBAC (this plan § Decisions) — **Composer**
- [x] Migration `0002_inbound_spam_log.sql` on the 2.0 baseline — **Composer**
- [x] Inbound journal-milter + Postfix wiring — **Composer**
- [x] mail.log tailer: inbound reject rows — **Composer**
- [x] List CRUD + validation + atomic rspamd map sync — **Composer**
- [x] Panel: Inbound › Log and Inbound › Filter lists from the kit partials, mockups `in-log` / `in-filter-lists` (add client IP and engine columns mockup-first), evidence pairs — **Composer**
- [x] Backup includes `/data/antispam/` — **Composer** (it always did: a backup walks all of `/data`; now held by a test, and the guide says the `antispam/` fragment directory is not in it)
- [x] Unit + handler tests — **Composer** (written with each step: store, milter, tailer, lists and their usage, view fixtures held to the outlines, handlers for both roles)
- [x] [guide.md](../guide.md), [architecture.md](../architecture.md) — **Composer** (kept up with each step; [security.md](../security.md) follows the security review)
- [x] `go vet`, `go test`, e2e if applicable — **Composer** (on the test server for every step; last on `7192c38`: vet ok, 22 packages ok, e2e ok, which relays, refuses, reads the log page and edits a list through the panel)
- [ ] Technical review of inbound antispam changes — **Opus**
- [ ] Security review inbound antispam path — **Fable**