docs: agree inbound antispam panel plan for 1.10.0.
test / test (push) Waiting to run

Add journal and allow/deny list design, quarantine candidate, roadmap entries, and Composer to Opus to Fable routing note.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
2026-08-19 00:32:03 +03:00
parent bb26e5022e
commit 1e4623b6e1
5 changed files with 390 additions and 0 deletions
+16
View File
@@ -7,6 +7,22 @@ Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); version
### Added
- [docs/plans/inbound-antispam-panel.md](docs/plans/inbound-antispam-panel.md) —
agreed plan for inbound antispam journal (inbound-journal milter + mail.log
rejects) and instance-wide allow/deny lists synced to rspamd maps; target
`1.10.0`; Composer → Opus → Fable workflow.
- [docs/plans/inbound-quarantine.md](docs/plans/inbound-quarantine.md) — candidate
plan for held-mail review/release (design TBD).
### Changed
- [docs/roadmap.md](docs/roadmap.md) — `inbound-antispam-panel` (agreed, 1/12)
and `inbound-quarantine` (candidate) in the index.
- [docs/development.md](docs/development.md) — plan checklists may override
model routing with Composer → Opus → Fable.
### Added
- [docs/schema-migrations.md](docs/schema-migrations.md) — living reference for the
SQLite migration chain (`user_version` head, per-file history, legacy artefacts,
1.x rules, and planned 2.x squash gate).
+5
View File
@@ -64,6 +64,11 @@ obligation — not because the convention around AI authorship shifts.
Default rule: risk-critical → Opus; UI / docs / boilerplate → Sonnet; trivial
mechanics → Haiku. Reviewers must not be the author of the code under review.
**Composer → Opus → Fable** (when a plan specifies it): **Composer** writes all
implementation for that item (code, tests, docs); **Opus** does the technical
review of the result; **Fable** does the security review. A plan's checklist
overrides the default routing table for that item only.
---
## Plan checklists
+251
View File
@@ -0,0 +1,251 @@
# Plan: inbound-antispam-panel
**Status:** agreed
**Date:** 2026-08-19
**Version:** `1.10.0` MINOR (opt-in; no change to outbound-only or inbound-without-filter paths).
---
## Goal
Give the operator a **panel view of inbound anti-spam decisions** and **editable
allow/deny lists** that tune filter behaviour — without SelfPost shipping or
starting an anti-spam engine ([product.md](../product.md) out-of-scope list
unchanged).
Typical journal row:
| When | From | To | Subject | Decision |
|------|------|----|---------|----------|
| 2026-08-19 09:14 | `spammer@evil.example` | `user@inbound.example` | Cheap pills | reject (rspamd: BAYES_SPAM +0.95) |
Lists let the operator correct false positives/negatives (e.g. always allow
`billing@vendor.example`, block `192.0.2.0/24`).
## Context (as-built)
- Inbound relay ships in `[1.4.0]` (`INBOUND_RELAY_ENABLE`); optional
`INBOUND_ANTISPAM_MILTER` attaches an **external** filter (documented rspamd
sidecar in [deploy/antispam/docker-compose.antispam.yml](../../deploy/antispam/docker-compose.antispam.yml)).
- The inbound smtpd has **no journal-milter** today — only outbound 465/587
writes `send_log` ([architecture.md](../architecture.md)).
- Inbound UI and handlers are **global-administrator only** today
(`requireInbound``requireGlobal`).
## Quarantine
**Not in this plan.** Held-mail review/release is tracked separately as roadmap
candidate [inbound-quarantine](inbound-quarantine.md). The journal may still
record a `quarantine` **decision label** when the external filter reports it,
without SelfPost storing the message.
**DMARC `p=quarantine`** remains unrelated (outbound policy for receivers).
## Decisions (closed 2026-08-19)
### Capture mechanism
**Inbound journal-milter in `panel` only** — no rspamd history API polling in v1.
A separate unix socket (`INBOUND_JOURNAL_MILTER_SOCKET`, default
`/run/selfpost/inbound-journal.sock`) on the inbound smtpd only; fail-open
(`default_action=accept`).
**Milter order on port 25** (when antispam is enabled):
```text
smtpd_milters = { antispam, inbound-journal }
```
Antispam runs first; inbound-journal runs at end-of-message on the **accept**
path and reads rspamd-added headers (`X-Rspamd-Action`, `X-Rspamd-Score`, symbol
summary when present).
**Antispam rejects** do not reach end-of-message on the journal milter. v1 also
tails `mail.log` for inbound `postfix/smtpd` **`milter-reject`** / **`reject`**
lines and inserts journal rows (client IP, envelope from/to when logged, reason
text, subject empty if rejected before DATA). DMARC-ingest pipe traffic is out
of scope for this tailer.
### Journal without `INBOUND_ANTISPAM_MILTER`
Journal stays **on** whenever `INBOUND_RELAY_ENABLE=true` (not tied to the
antispam hook). Without an external filter:
- **accept** rows — message relayed (engine `selfpost`, detail empty);
- **reject** / **tempfail** rows — from the mail.log tailer (Postfix policy,
size limit, unknown recipient, etc.).
Panel copy explains that rspamd symbols appear only when the antispam milter is
configured.
### RBAC
**Global administrator only** for journal and lists — same as inbound relay. No
domain-admin access in v1.
### Retention
Mirror send-log retention:
- Settings key `inbound_spam_log_retention_days` on `/settings` (global admin).
- Default **90** days; validation **7365**; seed from env
`INBOUND_SPAM_LOG_RETENTION_DAYS` when the setting is missing.
- Prune every **6 hours** (same cadence as send-log retention).
- Hard cap **10 000** rows after age prune (drop oldest) — busier than DMARC
reports, lighter than unbounded growth.
### Allow/deny lists
| Topic | Decision |
|-------|----------|
| Entry types | sender address, sender domain (`@domain` suffix), client IP / CIDR |
| Precedence | **deny overrides allow** |
| Scope | instance-wide (not per inbound domain in v1) |
| Where applied | **rspamd map files** under `/data/antispam/` (allow.map, deny.map); documented compose fragment mounts them into the sidecar |
| SelfPost pre-check | **out of v1** — lists live in rspamd only |
| Edit access | global administrator only |
| When inactive | lists editable in the panel but UI warns that sync applies only when `INBOUND_ANTISPAM_MILTER` is set; `postfix reload` / rspamd reload documented in guide |
List change → atomic map rewrite (same pattern as Postfix maps) → rspamd reload
via documented operator step or sidecar `SIGHUP` in the compose fragment.
### Activation
| Condition | Behaviour |
|-----------|-----------|
| `INBOUND_RELAY_ENABLE=false` | no inbound journal milter, no UI, no tables written |
| inbound on, antispam empty | journal + UI; lists visible but marked inactive for filter sync |
| inbound on + antispam set | full journal (symbols on accept) + list sync |
No separate feature flag beyond inbound relay + existing antispam env vars.
### UI
New nav item **Inbound spam** (or subsection under **Inbound**) — journal table
with filters (decision, domain, date range), list management on the same page or
a tab. Mockup: add `docs/assets/panel-ui/inbound_spam.html` in the panel step.
## Schema (migration `0010_inbound_spam_log.sql`)
**`inbound_spam_log`**
| Column | Type | Notes |
|--------|------|-------|
| `id` | INTEGER PK | |
| `inbound_domain` | TEXT NOT NULL | recipient domain (from `relay_domains`) |
| `client_ip` | TEXT NOT NULL | |
| `from_addr` | TEXT NOT NULL | envelope from |
| `to_addr` | TEXT NOT NULL | envelope to (one row per recipient) |
| `subject` | TEXT NOT NULL | truncated with `mailhdr.SubjectMaxRunes` (200) |
| `decision` | TEXT NOT NULL | `accept`, `reject`, `tempfail`, `quarantine` |
| `engine` | TEXT NOT NULL | `rspamd`, `postfix`, `selfpost` |
| `detail` | TEXT NOT NULL | symbol/score summary or log-line reason; may be empty |
| `created_at` | TEXT NOT NULL | RFC3339 UTC |
Index on `(created_at)`, `(inbound_domain, created_at)`.
**`inbound_spam_list`**
| Column | Type | Notes |
|--------|------|-------|
| `id` | INTEGER PK | |
| `list_type` | TEXT NOT NULL | `allow` or `deny` |
| `entry_type` | TEXT NOT NULL | `address`, `domain`, `ip` |
| `value` | TEXT NOT NULL | normalized entry |
| `note` | TEXT NOT NULL | operator comment; default `''` |
| `created_at` | TEXT NOT NULL | |
Unique on `(list_type, entry_type, value)`.
## Scope
**In:**
- Schema, store CRUD, retention loop, mail.log tailer extension for inbound
rejects.
- Inbound journal-milter (fail-open) + Postfix `postfix-config.sh` wiring.
- rspamd map sync under `/data/antispam/`.
- Panel journal + list UI (global admin).
- Full backup includes `/data/antispam/` and new SQLite tables.
- Tests; [guide.md](../guide.md); [architecture.md](../architecture.md).
**Out:**
- Shipping rspamd, ClamAV, or any filter binary inside the SelfPost image.
- Outbound spam filtering.
- [inbound-quarantine.md](inbound-quarantine.md) storage/release.
- rspamd rule editing, Bayes training UI, antivirus.
- MIME replay from the panel (metadata only).
- Per-domain lists; domain-admin RBAC.
- rspamd history API polling.
## Architecture
```mermaid
flowchart LR
internet["Internet :25"] --> inbound["Postfix inbound smtpd"]
inbound --> antispam["External antispam milter optional"]
antispam --> journalM["inbound-journal milter fail-open"]
journalM --> upstream["smtp upstream transport"]
journalM --> sqlite["inbound_spam_log"]
mailog["mail.log tailer"] --> sqlite
listsUI["Panel allow/deny lists"] --> maps["/data/antispam/*.map"]
maps --> antispam
sqlite --> journalUI["Panel inbound spam journal"]
```
## Security
- Same validation whitelists as inbound relay ([security.md](../security.md)).
- Journal rows contain PII — retention and global-admin-only access.
- Inbound journal milter fail-open must not block mail (backup-MX role).
- List map writes injection-safe; deny wins over allow.
## Done when
- With inbound relay + antispam sidecar, panel shows accept rows with symbols
and reject rows from mail.log tailing.
- Operator can add/remove list entries; a test message reflects allow/deny.
- With antispam hook off, journal still shows Postfix-level decisions; list UI
shows inactive sync notice.
- `go vet`, `go test`, image build green; guide and architecture updated.
## Risks
- rspamd header/symbol format drift — thin parser, pin sidecar tag in compose.
- Journal volume — retention + 10k cap.
- mail.log parse brittleness — unit tests on sample lines; inbound smtpd only.
## Dependencies
- [inbound-relay.md](inbound-relay.md) (shipped).
## Model routing (this plan)
| Role | Model |
|------|-------|
| All implementation (code, tests, docs) | **Composer** |
| Technical review (reviewer ≠ author) | **Opus** |
| Security review | **Fable** |
Composer ships each step; Opus reviews the accumulated diff once implementation
steps are done (fix loop: Composer addresses Opus findings, Opus re-checks).
Fable runs after Opus sign-off.
## Implementation checklist
Target version cut: **`1.10.0`** (MINOR). One commit per step;
[development.md](../development.md) § Plan checklists.
- [x] Agree journal fields, retention, and RBAC (this plan § Decisions) — **Composer**
- [ ] Migration `0010_inbound_spam_log.sql`**Composer**
- [ ] Inbound journal-milter + Postfix wiring — **Composer**
- [ ] mail.log tailer: inbound reject rows — **Composer**
- [ ] List CRUD + validation + atomic rspamd map sync — **Composer**
- [ ] Panel: journal + lists UI (+ mockup) — **Composer**
- [ ] Backup includes `/data/antispam/`**Composer**
- [ ] Unit + handler tests — **Composer**
- [ ] [guide.md](../guide.md), [architecture.md](../architecture.md) — **Composer**
- [ ] `go vet`, `go test`, e2e if applicable — **Composer**
- [ ] Technical review of inbound antispam changes — **Opus**
- [ ] Security review inbound antispam path — **Fable**
+72
View File
@@ -0,0 +1,72 @@
# Plan: inbound-quarantine
**Status:** candidate — design TBD; needs explicit agreement before any code
**Version:** TBD (`1.x` MINOR if opt-in and backward-compatible; may warrant `2.x`
if it changes inbound storage or backup contract).
---
## Goal (draft)
Let the operator **hold suspicious inbound mail** instead of only accept/reject
at the milter — review in the panel, then release to the configured upstream,
delete, or add sender to allow/deny lists.
This is a **product boundary question**, not a settled design. The item exists
so we can compare approaches before committing.
## Why it is separate from inbound-antispam-panel
[inbound-antispam-panel.md](inbound-antispam-panel.md) covers **metadata**
(journal + lists). Quarantine implies **storing message bodies** (or a durable
reference to them) under `/data`, retention, release workflow, and a wider
attack/backup surface — closer to "mini-mailbox" than pure relay.
## Open questions (to decide before a checklist)
1. **Where mail lives**
- SelfPost store under `/data/quarantine/` (panel owns lifecycle), or
- delegate to rspamd/Redis and panel only proxies release (less storage in
SelfPost, tighter rspamd coupling).
2. **What "release" means**
- inject into Postfix as a new delivery to the domain's upstream transport;
- or manual download only (no automatic forward).
3. **Retention and caps**
- max age, max total bytes, per-domain limits; interaction with backup size.
4. **RBAC**
- global admin only vs domain-admin for assigned inbound domains.
5. **Threat model**
- malware in stored MIME, path traversal on extract, quota DoS on busy MX.
6. **Product fit**
- does this violate [product.md](../product.md) "no mailboxes" if we only
hold spam suspects briefly? Explicit product decision required.
## Context (as-built)
- Inbound relay forwards or rejects; no local delivery ([inbound-relay.md](inbound-relay.md)).
- Optional `INBOUND_ANTISPAM_MILTER` — external rspamd may quarantine on its
own today, without panel integration.
- DMARC `p=quarantine` is unrelated (outbound policy for receivers).
## Likely out of scope (until revisited)
- IMAP/webmail for quarantined mail.
- End-user self-service quarantine (non-admin recipients).
- Outbound quarantine.
## Dependencies
- [inbound-relay.md](inbound-relay.md) (shipped).
- Sensible to decide **after** or **alongside**
[inbound-antispam-panel.md](inbound-antispam-panel.md) (journal/lists), but
not blocked on it for design discussion.
## Done when (placeholder)
TBD once approach is chosen. Minimum bar would include: opt-in flag, panel
list/detail, release or discard action, retention, backup inclusion documented,
security review.
## Implementation checklist
None yet — expand this plan after the open questions above are answered.
+46
View File
@@ -30,6 +30,8 @@ in `git log` and [CHANGELOG.md](../CHANGELOG.md).
|---|---|---|---|---|
| contributing | `CONTRIBUTING.md` | candidate | — | — |
| panel-docs | In-panel operator documentation | agreed | 6/6 | [plans/panel-docs.md](plans/panel-docs.md) |
| inbound-antispam-panel | Inbound antispam journal + allow/deny lists | agreed | 1/12 | [plans/inbound-antispam-panel.md](plans/inbound-antispam-panel.md) |
| inbound-quarantine | Inbound spam quarantine (hold / review / release) | candidate | — | [plans/inbound-quarantine.md](plans/inbound-quarantine.md) |
| schema-squash | Squash SQLite migrations into a 2.x baseline | **2.x** | — | — |
**Recommended order** (not binding): the next feature is **panel-docs** once
@@ -89,6 +91,50 @@ when checks change; not bloating every page with a second column of prose.
---
## inbound-antispam-panel
**Goal:** panel view of inbound anti-spam decisions (when, from, to, subject,
action) plus editable allow/deny lists that tune the external filter — without
shipping rspamd or another engine inside the image.
**Boundary:** metadata journal and list sync only; SelfPost does not run the
filter ([product.md](product.md)). Quarantine:
[inbound-quarantine](#inbound-quarantine). Design closed in
[plans/inbound-antispam-panel.md](plans/inbound-antispam-panel.md) § Decisions.
**Done when:** with inbound relay enabled, the operator sees journal rows
(accepts via inbound-journal milter, rejects via mail.log tailer) and can
maintain instance-wide allow/deny lists synced to rspamd maps when the antispam
hook is set.
**Dependencies / risks:** inbound relay `[1.4.0]`; rspamd map/header coupling;
journal PII; global-admin-only RBAC.
**Version:** `1.10.0` MINOR; **agreed**.
---
## inbound-quarantine
**Goal:** optional hold for suspicious inbound mail — review in the panel, then
release to upstream, discard, or tune lists. Design not settled.
**Boundary:** expands inbound from pure relay toward short-lived message
storage; may conflict with "no mailboxes" in [product.md](product.md) until
product explicitly agrees. Alternatives (rspamd-only quarantine vs SelfPost
store) are listed in [plans/inbound-quarantine.md](plans/inbound-quarantine.md).
**Done when:** TBD after open questions in the plan are answered.
**Dependencies / risks:** inbound relay `[1.4.0]`; storage/retention/backup
size; malware in held MIME; overlap with
[inbound-antispam-panel](#inbound-antispam-panel). No implementation checklist
yet.
**Version:** TBD; `candidate` until explicitly agreed.
---
## schema-squash
**Goal:** when 2.x is cut, stop shipping the 1.x migration chain in the binary