diff --git a/CHANGELOG.md b/CHANGELOG.md index 38e0891..da13587 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,22 @@ Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); version ### Added +- [docs/plans/inbound-antispam-panel.md](docs/plans/inbound-antispam-panel.md) — + agreed plan for inbound antispam journal (inbound-journal milter + mail.log + rejects) and instance-wide allow/deny lists synced to rspamd maps; target + `1.10.0`; Composer → Opus → Fable workflow. +- [docs/plans/inbound-quarantine.md](docs/plans/inbound-quarantine.md) — candidate + plan for held-mail review/release (design TBD). + +### Changed + +- [docs/roadmap.md](docs/roadmap.md) — `inbound-antispam-panel` (agreed, 1/12) + and `inbound-quarantine` (candidate) in the index. +- [docs/development.md](docs/development.md) — plan checklists may override + model routing with Composer → Opus → Fable. + +### Added + - [docs/schema-migrations.md](docs/schema-migrations.md) — living reference for the SQLite migration chain (`user_version` head, per-file history, legacy artefacts, 1.x rules, and planned 2.x squash gate). diff --git a/docs/development.md b/docs/development.md index af7acf1..50edd3c 100644 --- a/docs/development.md +++ b/docs/development.md @@ -64,6 +64,11 @@ obligation — not because the convention around AI authorship shifts. Default rule: risk-critical → Opus; UI / docs / boilerplate → Sonnet; trivial mechanics → Haiku. Reviewers must not be the author of the code under review. +**Composer → Opus → Fable** (when a plan specifies it): **Composer** writes all +implementation for that item (code, tests, docs); **Opus** does the technical +review of the result; **Fable** does the security review. A plan's checklist +overrides the default routing table for that item only. + --- ## Plan checklists diff --git a/docs/plans/inbound-antispam-panel.md b/docs/plans/inbound-antispam-panel.md new file mode 100644 index 0000000..bb2c5da --- /dev/null +++ b/docs/plans/inbound-antispam-panel.md @@ -0,0 +1,251 @@ +# Plan: inbound-antispam-panel + +**Status:** agreed +**Date:** 2026-08-19 +**Version:** `1.10.0` MINOR (opt-in; no change to outbound-only or inbound-without-filter paths). + +--- + +## Goal + +Give the operator a **panel view of inbound anti-spam decisions** and **editable +allow/deny lists** that tune filter behaviour — without SelfPost shipping or +starting an anti-spam engine ([product.md](../product.md) out-of-scope list +unchanged). + +Typical journal row: + +| When | From | To | Subject | Decision | +|------|------|----|---------|----------| +| 2026-08-19 09:14 | `spammer@evil.example` | `user@inbound.example` | Cheap pills | reject (rspamd: BAYES_SPAM +0.95) | + +Lists let the operator correct false positives/negatives (e.g. always allow +`billing@vendor.example`, block `192.0.2.0/24`). + +## Context (as-built) + +- Inbound relay ships in `[1.4.0]` (`INBOUND_RELAY_ENABLE`); optional + `INBOUND_ANTISPAM_MILTER` attaches an **external** filter (documented rspamd + sidecar in [deploy/antispam/docker-compose.antispam.yml](../../deploy/antispam/docker-compose.antispam.yml)). +- The inbound smtpd has **no journal-milter** today — only outbound 465/587 + writes `send_log` ([architecture.md](../architecture.md)). +- Inbound UI and handlers are **global-administrator only** today + (`requireInbound` → `requireGlobal`). + +## Quarantine + +**Not in this plan.** Held-mail review/release is tracked separately as roadmap +candidate [inbound-quarantine](inbound-quarantine.md). The journal may still +record a `quarantine` **decision label** when the external filter reports it, +without SelfPost storing the message. + +**DMARC `p=quarantine`** remains unrelated (outbound policy for receivers). + +## Decisions (closed 2026-08-19) + +### Capture mechanism + +**Inbound journal-milter in `panel` only** — no rspamd history API polling in v1. +A separate unix socket (`INBOUND_JOURNAL_MILTER_SOCKET`, default +`/run/selfpost/inbound-journal.sock`) on the inbound smtpd only; fail-open +(`default_action=accept`). + +**Milter order on port 25** (when antispam is enabled): + +```text +smtpd_milters = { antispam, inbound-journal } +``` + +Antispam runs first; inbound-journal runs at end-of-message on the **accept** +path and reads rspamd-added headers (`X-Rspamd-Action`, `X-Rspamd-Score`, symbol +summary when present). + +**Antispam rejects** do not reach end-of-message on the journal milter. v1 also +tails `mail.log` for inbound `postfix/smtpd` **`milter-reject`** / **`reject`** +lines and inserts journal rows (client IP, envelope from/to when logged, reason +text, subject empty if rejected before DATA). DMARC-ingest pipe traffic is out +of scope for this tailer. + +### Journal without `INBOUND_ANTISPAM_MILTER` + +Journal stays **on** whenever `INBOUND_RELAY_ENABLE=true` (not tied to the +antispam hook). Without an external filter: + +- **accept** rows — message relayed (engine `selfpost`, detail empty); +- **reject** / **tempfail** rows — from the mail.log tailer (Postfix policy, + size limit, unknown recipient, etc.). + +Panel copy explains that rspamd symbols appear only when the antispam milter is +configured. + +### RBAC + +**Global administrator only** for journal and lists — same as inbound relay. No +domain-admin access in v1. + +### Retention + +Mirror send-log retention: + +- Settings key `inbound_spam_log_retention_days` on `/settings` (global admin). +- Default **90** days; validation **7–365**; seed from env + `INBOUND_SPAM_LOG_RETENTION_DAYS` when the setting is missing. +- Prune every **6 hours** (same cadence as send-log retention). +- Hard cap **10 000** rows after age prune (drop oldest) — busier than DMARC + reports, lighter than unbounded growth. + +### Allow/deny lists + +| Topic | Decision | +|-------|----------| +| Entry types | sender address, sender domain (`@domain` suffix), client IP / CIDR | +| Precedence | **deny overrides allow** | +| Scope | instance-wide (not per inbound domain in v1) | +| Where applied | **rspamd map files** under `/data/antispam/` (allow.map, deny.map); documented compose fragment mounts them into the sidecar | +| SelfPost pre-check | **out of v1** — lists live in rspamd only | +| Edit access | global administrator only | +| When inactive | lists editable in the panel but UI warns that sync applies only when `INBOUND_ANTISPAM_MILTER` is set; `postfix reload` / rspamd reload documented in guide | + +List change → atomic map rewrite (same pattern as Postfix maps) → rspamd reload +via documented operator step or sidecar `SIGHUP` in the compose fragment. + +### Activation + +| Condition | Behaviour | +|-----------|-----------| +| `INBOUND_RELAY_ENABLE=false` | no inbound journal milter, no UI, no tables written | +| inbound on, antispam empty | journal + UI; lists visible but marked inactive for filter sync | +| inbound on + antispam set | full journal (symbols on accept) + list sync | + +No separate feature flag beyond inbound relay + existing antispam env vars. + +### UI + +New nav item **Inbound spam** (or subsection under **Inbound**) — journal table +with filters (decision, domain, date range), list management on the same page or +a tab. Mockup: add `docs/assets/panel-ui/inbound_spam.html` in the panel step. + +## Schema (migration `0010_inbound_spam_log.sql`) + +**`inbound_spam_log`** + +| Column | Type | Notes | +|--------|------|-------| +| `id` | INTEGER PK | | +| `inbound_domain` | TEXT NOT NULL | recipient domain (from `relay_domains`) | +| `client_ip` | TEXT NOT NULL | | +| `from_addr` | TEXT NOT NULL | envelope from | +| `to_addr` | TEXT NOT NULL | envelope to (one row per recipient) | +| `subject` | TEXT NOT NULL | truncated with `mailhdr.SubjectMaxRunes` (200) | +| `decision` | TEXT NOT NULL | `accept`, `reject`, `tempfail`, `quarantine` | +| `engine` | TEXT NOT NULL | `rspamd`, `postfix`, `selfpost` | +| `detail` | TEXT NOT NULL | symbol/score summary or log-line reason; may be empty | +| `created_at` | TEXT NOT NULL | RFC3339 UTC | + +Index on `(created_at)`, `(inbound_domain, created_at)`. + +**`inbound_spam_list`** + +| Column | Type | Notes | +|--------|------|-------| +| `id` | INTEGER PK | | +| `list_type` | TEXT NOT NULL | `allow` or `deny` | +| `entry_type` | TEXT NOT NULL | `address`, `domain`, `ip` | +| `value` | TEXT NOT NULL | normalized entry | +| `note` | TEXT NOT NULL | operator comment; default `''` | +| `created_at` | TEXT NOT NULL | | + +Unique on `(list_type, entry_type, value)`. + +## Scope + +**In:** + +- Schema, store CRUD, retention loop, mail.log tailer extension for inbound + rejects. +- Inbound journal-milter (fail-open) + Postfix `postfix-config.sh` wiring. +- rspamd map sync under `/data/antispam/`. +- Panel journal + list UI (global admin). +- Full backup includes `/data/antispam/` and new SQLite tables. +- Tests; [guide.md](../guide.md); [architecture.md](../architecture.md). + +**Out:** + +- Shipping rspamd, ClamAV, or any filter binary inside the SelfPost image. +- Outbound spam filtering. +- [inbound-quarantine.md](inbound-quarantine.md) storage/release. +- rspamd rule editing, Bayes training UI, antivirus. +- MIME replay from the panel (metadata only). +- Per-domain lists; domain-admin RBAC. +- rspamd history API polling. + +## Architecture + +```mermaid +flowchart LR + internet["Internet :25"] --> inbound["Postfix inbound smtpd"] + inbound --> antispam["External antispam milter optional"] + antispam --> journalM["inbound-journal milter fail-open"] + journalM --> upstream["smtp upstream transport"] + journalM --> sqlite["inbound_spam_log"] + mailog["mail.log tailer"] --> sqlite + listsUI["Panel allow/deny lists"] --> maps["/data/antispam/*.map"] + maps --> antispam + sqlite --> journalUI["Panel inbound spam journal"] +``` + +## Security + +- Same validation whitelists as inbound relay ([security.md](../security.md)). +- Journal rows contain PII — retention and global-admin-only access. +- Inbound journal milter fail-open must not block mail (backup-MX role). +- List map writes injection-safe; deny wins over allow. + +## Done when + +- With inbound relay + antispam sidecar, panel shows accept rows with symbols + and reject rows from mail.log tailing. +- Operator can add/remove list entries; a test message reflects allow/deny. +- With antispam hook off, journal still shows Postfix-level decisions; list UI + shows inactive sync notice. +- `go vet`, `go test`, image build green; guide and architecture updated. + +## Risks + +- rspamd header/symbol format drift — thin parser, pin sidecar tag in compose. +- Journal volume — retention + 10k cap. +- mail.log parse brittleness — unit tests on sample lines; inbound smtpd only. + +## Dependencies + +- [inbound-relay.md](inbound-relay.md) (shipped). + +## Model routing (this plan) + +| Role | Model | +|------|-------| +| All implementation (code, tests, docs) | **Composer** | +| Technical review (reviewer ≠ author) | **Opus** | +| Security review | **Fable** | + +Composer ships each step; Opus reviews the accumulated diff once implementation +steps are done (fix loop: Composer addresses Opus findings, Opus re-checks). +Fable runs after Opus sign-off. + +## Implementation checklist + +Target version cut: **`1.10.0`** (MINOR). One commit per step; +[development.md](../development.md) § Plan checklists. + +- [x] Agree journal fields, retention, and RBAC (this plan § Decisions) — **Composer** +- [ ] Migration `0010_inbound_spam_log.sql` — **Composer** +- [ ] Inbound journal-milter + Postfix wiring — **Composer** +- [ ] mail.log tailer: inbound reject rows — **Composer** +- [ ] List CRUD + validation + atomic rspamd map sync — **Composer** +- [ ] Panel: journal + lists UI (+ mockup) — **Composer** +- [ ] Backup includes `/data/antispam/` — **Composer** +- [ ] Unit + handler tests — **Composer** +- [ ] [guide.md](../guide.md), [architecture.md](../architecture.md) — **Composer** +- [ ] `go vet`, `go test`, e2e if applicable — **Composer** +- [ ] Technical review of inbound antispam changes — **Opus** +- [ ] Security review inbound antispam path — **Fable** diff --git a/docs/plans/inbound-quarantine.md b/docs/plans/inbound-quarantine.md new file mode 100644 index 0000000..fd0ec8c --- /dev/null +++ b/docs/plans/inbound-quarantine.md @@ -0,0 +1,72 @@ +# Plan: inbound-quarantine + +**Status:** candidate — design TBD; needs explicit agreement before any code +**Version:** TBD (`1.x` MINOR if opt-in and backward-compatible; may warrant `2.x` +if it changes inbound storage or backup contract). + +--- + +## Goal (draft) + +Let the operator **hold suspicious inbound mail** instead of only accept/reject +at the milter — review in the panel, then release to the configured upstream, +delete, or add sender to allow/deny lists. + +This is a **product boundary question**, not a settled design. The item exists +so we can compare approaches before committing. + +## Why it is separate from inbound-antispam-panel + +[inbound-antispam-panel.md](inbound-antispam-panel.md) covers **metadata** +(journal + lists). Quarantine implies **storing message bodies** (or a durable +reference to them) under `/data`, retention, release workflow, and a wider +attack/backup surface — closer to "mini-mailbox" than pure relay. + +## Open questions (to decide before a checklist) + +1. **Where mail lives** + - SelfPost store under `/data/quarantine/` (panel owns lifecycle), or + - delegate to rspamd/Redis and panel only proxies release (less storage in + SelfPost, tighter rspamd coupling). +2. **What "release" means** + - inject into Postfix as a new delivery to the domain's upstream transport; + - or manual download only (no automatic forward). +3. **Retention and caps** + - max age, max total bytes, per-domain limits; interaction with backup size. +4. **RBAC** + - global admin only vs domain-admin for assigned inbound domains. +5. **Threat model** + - malware in stored MIME, path traversal on extract, quota DoS on busy MX. +6. **Product fit** + - does this violate [product.md](../product.md) "no mailboxes" if we only + hold spam suspects briefly? Explicit product decision required. + +## Context (as-built) + +- Inbound relay forwards or rejects; no local delivery ([inbound-relay.md](inbound-relay.md)). +- Optional `INBOUND_ANTISPAM_MILTER` — external rspamd may quarantine on its + own today, without panel integration. +- DMARC `p=quarantine` is unrelated (outbound policy for receivers). + +## Likely out of scope (until revisited) + +- IMAP/webmail for quarantined mail. +- End-user self-service quarantine (non-admin recipients). +- Outbound quarantine. + +## Dependencies + +- [inbound-relay.md](inbound-relay.md) (shipped). +- Sensible to decide **after** or **alongside** + [inbound-antispam-panel.md](inbound-antispam-panel.md) (journal/lists), but + not blocked on it for design discussion. + +## Done when (placeholder) + +TBD once approach is chosen. Minimum bar would include: opt-in flag, panel +list/detail, release or discard action, retention, backup inclusion documented, +security review. + +## Implementation checklist + +None yet — expand this plan after the open questions above are answered. diff --git a/docs/roadmap.md b/docs/roadmap.md index b4355ff..40a6343 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -30,6 +30,8 @@ in `git log` and [CHANGELOG.md](../CHANGELOG.md). |---|---|---|---|---| | contributing | `CONTRIBUTING.md` | candidate | — | — | | panel-docs | In-panel operator documentation | agreed | 6/6 | [plans/panel-docs.md](plans/panel-docs.md) | +| inbound-antispam-panel | Inbound antispam journal + allow/deny lists | agreed | 1/12 | [plans/inbound-antispam-panel.md](plans/inbound-antispam-panel.md) | +| inbound-quarantine | Inbound spam quarantine (hold / review / release) | candidate | — | [plans/inbound-quarantine.md](plans/inbound-quarantine.md) | | schema-squash | Squash SQLite migrations into a 2.x baseline | **2.x** | — | — | **Recommended order** (not binding): the next feature is **panel-docs** once @@ -89,6 +91,50 @@ when checks change; not bloating every page with a second column of prose. --- +## inbound-antispam-panel + +**Goal:** panel view of inbound anti-spam decisions (when, from, to, subject, +action) plus editable allow/deny lists that tune the external filter — without +shipping rspamd or another engine inside the image. + +**Boundary:** metadata journal and list sync only; SelfPost does not run the +filter ([product.md](product.md)). Quarantine: +[inbound-quarantine](#inbound-quarantine). Design closed in +[plans/inbound-antispam-panel.md](plans/inbound-antispam-panel.md) § Decisions. + +**Done when:** with inbound relay enabled, the operator sees journal rows +(accepts via inbound-journal milter, rejects via mail.log tailer) and can +maintain instance-wide allow/deny lists synced to rspamd maps when the antispam +hook is set. + +**Dependencies / risks:** inbound relay `[1.4.0]`; rspamd map/header coupling; +journal PII; global-admin-only RBAC. + +**Version:** `1.10.0` MINOR; **agreed**. + +--- + +## inbound-quarantine + +**Goal:** optional hold for suspicious inbound mail — review in the panel, then +release to upstream, discard, or tune lists. Design not settled. + +**Boundary:** expands inbound from pure relay toward short-lived message +storage; may conflict with "no mailboxes" in [product.md](product.md) until +product explicitly agrees. Alternatives (rspamd-only quarantine vs SelfPost +store) are listed in [plans/inbound-quarantine.md](plans/inbound-quarantine.md). + +**Done when:** TBD after open questions in the plan are answered. + +**Dependencies / risks:** inbound relay `[1.4.0]`; storage/retention/backup +size; malware in held MIME; overlap with +[inbound-antispam-panel](#inbound-antispam-panel). No implementation checklist +yet. + +**Version:** TBD; `candidate` until explicitly agreed. + +--- + ## schema-squash **Goal:** when 2.x is cut, stop shipping the 1.x migration chain in the binary