Compare commits

...
17 Commits
Author SHA1 Message Date
mixandClaude Opus 5.5 d2a778edc0 inbound-antispam-panel: Filter lists accepted; steps 8-10 closed
test / test (push) Canceled after 0s
The owner accepted Inbound > Filter lists on 2026-10-12 ("Да"), after
a deny entry he added refused my test message on the live test server
with the list's own reply (554 5.7.1 Refused by this server's deny
list), mail.log carried the milter-reject line the count reads and the
journal its row.

Steps 8-10 of the plan were done with each earlier step and are ticked
with where. 10/12: the technical and the security review remain.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 07:22:09 +03:00
mixandClaude Opus 5.5 7192c3861e inbound-antispam-panel: Filter lists, kind column, matches, note on a sheet
Follows the design commit before it (owner's words quoted there).

The lists' tables are Value, Kind, Matched and Remove. The kind tag
opens the entry's sheet: note, day added, how often it decided a
message in the days the log keeps, and the last match.

Matches are read off the journal on each view, nothing is stored
(inbound.Lists.Usage, store.ListSpamListHits). A row counts for an
entry when the list decided the message - a rejected row whose reply is
the deny rule's message, an accepted row whose verdict carries the
allow rule's symbol for that kind - and the entry fits the row's sender
or client address the way rspamd matches it: an address whole, a domain
as the sender's domain, a network containing the client (an
IPv4-mapped address as IPv4). A failure to count shows the lists with
"never" and logs why.

For that the journal milter keeps the lists' own symbols ahead of its
six-symbol cap: rspamd writes symbols heaviest first, so an allow
entry's -100 came last and was cut behind any six others.

The plan gains "Filter list usage": what counting from the journal
implies, and - the owner asked for it in writing - the conditions under
which a counter kept with each entry becomes worth a migration (the
journal cut by its row cap and not by age; a short retention; counts
that must outlive the journal; a slow page). multimap.conf says which
of its texts SelfPost reads back.

Tests: Usage over every kind on both lists and the rows that are
nobody's; the milter's symbol order; the store query; the page and its
sheets; a handler test from journal rows to the numbers on the page.

Seen: the fixture page with a sheet open, headless Edge at 1360 px -
the same as the accepted drawing.

Checked on the test server, uncommitted tree: go vet ./... ok,
go test ./... ok (22 packages), make e2e ok.

Not seen in a real run: a count above zero. It needs an entry added in
the panel, which I cannot sign in to, and then mail that the list
decides; in particular the deny rule's message arriving in mail.log as
the reply of a milter-reject line is what the trial with rspamc showed,
not yet what Postfix logged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 01:08:26 +03:00
mixandClaude Opus 5.5 e0ccff257d design: in-filter-lists, kind column, matches, the note on a sheet
Owner, 2026-10-12, on Inbound > Filter lists as built: "Note хочется
вынести в лист, подсказку, раскрывающийся блок. Тип в отдельную колонку.
Ещё полезно будет дать статистику использования правила, чтобы удалять
устаревшие." Drawn, then accepted ("Да") together with counting the
matches from the Inbound log.

  - Kind is a column of its own; the tag no longer wraps under the
    value.
  - Note and Added leave the table for the entry's sheet, opened by the
    kind tag (of his three - sheet, tooltip, disclosure - the sheet: it
    exists and works under a finger).
  - Matched: how often the entry decided a message, or "never". The
    sheet adds the window ("in the last 90 days") and the last match.
    A fifth column for the last match did not fit half the page width.

Help, Filter lists: what Matched counts and what "never" suggests.

check.py: 32 screens, 0 violations; outlines.json unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 00:54:04 +03:00
mixandClaude Opus 5.5 c1da784193 inbound-antispam-panel: the backup holds the lists; Inbound log accepted
Step 7/12 of docs/plans/inbound-antispam-panel.md. A full backup walks
all of /data, so data/antispam/ was in it from the day the directory
existed; TestCreateIncludesStateExcludesTransient now holds that. On
the first start after a restore the maps are written again from the
database (mailStack.Resync, already there). The guide says what is not
in a backup: the antispam/ directory with the compose fragment and the
rspamd configuration, which is installation and not state.

The plan records the owner's acceptance of Inbound > Log (2026-10-12,
"Этот экран утверждаем") and what he changed on the way to it.

go test ./internal/backup/ ok.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 00:36:59 +03:00
mixandClaude Opus 5.5 9d330bed77 inbound-antispam-panel: Inbound log, Why as a sheet; no Deny on a Postfix reject
Follows the two design commits before it (owner's decisions quoted
there).

Kit: a new component, Sheet / SheetRow (components.go), partials
sheet_tag and sheet (components.html), its rules in panel.css as in
theme.css, and its sample on the kit page next to status_tag. It is
the browser's popover: no script.

Inbound log: the Why column is gone; the decision tag opens a sheet
with who decided, then either the filter's verdict and its symbols one
a line (a detail the journal milter wrote) or the reply (a detail the
tailer read from mail.log - it starts with its code), and the subject
whole. The sheet's id is the journal row's. A row Postfix itself
refused no longer links to the filter lists.

Tests: the page fixture and view tests for the sheet's markup and
content (TestWhyRows), the link rule, three Deny and two Allow on the
mockup's six rows; e2e reads "Decided by" and "Reply" from the sheet
and wants no sender link on the refused row.

Seen: the fixture page with a sheet open, headless Edge at 1360 px -
the same as the accepted drawing.

Checked on the test server, uncommitted tree: go vet ./... ok,
go test ./... ok (22 packages), make e2e ok.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 00:23:51 +03:00
mixandClaude Opus 5.5 970c8ca8da design: the sheet on the components page is a table; no empty Subject
The sample sheet on the components page still held facts; it shows the
table of name and value the accepted sheet is. In in-log a refused
message's sheet has no Subject row - there is no subject to show whole.

check.py: 32 screens, 0 violations.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 00:13:14 +03:00
mixandClaude Opus 5.5 19156814b8 design: in-log, Why is a sheet over the page; no Deny on a Postfix reject
Owner, 2026-10-11/12, on the Inbound log as built:
  - "Why сделать не столбцом, а листом поверх." Opened by the decision
    tag ("Тег."), drawn first, then: its content as a table ("В лист
    может быть таблицей?" - "Да. Но хорошо бы ещё её отцентрировать."),
    with a narrow column of names and only what the row does not show.
  - "Почему Deny у того, кто rejected?" - "Убирай": a message Postfix
    itself refused offers no Deny sender. Accepted mail offers Deny,
    mail the filter refused offers Allow.

A new component, sheet (theme.css, shown on the components page next to
status_tag): the browser's own popover, so no script - the tag is a
button with popovertarget; Esc, a click outside and the cross close it.
It rises from the bottom edge across the window; its content is a table
of name and value in a 44rem column in the middle. A tag that opens a
sheet carries a dotted underline to tell it from a plain tag.

in-log: the Why column is gone and rows are one height. A sheet holds
who decided, the verdict and the symbols (or the reply), and the
subject whole.

Guard: check.py WIDTH_OK gains ".sp-sheet" - the one component laid out
by the window and not by the page. The owner accepted the drawing with
that said.

check.py: 32 screens, 52 sp- classes, 0 violations; outlines.json
unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 00:12:04 +03:00
mixandClaude Opus 5.5 d2b97b3963 inbound-antispam-panel: Inbound log, the sender column is headed From
Follows the design commit before it.

go test ./internal/web/view/ ok.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 23:59:47 +03:00
mixandClaude Opus 5.5 d4efdb483d design: in-log, the sender column is headed From
Owner, 2026-10-11, of "From · client": "Client в названии столбца
зачем? Мне кажется, оно лишним." The client address stays under the
sender, in mono and muted; an address of that shape needs no label.

check.py: 32 screens, 0 violations; outlines.json unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 23:59:18 +03:00
mixandClaude Opus 5.5 56c5dfee40 inbound-antispam-panel: Inbound > Log and Inbound > Filter lists
Step 6/12 of docs/plans/inbound-antispam-panel.md. Written by a Sonnet
subagent from the mockups, reviewed and corrected by me.

GET /inbound/log - the journal a page at a time (50), filtered by
domain, decision and day; QueryInboundSpam / CountInboundSpam with the
viewer's inbound domains as the scope. The global role reads all of it
and gets "Deny sender" / "Allow sender" links into the lists; a user
with the domain role reads the rows of the inbound domains assigned to
them, without the links. The plan said global-only for the journal;
the accepted mockup and 2.0's delegation of Inbound say otherwise -
recorded in the plan as built.

GET, POST /inbound/filter-lists and POST .../{id}/delete - global role,
404 otherwise. Add form (prefilled from the query), the two lists with
Remove as the kit's row action, and "Where the lists go" with the
filter's state and when the maps were last written
(Lists.LastWritten). A refused or duplicate entry is 422 with the
message and what was typed.

Server > Settings: "Keep inbound log rows, days" with the relay on.
Help: Inbound log, Filter lists. Navigation: Log for everyone who sees
Inbound, Filter lists for the global role (Shell.ShowFilterLists).
Status tags accept / reject / tempfail join the kit's mapping.

Tests: page fixtures for both screens, held to outlines.json; view and
handler tests (both roles, filters, paging, relay off, add / duplicate
/ invalid / delete, map file content, settings field, help). e2e reads
the journal from the page instead of the database and adds and removes
a deny entry through the panel, reading deny_domain.map after each.

Seen: both pages and Settings rendered from their fixtures and compared
with the mockups side by side (headless Edge, 1360 px) - same layout
and wording.

Checked on the test server, uncommitted tree: go vet ./... ok,
go test ./... ok (22 packages), make e2e ok.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 23:50:34 +03:00
mixandClaude Opus 5.5 82d4a209be design: in-log and in-filter-lists leave the deferred screens
Guards and mockups only, ahead of the pages (inbound-antispam-panel
6/12). The owner agreed the feature ("Следующая по дорожной карте -
inbound-antispam-panel" / "Да", 2026-10-11) and started this step
("Начинай"), so the two screens are no longer "drawn but not built":

  - guard_panel_test.go: in-log and in-filter-lists removed from
    deferredScreens. From here TestLegacyRatchetMatchesTheEngine wants
    the engine to have both pages - red until the next commit.
  - guard_routes_test.go: the four routes join routes.inbound, so they
    must exist with the relay on and answer 404 with it off.

Two mockup details brought to what the kit can draw, found when the
pages were built from it:
  - help: the two new topics are one paragraph with a break in it, as
    every topic is (panel.css has no gap between paragraphs of a topic);
  - in-filter-lists: the directory in "Files" is mono at the text size -
    the kit's inline code has no small variant.

check.py: 32 screens, 0 violations; outlines.json unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 23:50:33 +03:00
mixandClaude Opus 5.5 737611a07e design: Inbound log and Filter lists as built; inbound retention; Help
Mockup-first step of inbound-antispam-panel 6/12. The two screens were
drawn for the redesign before the feature existed; this brings them to
what steps 3-5 built.

in-log: the lead says where rows come from and how long they are kept;
a help link in the box head; Why is the engine and the detail as the
journal holds them (rspamd: action, score / threshold, weighted
symbols; postfix and a refusing filter: the reply text). No row
invents what the journal cannot hold: the "selfpost - upstream timed
out" tempfail and the Review link to a quarantine page that does not
exist are gone, a refused message has no subject, an allow-listed one
shows SELFPOST_ALLOW_FROM(-100.00).

in-filter-lists: Remove is a POST - the kit's row action, not a link;
a domain entry is shown as stored, without "@"; "Where the lists go"
names the directory and its six files instead of allow.map / deny.map,
says rspamd rereads them by itself and that Deny wins.

settings: "Keep inbound log rows, days" in the Log retention box.

help: two topics under Inbound, Inbound log and Filter lists, and a
sentence in Settings.

outlines.json rewritten: Help's Inbound box has six topics, was four.
check.py: 32 screens, 0 violations.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 23:18:57 +03:00
mixandClaude Opus 5.5 eea40bcd49 inbound-antispam-panel: filter lists, written as rspamd maps
Step 5/12 of docs/plans/inbound-antispam-panel.md.

Store: AddSpamListEntry, ListSpamListEntries, DeleteSpamListEntry.
internal/inbound/lists.go validates and normalises an entry - sender
address, sender domain, client address or network (canonical text, an
IPv4-mapped address as IPv4, /0 refused) - and writes the lists as six
map files under /data/antispam (ANTISPAM_DIR), each atomically, 0644 in
a 0755 directory: the filter is another user in another container. All
six are written on every change and on every start with the relay on,
empty ones included. An entry whose maps could not be written is taken
back out of the table. The writer refuses a stored value with
whitespace, a newline or # in it.

The plan's allow.map / deny.map became one file per list and entry
type: to rspamd an IP map and a sender map are different map types.

deploy/antispam/rspamd/multimap.conf holds the rules; the fragment
mounts it and ./data/antispam into the sidecar. Deny is a prefilter
reject, allow a symbol weighing -100. Tried in a throwaway rspamd 3.11
container on the test server:
  - with allow as a prefilter accept, an address on both lists was
    accepted and an IP on both was refused - the rule that ran first
    decided; with the weight, deny won in all five mixed cases;
  - IPv4, IPv6, address (any case), domain (not its subdomains) match;
  - a rewritten file is reread in about ten seconds, no restart;
  - files that appear after rspamd started are loaded the same way.

No page edits the lists yet (step 6).

Checked on the test server, uncommitted tree: go vet ./... ok,
go test ./... ok (22 packages), make e2e ok - with the relay on
/data/antispam is 755 and holds the six maps, 644, empty.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 23:07:10 +03:00
mixandClaude Opus 5.5 27fd355e53 inbound-antispam-panel: refused inbound mail from mail.log; retention
Step 4/12 of docs/plans/inbound-antispam-panel.md.

The log tailer writes an inbound_spam_log row for mail port 25 refused
(internal/logtail/inbound.go): a "reject:" line of the port-25 smtpd is
Postfix's own policy; a milter-reject / milter-hold / milter-discard
line of cleanup is the filter's verdict. Only a recipient at an inbound
domain is a row, so "Relay access denied" for a stranger's domain is
not. The row is dated by its log line; the reply text is cut to 255
bytes of printable ASCII; a reply starting with 4 is a tempfail.

To tell port 25 from 465 / 587 the listener now logs as
postfix/inbound (syslog_name). cleanup logs under one name for every
listener, so its verdict counts only for a queue id the port-25 smtpd
was seen to open (4096 remembered). Lines are matched from the head,
after timestamp and host: their tail is the sender's own text.

Retention: PruneInboundSpam every six hours - older than the
inbound_spam_log_retention_days setting (seeded from
INBOUND_SPAM_LOG_RETENTION_DAYS, 90, range 7-365), then down to 10 000
rows. The setting has no field in the panel yet.

Known limits, in architecture.md: a log line names one recipient, so a
message refused at end-of-message is one row; a line re-read after an
unclean stop would be a second row.

Checked on the test server, uncommitted tree: go vet ./... ok,
go test ./... ok (22 packages), make e2e ok - the unknown recipient at
the inbound domain refused on port 25 has its row (reject by postfix),
the stranger's domain has none.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 22:42:25 +03:00
mixandClaude Opus 5.5 94e94490af roadmap: phone in "desktop site" mode gets the phone navigation
A candidate, recorded at the owner's word (2026-10-11, screenshot of
Server > Health on selfpost.mixfed.ru): the body is laid out in columns
while the navigation is the phone stack with every section open. The
item states what was seen and a first guess at the cause (980 px falls
between Bulma's 769 and 1024 px breakpoints); nothing is investigated
yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 22:42:08 +03:00
mixandClaude Opus 5.5 28ec784ae2 inbound-antispam-panel: inbound journal milter on port 25
Step 3/12 of docs/plans/inbound-antispam-panel.md.

The panel serves a second milter while the inbound relay is on
(internal/milter/inbound.go, socket INBOUND_JOURNAL_MILTER_SOCKET,
default /run/selfpost/inbound-journal.sock). build/postfix-config.sh
puts it last on port 25, after the optional antispam milter, with
default_action=accept; postfix-wrapper.sh waits for its socket. At
end-of-message it writes one inbound_spam_log row per recipient at an
inbound domain - other port-25 recipients (DMARC report addresses) get
none - and always accepts. With a filter attached it reads the verdict
from X-Rspamd-Action and X-Spamd-Result (action, score, up to six
weighted symbols, only what the patterns recognise, 255 bytes); without
one those headers are the sender's and are not read. The plan named
X-Rspamd-Score, which rspamd does not have.

The antispam compose fragment mounts rspamd/milter_headers.conf into
the sidecar so that it adds the two headers (checked in rspamd 3.11:
both routines exist and remove a sender's own headers of those names).

Store: InsertInboundSpam, IsInboundDomain.

Checked on the test server, uncommitted tree: go vet ./... ok,
go test ./... ok (22 packages), make e2e ok - the relayed message of
inbound_relay_delivers_and_refuses has its journal row (accept by
selfpost) and the DMARC report sent through the same listener has none.

Not yet seen in a real run: a verdict written by rspamd reaching this
milter. The e2e stand has no filter, and the test server's only inbound
domain has no upstream, so it accepts no mail.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 22:18:14 +03:00
mixandClaude Opus 5.5 a3cc52d6d4 inbound-antispam-panel: migration 0002, journal and filter-list tables
Step 2/12 of docs/plans/inbound-antispam-panel.md, the first migration
after the 2.0 baseline (2.0.1 was cut before it, as agreed).

0002_inbound_spam_log.sql adds inbound_spam_log (one row per recipient:
domain, client IP, envelope, subject, decision, engine, detail) with
indexes on created_at and (inbound_domain, created_at), and
inbound_spam_list (allow / deny entries, unique per list). DDL only;
decision, engine, list_type and entry_type carry CHECKs, as the
baseline does for its own enumerations. Nothing reads or writes the
tables yet.

Tests: TestBaselineSchema pins both tables and user_version 2;
TestOpenMigratesA20Database opens a file holding the baseline alone and
finds its user kept; TestInboundSpamConstraints.

go test ./internal/store/ ok; go vet ./... ok.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 22:05:20 +03:00
73 changed files with 5292 additions and 137 deletions
+63
View File
@@ -5,6 +5,69 @@ Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); version
## [Unreleased]
### Changed
- **Port 25 logs as `postfix/inbound`.** Its lines in the mail log (Server ›
System log) now read `postfix/inbound/smtpd[…]` where they read
`postfix/smtpd[…]`; the submission ports keep `postfix/smtpd`. This is how a
refusal on port 25 is told from one on 465 / 587.
### Added
- Schema: migration `0002_inbound_spam_log.sql`, the first after the 2.0
baseline - two empty tables for the inbound antispam journal and the filter
lists (inbound-antispam-panel). A 2.0.x database is migrated when the panel
starts.
- **Inbound mail is recorded.** With the inbound relay on, the panel runs a
second milter, the last one on port 25, and writes a row for every recipient
of a message that was accepted for an inbound domain: client address,
envelope, subject and - with a spam filter attached - the filter's action,
score and heaviest symbols. It only watches: it refuses nothing and Postfix
accepts mail when it is down.
- **Refused inbound mail is recorded too**, from the mail log: a recipient
Postfix refused at an inbound domain (unknown user, sender domain not
found), and a message the spam filter rejected, deferred, held or
discarded. The row is dated by its log line and carries the reply text. A
refusal to relay for a domain that is not an inbound one is not recorded.
- The inbound journal is pruned every six hours: rows older than 90 days
(`INBOUND_SPAM_LOG_RETENTION_DAYS`, 7-365) and anything beyond 10 000 rows.
- The antispam compose fragment mounts `antispam/rspamd/milter_headers.conf`
into the rspamd sidecar, which makes it write its verdict into
`X-Rspamd-Action` and `X-Spamd-Result`. **Copy `deploy/antispam/rspamd/`
next to the fragment before starting it** - Docker creates a directory in
place of a missing file and the sidecar then fails to read its
configuration.
- **Allow and deny lists for the spam filter**, kept by SelfPost and applied
by rspamd: sender addresses, sender domains, client addresses and networks.
They are written as six files under `data/antispam/`, which the fragment
mounts into the sidecar together with `antispam/rspamd/multimap.conf`;
rspamd rereads a changed file by itself. Deny wins over allow. The files
are there from the first start with the inbound relay on.
- **Inbound › Log**: the journal as a page - time, sender and client address,
recipient, subject and decision, filtered by domain, decision and day. The
decision opens a sheet over the page with who decided and why: the filter's
verdict and symbols, or the reply a refusal was given. A user with the
*domain* role sees the rows of the inbound domains assigned to them. For the
global role a row links to the filter lists with the sender filled in:
*Deny sender* on accepted mail, *Allow sender* on mail the filter refused.
- **Inbound › Filter lists** (global role): add and remove allow and deny
entries; the page says where the lists go and whether the filter is on to
read them. Each entry shows how often it decided a message - *Matched*, or
*never* - and its kind opens a sheet with the note, the day it was added and
the last match, so that entries nothing matches any more can be found and
removed. The count is read from the inbound log and reaches back as far as
the log does.
- **Server › Settings**: *Keep inbound log rows, days*, shown with the inbound
relay on.
- Help: *Inbound log* and *Filter lists*.
- New setting `ANTISPAM_DIR` (default `/data/antispam`).
- A full backup carries the inbound log, the filter lists and their files
under `data/antispam/`. It does not carry the `antispam/` directory with the
compose fragment and the rspamd configuration - copy that from
`deploy/antispam/` when restoring.
- New setting `INBOUND_JOURNAL_MILTER_SOCKET` (default
`/run/selfpost/inbound-journal.sock`).
## [2.0.1] - 2026-10-11
### Changed
+9
View File
@@ -112,6 +112,15 @@ for f in relay_domains transport relay_recipients tls_policy; do
done
chown panel:selfpost /data/postfix
chmod 2750 /data/postfix
# Filter lists for the external spam filter (written by the panel when
# INBOUND_RELAY_ENABLE=true, read by the antispam sidecar, which mounts this
# directory). The sidecar runs under a user of its own in another container,
# so the directory and its files are readable by anyone: they hold sender
# addresses and networks, no secrets.
mkdir -p /data/antispam
chown panel:selfpost /data/antispam
chmod 0755 /data/antispam
chown panel:selfpost /data/postfix/sender_login_maps
chmod 0640 /data/postfix/sender_login_maps
+25 -1
View File
@@ -117,6 +117,9 @@ SUBMISSION_ENABLE="${SUBMISSION_ENABLE:-false}"
INBOUND_RELAY_ENABLE="${INBOUND_RELAY_ENABLE:-false}"
INBOUND_ANTISPAM_MILTER="${INBOUND_ANTISPAM_MILTER:-}"
INBOUND_ANTISPAM_MILTER_ACTION="${INBOUND_ANTISPAM_MILTER_ACTION:-accept}"
# The panel's inbound journal milter (monitoring, fail-open), on port 25 only
# and only while the inbound relay is on. Matched by postfix-wrapper.sh.
INBOUND_JOURNAL_SOCK="${INBOUND_JOURNAL_MILTER_SOCKET:-/run/selfpost/inbound-journal.sock}"
INBOUND_RATE_LIMIT_MESSAGES_PER_IP="${INBOUND_RATE_LIMIT_MESSAGES_PER_IP:-20}"
INBOUND_MESSAGE_SIZE_LIMIT="${INBOUND_MESSAGE_SIZE_LIMIT:-26214400}"
RELAY_DOMAINS_MAP="${POSTFIX_RELAY_DOMAINS:-/data/postfix/relay_domains}"
@@ -357,7 +360,8 @@ if [ "${INBOUND_RELAY_ENABLE}" = "true" ] || [ "${DMARC_REPORTS_ENABLE}" = "true
# would file inbound mail into the outbound send log under the *sender's*
# domain — polluting Deliveries and letting a forged From: consume that
# domain's level-2 rate-limit budget. So smtpd_milters is always overridden
# here: either the optional antispam milter, or nothing at all.
# here: the optional antispam milter, then - with the inbound relay on - the
# panel's inbound journal milter, or nothing at all.
INBOUND_MILTERS=""
if [ "${INBOUND_RELAY_ENABLE}" = "true" ] && [ -n "${INBOUND_ANTISPAM_MILTER}" ]; then
case "${INBOUND_ANTISPAM_MILTER}" in
@@ -376,6 +380,22 @@ if [ "${INBOUND_RELAY_ENABLE}" = "true" ] || [ "${DMARC_REPORTS_ENABLE}" = "true
esac
INBOUND_MILTERS="{${INBOUND_ANTISPAM_MILTER},default_action=${INBOUND_ANTISPAM_MILTER_ACTION}}"
fi
if [ "${INBOUND_RELAY_ENABLE}" = "true" ]; then
# Last in the chain: Postfix hands a message to its milters one after
# another at end-of-message, so this one reads the verdict headers the
# filter added and sees only what the filter let through. It watches and
# never refuses, and must never hold up someone else's mail: accept when
# it is down. A comma, not a space, between the two - see
# master_cf_set_smtp_inet_o.
case "${INBOUND_JOURNAL_SOCK}" in
/[A-Za-z0-9._/-]*) ;;
*)
echo "FATAL: INBOUND_JOURNAL_MILTER_SOCKET must be an absolute path, got: ${INBOUND_JOURNAL_SOCK}" >&2
exit 1
;;
esac
INBOUND_MILTERS="${INBOUND_MILTERS:+${INBOUND_MILTERS},}{unix:${INBOUND_JOURNAL_SOCK},default_action=accept}"
fi
PORT25_RATE="${INBOUND_RATE_LIMIT_MESSAGES_PER_IP}"
PORT25_SIZE="${INBOUND_MESSAGE_SIZE_LIMIT}"
@@ -392,8 +412,12 @@ if [ "${INBOUND_RELAY_ENABLE}" = "true" ] || [ "${DMARC_REPORTS_ENABLE}" = "true
fi
fi
# syslog_name: this listener's lines read postfix/inbound/smtpd, the
# submission ports' postfix/smtpd. The log tailer files a refusal on port 25
# into the inbound journal and has to tell the two apart.
postconf -M "smtp/inet=smtp inet n - n - - smtpd"
postconf -P \
"smtp/inet/syslog_name=postfix/inbound" \
"smtp/inet/smtpd_sasl_auth_enable=no" \
"smtp/inet/smtpd_tls_auth_only=no" \
"smtp/inet/smtpd_sender_login_maps=" \
+10 -4
View File
@@ -2,8 +2,9 @@
# Postfix start wrapper: solves the cold-start race where Postfix would try to
# reach the milter sockets before they are listening.
#
# It blocks until BOTH milter sockets — OpenDKIM and the panel's journal-milter
# — are present, then execs `postfix start-fg`. If they are not ready within the
# It blocks until the milter sockets — OpenDKIM, the panel's journal-milter
# and, with the inbound relay on, its inbound journal milter — are present,
# then execs `postfix start-fg`. If they are not ready within the
# timeout it exits non-zero WITHOUT starting Postfix, so supervisord/Docker see
# an explicit startup failure instead of a relay running blind.
#
@@ -16,8 +17,13 @@ JOURNAL_SOCK="${JOURNAL_MILTER_SOCKET:-/run/selfpost/journal.sock}"
TIMEOUT="${MILTER_WAIT_TIMEOUT:-30}"
INTERVAL=1
SOCKS="$OPENDKIM_SOCK $JOURNAL_SOCK"
if [ "${INBOUND_RELAY_ENABLE:-false}" = "true" ]; then
SOCKS="$SOCKS ${INBOUND_JOURNAL_MILTER_SOCKET:-/run/selfpost/inbound-journal.sock}"
fi
elapsed=0
for sock in "$OPENDKIM_SOCK" "$JOURNAL_SOCK"; do
for sock in $SOCKS; do
while [ ! -S "$sock" ]; do
if [ "$elapsed" -ge "$TIMEOUT" ]; then
echo "postfix-wrapper: timed out after ${TIMEOUT}s waiting for milter socket $sock" >&2
@@ -29,5 +35,5 @@ for sock in "$OPENDKIM_SOCK" "$JOURNAL_SOCK"; do
echo "postfix-wrapper: milter socket ready: $sock"
done
echo "postfix-wrapper: both milter sockets ready, starting postfix"
echo "postfix-wrapper: milter sockets ready, starting postfix"
exec postfix start-fg
+11 -2
View File
@@ -24,6 +24,7 @@ type mailStack struct {
Domains *domain.Service
Apps *app.Service
Inbound *inbound.Service
Lists *inbound.Lists
DMARC *dmarc.Service
pf *postfix.Postfix
odk *domain.OpenDKIM
@@ -40,7 +41,8 @@ func newMailStack(cfg config, st *store.Store) *mailStack {
dmarcSvc := dmarc.NewService(st, pf, cfg.hostname, cfg.dmarcEnabled)
return &mailStack{
Domains: domains, Apps: apps, Inbound: inb, DMARC: dmarcSvc,
pf: pf, odk: odk, inboundEnabled: cfg.inboundEnabled, dmarcEnabled: cfg.dmarcEnabled,
Lists: inbound.NewLists(st, cfg.antispamDir),
pf: pf, odk: odk, inboundEnabled: cfg.inboundEnabled, dmarcEnabled: cfg.dmarcEnabled,
}
}
@@ -57,6 +59,11 @@ func (m *mailStack) Resync() error {
if err := m.Inbound.Resync(); err != nil {
return fmt.Errorf("inbound maps resync: %w", err)
}
// The filter's configuration names all six list files; they exist from
// the first start, empty until an entry is added.
if err := m.Lists.Sync(); err != nil {
return fmt.Errorf("filter lists resync: %w", err)
}
}
if m.dmarcEnabled {
if err := m.DMARC.Resync(); err != nil {
@@ -91,7 +98,7 @@ func newPanel(cfg config, st *store.Store) (*web.Server, error) {
// effective config, including a manual override; the panel keeps this
// snapshot for the process lifetime (architecture.md).
retryPolicy := postfix.LoadRetryPolicy()
return web.New(st, ms.Domains, ms.Apps, ms.Inbound, ms.DMARC, web.Config{
return web.New(st, ms.Domains, ms.Apps, ms.Inbound, ms.Lists, ms.DMARC, web.Config{
Hostname: cfg.hostname,
CookieSecure: cfg.cookieSecure,
SubmissionEnabled: cfg.submissionEnabled,
@@ -114,6 +121,8 @@ func newPanel(cfg config, st *store.Store) (*web.Server, error) {
InboundAntispamAction: cfg.inboundAntispamAction,
DMARCEnabled: cfg.dmarcEnabled,
SendLogRetentionEnvDefault: cfg.retentionDays,
InboundSpamLogRetentionEnvDefault: cfg.inboundSpamLogRetentionDays,
}, cfg.setupTokenPath)
}
+28 -9
View File
@@ -15,21 +15,42 @@ import (
// serveJournal opens the journal-milter Unix socket and runs the real milter
// (architecture.md § Mail path), recording accepted messages into the send
// log. Socket lifecycle (creation, stale cleanup, group permissions) lives
// here; the protocol handler lives in internal/milter.
// in listenMilter; the protocol handler lives in internal/milter.
func serveJournal(ctx context.Context, cfg config, st *store.Store) error {
socketPath := cfg.journalSocket
if err := os.MkdirAll(filepath.Dir(socketPath), 0o755); err != nil {
ln, err := listenMilter(cfg.journalSocket)
if err != nil {
return err
}
log.Printf("journal-milter listening on %s", cfg.journalSocket)
return milter.Serve(ctx, ln, st)
}
// serveInboundJournal runs the inbound journal milter, the last milter on
// port 25 (internal/milter/inbound.go). It exists only while the inbound relay
// is on: build/postfix-config.sh attaches the socket under the same condition.
func serveInboundJournal(ctx context.Context, cfg config, st *store.Store) error {
ln, err := listenMilter(cfg.inboundJournalSocket)
if err != nil {
return err
}
log.Printf("inbound-journal-milter listening on %s", cfg.inboundJournalSocket)
return milter.ServeInbound(ctx, ln, st, cfg.inboundAntispamMilter != "")
}
// listenMilter opens a milter's Unix socket where Postfix can reach it.
func listenMilter(socketPath string) (net.Listener, error) {
if err := os.MkdirAll(filepath.Dir(socketPath), 0o755); err != nil {
return nil, err
}
// Clear a stale socket left behind by an unclean shutdown, otherwise the
// listen below fails with "address already in use".
if err := os.Remove(socketPath); err != nil && !errors.Is(err, os.ErrNotExist) {
return err
return nil, err
}
ln, err := net.Listen("unix", socketPath)
if err != nil {
return err
return nil, err
}
// Postfix (user `postfix`) connects to this socket as a milter and needs
// group access. The socket inherits group `selfpost` from the setgid parent
@@ -37,9 +58,7 @@ func serveJournal(ctx context.Context, cfg config, st *store.Store) error {
// it (connecting to a Unix socket needs write permission on the node).
if err := os.Chmod(socketPath, 0o660); err != nil {
ln.Close()
return err
return nil, err
}
log.Printf("journal-milter listening on %s", socketPath)
return milter.Serve(ctx, ln, st)
return ln, nil
}
+51 -4
View File
@@ -23,6 +23,7 @@ import (
"github.com/mixeme/selfpost/internal/backup"
"github.com/mixeme/selfpost/internal/buildinfo"
"github.com/mixeme/selfpost/internal/dnscheck"
"github.com/mixeme/selfpost/internal/inbound"
"github.com/mixeme/selfpost/internal/logtail"
"github.com/mixeme/selfpost/internal/store"
)
@@ -89,6 +90,9 @@ type config struct {
saslRealm string
postfixDir string
deployRoot string
// Where the filter lists are written as map files for the external spam
// filter; the antispam compose fragment mounts it into the sidecar.
antispamDir string
inboundEnabled bool
// The optional milter on the inbound listener (INBOUND_ANTISPAM_MILTER) and
@@ -96,7 +100,12 @@ type config struct {
// what attaches it; the panel only says so. See antispamAction.
inboundAntispamMilter string
inboundAntispamAction string
dmarcEnabled bool
// The inbound journal milter's socket; the default mirrors
// build/postfix-config.sh. Served only while the inbound relay is on.
inboundJournalSocket string
// Retention of the inbound journal until Server > Settings says otherwise.
inboundSpamLogRetentionDays int
dmarcEnabled bool
}
func loadConfig() config {
@@ -157,11 +166,16 @@ func loadConfig() config {
saslDBPath: envDefault("SASL_DB_PATH", filepath.Join(dataDir, "sasl", "sasldb2")),
saslRealm: saslRealm(),
postfixDir: envDefault("POSTFIX_DIR", filepath.Join(dataDir, "postfix")),
deployRoot: envDefault("SELFPOST_DEPLOY_ROOT", "/selfpost-deploy"),
antispamDir: envDefault("ANTISPAM_DIR", filepath.Join(dataDir, "antispam")),
deployRoot: envDefault("SELFPOST_DEPLOY_ROOT", "/selfpost-deploy"),
// Optional inbound relay (backup-MX / forwarder). Off unless exactly "true".
inboundEnabled: os.Getenv("INBOUND_RELAY_ENABLE") == "true",
inboundAntispamMilter: os.Getenv("INBOUND_ANTISPAM_MILTER"),
inboundAntispamAction: antispamAction(os.Getenv("INBOUND_ANTISPAM_MILTER_ACTION")),
inboundJournalSocket: envDefault("INBOUND_JOURNAL_MILTER_SOCKET", "/run/selfpost/inbound-journal.sock"),
inboundSpamLogRetentionDays: envInt("INBOUND_SPAM_LOG_RETENTION_DAYS", store.InboundSpamLogRetentionDaysDefault),
// Optional DMARC aggregate ingest on port 25. Off unless exactly "true".
dmarcEnabled: os.Getenv("DMARC_REPORTS_ENABLE") == "true",
}
@@ -289,12 +303,38 @@ func run() error {
}
}
// The filter lists as files, on every start: the spam filter's
// configuration names all six, and a data directory from before the lists
// existed has none. Not fatal - the lists are the filter's business, and
// the panel has mail to relay either way.
if cfg.inboundEnabled && !restored {
if err := inbound.NewLists(st, cfg.antispamDir).Sync(); err != nil {
log.Printf("filter lists bootstrap: %v", err)
}
}
if err := st.EnsureSendLogRetentionDays(cfg.retentionDays); err != nil {
return fmt.Errorf("bootstrap send-log retention: %w", err)
}
// With the inbound relay on, the tailer also writes the inbound journal's
// rows for refused mail and prunes that journal.
var inboundTail *logtail.Inbound
if cfg.inboundEnabled {
if err := st.EnsureInboundSpamLogRetentionDays(cfg.inboundSpamLogRetentionDays); err != nil {
return fmt.Errorf("bootstrap inbound journal retention: %w", err)
}
inboundTail = &logtail.Inbound{Store: st, Retention: func() int {
days, err := st.GetInboundSpamLogRetentionDays(cfg.inboundSpamLogRetentionDays)
if err != nil {
log.Printf("inbound journal retention: %v", err)
return cfg.inboundSpamLogRetentionDays
}
return days
}}
}
var wg sync.WaitGroup
errc := make(chan error, 4)
errc := make(chan error, 5)
roles := []struct {
name string
@@ -310,7 +350,7 @@ func run() error {
return cfg.retentionDays
}
return days
})
}, inboundTail)
}},
{"rate-limit-recalc", func(ctx context.Context) error {
runAutoRateLimitRecalc(ctx, cfg, st)
@@ -318,6 +358,13 @@ func run() error {
}},
}
if cfg.inboundEnabled {
roles = append(roles, struct {
name string
fn func(context.Context) error
}{"inbound-journal-milter", func(ctx context.Context) error { return serveInboundJournal(ctx, cfg, st) }})
}
for _, r := range roles {
wg.Add(1)
go func(name string, fn func(context.Context) error) {
+4
View File
@@ -30,6 +30,10 @@ DMARC_REPORTS_ENABLE=false
# INBOUND_ANTISPAM_MILTER=inet:antispam:11332
# INBOUND_ANTISPAM_MILTER_ACTION=accept
# Days the inbound journal keeps a row (7-365) until the panel says otherwise;
# it never holds more than 10 000 rows.
# INBOUND_SPAM_LOG_RETENTION_DAYS=90
# Coarse per-client-IP limit and message size on inbound smtpd (port 25).
# INBOUND_RATE_LIMIT_MESSAGES_PER_IP=20
# INBOUND_MESSAGE_SIZE_LIMIT=26214400
@@ -27,3 +27,10 @@ services:
image: rspamd/rspamd:3.11
restart: unless-stopped
# No host ports: only SelfPost talks to the milter on the compose network.
volumes:
# The verdict headers SelfPost's inbound journal reads.
- ./antispam/rspamd/milter_headers.conf:/etc/rspamd/local.d/milter_headers.conf:ro
# The allow and deny lists kept in the panel: the rules, and the files
# SelfPost writes them to.
- ./antispam/rspamd/multimap.conf:/etc/rspamd/local.d/multimap.conf:ro
- ./data/antispam:/etc/rspamd/selfpost:ro
@@ -0,0 +1,11 @@
# Mounted into the sidecar as /etc/rspamd/local.d/milter_headers.conf.
#
# rspamd writes its verdict into two headers of every message it lets through:
# the action it chose and the score with the symbols behind it. SelfPost's
# inbound journal reads them (Inbound > Log shows them next to the message),
# and the upstream mailbox server receives them with the mail.
#
# rspamd removes headers of these names that a sender wrote himself before it
# adds its own. Without this file the journal still lists accepted mail, with
# no verdict next to it.
use = ["x-spamd-result", "x-rspamd-action"];
+63
View File
@@ -0,0 +1,63 @@
# Mounted into the sidecar as /etc/rspamd/local.d/multimap.conf.
#
# The allow and deny lists kept in the panel (Inbound > Filter lists). SelfPost
# writes them as six files under data/antispam/, which the fragment mounts at
# /etc/rspamd/selfpost; rspamd rereads a file when it changes, no restart.
#
# Deny wins over allow, whichever entries match. A denied sender is refused
# before anything else is looked at (a prefilter). An allowed one is not
# accepted outright - rspamd would then let whichever of the two rules ran
# first decide - but given a weight no spam score outweighs, which a refusal
# by the deny list still overrides.
#
# A domain entry is that domain, not its subdomains.
#
# SelfPost counts how often an entry decided a message by what these rules
# leave behind: the deny message below and the names of the allow symbols.
# Change either and Inbound > Filter lists shows "never" for every entry.
SELFPOST_DENY_IP {
type = "ip";
map = "/etc/rspamd/selfpost/deny_ip.map";
prefilter = true;
action = "reject";
message = "Refused by this server's deny list";
description = "Client address on the deny list";
}
SELFPOST_DENY_FROM {
type = "from";
map = "/etc/rspamd/selfpost/deny_from.map";
prefilter = true;
action = "reject";
message = "Refused by this server's deny list";
description = "Sender address on the deny list";
}
SELFPOST_DENY_DOMAIN {
type = "from";
filter = "email:domain";
map = "/etc/rspamd/selfpost/deny_domain.map";
prefilter = true;
action = "reject";
message = "Refused by this server's deny list";
description = "Sender domain on the deny list";
}
SELFPOST_ALLOW_IP {
type = "ip";
map = "/etc/rspamd/selfpost/allow_ip.map";
score = -100.0;
description = "Client address on the allow list";
}
SELFPOST_ALLOW_FROM {
type = "from";
map = "/etc/rspamd/selfpost/allow_from.map";
score = -100.0;
description = "Sender address on the allow list";
}
SELFPOST_ALLOW_DOMAIN {
type = "from";
filter = "email:domain";
map = "/etc/rspamd/selfpost/allow_domain.map";
score = -100.0;
description = "Sender domain on the allow list";
}
+1
View File
@@ -47,6 +47,7 @@ services:
# How long the send log keeps rows before the background sweep deletes
# them — the main driver of /data growth over time.
SEND_LOG_RETENTION_DAYS: "${SEND_LOG_RETENTION_DAYS:-90}"
INBOUND_SPAM_LOG_RETENTION_DAYS: "${INBOUND_SPAM_LOG_RETENTION_DAYS:-90}"
# Sliding idle timeout for the panel login session, in days.
PANEL_SESSION_IDLE_DAYS: "${PANEL_SESSION_IDLE_DAYS:-7}"
# Resolvers the panel's PTR/SPF/DKIM/DMARC checks query directly, so they
+53 -9
View File
@@ -27,7 +27,7 @@ Managed programs ([build/supervisord.conf](../build/supervisord.conf)):
| Program | User | Priority | Role |
|---|---|---|---|
| `opendkim` | root → `opendkim` (`UserID` in opendkim.conf) | 100 | DKIM signing milter |
| `panel` | panel | 200 | HTTP UI + journal-milter + log-tailer + rate-limit recalc |
| `panel` | panel | 200 | HTTP UI + journal-milter + log-tailer + rate-limit recalc; with the inbound relay on also the inbound journal milter |
| `postfix` | root (wrapper) | 300 | MTA — started only after both milter sockets exist |
| `postfix-reload` | root | — | On-demand `postfix reload` (autostart off) |
| `cert-reload` | root | 400 | Daily `postfix reload` for renewed TLS certs |
@@ -74,10 +74,53 @@ smtp/inet` removes Debian's stock smtpd). Outbound delivery still uses the
`smtpd_tls_security_level=encrypt`.
- **25/smtp inet** — when `INBOUND_RELAY_ENABLE=true` and/or
`DMARC_REPORTS_ENABLE=true`. No SASL, no OpenDKIM, no journal-milter —
`smtpd_milters` is always overridden on this service (to the antispam milter
or to nothing), because main.cf's submission chain would otherwise let a
signing outage defer relayed mail and would file inbound mail into the
outbound send log under the sender's domain.
`smtpd_milters` is always overridden on this service, because main.cf's
submission chain would otherwise let a signing outage defer relayed mail and
would file inbound mail into the outbound send log under the sender's
domain. With the inbound relay on the chain is the optional antispam milter,
then the panel's **inbound journal milter** (unix socket
`INBOUND_JOURNAL_MILTER_SOCKET`, default
`/run/selfpost/inbound-journal.sock`, `default_action=accept`); with DMARC
ingest alone it is empty. Postfix hands a message to its milters one after
another at end-of-message, each seeing what the one before added, so the
journal milter sees only mail the filter let through and reads the filter's
verdict headers (`X-Rspamd-Action`, `X-Spamd-Result`) off it. It writes one
`inbound_spam_log` row per recipient at an inbound domain - a DMARC report
address is not one - and never refuses anything
([internal/milter/inbound.go](../internal/milter/inbound.go)). The headers
are read only while a filter is attached; refused mail never reaches it.
Refusals come from `mail.log`: this listener logs as `postfix/inbound`
(`syslog_name`), and the log tailer turns a `reject:` line of
`postfix/inbound/smtpd` (Postfix's own policy) and a `milter-reject` /
`milter-hold` / `milter-discard` line of `cleanup` (the filter's verdict,
counted only for a queue id that listener opened) into a row, dated by the
line, when the recipient is at an inbound domain
([internal/logtail/inbound.go](../internal/logtail/inbound.go)). A line
names one recipient, so a message refused at end-of-message is one row
whatever its recipient count, and a line read twice (the tailer saves its
place once a second; an unclean stop can lose that second) is two rows. The
journal is pruned every six hours: rows
older than the `inbound_spam_log_retention_days` setting, then down to
10 000.
**Filter lists** (`inbound_spam_list`): allow and deny entries for the
external filter - sender address, sender domain, client address or network -
validated and normalised by `internal/inbound` (`lists.go`) and written as
six map files under `/data/antispam/` (`ANTISPAM_DIR`), each atomically, on
every change and on every start with the relay on. SelfPost never consults
them. The sidecar mounts the directory and rspamd's multimap rules
([deploy/antispam/rspamd/multimap.conf](../deploy/antispam/rspamd/multimap.conf))
read them: a deny entry is a prefilter `reject`, an allow entry a symbol
weighing -100. Allow is deliberately not a prefilter `accept`: with two
prefilters rspamd lets the one that ran first decide, and deny then won or
lost by rule order (seen on rspamd 3.11). A refusal by the deny list
overrides any score, so deny wins whichever entries match.
How often an entry decided a message is not stored: `Lists.Usage` reads it
off the journal on each view of Filter lists - the rows a list decided (the
deny rule's reply, the allow rule's symbol, which the journal milter keeps
ahead of its six-symbol cap) matched against the entries as rspamd matches
them. When that stops being good enough is written in
[plans/inbound-antispam-panel.md](plans/inbound-antispam-panel.md) § Filter
list usage.
**Inbound relay** (`INBOUND_RELAY_ENABLE=true`): accepts only
`relay_domains` + `relay_recipient_maps` (`reject_unauth_destination`,
`reject_unlisted_recipient`). Maps under `/data/postfix/` (`relay_domains`,
@@ -531,16 +574,17 @@ full backups also archive the operator's `./certs` PEM files when present.
### Database
The schema is one embedded baseline, `internal/store/migrations/0001_init.sql`,
applied by `store.Open` into an empty file; `PRAGMA user_version` is `1` and
The schema is an embedded baseline, `internal/store/migrations/0001_init.sql`,
and the migrations after it (`0002_inbound_spam_log.sql`), applied by
`store.Open` in file order; `PRAGMA user_version` counts the files applied and
`PRAGMA application_id` is `0x53503230` ("SP20"). There is no 1.x migration
chain. **2.0 starts from an empty data directory:** a database with a schema
version but without the 2.x `application_id` — what every 1.x file looks like —
is refused at start with `ErrForeignSchema` and left untouched, as is one whose
version is newer than the build knows. A restored 1.x backup fails the same
way. Domains are carried across with a domain export, whose format is not the
database. Later schema changes are new files (`0002_*.sql`, …); once `2.0.0` is
released the baseline is never edited. History and the per-table list:
database. A schema change is a new file; the baseline has not been edited
since `2.0.0`. History and the per-table list:
[schema-migrations.md](schema-migrations.md).
The tables that carry the panel's model:
+1 -1
View File
@@ -37,7 +37,7 @@ SIGNED_OUT = {"sp-signin", "sp-signin-brand", "sp-signin-form", "sp-foot"}
# Selectors allowed to set a width. Everything else gets its width from the
# layout it sits in — that is the whole point of the redesign.
WIDTH_OK = (".navbar-brand img", ".sp-head .sp-lead", ".sp-head .sp-grow", ".sp-postmark", ".table td.sp-clip",
".sp-h-icon", ".sp-signin", ".sp-timeline li::before", ".sp-perf")
".sp-h-icon", ".sp-signin", ".sp-timeline li::before", ".sp-perf", ".sp-sheet")
class Node:
@@ -114,6 +114,7 @@
<tr><td>error, fail, mismatch, bounced</td><td><span class="tag is-danger is-light">fail</span></td><td class="sp-mono sp-small">tag is-danger is-light</td></tr>
<tr><td>unknown, off, neutral counts</td><td><span class="tag is-light">off</span></td><td class="sp-mono sp-small">tag is-light</td></tr>
<tr><td>held: quarantine · role: global</td><td><span class="tag is-primary is-light">quarantine</span></td><td class="sp-mono sp-small">tag is-primary is-light</td></tr>
<tr><td>a tag that opens a <code>sheet</code></td><td><button type="button" class="tag is-danger is-light" popovertarget="kit-sheet">reject</button><div class="sp-sheet" id="kit-sheet" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-danger is-light">reject</span><h2>sheet</h2><button type="button" popovertarget="kit-sheet" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>What it holds</th><td>What does not fit the row, as name and value</td></tr><tr><th>How it closes</th><td class="sp-mono sp-small">Esc<br>a click outside<br>the cross</td></tr></tbody></table></div></div></div></div></td><td class="sp-mono sp-small">button.tag + sp-sheet (popover)</td></tr>
</tbody>
</table></div>
</div>
+5 -1
View File
@@ -97,6 +97,8 @@
<li><a href="#upstream"><i class="ti ti-arrow-forward-up"></i>Upstream</a></li>
<li><a href="#recipients"><i class="ti ti-address-book"></i>Valid recipients</a></li>
<li><a href="#filter"><i class="ti ti-shield-check"></i>Spam filter</a></li>
<li><a href="#inbound-log"><i class="ti ti-list-details"></i>Inbound log</a></li>
<li><a href="#filter-lists"><i class="ti ti-filter"></i>Filter lists</a></li>
</ul>
<p class="menu-label">Server</p>
<ul class="menu-list">
@@ -132,12 +134,14 @@
<div class="sp-help-topic" id="upstream"><h3>Upstream</h3><p>The upstream is the server that actually holds the mailboxes. Give its host and port — used exactly as written, with no MX lookup — and choose the TLS for the hop to it: Opportunistic uses TLS when the upstream offers it and otherwise sends in the clear, Required holds the mail until a TLS session is possible, Off never uses TLS. While the upstream does not answer, Postfix keeps the mail and retries as for any other mail.</p></div>
<div class="sp-help-topic" id="recipients"><h3>Valid recipients</h3><p>Choose whom this domain accepts. Listed addresses only: the relay accepts exactly the addresses you list, one per line or comma-separated, and refuses any other while the sender is still connected, so it never has to send a bounce for them. Any recipient: everything at the domain is accepted and passed on, and an address the upstream then rejects comes back as a bounce from this server to a sender who is often forged — backscatter. Prefer a list, and keep it in step with the mailboxes upstream.</p></div>
<div class="sp-help-topic" id="filter"><h3>Spam filter</h3><p>The spam filter belongs to the server, not to a domain. The server's administrator attaches it in the server's <code>.env</code>, and from then on every inbound domain is screened by it before mail is handed on; the box shows on or off and has nothing to switch. Off means mail goes to the upstream unfiltered. While the filter is down, mail is either accepted unfiltered (the default) or deferred, so the sending server tries again later, as the server's setting says; the box states which.</p></div>
<div class="sp-help-topic" id="inbound-log"><h3>Inbound log</h3><p>One row per recipient of a message offered on port 25 for an inbound domain, with what was decided: <span class="tag is-success is-light">accept</span> — handed on to the upstream; <span class="tag is-danger is-light">reject</span> — refused for good; <span class="tag is-warning is-light">tempfail</span> — the sender was asked to come back later; <span class="tag is-primary is-light">quarantine</span> — held by the filter. <em>Why</em> names who decided. <code>rspamd</code> is the spam filter, with its action, the score against the reject threshold and the heaviest symbols; <code>postfix</code> is the relay's own rule — an unknown recipient, a sender domain that does not exist — with its reply; <code>selfpost</code> means nobody judged the message: it was relayed with no filter attached. Accept says what this server did, not what the upstream did with the message afterwards.<br><br>Accepted mail is recorded as it passes; refused mail is read from <code>mail.log</code>, so its row has no subject, and a message refused after its text was sent is one row however many recipients it had. A refusal to relay for a domain that is not an inbound one is every scanner on the internet and is not recorded. Rows are kept for the number of days set in Settings, and never more than 10 000.</p></div>
<div class="sp-help-topic" id="filter-lists"><h3>Filter lists</h3><p>Two lists for the whole server that correct the spam filter by hand. An entry is a sender address, a sender domain — that domain, not its subdomains — or a client address or network in CIDR form. A sender on the Deny list is refused outright. A sender on the Allow list is given a weight no spam score outweighs, so the filter lets it through. When entries of both lists match one message, <strong>Deny wins</strong>.<br><br>SelfPost does not apply the lists itself: it writes them as files under <code>/data/antispam/</code>, and rspamd rereads a file within seconds of a change. With the spam filter off the lists are kept and can be edited, but nothing reads them. The sender matched is the envelope sender — the one in the log — not the From line a mail program shows. Matched counts the messages in the Inbound log that the entry decided, so it reaches back as far as the log does; never means not once in that time, and such an entry is one to consider removing.</p></div>
</div>
<div class="box">
<div class="sp-box-head"><span class="sp-no">04</span><h2>Server</h2></div>
<div class="sp-help-topic" id="backup"><h3>Backup and moving a domain</h3><p>Full backup downloads one archive of the whole instance: <code>data/</code> (the database, DKIM keys and application credentials), <code>docker-compose.yml</code>, <code>.env</code> and <code>certs/</code>. The Postfix queue is not in it. The backup is taken while the server is running, and Postfix keeps moving messages through the queue, so a copy could hold a message twice or miss it; mail still waiting for delivery is therefore not carried over, and a restored server starts with an empty queue. To keep the queue, stop the container and archive the project directory instead. The private key in <code>certs/</code> has to be readable by the panel: give it to group 998 on the host (<code>chgrp 998</code>, <code>chmod 640</code>). Until then the panel refuses to make a backup and names the file, rather than hand over an archive without the key. It holds secrets, so leave Encrypt with a password ticked: the file is sealed as <code>.spbk</code>, the password is not stored, and without it nothing can be recovered. To restore, unpack it into an empty project directory (decrypt a <code>.spbk</code> first with <code>selfpost-backup -decrypt</code>) and start the same SelfPost version that made it; the panel refuses data from another version. The reverse-proxy configuration and <code>mail.log</code> are not included. One domain moves on its own: Export domain on its settings page writes a <code>.json</code> (or encrypted <code>.spde</code>) with its DKIM key and its applications' passwords, and Import a domain here reads it into another instance, so the published DNS needs no change.</p></div>
<div class="sp-help-topic" id="users"><h3>Users and roles</h3><p>The two roles are a reach, not a rank: every user is an administrator and signs in with a username and password. Global reaches the whole server — Overview, Server, the queue and every domain — and is the only role that adds or deletes domains and users. Domain reaches only the domains assigned to it, outbound and inbound separately (the same name on both lists is two assignments); <em>All</em> includes domains added later, and a domain user needs at least one domain or <em>All</em> on either list. The only global user cannot be demoted or deleted, and you cannot delete the account you are signed in with. A new password set here signs that user out of their other sessions.</p></div>
<div class="sp-help-topic" id="settings"><h3>Settings</h3><p>Settings holds what is true of the whole instance. Log retention is how many days the Outbound log keeps its rows, from 7 to 365 and 90 by default; a sweep every six hours deletes older ones, and it is the main driver of how fast <code>/data</code> grows. It does not change the rotation of <code>mail.log</code>, which is daily with about two weeks kept. The sending rate limit shown under it is level 1, the per-client-IP ceiling set in <code>.env</code>: changing it needs a restart, and no level-2 limit can exceed it.</p></div>
<div class="sp-help-topic" id="settings"><h3>Settings</h3><p>Settings holds what is true of the whole instance. Log retention is how many days the Outbound log keeps its rows, from 7 to 365 and 90 by default; a sweep every six hours deletes older ones, and it is the main driver of how fast <code>/data</code> grows. With the inbound relay on there is a second number, for the Inbound log, with the same range. It does not change the rotation of <code>mail.log</code>, which is daily with about two weeks kept. The sending rate limit shown under it is level 1, the per-client-IP ceiling set in <code>.env</code>: changing it needs a restart, and no level-2 limit can exceed it.</p></div>
</div>
<div class="box">
<div class="sp-box-head"><span class="sp-no">05</span><h2>Account</h2></div>
@@ -74,7 +74,7 @@
</div>
<div class="box">
<div class="sp-box-head"><span class="sp-no">01</span><h2>Add an entry</h2><a class="sp-help" href="help.html#inbound" title="How the lists are applied"><i class="ti ti-help-circle"></i></a></div>
<div class="sp-box-head"><span class="sp-no">01</span><h2>Add an entry</h2><a class="sp-help" href="help.html#filter-lists" title="How the lists are applied"><i class="ti ti-help-circle"></i></a></div>
<div class="sp-box-body">
<form class="sp-form" method="post" action="#">
<div class="columns">
@@ -84,7 +84,7 @@
</div>
<div class="column is-3">
<div class="field"><label class="label" for="etype">Entry is</label>
<div class="control"><div class="select is-fullwidth"><select id="etype" name="entry_type"><option>a sender address</option><option>a sender domain</option><option>a client IP or CIDR</option></select></div></div></div>
<div class="control"><div class="select is-fullwidth"><select id="etype" name="entry_type"><option>a sender address</option><option>a sender domain</option><option>a client address or network</option></select></div></div></div>
</div>
<div class="column is-6">
<div class="field"><label class="label" for="value">Value</label>
@@ -103,11 +103,11 @@
<div class="box">
<div class="sp-box-head"><span class="sp-no">02</span><h2>Deny</h2><div class="sp-end">3 entries · wins over Allow</div></div>
<div class="table-container"><table class="table is-fullwidth is-hoverable">
<thead><tr><th>Value</th><th>Note</th><th>Added</th><th></th></tr></thead>
<thead><tr><th>Value</th><th>Kind</th><th>Matched</th><th></th></tr></thead>
<tbody>
<tr><td class="sp-mono sp-small">@bulk.example <span class="tag is-light">domain</span></td><td class="sp-muted">newsletter nobody subscribed to</td><td class="sp-muted sp-nowrap sp-small">2026-09-02</td><td class="sp-actions"><a class="has-text-danger" href="#">Remove</a></td></tr>
<tr><td class="sp-mono sp-small">198.51.100.0/24 <span class="tag is-light">ip</span></td><td class="sp-muted">dictionary attack on lists.example.org</td><td class="sp-muted sp-nowrap sp-small">2026-08-19</td><td class="sp-actions"><a class="has-text-danger" href="#">Remove</a></td></tr>
<tr><td class="sp-mono sp-small">win@lottery.example <span class="tag is-light">address</span></td><td class="sp-muted">—</td><td class="sp-muted sp-nowrap sp-small">2026-07-30</td><td class="sp-actions"><a class="has-text-danger" href="#">Remove</a></td></tr>
<tr><td class="sp-mono sp-small">bulk.example</td><td><button type="button" class="tag is-light" popovertarget="deny-1">domain</button><div class="sp-sheet" id="deny-1" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-light">domain</span><h2>bulk.example</h2><button type="button" popovertarget="deny-1" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Note</th><td>newsletter nobody subscribed to</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-09-02</td></tr><tr><th>Matched</th><td>41 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-09-21 13:57</td></tr></tbody></table></div></div></div></div></td><td class="sp-nowrap">41</td><td class="sp-actions"><form method="post" action="#"><button type="submit" class="has-text-danger">Remove</button></form></td></tr>
<tr><td class="sp-mono sp-small">198.51.100.0/24</td><td><button type="button" class="tag is-light" popovertarget="deny-2">ip</button><div class="sp-sheet" id="deny-2" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-light">ip</span><h2>198.51.100.0/24</h2><button type="button" popovertarget="deny-2" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Note</th><td>dictionary attack on lists.example.org</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-08-19</td></tr><tr><th>Matched</th><td>1203 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-09-21 09:12</td></tr></tbody></table></div></div></div></div></td><td class="sp-nowrap">1203</td><td class="sp-actions"><form method="post" action="#"><button type="submit" class="has-text-danger">Remove</button></form></td></tr>
<tr><td class="sp-mono sp-small">win@lottery.example</td><td><button type="button" class="tag is-light" popovertarget="deny-3">address</button><div class="sp-sheet" id="deny-3" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-light">address</span><h2>win@lottery.example</h2><button type="button" popovertarget="deny-3" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Note</th><td>—</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-07-30</td></tr><tr><th>Matched</th><td>never in the last 90 days</td></tr></tbody></table></div></div></div></div></td><td class="sp-muted sp-nowrap">never</td><td class="sp-actions"><form method="post" action="#"><button type="submit" class="has-text-danger">Remove</button></form></td></tr>
</tbody>
</table></div>
</div>
@@ -116,10 +116,10 @@
<div class="box">
<div class="sp-box-head"><span class="sp-no">03</span><h2>Allow</h2><div class="sp-end">2 entries</div></div>
<div class="table-container"><table class="table is-fullwidth is-hoverable">
<thead><tr><th>Value</th><th>Note</th><th>Added</th><th></th></tr></thead>
<thead><tr><th>Value</th><th>Kind</th><th>Matched</th><th></th></tr></thead>
<tbody>
<tr><td class="sp-mono sp-small">@partner.example <span class="tag is-light">domain</span></td><td class="sp-muted">invoices were scored as spam</td><td class="sp-muted sp-nowrap sp-small">2026-09-11</td><td class="sp-actions"><a class="has-text-danger" href="#">Remove</a></td></tr>
<tr><td class="sp-mono sp-small">noreply@github.example <span class="tag is-light">address</span></td><td class="sp-muted">security advisories</td><td class="sp-muted sp-nowrap sp-small">2026-06-04</td><td class="sp-actions"><a class="has-text-danger" href="#">Remove</a></td></tr>
<tr><td class="sp-mono sp-small">partner.example</td><td><button type="button" class="tag is-light" popovertarget="allow-1">domain</button><div class="sp-sheet" id="allow-1" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-light">domain</span><h2>partner.example</h2><button type="button" popovertarget="allow-1" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Note</th><td>invoices were scored as spam</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-09-11</td></tr><tr><th>Matched</th><td>7 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-09-19 16:40</td></tr></tbody></table></div></div></div></div></td><td class="sp-nowrap">7</td><td class="sp-actions"><form method="post" action="#"><button type="submit" class="has-text-danger">Remove</button></form></td></tr>
<tr><td class="sp-mono sp-small">noreply@github.example</td><td><button type="button" class="tag is-light" popovertarget="allow-2">address</button><div class="sp-sheet" id="allow-2" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-light">address</span><h2>noreply@github.example</h2><button type="button" popovertarget="allow-2" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Note</th><td>security advisories</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-06-04</td></tr><tr><th>Matched</th><td>18 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-09-21 13:30</td></tr></tbody></table></div></div></div></div></td><td class="sp-nowrap">18</td><td class="sp-actions"><form method="post" action="#"><button type="submit" class="has-text-danger">Remove</button></form></td></tr>
</tbody>
</table></div>
</div>
@@ -130,10 +130,10 @@
<div class="sp-box-head"><span class="sp-no">04</span><h2>Where the lists go</h2><div class="sp-end"><span class="tag is-success is-light">applied</span></div></div>
<div class="sp-box-body">
<dl class="sp-facts">
<div><dt>Applied by</dt><dd>rspamd, from map files</dd></div>
<div><dt>Files</dt><dd class="sp-mono sp-small">/data/antispam/allow.map<br>/data/antispam/deny.map</dd></div>
<div><dt>Applied by</dt><dd>rspamd, from map files it rereads within seconds of a change</dd></div>
<div><dt>Files</dt><dd><span class="sp-mono">/data/antispam/</span> — six, one for each list and kind of entry</dd></div>
<div><dt>Last written</dt><dd>2026-09-11 08:40 UTC</dd></div>
<div><dt>When the filter is off</dt><dd class="sp-muted">The lists are kept and editable, but nothing reads them until <code>INBOUND_ANTISPAM_MILTER</code> is set.</dd></div>
<div><dt>When both lists match</dt><dd>Deny wins, whichever entries they are.</dd></div>
</dl>
</div>
</div>
+10 -9
View File
@@ -69,7 +69,7 @@
<div class="sp-grow">
<p class="sp-kicker">Inbound</p>
<h1 class="title is-3">Inbound log</h1>
<p class="sp-lead">What port 25 decided about each message, one row per recipient. Accepts come from the inbound journal, rejects from <code>mail.log</code>; the filter's reasons appear when the spam filter is on.</p>
<p class="sp-lead">What port 25 decided about each message, one row per recipient. Accepted mail is recorded as it passes, refused mail is read from <code>mail.log</code>; the filter's verdict appears when the spam filter is on. Rows older than 90 days are deleted — change that in <a href="settings.html">Settings</a>.</p>
</div>
</div>
@@ -83,19 +83,20 @@
<input class="input" type="date" name="to" value="2026-09-21" aria-label="To date">
<button type="submit" class="button"><i class="ti ti-filter"></i>Filter</button>
</form>
<a class="sp-help" href="help.html#inbound-log" title="What the decisions mean"><i class="ti ti-help-circle"></i></a>
</div>
<div class="table-container"><table class="table is-fullwidth is-hoverable">
<thead><tr><th>Time, UTC</th><th>From · client</th><th>To</th><th>Subject</th><th>Decision</th><th>Why</th><th></th></tr></thead>
<thead><tr><th>Time, UTC</th><th>From</th><th>To</th><th>Subject</th><th>Decision</th><th></th></tr></thead>
<tbody>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 14:00:18</td><td>t.berg@<wbr>uni.example<br><span class="sp-mono sp-muted sp-small">192.0.2.77</span></td><td>dev@<wbr>lists.example.org</td><td class="sp-clip">Re: [dev] patch series v3</td><td><span class="tag is-success is-light">accept</span></td><td class="sp-muted sp-small">rspamd · score 0.4</td><td class="sp-actions"><a href="in-filter-lists.html">Deny sender</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:59:02</td><td>invoice@<wbr>partner-billing.example<br><span class="sp-mono sp-muted sp-small">203.0.113.140</span></td><td>owner@<wbr>lists.example.org</td><td class="sp-clip">Outstanding invoice — action required</td><td><span class="tag is-primary is-light">quarantine</span></td><td class="sp-muted sp-small">rspamd · score 9.8 · PHISHING, FORGED_SENDER</td><td class="sp-actions"><a href="in-quarantine-message.html">Review</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:57:41</td><td>promo@<wbr>bulk.example<br><span class="sp-mono sp-muted sp-small">198.51.100.90</span></td><td>sales@<wbr>lists.example.org</td><td class="sp-clip sp-muted">—</td><td><span class="tag is-danger is-light">reject</span></td><td class="sp-muted sp-small">postfix · 550 5.1.1 unknown recipient</td><td class="sp-actions"><a href="in-filter-lists.html">Deny sender</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:51:30</td><td>billing@<wbr>partner.example<br><span class="sp-mono sp-muted sp-small">198.51.100.12</span></td><td>owner@<wbr>lists.example.org</td><td class="sp-clip">Invoice 2026-0917</td><td><span class="tag is-danger is-light">reject</span></td><td class="sp-muted sp-small">rspamd · score 16.2 · BAYES_SPAM</td><td class="sp-actions"><a href="in-filter-lists.html">Allow sender</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:44:02</td><td>hr@<wbr>partner.example<br><span class="sp-mono sp-muted sp-small">192.0.2.31</span></td><td>jobs@<wbr>acme.io</td><td class="sp-clip">Application: backend engineer</td><td><span class="tag is-warning is-light">tempfail</span></td><td class="sp-muted sp-small">selfpost · upstream 10.0.4.12 timed out</td><td class="sp-actions"></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:30:55</td><td>noreply@<wbr>github.example<br><span class="sp-mono sp-muted sp-small">192.0.2.200</span></td><td>security@<wbr>lists.example.org</td><td class="sp-clip">[advisory] new report</td><td><span class="tag is-success is-light">accept</span></td><td class="sp-muted sp-small">rspamd · allow list</td><td class="sp-actions"></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 14:00:18</td><td>t.berg@<wbr>uni.example<br><span class="sp-mono sp-muted sp-small">192.0.2.77</span></td><td>dev@<wbr>lists.example.org</td><td class="sp-clip">Re: [dev] patch series v3</td><td><button type="button" class="tag is-success is-light" popovertarget="why-1">accept</button><div class="sp-sheet" id="why-1" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-success is-light">accept</span><h2>Why</h2><button type="button" popovertarget="why-1" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Verdict</th><td>no action, score 0.40 / 15.00</td></tr><tr><th>Subject</th><td>Re: [dev] patch series v3</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"><a href="in-filter-lists.html">Deny sender</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:59:02</td><td>invoice@<wbr>partner-billing.example<br><span class="sp-mono sp-muted sp-small">203.0.113.140</span></td><td>owner@<wbr>lists.example.org</td><td class="sp-clip">Outstanding invoice — action required</td><td><button type="button" class="tag is-success is-light" popovertarget="why-2">accept</button><div class="sp-sheet" id="why-2" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-success is-light">accept</span><h2>Why</h2><button type="button" popovertarget="why-2" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Verdict</th><td>add header, score 9.80 / 15.00</td></tr><tr><th>Symbols</th><td class="sp-mono sp-small">PHISHING(5.00)<br>FORGED_SENDER(3.00)<br>R_SPF_FAIL(1.00)</td></tr><tr><th>Subject</th><td>Outstanding invoice — action required</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"><a href="in-filter-lists.html">Deny sender</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:57:41</td><td>promo@<wbr>bulk.example<br><span class="sp-mono sp-muted sp-small">198.51.100.90</span></td><td>sales@<wbr>lists.example.org</td><td class="sp-clip sp-muted">—</td><td><button type="button" class="tag is-danger is-light" popovertarget="why-3">reject</button><div class="sp-sheet" id="why-3" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-danger is-light">reject</span><h2>Why</h2><button type="button" popovertarget="why-3" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>postfix</td></tr><tr><th>Reply</th><td class="sp-mono sp-small">550 5.1.1 &lt;sales@lists.example.org&gt;: Recipient address rejected: User unknown in relay recipient table</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:51:30</td><td>billing@<wbr>partner.example<br><span class="sp-mono sp-muted sp-small">198.51.100.12</span></td><td>owner@<wbr>lists.example.org</td><td class="sp-clip sp-muted">—</td><td><button type="button" class="tag is-danger is-light" popovertarget="why-4">reject</button><div class="sp-sheet" id="why-4" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-danger is-light">reject</span><h2>Why</h2><button type="button" popovertarget="why-4" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Reply</th><td class="sp-mono sp-small">5.7.1 Spam message rejected</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"><a href="in-filter-lists.html">Allow sender</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:44:02</td><td>hr@<wbr>partner.example<br><span class="sp-mono sp-muted sp-small">192.0.2.31</span></td><td>jobs@<wbr>acme.io</td><td class="sp-clip sp-muted">—</td><td><button type="button" class="tag is-warning is-light" popovertarget="why-5">tempfail</button><div class="sp-sheet" id="why-5" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-warning is-light">tempfail</span><h2>Why</h2><button type="button" popovertarget="why-5" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Reply</th><td class="sp-mono sp-small">4.7.1 Try again later</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"><a href="in-filter-lists.html">Allow sender</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:30:55</td><td>noreply@<wbr>github.example<br><span class="sp-mono sp-muted sp-small">192.0.2.200</span></td><td>security@<wbr>lists.example.org</td><td class="sp-clip">[advisory] new report</td><td><button type="button" class="tag is-success is-light" popovertarget="why-6">accept</button><div class="sp-sheet" id="why-6" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-success is-light">accept</span><h2>Why</h2><button type="button" popovertarget="why-6" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Verdict</th><td>no action, score -98.10 / 15.00</td></tr><tr><th>Symbols</th><td class="sp-mono sp-small">SELFPOST_ALLOW_FROM(-100.00)<br>R_DKIM_ALLOW(-0.20)</td></tr><tr><th>Subject</th><td>[advisory] new report</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"><a href="in-filter-lists.html">Deny sender</a></td></tr>
</tbody>
</table></div>
<div class="sp-box-foot"><span>Page 1 of 38</span><a href="#">Older →</a><span class="sp-end">Allow / Deny sender opens <a href="in-filter-lists.html">Filter lists</a> with the entry filled in · global administrator only</span></div>
<div class="sp-box-foot"><span>Page 1 of 38</span><a href="#">Older →</a><span class="sp-end">A decision opens its reason · Allow / Deny sender opens <a href="in-filter-lists.html">Filter lists</a> with the entry filled in</span></div>
</div>
<footer class="sp-foot">SelfPost 1.8.0 · © 2026 Mix · <a href="#">License (AGPL-3.0)</a> · <a href="#">Source</a> · No warranty</footer>
@@ -123,7 +123,7 @@
"is-9": [
"box[Overview]:help×3",
"box[Outbound]:help×6",
"box[Inbound]:help×4",
"box[Inbound]:help×6",
"box[Server]:help×3",
"box[Account]:help"
]
@@ -82,6 +82,9 @@
<div class="field"><label class="label" for="ret">Keep outbound log rows, days</label>
<div class="control"><input class="input" id="ret" name="send_log_retention_days" type="number" min="7" max="365" value="30" required></div>
<p class="help">The main driver of <code>/data</code> growth. Does not affect <code>mail.log</code> rotation (fourteen daily files).</p></div>
<div class="field"><label class="label" for="inret">Keep inbound log rows, days</label>
<div class="control"><input class="input" id="inret" name="inbound_spam_log_retention_days" type="number" min="7" max="365" value="90" required></div>
<p class="help">The inbound log also never holds more than 10 000 rows: refused mail is logged at the rate strangers knock on port 25.</p></div>
</div>
</div>
</div>
@@ -48,6 +48,7 @@
<tr><td>error, fail, mismatch, bounced</td><td><span class="tag is-danger is-light">fail</span></td><td class="sp-mono sp-small">tag is-danger is-light</td></tr>
<tr><td>unknown, off, neutral counts</td><td><span class="tag is-light">off</span></td><td class="sp-mono sp-small">tag is-light</td></tr>
<tr><td>held: quarantine · role: global</td><td><span class="tag is-primary is-light">quarantine</span></td><td class="sp-mono sp-small">tag is-primary is-light</td></tr>
<tr><td>a tag that opens a <code>sheet</code></td><td><button type="button" class="tag is-danger is-light" popovertarget="kit-sheet">reject</button><div class="sp-sheet" id="kit-sheet" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-danger is-light">reject</span><h2>sheet</h2><button type="button" popovertarget="kit-sheet" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>What it holds</th><td>What does not fit the row, as name and value</td></tr><tr><th>How it closes</th><td class="sp-mono sp-small">Esc<br>a click outside<br>the cross</td></tr></tbody></table></div></div></div></div></td><td class="sp-mono sp-small">button.tag + sp-sheet (popover)</td></tr>
</tbody>
</table></div>
</div>
@@ -31,6 +31,8 @@
<li><a href="#upstream"><i class="ti ti-arrow-forward-up"></i>Upstream</a></li>
<li><a href="#recipients"><i class="ti ti-address-book"></i>Valid recipients</a></li>
<li><a href="#filter"><i class="ti ti-shield-check"></i>Spam filter</a></li>
<li><a href="#inbound-log"><i class="ti ti-list-details"></i>Inbound log</a></li>
<li><a href="#filter-lists"><i class="ti ti-filter"></i>Filter lists</a></li>
</ul>
<p class="menu-label">Server</p>
<ul class="menu-list">
@@ -66,12 +68,14 @@
<div class="sp-help-topic" id="upstream"><h3>Upstream</h3><p>The upstream is the server that actually holds the mailboxes. Give its host and port — used exactly as written, with no MX lookup — and choose the TLS for the hop to it: Opportunistic uses TLS when the upstream offers it and otherwise sends in the clear, Required holds the mail until a TLS session is possible, Off never uses TLS. While the upstream does not answer, Postfix keeps the mail and retries as for any other mail.</p></div>
<div class="sp-help-topic" id="recipients"><h3>Valid recipients</h3><p>Choose whom this domain accepts. Listed addresses only: the relay accepts exactly the addresses you list, one per line or comma-separated, and refuses any other while the sender is still connected, so it never has to send a bounce for them. Any recipient: everything at the domain is accepted and passed on, and an address the upstream then rejects comes back as a bounce from this server to a sender who is often forged — backscatter. Prefer a list, and keep it in step with the mailboxes upstream.</p></div>
<div class="sp-help-topic" id="filter"><h3>Spam filter</h3><p>The spam filter belongs to the server, not to a domain. The server's administrator attaches it in the server's <code>.env</code>, and from then on every inbound domain is screened by it before mail is handed on; the box shows on or off and has nothing to switch. Off means mail goes to the upstream unfiltered. While the filter is down, mail is either accepted unfiltered (the default) or deferred, so the sending server tries again later, as the server's setting says; the box states which.</p></div>
<div class="sp-help-topic" id="inbound-log"><h3>Inbound log</h3><p>One row per recipient of a message offered on port 25 for an inbound domain, with what was decided: <span class="tag is-success is-light">accept</span> — handed on to the upstream; <span class="tag is-danger is-light">reject</span> — refused for good; <span class="tag is-warning is-light">tempfail</span> — the sender was asked to come back later; <span class="tag is-primary is-light">quarantine</span> — held by the filter. <em>Why</em> names who decided. <code>rspamd</code> is the spam filter, with its action, the score against the reject threshold and the heaviest symbols; <code>postfix</code> is the relay's own rule — an unknown recipient, a sender domain that does not exist — with its reply; <code>selfpost</code> means nobody judged the message: it was relayed with no filter attached. Accept says what this server did, not what the upstream did with the message afterwards.<br><br>Accepted mail is recorded as it passes; refused mail is read from <code>mail.log</code>, so its row has no subject, and a message refused after its text was sent is one row however many recipients it had. A refusal to relay for a domain that is not an inbound one is every scanner on the internet and is not recorded. Rows are kept for the number of days set in Settings, and never more than 10 000.</p></div>
<div class="sp-help-topic" id="filter-lists"><h3>Filter lists</h3><p>Two lists for the whole server that correct the spam filter by hand. An entry is a sender address, a sender domain — that domain, not its subdomains — or a client address or network in CIDR form. A sender on the Deny list is refused outright. A sender on the Allow list is given a weight no spam score outweighs, so the filter lets it through. When entries of both lists match one message, <strong>Deny wins</strong>.<br><br>SelfPost does not apply the lists itself: it writes them as files under <code>/data/antispam/</code>, and rspamd rereads a file within seconds of a change. With the spam filter off the lists are kept and can be edited, but nothing reads them. The sender matched is the envelope sender — the one in the log — not the From line a mail program shows. Matched counts the messages in the Inbound log that the entry decided, so it reaches back as far as the log does; never means not once in that time, and such an entry is one to consider removing.</p></div>
</div>
<div class="box">
<div class="sp-box-head"><span class="sp-no">04</span><h2>Server</h2></div>
<div class="sp-help-topic" id="backup"><h3>Backup and moving a domain</h3><p>Full backup downloads one archive of the whole instance: <code>data/</code> (the database, DKIM keys and application credentials), <code>docker-compose.yml</code>, <code>.env</code> and <code>certs/</code>. The Postfix queue is not in it. The backup is taken while the server is running, and Postfix keeps moving messages through the queue, so a copy could hold a message twice or miss it; mail still waiting for delivery is therefore not carried over, and a restored server starts with an empty queue. To keep the queue, stop the container and archive the project directory instead. The private key in <code>certs/</code> has to be readable by the panel: give it to group 998 on the host (<code>chgrp 998</code>, <code>chmod 640</code>). Until then the panel refuses to make a backup and names the file, rather than hand over an archive without the key. It holds secrets, so leave Encrypt with a password ticked: the file is sealed as <code>.spbk</code>, the password is not stored, and without it nothing can be recovered. To restore, unpack it into an empty project directory (decrypt a <code>.spbk</code> first with <code>selfpost-backup -decrypt</code>) and start the same SelfPost version that made it; the panel refuses data from another version. The reverse-proxy configuration and <code>mail.log</code> are not included. One domain moves on its own: Export domain on its settings page writes a <code>.json</code> (or encrypted <code>.spde</code>) with its DKIM key and its applications' passwords, and Import a domain here reads it into another instance, so the published DNS needs no change.</p></div>
<div class="sp-help-topic" id="users"><h3>Users and roles</h3><p>The two roles are a reach, not a rank: every user is an administrator and signs in with a username and password. Global reaches the whole server — Overview, Server, the queue and every domain — and is the only role that adds or deletes domains and users. Domain reaches only the domains assigned to it, outbound and inbound separately (the same name on both lists is two assignments); <em>All</em> includes domains added later, and a domain user needs at least one domain or <em>All</em> on either list. The only global user cannot be demoted or deleted, and you cannot delete the account you are signed in with. A new password set here signs that user out of their other sessions.</p></div>
<div class="sp-help-topic" id="settings"><h3>Settings</h3><p>Settings holds what is true of the whole instance. Log retention is how many days the Outbound log keeps its rows, from 7 to 365 and 90 by default; a sweep every six hours deletes older ones, and it is the main driver of how fast <code>/data</code> grows. It does not change the rotation of <code>mail.log</code>, which is daily with about two weeks kept. The sending rate limit shown under it is level 1, the per-client-IP ceiling set in <code>.env</code>: changing it needs a restart, and no level-2 limit can exceed it.</p></div>
<div class="sp-help-topic" id="settings"><h3>Settings</h3><p>Settings holds what is true of the whole instance. Log retention is how many days the Outbound log keeps its rows, from 7 to 365 and 90 by default; a sweep every six hours deletes older ones, and it is the main driver of how fast <code>/data</code> grows. With the inbound relay on there is a second number, for the Inbound log, with the same range. It does not change the rotation of <code>mail.log</code>, which is daily with about two weeks kept. The sending rate limit shown under it is level 1, the per-client-IP ceiling set in <code>.env</code>: changing it needs a restart, and no level-2 limit can exceed it.</p></div>
</div>
<div class="box">
<div class="sp-box-head"><span class="sp-no">05</span><h2>Account</h2></div>
@@ -8,7 +8,7 @@
</div>
<div class="box">
<div class="sp-box-head"><span class="sp-no">01</span><h2>Add an entry</h2><a class="sp-help" href="help.html#inbound" title="How the lists are applied"><i class="ti ti-help-circle"></i></a></div>
<div class="sp-box-head"><span class="sp-no">01</span><h2>Add an entry</h2><a class="sp-help" href="help.html#filter-lists" title="How the lists are applied"><i class="ti ti-help-circle"></i></a></div>
<div class="sp-box-body">
<form class="sp-form" method="post" action="#">
<div class="columns">
@@ -18,7 +18,7 @@
</div>
<div class="column is-3">
<div class="field"><label class="label" for="etype">Entry is</label>
<div class="control"><div class="select is-fullwidth"><select id="etype" name="entry_type"><option>a sender address</option><option>a sender domain</option><option>a client IP or CIDR</option></select></div></div></div>
<div class="control"><div class="select is-fullwidth"><select id="etype" name="entry_type"><option>a sender address</option><option>a sender domain</option><option>a client address or network</option></select></div></div></div>
</div>
<div class="column is-6">
<div class="field"><label class="label" for="value">Value</label>
@@ -37,11 +37,11 @@
<div class="box">
<div class="sp-box-head"><span class="sp-no">02</span><h2>Deny</h2><div class="sp-end">3 entries · wins over Allow</div></div>
<div class="table-container"><table class="table is-fullwidth is-hoverable">
<thead><tr><th>Value</th><th>Note</th><th>Added</th><th></th></tr></thead>
<thead><tr><th>Value</th><th>Kind</th><th>Matched</th><th></th></tr></thead>
<tbody>
<tr><td class="sp-mono sp-small">@bulk.example <span class="tag is-light">domain</span></td><td class="sp-muted">newsletter nobody subscribed to</td><td class="sp-muted sp-nowrap sp-small">2026-09-02</td><td class="sp-actions"><a class="has-text-danger" href="#">Remove</a></td></tr>
<tr><td class="sp-mono sp-small">198.51.100.0/24 <span class="tag is-light">ip</span></td><td class="sp-muted">dictionary attack on lists.example.org</td><td class="sp-muted sp-nowrap sp-small">2026-08-19</td><td class="sp-actions"><a class="has-text-danger" href="#">Remove</a></td></tr>
<tr><td class="sp-mono sp-small">win@lottery.example <span class="tag is-light">address</span></td><td class="sp-muted">—</td><td class="sp-muted sp-nowrap sp-small">2026-07-30</td><td class="sp-actions"><a class="has-text-danger" href="#">Remove</a></td></tr>
<tr><td class="sp-mono sp-small">bulk.example</td><td><button type="button" class="tag is-light" popovertarget="deny-1">domain</button><div class="sp-sheet" id="deny-1" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-light">domain</span><h2>bulk.example</h2><button type="button" popovertarget="deny-1" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Note</th><td>newsletter nobody subscribed to</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-09-02</td></tr><tr><th>Matched</th><td>41 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-09-21 13:57</td></tr></tbody></table></div></div></div></div></td><td class="sp-nowrap">41</td><td class="sp-actions"><form method="post" action="#"><button type="submit" class="has-text-danger">Remove</button></form></td></tr>
<tr><td class="sp-mono sp-small">198.51.100.0/24</td><td><button type="button" class="tag is-light" popovertarget="deny-2">ip</button><div class="sp-sheet" id="deny-2" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-light">ip</span><h2>198.51.100.0/24</h2><button type="button" popovertarget="deny-2" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Note</th><td>dictionary attack on lists.example.org</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-08-19</td></tr><tr><th>Matched</th><td>1203 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-09-21 09:12</td></tr></tbody></table></div></div></div></div></td><td class="sp-nowrap">1203</td><td class="sp-actions"><form method="post" action="#"><button type="submit" class="has-text-danger">Remove</button></form></td></tr>
<tr><td class="sp-mono sp-small">win@lottery.example</td><td><button type="button" class="tag is-light" popovertarget="deny-3">address</button><div class="sp-sheet" id="deny-3" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-light">address</span><h2>win@lottery.example</h2><button type="button" popovertarget="deny-3" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Note</th><td>—</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-07-30</td></tr><tr><th>Matched</th><td>never in the last 90 days</td></tr></tbody></table></div></div></div></div></td><td class="sp-muted sp-nowrap">never</td><td class="sp-actions"><form method="post" action="#"><button type="submit" class="has-text-danger">Remove</button></form></td></tr>
</tbody>
</table></div>
</div>
@@ -50,10 +50,10 @@
<div class="box">
<div class="sp-box-head"><span class="sp-no">03</span><h2>Allow</h2><div class="sp-end">2 entries</div></div>
<div class="table-container"><table class="table is-fullwidth is-hoverable">
<thead><tr><th>Value</th><th>Note</th><th>Added</th><th></th></tr></thead>
<thead><tr><th>Value</th><th>Kind</th><th>Matched</th><th></th></tr></thead>
<tbody>
<tr><td class="sp-mono sp-small">@partner.example <span class="tag is-light">domain</span></td><td class="sp-muted">invoices were scored as spam</td><td class="sp-muted sp-nowrap sp-small">2026-09-11</td><td class="sp-actions"><a class="has-text-danger" href="#">Remove</a></td></tr>
<tr><td class="sp-mono sp-small">noreply@github.example <span class="tag is-light">address</span></td><td class="sp-muted">security advisories</td><td class="sp-muted sp-nowrap sp-small">2026-06-04</td><td class="sp-actions"><a class="has-text-danger" href="#">Remove</a></td></tr>
<tr><td class="sp-mono sp-small">partner.example</td><td><button type="button" class="tag is-light" popovertarget="allow-1">domain</button><div class="sp-sheet" id="allow-1" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-light">domain</span><h2>partner.example</h2><button type="button" popovertarget="allow-1" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Note</th><td>invoices were scored as spam</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-09-11</td></tr><tr><th>Matched</th><td>7 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-09-19 16:40</td></tr></tbody></table></div></div></div></div></td><td class="sp-nowrap">7</td><td class="sp-actions"><form method="post" action="#"><button type="submit" class="has-text-danger">Remove</button></form></td></tr>
<tr><td class="sp-mono sp-small">noreply@github.example</td><td><button type="button" class="tag is-light" popovertarget="allow-2">address</button><div class="sp-sheet" id="allow-2" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-light">address</span><h2>noreply@github.example</h2><button type="button" popovertarget="allow-2" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Note</th><td>security advisories</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-06-04</td></tr><tr><th>Matched</th><td>18 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-09-21 13:30</td></tr></tbody></table></div></div></div></div></td><td class="sp-nowrap">18</td><td class="sp-actions"><form method="post" action="#"><button type="submit" class="has-text-danger">Remove</button></form></td></tr>
</tbody>
</table></div>
</div>
@@ -64,10 +64,10 @@
<div class="sp-box-head"><span class="sp-no">04</span><h2>Where the lists go</h2><div class="sp-end"><span class="tag is-success is-light">applied</span></div></div>
<div class="sp-box-body">
<dl class="sp-facts">
<div><dt>Applied by</dt><dd>rspamd, from map files</dd></div>
<div><dt>Files</dt><dd class="sp-mono sp-small">/data/antispam/allow.map<br>/data/antispam/deny.map</dd></div>
<div><dt>Applied by</dt><dd>rspamd, from map files it rereads within seconds of a change</dd></div>
<div><dt>Files</dt><dd><span class="sp-mono">/data/antispam/</span> — six, one for each list and kind of entry</dd></div>
<div><dt>Last written</dt><dd>2026-09-11 08:40 UTC</dd></div>
<div><dt>When the filter is off</dt><dd class="sp-muted">The lists are kept and editable, but nothing reads them until <code>INBOUND_ANTISPAM_MILTER</code> is set.</dd></div>
<div><dt>When both lists match</dt><dd>Deny wins, whichever entries they are.</dd></div>
</dl>
</div>
</div>
@@ -3,7 +3,7 @@
<div class="sp-grow">
<p class="sp-kicker">Inbound</p>
<h1 class="title is-3">Inbound log</h1>
<p class="sp-lead">What port 25 decided about each message, one row per recipient. Accepts come from the inbound journal, rejects from <code>mail.log</code>; the filter's reasons appear when the spam filter is on.</p>
<p class="sp-lead">What port 25 decided about each message, one row per recipient. Accepted mail is recorded as it passes, refused mail is read from <code>mail.log</code>; the filter's verdict appears when the spam filter is on. Rows older than 90 days are deleted — change that in <a href="settings.html">Settings</a>.</p>
</div>
</div>
@@ -17,17 +17,18 @@
<input class="input" type="date" name="to" value="2026-09-21" aria-label="To date">
<button type="submit" class="button"><i class="ti ti-filter"></i>Filter</button>
</form>
<a class="sp-help" href="help.html#inbound-log" title="What the decisions mean"><i class="ti ti-help-circle"></i></a>
</div>
<div class="table-container"><table class="table is-fullwidth is-hoverable">
<thead><tr><th>Time, UTC</th><th>From · client</th><th>To</th><th>Subject</th><th>Decision</th><th>Why</th><th></th></tr></thead>
<thead><tr><th>Time, UTC</th><th>From</th><th>To</th><th>Subject</th><th>Decision</th><th></th></tr></thead>
<tbody>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 14:00:18</td><td>t.berg@<wbr>uni.example<br><span class="sp-mono sp-muted sp-small">192.0.2.77</span></td><td>dev@<wbr>lists.example.org</td><td class="sp-clip">Re: [dev] patch series v3</td><td><span class="tag is-success is-light">accept</span></td><td class="sp-muted sp-small">rspamd · score 0.4</td><td class="sp-actions"><a href="in-filter-lists.html">Deny sender</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:59:02</td><td>invoice@<wbr>partner-billing.example<br><span class="sp-mono sp-muted sp-small">203.0.113.140</span></td><td>owner@<wbr>lists.example.org</td><td class="sp-clip">Outstanding invoice — action required</td><td><span class="tag is-primary is-light">quarantine</span></td><td class="sp-muted sp-small">rspamd · score 9.8 · PHISHING, FORGED_SENDER</td><td class="sp-actions"><a href="in-quarantine-message.html">Review</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:57:41</td><td>promo@<wbr>bulk.example<br><span class="sp-mono sp-muted sp-small">198.51.100.90</span></td><td>sales@<wbr>lists.example.org</td><td class="sp-clip sp-muted">—</td><td><span class="tag is-danger is-light">reject</span></td><td class="sp-muted sp-small">postfix · 550 5.1.1 unknown recipient</td><td class="sp-actions"><a href="in-filter-lists.html">Deny sender</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:51:30</td><td>billing@<wbr>partner.example<br><span class="sp-mono sp-muted sp-small">198.51.100.12</span></td><td>owner@<wbr>lists.example.org</td><td class="sp-clip">Invoice 2026-0917</td><td><span class="tag is-danger is-light">reject</span></td><td class="sp-muted sp-small">rspamd · score 16.2 · BAYES_SPAM</td><td class="sp-actions"><a href="in-filter-lists.html">Allow sender</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:44:02</td><td>hr@<wbr>partner.example<br><span class="sp-mono sp-muted sp-small">192.0.2.31</span></td><td>jobs@<wbr>acme.io</td><td class="sp-clip">Application: backend engineer</td><td><span class="tag is-warning is-light">tempfail</span></td><td class="sp-muted sp-small">selfpost · upstream 10.0.4.12 timed out</td><td class="sp-actions"></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:30:55</td><td>noreply@<wbr>github.example<br><span class="sp-mono sp-muted sp-small">192.0.2.200</span></td><td>security@<wbr>lists.example.org</td><td class="sp-clip">[advisory] new report</td><td><span class="tag is-success is-light">accept</span></td><td class="sp-muted sp-small">rspamd · allow list</td><td class="sp-actions"></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 14:00:18</td><td>t.berg@<wbr>uni.example<br><span class="sp-mono sp-muted sp-small">192.0.2.77</span></td><td>dev@<wbr>lists.example.org</td><td class="sp-clip">Re: [dev] patch series v3</td><td><button type="button" class="tag is-success is-light" popovertarget="why-1">accept</button><div class="sp-sheet" id="why-1" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-success is-light">accept</span><h2>Why</h2><button type="button" popovertarget="why-1" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Verdict</th><td>no action, score 0.40 / 15.00</td></tr><tr><th>Subject</th><td>Re: [dev] patch series v3</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"><a href="in-filter-lists.html">Deny sender</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:59:02</td><td>invoice@<wbr>partner-billing.example<br><span class="sp-mono sp-muted sp-small">203.0.113.140</span></td><td>owner@<wbr>lists.example.org</td><td class="sp-clip">Outstanding invoice — action required</td><td><button type="button" class="tag is-success is-light" popovertarget="why-2">accept</button><div class="sp-sheet" id="why-2" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-success is-light">accept</span><h2>Why</h2><button type="button" popovertarget="why-2" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Verdict</th><td>add header, score 9.80 / 15.00</td></tr><tr><th>Symbols</th><td class="sp-mono sp-small">PHISHING(5.00)<br>FORGED_SENDER(3.00)<br>R_SPF_FAIL(1.00)</td></tr><tr><th>Subject</th><td>Outstanding invoice — action required</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"><a href="in-filter-lists.html">Deny sender</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:57:41</td><td>promo@<wbr>bulk.example<br><span class="sp-mono sp-muted sp-small">198.51.100.90</span></td><td>sales@<wbr>lists.example.org</td><td class="sp-clip sp-muted">—</td><td><button type="button" class="tag is-danger is-light" popovertarget="why-3">reject</button><div class="sp-sheet" id="why-3" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-danger is-light">reject</span><h2>Why</h2><button type="button" popovertarget="why-3" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>postfix</td></tr><tr><th>Reply</th><td class="sp-mono sp-small">550 5.1.1 &lt;sales@lists.example.org&gt;: Recipient address rejected: User unknown in relay recipient table</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:51:30</td><td>billing@<wbr>partner.example<br><span class="sp-mono sp-muted sp-small">198.51.100.12</span></td><td>owner@<wbr>lists.example.org</td><td class="sp-clip sp-muted">—</td><td><button type="button" class="tag is-danger is-light" popovertarget="why-4">reject</button><div class="sp-sheet" id="why-4" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-danger is-light">reject</span><h2>Why</h2><button type="button" popovertarget="why-4" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Reply</th><td class="sp-mono sp-small">5.7.1 Spam message rejected</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"><a href="in-filter-lists.html">Allow sender</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:44:02</td><td>hr@<wbr>partner.example<br><span class="sp-mono sp-muted sp-small">192.0.2.31</span></td><td>jobs@<wbr>acme.io</td><td class="sp-clip sp-muted">—</td><td><button type="button" class="tag is-warning is-light" popovertarget="why-5">tempfail</button><div class="sp-sheet" id="why-5" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-warning is-light">tempfail</span><h2>Why</h2><button type="button" popovertarget="why-5" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Reply</th><td class="sp-mono sp-small">4.7.1 Try again later</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"><a href="in-filter-lists.html">Allow sender</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:30:55</td><td>noreply@<wbr>github.example<br><span class="sp-mono sp-muted sp-small">192.0.2.200</span></td><td>security@<wbr>lists.example.org</td><td class="sp-clip">[advisory] new report</td><td><button type="button" class="tag is-success is-light" popovertarget="why-6">accept</button><div class="sp-sheet" id="why-6" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-success is-light">accept</span><h2>Why</h2><button type="button" popovertarget="why-6" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Verdict</th><td>no action, score -98.10 / 15.00</td></tr><tr><th>Symbols</th><td class="sp-mono sp-small">SELFPOST_ALLOW_FROM(-100.00)<br>R_DKIM_ALLOW(-0.20)</td></tr><tr><th>Subject</th><td>[advisory] new report</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"><a href="in-filter-lists.html">Deny sender</a></td></tr>
</tbody>
</table></div>
<div class="sp-box-foot"><span>Page 1 of 38</span><a href="#">Older →</a><span class="sp-end">Allow / Deny sender opens <a href="in-filter-lists.html">Filter lists</a> with the entry filled in · global administrator only</span></div>
<div class="sp-box-foot"><span>Page 1 of 38</span><a href="#">Older →</a><span class="sp-end">A decision opens its reason · Allow / Deny sender opens <a href="in-filter-lists.html">Filter lists</a> with the entry filled in</span></div>
</div>
@@ -16,6 +16,9 @@
<div class="field"><label class="label" for="ret">Keep outbound log rows, days</label>
<div class="control"><input class="input" id="ret" name="send_log_retention_days" type="number" min="7" max="365" value="30" required></div>
<p class="help">The main driver of <code>/data</code> growth. Does not affect <code>mail.log</code> rotation (fourteen daily files).</p></div>
<div class="field"><label class="label" for="inret">Keep inbound log rows, days</label>
<div class="control"><input class="input" id="inret" name="inbound_spam_log_retention_days" type="number" min="7" max="365" value="90" required></div>
<p class="help">The inbound log also never holds more than 10 000 rows: refused mail is logged at the rate strangers knock on port 25.</p></div>
</div>
</div>
</div>
@@ -270,3 +270,27 @@ code { background: var(--sp-well); color: var(--sp-ink); border-radius: 3px; pad
.sp-record-head small { margin-left: 0; text-align: left; flex-basis: 100%; }
.sp-postmark { display: none; }
}
/* sheet: what does not fit a table row, opened over the page from the row's
tag. It is the browser's own popover (the tag is a button with
popovertarget), so it needs no script: Esc, a click outside and the close
button all shut it, and focus returns to the tag. It rises from the bottom
edge across the whole window - the one component that is not laid out by the
page, hence the widths - with its content, a table of name and value, in a
column of reading width in the middle. */
.sp-sheet {
inset: auto 0 0 0; width: auto; margin: 0; padding: 0; border: 0; border-top: 3px solid var(--sp-brick);
max-height: 75vh; overflow: auto; background: #fff; color: inherit; box-shadow: 0 -12px 40px rgba(40, 20, 15, .22);
}
.sp-sheet-in { max-width: 44rem; margin: 0 auto; padding: 0 1.25rem; }
.sp-sheet::backdrop { background: rgba(40, 20, 15, .38); }
.sp-sheet-head { display: flex; align-items: center; gap: .7rem; padding: 1rem 0 .8rem; border-bottom: 1px solid var(--sp-rule); }
.sp-sheet-head h2 { font-size: 1.15rem; font-weight: 600; margin-right: auto; }
.sp-sheet-head button { border: 0; background: none; cursor: pointer; font-size: 1.3rem; color: var(--sp-muted); }
.sp-sheet-body { padding: .4rem 0 1.4rem; }
/* The sheet's content is a table of name and value: the name is a label, set like the dt of facts. */
.sp-sheet .table th { font-family: var(--sp-mono); font-size: var(--sp-fs-xs); font-weight: 500; letter-spacing: .08em; text-transform: uppercase; color: var(--sp-muted); white-space: nowrap; vertical-align: top; width: 1%; padding-right: 2rem; }
.sp-sheet .table td { overflow-wrap: anywhere; }
/* A tag that opens a sheet is a button: it keeps the tag's look and says it can
be pressed with the dotted underline links to more detail carry. */
button.tag { border: 0; cursor: pointer; text-decoration: underline dotted; text-underline-offset: 3px; }
+45
View File
@@ -209,6 +209,8 @@ expected to set; defaults match the code exactly.
| `DMARC_MESSAGE_SIZE_LIMIT` | Maximum report message size in bytes when DMARC ingest is on. | `5242880` (5 MiB) | `.env` |
| `INBOUND_ANTISPAM_MILTER` | Optional milter on the inbound relay listener only (requires `INBOUND_RELAY_ENABLE=true`; not 465/587 or DMARC-only port 25). Empty = off. Format `inet:host:port` or `unix:/path`. Example with [deploy/antispam/docker-compose.antispam.yml](../deploy/antispam/docker-compose.antispam.yml): `inet:antispam:11332`. The panel reads it to report on the *Spam filter* box of the inbound pages and to check that the filter answers (Server › Health). | *(empty)* | `.env` |
| `INBOUND_ANTISPAM_MILTER_ACTION` | What Postfix does if that milter is down: `accept` (fail-open) or `tempfail` (defer). The *Spam filter* box states which. | `accept` | `.env` |
| `INBOUND_SPAM_LOG_RETENTION_DAYS` | How long the inbound journal keeps a row, 7-365 days; it also never holds more than 10 000 rows. Seeds the setting on the first start with the inbound relay on. | `90` | `.env` |
| `INBOUND_JOURNAL_MILTER_SOCKET` | Unix socket of the panel's inbound journal milter, the last milter on port 25 while the inbound relay is on. It records accepted inbound mail and never refuses any; there is no reason to change it. | `/run/selfpost/inbound-journal.sock` | compose |
| `INBOUND_RATE_LIMIT_MESSAGES_PER_IP` | Coarse per-client-IP cap on inbound smtpd (`smtpd_client_message_rate_limit`). Uses the same window as `RATE_LIMIT_WINDOW_SECONDS`. | `20` | `.env` |
| `INBOUND_MESSAGE_SIZE_LIMIT` | Maximum message size in bytes on inbound smtpd (`message_size_limit`). When both inbound relay and DMARC ingest are on, the shared listener uses the **greater** of this and `DMARC_MESSAGE_SIZE_LIMIT`. | `26214400` (25 MiB) | `.env` |
| `RATE_LIMIT_MESSAGES_PER_IP` | Level-1 backstop: maximum messages one client IP may submit per window (Postfix `smtpd_client_message_rate_limit`). See [Rate limiting — level 1](#rate-limiting--level-1-ip-backstop). | `100` | `.env` |
@@ -710,6 +712,15 @@ needed) for the new hostname or proxy, then `docker compose up -d`. The archive
carries `certs/` from the old host — re-issue certificates when the hostname or
IP changes. Set up the reverse-proxy vhost separately (not in the backup).
The inbound log and the filter lists are in the database and so in the
backup, and so are the list files under `data/antispam/`; the panel writes
those files again from the database on the first start after a restore. What
the backup does not hold is the spam filter's own side: the `antispam/`
directory with the compose fragment and the rspamd configuration. Like the
reverse-proxy fragments it is part of the installation, not of its state -
copy it from [deploy/antispam/](../deploy/antispam/) again, or the restored
instance runs with no filter.
**Restoring an encrypted (`.spbk`) backup** needs a running container to
decrypt it first — any container with the `selfpost-backup` CLI works; decryption
does not read `/data` and performs no version check. Start one normally
@@ -841,6 +852,40 @@ only sees SelfPost. Default action is fail-open (`accept`) so a down sidecar
does not block backup-MX; set `INBOUND_ANTISPAM_MILTER_ACTION=tempfail` to
defer instead.
The fragment mounts one file into the sidecar,
[deploy/antispam/rspamd/milter_headers.conf](../deploy/antispam/rspamd/milter_headers.conf):
it makes rspamd write its verdict - the action and the score with its symbols
- into two headers of every message it lets through. SelfPost records them
with the message, and the upstream receives them with the mail. Copy the
`antispam/rspamd/` directory next to the fragment; without the file accepted
mail is still recorded, with no verdict.
The same directory holds
[multimap.conf](../deploy/antispam/rspamd/multimap.conf), the rules for the
**allow and deny lists** kept in the panel. SelfPost does not apply the lists
itself: it writes them as six files under `data/antispam/` (`allow_ip.map`,
`allow_from.map`, `allow_domain.map` and the three `deny_` ones), the fragment
mounts that directory into the sidecar, and rspamd rereads a file within
seconds of a change - no restart. An entry is a sender address, a sender
domain (that domain, not its subdomains) or a client address or network. A
denied sender is refused outright. An allowed one is given a weight that no
spam score outweighs, so it is accepted - unless a deny entry matches the same
message: **deny wins**, whichever entries they are. The files are readable by
anyone on the host; they hold addresses, not secrets.
In the panel the lists are **Inbound › Filter lists** (global role), and what
port 25 decided about each message is **Inbound › Log**: accepted mail with the
filter's verdict, refused mail with the reply it was given. A row of the log
leads to the lists with its sender filled in. Beside each entry of a list is
how often it decided a message, counted from the log - so over the days the
log keeps - and *never* marks an entry that may have outlived its use. The
count depends on two things in
[multimap.conf](../deploy/antispam/rspamd/multimap.conf): the message of the
deny rules and the names of the allow symbols; change them and every entry
reads *never*. A user with the *domain* role
sees the log of the inbound domains assigned to them, and not the lists. How
long the log keeps a row is set under **Server › Settings**.
The panel only reports this setting; it does not change it. Every inbound
domain page has a **Spam filter** box — on or off, what happens to mail while
the filter is down, and that the filter belongs to the server, not to the
+74 -14
View File
@@ -57,8 +57,9 @@ smtpd_milters = { antispam, inbound-journal }
```
Antispam runs first; inbound-journal runs at end-of-message on the **accept**
path and reads rspamd-added headers (`X-Rspamd-Action`, `X-Rspamd-Score`, symbol
summary when present).
path and reads rspamd-added headers (`X-Rspamd-Action`, and `X-Spamd-Result`
for the score and symbols - rspamd has no `X-Rspamd-Score`; the sidecar adds
both through `deploy/antispam/rspamd/milter_headers.conf`).
**Antispam rejects** do not reach end-of-message on the journal milter. v1 also
tails `mail.log` for inbound `postfix/smtpd` **`milter-reject`** / **`reject`**
@@ -83,6 +84,25 @@ configured.
**Global administrator only** for journal and lists — same as inbound relay. No
domain-admin access in v1.
*As built (2026-10-11):* the lists are the global role's; the **log follows
the delegation of Inbound** that 2.0 introduced after this was written - a
user with the `domain` role sees the rows of the inbound domains assigned to
them and nothing else, without the *Allow / Deny sender* links. That is what
the accepted `in-log` mockup draws (no "global role" in its head). The owner
was told and can turn it back into a global-only page.
**Inbound › Log accepted by the owner on 2026-10-12** ("Этот экран
утверждаем"), from screenshots of the page built from its fixture, after his
changes to it: the sender column is headed *From*; *Why* is not a column but
a sheet over the page, opened by the decision tag, holding a centred table of
who decided, the verdict and symbols or the reply, and the subject whole
(kept on the sheet at his word: the table clips it); a message Postfix itself
refused offers no *Deny sender*. Inbound › Filter lists was redrawn at his word
the same day - kind in a column of its own, the note on the entry's sheet,
matches counted (§ Filter list usage) - and **accepted by him on 2026-10-12**
("Да"), after a deny entry he added in the panel refused a test message with
the list's reply and the journal recorded it.
### Retention
Mirror send-log retention:
@@ -101,13 +121,53 @@ Mirror send-log retention:
| Entry types | sender address, sender domain (`@domain` suffix), client IP / CIDR |
| Precedence | **deny overrides allow** |
| Scope | instance-wide (not per inbound domain in v1) |
| Where applied | **rspamd map files** under `/data/antispam/` (allow.map, deny.map); documented compose fragment mounts them into the sidecar |
| Where applied | **rspamd map files** under `/data/antispam/`, one per list and entry type (`allow_ip.map`, `allow_from.map`, `allow_domain.map`, `deny_…`) - an IP map and a sender map are different map types to rspamd; documented compose fragment mounts them into the sidecar. Deny is a prefilter reject, allow a weight of -100 (as built: two prefilters do not give "deny overrides allow") |
| SelfPost pre-check | **out of v1** — lists live in rspamd only |
| Edit access | global administrator only |
| When inactive | lists editable in the panel but UI warns that sync applies only when `INBOUND_ANTISPAM_MILTER` is set; `postfix reload` / rspamd reload documented in guide |
List change → atomic map rewrite (same pattern as Postfix maps) → rspamd reload
via documented operator step or sidecar `SIGHUP` in the compose fragment.
List change → atomic map rewrite (same pattern as Postfix maps). No reload:
rspamd rereads a changed map file by itself (about ten seconds on 3.11).
### Filter list usage (owner, 2026-10-12)
Each entry shows how often it decided a message (*Matched*, or *never*) and,
on its sheet, when it last did - so that entries nothing matches any more can
be removed. **The count is read off the journal on every view of the page;
nothing is stored for it.** A journal row counts for an entry when the list
decided the message - a rejected row whose reply is the deny rule's message,
an accepted row whose verdict carries the allow rule's symbol
(`SELFPOST_ALLOW_IP` / `_FROM` / `_DOMAIN`, which the journal milter always
keeps) - and the entry fits the row's sender or client address the way rspamd
matches it (`inbound.Lists.Usage`).
What follows from counting this way, and is accepted:
- the count reaches back as far as the journal does: its retention (90 days by
default), which is what the sheet says;
- a row counts for every fitting entry of the deciding list, since the journal
does not say which of two the filter met first;
- a message refused at end-of-message is one row whatever its recipient count,
so a count is of messages, roughly.
**When to move to a counter kept with each entry** (a migration adding hits
and last-hit to `inbound_spam_list`, bumped where the journal row is written;
the owner asked for the condition to be written down). Any one of:
1. **The journal is cut by its 10 000-row cap and not by age.** Then it holds
fewer days than its retention, the sheet's "in the last 90 days" is no
longer true, and *never* stops being evidence that an entry is stale. The
sign: the log tailer's prune line keeps reporting rows removed while the
oldest row of Inbound › Log is younger than the retention.
2. **The retention is set short** (towards its minimum of 7 days) for reasons
of its own, privacy or disk. A week without a match does not make an entry
stale.
3. **Counts are wanted that outlive the journal** - since the entry was added.
4. **The page is slow.** A view reads every journal row a list decided and
tries it against every entry. With the cap that is at most 10 000 rows
times the number of entries; past a few hundred entries, measure it.
Until then no migration is spent on it.
### Activation
@@ -243,14 +303,14 @@ Target version cut: **`2.1.0`** (MINOR), after `2.0.0`. One commit per step;
[development.md](../development.md) § Plan checklists.
- [x] Agree journal fields, retention, and RBAC (this plan § Decisions) — **Composer**
- [ ] Migration `0002_inbound_spam_log.sql` on the 2.0 baseline — **Composer**
- [ ] Inbound journal-milter + Postfix wiring — **Composer**
- [ ] mail.log tailer: inbound reject rows — **Composer**
- [ ] List CRUD + validation + atomic rspamd map sync — **Composer**
- [ ] Panel: Inbound › Log and Inbound › Filter lists from the kit partials, mockups `in-log` / `in-filter-lists` (add client IP and engine columns mockup-first), evidence pairs — **Composer**
- [ ] Backup includes `/data/antispam/` — **Composer**
- [ ] Unit + handler tests — **Composer**
- [ ] [guide.md](../guide.md), [architecture.md](../architecture.md) — **Composer**
- [ ] `go vet`, `go test`, e2e if applicable — **Composer**
- [x] Migration `0002_inbound_spam_log.sql` on the 2.0 baseline — **Composer**
- [x] Inbound journal-milter + Postfix wiring — **Composer**
- [x] mail.log tailer: inbound reject rows — **Composer**
- [x] List CRUD + validation + atomic rspamd map sync — **Composer**
- [x] Panel: Inbound › Log and Inbound › Filter lists from the kit partials, mockups `in-log` / `in-filter-lists` (add client IP and engine columns mockup-first), evidence pairs — **Composer**
- [x] Backup includes `/data/antispam/` — **Composer** (it always did: a backup walks all of `/data`; now held by a test, and the guide says the `antispam/` fragment directory is not in it)
- [x] Unit + handler tests — **Composer** (written with each step: store, milter, tailer, lists and their usage, view fixtures held to the outlines, handlers for both roles)
- [x] [guide.md](../guide.md), [architecture.md](../architecture.md) — **Composer** (kept up with each step; [security.md](../security.md) follows the security review)
- [x] `go vet`, `go test`, e2e if applicable — **Composer** (on the test server for every step; last on `7192c38`: vet ok, 22 packages ok, e2e ok, which relays, refuses, reads the log page and edits a list through the panel)
- [ ] Technical review of inbound antispam changes — **Opus**
- [ ] Security review inbound antispam path — **Fable**
+37 -1
View File
@@ -30,7 +30,7 @@ in `git log` and [CHANGELOG.md](../CHANGELOG.md).
|---|---|---|---|---|
| preflight | Installation check page (`/preflight`) | candidate | — | [plans/preflight.md](plans/preflight.md) |
| contributing | `CONTRIBUTING.md` | candidate | — | — |
| inbound-antispam-panel | Inbound antispam journal + allow/deny lists | agreed | 1/12 | [plans/inbound-antispam-panel.md](plans/inbound-antispam-panel.md) |
| inbound-antispam-panel | Inbound antispam journal + allow/deny lists | in progress | 10/12 | [plans/inbound-antispam-panel.md](plans/inbound-antispam-panel.md) |
| inbound-quarantine | Inbound spam quarantine (hold / review / release) | candidate | — | [plans/inbound-quarantine.md](plans/inbound-quarantine.md) |
| csrf-tokens | Per-session CSRF tokens on state-changing forms | candidate | — | — |
| template-data-typing | Typed structs for template data instead of `map[string]any` | candidate | — | — |
@@ -39,6 +39,7 @@ in `git log` and [CHANGELOG.md](../CHANGELOG.md).
| panel-notifications | E-mail notifications about important events | candidate | — | — |
| delivery-log-storage | Keep a message's `mail.log` lines with its send-log row | candidate | — | — |
| password-reset | Password reset by e-mail | candidate | — | — |
| phone-desktop-mode | A phone browser in "desktop site" mode gets the phone navigation | candidate | — | — |
| panel-redesign | Panel redesign (accepted mockups, design contract) | shipped `2.0.0` | 27/27 | [plans/panel-redesign.md](plans/panel-redesign.md) |
| schema-squash | Squash SQLite migrations into a 2.x baseline | built in panel-redesign (stage 1) | — | [plans/panel-redesign.md](plans/panel-redesign.md) |
@@ -406,6 +407,41 @@ goes and today Account saves it without the password.
---
## phone-desktop-mode
**Problem (owner, 2026-10-11):** the panel opened on a phone with the
browser's "desktop site" switched on does not become the desktop panel.
Seen on Server › Health at `selfpost.mixfed.ru`: the page body is laid out in
columns (Machine and Processes side by side, three cards in the next row),
while the navigation is the phone one - not a bar but a stack with every
section open (Outbound, Inbound, Server, the user menu), filling the first
screen before the page starts. The content also sits flush against the left
edge of the screen.
**Goal:** find out why, then decide what the panel should be in that mode -
the desktop panel, or a phone panel that is at least consistent.
**Where to look first (a guess, not checked):** "desktop site" on Android
lays a page out about 980 px wide. Bulma turns the navigation into a bar at
1024 px and puts columns side by side from 769 px, so 980 px falls between
the two: desktop columns under a touch navigation. `panel.css` has its own
breakpoints at 768, 1023 and 1407 px. Check the width the phone really
reports, why every dropdown is open rather than folded behind the burger,
and where the left padding went.
**Boundary:** the navigation and page gutter between 769 and 1023 px, which
is also a tablet in portrait and a narrow desktop window - the same defect is
expected there. A design change is mockup-first
([plans/panel-redesign.md](plans/panel-redesign.md) § Enforcement).
**Done when:** the cause is written down here, and the panel at that width
has one coherent layout, checked on a phone in "desktop site" mode and in a
980 px window.
**Version:** `2.x` PATCH or MINOR; `candidate`.
---
## panel-redesign
**Goal:** a panel an operator opens a few times a year and still finds their
+12 -9
View File
@@ -17,8 +17,8 @@ When you add or review a migration, update this file in the same change.
| Field | Value |
|---|---|
| Chain head | `user_version = 1` |
| Files | `0001_init.sql` (the 2.0 baseline) |
| Chain head | `user_version = 2` |
| Files | `0001_init.sql` (the 2.0 baseline), `0002_inbound_spam_log.sql` |
| File mark | `PRAGMA application_id = 0x53503230` ("SP20") |
| Database file | `/data/selfpost.db` (bind mount) |
| Compatibility | 2.x MINOR releases must boot a `2.0.0` data directory. **Nothing is carried over from 1.x.** |
@@ -60,10 +60,9 @@ What that means in practice:
**Implication:** once `2.0.0` is released, `0001_init.sql` is never edited
again and no file is deleted, renamed or reordered — every 2.x database has
already run it. Until that release the baseline is still being written:
later steps of the redesign and the features queued behind it (the next one
is `0002_inbound_spam_log.sql` of inbound-antispam-panel, `2.1.0`) add files
after it.
already run it. `2.0.0` and `2.0.1` shipped with the baseline alone; the
features after them add files, the first being `0002_inbound_spam_log.sql` of
inbound-antispam-panel (`2.1.0`).
---
@@ -72,13 +71,14 @@ after it.
| Ver | File | Shipped | Kind | Summary |
|-----|------|---------|------|---------|
| 1 | `0001_init.sql` | 2.0.0 | DDL | The whole 2.0 schema, below |
| 2 | `0002_inbound_spam_log.sql` | 2.1.0 (unreleased) | DDL | `inbound_spam_log`, `inbound_spam_list` and two indexes; nothing existing changes |
**Kind:** *DDL* — schema only; *data* — `INSERT`/`UPDATE` that must stay correct
for operators upgrading from older 2.x images.
---
## Schema at head (v1)
## Schema at head (v2)
| Table | Role |
|-------|------|
@@ -98,9 +98,12 @@ for operators upgrading from older 2.x images.
| `user_inbound_domains` | Inbound domains assigned to a `domain` user — granted separately from the outbound ones |
| `dmarc_reports` | Parsed aggregate report summaries |
| `dmarc_report_records` | Per-source rows inside a report |
| `inbound_spam_log` | Inbound antispam journal: one row per recipient of a message offered on port 25, with the decision (`accept`, `reject`, `tempfail`, `quarantine`) and who made it (`rspamd`, `postfix`, `selfpost`). Metadata only (v2) |
| `inbound_spam_list` | Allow / deny entries for the external filter (`address`, `domain`, `ip`), instance-wide; unique per list (v2) |
`TestBaselineSchema` (`internal/store/schema_test.go`) pins every table, column
and index of this list.
and index of this list; `TestOpenMigratesA20Database` opens a database that
holds the baseline alone.
### What changed against the last 1.x schema
@@ -129,7 +132,7 @@ for the domain. Nothing refers to a user: deleting one changes no domain.
1. Add `internal/store/migrations/00NN_short_name.sql` (next number).
2. Update the table in [Migration chain](#migration-chain), the snapshot at the
top, and the affected rows of [Schema at head](#schema-at-head-v1).
top, and the affected rows of [Schema at head](#schema-at-head-v2).
3. Update `TestBaselineSchema` if tables, columns or indexes change, and add a
test that opens a database at the previous version and migrates it.
4. A *data* migration states in a comment which 2.x versions it is written for.
+5
View File
@@ -42,6 +42,7 @@ func seedDataDir(t *testing.T) (dataDir, dbPath, deployRoot string) {
writeFile(t, filepath.Join(dataDir, "opendkim", "keys", "example.com", "selfpost.private"), "PRIVATE KEY")
writeFile(t, filepath.Join(dataDir, "sasl", "sasldb2"), "SASLDB")
writeFile(t, filepath.Join(dataDir, "postfix", "sender_login_maps"), "@example.com login")
writeFile(t, filepath.Join(dataDir, "antispam", "deny_domain.map"), "evil.example\n")
writeFile(t, filepath.Join(dataDir, "postfix", "queue", "deferred", "sample"), "queue-file")
// Transient files that must NOT be archived.
writeFile(t, filepath.Join(dataDir, "setup-token"), "secret-token")
@@ -114,6 +115,10 @@ func TestCreateIncludesStateExcludesTransient(t *testing.T) {
DataArchivePrefix + "opendkim/keys/example.com/selfpost.private",
DataArchivePrefix + "sasl/sasldb2",
DataArchivePrefix + "postfix/sender_login_maps",
// The filter lists as the spam filter reads them. They are rewritten
// from the database on the first start after a restore; carrying them
// means the filter has its lists before the panel is up.
DataArchivePrefix + "antispam/deny_domain.map",
ComposeFileName,
EnvFileName,
CertsDirName + "/fullchain.pem",
+351
View File
@@ -0,0 +1,351 @@
package inbound
import (
"fmt"
"net/netip"
"os"
"path/filepath"
"sort"
"strings"
"time"
"unicode/utf8"
"github.com/mixeme/selfpost/internal/atomicfile"
"github.com/mixeme/selfpost/internal/configsafe"
"github.com/mixeme/selfpost/internal/store"
)
// ListStore is the slice of the store the filter lists need. *store.Store
// satisfies it.
type ListStore interface {
AddSpamListEntry(list, kind, value, note string) (store.SpamListEntry, error)
ListSpamListEntries() ([]store.SpamListEntry, error)
DeleteSpamListEntry(id int64) error
ListSpamListHits(marks ...string) ([]store.SpamListHit, error)
}
// Lists owns the allow and deny lists of the external spam filter: the rows in
// SQLite and the map files the filter reads. SelfPost does not consult the
// lists itself - it writes them out, six files in one directory, one entry a
// line, and the filter (rspamd, deploy/antispam/rspamd/multimap.conf) rereads
// a file when it changes. Which list wins when both match is the filter's
// configuration: deny.
type Lists struct {
store ListStore
dir string
}
// NewLists builds the filter lists over the store and the map directory
// (/data/antispam).
func NewLists(st ListStore, dir string) *Lists {
return &Lists{store: st, dir: dir}
}
// noteMaxRunes caps the operator's comment on an entry.
const noteMaxRunes = 200
// mapFiles names the file of every (list, kind). An IP map and a sender map
// are different things to the filter, and a domain entry is matched against
// another part of the sender than an address, so each has its own file.
var mapFiles = map[[2]string]string{
{store.SpamListAllow, store.SpamEntryIP}: "allow_ip.map",
{store.SpamListAllow, store.SpamEntryAddress}: "allow_from.map",
{store.SpamListAllow, store.SpamEntryDomain}: "allow_domain.map",
{store.SpamListDeny, store.SpamEntryIP}: "deny_ip.map",
{store.SpamListDeny, store.SpamEntryAddress}: "deny_from.map",
{store.SpamListDeny, store.SpamEntryDomain}: "deny_domain.map",
}
// Entries returns every entry of both lists.
func (l *Lists) Entries() ([]store.SpamListEntry, error) {
return l.store.ListSpamListEntries()
}
// Add validates an entry, stores it normalised and rewrites the maps. A
// refusal is a *ValidationError; an entry already on that list is
// store.ErrSpamListEntryExists.
func (l *Lists) Add(list, kind, rawValue, rawNote string) (store.SpamListEntry, error) {
if list != store.SpamListAllow && list != store.SpamListDeny {
return store.SpamListEntry{}, invalid("choose the allow or the deny list")
}
value, err := normalizeListValue(kind, rawValue)
if err != nil {
return store.SpamListEntry{}, err
}
note, err := normalizeNote(rawNote)
if err != nil {
return store.SpamListEntry{}, err
}
e, err := l.store.AddSpamListEntry(list, kind, value, note)
if err != nil {
return store.SpamListEntry{}, err
}
if err := l.Sync(); err != nil {
// An entry the filter never got is worse than no entry: the page
// would show a rule that does nothing.
_ = l.store.DeleteSpamListEntry(e.ID)
return store.SpamListEntry{}, err
}
return e, nil
}
// Delete removes an entry and rewrites the maps.
func (l *Lists) Delete(id int64) error {
if err := l.store.DeleteSpamListEntry(id); err != nil {
return err
}
return l.Sync()
}
// Sync writes all six map files from SQLite, each atomically, so the filter
// reads a whole old file or a whole new one. Every file is written even when
// empty: the filter's configuration names all six.
func (l *Lists) Sync() error {
entries, err := l.store.ListSpamListEntries()
if err != nil {
return err
}
lines := make(map[string][]string, len(mapFiles))
for _, name := range mapFiles {
lines[name] = nil
}
for _, e := range entries {
name, ok := mapFiles[[2]string{e.List, e.Kind}]
if !ok {
return fmt.Errorf("filter list entry %d: unknown list %q or kind %q", e.ID, e.List, e.Kind)
}
// Values were validated on the way in; this is the writer's own guard
// against a row that got into the table some other way. In a map file a
// newline ends the entry, whitespace starts its value and # a comment.
if err := configsafe.Token("filter list entry", e.Value, " \t\r\n#"); err != nil {
return err
}
lines[name] = append(lines[name], e.Value)
}
// The filter runs under another user in another container: the directory
// and the files are readable by anyone. They hold addresses, not secrets.
if err := os.MkdirAll(l.dir, 0o755); err != nil {
return fmt.Errorf("create %s: %w", l.dir, err)
}
for name, vals := range lines {
sort.Strings(vals)
var b strings.Builder
b.WriteString("# Written by SelfPost (Inbound > Filter lists). Do not edit: the next change in the panel replaces this file.\n")
for _, v := range vals {
b.WriteString(v)
b.WriteByte('\n')
}
if err := atomicfile.Write(filepath.Join(l.dir, name), []byte(b.String()), 0o644); err != nil {
return err
}
}
return nil
}
// LastWritten is when the maps were last written: the newest modification
// time of the six files, the zero time while none exists (the filter lists
// have never been touched, or the directory is gone).
func (l *Lists) LastWritten() time.Time {
var newest time.Time
for _, name := range mapFiles {
if fi, err := os.Stat(filepath.Join(l.dir, name)); err == nil && fi.ModTime().After(newest) {
newest = fi.ModTime()
}
}
return newest
}
// normalizeListValue validates one entry and returns the form it is stored and
// written in: lower case for an address or a domain, the canonical text of an
// address or a network for an IP.
func normalizeListValue(kind, raw string) (string, error) {
v := strings.ToLower(strings.TrimSpace(raw))
switch kind {
case store.SpamEntryAddress:
if v == "" {
return "", invalid("sender address is required")
}
if len(v) > 254 {
return "", invalid("sender address must be at most 254 characters")
}
at := strings.LastIndexByte(v, '@')
if at <= 0 || at >= len(v)-1 {
return "", invalid("%q is not a valid email address", v)
}
local, domain := v[:at], v[at+1:]
if local[0] == '.' || local[len(local)-1] == '.' || len(local) > 64 {
return "", invalid("%q: invalid local part", v)
}
for i := 0; i < len(local); i++ {
c := local[i]
lower := c >= 'a' && c <= 'z'
digit := c >= '0' && c <= '9'
// '=' for the rewritten senders of forwarders (SRS0=…).
if !lower && !digit && !strings.ContainsRune("._+-=", rune(c)) {
return "", invalid("%q: local part contains invalid characters", v)
}
}
if err := checkDomain(domain); err != nil {
return "", err
}
return v, nil
case store.SpamEntryDomain:
// "@example.com" is how a domain is often written in such a list.
v = strings.TrimPrefix(v, "@")
if err := checkDomain(v); err != nil {
return "", err
}
return v, nil
case store.SpamEntryIP:
if v == "" {
return "", invalid("address or network is required")
}
if strings.Contains(v, "/") {
p, err := netip.ParsePrefix(v)
if err != nil {
return "", invalid("%q is not a network in CIDR form (e.g. 192.0.2.0/24)", v)
}
if p.Addr().Is4In6() {
return "", invalid("%q: write an IPv4 network as IPv4 (e.g. 192.0.2.0/24)", v)
}
if p.Bits() == 0 {
return "", invalid("%q is every address there is", v)
}
return p.Masked().String(), nil
}
a, err := netip.ParseAddr(v)
if err != nil || a.Zone() != "" {
return "", invalid("%q is not an IP address or a network in CIDR form", v)
}
return a.Unmap().String(), nil
default:
return "", invalid("choose what the entry is: a sender address, a sender domain or a client address")
}
}
// normalizeNote trims the operator's comment and refuses what a single line of
// text cannot be.
func normalizeNote(raw string) (string, error) {
note := strings.TrimSpace(raw)
if utf8.RuneCountInString(note) > noteMaxRunes {
return "", invalid("note must be at most %d characters", noteMaxRunes)
}
for _, r := range note {
if r < 0x20 || r == 0x7f || r == utf8.RuneError {
return "", invalid("note must be one line of plain text")
}
}
return note, nil
}
// What a list leaves in the journal when it decides a message - the two ends
// of deploy/antispam/rspamd/multimap.conf this package depends on. A deny rule
// refuses with a message, which the log tailer files as the reply of a
// rejected row. An allow rule adds a symbol named after the kind of entry,
// which the journal milter keeps in the verdict of an accepted row.
const (
denyReplyMark = "this server's deny list"
allowMark = "SELFPOST_ALLOW_"
)
var allowSymbol = map[string]string{
store.SpamEntryIP: "SELFPOST_ALLOW_IP(",
store.SpamEntryAddress: "SELFPOST_ALLOW_FROM(",
store.SpamEntryDomain: "SELFPOST_ALLOW_DOMAIN(",
}
// Usage is how often an entry decided a message, as far back as the journal
// reaches, and when it last did. The zero value is an entry that never did.
type Usage struct {
Hits int
Last time.Time
}
// Usage counts, for every entry, the journal rows it decided. Nothing is
// stored for this: the journal already says which list decided a message and
// who the message was from, and an entry is matched against that the way the
// filter matches it - an address whole, a domain as the sender's domain, an
// address or network against the client's. So the count reaches back as far
// as the journal does (its retention, or less when it is full), and a row
// counts for every entry of the deciding list that fits it: the journal does
// not say which of two fitting entries the filter met first.
//
// A counter kept with each entry would not depend on the journal; the plan
// (docs/plans/inbound-antispam-panel.md, Filter list usage) says when that is
// worth a migration.
func (l *Lists) Usage() (map[int64]Usage, error) {
entries, err := l.store.ListSpamListEntries()
if err != nil {
return nil, err
}
out := make(map[int64]Usage, len(entries))
if len(entries) == 0 {
return out, nil
}
hits, err := l.store.ListSpamListHits(denyReplyMark, allowMark)
if err != nil {
return nil, err
}
type matcher struct {
id int64
list string
kind string
value string
prefix netip.Prefix
}
matchers := make([]matcher, 0, len(entries))
for _, e := range entries {
m := matcher{id: e.ID, list: e.List, kind: e.Kind, value: e.Value}
if e.Kind == store.SpamEntryIP {
if p, err := netip.ParsePrefix(e.Value); err == nil {
m.prefix = p
} else if a, err := netip.ParseAddr(e.Value); err == nil {
m.prefix = netip.PrefixFrom(a, a.BitLen())
} else {
continue // not an entry this package wrote
}
}
matchers = append(matchers, m)
}
for _, h := range hits {
from := strings.ToLower(h.From)
domain := ""
if i := strings.LastIndexByte(from, '@'); i >= 0 {
domain = from[i+1:]
}
client, clientErr := netip.ParseAddr(h.ClientIP)
client = client.Unmap()
denied := h.Decision != store.SpamDecisionAccept && strings.Contains(h.Detail, denyReplyMark)
for _, m := range matchers {
switch m.list {
case store.SpamListDeny:
if !denied {
continue
}
case store.SpamListAllow:
if h.Decision != store.SpamDecisionAccept || !strings.Contains(h.Detail, allowSymbol[m.kind]) {
continue
}
}
fits := false
switch m.kind {
case store.SpamEntryAddress:
fits = from == m.value
case store.SpamEntryDomain:
fits = domain == m.value
case store.SpamEntryIP:
fits = clientErr == nil && m.prefix.Contains(client)
}
if !fits {
continue
}
u := out[m.id]
u.Hits++
if h.CreatedAt.After(u.Last) {
u.Last = h.CreatedAt
}
out[m.id] = u
}
}
return out, nil
}
+370
View File
@@ -0,0 +1,370 @@
package inbound
import (
"errors"
"os"
"path/filepath"
"strings"
"testing"
"time"
"github.com/mixeme/selfpost/internal/store"
)
func testLists(t *testing.T) (*Lists, *store.Store, string) {
t.Helper()
st, err := store.Open(filepath.Join(t.TempDir(), "test.db"))
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { st.Close() })
dir := filepath.Join(t.TempDir(), "antispam")
return NewLists(st, dir), st, dir
}
// mapLines reads a map file without its comment line.
func mapLines(t *testing.T, dir, name string) []string {
t.Helper()
raw, err := os.ReadFile(filepath.Join(dir, name))
if err != nil {
t.Fatalf("read %s: %v", name, err)
}
var out []string
for _, line := range strings.Split(strings.TrimRight(string(raw), "\n"), "\n") {
if !strings.HasPrefix(line, "#") {
out = append(out, line)
}
}
return out
}
func TestNormalizeListValue(t *testing.T) {
for _, c := range []struct{ kind, raw, want string }{
{store.SpamEntryAddress, " Billing@Vendor.Example ", "billing@vendor.example"},
{store.SpamEntryAddress, "srs0=ab12=cd=example.org=user@forwarder.example", "srs0=ab12=cd=example.org=user@forwarder.example"},
{store.SpamEntryDomain, "@Evil.Example", "evil.example"},
{store.SpamEntryDomain, "sub.evil.example", "sub.evil.example"},
{store.SpamEntryIP, "192.0.2.7", "192.0.2.7"},
{store.SpamEntryIP, "192.0.2.77/24", "192.0.2.0/24"},
{store.SpamEntryIP, "::ffff:192.0.2.7", "192.0.2.7"},
{store.SpamEntryIP, "2001:DB8:0:0::1", "2001:db8::1"},
{store.SpamEntryIP, "2001:db8:1::5/48", "2001:db8:1::/48"},
} {
got, err := normalizeListValue(c.kind, c.raw)
if err != nil || got != c.want {
t.Errorf("normalizeListValue(%s, %q) = %q, %v; want %q", c.kind, c.raw, got, err, c.want)
}
}
for _, c := range []struct{ kind, raw string }{
{store.SpamEntryAddress, ""},
{store.SpamEntryAddress, "no-at-sign"},
{store.SpamEntryAddress, "@example.com"},
{store.SpamEntryAddress, "a@localhost"},
{store.SpamEntryAddress, "a b@example.com"},
{store.SpamEntryAddress, "a#b@example.com"},
{store.SpamEntryAddress, "a\nb@example.com"},
{store.SpamEntryAddress, ".a@example.com"},
{store.SpamEntryDomain, ""},
{store.SpamEntryDomain, "example"},
{store.SpamEntryDomain, "exa mple.com"},
{store.SpamEntryDomain, "example.com # all of it"},
{store.SpamEntryDomain, "user@example.com"},
{store.SpamEntryIP, ""},
{store.SpamEntryIP, "example.com"},
{store.SpamEntryIP, "192.0.2.300"},
{store.SpamEntryIP, "192.0.2.0/33"},
{store.SpamEntryIP, "0.0.0.0/0"},
{store.SpamEntryIP, "::/0"},
{store.SpamEntryIP, "fe80::1%eth0"},
{store.SpamEntryIP, "::ffff:192.0.2.0/120"},
{store.SpamEntryIP, "192.0.2.7 192.0.2.8"},
{"subject", "pills"},
{"", "x"},
} {
got, err := normalizeListValue(c.kind, c.raw)
var ve *ValidationError
if err == nil || !errors.As(err, &ve) {
t.Errorf("normalizeListValue(%q, %q) = %q, %v; want a ValidationError", c.kind, c.raw, got, err)
}
}
}
func TestListsAddWritesTheMaps(t *testing.T) {
l, _, dir := testLists(t)
for _, e := range []struct{ list, kind, value string }{
{store.SpamListAllow, store.SpamEntryAddress, "Billing@Vendor.Example"},
{store.SpamListAllow, store.SpamEntryAddress, "accounts@vendor.example"},
{store.SpamListDeny, store.SpamEntryDomain, "@evil.example"},
{store.SpamListDeny, store.SpamEntryIP, "198.51.100.9/24"},
// The same value on both lists is two entries: deny wins in the filter.
{store.SpamListDeny, store.SpamEntryAddress, "billing@vendor.example"},
} {
if _, err := l.Add(e.list, e.kind, e.value, " why "); err != nil {
t.Fatalf("Add(%s, %s, %q): %v", e.list, e.kind, e.value, err)
}
}
want := map[string]string{
"allow_from.map": "accounts@vendor.example billing@vendor.example",
"allow_domain.map": "",
"allow_ip.map": "",
"deny_from.map": "billing@vendor.example",
"deny_domain.map": "evil.example",
"deny_ip.map": "198.51.100.0/24",
}
for name, lines := range want {
if got := strings.Join(mapLines(t, dir, name), " "); got != lines {
t.Errorf("%s = %q, want %q", name, got, lines)
}
// The filter is another user in another container.
if info, err := os.Stat(filepath.Join(dir, name)); err != nil {
t.Errorf("stat %s: %v", name, err)
} else if info.Mode().Perm()&0o004 == 0 && os.PathSeparator == '/' {
t.Errorf("%s is mode %o, not readable by the filter", name, info.Mode().Perm())
}
}
if files, _ := os.ReadDir(dir); len(files) != len(want) {
t.Errorf("the directory holds %d files, want %d (no temp file left behind)", len(files), len(want))
}
entries, err := l.Entries()
if err != nil || len(entries) != 5 {
t.Fatalf("Entries = %d, %v", len(entries), err)
}
if entries[0].Note != "why" {
t.Errorf("note = %q, want it trimmed", entries[0].Note)
}
}
func TestListsRefusals(t *testing.T) {
l, _, dir := testLists(t)
if _, err := l.Add(store.SpamListAllow, store.SpamEntryIP, "192.0.2.7", ""); err != nil {
t.Fatalf("Add: %v", err)
}
// Written differently, the same entry.
if _, err := l.Add(store.SpamListAllow, store.SpamEntryIP, "::ffff:192.0.2.7", ""); !errors.Is(err, store.ErrSpamListEntryExists) {
t.Errorf("the same address again: %v, want ErrSpamListEntryExists", err)
}
var ve *ValidationError
for name, add := range map[string]func() error{
"an unknown list": func() error { _, err := l.Add("block", store.SpamEntryIP, "192.0.2.8", ""); return err },
"a bad value": func() error { _, err := l.Add(store.SpamListDeny, store.SpamEntryIP, "nope", ""); return err },
"a long note": func() error {
_, err := l.Add(store.SpamListDeny, store.SpamEntryIP, "192.0.2.8", strings.Repeat("x", noteMaxRunes+1))
return err
},
"a note of two lines": func() error {
_, err := l.Add(store.SpamListDeny, store.SpamEntryIP, "192.0.2.8", "one\ntwo")
return err
},
} {
if err := add(); !errors.As(err, &ve) {
t.Errorf("%s: %v, want a ValidationError", name, err)
}
}
if got := strings.Join(mapLines(t, dir, "deny_ip.map"), " "); got != "" {
t.Errorf("deny_ip.map = %q after refusals only", got)
}
if entries, _ := l.Entries(); len(entries) != 1 {
t.Errorf("entries = %d, want the first one alone", len(entries))
}
}
func TestListsDelete(t *testing.T) {
l, _, dir := testLists(t)
e, err := l.Add(store.SpamListDeny, store.SpamEntryDomain, "evil.example", "")
if err != nil {
t.Fatalf("Add: %v", err)
}
if err := l.Delete(e.ID); err != nil {
t.Fatalf("Delete: %v", err)
}
if got := mapLines(t, dir, "deny_domain.map"); len(got) != 0 {
t.Errorf("deny_domain.map = %v after the delete", got)
}
if err := l.Delete(e.ID); !errors.Is(err, store.ErrSpamListEntryNotFound) {
t.Errorf("a second delete: %v", err)
}
}
// An entry the filter never got must not stay on the page.
func TestListsAddRollsBackWhenTheMapsCannotBeWritten(t *testing.T) {
st, err := store.Open(filepath.Join(t.TempDir(), "test.db"))
if err != nil {
t.Fatal(err)
}
defer st.Close()
// A file where the directory should be.
blocked := filepath.Join(t.TempDir(), "antispam")
if err := os.WriteFile(blocked, nil, 0o600); err != nil {
t.Fatal(err)
}
l := NewLists(st, blocked)
if _, err := l.Add(store.SpamListDeny, store.SpamEntryDomain, "evil.example", ""); err == nil {
t.Fatal("Add succeeded with nowhere to write the maps")
}
if entries, _ := l.Entries(); len(entries) != 0 {
t.Errorf("the entry stayed: %+v", entries)
}
}
// The writer's own guard: a row that got into the table some other way is not
// written into a map.
func TestListsSyncRefusesAnUnsafeRow(t *testing.T) {
l, st, dir := testLists(t)
if _, err := st.AddSpamListEntry(store.SpamListDeny, store.SpamEntryDomain, "evil.example\ngood.example", ""); err != nil {
t.Fatalf("AddSpamListEntry: %v", err)
}
if err := l.Sync(); err == nil {
t.Fatal("Sync wrote a value with a newline in it")
}
if _, err := os.Stat(filepath.Join(dir, "deny_domain.map")); err == nil {
t.Error("a map file was written")
}
}
// With nothing on the lists all six files are there, empty.
func TestListsSyncWritesEmptyMaps(t *testing.T) {
l, _, dir := testLists(t)
if err := l.Sync(); err != nil {
t.Fatalf("Sync: %v", err)
}
for _, name := range mapFiles {
if got := mapLines(t, dir, name); len(got) != 0 {
t.Errorf("%s = %v", name, got)
}
}
}
func TestListsLastWritten(t *testing.T) {
l, _, dir := testLists(t)
if got := l.LastWritten(); !got.IsZero() {
t.Fatalf("LastWritten before anything was written = %v", got)
}
if err := l.Sync(); err != nil {
t.Fatal(err)
}
if got := l.LastWritten(); got.IsZero() || time.Since(got) > time.Minute {
t.Errorf("LastWritten after a sync = %v", got)
}
// The newest of the six counts, whichever it is.
old := time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC)
for _, name := range mapFiles {
if err := os.Chtimes(filepath.Join(dir, name), old, old); err != nil {
t.Fatal(err)
}
}
newest := old.Add(48 * time.Hour)
if err := os.Chtimes(filepath.Join(dir, "deny_domain.map"), newest, newest); err != nil {
t.Fatal(err)
}
if got := l.LastWritten(); !got.Equal(newest) {
t.Errorf("LastWritten = %v, want %v", got, newest)
}
// A file that is not one of the six does not count.
other := newest.Add(time.Hour)
if err := os.WriteFile(filepath.Join(dir, "notes.txt"), nil, 0o644); err != nil {
t.Fatal(err)
}
if err := os.Chtimes(filepath.Join(dir, "notes.txt"), other, other); err != nil {
t.Fatal(err)
}
if got := l.LastWritten(); !got.Equal(newest) {
t.Errorf("LastWritten counts a stray file: %v", got)
}
}
// How often an entry decided a message is read off the journal: the list that
// decided, and whether the entry fits who the message was from.
func TestListsUsage(t *testing.T) {
l, st, _ := testLists(t)
add := func(list, kind, value string) int64 {
t.Helper()
e, err := l.Add(list, kind, value, "")
if err != nil {
t.Fatalf("Add(%s, %s, %s): %v", list, kind, value, err)
}
return e.ID
}
denyDomain := add(store.SpamListDeny, store.SpamEntryDomain, "evil.example")
denyAddr := add(store.SpamListDeny, store.SpamEntryAddress, "spammer@evil.example")
denyNet := add(store.SpamListDeny, store.SpamEntryIP, "198.51.100.0/24")
denyV6 := add(store.SpamListDeny, store.SpamEntryIP, "2001:db8::/32")
allowAddr := add(store.SpamListAllow, store.SpamEntryAddress, "billing@vendor.example")
allowDomain := add(store.SpamListAllow, store.SpamEntryDomain, "vendor.example")
allowIP := add(store.SpamListAllow, store.SpamEntryIP, "203.0.113.5")
unused := add(store.SpamListAllow, store.SpamEntryDomain, "quiet.example")
const denied = "5.7.1 Refused by this server's deny list"
at := time.Date(2026, 10, 1, 0, 0, 0, 0, time.UTC)
hour := 0
row := func(from, ip, decision, engine, detail string) {
t.Helper()
hour++
err := st.InsertInboundSpam(store.InboundSpamEntry{
InboundDomain: "in.example.com", To: "a@in.example.com", From: from, ClientIP: ip,
Decision: decision, Engine: engine, Detail: detail, At: at.Add(time.Duration(hour) * time.Hour),
})
if err != nil {
t.Fatal(err)
}
}
// hour 1: the address and its domain are both on the deny list, and both fit.
row("Spammer@Evil.Example", "192.0.2.9", store.SpamDecisionReject, store.SpamEngineRspamd, denied)
// hour 2: another address of the domain.
row("other@evil.example", "192.0.2.9", store.SpamDecisionReject, store.SpamEngineRspamd, denied)
// hour 3, 4: the networks, v4 (as Postfix may log it, mapped) and v6.
row("x@elsewhere.example", "::ffff:198.51.100.77", store.SpamDecisionReject, store.SpamEngineRspamd, denied)
row("x@elsewhere.example", "2001:db8::1", store.SpamDecisionReject, store.SpamEngineRspamd, denied)
// hour 5: a subdomain is not the domain.
row("x@sub.evil.example", "192.0.2.9", store.SpamDecisionReject, store.SpamEngineRspamd, denied)
// hour 6: allowed by its address and its domain - two symbols, two entries.
row("billing@vendor.example", "192.0.2.50", store.SpamDecisionAccept, store.SpamEngineRspamd,
"no action, score -199.00 / 15.00; SELFPOST_ALLOW_DOMAIN(-100.00) SELFPOST_ALLOW_FROM(-100.00)")
// hour 7: allowed by its domain alone.
row("accounts@vendor.example", "192.0.2.50", store.SpamDecisionAccept, store.SpamEngineRspamd,
"no action, score -99.00 / 15.00; SELFPOST_ALLOW_DOMAIN(-100.00)")
// hour 8: allowed by the client address; the sender's domain is not why.
row("x@quiet.example", "203.0.113.5", store.SpamDecisionAccept, store.SpamEngineRspamd,
"no action, score -99.00 / 15.00; SELFPOST_ALLOW_IP(-100.00)")
// Nobody's: the filter's own refusal, an accept with no list symbol, a
// refusal by Postfix, and a sender who spells the mark in his own address.
row("spammer@evil.example", "192.0.2.9", store.SpamDecisionReject, store.SpamEngineRspamd, "5.7.1 Spam message rejected")
row("billing@vendor.example", "192.0.2.50", store.SpamDecisionAccept, store.SpamEngineRspamd, "no action, score 0.10 / 15.00")
row("spammer@evil.example", "198.51.100.1", store.SpamDecisionReject, store.SpamEnginePostfix, "550 5.1.1 <nobody@in.example.com>: this server's deny list")
row("billing@vendor.example", "203.0.113.5", store.SpamDecisionAccept, store.SpamEngineSelfPost, "")
usage, err := l.Usage()
if err != nil {
t.Fatalf("Usage: %v", err)
}
for name, c := range map[string]struct {
id int64
hits int
last int // hour
}{
"deny domain": {denyDomain, 2, 2}, "deny address": {denyAddr, 1, 1},
"deny network": {denyNet, 1, 3}, "deny v6 network": {denyV6, 1, 4},
"allow address": {allowAddr, 1, 6}, "allow domain": {allowDomain, 2, 7}, "allow ip": {allowIP, 1, 8},
"unused": {unused, 0, 0},
} {
got := usage[c.id]
wantLast := time.Time{}
if c.hits > 0 {
wantLast = at.Add(time.Duration(c.last) * time.Hour)
}
if got.Hits != c.hits || !got.Last.Equal(wantLast) {
t.Errorf("%s: %d hits, last %v; want %d, %v", name, got.Hits, got.Last, c.hits, wantLast)
}
}
}
func TestListsUsageWithNoEntries(t *testing.T) {
l, _, _ := testLists(t)
if usage, err := l.Usage(); err != nil || len(usage) != 0 {
t.Errorf("Usage = %v, %v", usage, err)
}
}
+231
View File
@@ -0,0 +1,231 @@
package logtail
import (
"context"
"log"
"regexp"
"strings"
"time"
"github.com/mixeme/selfpost/internal/store"
)
// InboundStore is what the tailer needs of the inbound antispam journal:
// writing the rows for refused mail, knowing which domains are inbound, and
// pruning. *store.Store satisfies it.
type InboundStore interface {
InsertInboundSpam(e store.InboundSpamEntry) error
IsInboundDomain(name string) (bool, error)
PruneInboundSpam(cutoff time.Time, keep int) (int64, error)
}
// Inbound turns on the inbound half of the tailer: journal rows for mail that
// port 25 refused, and the journal's retention. Run takes nil when the inbound
// relay is off.
type Inbound struct {
Store InboundStore
Retention RetentionDays
}
// The inbound journal milter writes a row for mail that was accepted; mail
// that was refused never reaches it, and what is known about it is one line
// of mail.log:
//
// postfix/inbound/smtpd[41]: NOQUEUE: reject: RCPT from mx.example.net[203.0.113.9]: 550 5.1.1 <nobody@in.example.com>: Recipient address rejected: User unknown in relay recipient table; from=<a@example.net> to=<nobody@in.example.com> proto=ESMTP helo=<mx.example.net>
// postfix/cleanup[52]: 4A1B2C3D: milter-reject: END-OF-MESSAGE from mx.example.net[203.0.113.9]: 5.7.1 Spam message rejected; from=<a@example.net> to=<b@in.example.com> proto=ESMTP helo=<mx.example.net>
//
// The first is Postfix's own policy, written by the port-25 smtpd, which
// build/postfix-config.sh names postfix/inbound so that its lines are told
// from those of the submission ports. The second is the filter's verdict at
// end-of-message, which cleanup(8) writes under one name for every listener -
// so it counts only for a queue id the port-25 smtpd was seen to open
// (inboundClientRe).
//
// Both are matched from the head of the line, after the timestamp and host
// name: the tail of such a line is the sender's own words (HELO name,
// addresses), and a pattern that could start anywhere would let him write
// journal rows.
var (
inboundRejectRe = regexp.MustCompile(`^\S+ postfix/(inbound/smtpd|cleanup)\[\d+\]: ([0-9A-Za-z]+): (reject|milter-reject|milter-hold|milter-discard): \S+ from [^\[\s]*\[([0-9A-Fa-f:.]+)\]: (.*); from=<([^>]*)> to=<([^>]*)> proto=\S+(?: helo=<[^>]*>)?$`)
inboundClientRe = regexp.MustCompile(`^\S+ postfix/inbound/smtpd\[\d+\]: ([0-9A-Za-z]+): client=`)
)
// A row holds one line of reason; Postfix's reply text carries the far end's
// bytes and is cut and reduced to printable ASCII like any of them.
const inboundDetailMax = 255
// inboundQueueIDs is how many port-25 queue ids are remembered while waiting
// for cleanup's verdict line on them. A message is in that state for the
// seconds its DATA takes; the cap only bounds a flood.
const inboundQueueIDs = 4096
// inboundJournal is the per-run state of the inbound half.
type inboundJournal struct {
st InboundStore
// port25 holds the queue ids the port-25 smtpd opened and cleanup has not
// reported on yet; order is their arrival, for dropping the oldest.
port25 map[string]struct{}
order []string
}
func newInboundJournal(st InboundStore) *inboundJournal {
return &inboundJournal{st: st, port25: make(map[string]struct{})}
}
// handle looks at one mail.log line and writes a journal row when it is a
// refusal of mail for an inbound domain. Errors are logged; the tailer goes on.
func (j *inboundJournal) handle(line string, now time.Time) {
at, rest := lineTime(line, now)
if m := inboundClientRe.FindStringSubmatch(rest); m != nil {
j.remember(m[1])
return
}
m := inboundRejectRe.FindStringSubmatch(rest)
if m == nil {
return
}
program, queueID, kind, clientIP, reason, from, to := m[1], m[2], m[3], m[4], m[5], m[6], m[7]
if program == "cleanup" {
if _, ok := j.port25[queueID]; !ok {
return // a submission-port message, or one opened before this run
}
delete(j.port25, queueID)
}
// Only mail for an inbound domain is the relay's. "Relay access denied"
// for a stranger's domain is every scanner on the internet, and a line
// with no recipient (size limit at MAIL) belongs to no domain.
domain := ""
if i := strings.LastIndexByte(to, '@'); i >= 0 {
domain = strings.ToLower(to[i+1:])
}
if domain == "" {
return
}
ok, err := j.st.IsInboundDomain(domain)
if err != nil {
log.Printf("log-tailer: inbound journal: look up %s: %v", domain, err)
return
}
if !ok {
return
}
e := store.InboundSpamEntry{
InboundDomain: domain,
ClientIP: clientIP,
From: from,
To: to,
Decision: store.SpamDecisionReject,
Engine: store.SpamEngineRspamd,
Detail: printable(reason, inboundDetailMax),
At: at,
}
switch {
case kind == "reject":
e.Engine = store.SpamEnginePostfix
case kind == "milter-hold":
e.Decision = store.SpamDecisionQuarantine
}
// "450 4.7.1 …" from Postfix, "4.7.1 …" from a milter: either way a reply
// that starts with 4 asks the sender to come back.
if e.Decision == store.SpamDecisionReject && strings.HasPrefix(reason, "4") {
e.Decision = store.SpamDecisionTempfail
}
if err := j.st.InsertInboundSpam(e); err != nil {
log.Printf("log-tailer: inbound journal: record %s -> %s: %v", from, to, err)
}
}
func (j *inboundJournal) remember(queueID string) {
if _, ok := j.port25[queueID]; ok {
return
}
if len(j.order) >= inboundQueueIDs {
delete(j.port25, j.order[0])
j.order = j.order[1:]
}
j.port25[queueID] = struct{}{}
j.order = append(j.order, queueID)
}
// printable cuts s to max bytes and replaces everything outside printable
// ASCII.
func printable(s string, max int) string {
if len(s) > max {
s = s[:max]
}
b := []byte(s)
for i, c := range b {
if c < 0x20 || c > 0x7e {
b[i] = '?'
}
}
return string(b)
}
// lineTime reads the time at the head of a mail.log line and returns it with
// the rest of the line. A refusal is dated by its line and not by when the
// tailer got to it: after the panel was down the two are hours apart. The
// syslog format carries neither year nor zone - the year is taken from now
// (a stamp that would then lie in the future is last year's), the zone is the
// container's, which is what postlogd wrote it in. A line with no readable
// stamp is dated now.
func lineTime(line string, now time.Time) (time.Time, string) {
if m := isoStampRe.FindStringSubmatch(line); m != nil {
rest := strings.TrimSpace(line[len(m[0]):])
if t, err := time.Parse(time.RFC3339Nano, strings.TrimSpace(m[0])); err == nil {
return t.UTC(), rest
}
return now, rest
}
if m := syslogStampRe.FindStringSubmatch(line); m != nil {
rest := strings.TrimSpace(line[len(m[0]):])
stamp := strings.Join(strings.Fields(m[1]), " ")
t, err := time.ParseInLocation("Jan 2 15:04:05", stamp, time.Local)
if err != nil {
return now, rest
}
t = t.AddDate(now.In(time.Local).Year(), 0, 0)
if t.After(now.Add(24 * time.Hour)) {
t = t.AddDate(-1, 0, 0)
}
return t.UTC(), rest
}
return now, line
}
// The journal keeps at most this many rows whatever their age: refusals are
// written at the rate strangers knock on port 25.
const inboundSpamLogMaxRows = 10000
// inboundRetentionLoop prunes the inbound journal at once and then on the
// cadence of the send log.
func inboundRetentionLoop(ctx context.Context, in *Inbound) {
prune := func() {
days := in.Retention()
if days <= 0 {
days = store.InboundSpamLogRetentionDaysDefault
}
n, err := in.Store.PruneInboundSpam(time.Now().UTC().AddDate(0, 0, -days), inboundSpamLogMaxRows)
if err != nil {
log.Printf("log-tailer: inbound journal prune: %v", err)
return
}
if n > 0 {
log.Printf("log-tailer: pruned %d inbound journal rows (older than %d days or beyond %d rows)", n, days, inboundSpamLogMaxRows)
}
}
prune()
t := time.NewTicker(retentionInterval)
defer t.Stop()
for {
select {
case <-ctx.Done():
return
case <-t.C:
prune()
}
}
}
+218
View File
@@ -0,0 +1,218 @@
package logtail
import (
"context"
"errors"
"strings"
"testing"
"time"
"github.com/mixeme/selfpost/internal/store"
)
type fakeInboundStore struct {
domains map[string]bool
rows []store.InboundSpamEntry
lookupErr error
pruned []int
}
func (f *fakeInboundStore) InsertInboundSpam(e store.InboundSpamEntry) error {
f.rows = append(f.rows, e)
return nil
}
func (f *fakeInboundStore) IsInboundDomain(name string) (bool, error) {
if f.lookupErr != nil {
return false, f.lookupErr
}
return f.domains[name], nil
}
func (f *fakeInboundStore) PruneInboundSpam(_ time.Time, keep int) (int64, error) {
f.pruned = append(f.pruned, keep)
return 0, nil
}
func inboundFixture() (*inboundJournal, *fakeInboundStore) {
st := &fakeInboundStore{domains: map[string]bool{"in.example.com": true}}
return newInboundJournal(st), st
}
var tailNow = time.Date(2026, 10, 11, 18, 0, 0, 0, time.UTC)
const (
head = "2026-10-11T12:00:05.123456+00:00 mail "
unknownRcp = head + "postfix/inbound/smtpd[41]: NOQUEUE: reject: RCPT from mx.example.net[203.0.113.9]: 550 5.1.1 <nobody@in.example.com>: Recipient address rejected: User unknown in relay recipient table; from=<a@example.net> to=<nobody@in.example.com> proto=ESMTP helo=<mx.example.net>"
)
// Postfix's own refusal of a recipient at an inbound domain is a row, dated by
// its line.
func TestInboundRejectByPostfix(t *testing.T) {
j, st := inboundFixture()
j.handle(unknownRcp, tailNow)
if len(st.rows) != 1 {
t.Fatalf("rows = %+v", st.rows)
}
want := store.InboundSpamEntry{
InboundDomain: "in.example.com", ClientIP: "203.0.113.9",
From: "a@example.net", To: "nobody@in.example.com",
Decision: store.SpamDecisionReject, Engine: store.SpamEnginePostfix,
Detail: "550 5.1.1 <nobody@in.example.com>: Recipient address rejected: User unknown in relay recipient table",
At: time.Date(2026, 10, 11, 12, 0, 5, 123456000, time.UTC),
}
if st.rows[0] != want {
t.Errorf("row = %+v\nwant %+v", st.rows[0], want)
}
}
func TestInboundTempfailByPostfix(t *testing.T) {
j, st := inboundFixture()
j.handle(head+"postfix/inbound/smtpd[41]: NOQUEUE: reject: RCPT from unknown[2001:db8::9]: 450 4.1.8 <a@nx.example>: Sender address rejected: Domain not found; from=<a@nx.example> to=<B@IN.example.com> proto=ESMTP helo=<x>", tailNow)
if len(st.rows) != 1 || st.rows[0].Decision != store.SpamDecisionTempfail || st.rows[0].ClientIP != "2001:db8::9" || st.rows[0].InboundDomain != "in.example.com" {
t.Errorf("rows = %+v", st.rows)
}
}
// The filter's verdict is written by cleanup, under one name for every
// listener: it is the journal's only for a queue id port 25 opened.
func TestInboundMilterVerdictNeedsAPort25QueueID(t *testing.T) {
const verdict = "postfix/cleanup[52]: 4A1B2C3D: milter-reject: END-OF-MESSAGE from mx.example.net[203.0.113.9]: 5.7.1 Spam message rejected; from=<a@example.net> to=<b@in.example.com> proto=ESMTP helo=<mx.example.net>"
j, st := inboundFixture()
// Opened by a submission port: postfix/smtpd, not postfix/inbound/smtpd.
j.handle(head+"postfix/smtpd[20]: 4A1B2C3D: client=app.example.com[203.0.113.4], sasl_method=PLAIN, sasl_username=app", tailNow)
j.handle(head+verdict, tailNow)
if len(st.rows) != 0 {
t.Fatalf("a submission-port verdict was journalled: %+v", st.rows)
}
j.handle(head+"postfix/inbound/smtpd[41]: 4A1B2C3D: client=mx.example.net[203.0.113.9]", tailNow)
j.handle(head+verdict, tailNow)
if len(st.rows) != 1 {
t.Fatalf("rows = %+v", st.rows)
}
got := st.rows[0]
if got.Decision != store.SpamDecisionReject || got.Engine != store.SpamEngineRspamd || got.Detail != "5.7.1 Spam message rejected" {
t.Errorf("row = %+v", got)
}
// One verdict per message: the id is forgotten with it.
j.handle(head+verdict, tailNow)
if len(st.rows) != 1 {
t.Errorf("the same queue id was journalled twice")
}
}
func TestInboundMilterDecisions(t *testing.T) {
for kind, c := range map[string]struct{ reply, decision string }{
"milter-reject": {"4.7.1 Try again later", store.SpamDecisionTempfail},
"milter-hold": {"milter triggers HOLD action", store.SpamDecisionQuarantine},
"milter-discard": {"milter triggers DISCARD action", store.SpamDecisionReject},
} {
j, st := inboundFixture()
j.handle(head+"postfix/inbound/smtpd[41]: Q1: client=mx.example.net[203.0.113.9]", tailNow)
j.handle(head+"postfix/cleanup[52]: Q1: "+kind+": END-OF-MESSAGE from mx.example.net[203.0.113.9]: "+c.reply+"; from=<a@example.net> to=<b@in.example.com> proto=ESMTP helo=<mx.example.net>", tailNow)
if len(st.rows) != 1 || st.rows[0].Decision != c.decision || st.rows[0].Engine != store.SpamEngineRspamd {
t.Errorf("%s: rows = %+v", kind, st.rows)
}
}
}
// What is not the relay's mail, or is not a refusal at all, leaves no row.
func TestInboundIgnores(t *testing.T) {
for name, line := range map[string]string{
"another domain (relay access denied)": head + "postfix/inbound/smtpd[41]: NOQUEUE: reject: RCPT from x[203.0.113.9]: 554 5.7.1 <c@elsewhere.example>: Relay access denied; from=<a@example.net> to=<c@elsewhere.example> proto=ESMTP helo=<x>",
"no recipient (size limit at MAIL)": head + "postfix/inbound/smtpd[41]: NOQUEUE: reject: MAIL from x[203.0.113.9]: 552 5.3.4 Message size exceeds fixed limit; proto=ESMTP helo=<x>",
"a submission port": head + "postfix/smtpd[20]: NOQUEUE: reject: RCPT from x[203.0.113.9]: 553 5.7.1 <b@in.example.com>: Sender address rejected: not owned by user app; from=<a@example.net> to=<b@in.example.com> proto=ESMTP helo=<x>",
"a delivery line": head + "postfix/smtp[26]: 41E862C00D9E: to=<b@in.example.com>, relay=mx[203.0.113.9]:25, dsn=2.0.0, status=sent (250 OK)",
"a connect line": head + "postfix/inbound/smtpd[41]: connect from mx.example.net[203.0.113.9]",
// The HELO name is the sender's; a line cannot be started from inside one.
"a refusal spelled inside another line": head + "postfix/inbound/smtpd[41]: warning: hostname x does not resolve: mail postfix/inbound/smtpd[1]: NOQUEUE: reject: RCPT from x[203.0.113.9]: 550 no; from=<a@example.net> to=<b@in.example.com> proto=ESMTP helo=<x>",
"an address with a bracket in it": head + "postfix/inbound/smtpd[41]: NOQUEUE: reject: RCPT from x[203.0.113.9]: 550 no; from=<a> to=<v@in.example.com> proto=X helo=<y>@example.net> to=<b@elsewhere.example> proto=ESMTP helo=<x>",
} {
j, st := inboundFixture()
j.handle(line, tailNow)
if len(st.rows) != 0 {
t.Errorf("%s: journalled %+v", name, st.rows)
}
}
}
func TestInboundDetailIsCutAndPrintable(t *testing.T) {
j, st := inboundFixture()
j.handle(head+"postfix/inbound/smtpd[41]: NOQUEUE: reject: RCPT from x[203.0.113.9]: 550 5.7.1 caf\xc3\xa9 \x1b[31m"+strings.Repeat("x", 400)+"; from=<a@example.net> to=<b@in.example.com> proto=ESMTP helo=<x>", tailNow)
if len(st.rows) != 1 {
t.Fatalf("rows = %+v", st.rows)
}
d := st.rows[0].Detail
if len(d) != inboundDetailMax || !strings.HasPrefix(d, "550 5.7.1 caf?? ?[31mxxx") {
t.Errorf("detail (%d bytes) = %q", len(d), d)
}
}
func TestInboundLookupFailureLeavesNoRow(t *testing.T) {
j, st := inboundFixture()
st.lookupErr = errors.New("boom")
j.handle(unknownRcp, tailNow)
if len(st.rows) != 0 {
t.Errorf("rows = %+v", st.rows)
}
}
// The queue ids waiting for a verdict are bounded.
func TestInboundQueueIDsAreBounded(t *testing.T) {
j, _ := inboundFixture()
for i := 0; i < inboundQueueIDs+10; i++ {
j.remember("Q" + strings.Repeat("A", i%7) + string(rune('0'+i%10)) + time.Duration(i).String())
}
if len(j.port25) != inboundQueueIDs || len(j.order) != inboundQueueIDs {
t.Errorf("remembered %d ids (%d in order), want %d", len(j.port25), len(j.order), inboundQueueIDs)
}
}
func TestLineTime(t *testing.T) {
now := time.Date(2026, 1, 2, 10, 0, 0, 0, time.UTC)
for _, c := range []struct {
line string
want time.Time
rest string
}{
{"2026-01-02T09:00:00.5+03:00 mail postfix/x: y", time.Date(2026, 1, 2, 6, 0, 0, 500000000, time.UTC), "mail postfix/x: y"},
{"Jan 2 09:30:00 mail postfix/x: y", time.Date(2026, 1, 2, 9, 30, 0, 0, time.Local).UTC(), "mail postfix/x: y"},
// December read in January is last year's.
{"Dec 31 23:59:00 mail postfix/x: y", time.Date(2025, 12, 31, 23, 59, 0, 0, time.Local).UTC(), "mail postfix/x: y"},
{"no stamp here", now, "no stamp here"},
} {
got, rest := lineTime(c.line, now)
if !got.Equal(c.want) || rest != c.rest {
t.Errorf("lineTime(%q) = %v, %q; want %v, %q", c.line, got, rest, c.want, c.rest)
}
}
}
func TestInboundRetentionPrunesAtStart(t *testing.T) {
old := retentionInterval
retentionInterval = time.Hour
defer func() { retentionInterval = old }()
st := &fakeInboundStore{}
ctx, cancel := contextWithCancel()
done := make(chan struct{})
go func() {
inboundRetentionLoop(ctx, &Inbound{Store: st, Retention: func() int { return 0 }})
close(done)
}()
deadline := time.Now().Add(2 * time.Second)
for len(st.pruned) == 0 && time.Now().Before(deadline) {
time.Sleep(10 * time.Millisecond)
}
cancel()
<-done
if len(st.pruned) != 1 || st.pruned[0] != inboundSpamLogMaxRows {
t.Errorf("prune calls = %v", st.pruned)
}
}
func contextWithCancel() (context.Context, context.CancelFunc) {
return context.WithCancel(context.Background())
}
+12 -2
View File
@@ -107,9 +107,16 @@ func parseDelivery(line string) (queueID, recipient, status string, ok bool) {
// Run follows path and updates send-log statuses until ctx is cancelled, while
// a background sweep prunes rows older than retention(). Reading resumes at
// the offset the previous run persisted, so a restart parses the delivery lines
// written while the panel was down. It returns nil on a clean shutdown.
func Run(ctx context.Context, path string, st StatusStore, retention RetentionDays) error {
// written while the panel was down. With inbound set it also writes the
// inbound journal's rows for refused mail and prunes that journal
// (inbound.go). It returns nil on a clean shutdown.
func Run(ctx context.Context, path string, st StatusStore, retention RetentionDays, inbound *Inbound) error {
go retentionLoop(ctx, st, retention)
var journal *inboundJournal
if inbound != nil {
go inboundRetentionLoop(ctx, inbound)
journal = newInboundJournal(inbound.Store)
}
// The reconcile sweep must not run against a backlog the tailer has not
// read yet: on a restart the log holds the very lines that resolve the rows
@@ -119,6 +126,9 @@ func Run(ctx context.Context, path string, st StatusStore, retention RetentionDa
go reconcileLoop(ctx, st, caughtUp)
return follow(ctx, path, &tracker{st: st, path: path}, caughtUp, func(line string) {
if journal != nil {
journal.handle(line, time.Now().UTC())
}
queueID, recipient, status, ok := parseDelivery(line)
if !ok {
return
+2 -2
View File
@@ -212,7 +212,7 @@ func TestFollowTailsAndRotates(t *testing.T) {
cs := &captureStore{}
ctx, cancel := context.WithCancel(context.Background())
done := make(chan error, 1)
go func() { done <- Run(ctx, path, cs, func() int { return 90 }) }()
go func() { done <- Run(ctx, path, cs, func() int { return 90 }, nil) }()
// Give follow() time to open at EOF (it seeks to end immediately on start,
// so the seed line above is ignored), then append a delivery line.
@@ -295,7 +295,7 @@ func startRun(t *testing.T, path string, cs *captureStore) func() {
t.Helper()
ctx, cancel := context.WithCancel(context.Background())
done := make(chan error, 1)
go func() { done <- Run(ctx, path, cs, func() int { return 90 }) }()
go func() { done <- Run(ctx, path, cs, func() int { return 90 }, nil) }()
// follow() opens and seeks on start; give it a moment before the caller
// appends, so the append is not raced by the initial open.
time.Sleep(50 * time.Millisecond)
+246
View File
@@ -0,0 +1,246 @@
package milter
import (
"context"
"log"
"net"
"net/textproto"
"regexp"
"strings"
"github.com/emersion/go-milter"
"github.com/mixeme/selfpost/internal/mailhdr"
"github.com/mixeme/selfpost/internal/store"
)
// InboundStore is what the inbound journal milter needs: the journal itself
// and the list of inbound domains, since port 25 also takes mail that is not
// the relay's. *store.Store satisfies it; tests substitute a fake.
type InboundStore interface {
InsertInboundSpam(e store.InboundSpamEntry) error
IsInboundDomain(name string) (bool, error)
}
// inboundSession is the inbound journal milter: the last milter on port 25,
// after the optional external filter. It writes one journal row per recipient
// of a message that reached end-of-message - that is, one the filter did not
// refuse; refusals never get this far and are read from mail.log instead.
//
// Like the send-log milter it only watches: every callback continues or
// accepts, and Postfix attaches the socket with default_action=accept, so
// neither an error here nor the panel being down holds up inbound mail.
type inboundSession struct {
milter.NoOpMilter
rec InboundStore
// filter says whether an external filter is attached in front of this
// milter. Without one the verdict headers below can only have been written
// by the sender, and are not read.
filter bool
clientIP string // once per connection
from string
rcpts []string
subject string
action string // X-Rspamd-Action
result string // X-Spamd-Result
seen bool // any verdict header of the filter
}
func (s *inboundSession) Connect(host, family string, port uint16, addr net.IP, m *milter.Modifier) (milter.Response, error) {
if addr != nil {
s.clientIP = addr.String()
}
return milter.RespContinue, nil
}
// MailFrom starts a message: a connection may carry several.
func (s *inboundSession) MailFrom(from string, m *milter.Modifier) (milter.Response, error) {
s.reset()
s.from = cleanAddress(from)
return milter.RespContinue, nil
}
func (s *inboundSession) RcptTo(rcpt string, m *milter.Modifier) (milter.Response, error) {
s.rcpts = append(s.rcpts, cleanAddress(rcpt))
return milter.RespContinue, nil
}
// Header keeps the first Subject and the filter's verdict. Postfix hands the
// message to its milters one after another at end-of-message, each seeing what
// the one before added, which is how the filter's headers arrive here. The
// filter replaces headers of these names, so the last one read is its own.
func (s *inboundSession) Header(name, value string, m *milter.Modifier) (milter.Response, error) {
switch textproto.CanonicalMIMEHeaderKey(name) {
case "Subject":
if s.subject == "" {
s.subject = mailhdr.DecodeSubject(value)
}
case "X-Rspamd-Action":
if s.filter {
s.action, s.seen = value, true
}
case "X-Spamd-Result":
if s.filter {
s.result, s.seen = value, true
}
}
return milter.RespContinue, nil
}
// Body fires at end-of-message: the message is about to be queued for its
// upstream.
func (s *inboundSession) Body(m *milter.Modifier) (milter.Response, error) {
s.record()
s.reset()
return milter.RespAccept, nil
}
func (s *inboundSession) Abort(m *milter.Modifier) error {
s.reset()
return nil
}
func (s *inboundSession) reset() {
s.from, s.rcpts, s.subject = "", nil, ""
s.action, s.result, s.seen = "", "", false
}
// record writes one row per recipient at an inbound domain. Other recipients
// of port 25 (a DMARC report address at the server's own name) are not the
// relay's mail and get no row. Failures are logged, never propagated.
func (s *inboundSession) record() {
e := store.InboundSpamEntry{
ClientIP: s.clientIP,
From: s.from,
Subject: s.subject,
Decision: store.SpamDecisionAccept,
Engine: store.SpamEngineSelfPost,
}
if s.seen {
action, detail := rspamdVerdict(s.action, s.result)
e.Engine, e.Detail = store.SpamEngineRspamd, detail
if action == "quarantine" {
e.Decision = store.SpamDecisionQuarantine
}
}
for _, to := range s.rcpts {
domain := domainOf(to)
ok, err := s.rec.IsInboundDomain(domain)
if err != nil {
log.Printf("inbound-journal: look up %s: %v", domain, err)
continue
}
if !ok {
continue
}
e.InboundDomain, e.To = domain, to
if err := s.rec.InsertInboundSpam(e); err != nil {
log.Printf("inbound-journal: record %s -> %s: %v", s.from, to, err)
}
}
}
// What the journal keeps of a verdict: enough to see why, on one line.
const (
verdictMaxSymbols = 6
verdictMaxBytes = 255
)
// ListSymbolPrefix starts the names of the symbols the filter lists' rules add
// (deploy/antispam/rspamd/multimap.conf): SELFPOST_ALLOW_IP, _FROM, _DOMAIN.
const ListSymbolPrefix = "SELFPOST_"
var (
// "default: False [7.20 / 15.00]" - the score and the reject threshold.
rspamdScore = regexp.MustCompile(`\[(-?[0-9]+(?:\.[0-9]+)?) / (-?[0-9]+(?:\.[0-9]+)?)\]`)
// "BAYES_SPAM(5.10)[100.00%]" - a symbol and its weight.
rspamdSymbol = regexp.MustCompile(`([A-Z][A-Z0-9_]{1,63})\((-?[0-9]+(?:\.[0-9]+)?)\)`)
)
// rspamdVerdict reduces the filter's headers to an action and a one-line
// summary: the action, the score against the threshold, and the first few
// symbols that carried weight. It reads only what it recognises - the headers
// are the filter's when it is up, but the sender's own when it is not (it is
// attached fail-open), so nothing of them is kept verbatim except through
// these patterns.
func rspamdVerdict(actionHeader, resultHeader string) (action, detail string) {
action = strings.ToLower(strings.TrimSpace(actionHeader))
for _, r := range action {
if (r < 'a' || r > 'z') && r != ' ' {
action = ""
break
}
}
if len(action) > 32 {
action = ""
}
var parts []string
if action != "" {
parts = append(parts, action)
}
if m := rspamdScore.FindStringSubmatch(resultHeader); m != nil {
parts = append(parts, "score "+m[1]+" / "+m[2])
}
detail = strings.Join(parts, ", ")
// The filter lists' own symbols come first and are kept whatever else there
// is: Filter lists counts how often an entry decided by finding them here,
// and rspamd writes symbols heaviest first, which puts an allow entry's
// -100 last, behind any six others.
var symbols, rest []string
for _, m := range rspamdSymbol.FindAllStringSubmatch(resultHeader, -1) {
if strings.Trim(m[2], "-0.") == "" { // weight zero: informational
continue
}
if strings.HasPrefix(m[1], ListSymbolPrefix) {
if len(symbols) < verdictMaxSymbols {
symbols = append(symbols, m[1]+"("+m[2]+")")
}
continue
}
rest = append(rest, m[1]+"("+m[2]+")")
}
for _, sym := range rest {
if len(symbols) == verdictMaxSymbols {
break
}
symbols = append(symbols, sym)
}
if len(symbols) > 0 {
if detail != "" {
detail += "; "
}
detail += strings.Join(symbols, " ")
}
if len(detail) > verdictMaxBytes {
detail = detail[:verdictMaxBytes]
}
return action, detail
}
// ServeInbound runs the inbound journal milter on ln until ctx is cancelled.
// filter says whether an external filter is attached in front of it
// (INBOUND_ANTISPAM_MILTER). It returns nil on a clean shutdown.
func ServeInbound(ctx context.Context, ln net.Listener, rec InboundStore, filter bool) error {
srv := &milter.Server{
NewMilter: func() milter.Milter { return &inboundSession{rec: rec, filter: filter} },
Actions: 0, // read-only
Protocol: milter.OptNoBody, // headers and end-of-message, not the body
}
go func() {
<-ctx.Done()
_ = srv.Close()
}()
if err := srv.Serve(ln); err != nil {
if ctx.Err() != nil {
return nil
}
return err
}
return nil
}
+236
View File
@@ -0,0 +1,236 @@
package milter
import (
"errors"
"net"
"strings"
"testing"
"github.com/emersion/go-milter"
"github.com/mixeme/selfpost/internal/store"
)
// fakeInbound is the journal and the inbound-domain list of one test.
type fakeInbound struct {
domains map[string]bool
rows []store.InboundSpamEntry
insertErr error
lookupErr error
}
func (f *fakeInbound) InsertInboundSpam(e store.InboundSpamEntry) error {
if f.insertErr != nil {
return f.insertErr
}
f.rows = append(f.rows, e)
return nil
}
func (f *fakeInbound) IsInboundDomain(name string) (bool, error) {
if f.lookupErr != nil {
return false, f.lookupErr
}
return f.domains[name], nil
}
// A header of the kind the documented rspamd sidecar adds.
const spamdResult = "default: False [7.20 / 15.00];\n\tBAYES_SPAM(5.10)[100.00%];\n\tMIME_GOOD(-0.10)[text/plain];\n\tARC_NA(0.00)[];\n\tRCVD_COUNT_ZERO(0.00)[0];\n\tR_DKIM_NA(2.20)[]"
// message replays one message through s: envelope, headers in order, end.
func message(t *testing.T, s *inboundSession, from string, rcpts []string, headers [][2]string) milter.Response {
t.Helper()
if _, err := s.MailFrom(from, mods(nil)); err != nil {
t.Fatalf("MailFrom: %v", err)
}
for _, r := range rcpts {
if _, err := s.RcptTo(r, mods(nil)); err != nil {
t.Fatalf("RcptTo: %v", err)
}
}
for _, h := range headers {
if _, err := s.Header(h[0], h[1], mods(nil)); err != nil {
t.Fatalf("Header: %v", err)
}
}
resp, err := s.Body(mods(map[string]string{"i": "ABC123"}))
if err != nil {
t.Fatalf("Body: %v", err)
}
return resp
}
func inboundSessionFor(t *testing.T, rec InboundStore, filter bool) *inboundSession {
t.Helper()
s := &inboundSession{rec: rec, filter: filter}
if _, err := s.Connect("mx.example.net", "tcp4", 0, net.ParseIP("203.0.113.7"), mods(nil)); err != nil {
t.Fatalf("Connect: %v", err)
}
return s
}
// With no filter attached an accepted message is a plain row, one per
// recipient, decided by nobody.
func TestInboundRecordsAcceptedMessage(t *testing.T) {
rec := &fakeInbound{domains: map[string]bool{"in.example.com": true}}
s := inboundSessionFor(t, rec, false)
resp := message(t, s, "<sender@example.net> SIZE=1200",
[]string{"<a@in.example.com>", "b@IN.example.com"},
[][2]string{{"Subject", "=?utf-8?Q?Sch=C3=B6n?="}})
if resp != milter.RespAccept {
t.Errorf("end-of-message response = %v, want accept", resp)
}
if len(rec.rows) != 2 {
t.Fatalf("rows = %+v, want one per recipient", rec.rows)
}
want := store.InboundSpamEntry{
InboundDomain: "in.example.com", ClientIP: "203.0.113.7",
From: "sender@example.net", To: "a@in.example.com", Subject: "Schön",
Decision: store.SpamDecisionAccept, Engine: store.SpamEngineSelfPost,
}
if rec.rows[0] != want {
t.Errorf("row = %+v\nwant %+v", rec.rows[0], want)
}
if rec.rows[1].To != "b@IN.example.com" || rec.rows[1].InboundDomain != "in.example.com" {
t.Errorf("second row = %+v", rec.rows[1])
}
}
// Port 25 also takes DMARC reports for the server's own name. They are not
// the relay's mail and the journal says nothing about them.
func TestInboundSkipsRecipientsOutsideInboundDomains(t *testing.T) {
rec := &fakeInbound{domains: map[string]bool{"in.example.com": true}}
s := inboundSessionFor(t, rec, false)
message(t, s, "noreply-dmarc@google.com",
[]string{"dmarc-example.com@mail.example.org", "a@in.example.com"}, nil)
if len(rec.rows) != 1 || rec.rows[0].To != "a@in.example.com" {
t.Errorf("rows = %+v, want the inbound recipient alone", rec.rows)
}
}
// With the filter attached its verdict headers become the engine and detail.
func TestInboundReadsTheFiltersVerdict(t *testing.T) {
rec := &fakeInbound{domains: map[string]bool{"in.example.com": true}}
s := inboundSessionFor(t, rec, true)
message(t, s, "sender@example.net", []string{"a@in.example.com"}, [][2]string{
{"Subject", "Cheap pills"},
{"X-Rspamd-Action", "add header"},
{"X-Spamd-Result", spamdResult},
})
if len(rec.rows) != 1 {
t.Fatalf("rows = %+v", rec.rows)
}
got := rec.rows[0]
if got.Engine != store.SpamEngineRspamd || got.Decision != store.SpamDecisionAccept {
t.Errorf("engine %q, decision %q", got.Engine, got.Decision)
}
if want := "add header, score 7.20 / 15.00; BAYES_SPAM(5.10) MIME_GOOD(-0.10) R_DKIM_NA(2.20)"; got.Detail != want {
t.Errorf("detail = %q\nwant %q", got.Detail, want)
}
}
func TestInboundQuarantineAction(t *testing.T) {
rec := &fakeInbound{domains: map[string]bool{"in.example.com": true}}
s := inboundSessionFor(t, rec, true)
message(t, s, "sender@example.net", []string{"a@in.example.com"},
[][2]string{{"x-rspamd-action", "quarantine"}})
if len(rec.rows) != 1 || rec.rows[0].Decision != store.SpamDecisionQuarantine || rec.rows[0].Detail != "quarantine" {
t.Errorf("rows = %+v", rec.rows)
}
}
// Without a filter in front, a verdict header is the sender's own writing.
func TestInboundIgnoresVerdictHeadersWithoutAFilter(t *testing.T) {
rec := &fakeInbound{domains: map[string]bool{"in.example.com": true}}
s := inboundSessionFor(t, rec, false)
message(t, s, "sender@example.net", []string{"a@in.example.com"}, [][2]string{
{"X-Rspamd-Action", "no action"},
{"X-Spamd-Result", spamdResult},
})
if len(rec.rows) != 1 || rec.rows[0].Engine != store.SpamEngineSelfPost || rec.rows[0].Detail != "" {
t.Errorf("rows = %+v", rec.rows)
}
}
// The filter is attached fail-open, so with it down the headers are again the
// sender's: nothing of them reaches the journal except through the patterns.
func TestRspamdVerdictKeepsOnlyWhatItRecognises(t *testing.T) {
action, detail := rspamdVerdict("<script>alert(1)</script>", "default: False [1 / 15]; <b>HTML</b> lower_case(9.9) \x1b[31m")
if action != "" {
t.Errorf("action = %q, want it dropped", action)
}
if detail != "score 1 / 15" {
t.Errorf("detail = %q", detail)
}
var many strings.Builder
for i := 0; i < 400; i++ {
many.WriteString("SYMBOL_WITH_A_LONG_NAME_" + strings.Repeat("X", 30) + "(1.00)[];")
}
if _, detail := rspamdVerdict("reject", many.String()); len(detail) > verdictMaxBytes || strings.Count(detail, "(") > verdictMaxSymbols {
t.Errorf("detail of %d bytes with %d symbols", len(detail), strings.Count(detail, "("))
}
if action, _ := rspamdVerdict(strings.Repeat("a", 33), ""); action != "" {
t.Errorf("an action of 33 letters was kept")
}
}
// A second message on the same connection starts clean.
func TestInboundResetsBetweenMessages(t *testing.T) {
rec := &fakeInbound{domains: map[string]bool{"in.example.com": true}}
s := inboundSessionFor(t, rec, true)
message(t, s, "one@example.net", []string{"a@in.example.com"}, [][2]string{
{"Subject", "First"}, {"X-Rspamd-Action", "add header"},
})
message(t, s, "two@example.net", []string{"b@in.example.com"}, nil)
if len(rec.rows) != 2 {
t.Fatalf("rows = %+v", rec.rows)
}
want := store.InboundSpamEntry{
InboundDomain: "in.example.com", ClientIP: "203.0.113.7",
From: "two@example.net", To: "b@in.example.com",
Decision: store.SpamDecisionAccept, Engine: store.SpamEngineSelfPost,
}
if rec.rows[1] != want {
t.Errorf("second row = %+v\nwant %+v", rec.rows[1], want)
}
// An aborted transaction leaves nothing behind for the next one either.
if _, err := s.MailFrom("three@example.net", mods(nil)); err != nil {
t.Fatal(err)
}
if _, err := s.RcptTo("c@in.example.com", mods(nil)); err != nil {
t.Fatal(err)
}
if err := s.Abort(mods(nil)); err != nil {
t.Fatal(err)
}
if len(rec.rows) != 2 || len(s.rcpts) != 0 {
t.Errorf("after an abort: rows %d, pending recipients %v", len(rec.rows), s.rcpts)
}
}
// The journal failing is never the mail's problem.
func TestInboundAcceptsWhenTheStoreFails(t *testing.T) {
for name, rec := range map[string]*fakeInbound{
"insert": {domains: map[string]bool{"in.example.com": true}, insertErr: errors.New("boom")},
"lookup": {lookupErr: errors.New("boom")},
} {
s := inboundSessionFor(t, rec, false)
if resp := message(t, s, "sender@example.net", []string{"a@in.example.com"}, nil); resp != milter.RespAccept {
t.Errorf("%s failure: response = %v, want accept", name, resp)
}
}
}
// rspamd writes symbols heaviest first, so an allow entry's -100 comes last.
// Filter lists counts an entry's matches by finding that symbol in the journal:
// it is kept whatever else there is.
func TestRspamdVerdictKeepsTheListSymbols(t *testing.T) {
result := "default: False [-91.00 / 15.00]; SYM_A(3.00)[]; SYM_B(2.00)[]; SYM_C(1.50)[]; SYM_D(1.00)[]; SYM_E(0.80)[]; SYM_F(0.50)[]; SYM_G(0.20)[]; ARC_NA(0.00)[]; SELFPOST_ALLOW_FROM(-100.00)[billing@vendor.example]"
_, detail := rspamdVerdict("no action", result)
want := "no action, score -91.00 / 15.00; SELFPOST_ALLOW_FROM(-100.00) SYM_A(3.00) SYM_B(2.00) SYM_C(1.50) SYM_D(1.00) SYM_E(0.80)"
if detail != want {
t.Errorf("detail = %q\nwant %q", detail, want)
}
}
+351
View File
@@ -0,0 +1,351 @@
package store
import (
"database/sql"
"errors"
"fmt"
"strings"
"time"
)
// Decisions of the inbound antispam journal. Kept in sync with the CHECK
// constraint of inbound_spam_log.
const (
SpamDecisionAccept = "accept"
SpamDecisionReject = "reject"
SpamDecisionTempfail = "tempfail"
SpamDecisionQuarantine = "quarantine"
)
// Who made a decision of the inbound antispam journal. Kept in sync with the
// CHECK constraint of inbound_spam_log.
const (
SpamEngineRspamd = "rspamd" // the external filter
SpamEnginePostfix = "postfix" // Postfix's own policy: unknown recipient, size, rate
SpamEngineSelfPost = "selfpost" // nobody: relayed with no filter attached
)
// InboundSpamEntry is one row of the inbound antispam journal: what happened
// to a message offered on port 25 for one recipient. Metadata only - the
// message itself is relayed, not stored. Subject and Detail may be empty.
type InboundSpamEntry struct {
InboundDomain string
ClientIP string
From string
To string
Subject string
Decision string
Engine string
Detail string
// At is when it happened; zero means now. The log tailer dates a refusal
// by its mail.log line, which it may read long after.
At time.Time
}
// InsertInboundSpam records one journal row. It is called from the inbound
// journal milter's hot path and from the log tailer, so it returns any error
// for the caller to log; neither of them may hold up mail over it.
func (s *Store) InsertInboundSpam(e InboundSpamEntry) error {
at := e.At
if at.IsZero() {
at = time.Now()
}
_, err := s.db.Exec(
`INSERT INTO inbound_spam_log
(inbound_domain, client_ip, from_addr, to_addr, subject, decision, engine, detail, created_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)`,
e.InboundDomain, e.ClientIP, e.From, e.To, e.Subject, e.Decision, e.Engine, e.Detail,
at.UTC().Format(time.RFC3339),
)
if err != nil {
return fmt.Errorf("insert inbound_spam_log: %w", err)
}
return nil
}
// PruneInboundSpam deletes journal rows older than cutoff and then, if more
// than keep rows are left, the oldest of those. It returns how many went.
func (s *Store) PruneInboundSpam(cutoff time.Time, keep int) (int64, error) {
res, err := s.db.Exec("DELETE FROM inbound_spam_log WHERE created_at < ?", cutoff.UTC().Format(time.RFC3339))
if err != nil {
return 0, fmt.Errorf("prune inbound_spam_log by age: %w", err)
}
n, _ := res.RowsAffected()
res, err = s.db.Exec(
`DELETE FROM inbound_spam_log WHERE id <=
(SELECT id FROM inbound_spam_log ORDER BY id DESC LIMIT 1 OFFSET ?)`, keep)
if err != nil {
return n, fmt.Errorf("prune inbound_spam_log by count: %w", err)
}
m, _ := res.RowsAffected()
return n + m, nil
}
// InboundSpamRow is one row of the inbound antispam journal as it is read
// back for the panel: the entry as it was written, with its id and when it
// happened.
type InboundSpamRow struct {
ID int64
InboundDomain string
ClientIP string
From string
To string
Subject string
Decision string
Engine string
Detail string
CreatedAt time.Time
}
// InboundSpamFilter narrows QueryInboundSpam and CountInboundSpam. Like
// SendLogFilter it carries two kinds of narrowing that behave in opposite ways
// on purpose.
//
// Domain, Decision, Since and Until are the operator's own filters, chosen in
// the UI: an empty field matches everything. Since and Until are days, both
// inclusive and read as UTC days whatever the time of day they carry.
//
// Domains is the authorization scope, which no query parameter may widen: the
// exhaustive set of inbound domain names the caller may read, applied as an IN
// constraint. nil lifts it (the global role reads the whole journal, including
// rows of a domain deleted since); an empty set is a caller who reaches
// nothing and gets nothing. Unlike SendLogFilter, forgetting the scope is
// therefore the permissive case - the caller states "all" by leaving it out -
// so every caller that serves someone other than the global role must set it.
type InboundSpamFilter struct {
Domain string
Decision string
Since time.Time
Until time.Time
Domains []string
}
// QueryInboundSpam returns journal rows matching filter, newest first, for
// the Inbound log's server-side pagination. A refusal is dated by the mail.log
// line the tailer read it from, which can be older than rows written since, so
// the order is by time and not by id.
func (s *Store) QueryInboundSpam(filter InboundSpamFilter, limit, offset int) ([]InboundSpamRow, error) {
where, args := inboundSpamWhere(filter)
args = append(args, limit, offset)
rows, err := s.db.Query(
`SELECT id, inbound_domain, client_ip, from_addr, to_addr, subject, decision, engine, detail, created_at
FROM inbound_spam_log`+where+`
ORDER BY created_at DESC, id DESC
LIMIT ? OFFSET ?`,
args...,
)
if err != nil {
return nil, fmt.Errorf("query inbound_spam_log: %w", err)
}
defer rows.Close()
var out []InboundSpamRow
for rows.Next() {
var (
row InboundSpamRow
created string
)
if err := rows.Scan(&row.ID, &row.InboundDomain, &row.ClientIP, &row.From, &row.To,
&row.Subject, &row.Decision, &row.Engine, &row.Detail, &created); err != nil {
return nil, fmt.Errorf("scan inbound_spam_log row: %w", err)
}
row.CreatedAt, _ = time.Parse(time.RFC3339, created)
out = append(out, row)
}
return out, rows.Err()
}
// CountInboundSpam returns how many journal rows match filter, for the page
// numbers under the Inbound log.
func (s *Store) CountInboundSpam(filter InboundSpamFilter) (int64, error) {
where, args := inboundSpamWhere(filter)
var n int64
if err := s.db.QueryRow(`SELECT COUNT(*) FROM inbound_spam_log`+where, args...).Scan(&n); err != nil {
return 0, fmt.Errorf("count inbound_spam_log: %w", err)
}
return n, nil
}
func inboundSpamWhere(f InboundSpamFilter) (string, []any) {
var clauses []string
var args []any
if f.Domains != nil {
if len(f.Domains) == 0 {
return " WHERE 1 = 0", nil
}
marks := make([]string, len(f.Domains))
for i, name := range f.Domains {
marks[i] = "?"
args = append(args, name)
}
clauses = append(clauses, "inbound_domain IN ("+strings.Join(marks, ", ")+")")
}
if f.Domain != "" {
clauses = append(clauses, "inbound_domain = ?")
args = append(args, f.Domain)
}
if f.Decision != "" {
clauses = append(clauses, "decision = ?")
args = append(args, f.Decision)
}
// created_at is RFC3339 UTC, so comparing it as text against the same
// format is comparing moments: from midnight of the first day up to, not
// including, midnight after the last.
if !f.Since.IsZero() {
y, m, d := f.Since.UTC().Date()
clauses = append(clauses, "created_at >= ?")
args = append(args, time.Date(y, m, d, 0, 0, 0, 0, time.UTC).Format(time.RFC3339))
}
if !f.Until.IsZero() {
y, m, d := f.Until.UTC().Date()
clauses = append(clauses, "created_at < ?")
args = append(args, time.Date(y, m, d+1, 0, 0, 0, 0, time.UTC).Format(time.RFC3339))
}
if len(clauses) == 0 {
return "", nil
}
return " WHERE " + strings.Join(clauses, " AND "), args
}
// IsInboundDomain reports whether name is a configured inbound domain. Port 25
// also takes mail that is not the relay's - DMARC reports for the server's own
// name - and the journal keeps rows for inbound domains only.
func (s *Store) IsInboundDomain(name string) (bool, error) {
var one int
err := s.db.QueryRow("SELECT 1 FROM inbound_domains WHERE name = ?", strings.ToLower(name)).Scan(&one)
if errors.Is(err, sql.ErrNoRows) {
return false, nil
}
if err != nil {
return false, fmt.Errorf("look up inbound domain: %w", err)
}
return true, nil
}
// Lists and entry kinds of the filter lists. Kept in sync with the CHECK
// constraints of inbound_spam_list.
const (
SpamListAllow = "allow"
SpamListDeny = "deny"
SpamEntryAddress = "address" // sender address
SpamEntryDomain = "domain" // sender domain
SpamEntryIP = "ip" // client address or network
)
// ErrSpamListEntryExists is returned when the entry is already on that list.
var ErrSpamListEntryExists = errors.New("filter list entry already exists")
// ErrSpamListEntryNotFound is returned when a filter list entry id does not exist.
var ErrSpamListEntryNotFound = errors.New("filter list entry not found")
// SpamListEntry is one allow or deny entry for the external filter. Value is
// stored normalised by the caller (internal/inbound), which is what makes the
// uniqueness of an entry on its list mean anything.
type SpamListEntry struct {
ID int64
List string
Kind string
Value string
Note string
CreatedAt time.Time
}
// AddSpamListEntry inserts an entry. The caller must have validated it.
func (s *Store) AddSpamListEntry(list, kind, value, note string) (SpamListEntry, error) {
now := time.Now().UTC()
res, err := s.db.Exec(
"INSERT INTO inbound_spam_list (list_type, entry_type, value, note, created_at) VALUES (?, ?, ?, ?, ?)",
list, kind, value, note, now.Format(time.RFC3339),
)
if err != nil {
if isUniqueViolation(err) {
return SpamListEntry{}, ErrSpamListEntryExists
}
return SpamListEntry{}, fmt.Errorf("insert inbound_spam_list: %w", err)
}
id, err := res.LastInsertId()
if err != nil {
return SpamListEntry{}, fmt.Errorf("filter list entry id: %w", err)
}
return SpamListEntry{ID: id, List: list, Kind: kind, Value: value, Note: note, CreatedAt: now.Truncate(time.Second)}, nil
}
// ListSpamListEntries returns every entry of both lists: by list, then kind,
// then value.
func (s *Store) ListSpamListEntries() ([]SpamListEntry, error) {
rows, err := s.db.Query(
"SELECT id, list_type, entry_type, value, note, created_at FROM inbound_spam_list ORDER BY list_type, entry_type, value")
if err != nil {
return nil, fmt.Errorf("list inbound_spam_list: %w", err)
}
defer rows.Close()
var out []SpamListEntry
for rows.Next() {
var e SpamListEntry
var created string
if err := rows.Scan(&e.ID, &e.List, &e.Kind, &e.Value, &e.Note, &created); err != nil {
return nil, fmt.Errorf("scan inbound_spam_list: %w", err)
}
e.CreatedAt, _ = time.Parse(time.RFC3339, created)
out = append(out, e)
}
return out, rows.Err()
}
// DeleteSpamListEntry removes an entry by id.
func (s *Store) DeleteSpamListEntry(id int64) error {
res, err := s.db.Exec("DELETE FROM inbound_spam_list WHERE id = ?", id)
if err != nil {
return fmt.Errorf("delete inbound_spam_list: %w", err)
}
if n, _ := res.RowsAffected(); n == 0 {
return ErrSpamListEntryNotFound
}
return nil
}
// SpamListHit is what the journal holds about one message a filter list
// decided: who it was from, which server offered it, when, and the detail that
// says which list it was.
type SpamListHit struct {
ClientIP string
From string
Decision string
Detail string
CreatedAt time.Time
}
// ListSpamListHits returns the journal rows of the filter in whose detail one
// of the marks occurs - the marks the filter lists leave (internal/inbound).
// Which entry a row belongs to is for the caller to work out; here the rows
// are only narrowed, so that a page view does not read the whole journal.
func (s *Store) ListSpamListHits(marks ...string) ([]SpamListHit, error) {
if len(marks) == 0 {
return nil, nil
}
likes := make([]string, len(marks))
args := []any{SpamEngineRspamd}
for i, m := range marks {
likes[i] = "instr(detail, ?) > 0"
args = append(args, m)
}
rows, err := s.db.Query(
`SELECT client_ip, from_addr, decision, detail, created_at FROM inbound_spam_log
WHERE engine = ? AND (`+strings.Join(likes, " OR ")+`)`, args...)
if err != nil {
return nil, fmt.Errorf("query filter list hits: %w", err)
}
defer rows.Close()
var out []SpamListHit
for rows.Next() {
var h SpamListHit
var created string
if err := rows.Scan(&h.ClientIP, &h.From, &h.Decision, &h.Detail, &created); err != nil {
return nil, fmt.Errorf("scan filter list hit: %w", err)
}
h.CreatedAt, _ = time.Parse(time.RFC3339, created)
out = append(out, h)
}
return out, rows.Err()
}
+286
View File
@@ -0,0 +1,286 @@
package store
import (
"testing"
"time"
)
func TestInsertInboundSpam(t *testing.T) {
st := openTestStore(t)
e := InboundSpamEntry{
InboundDomain: "in.example.com", ClientIP: "203.0.113.7",
From: "sender@example.net", To: "a@in.example.com", Subject: "Hello",
Decision: SpamDecisionAccept, Engine: SpamEngineRspamd, Detail: "no action, score 0.10 / 15.00",
}
if err := st.InsertInboundSpam(e); err != nil {
t.Fatalf("InsertInboundSpam: %v", err)
}
var got InboundSpamEntry
var created string
err := st.db.QueryRow(
"SELECT inbound_domain, client_ip, from_addr, to_addr, subject, decision, engine, detail, created_at FROM inbound_spam_log",
).Scan(&got.InboundDomain, &got.ClientIP, &got.From, &got.To, &got.Subject, &got.Decision, &got.Engine, &got.Detail, &created)
if err != nil {
t.Fatalf("read back: %v", err)
}
if got != e {
t.Errorf("row = %+v\nwant %+v", got, e)
}
if at, err := time.Parse(time.RFC3339, created); err != nil || time.Since(at) > time.Minute {
t.Errorf("created_at = %q (%v)", created, err)
}
// Every decision and engine the code names is one the table takes.
for _, d := range []string{SpamDecisionAccept, SpamDecisionReject, SpamDecisionTempfail, SpamDecisionQuarantine} {
for _, eng := range []string{SpamEngineRspamd, SpamEnginePostfix, SpamEngineSelfPost} {
e.Decision, e.Engine = d, eng
if err := st.InsertInboundSpam(e); err != nil {
t.Errorf("decision %s by %s: %v", d, eng, err)
}
}
}
e.Decision = "bounce"
if err := st.InsertInboundSpam(e); err == nil {
t.Error("an unknown decision was stored")
}
}
func TestIsInboundDomain(t *testing.T) {
st := openTestStore(t)
if _, err := st.AddInboundDomain("in.example.com"); err != nil {
t.Fatalf("AddInboundDomain: %v", err)
}
for name, want := range map[string]bool{
"in.example.com": true, "IN.Example.COM": true,
"example.com": false, "": false, "other.example.com": false,
} {
if got, err := st.IsInboundDomain(name); err != nil || got != want {
t.Errorf("IsInboundDomain(%q) = %v, %v; want %v", name, got, err, want)
}
}
}
func TestInsertInboundSpamKeepsItsOwnTime(t *testing.T) {
st := openTestStore(t)
at := time.Date(2026, 10, 1, 9, 30, 0, 0, time.FixedZone("msk", 3*3600))
err := st.InsertInboundSpam(InboundSpamEntry{
InboundDomain: "in.example.com", ClientIP: "203.0.113.7", To: "a@in.example.com",
Decision: SpamDecisionReject, Engine: SpamEnginePostfix, At: at,
})
if err != nil {
t.Fatalf("InsertInboundSpam: %v", err)
}
var created string
if err := st.db.QueryRow("SELECT created_at FROM inbound_spam_log").Scan(&created); err != nil || created != "2026-10-01T06:30:00Z" {
t.Errorf("created_at = %q, %v", created, err)
}
}
func TestPruneInboundSpam(t *testing.T) {
st := openTestStore(t)
now := time.Now().UTC()
add := func(to string, age time.Duration) {
t.Helper()
err := st.InsertInboundSpam(InboundSpamEntry{
InboundDomain: "in.example.com", ClientIP: "203.0.113.7", To: to,
Decision: SpamDecisionReject, Engine: SpamEnginePostfix, At: now.Add(-age),
})
if err != nil {
t.Fatalf("InsertInboundSpam: %v", err)
}
}
left := func() string {
t.Helper()
rows, err := st.db.Query("SELECT to_addr FROM inbound_spam_log ORDER BY id")
if err != nil {
t.Fatal(err)
}
defer rows.Close()
var out string
for rows.Next() {
var to string
if err := rows.Scan(&to); err != nil {
t.Fatal(err)
}
out += to + " "
}
return out
}
add("old", 100*24*time.Hour)
add("a", 3*time.Hour)
add("b", 2*time.Hour)
add("c", time.Hour)
// Age first: nothing else goes while the rest fits.
if n, err := st.PruneInboundSpam(now.AddDate(0, 0, -90), 10); err != nil || n != 1 {
t.Fatalf("prune by age = %d, %v", n, err)
}
if got := left(); got != "a b c " {
t.Fatalf("left = %q", got)
}
// Then the cap, oldest first.
if n, err := st.PruneInboundSpam(now.AddDate(0, 0, -90), 2); err != nil || n != 1 {
t.Fatalf("prune by count = %d, %v", n, err)
}
if got := left(); got != "b c " {
t.Fatalf("left = %q", got)
}
if n, err := st.PruneInboundSpam(now.AddDate(0, 0, -90), 2); err != nil || n != 0 {
t.Errorf("a second prune removed %d, %v", n, err)
}
}
func TestInboundSpamLogRetentionDays(t *testing.T) {
st := openTestStore(t)
if days, err := st.GetInboundSpamLogRetentionDays(30); err != nil || days != 30 {
t.Errorf("unset, env 30: %d, %v", days, err)
}
if days, err := st.GetInboundSpamLogRetentionDays(3); err != nil || days != InboundSpamLogRetentionDaysDefault {
t.Errorf("unset, env out of range: %d, %v", days, err)
}
if err := st.EnsureInboundSpamLogRetentionDays(30); err != nil {
t.Fatalf("Ensure: %v", err)
}
if err := st.EnsureInboundSpamLogRetentionDays(60); err != nil {
t.Fatalf("Ensure again: %v", err)
}
if days, _ := st.GetInboundSpamLogRetentionDays(90); days != 30 {
t.Errorf("after Ensure(30), Ensure(60): %d, want the first seed kept", days)
}
if err := st.SetInboundSpamLogRetentionDays(6); err == nil {
t.Error("6 days was accepted")
}
if err := st.SetInboundSpamLogRetentionDays(365); err != nil {
t.Errorf("365 days: %v", err)
}
if days, _ := st.GetInboundSpamLogRetentionDays(90); days != 365 {
t.Errorf("after Set(365): %d", days)
}
}
func TestQueryInboundSpamFilters(t *testing.T) {
st := openTestStore(t)
day := func(d, h int) time.Time { return time.Date(2026, 9, d, h, 0, 0, 0, time.UTC) }
add := func(domain, to, decision string, at time.Time) {
t.Helper()
err := st.InsertInboundSpam(InboundSpamEntry{
InboundDomain: domain, ClientIP: "203.0.113.7", From: "s@example.net", To: to,
Subject: "S " + to, Decision: decision, Engine: SpamEngineRspamd, Detail: "d", At: at,
})
if err != nil {
t.Fatalf("InsertInboundSpam: %v", err)
}
}
add("a.example", "1@a.example", SpamDecisionAccept, day(20, 23))
add("a.example", "2@a.example", SpamDecisionReject, day(21, 0))
add("b.example", "3@b.example", SpamDecisionAccept, day(21, 12))
add("b.example", "4@b.example", SpamDecisionTempfail, day(22, 0))
// Written last but dated earliest: the tailer reads a refusal after the fact.
add("a.example", "5@a.example", SpamDecisionReject, day(19, 8))
to := func(f InboundSpamFilter) string {
t.Helper()
rows, err := st.QueryInboundSpam(f, 50, 0)
if err != nil {
t.Fatalf("QueryInboundSpam(%+v): %v", f, err)
}
n, err := st.CountInboundSpam(f)
if err != nil || n != int64(len(rows)) {
t.Fatalf("CountInboundSpam(%+v) = %d, %v; the query returned %d", f, n, err, len(rows))
}
var out string
for _, r := range rows {
out += r.To[:1]
}
return out
}
// Newest first by time, not by the order of writing.
if got := to(InboundSpamFilter{}); got != "43215" {
t.Errorf("no filter = %q, want 43215", got)
}
if got := to(InboundSpamFilter{Domain: "a.example"}); got != "215" {
t.Errorf("domain = %q, want 215", got)
}
if got := to(InboundSpamFilter{Decision: SpamDecisionReject}); got != "25" {
t.Errorf("decision = %q, want 25", got)
}
// Days are whole UTC days and both ends are in: 21 st is 00:00 up to 23:59.
if got := to(InboundSpamFilter{Since: day(21, 15), Until: day(21, 3)}); got != "32" {
t.Errorf("one day = %q, want 32", got)
}
if got := to(InboundSpamFilter{Since: day(22, 0)}); got != "4" {
t.Errorf("since = %q, want 4", got)
}
if got := to(InboundSpamFilter{Until: day(20, 0)}); got != "15" {
t.Errorf("until = %q, want 15", got)
}
// A time in another zone is read as the UTC day it falls on.
msk := time.FixedZone("msk", 3*3600)
if got := to(InboundSpamFilter{Since: time.Date(2026, 9, 22, 2, 0, 0, 0, msk)}); got != "432" {
t.Errorf("since in another zone = %q, want 432 (it is the 21st in UTC)", got)
}
if got := to(InboundSpamFilter{Domain: "b.example", Decision: SpamDecisionAccept, Since: day(21, 0), Until: day(21, 0)}); got != "3" {
t.Errorf("all filters = %q, want 3", got)
}
// The scope: nil is everything, a set is its domains, an empty set nothing,
// and a chosen domain outside the set is nothing too.
if got := to(InboundSpamFilter{Domains: []string{"b.example"}}); got != "43" {
t.Errorf("scope = %q, want 43", got)
}
if got := to(InboundSpamFilter{Domains: []string{}}); got != "" {
t.Errorf("empty scope = %q, want nothing", got)
}
if got := to(InboundSpamFilter{Domains: []string{"b.example"}, Domain: "a.example"}); got != "" {
t.Errorf("a chosen domain outside the scope = %q, want nothing", got)
}
// Paging.
rows, err := st.QueryInboundSpam(InboundSpamFilter{}, 2, 2)
if err != nil || len(rows) != 2 || rows[0].To != "2@a.example" || rows[1].To != "1@a.example" {
t.Errorf("page 2 of 2 rows = %+v, %v", rows, err)
}
if rows[0].ID == 0 || !rows[0].CreatedAt.Equal(day(21, 0)) || rows[0].Subject != "S 2@a.example" ||
rows[0].Engine != SpamEngineRspamd || rows[0].Detail != "d" || rows[0].ClientIP != "203.0.113.7" || rows[0].From != "s@example.net" {
t.Errorf("a row read back = %+v", rows[0])
}
// A value is data, not SQL.
if got := to(InboundSpamFilter{Domain: "a.example' OR '1'='1"}); got != "" {
t.Errorf("injected domain = %q", got)
}
}
func TestListSpamListHits(t *testing.T) {
st := openTestStore(t)
add := func(engine, decision, detail string) {
t.Helper()
err := st.InsertInboundSpam(InboundSpamEntry{
InboundDomain: "in.example.com", ClientIP: "203.0.113.7", From: "a@example.net", To: "a@in.example.com",
Decision: decision, Engine: engine, Detail: detail, At: time.Date(2026, 10, 1, 9, 0, 0, 0, time.UTC),
})
if err != nil {
t.Fatal(err)
}
}
add(SpamEngineRspamd, SpamDecisionReject, "5.7.1 Refused by this server's deny list")
add(SpamEngineRspamd, SpamDecisionAccept, "no action, score -99.00 / 15.00; SELFPOST_ALLOW_IP(-100.00)")
add(SpamEngineRspamd, SpamDecisionReject, "5.7.1 Spam message rejected")
add(SpamEnginePostfix, SpamDecisionReject, "550 5.1.1 this server's deny list")
// a mark with the characters LIKE would treat as wildcards is text
add(SpamEngineRspamd, SpamDecisionAccept, "no action; SELFPOSTXALLOWX(1.00)")
hits, err := st.ListSpamListHits("this server's deny list", "SELFPOST_ALLOW_")
if err != nil || len(hits) != 2 {
t.Fatalf("hits = %+v, %v; want the two rows a list decided", hits, err)
}
h := hits[0]
if h.From != "a@example.net" || h.ClientIP != "203.0.113.7" || h.Decision != SpamDecisionReject || h.CreatedAt.Hour() != 9 {
t.Errorf("hit = %+v", h)
}
if hits, err := st.ListSpamListHits(); err != nil || hits != nil {
t.Errorf("no marks: %v, %v", hits, err)
}
}
@@ -0,0 +1,40 @@
-- Inbound antispam journal and filter lists (inbound-antispam-panel, 2.1.0).
-- The first migration after the 2.0 baseline: DDL only, so a 2.0.x database
-- gains two empty tables and nothing in it changes.
-- What happened to a message offered on port 25, one row per recipient.
-- Metadata only - SelfPost relays inbound mail and stores none of it. Accepted
-- messages are written by the inbound journal milter, refused ones by the
-- mail.log tailer, which is why a row can lack a subject (refused before DATA)
-- and why engine says who decided: the external filter, Postfix's own policy,
-- or nobody (selfpost: relayed with no filter attached).
CREATE TABLE inbound_spam_log (
id INTEGER PRIMARY KEY AUTOINCREMENT,
inbound_domain TEXT NOT NULL,
client_ip TEXT NOT NULL,
from_addr TEXT NOT NULL,
to_addr TEXT NOT NULL,
subject TEXT NOT NULL,
decision TEXT NOT NULL CHECK (decision IN ('accept', 'reject', 'tempfail', 'quarantine')),
engine TEXT NOT NULL CHECK (engine IN ('rspamd', 'postfix', 'selfpost')),
detail TEXT NOT NULL,
created_at TEXT NOT NULL
);
-- Retention prunes by age; the Log page lists newest first, for the whole
-- server or for one domain.
CREATE INDEX idx_inbound_spam_log_created_at ON inbound_spam_log (created_at);
CREATE INDEX idx_inbound_spam_log_domain_created_at ON inbound_spam_log (inbound_domain, created_at);
-- Allow / deny entries for the external filter, instance-wide. The panel
-- writes them out as map files under /data/antispam/; SelfPost itself does not
-- consult them. value is stored normalised, so the same entry cannot be added
-- twice to one list.
CREATE TABLE inbound_spam_list (
id INTEGER PRIMARY KEY AUTOINCREMENT,
list_type TEXT NOT NULL CHECK (list_type IN ('allow', 'deny')),
entry_type TEXT NOT NULL CHECK (entry_type IN ('address', 'domain', 'ip')),
value TEXT NOT NULL,
note TEXT NOT NULL DEFAULT '',
created_at TEXT NOT NULL,
UNIQUE (list_type, entry_type, value)
);
+95 -2
View File
@@ -55,6 +55,8 @@ func TestBaselineSchema(t *testing.T) {
"user_inbound_domains": "user_id inbound_domain_id",
"dmarc_reports": "id domain reporter report_id period_begin period_end received_at contact_email policy_p policy_sp policy_pct policy_adkim policy_aspf pass_count fail_count recipient",
"dmarc_report_records": "id report_row_id source_ip count disposition spf_result dkim_result header_from",
"inbound_spam_log": "id inbound_domain client_ip from_addr to_addr subject decision engine detail created_at",
"inbound_spam_list": "id list_type entry_type value note created_at",
}
rows, err := st.db.Query("SELECT name FROM sqlite_master WHERE type = 'table' AND name NOT LIKE 'sqlite_%' ORDER BY name")
if err != nil {
@@ -99,6 +101,7 @@ func TestBaselineSchema(t *testing.T) {
rows.Close()
wantIdx := []string{
"idx_dmarc_report_records_report", "idx_dmarc_reports_domain", "idx_dmarc_reports_received",
"idx_inbound_spam_log_created_at", "idx_inbound_spam_log_domain_created_at",
"idx_send_log_app_login_created_at", "idx_send_log_created_at", "idx_send_log_domain", "idx_send_log_queue_id",
"idx_sessions_expires_at",
}
@@ -108,8 +111,8 @@ func TestBaselineSchema(t *testing.T) {
}
var version, appID int64
if err := st.db.QueryRow("PRAGMA user_version").Scan(&version); err != nil || version != 1 {
t.Errorf("user_version = %d, %v; want 1", version, err)
if err := st.db.QueryRow("PRAGMA user_version").Scan(&version); err != nil || version != 2 {
t.Errorf("user_version = %d, %v; want 2", version, err)
}
if err := st.db.QueryRow("PRAGMA application_id").Scan(&appID); err != nil || appID != applicationID {
t.Errorf("application_id = %#x, %v; want %#x", appID, err, applicationID)
@@ -156,6 +159,96 @@ func TestReopenKeepsTheDatabase(t *testing.T) {
}
}
// A 2.0.x data directory - the baseline and nothing after it - is brought to
// the head by opening it, with what it holds left alone.
func TestOpenMigratesA20Database(t *testing.T) {
path := filepath.Join(t.TempDir(), "test.db")
baseline, err := migrationsFS.ReadFile("migrations/0001_init.sql")
if err != nil {
t.Fatalf("read the baseline: %v", err)
}
db, err := sql.Open("sqlite", "file:"+path)
if err != nil {
t.Fatalf("create a 2.0 db: %v", err)
}
for _, stmt := range []string{
string(baseline),
"PRAGMA user_version = 1",
"INSERT INTO users (username, password_hash, role, created_at) VALUES ('admin', 'hash', 'global', '2026-10-11T00:00:00Z')",
} {
if _, err := db.Exec(stmt); err != nil {
t.Fatalf("prepare a 2.0 db: %v", err)
}
}
db.Close()
st, err := Open(path)
if err != nil {
t.Fatalf("open a 2.0 database: %v", err)
}
defer st.Close()
var version int
if err := st.db.QueryRow("PRAGMA user_version").Scan(&version); err != nil || version != 2 {
t.Errorf("user_version = %d, %v; want 2", version, err)
}
if ok, err := st.UserExists(); err != nil || !ok {
t.Errorf("the user of the 2.0 database: exists = %v, %v", ok, err)
}
for _, table := range []string{"inbound_spam_log", "inbound_spam_list"} {
if len(tableColumns(t, st, table)) == 0 {
t.Errorf("table %s was not created", table)
}
}
}
// What the antispam tables refuse: a decision, engine or list the panel does
// not know, and the same entry twice in one list.
func TestInboundSpamConstraints(t *testing.T) {
st := openTestStore(t)
logRow := func(decision, engine string) error {
_, err := st.db.Exec(
"INSERT INTO inbound_spam_log (inbound_domain, client_ip, from_addr, to_addr, subject, decision, engine, detail, created_at)"+
" VALUES ('in.example.com', '192.0.2.1', 'a@example.net', 'b@in.example.com', '', ?, ?, '', '2026-10-11T00:00:00Z')",
decision, engine)
return err
}
if err := logRow("reject", "postfix"); err != nil {
t.Fatalf("a reject by postfix: %v", err)
}
if err := logRow("bounce", "postfix"); err == nil {
t.Error("decision bounce was accepted")
}
if err := logRow("accept", "spamassassin"); err == nil {
t.Error("engine spamassassin was accepted")
}
entry := func(list, kind, value string) error {
_, err := st.db.Exec(
"INSERT INTO inbound_spam_list (list_type, entry_type, value, created_at) VALUES (?, ?, ?, '2026-10-11T00:00:00Z')",
list, kind, value)
return err
}
if err := entry("allow", "address", "billing@vendor.example"); err != nil {
t.Fatalf("an allow entry: %v", err)
}
if err := entry("allow", "address", "billing@vendor.example"); err == nil {
t.Error("the same entry was added to one list twice")
}
if err := entry("deny", "address", "billing@vendor.example"); err != nil {
t.Errorf("the same value on the other list: %v", err)
}
if err := entry("block", "address", "x@example.net"); err == nil {
t.Error("list type block was accepted")
}
if err := entry("deny", "subject", "pills"); err == nil {
t.Error("entry type subject was accepted")
}
var note string
if err := st.db.QueryRow("SELECT note FROM inbound_spam_list WHERE list_type = 'allow'").Scan(&note); err != nil || note != "" {
t.Errorf("note of an entry added without one = %q, %v", note, err)
}
}
// A 1.x data directory must be refused, loudly. Its user_version (10 at the
// last 1.x release) would otherwise read as "the baseline is already applied",
// and the panel would start on tables it does not know.
+66
View File
@@ -77,6 +77,72 @@ func (s *Store) EnsureSendLogRetentionDays(envDefault int) error {
return s.SetSendLogRetentionDays(sendLogRetentionFallback(envDefault))
}
const (
// InboundSpamLogRetentionDaysKey is the settings table key for the
// inbound antispam journal's retention. Same range as the send log's.
InboundSpamLogRetentionDaysKey = "inbound_spam_log_retention_days"
InboundSpamLogRetentionDaysMin = 7
InboundSpamLogRetentionDaysMax = 365
// InboundSpamLogRetentionDaysDefault matches
// INBOUND_SPAM_LOG_RETENTION_DAYS when unset.
InboundSpamLogRetentionDaysDefault = 90
)
// ErrInboundSpamLogRetentionDaysOutOfRange is returned when the inbound
// journal's retention is outside the allowed panel range.
var ErrInboundSpamLogRetentionDaysOutOfRange = errors.New("inbound journal retention days out of range")
// ValidateInboundSpamLogRetentionDays checks the panel-allowed window.
func ValidateInboundSpamLogRetentionDays(days int) error {
if days < InboundSpamLogRetentionDaysMin || days > InboundSpamLogRetentionDaysMax {
return fmt.Errorf("%w: must be between %d and %d days", ErrInboundSpamLogRetentionDaysOutOfRange, InboundSpamLogRetentionDaysMin, InboundSpamLogRetentionDaysMax)
}
return nil
}
func inboundSpamLogRetentionFallback(envDefault int) int {
if ValidateInboundSpamLogRetentionDays(envDefault) == nil {
return envDefault
}
return InboundSpamLogRetentionDaysDefault
}
// GetInboundSpamLogRetentionDays returns the effective retention window of
// the inbound journal. When the setting is missing or invalid, envDefault is
// used (INBOUND_SPAM_LOG_RETENTION_DAYS).
func (s *Store) GetInboundSpamLogRetentionDays(envDefault int) (int, error) {
raw, err := s.GetSetting(InboundSpamLogRetentionDaysKey)
if err != nil {
return 0, err
}
days, err := strconv.Atoi(raw)
if err != nil || ValidateInboundSpamLogRetentionDays(days) != nil {
return inboundSpamLogRetentionFallback(envDefault), nil
}
return days, nil
}
// SetInboundSpamLogRetentionDays persists the inbound journal's retention.
func (s *Store) SetInboundSpamLogRetentionDays(days int) error {
if err := ValidateInboundSpamLogRetentionDays(days); err != nil {
return err
}
return s.SetSetting(InboundSpamLogRetentionDaysKey, strconv.Itoa(days))
}
// EnsureInboundSpamLogRetentionDays seeds the setting from envDefault when it
// has never been written.
func (s *Store) EnsureInboundSpamLogRetentionDays(envDefault int) error {
raw, err := s.GetSetting(InboundSpamLogRetentionDaysKey)
if err != nil {
return err
}
if raw != "" {
return nil
}
return s.SetInboundSpamLogRetentionDays(inboundSpamLogRetentionFallback(envDefault))
}
// GetSetting returns a settings value or empty string when missing.
func (s *Store) GetSetting(key string) (string, error) {
var value string
+3
View File
@@ -56,6 +56,9 @@ var routes = struct{ always, inbound, dmarc []string }{
"POST /inbound/domains/{id}/upstream",
"POST /inbound/domains/{id}/recipients",
"GET /inbound/domains/{id}/delete", "POST /inbound/domains/{id}/delete",
"GET /inbound/log",
"GET /inbound/filter-lists", "POST /inbound/filter-lists",
"POST /inbound/filter-lists/{id}/delete",
},
dmarc: []string{
"GET /outbound/dmarc", "GET /outbound/dmarc/domains/{id}", "GET /outbound/dmarc/reports/{id}",
+6
View File
@@ -50,6 +50,9 @@ type Config struct {
// SendLogRetentionEnvDefault is SEND_LOG_RETENTION_DAYS at panel start; used
// as bootstrap and fallback when the settings row is missing or invalid.
SendLogRetentionEnvDefault int
// InboundSpamLogRetentionEnvDefault is INBOUND_SPAM_LOG_RETENTION_DAYS at
// panel start, the same bootstrap and fallback for the inbound journal.
InboundSpamLogRetentionEnvDefault int
}
// Handlers holds dependencies for authenticated panel routes.
@@ -58,6 +61,7 @@ type Handlers struct {
domains *domain.Service
apps *app.Service
inbound *inbound.Service
lists *inbound.Lists
dmarc *dmarc.Service
cfg Config
view *view.Engine
@@ -72,6 +76,7 @@ func New(
domains *domain.Service,
apps *app.Service,
inboundSvc *inbound.Service,
lists *inbound.Lists,
dmarcSvc *dmarc.Service,
cfg Config,
v *view.Engine,
@@ -84,6 +89,7 @@ func New(
domains: domains,
apps: apps,
inbound: inboundSvc,
lists: lists,
dmarc: dmarcSvc,
cfg: cfg,
view: v,
@@ -0,0 +1,247 @@
package handlers
import (
"errors"
"net/http"
"strconv"
"time"
"github.com/mixeme/selfpost/internal/inbound"
"github.com/mixeme/selfpost/internal/store"
"github.com/mixeme/selfpost/internal/web/view"
)
// Inbound › Log and Inbound › Filter lists. The log is for everyone who reaches
// an inbound domain, with the rows of their domains only (requireInbound); the
// lists are instance-wide and so the global role's (requireInboundGlobal), and
// answer 404 to anyone else, like the rest of what only that role may do.
const inboundLogPageSize = 50
// logDay is the form of a day in the log's date filters and in <input
// type="date">.
const logDay = "2006-01-02"
// HandleInboundLog renders the Inbound log: the journal of port 25's decisions,
// filtered server-side by domain, decision and day, a page at a time. A filter
// value that is not one the page offers is dropped rather than refused, and the
// domains a domain administrator may read are decided here, before the query
// runs - the dropdown only offers them.
func (h *Handlers) HandleInboundLog(w http.ResponseWriter, r *http.Request) {
p, ok := h.requireInbound(w, r)
if !ok {
return
}
assigned, err := h.assignedInboundDomains(p)
if err != nil {
logf("panel: inbound log: %v", err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
names := make([]string, 0, len(assigned))
offered := make(map[string]bool, len(assigned))
for _, d := range assigned {
names = append(names, d.Name)
offered[d.Name] = true
}
q := r.URL.Query()
form := view.InLogFilter{}
query := store.InboundSpamFilter{}
if d := q.Get("domain"); offered[d] {
form.Domain, query.Domain = d, d
}
switch d := q.Get("decision"); d {
case store.SpamDecisionAccept, store.SpamDecisionReject, store.SpamDecisionTempfail, store.SpamDecisionQuarantine:
form.Decision, query.Decision = d, d
}
if t, err := time.Parse(logDay, q.Get("from")); err == nil {
form.From, query.Since = t.Format(logDay), t
}
if t, err := time.Parse(logDay, q.Get("to")); err == nil {
form.To, query.Until = t.Format(logDay), t
}
// A global administrator reads the whole journal, including rows left
// behind by a domain that has since been deleted; anyone else only what is
// assigned to them, however empty that is.
if !p.IsGlobal() {
query.Domains = names
}
total, err := h.store.CountInboundSpam(query)
if err != nil {
logf("panel: inbound log: %v", err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
lastPage := 1
if total > 0 {
lastPage = int((total + inboundLogPageSize - 1) / inboundLogPageSize)
}
page := min(parsePage(q.Get("p")), lastPage)
rows, err := h.store.QueryInboundSpam(query, inboundLogPageSize, (page-1)*inboundLogPageSize)
if err != nil {
logf("panel: inbound log: %v", err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
out := view.NewInLog(h.shellMeta(r), h.inboundSpamLogRetentionDays(), names, form)
table := make([]view.InLogRow, len(rows))
for i, row := range rows {
table[i] = out.NewRow(view.InLogRowInput{
ID: row.ID, At: row.CreatedAt, From: row.From, Client: row.ClientIP, To: row.To, Subject: row.Subject,
Decision: row.Decision, Engine: row.Engine, Detail: row.Detail,
})
}
h.view.Render(w, http.StatusOK, "in-log", out.WithRows(table, page, lastPage))
}
// inboundSpamLogRetentionDays returns the effective inbound journal retention
// window.
func (h *Handlers) inboundSpamLogRetentionDays() int {
days, err := h.store.GetInboundSpamLogRetentionDays(h.cfg.InboundSpamLogRetentionEnvDefault)
if err != nil {
logf("panel: inbound log retention: %v", err)
if err := store.ValidateInboundSpamLogRetentionDays(h.cfg.InboundSpamLogRetentionEnvDefault); err == nil {
return h.cfg.InboundSpamLogRetentionEnvDefault
}
return store.InboundSpamLogRetentionDaysDefault
}
return days
}
// ---- Filter lists
func (h *Handlers) requireFilterLists(w http.ResponseWriter, r *http.Request) bool {
if _, ok := h.requireInboundGlobal(w, r); !ok {
return false
}
if h.lists == nil {
http.NotFound(w, r)
return false
}
return true
}
// filterListForm is what the add form shows: the list and kind it offers (only
// values it knows are kept) and the text of the two fields.
type filterListForm struct {
list, kind, value, note string
}
// HandleFilterLists shows the lists. The query fills the add form in - the
// Inbound log's Allow / Deny sender links lead here - and only a list and a
// kind the form offers are taken; the value is just put in the field.
func (h *Handlers) HandleFilterLists(w http.ResponseWriter, r *http.Request) {
if !h.requireFilterLists(w, r) {
return
}
q := r.URL.Query()
form := filterListForm{value: q.Get("value")}
switch l := q.Get("list"); l {
case store.SpamListAllow, store.SpamListDeny:
form.list = l
}
switch k := q.Get("kind"); k {
case store.SpamEntryAddress, store.SpamEntryDomain, store.SpamEntryIP:
form.kind = k
}
h.renderFilterLists(w, r, http.StatusOK, filterListsFlash(r), "", form)
}
func filterListsFlash(r *http.Request) string {
switch {
case r.URL.Query().Has("added"):
return "Entry added."
case r.URL.Query().Has("removed"):
return "Entry removed."
}
return ""
}
func (h *Handlers) renderFilterLists(w http.ResponseWriter, r *http.Request, status int, flash, formErr string, form filterListForm) {
entries, err := h.lists.Entries()
if err != nil {
logf("panel: filter lists: %v", err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
// How often each entry decided a message is counted from the journal. A
// failure there is not a reason to hide the lists: they are shown without
// the counts' worth - every entry "never" - and the log says why.
usage, err := h.lists.Usage()
if err != nil {
logf("panel: filter lists: usage: %v", err)
}
on, _, _ := h.inboundFilter()
in := make([]view.FilterEntryInput, len(entries))
for i, e := range entries {
in[i] = view.FilterEntryInput{ID: e.ID, List: e.List, Kind: e.Kind, Value: e.Value, Note: e.Note, Added: e.CreatedAt,
Hits: usage[e.ID].Hits, LastHit: usage[e.ID].Last}
}
page := view.NewFilterLists(h.shellMeta(r), view.FilterListsInput{
FilterOn: on, LastWritten: h.lists.LastWritten(), RetentionDays: h.inboundSpamLogRetentionDays()}).
WithEntries(in).
WithResult(flash, formErr)
// A blank list or kind leaves the form on its first choice.
if form.list == "" {
form.list = store.SpamListDeny
}
if form.kind == "" {
form.kind = store.SpamEntryAddress
}
h.view.Render(w, status, "in-filter-lists", page.WithForm(form.list, form.kind, form.value, form.note))
}
// HandleFilterListAdd adds an entry and rewrites the map files. A refusal - an
// entry the lists do not take, or one already on the list - shows the page again
// with the message and what was typed.
func (h *Handlers) HandleFilterListAdd(w http.ResponseWriter, r *http.Request) {
if !h.requireFilterLists(w, r) {
return
}
if err := r.ParseForm(); err != nil {
h.renderFilterLists(w, r, http.StatusBadRequest, "", "Invalid form submission.", filterListForm{})
return
}
form := filterListForm{
list: r.PostFormValue("list_type"), kind: r.PostFormValue("entry_type"),
value: r.PostFormValue("value"), note: r.PostFormValue("note"),
}
_, err := h.lists.Add(form.list, form.kind, form.value, form.note)
var refused *inbound.ValidationError
switch {
case err == nil:
http.Redirect(w, r, "/inbound/filter-lists?added=1", http.StatusSeeOther)
case errors.Is(err, store.ErrSpamListEntryExists):
h.renderFilterLists(w, r, http.StatusUnprocessableEntity, "", "That entry is already on the list.", form)
case errors.As(err, &refused):
h.renderFilterLists(w, r, http.StatusUnprocessableEntity, "", refused.Error(), form)
default:
logf("panel: add filter list entry: %v", err)
h.renderFilterLists(w, r, http.StatusInternalServerError, "", "Could not add the entry. Check the logs and try again.", form)
}
}
// HandleFilterListDelete removes an entry and rewrites the map files.
func (h *Handlers) HandleFilterListDelete(w http.ResponseWriter, r *http.Request) {
if !h.requireFilterLists(w, r) {
return
}
id, err := strconv.ParseInt(r.PathValue("id"), 10, 64)
if err != nil || id <= 0 {
http.NotFound(w, r)
return
}
if err := h.lists.Delete(id); err != nil {
if errors.Is(err, store.ErrSpamListEntryNotFound) {
http.NotFound(w, r)
return
}
logf("panel: delete filter list entry %d: %v", id, err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
http.Redirect(w, r, "/inbound/filter-lists?removed=1", http.StatusSeeOther)
}
@@ -0,0 +1,674 @@
package handlers
import (
"fmt"
"net/http"
"net/http/httptest"
"net/url"
"os"
"path/filepath"
"strings"
"testing"
"time"
"github.com/mixeme/selfpost/internal/inbound"
"github.com/mixeme/selfpost/internal/store"
"github.com/mixeme/selfpost/internal/web/auth"
)
// Inbound › Log and Inbound › Filter lists, rendered from the real journal and
// lists by the handlers. Their markup is held to the mockups by the guards in
// internal/web/view; what is checked here is what the handlers read and who may.
// inboundLogStand is an inbound relay with two domains, a journal row for each
// and the lists over a directory the test can read the map files from.
type inboundLogStand struct {
h *Handlers
st *store.Store
dir string
a, b store.InboundDomain
}
func newInboundLogStand(t *testing.T) inboundLogStand {
t.Helper()
h, st := inboundHandlers(t)
s := inboundLogStand{h: h, st: st, dir: filepath.Join(t.TempDir(), "antispam")}
h.lists = inbound.NewLists(st, s.dir)
var err error
if s.a, err = st.AddInboundDomain("a.example.com"); err != nil {
t.Fatal(err)
}
if s.b, err = st.AddInboundDomain("b.example.com"); err != nil {
t.Fatal(err)
}
s.row("a.example.com", "alice@a.example.com", "Hello A", store.SpamDecisionAccept, store.SpamEngineRspamd)
s.row("b.example.com", "bob@b.example.com", "Hello B", store.SpamDecisionAccept, store.SpamEngineRspamd)
return s
}
func (s inboundLogStand) row(domain, to, subject, decision, engine string) {
s.rowAt(domain, to, subject, decision, engine, time.Time{})
}
func (s inboundLogStand) rowAt(domain, to, subject, decision, engine string, at time.Time) {
err := s.st.InsertInboundSpam(store.InboundSpamEntry{
InboundDomain: domain, ClientIP: "203.0.113.9", From: "sender@example.net", To: to, Subject: subject,
Decision: decision, Engine: engine, Detail: "detail of " + subject, At: at,
})
if err != nil {
panic(err)
}
}
// get runs a handler over a GET as p.
func (s inboundLogStand) get(h http.HandlerFunc, target string, p auth.Principal) *httptest.ResponseRecorder {
rec := httptest.NewRecorder()
h(rec, auth.RequestWithPrincipal(httptest.NewRequest(http.MethodGet, target, nil), p))
return rec
}
func (s inboundLogStand) owner(t *testing.T, username string, domains ...store.InboundDomain) auth.Principal {
t.Helper()
var ids []int64
for _, d := range domains {
ids = append(ids, d.ID)
}
return userWith(t, s.st, username, store.RoleDomain, store.Reach{InboundDomainIDs: ids})
}
// The global role reads the whole journal, is offered every domain, and is given
// the way to correct a decision and to change how long rows are kept.
func TestInboundLogForTheGlobalRole(t *testing.T) {
s := newInboundLogStand(t)
s.row("a.example.com", "carol@a.example.com", "Quarantined", store.SpamDecisionQuarantine, store.SpamEngineRspamd)
if err := s.st.SetInboundSpamLogRetentionDays(30); err != nil {
t.Fatal(err)
}
rec := s.get(s.h.HandleInboundLog, "/inbound/log", globalPrincipal)
if rec.Code != http.StatusOK {
t.Fatalf("GET /inbound/log = %d:\n%s", rec.Code, rec.Body.String())
}
body := rec.Body.String()
has(t, "log", body,
`<h1 class="title is-3">Inbound log</h1>`,
`Rows older than 30 days are deleted — change that in <a href="/server/settings">Settings</a>.`,
`alice@<wbr>a.example.com`, `bob@<wbr>b.example.com`, `Hello A`, `Hello B`,
`<option value="a.example.com">a.example.com</option>`, `<option value="b.example.com">b.example.com</option>`,
`class="tag is-success is-light" popovertarget="why-`, `class="tag is-primary is-light" popovertarget="why-`,
`<span class="sp-mono sp-muted sp-small">203.0.113.9</span>`,
`<tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Verdict</th><td>detail of Hello A</td></tr>`,
`<a href="/inbound/filter-lists?list=deny&amp;kind=address&amp;value=sender%40example.net">Deny sender</a>`,
`<a href="/inbound/filter-lists?list=allow&amp;kind=address&amp;value=sender%40example.net">Allow sender</a>`,
`Page 1 of 1`, `Allow / Deny sender opens <a href="/inbound/filter-lists">Filter lists</a>`,
)
// Newest first: the last row written is the first on the page.
if strings.Index(body, `Quarantined`) > strings.Index(body, `Hello B`) || strings.Index(body, `Hello B`) > strings.Index(body, `Hello A`) {
t.Error("the rows are not newest first")
}
lacks(t, "log", body, `data-poll`, `hx-get`)
}
// A domain administrator reads the rows of the inbound domains assigned to them,
// is offered those domains alone, and is not given the lists or the setting.
func TestInboundLogForADomainAdministrator(t *testing.T) {
s := newInboundLogStand(t)
p := s.owner(t, "owns-a", s.a)
body := s.get(s.h.HandleInboundLog, "/inbound/log", p).Body.String()
has(t, "log", body, `Hello A`, `<option value="a.example.com">a.example.com</option>`, `Rows older than 90 days are deleted.`)
lacks(t, "log", body, `Hello B`, `b.example.com`, `href="/server/settings"`, `/inbound/filter-lists`,
`Deny sender`, `Allow sender`, `Allow / Deny`)
// A domain filter outside their reach is dropped, not honoured.
body = s.get(s.h.HandleInboundLog, "/inbound/log?domain=b.example.com", p).Body.String()
has(t, "log", body, `Hello A`)
lacks(t, "log", body, `Hello B`, `b.example.com`)
// One with All sees every domain, one with none sees no log at all.
all := userWith(t, s.st, "owns-all", store.RoleDomain, store.Reach{AllInbound: true})
body = s.get(s.h.HandleInboundLog, "/inbound/log", all).Body.String()
has(t, "log", body, `Hello A`, `Hello B`)
lacks(t, "log", body, `Deny sender`)
outboundOnly := userWith(t, s.st, "outbound-only", store.RoleDomain, store.Reach{AllDomains: true})
if rec := s.get(s.h.HandleInboundLog, "/inbound/log", outboundOnly); rec.Code != http.StatusNotFound {
t.Errorf("an outbound-only administrator got %d, want 404", rec.Code)
}
}
// A domain administrator whose assigned domain has been deleted since has nothing
// to read, not everything.
func TestInboundLogOfADeletedDomainIsEmptyForItsAdministrator(t *testing.T) {
s := newInboundLogStand(t)
p := s.owner(t, "owns-a", s.a)
both := s.owner(t, "owns-both", s.a, s.b)
if err := s.st.DeleteInboundDomain(s.a.ID); err != nil {
t.Fatal(err)
}
body := s.get(s.h.HandleInboundLog, "/inbound/log", p).Body.String()
has(t, "log", body, `No messages logged yet.`)
lacks(t, "log", body, `Hello A`, `Hello B`)
body = s.get(s.h.HandleInboundLog, "/inbound/log", both).Body.String()
has(t, "log", body, `Hello B`)
lacks(t, "log", body, `Hello A`)
// The global role still reads what the deleted domain left in the journal.
has(t, "log", s.get(s.h.HandleInboundLog, "/inbound/log", globalPrincipal).Body.String(), `Hello A`, `Hello B`)
}
func TestInboundLogFilters(t *testing.T) {
s := newInboundLogStand(t)
day := func(d, h int) time.Time { return time.Date(2026, 9, d, h, 0, 0, 0, time.UTC) }
s.rowAt("a.example.com", "r1@a.example.com", "Rejected on the 10th", store.SpamDecisionReject, store.SpamEnginePostfix, day(10, 23))
s.rowAt("a.example.com", "r2@a.example.com", "Rejected on the 11th", store.SpamDecisionReject, store.SpamEnginePostfix, day(11, 0))
s.rowAt("b.example.com", "r3@b.example.com", "Tempfail on the 11th", store.SpamDecisionTempfail, store.SpamEngineRspamd, day(11, 23))
s.rowAt("a.example.com", "r4@a.example.com", "Accepted on the 12th", store.SpamDecisionAccept, store.SpamEngineRspamd, day(12, 0))
rows := func(query string) string {
t.Helper()
rec := s.get(s.h.HandleInboundLog, "/inbound/log"+query, globalPrincipal)
if rec.Code != http.StatusOK {
t.Fatalf("GET /inbound/log%s = %d", query, rec.Code)
}
var got []string
body := rec.Body.String()
for _, n := range []string{"10th", "11th", "12th"} {
for _, kind := range []string{"Rejected", "Tempfail", "Accepted"} {
if strings.Contains(body, kind+" on the "+n) {
got = append(got, kind[:1]+n[:2])
}
}
}
return strings.Join(got, " ")
}
for query, want := range map[string]string{
"?decision=reject": "R10 R11",
"?domain=b.example.com": "T11",
"?from=2026-09-11&to=2026-09-11": "R11 T11",
"?from=2026-09-12": "A12",
"?to=2026-09-10": "R10",
"?domain=a.example.com&decision=reject&from=2026-09-11": "R11",
// A value that is not one the page offers is ignored.
"?decision=bounce": "R10 R11 T11 A12",
"?from=yesterday&to=2026-13-45": "R10 R11 T11 A12",
"?domain=nobody.example.com": "R10 R11 T11 A12",
"?decision=reject%27+OR+%271%27%3D%271": "R10 R11 T11 A12",
"?domain=a.example.com&domain=b.example.com": "R10 R11 A12",
} {
if got := rows(query); got != want {
t.Errorf("GET /inbound/log%s shows %q, want %q", query, got, want)
}
}
// The choice made is the one shown, and the page leads on with it.
body := s.get(s.h.HandleInboundLog, "/inbound/log?domain=a.example.com&decision=reject&from=2026-09-11&to=2026-09-12", globalPrincipal).Body.String()
has(t, "log", body, `<option value="a.example.com" selected>`, `<option value="reject" selected>`,
`name="from" value="2026-09-11"`, `name="to" value="2026-09-12"`)
body = s.get(s.h.HandleInboundLog, "/inbound/log?decision=quarantine", globalPrincipal).Body.String()
has(t, "log", body, `No messages match these filters.`)
}
func TestInboundLogPages(t *testing.T) {
s := newInboundLogStand(t)
for i := 0; i < inboundLogPageSize; i++ {
s.row("a.example.com", fmt.Sprintf("u%d@a.example.com", i), fmt.Sprintf("Message %02d", i), store.SpamDecisionAccept, store.SpamEngineRspamd)
}
// 51 rows with the one the stand wrote: 50 and 1.
first := s.get(s.h.HandleInboundLog, "/inbound/log?domain=a.example.com", globalPrincipal).Body.String()
has(t, "page 1", first, `Page 1 of 2`, `<a href="/inbound/log?domain=a.example.com&amp;p=2">Older →</a>`, `Message 49`, `Message 00`)
lacks(t, "page 1", first, `← Newer`, `Hello A`)
if got := strings.Count(first, `<tr><td`); got != inboundLogPageSize {
t.Errorf("page 1 has %d rows, want %d", got, inboundLogPageSize)
}
second := s.get(s.h.HandleInboundLog, "/inbound/log?domain=a.example.com&p=2", globalPrincipal).Body.String()
has(t, "page 2", second, `Page 2 of 2`, `<a href="/inbound/log?domain=a.example.com">← Newer</a>`, `Hello A`)
lacks(t, "page 2", second, `Older →`, `Message 49`, `Message 00`)
if got := strings.Count(second, `<tr><td`); got != 1 {
t.Errorf("page 2 has %d rows, want 1", got)
}
// A page past the end, or nonsense, is the last page, or the first.
has(t, "page 9", s.get(s.h.HandleInboundLog, "/inbound/log?domain=a.example.com&p=9", globalPrincipal).Body.String(), `Page 2 of 2`, `Hello A`)
has(t, "page x", s.get(s.h.HandleInboundLog, "/inbound/log?domain=a.example.com&p=x", globalPrincipal).Body.String(), `Page 1 of 2`)
}
func TestInboundLogEmptyAndOff(t *testing.T) {
h, _ := inboundHandlers(t)
s := inboundLogStand{h: h}
body := s.get(h.HandleInboundLog, "/inbound/log", globalPrincipal).Body.String()
has(t, "empty log", body, `No messages logged yet.`)
lacks(t, "empty log", body, `<table`, `Page 1`)
h.cfg.InboundEnabled = false
if rec := s.get(h.HandleInboundLog, "/inbound/log", globalPrincipal); rec.Code != http.StatusNotFound {
t.Errorf("with the inbound relay off GET /inbound/log = %d, want 404", rec.Code)
}
}
// What a sender or a server wrote is text on the page.
func TestInboundLogEscapesWhatSendersWrote(t *testing.T) {
s := newInboundLogStand(t)
err := s.st.InsertInboundSpam(store.InboundSpamEntry{
InboundDomain: "a.example.com", ClientIP: "203.0.113.9", From: `"><img src=x onerror=alert(1)>@e.example`,
To: "x@a.example.com", Subject: `<script>alert(3)</script>`, Decision: store.SpamDecisionReject,
Engine: store.SpamEnginePostfix, Detail: `550 <b>no</b>`,
})
if err != nil {
t.Fatal(err)
}
body := s.get(s.h.HandleInboundLog, "/inbound/log", globalPrincipal).Body.String()
lacks(t, "log", body, `<script>alert`, `<img src=x`, `<b>no</b>`)
has(t, "log", body, `&lt;script&gt;alert(3)&lt;/script&gt;`)
}
// ---- Filter lists
func (s inboundLogStand) post(h http.HandlerFunc, target string, form url.Values, p auth.Principal) *httptest.ResponseRecorder {
req := httptest.NewRequest(http.MethodPost, target, strings.NewReader(form.Encode()))
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
if rest, ok := strings.CutPrefix(target, "/inbound/filter-lists/"); ok {
id, _, _ := strings.Cut(rest, "/")
req.SetPathValue("id", id)
}
rec := httptest.NewRecorder()
h(rec, auth.RequestWithPrincipal(req, p))
return rec
}
func (s inboundLogStand) mapFile(t *testing.T, name string) string {
t.Helper()
b, err := os.ReadFile(filepath.Join(s.dir, name))
if err != nil {
t.Fatalf("read %s: %v", name, err)
}
return string(b)
}
func entry(list, kind, value, note string) url.Values {
return url.Values{"list_type": {list}, "entry_type": {kind}, "value": {value}, "note": {note}}
}
// The lists are the global role's: everything about them is 404 to a domain
// administrator and while the inbound relay is off, and nothing is written.
func TestFilterListsAreForTheGlobalRoleOnly(t *testing.T) {
s := newInboundLogStand(t)
e, err := s.h.lists.Add("deny", "domain", "bulk.example", "")
if err != nil {
t.Fatal(err)
}
ops := s.owner(t, "owns-a", s.a)
all := userWith(t, s.st, "owns-all", store.RoleDomain, store.Reach{AllInbound: true, AllDomains: true})
for _, p := range []auth.Principal{ops, all} {
if rec := s.get(s.h.HandleFilterLists, "/inbound/filter-lists", p); rec.Code != http.StatusNotFound {
t.Errorf("GET as %s = %d, want 404", p.Username, rec.Code)
}
if rec := s.post(s.h.HandleFilterListAdd, "/inbound/filter-lists", entry("deny", "domain", "evil.example", ""), p); rec.Code != http.StatusNotFound {
t.Errorf("POST as %s = %d, want 404", p.Username, rec.Code)
}
if rec := s.post(s.h.HandleFilterListDelete, "/inbound/filter-lists/"+itoa(e.ID)+"/delete", nil, p); rec.Code != http.StatusNotFound {
t.Errorf("POST delete as %s = %d, want 404", p.Username, rec.Code)
}
}
if list, _ := s.h.lists.Entries(); len(list) != 1 {
t.Errorf("a domain administrator changed the lists: %+v", list)
}
if got := s.mapFile(t, "deny_domain.map"); strings.Contains(got, "evil.example") || !strings.Contains(got, "bulk.example") {
t.Errorf("deny_domain.map = %q", got)
}
s.h.cfg.InboundEnabled = false
if rec := s.get(s.h.HandleFilterLists, "/inbound/filter-lists", globalPrincipal); rec.Code != http.StatusNotFound {
t.Errorf("with the inbound relay off GET = %d, want 404", rec.Code)
}
if rec := s.post(s.h.HandleFilterListAdd, "/inbound/filter-lists", entry("deny", "domain", "evil.example", ""), globalPrincipal); rec.Code != http.StatusNotFound {
t.Errorf("with the inbound relay off POST = %d, want 404", rec.Code)
}
if rec := s.post(s.h.HandleFilterListDelete, "/inbound/filter-lists/"+itoa(e.ID)+"/delete", nil, globalPrincipal); rec.Code != http.StatusNotFound {
t.Errorf("with the inbound relay off POST delete = %d, want 404", rec.Code)
}
// Nothing to write the lists to is nothing to show.
s.h.cfg.InboundEnabled, s.h.lists = true, nil
if rec := s.get(s.h.HandleFilterLists, "/inbound/filter-lists", globalPrincipal); rec.Code != http.StatusNotFound {
t.Errorf("without lists GET = %d, want 404", rec.Code)
}
}
func TestFilterListsAddAndRemove(t *testing.T) {
s := newInboundLogStand(t)
empty := s.get(s.h.HandleFilterLists, "/inbound/filter-lists", globalPrincipal).Body.String()
has(t, "empty lists", empty, `<h1 class="title is-3">Filter lists</h1>`, `<h2>Deny</h2><div class="sp-end">0 entries · wins over Allow</div>`,
`<h2>Allow</h2><div class="sp-end">0 entries</div>`, `No sender is turned away by hand.`, `No sender is let through by hand.`)
lacks(t, "empty lists", empty, `<table`)
// The domain is normalised, written to the file of its list and kind, and the
// page shows it.
rec := s.post(s.h.HandleFilterListAdd, "/inbound/filter-lists", entry("deny", "domain", " @Bulk.Example ", " newsletter "), globalPrincipal)
if rec.Code != http.StatusSeeOther || rec.Header().Get("Location") != "/inbound/filter-lists?added=1" {
t.Fatalf("add = %d to %q:\n%s", rec.Code, rec.Header().Get("Location"), rec.Body.String())
}
if got := s.mapFile(t, "deny_domain.map"); !strings.Contains(got, "\nbulk.example\n") {
t.Errorf("deny_domain.map = %q", got)
}
if got := s.mapFile(t, "allow_domain.map"); strings.Contains(got, "bulk.example") {
t.Errorf("allow_domain.map = %q", got)
}
rec = s.post(s.h.HandleFilterListAdd, "/inbound/filter-lists", entry("allow", "ip", "198.51.100.77/24", ""), globalPrincipal)
if rec.Code != http.StatusSeeOther {
t.Fatalf("add ip = %d:\n%s", rec.Code, rec.Body.String())
}
if got := s.mapFile(t, "allow_ip.map"); !strings.Contains(got, "\n198.51.100.0/24\n") {
t.Errorf("allow_ip.map = %q", got)
}
page := s.get(s.h.HandleFilterLists, "/inbound/filter-lists?added=1", globalPrincipal).Body.String()
has(t, "lists", page,
`<div class="notification is-success is-light">`, `Entry added.`,
`<h2>Deny</h2><div class="sp-end">1 entry · wins over Allow</div>`, `<h2>Allow</h2><div class="sp-end">1 entry</div>`,
`<td class="sp-mono sp-small">bulk.example</td><td><button type="button" class="tag is-light" popovertarget="entry-`,
`<tr><th>Note</th><td>newsletter</td></tr><tr><th>Added</th><td class="sp-mono sp-small">`+time.Now().UTC().Format("2006-01-02")+`</td></tr><tr><th>Matched</th><td>never in the last 90 days</td></tr>`,
`<td class="sp-mono sp-small">198.51.100.0/24</td>`, `<tr><th>Note</th><td>—</td></tr>`,
`<td class="sp-muted sp-nowrap">never</td>`,
)
lacks(t, "lists", page, `No sender is turned away`)
// Remove the deny entry: gone from the page and from the file, the allow
// entry untouched.
entries, _ := s.h.lists.Entries()
var denyID int64
for _, e := range entries {
if e.List == "deny" {
denyID = e.ID
}
}
has(t, "lists", page, `action="/inbound/filter-lists/`+itoa(denyID)+`/delete"`)
rec = s.post(s.h.HandleFilterListDelete, "/inbound/filter-lists/"+itoa(denyID)+"/delete", nil, globalPrincipal)
if rec.Code != http.StatusSeeOther || rec.Header().Get("Location") != "/inbound/filter-lists?removed=1" {
t.Fatalf("delete = %d to %q", rec.Code, rec.Header().Get("Location"))
}
if got := s.mapFile(t, "deny_domain.map"); strings.Contains(got, "bulk.example") {
t.Errorf("deny_domain.map still holds the entry: %q", got)
}
if got := s.mapFile(t, "allow_ip.map"); !strings.Contains(got, "198.51.100.0/24") {
t.Errorf("allow_ip.map lost its entry: %q", got)
}
page = s.get(s.h.HandleFilterLists, "/inbound/filter-lists?removed=1", globalPrincipal).Body.String()
has(t, "lists", page, `Entry removed.`, `<h2>Deny</h2><div class="sp-end">0 entries`)
lacks(t, "lists", page, `bulk.example <span`)
// An entry that is not there, or an id that is not one.
for _, id := range []string{itoa(denyID), "9999", "x", "0", "-1"} {
if rec := s.post(s.h.HandleFilterListDelete, "/inbound/filter-lists/"+id+"/delete", nil, globalPrincipal); rec.Code != http.StatusNotFound {
t.Errorf("delete %q = %d, want 404", id, rec.Code)
}
}
}
// A refusal comes back with 422 on the same page, the message between the head
// and the form and what was typed kept in it; nothing is written.
func TestFilterListsRefusals(t *testing.T) {
s := newInboundLogStand(t)
if rec := s.post(s.h.HandleFilterListAdd, "/inbound/filter-lists", entry("deny", "address", "win@lottery.example", ""), globalPrincipal); rec.Code != http.StatusSeeOther {
t.Fatalf("add = %d", rec.Code)
}
before := s.mapFile(t, "deny_from.map")
for _, c := range []struct {
name string
form url.Values
want string
}{
{"duplicate", entry("deny", "address", "WIN@lottery.example", "again"), "That entry is already on the list."},
{"not an address", entry("deny", "address", "not-an-address", ""), "is not a valid email address"},
{"not a network", entry("allow", "ip", "192.0.2.0/99", "kept note"), "is not a network in CIDR form"},
{"no kind", entry("deny", "everything", "x", ""), "choose what the entry is"},
{"no list", entry("maybe", "domain", "x.example", ""), "choose the allow or the deny list"},
{"empty", entry("deny", "domain", " ", ""), "domain is required"},
{"long note", entry("deny", "domain", "x.example", strings.Repeat("n", 201)), "note must be at most 200 characters"},
} {
rec := s.post(s.h.HandleFilterListAdd, "/inbound/filter-lists", c.form, globalPrincipal)
if rec.Code != http.StatusUnprocessableEntity {
t.Errorf("%s: %d, want 422:\n%s", c.name, rec.Code, rec.Body.String())
continue
}
body := rec.Body.String()
has(t, c.name, body, `<div class="notification is-danger is-light">`, c.want, `value="`+c.form.Get("value")+`"`)
if c.form.Get("list_type") == "allow" || c.form.Get("list_type") == "deny" {
has(t, c.name, body, `<option value="`+c.form.Get("list_type")+`" selected>`)
}
if c.form.Get("note") != "" && len(c.form.Get("note")) < 100 {
has(t, c.name, body, `value="`+c.form.Get("note")+`"`)
}
}
if got := s.mapFile(t, "deny_from.map"); got != before {
t.Errorf("a refused entry changed deny_from.map:\n%s", got)
}
if e, _ := s.h.lists.Entries(); len(e) != 1 {
t.Errorf("a refused entry was stored: %+v", e)
}
// What was typed is escaped when it comes back.
rec := s.post(s.h.HandleFilterListAdd, "/inbound/filter-lists", entry("deny", "address", `"><script>1</script>`, `<i>x</i>`), globalPrincipal)
if rec.Code != http.StatusUnprocessableEntity {
t.Fatalf("hostile entry = %d", rec.Code)
}
lacks(t, "refused form", rec.Body.String(), `<script>1`, `<i>x</i>`)
}
// The log's Allow / Deny sender links fill the form in. Only a list and a kind
// the form offers are taken; the value is text.
func TestFilterListsPrefillFromTheLog(t *testing.T) {
s := newInboundLogStand(t)
get := func(query string) string {
rec := s.get(s.h.HandleFilterLists, "/inbound/filter-lists"+query, globalPrincipal)
if rec.Code != http.StatusOK {
t.Fatalf("GET %s = %d", query, rec.Code)
}
return rec.Body.String()
}
body := get("?list=allow&kind=address&value=billing%40partner.example")
has(t, "prefilled", body, `<option value="allow" selected>`, `<option value="address" selected>`,
`id="value" name="value" value="billing@partner.example"`)
lacks(t, "prefilled", body, `<option value="deny" selected>`)
body = get("?list=allow&kind=domain&value=partner.example")
has(t, "prefilled", body, `<option value="allow" selected>`, `<option value="domain" selected>`, `value="partner.example"`)
// Unknown choices leave the form on its first; the value is still put in.
body = get("?list=everything&kind=whatever&value=x%40y.example")
has(t, "unknown choices", body, `<option value="deny" selected>`, `<option value="address" selected>`, `value="x@y.example"`)
// The value cannot leave its attribute.
body = get(`?value=` + url.QueryEscape(`"><script>alert(1)</script>`))
lacks(t, "hostile value", body, `<script>alert`)
has(t, "hostile value", body, `value="&#34;&gt;&lt;script&gt;alert(1)&lt;/script&gt;"`)
// Nothing in the query is a flash.
lacks(t, "plain", get(""), `notification`)
}
// The log's links and the form agree: what the log links to is what the form
// accepts, for every kind of sender it can name.
func TestFilterListsAcceptWhatTheLogLinksTo(t *testing.T) {
s := newInboundLogStand(t)
log := s.get(s.h.HandleInboundLog, "/inbound/log", globalPrincipal).Body.String()
has(t, "log", log, `href="/inbound/filter-lists?list=deny&amp;kind=address&amp;value=sender%40example.net"`)
body := s.get(s.h.HandleFilterLists, "/inbound/filter-lists?list=deny&kind=address&value=sender%40example.net", globalPrincipal).Body.String()
has(t, "form", body, `value="sender@example.net"`)
if rec := s.post(s.h.HandleFilterListAdd, "/inbound/filter-lists", entry("deny", "address", "sender@example.net", ""), globalPrincipal); rec.Code != http.StatusSeeOther {
t.Errorf("the entry the log leads to was refused: %d", rec.Code)
}
}
// Box 04 says whether anything reads the lists and when they were last written.
func TestFilterListsWhereTheyGo(t *testing.T) {
s := newInboundLogStand(t)
// Filter off: the lists are kept, the box says nothing reads them, and the
// files have never been written.
body := s.get(s.h.HandleFilterLists, "/inbound/filter-lists", globalPrincipal).Body.String()
has(t, "filter off", body, `<h2>Where the lists go</h2>`, `<span class="tag is-light">not applied</span>`,
`Nothing — the spam filter is off. The lists are kept and can be edited; nothing reads them until <code>INBOUND_ANTISPAM_MILTER</code> is set.`,
`<dt>Last written</dt><dd>never</dd>`, `Deny wins, whichever entries they are.`)
lacks(t, "filter off", body, `is-success is-light">applied`, `rspamd, from map files`)
// Filter on, and the files written.
s.h.cfg.InboundAntispamMilter = "inet:antispam:11332"
if err := s.h.lists.Sync(); err != nil {
t.Fatal(err)
}
stamp := time.Date(2026, 9, 11, 8, 40, 0, 0, time.UTC)
for _, name := range []string{"allow_ip.map", "allow_from.map", "allow_domain.map", "deny_ip.map", "deny_from.map", "deny_domain.map"} {
if err := os.Chtimes(filepath.Join(s.dir, name), stamp, stamp); err != nil {
t.Fatal(err)
}
}
body = s.get(s.h.HandleFilterLists, "/inbound/filter-lists", globalPrincipal).Body.String()
has(t, "filter on", body, `<span class="tag is-success is-light">applied</span>`,
`rspamd, from map files it rereads within seconds of a change`, `<dt>Last written</dt><dd>2026-09-11 08:40 UTC</dd>`,
`/data/antispam/`)
lacks(t, "filter on", body, `not applied`, `Nothing — the spam filter is off`, `INBOUND_ANTISPAM_MILTER`)
}
// ---- Settings and Help
func settingsWithInbound(t *testing.T, on bool) *Handlers {
t.Helper()
h, _ := settingsServer(t)
h.cfg.InboundEnabled = on
h.cfg.InboundSpamLogRetentionEnvDefault = 90
return h
}
func TestSettingsShowsAndSavesTheInboundRetention(t *testing.T) {
h := settingsWithInbound(t, true)
body := getBody(t, h.HandleServerSettings, "/server/settings")
has(t, "settings", body, `Keep outbound log rows, days`, `Keep inbound log rows, days`,
`name="inbound_spam_log_retention_days"`, `value="90"`,
`The inbound log also never holds more than 10 000 rows`)
if err := h.store.SetInboundSpamLogRetentionDays(45); err != nil {
t.Fatal(err)
}
has(t, "settings", getBody(t, h.HandleServerSettings, "/server/settings"), `name="inbound_spam_log_retention_days" type="number" min="7" max="365" step="1" value="45"`)
rec := postFormAs(h.HandleServerSettings, globalPrincipal, "/server/settings", nil,
url.Values{"send_log_retention_days": {"120"}, "inbound_spam_log_retention_days": {"200"}})
if rec.Code != http.StatusSeeOther || rec.Header().Get("Location") != "/server/settings?saved=1" {
t.Fatalf("POST /server/settings = %d to %q:\n%s", rec.Code, rec.Header().Get("Location"), rec.Body.String())
}
if got, _ := h.store.GetSendLogRetentionDays(90); got != 120 {
t.Errorf("outbound retention = %d, want 120", got)
}
if got, _ := h.store.GetInboundSpamLogRetentionDays(90); got != 200 {
t.Errorf("inbound retention = %d, want 200", got)
}
// The Inbound log's lead reads the same setting.
if got := h.inboundSpamLogRetentionDays(); got != 200 {
t.Errorf("inboundSpamLogRetentionDays = %d, want 200", got)
}
}
func TestSettingsRefuseAnInboundRetentionOutOfRange(t *testing.T) {
h := settingsWithInbound(t, true)
for _, bad := range []string{"3", "366", "nine", ""} {
rec := postFormAs(h.HandleServerSettings, globalPrincipal, "/server/settings", nil,
url.Values{"send_log_retention_days": {"120"}, "inbound_spam_log_retention_days": {bad}})
if rec.Code != http.StatusBadRequest {
t.Fatalf("POST (inbound %q) = %d, want 400:\n%s", bad, rec.Code, rec.Body.String())
}
if bad == "3" && !strings.Contains(rec.Body.String(), "between 7 and 365") {
t.Errorf("no range in the refusal:\n%s", rec.Body.String())
}
if bad == "nine" && !strings.Contains(rec.Body.String(), "Inbound log retention must be a whole number of days.") {
t.Errorf("no sentence in the refusal:\n%s", rec.Body.String())
}
// The outbound value that was fine is shown as typed, the bad one as it was.
has(t, "refused form", rec.Body.String(), `name="send_log_retention_days" type="number" min="7" max="365" step="1" value="120"`)
}
// Nothing was saved, not even the outbound value that was fine.
if got, _ := h.store.GetSendLogRetentionDays(90); got != 90 {
t.Errorf("a refused form changed the outbound retention to %d", got)
}
if got, _ := h.store.GetInboundSpamLogRetentionDays(90); got != 90 {
t.Errorf("a refused form changed the inbound retention to %d", got)
}
// A bad outbound value refuses the whole form too.
rec := postFormAs(h.HandleServerSettings, globalPrincipal, "/server/settings", nil,
url.Values{"send_log_retention_days": {"3"}, "inbound_spam_log_retention_days": {"200"}})
if rec.Code != http.StatusBadRequest {
t.Fatalf("POST (outbound 3) = %d", rec.Code)
}
if got, _ := h.store.GetInboundSpamLogRetentionDays(90); got != 90 {
t.Errorf("the inbound retention was saved by a refused form: %d", got)
}
}
// With the relay off there is no inbound journal to keep: the field is not on the
// page and a value posted for it is not read.
func TestSettingsHaveNoInboundRetentionWithTheRelayOff(t *testing.T) {
h := settingsWithInbound(t, false)
body := getBody(t, h.HandleServerSettings, "/server/settings")
has(t, "settings", body, `Keep outbound log rows, days`)
lacks(t, "settings", body, `inbound_spam_log_retention_days`, `Keep inbound log rows`)
rec := postFormAs(h.HandleServerSettings, globalPrincipal, "/server/settings", nil,
url.Values{"send_log_retention_days": {"120"}, "inbound_spam_log_retention_days": {"3"}})
if rec.Code != http.StatusSeeOther {
t.Fatalf("POST = %d:\n%s", rec.Code, rec.Body.String())
}
if got, _ := h.store.GetSendLogRetentionDays(90); got != 120 {
t.Errorf("outbound retention = %d, want 120", got)
}
if raw, _ := h.store.GetSetting(store.InboundSpamLogRetentionDaysKey); raw != "" {
t.Errorf("an inbound retention was saved with the relay off: %q", raw)
}
}
func TestHelpHasTheInboundLogAndFilterListsTopics(t *testing.T) {
h, _ := inboundHandlers(t)
rec := send(h.HandleHelp, &globalPrincipal, "GET", "/help", nil, nil)
has(t, "help", rec.Body.String(),
`id="inbound-log"`, `id="filter-lists"`, `href="#inbound-log"`, `href="#filter-lists"`,
`With the inbound relay on there is a second number, for the Inbound log, with the same range.`)
// Both pages' help links land on them.
s := inboundLogStand{h: h, st: h.store, dir: filepath.Join(t.TempDir(), "antispam")}
h.lists = inbound.NewLists(h.store, s.dir)
has(t, "log", s.get(h.HandleInboundLog, "/inbound/log", globalPrincipal).Body.String(), `href="/help#inbound-log"`)
has(t, "lists", s.get(h.HandleFilterLists, "/inbound/filter-lists", globalPrincipal).Body.String(), `href="/help#filter-lists"`)
}
// An entry's count comes from the journal: a row counts for the entry that
// fits it on the list that decided it.
func TestFilterListsCountMatchesFromTheJournal(t *testing.T) {
s := newInboundLogStand(t)
for _, e := range [][3]string{{"deny", "domain", "bulk.example"}, {"allow", "address", "billing@vendor.example"}, {"deny", "ip", "203.0.113.0/24"}} {
if _, err := s.h.lists.Add(e[0], e[1], e[2], ""); err != nil {
t.Fatalf("Add %v: %v", e, err)
}
}
at := time.Date(2026, 10, 1, 9, 30, 0, 0, time.UTC)
for i, row := range []store.InboundSpamEntry{
{From: "promo@bulk.example", ClientIP: "198.51.100.9", Decision: store.SpamDecisionReject, Engine: store.SpamEngineRspamd, Detail: "5.7.1 Refused by this server's deny list"},
{From: "other@BULK.example", ClientIP: "198.51.100.9", Decision: store.SpamDecisionReject, Engine: store.SpamEngineRspamd, Detail: "5.7.1 Refused by this server's deny list"},
{From: "billing@vendor.example", ClientIP: "192.0.2.5", Decision: store.SpamDecisionAccept, Engine: store.SpamEngineRspamd, Detail: "no action, score -99.00 / 15.00; SELFPOST_ALLOW_FROM(-100.00)"},
// refused by the filter for its own reasons: nobody's match
{From: "promo@bulk.example", ClientIP: "198.51.100.9", Decision: store.SpamDecisionReject, Engine: store.SpamEngineRspamd, Detail: "5.7.1 Spam message rejected"},
} {
row.InboundDomain, row.To, row.At = "a.example.com", "alice@a.example.com", at.Add(time.Duration(i)*time.Hour)
if err := s.st.InsertInboundSpam(row); err != nil {
t.Fatal(err)
}
}
page := s.get(s.h.HandleFilterLists, "/inbound/filter-lists", globalPrincipal).Body.String()
has(t, "lists", page,
`<tr><th>Matched</th><td>2 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-10-01 10:30</td></tr>`,
`<td class="sp-nowrap">2</td>`,
`<tr><th>Matched</th><td>1 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-10-01 11:30</td></tr>`,
`<td class="sp-nowrap">1</td>`,
// the network nobody came from
`<td class="sp-muted sp-nowrap">never</td>`,
)
if got := strings.Count(page, `<td class="sp-muted sp-nowrap">never</td>`); got != 1 {
t.Errorf("%d entries say never, want the one network", got)
}
}
+50 -8
View File
@@ -10,7 +10,8 @@ import (
)
// Server › Settings holds what is true of the whole instance, whoever is
// signed in: log retention and the read-only view of the rate limits. A
// signed in: log retention (the outbound log's, and the inbound log's while the
// inbound relay is on) and the read-only view of the rate limits. A
// user's own name, e-mail, password and DMARC default are on Account
// (handlers_account.go). Before placing a setting here, ask whose it is.
@@ -21,7 +22,7 @@ func (h *Handlers) HandleServerSettings(w http.ResponseWriter, r *http.Request)
}
switch r.Method {
case http.MethodGet:
h.renderServerSettings(w, r, http.StatusOK, "", h.sendLogRetentionDays())
h.renderServerSettings(w, r, http.StatusOK, "", h.currentRetentions())
case http.MethodPost:
h.submitServerSettings(w, r)
default:
@@ -30,8 +31,26 @@ func (h *Handlers) HandleServerSettings(w http.ResponseWriter, r *http.Request)
}
}
func (h *Handlers) renderServerSettings(w http.ResponseWriter, r *http.Request, status int, formErr string, formRetention int) {
page := view.NewSettings(h.shellMeta(r), strconv.Itoa(formRetention), view.FormatRate(h.l1Messages(), h.l1Window()))
// retentions are the two journal retentions the Settings form shows: the
// outbound log's, and the inbound log's while the inbound relay is on.
type retentions struct {
outbound, inbound int
}
func (h *Handlers) currentRetentions() retentions {
cur := retentions{outbound: h.sendLogRetentionDays()}
if h.cfg.InboundEnabled {
cur.inbound = h.inboundSpamLogRetentionDays()
}
return cur
}
func (h *Handlers) renderServerSettings(w http.ResponseWriter, r *http.Request, status int, formErr string, form retentions) {
page := view.NewSettings(h.shellMeta(r), strconv.Itoa(form.outbound), view.FormatRate(h.l1Messages(), h.l1Window()))
// The inbound field exists while there is an inbound journal to keep.
if h.cfg.InboundEnabled {
page.WithInbound(strconv.Itoa(form.inbound))
}
flash := ""
if r.URL.Query().Has("saved") {
flash = "Settings saved."
@@ -41,7 +60,7 @@ func (h *Handlers) renderServerSettings(w http.ResponseWriter, r *http.Request,
}
func (h *Handlers) submitServerSettings(w http.ResponseWriter, r *http.Request) {
current := h.sendLogRetentionDays()
current := h.currentRetentions()
if err := r.ParseForm(); err != nil {
h.renderServerSettings(w, r, http.StatusBadRequest, "Invalid form submission.", current)
return
@@ -51,17 +70,40 @@ func (h *Handlers) submitServerSettings(w http.ResponseWriter, r *http.Request)
h.renderServerSettings(w, r, http.StatusBadRequest, "Send log retention must be a whole number of days.", current)
return
}
form := retentions{outbound: days, inbound: current.inbound}
if err := store.ValidateSendLogRetentionDays(days); err != nil {
h.renderServerSettings(w, r, http.StatusBadRequest, err.Error(), days)
h.renderServerSettings(w, r, http.StatusBadRequest, err.Error(), form)
return
}
if h.cfg.InboundEnabled {
inboundDays, err := strconv.Atoi(strings.TrimSpace(r.PostFormValue("inbound_spam_log_retention_days")))
if err != nil {
h.renderServerSettings(w, r, http.StatusBadRequest, "Inbound log retention must be a whole number of days.", form)
return
}
form.inbound = inboundDays
if err := store.ValidateInboundSpamLogRetentionDays(inboundDays); err != nil {
h.renderServerSettings(w, r, http.StatusBadRequest, err.Error(), form)
return
}
}
if err := h.store.SetSendLogRetentionDays(days); err != nil {
logf("panel: settings: set send-log retention failed: %v", err)
h.renderServerSettings(w, r, http.StatusInternalServerError,
"Could not save send log retention. Please check the logs and try again.", days)
"Could not save send log retention. Please check the logs and try again.", form)
return
}
logf("panel: instance settings updated (send-log retention: %d days)", days)
if h.cfg.InboundEnabled {
if err := h.store.SetInboundSpamLogRetentionDays(form.inbound); err != nil {
logf("panel: settings: set inbound-log retention failed: %v", err)
h.renderServerSettings(w, r, http.StatusInternalServerError,
"Could not save inbound log retention. Please check the logs and try again.", form)
return
}
logf("panel: instance settings updated (send-log retention: %d days, inbound-log retention: %d days)", days, form.inbound)
} else {
logf("panel: instance settings updated (send-log retention: %d days)", days)
}
http.Redirect(w, r, "/server/settings?saved=1", http.StatusSeeOther)
}
+12
View File
@@ -56,6 +56,10 @@ type Settings struct {
Retention Box
RetentionDays string
// Inbound is whether the inbound relay is on, which gives the retention box
// its second field; InboundRetentionDays is what that field shows.
Inbound bool
InboundRetentionDays string
RateLimits Box
RateFacts []Fact
@@ -86,6 +90,14 @@ func NewSettings(m Meta, retention, level1 string) *Settings {
}
}
// WithInbound adds the retention of the Inbound log to the Log retention box.
// days is what the field shows, like retention in NewSettings. The field exists
// only while the inbound relay is on: with it off there is no journal to keep.
func (s *Settings) WithInbound(days string) *Settings {
s.Inbound, s.InboundRetentionDays = true, days
return s
}
// WithResult shows the result of the last action, or its refusal, between the
// head and the form.
func (s *Settings) WithResult(flash, formErr string) *Settings {
+29 -1
View File
@@ -105,18 +105,41 @@ type Tag struct {
Label string
}
// Sheet is what does not fit a table row, opened over the page from the row's
// tag (partials "sheet_tag" and "sheet"; used by the Inbound log, where it says
// why a message was decided the way it was). It is the browser's own popover:
// the tag is a button that names the sheet by ID, so ID is unique on the page.
// Its content is a table of name and value.
type Sheet struct {
ID string
Tag Tag
Title string
Rows []SheetRow
}
// SheetRow is one name and its value in a sheet. A value of several lines is
// one line each; Mono is for what is read as written - a reply, a list of
// symbols.
type SheetRow struct {
Label string
Lines []string
Mono bool
}
// statusClass is the one mapping of status to Bulma tag classes — box 03 of the
// kit page. A page passes the status it knows (a DKIM check says ok, a message
// says deferred) and never picks a colour.
var statusClass = map[string]string{
"ok": "tag is-success is-light", "published": "tag is-success is-light",
"running": "tag is-success is-light", "sent": "tag is-success is-light",
"accept": "tag is-success is-light",
"warn": "tag is-warning is-light", "deferred": "tag is-warning is-light",
"quiet": "tag is-warning is-light",
"quiet": "tag is-warning is-light", "tempfail": "tag is-warning is-light",
"error": "tag is-danger is-light", "fail": "tag is-danger is-light",
"mismatch": "tag is-danger is-light", "bounced": "tag is-danger is-light",
"reject": "tag is-danger is-light",
"quarantine": "tag is-primary is-light", "held": "tag is-primary is-light",
"global": "tag is-primary is-light",
@@ -503,6 +526,9 @@ type Shell struct {
ShowDMARC bool
ShowInbound bool
ShowServer bool
// ShowFilterLists is the Filter lists entry of Inbound: the lists are the
// instance's, so only the global role has them.
ShowFilterLists bool
}
// Frame is what layout.html is executed with: the shell and the page's own
@@ -523,6 +549,8 @@ func (e *Engine) shell(m Meta, copyright, sourceURL string) Shell {
ShowQueue: m.IsGlobal,
ShowOutbound: m.IsGlobal || m.HasOutbound,
ShowInbound: e.inboundEnabled && (m.IsGlobal || m.HasInbound),
ShowFilterLists: e.inboundEnabled && m.IsGlobal,
}
s.ShowDMARC = e.dmarcEnabled && s.ShowOutbound
switch {
+7 -6
View File
@@ -71,7 +71,8 @@ func TestStatusTagMapsEveryStatusOfTheKitTable(t *testing.T) {
"ok": "tag is-success is-light", "published": "tag is-success is-light",
"running": "tag is-success is-light", "sent": "tag is-success is-light",
"warn": "tag is-warning is-light", "deferred": "tag is-warning is-light",
"quiet": "tag is-warning is-light",
"quiet": "tag is-warning is-light",
"accept": "tag is-success is-light", "tempfail": "tag is-warning is-light", "reject": "tag is-danger is-light",
"error": "tag is-danger is-light", "fail": "tag is-danger is-light",
"mismatch": "tag is-danger is-light", "bounced": "tag is-danger is-light",
"quarantine": "tag is-primary is-light", "global": "tag is-primary is-light",
@@ -413,7 +414,7 @@ func TestShellVisibilityFollowsRoleAndFlags(t *testing.T) {
wantIn bool
}{
{"global, everything on", on, Meta{IsGlobal: true},
Shell{Home: "/overview", ShowOverview: true, ShowOutbound: true, ShowQueue: true, ShowDMARC: true, ShowInbound: true, ShowServer: true}, true},
Shell{Home: "/overview", ShowOverview: true, ShowOutbound: true, ShowQueue: true, ShowDMARC: true, ShowInbound: true, ShowServer: true, ShowFilterLists: true}, true},
{"global, inbound and DMARC off", off, Meta{IsGlobal: true},
Shell{Home: "/overview", ShowOverview: true, ShowOutbound: true, ShowQueue: true, ShowServer: true}, true},
{"domain admin with both reaches", on, Meta{HasOutbound: true, HasInbound: true},
@@ -446,19 +447,19 @@ func renderShell(t *testing.T, e *Engine, m Meta) string {
return rec.Body.String()
}
// The menu has the 2.0 entries and none of the ones that are not in 2.0 (plan
// § Routes): no Inbound Log, Queue, Quarantine or Filter lists, no Preflight.
// The menu has the entries that are built and none of the ones that are not (plan
// § Routes): no Inbound Queue or Quarantine, no Preflight.
func TestMenuHoldsTheTwoDotZeroEntries(t *testing.T) {
out := renderShell(t, kitEngine(t), Meta{Title: "t", User: "admin", IsGlobal: true})
for _, href := range []string{
"/overview", "/outbound/domains", "/outbound/log", "/outbound/queue", "/outbound/dmarc",
"/inbound/domains", "/server/health", "/server/log", "/server/backup", "/server/users", "/server/settings",
"/inbound/domains", "/inbound/log", "/inbound/filter-lists", "/server/health", "/server/log", "/server/backup", "/server/users", "/server/settings",
"/account", "/help", "/license",
} {
mustContain(t, out, `href="`+href+`"`)
}
for _, gone := range []string{
"/inbound/log", "/inbound/queue", "/inbound/quarantine", "/inbound/filter-lists", "/server/preflight",
"/inbound/queue", "/inbound/quarantine", "/server/preflight",
`href="/status"`, `href="/domains"`, `href="/users"`, `href="/settings"`,
} {
mustNotContain(t, out, gone)
+50 -1
View File
@@ -38,6 +38,8 @@ var pageFixtures = map[string]func() any{
"in-domains": func() any { return inDomainsFixture() },
"in-domain": func() any { return inDomainFixture() },
"in-domain-delete": func() any { return inDomainDeleteFixture() },
"in-log": func() any { return inLogFixture() },
"in-filter-lists": func() any { return inFilterListsFixture() },
"out-log": func() any { return outLogFixture() },
"out-message": func() any { return outMessageFixture() },
@@ -131,7 +133,7 @@ func accountFixture() *Account {
// settingsFixture is the mockup's Settings: thirty days of log, a limit of 600
// messages an hour.
func settingsFixture() *Settings {
return NewSettings(admin(), "30", FormatRate(600, 3600))
return NewSettings(admin(), "30", FormatRate(600, 3600)).WithInbound("90")
}
// outDomainsFixture is the mockup's Outbound domains: three domains, the first
@@ -411,3 +413,50 @@ func userFixture() *UserForm {
func userDeleteFixture() *UserDelete {
return NewUserDelete(admin(), UserDeleteInput{ID: 3, Username: "acme-ops", AllOutbound: true, Inbound: []string{"acme.io"}})
}
// inLogFixture is the mockup's Inbound log: six rows of a week's mail, as the
// global role sees them - an accepted message scored as spam and one let through
// by an Allow entry, refusals by Postfix and by the filter, and a tempfail.
func inLogFixture() *InLog {
p := NewInLog(admin(), 90, []string{"lists.example.org", "acme.io"},
InLogFilter{From: "2026-09-14", To: "2026-09-21"})
var id int64
row := func(clock, from, client, to, subject, decision, engine, detail string) InLogRow {
at, err := time.Parse("2006-01-02 15:04:05", "2026-09-21 "+clock)
if err != nil {
panic(err)
}
id++
return p.NewRow(InLogRowInput{ID: id, At: at, From: from, Client: client, To: to, Subject: subject,
Decision: decision, Engine: engine, Detail: detail})
}
return p.WithRows([]InLogRow{
row("14:00:18", "t.berg@uni.example", "192.0.2.77", "dev@lists.example.org", "Re: [dev] patch series v3",
"accept", "rspamd", "no action, score 0.40 / 15.00"),
row("13:59:02", "invoice@partner-billing.example", "203.0.113.140", "owner@lists.example.org", "Outstanding invoice — action required",
"accept", "rspamd", "add header, score 9.80 / 15.00; PHISHING(5.00) FORGED_SENDER(3.00) R_SPF_FAIL(1.00)"),
row("13:57:41", "promo@bulk.example", "198.51.100.90", "sales@lists.example.org", "",
"reject", "postfix", "550 5.1.1 <sales@lists.example.org>: Recipient address rejected: User unknown in relay recipient table"),
row("13:51:30", "billing@partner.example", "198.51.100.12", "owner@lists.example.org", "",
"reject", "rspamd", "5.7.1 Spam message rejected"),
row("13:44:02", "hr@partner.example", "192.0.2.31", "jobs@acme.io", "",
"tempfail", "rspamd", "4.7.1 Try again later"),
row("13:30:55", "noreply@github.example", "192.0.2.200", "security@lists.example.org", "[advisory] new report",
"accept", "rspamd", "no action, score -98.10 / 15.00; SELFPOST_ALLOW_FROM(-100.00) R_DKIM_ALLOW(-0.20)"),
}, 1, 38)
}
// inFilterListsFixture is the mockup's Filter lists: three senders denied, two
// allowed, and a filter that reads them.
func inFilterListsFixture() *FilterLists {
day := func(m time.Month, d int) time.Time { return time.Date(2026, m, d, 0, 0, 0, 0, time.UTC) }
at := func(d, h, m int) time.Time { return time.Date(2026, 9, d, h, m, 0, 0, time.UTC) }
return NewFilterLists(admin(), FilterListsInput{FilterOn: true, LastWritten: time.Date(2026, 9, 11, 8, 40, 0, 0, time.UTC), RetentionDays: 90}).
WithEntries([]FilterEntryInput{
{ID: 1, List: "deny", Kind: "domain", Value: "bulk.example", Note: "newsletter nobody subscribed to", Added: day(9, 2), Hits: 41, LastHit: at(21, 13, 57)},
{ID: 2, List: "deny", Kind: "ip", Value: "198.51.100.0/24", Note: "dictionary attack on lists.example.org", Added: day(8, 19), Hits: 1203, LastHit: at(21, 9, 12)},
{ID: 3, List: "deny", Kind: "address", Value: "win@lottery.example", Added: day(7, 30)},
{ID: 4, List: "allow", Kind: "domain", Value: "partner.example", Note: "invoices were scored as spam", Added: day(9, 11), Hits: 7, LastHit: at(19, 16, 40)},
{ID: 5, List: "allow", Kind: "address", Value: "noreply@github.example", Note: "security advisories", Added: day(6, 4), Hits: 18, LastHit: at(21, 13, 30)},
})
}
-2
View File
@@ -46,8 +46,6 @@ const mockupDir = "../../../docs/assets/panel-redesign/panel"
// with its own plan (panel-redesign.md § Room for roadmap items). They are the
// only keys of outlines.json the finished panel may lack.
var deferredScreens = map[string]string{
"in-log": "inbound-antispam-panel, 2.1.0",
"in-filter-lists": "inbound-antispam-panel, 2.1.0",
"in-queue": "queue as a table / inbound queue, undecided",
"in-quarantine": "inbound-quarantine, candidate",
"in-quarantine-message": "inbound-quarantine, candidate",
+24 -1
View File
@@ -52,6 +52,9 @@ const (
HelpUpstream = "upstream"
HelpRecipients = "recipients"
HelpFilter = "filter"
// HelpInboundLog and HelpFilterLists: the plain "log" is the Outbound log's.
HelpInboundLog = "inbound-log"
HelpFilterLists = "filter-lists"
// Server
HelpBackup = "backup"
HelpUsers = "users"
@@ -109,6 +112,8 @@ func NewHelp(m Meta, inbound bool) *Help {
{Label: "Upstream", Icon: "ti-arrow-forward-up", Href: "#" + HelpUpstream},
{Label: "Valid recipients", Icon: "ti-address-book", Href: "#" + HelpRecipients},
{Label: "Spam filter", Icon: "ti-shield-check", Href: "#" + HelpFilter},
{Label: "Inbound log", Icon: "ti-list-details", Href: "#" + HelpInboundLog},
{Label: "Filter lists", Icon: "ti-filter", Href: "#" + HelpFilterLists},
})
}
if m.IsGlobal {
@@ -184,6 +189,24 @@ var inboundTopics = []HelpTopic{
Rich("The spam filter belongs to the server, not to a domain. The server's administrator attaches it in the server's ", Code(".env"),
", and from then on every inbound domain is screened by it before mail is handed on; the box shows on or off and has nothing to switch. Off means mail goes to the upstream unfiltered. While the filter is down, mail is either accepted unfiltered (the default) or deferred, so the sending server tries again later, as the server's setting says; the box states which."),
}},
{ID: HelpInboundLog, Title: "Inbound log", Body: []Text{
Rich("One row per recipient of a message offered on port 25 for an inbound domain, with what was decided: ",
TagOf("accept", ""), " — handed on to the upstream; ", TagOf("reject", ""), " — refused for good; ",
TagOf("tempfail", ""), " — the sender was asked to come back later; ", TagOf("quarantine", ""), " — held by the filter. ",
Em("Why"), " names who decided. ", Code("rspamd"), " is the spam filter, with its action, the score against the reject threshold and the heaviest symbols; ",
Code("postfix"), " is the relay's own rule — an unknown recipient, a sender domain that does not exist — with its reply; ",
Code("selfpost"), " means nobody judged the message: it was relayed with no filter attached. Accept says what this server did, not what the upstream did with the message afterwards.",
Br(), Br(),
"Accepted mail is recorded as it passes; refused mail is read from ", Code("mail.log"),
", so its row has no subject, and a message refused after its text was sent is one row however many recipients it had. A refusal to relay for a domain that is not an inbound one is every scanner on the internet and is not recorded. Rows are kept for the number of days set in Settings, and never more than 10 000."),
}},
{ID: HelpFilterLists, Title: "Filter lists", Body: []Text{
Rich("Two lists for the whole server that correct the spam filter by hand. An entry is a sender address, a sender domain — that domain, not its subdomains — or a client address or network in CIDR form. A sender on the Deny list is refused outright. A sender on the Allow list is given a weight no spam score outweighs, so the filter lets it through. When entries of both lists match one message, ",
Strong("Deny wins"), ".",
Br(), Br(),
"SelfPost does not apply the lists itself: it writes them as files under ", Code("/data/antispam/"),
", and rspamd rereads a file within seconds of a change. With the spam filter off the lists are kept and can be edited, but nothing reads them. The sender matched is the envelope sender — the one in the log — not the From line a mail program shows. Matched counts the messages in the Inbound log that the entry decided, so it reaches back as far as the log does; never means not once in that time, and such an entry is one to consider removing."),
}},
}
var serverTopics = []HelpTopic{
@@ -208,7 +231,7 @@ var serverTopics = []HelpTopic{
}},
{ID: HelpSettings, Title: "Settings", Body: []Text{
Rich("Settings holds what is true of the whole instance. Log retention is how many days the Outbound log keeps its rows, from 7 to 365 and 90 by default; a sweep every six hours deletes older ones, and it is the main driver of how fast ",
Code("/data"), " grows. It does not change the rotation of ", Code("mail.log"),
Code("/data"), " grows. With the inbound relay on there is a second number, for the Inbound log, with the same range. It does not change the rotation of ", Code("mail.log"),
", which is daily with about two weeks kept. The sending rate limit shown under it is level 1, the per-client-IP ceiling set in ",
Code(".env"), ": changing it needs a restart, and no level-2 limit can exceed it."),
}},
+27
View File
@@ -164,3 +164,30 @@ func TestHelpLinksStayInsideWhatTheirViewerIsShown(t *testing.T) {
landsOn("DMARC hub", hub.Recent.Help)
}
}
// The Inbound log and Filter lists have their topics, after the spam filter's, in
// the menu and on the page, for everyone who is shown Inbound; the Settings topic
// mentions the second retention.
func TestHelpHasTheInboundLogAndFilterListsTopics(t *testing.T) {
out := renderSignedIn(t, "help", NewHelp(admin(), true))
pageHas(t, "Help", out,
`<li><a href="#inbound-log"><i class="ti ti-list-details"></i>Inbound log</a></li>`,
`<li><a href="#filter-lists"><i class="ti ti-filter"></i>Filter lists</a></li>`,
`<div class="sp-help-topic" id="inbound-log"><h3>Inbound log</h3>`,
`<span class="tag is-primary is-light">quarantine</span> — held by the filter. <em>Why</em> names who decided.`,
`a message refused after its text was sent is one row however many recipients it had.`,
`and never more than 10 000.`,
`<div class="sp-help-topic" id="filter-lists"><h3>Filter lists</h3>`,
`<strong>Deny wins</strong>.`, `files under <code>/data/antispam/</code>`,
`With the inbound relay on there is a second number, for the Inbound log, with the same range.`)
for _, v := range helpViewers {
out := renderSignedIn(t, "help", NewHelp(v.meta, v.inbound))
shown := strings.Contains(out, `id="inbound-log"`)
if want := v.inbound && (v.meta.IsGlobal || v.meta.HasInbound); shown != want {
t.Errorf("%s: inbound log topic shown = %v, want %v", v.name, shown, want)
}
}
if HelpInboundLog != "inbound-log" || HelpFilterLists != "filter-lists" {
t.Errorf("anchors %q and %q", HelpInboundLog, HelpFilterLists)
}
}
+1 -1
View File
@@ -19,7 +19,7 @@ func TestInDomainShowsTheMockupsBoxes(t *testing.T) {
`<h2>Spam filter</h2>`, `href="#filter"`, `href="/inbound/domains/1/delete"`, `Delete domain`,
)
// What has no source in this build is not drawn.
pageLacks(t, "Inbound domain", out, `rspamd`, `quarantine`, `Quarantine`, `Log for this domain`, `/inbound/log`, `/inbound/quarantine`, `Allow and deny`)
pageLacks(t, "Inbound domain", out, `rspamd`, `quarantine`, `Quarantine`, `Log for this domain`, `/inbound/quarantine`, `Allow and deny`)
}
// The Spam filter box states what the server is set up with, and what of it the
+202
View File
@@ -0,0 +1,202 @@
package view
// The data of Inbound › Filter lists (templates/in_filter_lists.html, mockup
// in-filter-lists.html): the allow and deny lists that correct the external spam
// filter by hand. The lists are instance-wide, so the page is the global role's.
//
// A page is a typed struct that embeds Meta, built by a constructor and filled
// by With… methods; the handler reads the lists, the view decides what is said
// and how it is worded.
import (
"net/url"
"strconv"
"time"
)
const inboundFilterLists = "/inbound/filter-lists"
// FilterListsHref is the address of the page with the add form filled in: list
// is "allow" or "deny", kind "address", "domain" or "ip". The Inbound log links
// its rows here.
func FilterListsHref(list, kind, value string) string {
return inboundFilterLists + "?list=" + url.QueryEscape(list) + "&kind=" + url.QueryEscape(kind) + "&value=" + url.QueryEscape(value)
}
// FilterLists is the data of the page: the form that adds an entry, the two
// lists side by side and what happens to them after that.
type FilterLists struct {
Meta
Head Head
Flash *Flash
Add Box
Action string
ListOptions []Option
KindOptions []Option
// List, Kind, Value and Note are what the form shows: blank, prefilled from
// the Inbound log, or what was typed into a form that was refused.
List string
Kind string
Value string
Note string
Deny FilterList
Allow FilterList
Where Box
WhereFacts []Fact
retentionDays int
}
// FilterList is one of the two boxes of entries.
type FilterList struct {
Box Box
Rows []FilterRow
Empty EmptyState
}
// FilterRow is one entry of a list: the value, the kind of entry as a tag that
// opens the entry's sheet (the note, the day it was added, how often it decided
// a message and when last), how often it decided, and the way to remove it.
// Matched is empty for an entry that never did; the table then says "never".
type FilterRow struct {
Value string
Entry Sheet
Matched string
Remove RowAction
}
// FilterEntryInput is what the page knows of one entry of a list. Hits and
// LastHit are counted from the Inbound log, so they reach back as far as it
// does.
type FilterEntryInput struct {
ID int64
List string
Kind string
Value string
Note string
Added time.Time
Hits int
LastHit time.Time
}
// FilterListsInput is the state of the filter the lists are written for: whether
// a filter is attached at all, and when the map files it reads were last written.
type FilterListsInput struct {
FilterOn bool
LastWritten time.Time
// RetentionDays is how long the Inbound log keeps a row: how far back an
// entry's matches are counted.
RetentionDays int
}
// NewFilterLists builds the page. The form starts on the Deny list for a sender
// address, the commonest correction.
func NewFilterLists(m Meta, in FilterListsInput) *FilterLists {
m.Title, m.Section, m.Page = "Filter lists", "inbound", "filter-lists"
p := &FilterLists{
Meta: m,
Head: Head{
Kicker: "Inbound · global role",
Title: "Filter lists",
Lead: Plain("Correct the spam filter by hand: senders it must always let through, and senders it must always turn away. One pair of lists for the whole instance."),
},
Add: Box{No: "01", Title: "Add an entry", Help: topicLink(HelpFilterLists, "How the lists are applied")},
Action: inboundFilterLists,
ListOptions: []Option{
{Value: "deny", Label: "Deny — always turn away"},
{Value: "allow", Label: "Allow — always let through"},
},
KindOptions: []Option{
{Value: "address", Label: "a sender address"},
{Value: "domain", Label: "a sender domain"},
{Value: "ip", Label: "a client address or network"},
},
List: "deny",
Kind: "address",
Deny: FilterList{Box: Box{No: "02", Title: "Deny"}, Empty: EmptyState{Icon: "ti-filter", Text: Plain("No sender is turned away by hand.")}},
Allow: FilterList{Box: Box{No: "03", Title: "Allow"}, Empty: EmptyState{Icon: "ti-filter", Text: Plain("No sender is let through by hand.")}},
Where: Box{No: "04", Title: "Where the lists go"},
}
p.Deny.Box.End = Plain(plural(0, "entry", "entries") + " · wins over Allow")
p.Allow.Box.End = Plain(plural(0, "entry", "entries"))
p.retentionDays = in.RetentionDays
p.setWhere(in)
return p
}
// setWhere writes box 04: the tag that says whether anything reads the lists,
// and the four facts about where they go. With no filter attached the lists are
// still kept and editable, and the box says so instead of promising an effect.
func (p *FilterLists) setWhere(in FilterListsInput) {
applied := Plain("rspamd, from map files it rereads within seconds of a change")
p.Where.End = Rich(TagOf("ok", "applied"))
if !in.FilterOn {
p.Where.End = Rich(TagOf("off", "not applied"))
applied = Rich("Nothing — the spam filter is off. The lists are kept and can be edited; nothing reads them until ",
Code("INBOUND_ANTISPAM_MILTER"), " is set.")
}
written := "never"
if !in.LastWritten.IsZero() {
written = in.LastWritten.UTC().Format("2006-01-02 15:04") + " UTC"
}
p.WhereFacts = []Fact{
{Label: "Applied by", Value: applied},
{Label: "Files", Value: Rich(Mono("/data/antispam/"), " — six, one for each list and kind of entry")},
{Label: "Last written", Value: Plain(written)},
{Label: "When both lists match", Value: Plain("Deny wins, whichever entries they are.")},
}
}
// WithEntries sets the two tables, each counted in its box head.
func (p *FilterLists) WithEntries(entries []FilterEntryInput) *FilterLists {
p.Deny.Rows, p.Allow.Rows = nil, nil
for _, e := range entries {
id := strconv.FormatInt(e.ID, 10)
window := "in the last " + plural(int64(p.retentionDays), "day", "days")
sheet := []SheetRow{
{Label: "Note", Lines: []string{orDash(e.Note)}},
{Label: "Added", Lines: []string{e.Added.UTC().Format("2006-01-02")}, Mono: true},
}
row := FilterRow{
Value: e.Value,
Remove: RowAction{Label: "Remove", Post: inboundFilterLists + "/" + id + "/delete", Danger: true},
}
if e.Hits > 0 {
row.Matched = strconv.Itoa(e.Hits)
sheet = append(sheet,
SheetRow{Label: "Matched", Lines: []string{row.Matched + " " + window}},
SheetRow{Label: "Last match", Lines: []string{e.LastHit.UTC().Format("2006-01-02 15:04")}, Mono: true})
} else {
sheet = append(sheet, SheetRow{Label: "Matched", Lines: []string{"never " + window}})
}
row.Entry = Sheet{ID: "entry-" + id, Tag: Tag{Label: e.Kind}, Title: e.Value, Rows: sheet}
switch e.List {
case "deny":
p.Deny.Rows = append(p.Deny.Rows, row)
case "allow":
p.Allow.Rows = append(p.Allow.Rows, row)
}
}
p.Deny.Box.End = Plain(plural(int64(len(p.Deny.Rows)), "entry", "entries") + " · wins over Allow")
p.Allow.Box.End = Plain(plural(int64(len(p.Allow.Rows)), "entry", "entries"))
return p
}
// WithForm sets what the add form shows. The handler passes only what it has
// checked to be a list and a kind the form offers, or what was submitted to a
// form it is showing again.
func (p *FilterLists) WithForm(list, kind, value, note string) *FilterLists {
p.List, p.Kind, p.Value, p.Note = list, kind, value, note
return p
}
// WithResult shows the result of the last action, or the refusal of the add
// form, between the head and the first box.
func (p *FilterLists) WithResult(flash, formErr string) *FilterLists {
p.Flash = resultFlash(flash, formErr)
return p
}
+231
View File
@@ -0,0 +1,231 @@
package view
// The data of the Inbound log (templates/in_log.html, mockup in-log.html): what
// port 25 decided about each message offered for an inbound domain, one row per
// recipient. Unlike the Outbound log it does not poll: a decision does not change
// after it is made, and the page is read to find one row, not watched.
//
// A page is a typed struct that embeds Meta, built by a constructor and filled
// by With… methods; the handler reads the journal, the view decides what is
// said and how it is worded.
import (
"net/url"
"strconv"
"strings"
"time"
)
const inboundLog = "/inbound/log"
// InLogFilter is what the filters of the Inbound log are set to, as the strings
// the form sends: a domain name, a decision and two days (2006-01-02). Empty
// means no narrowing.
type InLogFilter struct {
Domain string
Decision string
From string
To string
}
func (f InLogFilter) active() bool { return f != InLogFilter{} }
// query is the query string of a page of the log: the filters and the page, each
// left out when it says nothing, so the first unfiltered page is the bare path.
func (f InLogFilter) query(page int) string {
q := url.Values{}
for name, v := range map[string]string{"domain": f.Domain, "decision": f.Decision, "from": f.From, "to": f.To} {
if v != "" {
q.Set(name, v)
}
}
if page > 1 {
q.Set("p", strconv.Itoa(page))
}
if len(q) == 0 {
return ""
}
return "?" + q.Encode()
}
// InLogHref is the address of a page of the Inbound log.
func InLogHref(f InLogFilter, page int) string { return inboundLog + f.query(page) }
// InLog is the data of the Inbound log: the decisions about mail for the inbound
// domains the viewer reaches, filtered by domain, decision and day, a page at a
// time.
type InLog struct {
Meta
Head Head
Messages Box
Rows []InLogRow
Empty EmptyState
Foot Foot
filter InLogFilter
}
// InLogRow is one recipient of one message: when it was decided, who the mail
// was from (and the address of the server that offered it) and for, its subject,
// the decision as a tag that opens the sheet saying who decided and why, and —
// for the global role, who alone edits the filter lists — the way to correct
// the decision.
type InLogRow struct {
Time string
From string
Client string
To string
Subject string
Why Sheet
Action *Anchor
}
// InLogRowInput is what the journal holds about one row. Subject, Detail and
// From may be empty: refused mail is read from mail.log, which has no subject,
// and a filter that said nothing has no detail.
type InLogRowInput struct {
ID int64
At time.Time
From string
Client string
To string
Subject string
Decision string
Engine string
Detail string
}
// NewInLog builds the log for a viewer. domains are the choices of the domain
// filter (only the inbound domains the viewer reaches) and filter is what is in
// force. retentionDays is how long rows are kept; the global role, who can
// change it and who edits the lists, is read from m.
func NewInLog(m Meta, retentionDays int, domains []string, filter InLogFilter) *InLog {
m.Title, m.Section, m.Page = "Inbound log", "inbound", "log"
lead := Rich("What port 25 decided about each message, one row per recipient. Accepted mail is recorded as it passes, refused mail is read from ",
Code("mail.log"), "; the filter's verdict appears when the spam filter is on. Rows older than "+plural(int64(retentionDays), "day", "days")+" are deleted")
if m.IsGlobal {
lead = append(lead, Inline{Text: " — change that in "}, Link("/server/settings", "Settings"))
}
lead = append(lead, Inline{Text: "."})
domainOptions := []Option{{Value: "", Label: "All domains"}}
for _, d := range domains {
domainOptions = append(domainOptions, Option{Value: d, Label: d})
}
decisionOptions := []Option{{Value: "", Label: "Any decision"}}
for _, d := range []string{"accept", "reject", "tempfail", "quarantine"} {
decisionOptions = append(decisionOptions, Option{Value: d, Label: d})
}
p := &InLog{
Meta: m,
Head: Head{Kicker: "Inbound", Title: "Inbound log", Lead: lead},
Messages: Box{No: "01", Title: "Messages", Help: topicLink(HelpInboundLog, "What the decisions mean"), Filter: &Filter{
Action: inboundLog,
Selects: []Select{
{Name: "domain", Label: "Domain", Options: domainOptions, Selected: filter.Domain},
{Name: "decision", Label: "Decision", Options: decisionOptions, Selected: filter.Decision},
},
Dates: []DateInput{
{Name: "from", Label: "From date", Value: filter.From},
{Name: "to", Label: "To date", Value: filter.To},
},
}},
Empty: EmptyState{Icon: "ti-list-details", Text: Plain("No messages logged yet.")},
filter: filter,
}
if filter.active() {
p.Empty.Text = Plain("No messages match these filters.")
}
return p
}
// NewRow words one row of the journal for the viewer of the page.
func (p *InLog) NewRow(in InLogRowInput) InLogRow {
r := InLogRow{
Time: FormatLogTime(in.At), From: orDash(in.From), Client: in.Client, To: in.To, Subject: in.Subject,
Why: Sheet{
ID: "why-" + strconv.FormatInt(in.ID, 10), Tag: Tag{Status: in.Decision}, Title: "Why",
Rows: whyRows(in.Engine, in.Detail, in.Subject),
},
}
if p.IsGlobal {
r.Action = senderAction(in.Decision, in.Engine, in.From)
}
return r
}
// whyRows is the content of a row's sheet: who decided, what they said, and
// the subject whole, which the table clips. What was said has one of two
// shapes, told apart by how it starts. The inbound journal milter writes the
// filter's verdict on mail that passed - an action and a score, then the
// symbols after a semicolon. The log tailer writes the reply a refusal was
// given, which starts with its code.
func whyRows(engine, detail, subject string) []SheetRow {
rows := []SheetRow{{Label: "Decided by", Lines: []string{engine}}}
switch {
case detail == "":
case detail[0] >= '0' && detail[0] <= '9':
rows = append(rows, SheetRow{Label: "Reply", Lines: []string{detail}, Mono: true})
default:
verdict, symbols, _ := strings.Cut(detail, "; ")
rows = append(rows, SheetRow{Label: "Verdict", Lines: []string{verdict}})
if symbols != "" {
rows = append(rows, SheetRow{Label: "Symbols", Lines: strings.Fields(symbols), Mono: true})
}
}
if subject != "" {
rows = append(rows, SheetRow{Label: "Subject", Lines: []string{subject}})
}
return rows
}
// senderAction is the way to correct a decision from its row, if there is one:
// a message that got through is the one to deny its sender for, and a message
// the filter turned away, put off or held is the one to allow its sender for.
// A refusal by Postfix's own policy (an unknown recipient, say) offers nothing:
// the message is refused already and the lists would not change what Postfix
// thinks of the recipient. A row with no sender has nobody to list.
func senderAction(decision, engine, from string) *Anchor {
if from == "" {
return nil
}
list := ""
switch {
case decision == "accept":
list = "deny"
case engine == "rspamd":
list = "allow"
default:
return nil
}
label := "Deny sender"
if list == "allow" {
label = "Allow sender"
}
return &Anchor{Href: FilterListsHref(list, "address", from), Label: label}
}
// WithRows sets the table for page of last, and the foot under it: where the
// viewer is, and the way to the newer and older rows. For the global role the
// foot also says where the sender links lead.
func (p *InLog) WithRows(rows []InLogRow, page, last int) *InLog {
p.Rows = rows
if len(rows) == 0 {
return p
}
text := Rich("Page " + strconv.Itoa(page) + " of " + strconv.Itoa(last))
if page > 1 {
text = append(text, Inline{Text: " · "}, Link(InLogHref(p.filter, page-1), "← Newer"))
}
p.Foot.Text = text
if page < last {
p.Foot.Link = &Anchor{Href: InLogHref(p.filter, page+1), Label: "Older →"}
}
if p.IsGlobal {
p.Foot.End = Rich("A decision opens its reason · Allow / Deny sender opens ", Link(inboundFilterLists, "Filter lists"), " with the entry filled in")
} else {
p.Foot.End = Plain("A decision opens its reason")
}
return p
}
+301
View File
@@ -0,0 +1,301 @@
package view
import (
"strings"
"testing"
"time"
)
// ---- The log
// The log is the whole of what the journal holds about the viewer's inbound mail:
// the filters offer what the viewer reaches, the head says how long rows live and
// where to change it, and the global role is given the way to correct a decision.
func TestInLogShowsTheRowsAndTheirFilters(t *testing.T) {
out := renderSignedIn(t, "in-log", inLogFixture())
pageHas(t, "Inbound log", out,
`<h1 class="title is-3">Inbound log</h1>`,
`refused mail is read from <code>mail.log</code>; the filter&#39;s verdict appears when the spam filter is on. Rows older than 90 days are deleted — change that in <a href="/server/settings">Settings</a>.`,
`<form class="sp-end" method="get" action="/inbound/log">`,
`<select name="domain" aria-label="Domain"><option value="" selected>All domains</option><option value="lists.example.org">lists.example.org</option><option value="acme.io">acme.io</option></select>`,
`<select name="decision" aria-label="Decision"><option value="" selected>Any decision</option><option value="accept">accept</option><option value="reject">reject</option><option value="tempfail">tempfail</option><option value="quarantine">quarantine</option></select>`,
`<input class="input" type="date" name="from" value="2026-09-14" aria-label="From date">`,
`<input class="input" type="date" name="to" value="2026-09-21" aria-label="To date">`,
`href="/help#inbound-log" title="What the decisions mean"`,
`<th>Time, UTC</th><th>From</th><th>To</th><th>Subject</th><th>Decision</th><th></th></tr>`,
`09-21 14:00:18`,
`t.berg@<wbr>uni.example<br><span class="sp-mono sp-muted sp-small">192.0.2.77</span>`,
`dev@<wbr>lists.example.org`, `title="Re: [dev] patch series v3"`,
`<td class="sp-clip sp-muted">—</td>`,
// the decision is a tag that opens the sheet with who decided and why
`<button type="button" class="tag is-success is-light" popovertarget="why-1">accept</button><div class="sp-sheet" id="why-1" popover>`,
`<div class="sp-sheet-head"><span class="tag is-success is-light">accept</span><h2>Why</h2><button type="button" popovertarget="why-1" popovertargetaction="hide" aria-label="Close">`,
`<button type="button" class="tag is-danger is-light" popovertarget="why-3">reject</button>`,
`<button type="button" class="tag is-warning is-light" popovertarget="why-5">tempfail</button>`,
`<tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Verdict</th><td>no action, score 0.40 / 15.00</td></tr><tr><th>Subject</th><td>Re: [dev] patch series v3</td></tr>`,
`<tr><th>Verdict</th><td>add header, score 9.80 / 15.00</td></tr><tr><th>Symbols</th><td class="sp-mono sp-small">PHISHING(5.00)<br>FORGED_SENDER(3.00)<br>R_SPF_FAIL(1.00)</td></tr>`,
`<tr><th>Decided by</th><td>postfix</td></tr><tr><th>Reply</th><td class="sp-mono sp-small">550 5.1.1 &lt;sales@lists.example.org&gt;: Recipient address rejected: User unknown in relay recipient table</td></tr></tbody>`,
`Page 1 of 38`, `<a href="/inbound/log?from=2026-09-14&amp;p=2&amp;to=2026-09-21">Older →</a>`,
`A decision opens its reason · Allow / Deny sender opens <a href="/inbound/filter-lists">Filter lists</a> with the entry filled in`,
)
pageLacks(t, "Inbound log", out, `← Newer`, `data-poll`, `hx-get`, `<th>Why</th>`)
// A message Postfix itself refused offers no way into the lists: one Deny
// for each of the three accepted messages, one Allow for each the filter
// refused or put off.
if deny, allow := strings.Count(out, `>Deny sender</a>`), strings.Count(out, `>Allow sender</a>`); deny != 3 || allow != 2 {
t.Errorf("the fixture's six rows offer Deny %d times and Allow %d times, want 3 and 2", deny, allow)
}
// Without the global role there is nowhere to change the retention, and
// nowhere to send a sender.
domain := NewInLog(Meta{User: "ops", HasInbound: true}, 7, []string{"acme.io"}, InLogFilter{})
out = renderSignedIn(t, "in-log", domain.WithRows([]InLogRow{domain.NewRow(InLogRowInput{
At: time.Date(2026, 9, 21, 14, 0, 0, 0, time.UTC), From: "a@example.net", To: "b@acme.io",
Decision: "accept", Engine: "rspamd"})}, 1, 1))
pageHas(t, "Inbound log", out, `Rows older than 7 days are deleted.`, `rspamd</td>`)
pageLacks(t, "Inbound log", out, `/server/settings`, `/inbound/filter-lists`, `Deny sender`, `Allow sender`, `Allow / Deny`)
}
// The lead's day count reads right for one day too, though the setting's range
// starts at seven: the word follows the number, not the other way round.
func TestInLogLeadCountsDays(t *testing.T) {
out := renderSignedIn(t, "in-log", NewInLog(Meta{User: "ops", HasInbound: true}, 1, nil, InLogFilter{}))
pageHas(t, "Inbound log", out, `Rows older than 1 day are deleted.`)
}
// The filter in force is the one selected, and the paging and the empty state
// carry and name it.
func TestInLogCarriesItsFilters(t *testing.T) {
f := InLogFilter{Domain: "acme.io", Decision: "reject", From: "2026-09-01", To: "2026-09-30"}
p := NewInLog(admin(), 90, []string{"lists.example.org", "acme.io"}, f)
p.WithRows([]InLogRow{p.NewRow(InLogRowInput{At: time.Date(2026, 9, 2, 1, 2, 3, 0, time.UTC),
From: "a@example.net", To: "b@acme.io", Decision: "reject", Engine: "postfix"})}, 3, 5)
out := renderSignedIn(t, "in-log", p)
pageHas(t, "Inbound log", out,
`<option value="acme.io" selected>acme.io</option>`, `<option value="reject" selected>reject</option>`,
`name="from" value="2026-09-01"`, `name="to" value="2026-09-30"`,
`Page 3 of 5 · <a href="/inbound/log?decision=reject&amp;domain=acme.io&amp;from=2026-09-01&amp;p=2&amp;to=2026-09-30">← Newer</a>`,
`<a href="/inbound/log?decision=reject&amp;domain=acme.io&amp;from=2026-09-01&amp;p=4&amp;to=2026-09-30">Older →</a>`,
)
if got := InLogHref(InLogFilter{}, 1); got != "/inbound/log" {
t.Errorf("the first unfiltered page is %q", got)
}
if got := InLogHref(InLogFilter{Domain: "a b&c"}, 2); got != "/inbound/log?domain=a+b%26c&p=2" {
t.Errorf("a filter is not encoded: %q", got)
}
out = renderSignedIn(t, "in-log", NewInLog(admin(), 90, nil, f).WithRows(nil, 1, 1))
pageHas(t, "Inbound log", out, `No messages match these filters.`)
out = renderSignedIn(t, "in-log", NewInLog(admin(), 90, nil, InLogFilter{}).WithRows(nil, 1, 1))
pageHas(t, "Inbound log", out, `No messages logged yet.`)
pageLacks(t, "Inbound log", out, `<table`, `Page 1`, `sp-box-foot`)
}
// The last page has nothing older.
func TestInLogLastPage(t *testing.T) {
p := NewInLog(admin(), 90, nil, InLogFilter{})
row := p.NewRow(InLogRowInput{At: time.Now(), From: "a@example.net", To: "b@acme.io", Decision: "accept", Engine: "selfpost"})
out := renderSignedIn(t, "in-log", p.WithRows([]InLogRow{row}, 2, 2))
pageHas(t, "Inbound log", out, `Page 2 of 2`, `← Newer</a>`)
pageLacks(t, "Inbound log", out, `Older →`)
}
// A row says who decided, and what they said when they said something; a message
// refused before its sender was known has no sender to show or to list.
func TestInLogRowWording(t *testing.T) {
p := NewInLog(admin(), 90, nil, InLogFilter{})
at := time.Date(2026, 9, 21, 14, 0, 18, 0, time.UTC)
r := p.NewRow(InLogRowInput{At: at, From: "a@example.net", To: "b@acme.io", Decision: "accept", Engine: "selfpost"})
if len(r.Why.Rows) != 1 || r.Why.Rows[0].Lines[0] != "selfpost" || r.Time != "09-21 14:00:18" || r.Client != "" {
t.Errorf("row = %+v", r)
}
r = p.NewRow(InLogRowInput{ID: 7, At: at, To: "b@acme.io", Decision: "reject", Engine: "postfix", Detail: "550 5.1.1 no"})
if r.Why.ID != "why-7" || r.Why.Tag.Status != "reject" || len(r.Why.Rows) != 2 || r.Why.Rows[1].Label != "Reply" || r.From != "—" || r.Action != nil {
t.Errorf("row = %+v", r)
}
out := renderSignedIn(t, "in-log", p.WithRows([]InLogRow{r}, 1, 1))
pageHas(t, "Inbound log", out, `<td>—</td>`)
pageLacks(t, "Inbound log", out, `<br><span class="sp-mono sp-muted sp-small"></span>`)
}
// Which decisions lead to which list, for the global role: the sender of mail
// that got through can be denied, the sender of mail the filter turned away or
// put off can be allowed, and a refusal that is Postfix's own offers neither.
func TestInLogSenderAction(t *testing.T) {
for _, c := range []struct {
decision, engine, from, label, href string
}{
{"accept", "rspamd", "a@example.net", "Deny sender", "/inbound/filter-lists?list=deny&kind=address&value=a%40example.net"},
{"accept", "selfpost", "a@example.net", "Deny sender", "/inbound/filter-lists?list=deny&kind=address&value=a%40example.net"},
{"reject", "rspamd", "a+b@example.net", "Allow sender", "/inbound/filter-lists?list=allow&kind=address&value=a%2Bb%40example.net"},
{"tempfail", "rspamd", "a@example.net", "Allow sender", "/inbound/filter-lists?list=allow&kind=address&value=a%40example.net"},
{"quarantine", "rspamd", "a@example.net", "Allow sender", "/inbound/filter-lists?list=allow&kind=address&value=a%40example.net"},
{"reject", "postfix", "a@example.net", "", ""},
{"tempfail", "postfix", "a@example.net", "", ""},
{"reject", "selfpost", "a@example.net", "", ""},
{"accept", "rspamd", "", "", ""},
{"reject", "rspamd", "", "", ""},
} {
got := senderAction(c.decision, c.engine, c.from)
switch {
case c.label == "" && got != nil:
t.Errorf("%s by %s from %q is offered %+v", c.decision, c.engine, c.from, *got)
case c.label != "" && (got == nil || got.Label != c.label || got.Href != c.href):
t.Errorf("%s by %s from %q is offered %+v, want %s -> %s", c.decision, c.engine, c.from, got, c.label, c.href)
}
}
// The address in the link is data: it cannot leave the attribute.
p := NewInLog(admin(), 90, nil, InLogFilter{})
row := p.NewRow(InLogRowInput{At: time.Now(), From: `"><script>1</script>@x`, To: "b@acme.io", Decision: "accept", Engine: "rspamd"})
out := renderSignedIn(t, "in-log", p.WithRows([]InLogRow{row}, 1, 1))
pageLacks(t, "Inbound log", out, `<script>1`)
pageHas(t, "Inbound log", out, `value=%22%3E%3Cscript%3E1%3C%2Fscript%3E%40x`)
}
// What a sender wrote is text on the page: an address, a subject or a reply from
// a server is never markup.
func TestInLogEscapesWhatSendersWrote(t *testing.T) {
p := NewInLog(admin(), 90, nil, InLogFilter{})
row := p.NewRow(InLogRowInput{At: time.Now(), From: `<img src=x onerror=alert(1)>@e.example`, Client: `<b>1</b>`,
To: `<script>alert(2)</script>@acme.io`, Subject: `<script>alert(3)</script> "quoted"`,
Decision: "reject", Engine: "rspamd", Detail: `<i>5.7.1</i>`})
out := renderSignedIn(t, "in-log", p.WithRows([]InLogRow{row}, 1, 1))
pageLacks(t, "Inbound log", out, `<script>alert`, `<img src=x`, `<b>1</b>`, `<i>5.7.1</i>`)
pageHas(t, "Inbound log", out, `&lt;script&gt;alert(3)&lt;/script&gt;`, `&lt;i&gt;5.7.1&lt;/i&gt;`)
}
// ---- Filter lists
// The page is the add form, the two lists with their counts, and where the lists
// go - drawn as the mockup draws it.
func TestFilterListsShowsTheFormTheListsAndWhereTheyGo(t *testing.T) {
out := renderSignedIn(t, "in-filter-lists", inFilterListsFixture())
pageHas(t, "Filter lists", out,
`<p class="sp-kicker">Inbound · global role</p>`, `<h1 class="title is-3">Filter lists</h1>`,
`<form class="sp-form" method="post" action="/inbound/filter-lists">`,
`<select id="list" name="list_type">`, `<option value="deny" selected>Deny — always turn away</option>`,
`<option value="allow">Allow — always let through</option>`,
`<select id="etype" name="entry_type">`, `<option value="address" selected>a sender address</option>`,
`<option value="domain">a sender domain</option>`, `<option value="ip">a client address or network</option>`,
`<input class="input sp-mono" id="value" name="value" value="" placeholder="promo@bulk.example"`,
`<input class="input" id="note" name="note" value=""`, `Add entry`,
`href="/help#filter-lists" title="How the lists are applied"`,
`<h2>Deny</h2><div class="sp-end">3 entries · wins over Allow</div>`,
`<h2>Allow</h2><div class="sp-end">2 entries</div>`,
`<thead><tr><th>Value</th><th>Kind</th><th>Matched</th><th></th></tr></thead>`,
// the kind is a tag that opens the entry's sheet; the count is beside it
`<td class="sp-mono sp-small">bulk.example</td><td><button type="button" class="tag is-light" popovertarget="entry-1">domain</button><div class="sp-sheet" id="entry-1" popover>`,
`<div class="sp-sheet-head"><span class="tag is-light">domain</span><h2>bulk.example</h2>`,
`<tr><th>Note</th><td>newsletter nobody subscribed to</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-09-02</td></tr><tr><th>Matched</th><td>41 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-09-21 13:57</td></tr></tbody>`,
`</div></td><td class="sp-nowrap">41</td><td class="sp-actions">`,
`<td class="sp-nowrap">1203</td>`,
// an entry that never decided anything says so, in the row and on its sheet
`<td class="sp-mono sp-small">win@lottery.example</td><td><button type="button" class="tag is-light" popovertarget="entry-3">address</button>`,
`<tr><th>Note</th><td>—</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-07-30</td></tr><tr><th>Matched</th><td>never in the last 90 days</td></tr></tbody>`,
`</div></td><td class="sp-muted sp-nowrap">never</td><td class="sp-actions">`,
`<form method="post" action="/inbound/filter-lists/3/delete"><button type="submit" class="has-text-danger">Remove</button></form>`,
`<h2>Where the lists go</h2><div class="sp-end"><span class="tag is-success is-light">applied</span></div>`,
`rspamd, from map files it rereads within seconds of a change`,
`<span class="sp-mono">/data/antispam/</span> — six, one for each list and kind of entry`,
`2026-09-11 08:40 UTC`, `Deny wins, whichever entries they are.`,
)
if got := strings.Count(out, `<table class="table is-fullwidth is-hoverable">`); got != 2 {
t.Errorf("%d list tables, want the two lists", got)
}
pageLacks(t, "Filter lists", out, `<th>Note</th><th>`, `<th>Added</th><th>`)
}
// With no filter attached the lists are still the operator's to edit, and the
// box says nothing reads them; a list with no entries says so in its box.
func TestFilterListsOffAndEmpty(t *testing.T) {
p := NewFilterLists(admin(), FilterListsInput{}).WithEntries([]FilterEntryInput{
{ID: 9, List: "allow", Kind: "ip", Value: "192.0.2.1", Added: time.Date(2026, 9, 1, 0, 0, 0, 0, time.UTC)},
})
out := renderSignedIn(t, "in-filter-lists", p)
pageHas(t, "Filter lists", out,
`<span class="tag is-light">not applied</span>`,
`Nothing — the spam filter is off. The lists are kept and can be edited; nothing reads them until <code>INBOUND_ANTISPAM_MILTER</code> is set.`,
`<dt>Last written</dt><dd>never</dd>`,
`<h2>Deny</h2><div class="sp-end">0 entries · wins over Allow</div>`,
`<h2>Allow</h2><div class="sp-end">1 entry</div>`,
`No sender is turned away by hand.`,
)
pageLacks(t, "Filter lists", out, `rspamd, from map files`, `is-success is-light">applied`, `No sender is let through by hand.`)
if got := strings.Count(out, `<table class="table is-fullwidth is-hoverable">`); got != 1 {
t.Errorf("%d list tables, want the one list that has entries", got)
}
}
// The form shows what it is told to: prefilled from the log, or what was typed
// into a form that was refused, with the refusal between the head and the form.
func TestFilterListsFormAndRefusal(t *testing.T) {
p := inFilterListsFixture().
WithForm("allow", "domain", `x"><script>1</script>`, `a "note"`).
WithResult("", "That entry is already on the list.")
out := renderSignedIn(t, "in-filter-lists", p)
pageHas(t, "Filter lists", out,
`<option value="allow" selected>Allow — always let through</option>`, `<option value="domain" selected>a sender domain</option>`,
`value="x&#34;&gt;&lt;script&gt;1&lt;/script&gt;"`, `value="a &#34;note&#34;"`,
`<div class="notification is-danger is-light">`, `That entry is already on the list.`)
pageLacks(t, "Filter lists", out, `<script>1`, `<option value="deny" selected>`)
if strings.Index(out, `class="notification`) < strings.Index(out, `class="sp-head"`) ||
strings.Index(out, `class="notification`) > strings.Index(out, `<form class="sp-form"`) {
t.Error("the refusal is not between the head and the form")
}
out = renderSignedIn(t, "in-filter-lists", inFilterListsFixture().WithResult("Entry added.", ""))
pageHas(t, "Filter lists", out, `<div class="notification is-success is-light">`, `Entry added.`)
}
// What a person typed into a list is text on the page.
func TestFilterListsEscapesTheEntries(t *testing.T) {
p := NewFilterLists(admin(), FilterListsInput{FilterOn: true}).WithEntries([]FilterEntryInput{
{ID: 1, List: "deny", Kind: "address", Value: `<img src=x>@a.example`, Note: `<script>alert(1)</script>`, Added: time.Now()},
})
out := renderSignedIn(t, "in-filter-lists", p)
pageLacks(t, "Filter lists", out, `<img src=x>`, `<script>alert`)
}
// The menu leads to both pages, the lists for the global role only.
func TestInboundMenuHasLogAndFilterLists(t *testing.T) {
out := renderSignedIn(t, "in-log", inLogFixture())
pageHas(t, "Inbound log", out,
`<a class="navbar-item is-active" href="/inbound/log"><i class="ti ti-list-details"></i>Log</a>`,
`<a class="navbar-item" href="/inbound/filter-lists"><i class="ti ti-filter"></i>Filter lists</a>`,
`<a href="/inbound/log" aria-current="page"><i class="ti ti-list-details"></i>Log</a><a href="/inbound/filter-lists"><i class="ti ti-filter"></i>Filter lists</a>`)
out = renderSignedIn(t, "in-filter-lists", inFilterListsFixture())
pageHas(t, "Filter lists", out,
`<a class="navbar-item is-active" href="/inbound/filter-lists">`, `<a href="/inbound/filter-lists" aria-current="page">`)
domain := NewInLog(Meta{User: "ops", HasInbound: true}, 90, nil, InLogFilter{}).WithRows(nil, 1, 1)
out = renderSignedIn(t, "in-log", domain)
pageHas(t, "Inbound log", out, `<a class="navbar-item is-active" href="/inbound/log">`, `<a href="/inbound/log" aria-current="page">`)
pageLacks(t, "Inbound log", out, `/inbound/filter-lists`)
}
// What a sheet says, from the two shapes a row's detail has: the filter's
// verdict on mail that passed, and the reply a refusal was given.
func TestWhyRows(t *testing.T) {
flat := func(rows []SheetRow) string {
var out []string
for _, r := range rows {
out = append(out, r.Label+"="+strings.Join(r.Lines, "|"))
}
return strings.Join(out, "; ")
}
for _, c := range []struct{ engine, detail, subject, want string }{
{"selfpost", "", "", "Decided by=selfpost"},
{"selfpost", "", "Hello", "Decided by=selfpost; Subject=Hello"},
{"rspamd", "no action, score 0.40 / 15.00", "Hi", "Decided by=rspamd; Verdict=no action, score 0.40 / 15.00; Subject=Hi"},
{"rspamd", "add header, score 9.80 / 15.00; A(5.00) B(-0.10)", "", "Decided by=rspamd; Verdict=add header, score 9.80 / 15.00; Symbols=A(5.00)|B(-0.10)"},
// a verdict with symbols and no action or score
{"rspamd", "A(5.00)", "", "Decided by=rspamd; Verdict=A(5.00)"},
{"rspamd", "5.7.1 Spam message rejected", "", "Decided by=rspamd; Reply=5.7.1 Spam message rejected"},
{"postfix", "450 4.1.8 <a@nx.example>: Sender address rejected; see x", "", "Decided by=postfix; Reply=450 4.1.8 <a@nx.example>: Sender address rejected; see x"},
} {
if got := flat(whyRows(c.engine, c.detail, c.subject)); got != c.want {
t.Errorf("whyRows(%q, %q, %q)\n got: %s\nwant: %s", c.engine, c.detail, c.subject, got, c.want)
}
}
}
+8 -3
View File
@@ -21,6 +21,7 @@ type Kit struct {
Postmarks []Postmark
StatusTag Box
StatusRows []KitStatusRow
Sheet Sheet
BoxHead Box
BoxFoot Foot
DangerZone Box
@@ -91,10 +92,14 @@ func KitPage() *Kit {
},
StatusTag: Box{No: "03", Title: "status_tag", End: Plain("One mapping, one function")},
Sheet: Sheet{ID: "kit-sheet", Tag: Tag{Status: "reject"}, Title: "sheet", Rows: []SheetRow{
{Label: "What it holds", Lines: []string{"What does not fit the row, as name and value"}},
{Label: "How it closes", Lines: []string{"Esc", "a click outside", "the cross"}, Mono: true},
}},
StatusRows: []KitStatusRow{
{"ok, published, running, sent", Tag{Status: "ok"}},
{"warn, deferred, quiet", Tag{Status: "warn"}},
{"error, fail, mismatch, bounced", Tag{Status: "fail"}},
{"ok, published, running, sent, accept", Tag{Status: "ok"}},
{"warn, deferred, quiet, tempfail", Tag{Status: "warn"}},
{"error, fail, mismatch, bounced, reject", Tag{Status: "fail"}},
{"unknown, off, neutral counts", Tag{Status: "off"}},
{"held: quarantine · role: global", Tag{Status: "quarantine"}},
},
+20
View File
@@ -402,3 +402,23 @@ func TestOverviewFragmentIsTheBoxAndTheHeadOutOfBand(t *testing.T) {
}
}
}
// With the inbound relay on the retention box has a second field, for the Inbound
// log; with it off the field is not there, since there is no journal to keep.
func TestSettingsHasTheInboundRetentionOnlyWithTheRelay(t *testing.T) {
out := renderSignedIn(t, "settings", settingsFixture())
pageHas(t, "Settings", out,
`for="inbound_spam_log_retention_days">Keep inbound log rows, days</label>`,
`name="inbound_spam_log_retention_days" type="number" min="7" max="365" step="1" value="90" required`,
`The inbound log also never holds more than 10 000 rows: refused mail is logged at the rate strangers knock on port 25.`,
`name="send_log_retention_days"`)
if got := strings.Count(out, `<form class="sp-form"`); got != 1 {
t.Errorf("Settings has %d forms, want 1", got)
}
pageLacks(t, "Settings", renderSignedIn(t, "settings", NewSettings(admin(), "30", "600 / h")),
`inbound_spam_log_retention_days`, `Keep inbound log rows`, `10 000 rows`)
// A refused value comes back as typed.
out = renderSignedIn(t, "settings", NewSettings(admin(), "30", "600 / h").WithInbound("3").WithResult("", "Inbound log retention must be between 7 and 365."))
pageHas(t, "Settings", out, `value="3"`, `between 7 and 365`)
}
+24
View File
@@ -327,3 +327,27 @@ code { background: var(--sp-well); color: var(--sp-ink); border-radius: 3px; pad
.sp-record-head small { margin-left: 0; text-align: left; flex-basis: 100%; }
.sp-postmark { display: none; }
}
/* sheet: what does not fit a table row, opened over the page from the row's
tag. It is the browser's own popover (the tag is a button with
popovertarget), so it needs no script: Esc, a click outside and the close
button all shut it, and focus returns to the tag. It rises from the bottom
edge across the whole window - the one component that is not laid out by the
page, hence the widths - with its content, a table of name and value, in a
column of reading width in the middle. */
.sp-sheet {
inset: auto 0 0 0; width: auto; margin: 0; padding: 0; border: 0; border-top: 3px solid var(--sp-brick);
max-height: 75vh; overflow: auto; background: #fff; color: inherit; box-shadow: 0 -12px 40px rgba(40, 20, 15, .22);
}
.sp-sheet-in { max-width: 44rem; margin: 0 auto; padding: 0 1.25rem; }
.sp-sheet::backdrop { background: rgba(40, 20, 15, .38); }
.sp-sheet-head { display: flex; align-items: center; gap: .7rem; padding: 1rem 0 .8rem; border-bottom: 1px solid var(--sp-rule); }
.sp-sheet-head h2 { font-size: 1.15rem; font-weight: 600; margin-right: auto; }
.sp-sheet-head button { border: 0; background: none; cursor: pointer; font-size: 1.3rem; color: var(--sp-muted); }
.sp-sheet-body { padding: .4rem 0 1.4rem; }
/* The sheet's content is a table of name and value: the name is a label, set like the dt of facts. */
.sp-sheet .table th { font-family: var(--sp-mono); font-size: var(--sp-fs-xs); font-weight: 500; letter-spacing: .08em; text-transform: uppercase; color: var(--sp-muted); white-space: nowrap; vertical-align: top; width: 1%; padding-right: 2rem; }
.sp-sheet .table td { overflow-wrap: anywhere; }
/* A tag that opens a sheet is a button: it keeps the tag's look and says it can
be pressed with the dotted underline links to more detail carry. */
button.tag { border: 0; cursor: pointer; text-decoration: underline dotted; text-underline-offset: 3px; }
@@ -27,6 +27,13 @@
mapping of status to Bulma classes; the label defaults to the status. */}}
{{define "tag"}}<span class="{{status_tag .Status}}">{{if .Label}}{{.Label}}{{else}}{{.Status}}{{end}}</span>{{end}}
{{/* sheet_tag is a status tag that opens a sheet (input: Sheet): the tag as a
button, and the sheet it names right after it. sheet is what opens: the tag
again, a title, the cross that closes it, and a table of name and value.
The browser's popover does the rest - no script. */}}
{{define "sheet_tag"}}<button type="button" class="{{status_tag .Tag.Status}}" popovertarget="{{.ID}}">{{if .Tag.Label}}{{.Tag.Label}}{{else}}{{.Tag.Status}}{{end}}</button>{{template "sheet" .}}{{end}}
{{define "sheet"}}<div class="sp-sheet" id="{{.ID}}" popover><div class="sp-sheet-in"><div class="sp-sheet-head">{{template "tag" .Tag}}<h2>{{.Title}}</h2><button type="button" popovertarget="{{.ID}}" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody>{{range .Rows}}<tr><th>{{.Label}}</th><td{{if .Mono}} class="sp-mono sp-small"{{end}}>{{range $i, $line := .Lines}}{{if $i}}<br>{{end}}{{$line}}{{end}}</td></tr>{{end}}</tbody></table></div></div></div></div>{{end}}
{{/* kicker is the upper-case mono micro-label on its own (input: string).
page_head writes its own kicker as the line above the title. */}}
{{define "kicker"}}<span class="sp-kicker">{{.}}</span>{{end}}
@@ -0,0 +1,64 @@
{{/* Inbound > Filter lists (mockup in-filter-lists.html): the allow and deny
lists that correct the spam filter by hand, for the whole instance, and
where they go. Global role only. The form adds an entry; each row of a list
removes its own, says how often the entry decided a message, and opens a
sheet with the entry's note from the tag of its kind. */}}
{{define "content"}}
{{template "page_head" .Head}}
{{if .Flash}}{{template "flash" .Flash}}{{end}}
{{template "box_open" .Add}}
{{template "body_open"}}
<form class="sp-form" method="post" action="/inbound/filter-lists">
<div class="columns">
<div class="column is-3">
<div class="field"><label class="label" for="list">List</label>
<div class="control"><div class="select is-fullwidth"><select id="list" name="list_type">{{$list := .List}}{{range .ListOptions}}
<option value="{{.Value}}"{{if eq .Value $list}} selected{{end}}>{{.Label}}</option>{{end}}
</select></div></div></div>
</div>
<div class="column is-3">
<div class="field"><label class="label" for="etype">Entry is</label>
<div class="control"><div class="select is-fullwidth"><select id="etype" name="entry_type">{{$kind := .Kind}}{{range .KindOptions}}
<option value="{{.Value}}"{{if eq .Value $kind}} selected{{end}}>{{.Label}}</option>{{end}}
</select></div></div></div>
</div>
<div class="column is-6">
<div class="field"><label class="label" for="value">Value</label>
<div class="control"><input class="input sp-mono" id="value" name="value" value="{{.Value}}" placeholder="promo@bulk.example" autocomplete="off" autocapitalize="none" spellcheck="false" required></div></div>
</div>
</div>
<div class="field"><label class="label" for="note">Note</label>
<div class="control"><input class="input" id="note" name="note" value="{{.Note}}" placeholder="Why — for whoever reads this list next year"></div></div>
<button type="submit" class="button is-primary"><i class="ti ti-plus"></i>Add entry</button>
</form>
{{template "body_close"}}
{{template "box_close"}}
<div class="columns">
<div class="column is-6">
{{template "filter_list" .Deny}}
</div>
<div class="column is-6">
{{template "filter_list" .Allow}}
</div>
</div>
{{template "box_open" .Where}}
{{template "body_open"}}
{{template "facts" .WhereFacts}}
{{template "body_close"}}
{{template "box_close"}}
{{end}}
{{/* filter_list is one of the two boxes of entries (input: FilterList): the
table, or what is absent when the list is empty. */}}
{{define "filter_list"}}{{template "box_open" .Box}}
{{if .Rows}}<div class="table-container"><table class="table is-fullwidth is-hoverable">
<thead><tr><th>Value</th><th>Kind</th><th>Matched</th><th></th></tr></thead>
<tbody>{{range .Rows}}
<tr><td class="sp-mono sp-small">{{.Value}}</td><td>{{template "sheet_tag" .Entry}}</td>{{if .Matched}}<td class="sp-nowrap">{{.Matched}}</td>{{else}}<td class="sp-muted sp-nowrap">never</td>{{end}}<td class="sp-actions">{{template "row_action" .Remove}}</td></tr>{{end}}
</tbody>
</table></div>
{{else}}{{template "empty_state" .Empty}}
{{end}}{{template "box_close"}}{{end}}
+23
View File
@@ -0,0 +1,23 @@
{{/* Inbound log (mockup in-log.html): what port 25 decided about each message
offered for an inbound domain, one row per recipient, newest first,
filtered by domain, decision and day. It does not poll. The decision is a
tag that opens a sheet with who decided and why. The last cell is the way
to correct a decision in Filter lists, and only the global role, who alone
edits the lists, is given it. */}}
{{define "content"}}
{{template "page_head" .Head}}
{{template "box_open" .Messages}}
{{if .Rows}}
<div class="table-container"><table class="table is-fullwidth is-hoverable">
<thead><tr><th>Time, UTC</th><th>From</th><th>To</th><th>Subject</th><th>Decision</th><th></th></tr></thead>
<tbody>{{range .Rows}}
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">{{.Time}}</td><td>{{wbr_at .From}}{{if .Client}}<br><span class="sp-mono sp-muted sp-small">{{.Client}}</span>{{end}}</td><td>{{wbr_at .To}}</td>{{if .Subject}}<td class="sp-clip" title="{{.Subject}}">{{.Subject}}</td>{{else}}<td class="sp-clip sp-muted">—</td>{{end}}<td>{{template "sheet_tag" .Why}}</td><td class="sp-actions">{{if .Action}}<a href="{{.Action.Href}}">{{.Action.Label}}</a>{{end}}</td></tr>{{end}}
</tbody>
</table></div>
{{template "box_foot" .Foot}}
{{else}}
{{template "empty_state" .Empty}}
{{end}}
{{template "box_close"}}
{{end}}
+1
View File
@@ -31,6 +31,7 @@
<thead><tr><th>Status</th><th>Tag</th><th>Bulma classes</th></tr></thead>
<tbody>{{range .StatusRows}}
<tr><td>{{.Words}}</td><td>{{template "tag" .Tag}}</td><td class="sp-mono sp-small">{{status_tag .Tag.Status}}</td></tr>{{end}}
<tr><td>a tag that opens a <code>sheet</code></td><td>{{template "sheet_tag" .Sheet}}</td><td class="sp-mono sp-small">button.tag + sp-sheet (popover)</td></tr>
</tbody>
</table></div>
{{template "box_close"}}
+4 -2
View File
@@ -104,7 +104,9 @@
<a class="navbar-link" href="/inbound/domains">Inbound</a>
<div class="navbar-dropdown">
<a class="navbar-item{{if and (eq .Section "inbound") (eq .Page "domains")}} is-active{{end}}" href="/inbound/domains"><i class="ti ti-world-download"></i>Domains</a>
</div>
<a class="navbar-item{{if and (eq .Section "inbound") (eq .Page "log")}} is-active{{end}}" href="/inbound/log"><i class="ti ti-list-details"></i>Log</a>
{{if .ShowFilterLists}} <a class="navbar-item{{if and (eq .Section "inbound") (eq .Page "filter-lists")}} is-active{{end}}" href="/inbound/filter-lists"><i class="ti ti-filter"></i>Filter lists</a>
{{end}} </div>
</div>
{{end}}{{if .ShowServer}} <div class="navbar-item has-dropdown is-hoverable{{if eq .Section "server"}} sp-current{{end}}">
<a class="navbar-link" href="/server/health">Server</a>
@@ -143,6 +145,6 @@
pages of the current group, the current one marked. A page outside every
group (the user menu, the kit page's neighbours) has none. */}}
{{define "subnav"}}{{if and .ShowOutbound (eq .Section "outbound")}}<div class="sp-subnav"><div class="container"><a href="/outbound/domains"{{if eq .Page "domains"}} aria-current="page"{{end}}><i class="ti ti-world-upload"></i>Domains</a><a href="/outbound/log"{{if eq .Page "log"}} aria-current="page"{{end}}><i class="ti ti-list-details"></i>Log</a>{{if .ShowQueue}}<a href="/outbound/queue"{{if eq .Page "queue"}} aria-current="page"{{end}}><i class="ti ti-stack-2"></i>Queue</a>{{end}}{{if .ShowDMARC}}<a href="/outbound/dmarc"{{if eq .Page "dmarc"}} aria-current="page"{{end}}><i class="ti ti-report-analytics"></i>DMARC reports</a>{{end}}</div></div>
{{else if and .ShowInbound (eq .Section "inbound")}}<div class="sp-subnav"><div class="container"><a href="/inbound/domains"{{if eq .Page "domains"}} aria-current="page"{{end}}><i class="ti ti-world-download"></i>Domains</a></div></div>
{{else if and .ShowInbound (eq .Section "inbound")}}<div class="sp-subnav"><div class="container"><a href="/inbound/domains"{{if eq .Page "domains"}} aria-current="page"{{end}}><i class="ti ti-world-download"></i>Domains</a><a href="/inbound/log"{{if eq .Page "log"}} aria-current="page"{{end}}><i class="ti ti-list-details"></i>Log</a>{{if .ShowFilterLists}}<a href="/inbound/filter-lists"{{if eq .Page "filter-lists"}} aria-current="page"{{end}}><i class="ti ti-filter"></i>Filter lists</a>{{end}}</div></div>
{{else if and .ShowServer (eq .Section "server")}}<div class="sp-subnav"><div class="container"><a href="/server/health"{{if eq .Page "health"}} aria-current="page"{{end}}><i class="ti ti-heartbeat"></i>Health</a><a href="/server/log"{{if eq .Page "log"}} aria-current="page"{{end}}><i class="ti ti-file-text"></i>System log</a><a href="/server/backup"{{if eq .Page "backup"}} aria-current="page"{{end}}><i class="ti ti-archive"></i>Backup</a><a href="/server/users"{{if eq .Page "users"}} aria-current="page"{{end}}><i class="ti ti-users"></i>Users</a><a href="/server/settings"{{if eq .Page "settings"}} aria-current="page"{{end}}><i class="ti ti-settings"></i>Settings</a></div></div>
{{end}}{{end}}
@@ -13,6 +13,10 @@
<div class="field"><label class="label" for="send_log_retention_days">Keep outbound log rows, days</label>
<div class="control"><input class="input" id="send_log_retention_days" name="send_log_retention_days" type="number" min="7" max="365" step="1" value="{{.RetentionDays}}" required></div>
<p class="help">The main driver of <code>/data</code> growth. Does not affect <code>mail.log</code> rotation (fourteen daily files).</p></div>
{{if .Inbound}}<div class="field"><label class="label" for="inbound_spam_log_retention_days">Keep inbound log rows, days</label>
<div class="control"><input class="input" id="inbound_spam_log_retention_days" name="inbound_spam_log_retention_days" type="number" min="7" max="365" step="1" value="{{.InboundRetentionDays}}" required></div>
<p class="help">The inbound log also never holds more than 10 000 rows: refused mail is logged at the rate strangers knock on port 25.</p></div>
{{end}}
{{template "body_close"}}
{{template "box_close"}}
</div>
+2
View File
@@ -54,6 +54,8 @@ var pageFiles = map[string][]string{
"in-domains": {"templates/in_domains.html"},
"in-domain": {"templates/in_domain.html"},
"in-domain-delete": {"templates/in_domain_delete.html"},
"in-log": {"templates/in_log.html"},
"in-filter-lists": {"templates/in_filter_lists.html"},
"out-log": {"templates/out_log.html", "templates/out_log_rows.html"},
"out-message": {"templates/out_message.html"},
"dmarc": {"templates/dmarc.html"},
+14 -2
View File
@@ -97,6 +97,9 @@ type Config struct {
DMARCEnabled bool
// SendLogRetentionEnvDefault is SEND_LOG_RETENTION_DAYS at panel start.
SendLogRetentionEnvDefault int
// InboundSpamLogRetentionEnvDefault is INBOUND_SPAM_LOG_RETENTION_DAYS at
// panel start.
InboundSpamLogRetentionEnvDefault int
}
// Server is the panel HTTP application.
@@ -111,7 +114,7 @@ type Server struct {
// that owns DKIM keys and the OpenDKIM tables (architecture.md § OpenDKIM);
// apps owns application SASL accounts and the Postfix sender map
// (architecture.md § Mail path).
func New(st *store.Store, domains *domain.Service, apps *app.Service, inboundSvc *inbound.Service, dmarcSvc *dmarc.Service, cfg Config, setupTokenPath string) (*Server, error) {
func New(st *store.Store, domains *domain.Service, apps *app.Service, inboundSvc *inbound.Service, lists *inbound.Lists, dmarcSvc *dmarc.Service, cfg Config, setupTokenPath string) (*Server, error) {
v, err := view.New(cfg.Version)
if err != nil {
return nil, err
@@ -124,7 +127,7 @@ func New(st *store.Store, domains *domain.Service, apps *app.Service, inboundSvc
SessionIdleDays: cfg.SessionIdleDays,
TrustedProxyCIDRs: cfg.TrustedProxyCIDRs,
}, v, setupTokenPath)
h := handlers.New(st, domains, apps, inboundSvc, dmarcSvc, handlers.Config{
h := handlers.New(st, domains, apps, inboundSvc, lists, dmarcSvc, handlers.Config{
Hostname: cfg.Hostname,
SubmissionEnabled: cfg.SubmissionEnabled,
MailLogPath: cfg.MailLogPath,
@@ -143,6 +146,8 @@ func New(st *store.Store, domains *domain.Service, apps *app.Service, inboundSvc
InboundAntispamAction: cfg.InboundAntispamAction,
DMARCEnabled: cfg.DMARCEnabled,
SendLogRetentionEnvDefault: cfg.SendLogRetentionEnvDefault,
InboundSpamLogRetentionEnvDefault: cfg.InboundSpamLogRetentionEnvDefault,
}, v, dnscheck.New(cfg.DNSResolvers), &health.MachineSampler{}, a)
return &Server{cfg: cfg, auth: a, handlers: h}, nil
}
@@ -240,6 +245,13 @@ func (s *Server) muxes() (public, authed *http.ServeMux) {
authed.HandleFunc("POST /inbound/domains/{id}/recipients", h.HandleInboundRecipients)
authed.HandleFunc("GET /inbound/domains/{id}/delete", h.HandleInboundDeleteConfirm)
authed.HandleFunc("POST /inbound/domains/{id}/delete", h.HandleInboundDelete)
// The log is for everyone who reaches an inbound domain (scoped to
// theirs); the lists are the instance's, so the handlers keep the
// global role's check (requireInboundGlobal) and answer 404 to the rest.
authed.HandleFunc("GET /inbound/log", h.HandleInboundLog)
authed.HandleFunc("GET /inbound/filter-lists", h.HandleFilterLists)
authed.HandleFunc("POST /inbound/filter-lists", h.HandleFilterListAdd)
authed.HandleFunc("POST /inbound/filter-lists/{id}/delete", h.HandleFilterListDelete)
}
// Server: everything under /server/ is for the global role, so the subtree
+9 -5
View File
@@ -47,9 +47,13 @@ var port25Cases = []port25Case{
{param: "smtpd_sasl_auth_enable", contains: "no", wantExact: true},
{param: "smtpd_relay_restrictions", contains: "reject_unauth_destination", wantExact: true},
{param: "smtpd_recipient_restrictions", contains: "reject_unauth_destination,reject_unlisted_recipient", wantExact: true},
// No antispam milter configured: the submission chain (strict
// OpenDKIM + journal-milter) must not reach inbound mail.
{param: "smtpd_milters", contains: "", wantExact: true},
// The log tailer tells this listener's refusals from the
// submission ports' by the name they are logged under.
{param: "syslog_name", contains: "postfix/inbound", wantExact: true},
// No antispam milter configured: the inbound journal milter
// alone. The submission chain (strict OpenDKIM + journal-milter)
// must not reach inbound mail.
{param: "smtpd_milters", contains: "{unix:/run/selfpost/inbound-journal.sock,default_action=accept}", wantExact: true},
},
},
{
@@ -72,7 +76,7 @@ var port25Cases = []port25Case{
},
check: []optionCheck{
{param: "smtpd_recipient_restrictions", contains: "check_recipient_access,texthash:/data/postfix/dmarc_recipients,reject_unauth_destination,reject_unlisted_recipient", wantExact: true},
{param: "smtpd_milters", contains: "", wantExact: true},
{param: "smtpd_milters", contains: "{unix:/run/selfpost/inbound-journal.sock,default_action=accept}", wantExact: true},
},
// Both on: the report addresses sit on a relay domain, and the relay's
// unlisted-recipient check runs after the restriction list. Unless their
@@ -94,7 +98,7 @@ var port25Cases = []port25Case{
check: []optionCheck{
// Exact equality also covers the absence of OpenDKIM/journal-milter
// sockets — anything else there would fail the comparison.
{param: "smtpd_milters", contains: "{inet:antispam:11332,default_action=accept}", wantExact: true},
{param: "smtpd_milters", contains: "{inet:antispam:11332,default_action=accept},{unix:/run/selfpost/inbound-journal.sock,default_action=accept}", wantExact: true},
},
},
}
+147
View File
@@ -265,10 +265,157 @@ func testInboundRelay(t *testing.T, sc *scenario) {
t.Fatalf("mail for the inbound domain never reached its upstream: %v\n%s", err, mailLogTail(h, "postfix/"))
}
// The filter lists exist as files from the first start with the relay on,
// readable by a filter that is another user in another container. Nothing
// is on them until someone adds an entry.
maps, err := h.execIn("selfpost", "sh", "-c", `cd /data/antispam && stat -c "%a %n" . *.map && { grep -vc "^#" *.map; true; }`)
if err != nil {
t.Fatalf("list /data/antispam: %v\n%s", err, maps)
}
for _, want := range []string{"755 .",
"644 allow_domain.map", "644 allow_from.map", "644 allow_ip.map",
"644 deny_domain.map", "644 deny_from.map", "644 deny_ip.map",
"allow_ip.map:0", "deny_from.map:0",
} {
if !strings.Contains(maps, want) {
t.Fatalf("/data/antispam: no %q in\n%s", want, maps)
}
}
// The inbound journal milter saw it go through, and the Inbound log shows it.
// No filter is attached on the stand: the row is an accept by nobody, so its
// reason is the engine alone.
logRows := func(want string) []string {
t.Helper()
var rows []string
var body string
for deadline := time.Now().Add(15 * time.Second); ; time.Sleep(time.Second) {
resp, b, err := sc.panel.get("/inbound/log")
if err != nil || resp.StatusCode != 200 {
t.Fatalf("GET /inbound/log: status %v, %v\n%.800s", resp, err, b)
}
body = b
rows = rows[:0]
for _, row := range strings.Split(body, "<tr><td")[1:] {
// The row's own end is the last one: the sheet in its
// decision cell holds a table of its own.
rows = append(rows, row[:strings.LastIndex(row, "</tr>")])
}
if strings.Contains(body, want) {
return rows
}
if time.Now().After(deadline) {
t.Fatalf("no row with %q in the Inbound log:\n%.3000s\n%s", want, body, mailLogTail(h, "postfix/"))
}
}
}
// rowWith finds the one row that holds all of wants.
rowWith := func(rows []string, wants ...string) string {
t.Helper()
for _, row := range rows {
ok := true
for _, w := range wants {
ok = ok && strings.Contains(row, w)
}
if ok {
return row
}
}
t.Fatalf("no row of the Inbound log holds all of %q in\n%s", wants, strings.Join(rows, "\n"))
return ""
}
rows := logRows("e2e " + token)
rowWith(rows,
"someone@<wbr>stranger.e2e.test", "user@<wbr>"+inboundDomain, "e2e "+token,
`class="tag is-success is-light" popovertarget="why-`, `<th>Decided by</th><td>selfpost</td>`,
// the way to correct it, which only the global role is given
`href="/inbound/filter-lists?list=deny&amp;kind=address&amp;value=someone%40stranger.e2e.test">Deny sender</a>`)
// A report mailed to the server's own name earlier in the run came through
// the same listener and is not the relay's mail.
if len(rows) != 1 {
t.Fatalf("the Inbound log holds %d rows, want the relayed message alone:\n%s", len(rows), strings.Join(rows, "\n"))
}
for _, to := range []string{"nobody@" + inboundDomain, "someone@elsewhere.e2e.test"} {
res := deliverPort25("someone@stranger.e2e.test", to, message(to, uniqueToken("refused")))
if res.rcptErr == nil {
t.Fatalf("port 25 accepted the recipient %s (first error: %v)", to, res.firstErr())
}
}
// The refusal of an unknown recipient at the inbound domain is read from
// the mail log into the journal; the refusal to relay for a stranger's
// domain is not the relay's mail and is not.
rows = logRows("nobody@<wbr>" + inboundDomain)
refused := rowWith(rows,
"someone@<wbr>stranger.e2e.test", "nobody@<wbr>"+inboundDomain, `<td class="sp-clip sp-muted">—</td>`,
`class="tag is-danger is-light" popovertarget="why-`, `<th>Decided by</th><td>postfix</td>`,
`<th>Reply</th><td class="sp-mono sp-small">550 5.1.1 `)
// A message Postfix itself refused offers no way into the lists.
if strings.Contains(refused, "sender</a>") {
t.Fatalf("the refused row offers a way into the filter lists:\n%s", refused)
}
if len(rows) != 2 {
t.Fatalf("the Inbound log holds %d rows, want the relayed message and one refusal:\n%s", len(rows), strings.Join(rows, "\n"))
}
// The filter lists: an entry added in the panel is in the map file the
// filter reads, and one removed is gone from it. Nothing reads the file on
// this stand (no filter is attached), so the file is the whole effect.
const spammer = "spammer.e2e.test"
denyDomain := func() string {
t.Helper()
out, err := h.execIn("selfpost", "cat", "/data/antispam/deny_domain.map")
if err != nil {
t.Fatalf("read deny_domain.map: %v\n%s", err, out)
}
return out
}
resp, body, err = sc.panel.postForm("/inbound/filter-lists", url.Values{
"list_type": {"deny"}, "entry_type": {"domain"}, "value": {" @" + strings.ToUpper(spammer)}, "note": {"e2e"}})
if err != nil {
t.Fatal(err)
}
if resp.StatusCode != 200 || resp.Request.URL.Path != "/inbound/filter-lists" || !strings.Contains(body, "Entry added.") {
t.Fatalf("add a filter list entry: status %d, landed on %s:\n%.800s", resp.StatusCode, resp.Request.URL.Path, body)
}
if got := denyDomain(); !strings.Contains(got, "\n"+spammer+"\n") {
t.Fatalf("deny_domain.map after adding %s:\n%s", spammer, got)
}
if got, err := h.execIn("selfpost", "sh", "-c", `cd /data/antispam && { grep -vc "^#" *.map; true; }`); err != nil ||
strings.Count(got, ":0") != 5 || !strings.Contains(got, "deny_domain.map:1") {
t.Fatalf("the entry is in other files than deny_domain.map (%v):\n%s", err, got)
}
// The same entry again is refused on the page, and the file still holds it once.
resp, body, err = sc.panel.postForm("/inbound/filter-lists", url.Values{
"list_type": {"deny"}, "entry_type": {"domain"}, "value": {spammer}})
if err != nil {
t.Fatal(err)
}
if resp.StatusCode != 422 || !strings.Contains(body, "That entry is already on the list.") {
t.Fatalf("add the same entry again: status %d:\n%.800s", resp.StatusCode, body)
}
if got := denyDomain(); strings.Count(got, spammer) != 1 {
t.Fatalf("deny_domain.map after a duplicate:\n%s", got)
}
resp, body, err = sc.panel.get("/inbound/filter-lists")
if err != nil || resp.StatusCode != 200 {
t.Fatalf("GET /inbound/filter-lists: status %v, %v", resp, err)
}
m = regexp.MustCompile(`action="(/inbound/filter-lists/\d+/delete)"`).FindStringSubmatch(body)
if m == nil || !strings.Contains(body, spammer) {
t.Fatalf("the list does not show %s with a Remove:\n%.2000s", spammer, body)
}
resp, body, err = sc.panel.postForm(m[1], nil)
if err != nil {
t.Fatal(err)
}
if resp.StatusCode != 200 || resp.Request.URL.Path != "/inbound/filter-lists" || !strings.Contains(body, "Entry removed.") {
t.Fatalf("remove the entry: status %d, landed on %s:\n%.800s", resp.StatusCode, resp.Request.URL.Path, body)
}
if got := denyDomain(); strings.Contains(got, spammer) {
t.Fatalf("deny_domain.map after removing %s:\n%s", spammer, got)
}
}