31 Commits
Author SHA1 Message Date
mixandClaude Opus 5.5 d2a778edc0 inbound-antispam-panel: Filter lists accepted; steps 8-10 closed
test / test (push) Canceled after 0s
The owner accepted Inbound > Filter lists on 2026-10-12 ("Да"), after
a deny entry he added refused my test message on the live test server
with the list's own reply (554 5.7.1 Refused by this server's deny
list), mail.log carried the milter-reject line the count reads and the
journal its row.

Steps 8-10 of the plan were done with each earlier step and are ticked
with where. 10/12: the technical and the security review remain.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 07:22:09 +03:00
mixandClaude Opus 5.5 7192c3861e inbound-antispam-panel: Filter lists, kind column, matches, note on a sheet
Follows the design commit before it (owner's words quoted there).

The lists' tables are Value, Kind, Matched and Remove. The kind tag
opens the entry's sheet: note, day added, how often it decided a
message in the days the log keeps, and the last match.

Matches are read off the journal on each view, nothing is stored
(inbound.Lists.Usage, store.ListSpamListHits). A row counts for an
entry when the list decided the message - a rejected row whose reply is
the deny rule's message, an accepted row whose verdict carries the
allow rule's symbol for that kind - and the entry fits the row's sender
or client address the way rspamd matches it: an address whole, a domain
as the sender's domain, a network containing the client (an
IPv4-mapped address as IPv4). A failure to count shows the lists with
"never" and logs why.

For that the journal milter keeps the lists' own symbols ahead of its
six-symbol cap: rspamd writes symbols heaviest first, so an allow
entry's -100 came last and was cut behind any six others.

The plan gains "Filter list usage": what counting from the journal
implies, and - the owner asked for it in writing - the conditions under
which a counter kept with each entry becomes worth a migration (the
journal cut by its row cap and not by age; a short retention; counts
that must outlive the journal; a slow page). multimap.conf says which
of its texts SelfPost reads back.

Tests: Usage over every kind on both lists and the rows that are
nobody's; the milter's symbol order; the store query; the page and its
sheets; a handler test from journal rows to the numbers on the page.

Seen: the fixture page with a sheet open, headless Edge at 1360 px -
the same as the accepted drawing.

Checked on the test server, uncommitted tree: go vet ./... ok,
go test ./... ok (22 packages), make e2e ok.

Not seen in a real run: a count above zero. It needs an entry added in
the panel, which I cannot sign in to, and then mail that the list
decides; in particular the deny rule's message arriving in mail.log as
the reply of a milter-reject line is what the trial with rspamc showed,
not yet what Postfix logged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 01:08:26 +03:00
mixandClaude Opus 5.5 e0ccff257d design: in-filter-lists, kind column, matches, the note on a sheet
Owner, 2026-10-12, on Inbound > Filter lists as built: "Note хочется
вынести в лист, подсказку, раскрывающийся блок. Тип в отдельную колонку.
Ещё полезно будет дать статистику использования правила, чтобы удалять
устаревшие." Drawn, then accepted ("Да") together with counting the
matches from the Inbound log.

  - Kind is a column of its own; the tag no longer wraps under the
    value.
  - Note and Added leave the table for the entry's sheet, opened by the
    kind tag (of his three - sheet, tooltip, disclosure - the sheet: it
    exists and works under a finger).
  - Matched: how often the entry decided a message, or "never". The
    sheet adds the window ("in the last 90 days") and the last match.
    A fifth column for the last match did not fit half the page width.

Help, Filter lists: what Matched counts and what "never" suggests.

check.py: 32 screens, 0 violations; outlines.json unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 00:54:04 +03:00
mixandClaude Opus 5.5 c1da784193 inbound-antispam-panel: the backup holds the lists; Inbound log accepted
Step 7/12 of docs/plans/inbound-antispam-panel.md. A full backup walks
all of /data, so data/antispam/ was in it from the day the directory
existed; TestCreateIncludesStateExcludesTransient now holds that. On
the first start after a restore the maps are written again from the
database (mailStack.Resync, already there). The guide says what is not
in a backup: the antispam/ directory with the compose fragment and the
rspamd configuration, which is installation and not state.

The plan records the owner's acceptance of Inbound > Log (2026-10-12,
"Этот экран утверждаем") and what he changed on the way to it.

go test ./internal/backup/ ok.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 00:36:59 +03:00
mixandClaude Opus 5.5 9d330bed77 inbound-antispam-panel: Inbound log, Why as a sheet; no Deny on a Postfix reject
Follows the two design commits before it (owner's decisions quoted
there).

Kit: a new component, Sheet / SheetRow (components.go), partials
sheet_tag and sheet (components.html), its rules in panel.css as in
theme.css, and its sample on the kit page next to status_tag. It is
the browser's popover: no script.

Inbound log: the Why column is gone; the decision tag opens a sheet
with who decided, then either the filter's verdict and its symbols one
a line (a detail the journal milter wrote) or the reply (a detail the
tailer read from mail.log - it starts with its code), and the subject
whole. The sheet's id is the journal row's. A row Postfix itself
refused no longer links to the filter lists.

Tests: the page fixture and view tests for the sheet's markup and
content (TestWhyRows), the link rule, three Deny and two Allow on the
mockup's six rows; e2e reads "Decided by" and "Reply" from the sheet
and wants no sender link on the refused row.

Seen: the fixture page with a sheet open, headless Edge at 1360 px -
the same as the accepted drawing.

Checked on the test server, uncommitted tree: go vet ./... ok,
go test ./... ok (22 packages), make e2e ok.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 00:23:51 +03:00
mixandClaude Opus 5.5 970c8ca8da design: the sheet on the components page is a table; no empty Subject
The sample sheet on the components page still held facts; it shows the
table of name and value the accepted sheet is. In in-log a refused
message's sheet has no Subject row - there is no subject to show whole.

check.py: 32 screens, 0 violations.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 00:13:14 +03:00
mixandClaude Opus 5.5 19156814b8 design: in-log, Why is a sheet over the page; no Deny on a Postfix reject
Owner, 2026-10-11/12, on the Inbound log as built:
  - "Why сделать не столбцом, а листом поверх." Opened by the decision
    tag ("Тег."), drawn first, then: its content as a table ("В лист
    может быть таблицей?" - "Да. Но хорошо бы ещё её отцентрировать."),
    with a narrow column of names and only what the row does not show.
  - "Почему Deny у того, кто rejected?" - "Убирай": a message Postfix
    itself refused offers no Deny sender. Accepted mail offers Deny,
    mail the filter refused offers Allow.

A new component, sheet (theme.css, shown on the components page next to
status_tag): the browser's own popover, so no script - the tag is a
button with popovertarget; Esc, a click outside and the cross close it.
It rises from the bottom edge across the window; its content is a table
of name and value in a 44rem column in the middle. A tag that opens a
sheet carries a dotted underline to tell it from a plain tag.

in-log: the Why column is gone and rows are one height. A sheet holds
who decided, the verdict and the symbols (or the reply), and the
subject whole.

Guard: check.py WIDTH_OK gains ".sp-sheet" - the one component laid out
by the window and not by the page. The owner accepted the drawing with
that said.

check.py: 32 screens, 52 sp- classes, 0 violations; outlines.json
unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 00:12:04 +03:00
mixandClaude Opus 5.5 d2b97b3963 inbound-antispam-panel: Inbound log, the sender column is headed From
Follows the design commit before it.

go test ./internal/web/view/ ok.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 23:59:47 +03:00
mixandClaude Opus 5.5 d4efdb483d design: in-log, the sender column is headed From
Owner, 2026-10-11, of "From · client": "Client в названии столбца
зачем? Мне кажется, оно лишним." The client address stays under the
sender, in mono and muted; an address of that shape needs no label.

check.py: 32 screens, 0 violations; outlines.json unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 23:59:18 +03:00
mixandClaude Opus 5.5 56c5dfee40 inbound-antispam-panel: Inbound > Log and Inbound > Filter lists
Step 6/12 of docs/plans/inbound-antispam-panel.md. Written by a Sonnet
subagent from the mockups, reviewed and corrected by me.

GET /inbound/log - the journal a page at a time (50), filtered by
domain, decision and day; QueryInboundSpam / CountInboundSpam with the
viewer's inbound domains as the scope. The global role reads all of it
and gets "Deny sender" / "Allow sender" links into the lists; a user
with the domain role reads the rows of the inbound domains assigned to
them, without the links. The plan said global-only for the journal;
the accepted mockup and 2.0's delegation of Inbound say otherwise -
recorded in the plan as built.

GET, POST /inbound/filter-lists and POST .../{id}/delete - global role,
404 otherwise. Add form (prefilled from the query), the two lists with
Remove as the kit's row action, and "Where the lists go" with the
filter's state and when the maps were last written
(Lists.LastWritten). A refused or duplicate entry is 422 with the
message and what was typed.

Server > Settings: "Keep inbound log rows, days" with the relay on.
Help: Inbound log, Filter lists. Navigation: Log for everyone who sees
Inbound, Filter lists for the global role (Shell.ShowFilterLists).
Status tags accept / reject / tempfail join the kit's mapping.

Tests: page fixtures for both screens, held to outlines.json; view and
handler tests (both roles, filters, paging, relay off, add / duplicate
/ invalid / delete, map file content, settings field, help). e2e reads
the journal from the page instead of the database and adds and removes
a deny entry through the panel, reading deny_domain.map after each.

Seen: both pages and Settings rendered from their fixtures and compared
with the mockups side by side (headless Edge, 1360 px) - same layout
and wording.

Checked on the test server, uncommitted tree: go vet ./... ok,
go test ./... ok (22 packages), make e2e ok.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 23:50:34 +03:00
mixandClaude Opus 5.5 82d4a209be design: in-log and in-filter-lists leave the deferred screens
Guards and mockups only, ahead of the pages (inbound-antispam-panel
6/12). The owner agreed the feature ("Следующая по дорожной карте -
inbound-antispam-panel" / "Да", 2026-10-11) and started this step
("Начинай"), so the two screens are no longer "drawn but not built":

  - guard_panel_test.go: in-log and in-filter-lists removed from
    deferredScreens. From here TestLegacyRatchetMatchesTheEngine wants
    the engine to have both pages - red until the next commit.
  - guard_routes_test.go: the four routes join routes.inbound, so they
    must exist with the relay on and answer 404 with it off.

Two mockup details brought to what the kit can draw, found when the
pages were built from it:
  - help: the two new topics are one paragraph with a break in it, as
    every topic is (panel.css has no gap between paragraphs of a topic);
  - in-filter-lists: the directory in "Files" is mono at the text size -
    the kit's inline code has no small variant.

check.py: 32 screens, 0 violations; outlines.json unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 23:50:33 +03:00
mixandClaude Opus 5.5 737611a07e design: Inbound log and Filter lists as built; inbound retention; Help
Mockup-first step of inbound-antispam-panel 6/12. The two screens were
drawn for the redesign before the feature existed; this brings them to
what steps 3-5 built.

in-log: the lead says where rows come from and how long they are kept;
a help link in the box head; Why is the engine and the detail as the
journal holds them (rspamd: action, score / threshold, weighted
symbols; postfix and a refusing filter: the reply text). No row
invents what the journal cannot hold: the "selfpost - upstream timed
out" tempfail and the Review link to a quarantine page that does not
exist are gone, a refused message has no subject, an allow-listed one
shows SELFPOST_ALLOW_FROM(-100.00).

in-filter-lists: Remove is a POST - the kit's row action, not a link;
a domain entry is shown as stored, without "@"; "Where the lists go"
names the directory and its six files instead of allow.map / deny.map,
says rspamd rereads them by itself and that Deny wins.

settings: "Keep inbound log rows, days" in the Log retention box.

help: two topics under Inbound, Inbound log and Filter lists, and a
sentence in Settings.

outlines.json rewritten: Help's Inbound box has six topics, was four.
check.py: 32 screens, 0 violations.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 23:18:57 +03:00
mixandClaude Opus 5.5 eea40bcd49 inbound-antispam-panel: filter lists, written as rspamd maps
Step 5/12 of docs/plans/inbound-antispam-panel.md.

Store: AddSpamListEntry, ListSpamListEntries, DeleteSpamListEntry.
internal/inbound/lists.go validates and normalises an entry - sender
address, sender domain, client address or network (canonical text, an
IPv4-mapped address as IPv4, /0 refused) - and writes the lists as six
map files under /data/antispam (ANTISPAM_DIR), each atomically, 0644 in
a 0755 directory: the filter is another user in another container. All
six are written on every change and on every start with the relay on,
empty ones included. An entry whose maps could not be written is taken
back out of the table. The writer refuses a stored value with
whitespace, a newline or # in it.

The plan's allow.map / deny.map became one file per list and entry
type: to rspamd an IP map and a sender map are different map types.

deploy/antispam/rspamd/multimap.conf holds the rules; the fragment
mounts it and ./data/antispam into the sidecar. Deny is a prefilter
reject, allow a symbol weighing -100. Tried in a throwaway rspamd 3.11
container on the test server:
  - with allow as a prefilter accept, an address on both lists was
    accepted and an IP on both was refused - the rule that ran first
    decided; with the weight, deny won in all five mixed cases;
  - IPv4, IPv6, address (any case), domain (not its subdomains) match;
  - a rewritten file is reread in about ten seconds, no restart;
  - files that appear after rspamd started are loaded the same way.

No page edits the lists yet (step 6).

Checked on the test server, uncommitted tree: go vet ./... ok,
go test ./... ok (22 packages), make e2e ok - with the relay on
/data/antispam is 755 and holds the six maps, 644, empty.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 23:07:10 +03:00
mixandClaude Opus 5.5 27fd355e53 inbound-antispam-panel: refused inbound mail from mail.log; retention
Step 4/12 of docs/plans/inbound-antispam-panel.md.

The log tailer writes an inbound_spam_log row for mail port 25 refused
(internal/logtail/inbound.go): a "reject:" line of the port-25 smtpd is
Postfix's own policy; a milter-reject / milter-hold / milter-discard
line of cleanup is the filter's verdict. Only a recipient at an inbound
domain is a row, so "Relay access denied" for a stranger's domain is
not. The row is dated by its log line; the reply text is cut to 255
bytes of printable ASCII; a reply starting with 4 is a tempfail.

To tell port 25 from 465 / 587 the listener now logs as
postfix/inbound (syslog_name). cleanup logs under one name for every
listener, so its verdict counts only for a queue id the port-25 smtpd
was seen to open (4096 remembered). Lines are matched from the head,
after timestamp and host: their tail is the sender's own text.

Retention: PruneInboundSpam every six hours - older than the
inbound_spam_log_retention_days setting (seeded from
INBOUND_SPAM_LOG_RETENTION_DAYS, 90, range 7-365), then down to 10 000
rows. The setting has no field in the panel yet.

Known limits, in architecture.md: a log line names one recipient, so a
message refused at end-of-message is one row; a line re-read after an
unclean stop would be a second row.

Checked on the test server, uncommitted tree: go vet ./... ok,
go test ./... ok (22 packages), make e2e ok - the unknown recipient at
the inbound domain refused on port 25 has its row (reject by postfix),
the stranger's domain has none.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 22:42:25 +03:00
mixandClaude Opus 5.5 94e94490af roadmap: phone in "desktop site" mode gets the phone navigation
A candidate, recorded at the owner's word (2026-10-11, screenshot of
Server > Health on selfpost.mixfed.ru): the body is laid out in columns
while the navigation is the phone stack with every section open. The
item states what was seen and a first guess at the cause (980 px falls
between Bulma's 769 and 1024 px breakpoints); nothing is investigated
yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 22:42:08 +03:00
mixandClaude Opus 5.5 28ec784ae2 inbound-antispam-panel: inbound journal milter on port 25
Step 3/12 of docs/plans/inbound-antispam-panel.md.

The panel serves a second milter while the inbound relay is on
(internal/milter/inbound.go, socket INBOUND_JOURNAL_MILTER_SOCKET,
default /run/selfpost/inbound-journal.sock). build/postfix-config.sh
puts it last on port 25, after the optional antispam milter, with
default_action=accept; postfix-wrapper.sh waits for its socket. At
end-of-message it writes one inbound_spam_log row per recipient at an
inbound domain - other port-25 recipients (DMARC report addresses) get
none - and always accepts. With a filter attached it reads the verdict
from X-Rspamd-Action and X-Spamd-Result (action, score, up to six
weighted symbols, only what the patterns recognise, 255 bytes); without
one those headers are the sender's and are not read. The plan named
X-Rspamd-Score, which rspamd does not have.

The antispam compose fragment mounts rspamd/milter_headers.conf into
the sidecar so that it adds the two headers (checked in rspamd 3.11:
both routines exist and remove a sender's own headers of those names).

Store: InsertInboundSpam, IsInboundDomain.

Checked on the test server, uncommitted tree: go vet ./... ok,
go test ./... ok (22 packages), make e2e ok - the relayed message of
inbound_relay_delivers_and_refuses has its journal row (accept by
selfpost) and the DMARC report sent through the same listener has none.

Not yet seen in a real run: a verdict written by rspamd reaching this
milter. The e2e stand has no filter, and the test server's only inbound
domain has no upstream, so it accepts no mail.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 22:18:14 +03:00
mixandClaude Opus 5.5 a3cc52d6d4 inbound-antispam-panel: migration 0002, journal and filter-list tables
Step 2/12 of docs/plans/inbound-antispam-panel.md, the first migration
after the 2.0 baseline (2.0.1 was cut before it, as agreed).

0002_inbound_spam_log.sql adds inbound_spam_log (one row per recipient:
domain, client IP, envelope, subject, decision, engine, detail) with
indexes on created_at and (inbound_domain, created_at), and
inbound_spam_list (allow / deny entries, unique per list). DDL only;
decision, engine, list_type and entry_type carry CHECKs, as the
baseline does for its own enumerations. Nothing reads or writes the
tables yet.

Tests: TestBaselineSchema pins both tables and user_version 2;
TestOpenMigratesA20Database opens a file holding the baseline alone and
finds its user kept; TestInboundSpamConstraints.

go test ./internal/store/ ok; go vet ./... ok.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 22:05:20 +03:00
mixandClaude Opus 5.5 d65d632505 release: 2.0.1: DMARC ingest drops and bounds; Full backup from the panel
test / test (push) Canceled after 0s
A patch on 2.0.0, all of it found on the test server after the cut:
  - a message that is not a report this relay keeps is dropped at once
    instead of being retried for days;
  - a report is bounded before it is stored, and reports are kept per
    domain (security review of the ingest path, nothing Critical or High);
  - Full backup from the panel works (it had answered 200 with ten bytes
    since 1.3.0);
  - no backup holds the Postfix queue, and the panel says why.

Cut: CHANGELOG [Unreleased] -> [2.0.1]; image pin 2.0.1 in compose,
README, guide; version table.

On the test server, on dd0fefd: go vet ./... ok, go test ./... ok
(22 packages), make e2e ok.

Full detail: CHANGELOG.md (2.0.1).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 21:14:00 +03:00
mixandClaude Opus 5.5 dd0fefdcb7 backup: the panel's archive is the command's, private key included
Owner, 2026-10-11: the private key is needed, and a backup from the
panel and one from the console have to be the same.

The commit that made the panel's backup work skipped a file under
certs/ the panel could not open - on the test server the private key,
which the certificate hook leaves to root and Postfix's group. That gave
an archive with a certificate and no key, and two kinds of archive.

- Params.Unprivileged is gone. Nothing is left out on account of who
  makes the archive; the queue is out for everyone (previous commit).
- When the panel cannot read a file under certs/, backup.Check says so
  before the download and the panel refuses with what to do on the host:
  chgrp 998 certs/privkey.pem && chmod 640. backup.UnreadableCert tells
  that case from any other failure.
- 998 is the container's selfpost group, which holds the panel and
  Postfix. Its id was whatever groupadd gave; the Dockerfile now fixes
  it at the value it has always had, because an operator addresses it
  by number from the host.
- Guide: method 1 is the same archive as method 2, with the permission
  the key needs and a note for whatever renews the certificate. Help and
  the box follow the design commit before this one.
- e2e: the restore scenario first makes the key unreadable and expects
  the refusal naming the file and the way out, then takes the backup and
  expects the key in it; nothing is put back by hand before the restart.

  go vet ./internal/backup/ ./internal/web/handlers/: ok
  go test ./internal/backup/ ./cmd/... ./internal/web/...: ok
  make e2e on the test server: ok

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 20:43:10 +03:00
mixandClaude Opus 5.5 75ebbf6a57 design: the panel's backup needs the private key readable, and says so
Owner, 2026-10-11: the key is needed and the panel's archive must be the console's. The box no longer says a certificate file may be left out; Help says the key has to be readable by the panel (group 998 on the host) and that the panel refuses, naming the file, until it is.

check.py: 0 violations; outlines.json unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 20:43:10 +03:00
mixandClaude Opus 5.5 ede3783734 panel-redesign: Full backup and Help say why the queue is left out
Follows the design commit before it: the paragraph of the Full backup box and the Backup topic of Help.

  go test -count=1 ./internal/web/...: ok
  Backup page at 1280: no overflow, the box holds the longer paragraph.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 20:16:30 +03:00
mixandClaude Opus 5.5 d70371a50c design: Full backup and Help say why the queue is left out
Owner, 2026-10-11: the user has to be told why. The box gives the reason in a sentence - the queue cannot be copied reliably while the server is running; the Help topic explains it (Postfix keeps moving messages, a copy could hold one twice or miss it), what follows (waiting mail is not carried over, a restored server starts with an empty queue) and what to do to keep the queue (stop the container, archive the project directory).

check.py: 0 violations; outlines.json unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 20:16:30 +03:00
mixandClaude Opus 5.5 21e961cb29 backup: no full backup holds the Postfix queue
Owner, 2026-10-11: the queue goes out of the command's archive too.

The commit that made the panel's backup work left the queue in the
archive of selfpost-backup, which runs as root and can read it. But that
copy was never the queue: the archive is made while the instance runs,
Postfix moves messages between the queue's directories all the time, and
a message on its way could be in the archive twice or not at all. One
rule now - QueueDir is left out of every archive - and
Params.Unprivileged is only about a certs/ file the panel may not open.

What is not carried over is mail still waiting for delivery; a restored
instance builds an empty queue (the e2e restore scenario starts one).
To keep the queue: stop the instance and tar the project directory, as
the guide's methods 3 and 4 already describe.

Guide: the description of a full backup, methods 1 and 2 and the
comparison table. Help follows the design commit before this one.

  go vet ./internal/backup/: ok
  go test ./internal/backup/ ./cmd/... ./internal/web/...: ok

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 20:06:41 +03:00
mixandClaude Opus 5.5 82f212ab39 design: Help says no backup holds the Postfix queue
Owner, 2026-10-11: the queue is left out of the command's archive as well as the panel's. The Help topic no longer says selfpost-backup takes it; the command still takes the private key the panel cannot read.

check.py: 0 violations; outlines.json unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 20:06:41 +03:00
mixandClaude Opus 5.5 b55b33ae92 e2e: full backup from the panel, restore into an empty directory
After the rate-limit checks the run takes a full backup through the
panel, reads the archive through while the container can still say what
went wrong, then removes data/, certs/, docker-compose.yml and .env,
unpacks the archive into the project directory, puts the private key
back (the panel's archive comes without it and without the queue, both
asserted), and starts the stand again.

What has to hold: manifest.json is consumed by the first start, the
administrator signs in, the domain shows the same DKIM value, and a
message sent with the application's old password reaches the sink and
verifies against the record published before the backup.

The port-25 scenarios then run on the restored installation.

make e2e on the test server: ok.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 19:55:19 +03:00
mixandClaude Opus 5.5 8c9a71e6a0 backup: the panel's full backup works, without the queue
Found by the e2e restore scenario of the next commit: the Full backup
button answered 200 with ten bytes. The archive includes the Postfix
queue, Postfix keeps every queue file and directory readable by its own
user alone (not configurable), and the panel runs as its own user - the
walk stopped at /data/postfix/queue/active with "permission denied"
after the headers were out. So since the queue moved under /data in
1.3.0. Confirmed on the test server: selfpost-backup as root makes the
whole archive, the same command as user panel fails the same way.

Owner, 2026-10-11: the panel's archive comes without the queue.

- backup.Params.Unprivileged: the queue is left out, and so is a file
  under certs/ that cannot be opened (the private key on the test
  server is root and Postfix's group only), named through OnWarn. The
  selfpost-backup command leaves it unset and takes both, as before.
- backup.Check reads through everything Create would archive, writing
  nothing. The panel runs it before the first byte: a failure is a
  message on the page, not a download cut short.
- The box, Help (design commit before this one) and the guide say what
  the panel's archive leaves out; the guide's comparison table has rows
  for the queue and the private key, and tells to put the key back
  before starting a restored instance.

  go vet ./internal/backup/ ./internal/web/handlers/: ok
  go test ./internal/backup/ ./cmd/... ./internal/web/...: ok
  make e2e on the test server, with the restore scenario: ok

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 19:55:18 +03:00
mixandClaude Opus 5.5 646ec1e149 design: Full backup says what the panel's archive leaves out
Owner, 2026-10-11: the archive made from the panel comes without the Postfix queue (variant 1). The Full backup box and the Help topic say so, and name the other thing the panel may not read - a certificate file, usually the private key - and the command that takes both.

check.py: 0 violations; outlines.json unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 19:55:18 +03:00
mixandClaude Opus 5.5 8496b289ec e2e: the run ends on port 25, with a report, junk and inbound mail
Until now the suite only read back the configuration generated for the
two optional port-25 features. DMARC ingest shipped in 1.7.0 with a
delivery service Postfix could not start, and nothing sent a report
through it until a real one arrived on the test server on 2026-10-11.

The last steps recreate the container with INBOUND_RELAY_ENABLE and
DMARC_REPORTS_ENABLE on, over the same /data, and talk to port 25 as a
stranger's mail server:

- the container log has no postconf warning, the session survives;
- a domain takes the SelfPost-hosted report address; a gzip report
  mailed to it appears under Outbound > DMARC reports; an address nobody
  chose is refused at RCPT;
- a report on another domain and a message with no report are accepted,
  dropped with "dropped: ..." on their delivery lines, counted as two
  parse failures, absent from the queue and from the reports;
- an inbound domain with the sink as upstream and one listed recipient:
  mail for it reaches the sink, another recipient and another domain are
  refused at RCPT.

The stage directory now holds docker-compose.yml and .env, as an
operator's project directory does (a full backup needs them; the
restore scenario that uses it is not in this commit - the panel's full
backup turned out not to work, see the next one).

make e2e on the test server: ok.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 17:40:47 +03:00
mixandClaude Opus 5.5 4dd46ea355 dmarc: a report is bounded, kept per domain, and ingested one at a time
Security review of the DMARC report path (Fable, 2026-10-11), which ran
for the first time in 2.0.0. No Critical or High finding. Fixed:

- Medium: a report was bounded only as 8 MiB of XML - some 160 000
  records or one name of megabytes, all of it rows and table cells. At
  most 5 000 records (more is ErrRejected, dropped), text fields cut to
  255 bytes on a character boundary, a record's count capped.
- Medium: the cap of 500 reports was for the whole server, so a stranger
  filing under one domain's address evicted every other domain's
  reports. PruneDMARCReports counts per domain.
- Low: the pipe service ran up to default_process_limit (100) commands
  at once against the panel's database; maxproc is 1.
- Low: the reason logged for a dropped message carried the sender's
  bytes (header line, zip entry name) as they came; cut to 200 bytes and
  reduced to printable ASCII, zip entry names quoted.
- Low: SELFPOST_DATA_DIR / SELFPOST_DB_PATH with whitespace or "$" would
  reach the command through export_environment as another path; refused
  at start when DMARC ingest is on.

Left as they are, told to the owner: the parse-failure count is a
lifetime figure shown under "This week"; a report larger than the
command reads is dropped as unreadable; nested multipart is not searched.

The hub's Retention line follows the design commit before this one.
security.md: the review, and the accepted-risk entry says what a
stranger can and cannot do now. The roadmap no longer calls the review
pending.

  go vet ./cmd/... ./internal/dmarc/ ./internal/store/: ok
  go test ./cmd/... ./internal/dmarc/ ./internal/store/ ./internal/web/...: ok
  make e2e on the test server (with the port-25 scenarios of the next
  commit): ok

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 17:40:31 +03:00
mixandClaude Opus 5.5 505260e227 design: the DMARC hub says reports are kept 500 a domain
The retention cap is counted per domain (the security commit after this one); the Ingest box says so: 500 reports a domain or 90 days.

check.py: 0 violations; outlines.json unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 17:40:30 +03:00
mixandClaude Opus 5.5 09d9625f32 dmarc-ingest: what is not a report this relay keeps is dropped, not retried
Owner, 2026-10-11: foreign and malformed reports are turned away.

The command answered every failure with EX_TEMPFAIL, so Postfix kept a
message that could never be stored and offered it again for days, each
attempt adding to the parse-failure count. IngestMessage now marks those
failures (unreadable as an aggregate report, a domain not sent from
here, another domain's report address) with ErrRejected; the command
counts one failure, writes the reason where Postfix puts it on the delivery line of the mail log
and reports the delivery as done. No bounce: the envelope sender of such
a message is whatever its author wrote, and the report addresses are
public. A failing store is still EX_TEMPFAIL.

  go vet ./cmd/... ./internal/dmarc/: ok
  go test ./cmd/panel/ ./internal/dmarc/ ./internal/web/...: ok

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 16:28:12 +03:00
100 changed files with 6542 additions and 223 deletions
+137
View File
@@ -5,6 +5,143 @@ Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); version
## [Unreleased] ## [Unreleased]
### Changed
- **Port 25 logs as `postfix/inbound`.** Its lines in the mail log (Server ›
System log) now read `postfix/inbound/smtpd[…]` where they read
`postfix/smtpd[…]`; the submission ports keep `postfix/smtpd`. This is how a
refusal on port 25 is told from one on 465 / 587.
### Added
- Schema: migration `0002_inbound_spam_log.sql`, the first after the 2.0
baseline - two empty tables for the inbound antispam journal and the filter
lists (inbound-antispam-panel). A 2.0.x database is migrated when the panel
starts.
- **Inbound mail is recorded.** With the inbound relay on, the panel runs a
second milter, the last one on port 25, and writes a row for every recipient
of a message that was accepted for an inbound domain: client address,
envelope, subject and - with a spam filter attached - the filter's action,
score and heaviest symbols. It only watches: it refuses nothing and Postfix
accepts mail when it is down.
- **Refused inbound mail is recorded too**, from the mail log: a recipient
Postfix refused at an inbound domain (unknown user, sender domain not
found), and a message the spam filter rejected, deferred, held or
discarded. The row is dated by its log line and carries the reply text. A
refusal to relay for a domain that is not an inbound one is not recorded.
- The inbound journal is pruned every six hours: rows older than 90 days
(`INBOUND_SPAM_LOG_RETENTION_DAYS`, 7-365) and anything beyond 10 000 rows.
- The antispam compose fragment mounts `antispam/rspamd/milter_headers.conf`
into the rspamd sidecar, which makes it write its verdict into
`X-Rspamd-Action` and `X-Spamd-Result`. **Copy `deploy/antispam/rspamd/`
next to the fragment before starting it** - Docker creates a directory in
place of a missing file and the sidecar then fails to read its
configuration.
- **Allow and deny lists for the spam filter**, kept by SelfPost and applied
by rspamd: sender addresses, sender domains, client addresses and networks.
They are written as six files under `data/antispam/`, which the fragment
mounts into the sidecar together with `antispam/rspamd/multimap.conf`;
rspamd rereads a changed file by itself. Deny wins over allow. The files
are there from the first start with the inbound relay on.
- **Inbound › Log**: the journal as a page - time, sender and client address,
recipient, subject and decision, filtered by domain, decision and day. The
decision opens a sheet over the page with who decided and why: the filter's
verdict and symbols, or the reply a refusal was given. A user with the
*domain* role sees the rows of the inbound domains assigned to them. For the
global role a row links to the filter lists with the sender filled in:
*Deny sender* on accepted mail, *Allow sender* on mail the filter refused.
- **Inbound › Filter lists** (global role): add and remove allow and deny
entries; the page says where the lists go and whether the filter is on to
read them. Each entry shows how often it decided a message - *Matched*, or
*never* - and its kind opens a sheet with the note, the day it was added and
the last match, so that entries nothing matches any more can be found and
removed. The count is read from the inbound log and reaches back as far as
the log does.
- **Server › Settings**: *Keep inbound log rows, days*, shown with the inbound
relay on.
- Help: *Inbound log* and *Filter lists*.
- New setting `ANTISPAM_DIR` (default `/data/antispam`).
- A full backup carries the inbound log, the filter lists and their files
under `data/antispam/`. It does not carry the `antispam/` directory with the
compose fragment and the rspamd configuration - copy that from
`deploy/antispam/` when restoring.
- New setting `INBOUND_JOURNAL_MILTER_SOCKET` (default
`/run/selfpost/inbound-journal.sock`).
## [2.0.1] - 2026-10-11
### Changed
- **A message that is not a report this relay keeps is dropped at once.**
One that cannot be read as an aggregate report, reports on a domain not
sent from here, or came to another domain's report address used to be put
off and offered again by Postfix for days, counted as a parse failure each
time. It is now accepted and discarded - no retry, and no bounce, since the
envelope sender of such a message is whatever its author wrote. It counts
as one parse failure and the reason is in the mail log, on the message's
own delivery line (Server › System log, `dropped: …`). Only a failure of
the database is still retried.
- **A full backup no longer holds the Postfix queue**, from the panel or from
`selfpost-backup`. It is taken while the instance runs, and a copy of a
queue Postfix is working on can hold a message twice or not at all. Mail
still waiting for delivery is not carried over; a restored instance starts
with an empty queue. To keep the queue, stop the instance and `tar` the
project directory.
- DMARC reports are kept 500 **for each domain** (and 90 days), not 500 for
the whole server - see Security.
### Fixed
- **Full backup from the panel works.** Since the Postfix queue moved under
`/data` in 1.3.0 the button answered with a 200 and ten bytes: the archive
includes the queue, Postfix keeps the queue readable by its own user alone,
and the panel is not that user ("open /data/postfix/queue/active:
permission denied" in the log). `selfpost-backup` run with `docker exec`
was not affected - it runs as root. The queue is no longer part of a backup
(see Changed), so the two archives are the same again. Everything is read
through before the download starts, so a failure is a message on the page
and not a file cut short. One thing the panel may still be unable to read
is the private key in `certs/`: it then refuses and says what to do -
`chgrp 998 certs/privkey.pem && chmod 640 certs/privkey.pem` on the host,
998 being the container's `selfpost` group, whose id is now fixed in the
image.
### Security
Review of the DMARC report path (Fable, 2026-10-11), which started to run for
the first time in 2.0.0. Nothing Critical or High.
- **A report is bounded before it is stored.** The XML comes from anyone who
knows a report address, and its 8 MiB limit left room for some 160 000
records or a single name of megabytes - rows in the database and cells of
the report page. A report holds at most 5 000 records (a larger one is
dropped), every text field is cut to 255 bytes, and a record's count is
capped so the sums cannot wrap.
- **One domain's reports cannot push out another's.** The cap of 500 was for
the whole server, so a stranger filing reports under one domain's address
could evict every other domain's history. It is counted per domain now.
- The delivery command runs one at a time, not up to a hundred in parallel.
- The reason written to the mail log for a dropped message is cut to 200
bytes and anything outside printable ASCII is replaced: parts of it are the
sender's own bytes (a header line, a file name inside an archive).
- `SELFPOST_DATA_DIR` / `SELFPOST_DB_PATH` with whitespace or `$` are refused
at start when DMARC ingest is on, instead of handing the delivery command
another path and so an empty database of its own.
### Added
- e2e: the run now ends on port 25 with both optional features on. A DMARC
report is mailed to a domain's hosted address and has to appear under
Outbound › DMARC reports; a report on another domain and a message with no
report are accepted and dropped; mail for an inbound domain reaches its
upstream and other recipients are refused. Until now the suite only read
back the generated configuration of these features.
- e2e: full backup and restore. The run takes a backup from the panel, throws
the installation away, unpacks the archive into the empty project directory
and starts it: the administrator signs in, the domain has the same DKIM key,
and mail sent by the application verifies against the record published
before the backup.
## [2.0.0] - 2026-10-11 ## [2.0.0] - 2026-10-11
The panel redesign. A breaking release: every panel URL changed and the The panel redesign. A breaking release: every panel URL changed and the
+1 -1
View File
@@ -112,7 +112,7 @@ docker run --rm -d --name selfpost-try \
-e SELFPOST_HOSTNAME=mail.local.test \ -e SELFPOST_HOSTNAME=mail.local.test \
-e PANEL_COOKIE_SECURE=false \ -e PANEL_COOKIE_SECURE=false \
-v selfpost-try-data:/data \ -v selfpost-try-data:/data \
ghcr.io/mixeme/selfpost:2.0.0 ghcr.io/mixeme/selfpost:2.0.1
``` ```
**Get the setup URL** (pick one): **Get the setup URL** (pick one):
+4 -1
View File
@@ -62,7 +62,10 @@ RUN useradd --system --no-create-home --shell /usr/sbin/nologin panel
# entrypoint.sh) — lets OpenDKIM read the panel-owned keys and lets Postfix read # entrypoint.sh) — lets OpenDKIM read the panel-owned keys and lets Postfix read
# the sasldb2/sender map, and lets the panel reach the supervisor control socket # the sasldb2/sender map, and lets the panel reach the supervisor control socket
# to signal OpenDKIM/Postfix reloads without any process running as root. # to signal OpenDKIM/Postfix reloads without any process running as root.
RUN groupadd --system selfpost \ # The id is fixed, and it is the one the group has always had: an operator
# gives the TLS private key on the host to this group by number, so that the
# panel can put it into a full backup (guide § Taking a full backup).
RUN groupadd --system --gid 998 selfpost \
&& usermod -aG selfpost panel \ && usermod -aG selfpost panel \
&& usermod -aG selfpost opendkim \ && usermod -aG selfpost opendkim \
&& usermod -aG selfpost postfix && usermod -aG selfpost postfix
+9
View File
@@ -112,6 +112,15 @@ for f in relay_domains transport relay_recipients tls_policy; do
done done
chown panel:selfpost /data/postfix chown panel:selfpost /data/postfix
chmod 2750 /data/postfix chmod 2750 /data/postfix
# Filter lists for the external spam filter (written by the panel when
# INBOUND_RELAY_ENABLE=true, read by the antispam sidecar, which mounts this
# directory). The sidecar runs under a user of its own in another container,
# so the directory and its files are readable by anyone: they hold sender
# addresses and networks, no secrets.
mkdir -p /data/antispam
chown panel:selfpost /data/antispam
chmod 0755 /data/antispam
chown panel:selfpost /data/postfix/sender_login_maps chown panel:selfpost /data/postfix/sender_login_maps
chmod 0640 /data/postfix/sender_login_maps chmod 0640 /data/postfix/sender_login_maps
+38 -2
View File
@@ -117,6 +117,9 @@ SUBMISSION_ENABLE="${SUBMISSION_ENABLE:-false}"
INBOUND_RELAY_ENABLE="${INBOUND_RELAY_ENABLE:-false}" INBOUND_RELAY_ENABLE="${INBOUND_RELAY_ENABLE:-false}"
INBOUND_ANTISPAM_MILTER="${INBOUND_ANTISPAM_MILTER:-}" INBOUND_ANTISPAM_MILTER="${INBOUND_ANTISPAM_MILTER:-}"
INBOUND_ANTISPAM_MILTER_ACTION="${INBOUND_ANTISPAM_MILTER_ACTION:-accept}" INBOUND_ANTISPAM_MILTER_ACTION="${INBOUND_ANTISPAM_MILTER_ACTION:-accept}"
# The panel's inbound journal milter (monitoring, fail-open), on port 25 only
# and only while the inbound relay is on. Matched by postfix-wrapper.sh.
INBOUND_JOURNAL_SOCK="${INBOUND_JOURNAL_MILTER_SOCKET:-/run/selfpost/inbound-journal.sock}"
INBOUND_RATE_LIMIT_MESSAGES_PER_IP="${INBOUND_RATE_LIMIT_MESSAGES_PER_IP:-20}" INBOUND_RATE_LIMIT_MESSAGES_PER_IP="${INBOUND_RATE_LIMIT_MESSAGES_PER_IP:-20}"
INBOUND_MESSAGE_SIZE_LIMIT="${INBOUND_MESSAGE_SIZE_LIMIT:-26214400}" INBOUND_MESSAGE_SIZE_LIMIT="${INBOUND_MESSAGE_SIZE_LIMIT:-26214400}"
RELAY_DOMAINS_MAP="${POSTFIX_RELAY_DOMAINS:-/data/postfix/relay_domains}" RELAY_DOMAINS_MAP="${POSTFIX_RELAY_DOMAINS:-/data/postfix/relay_domains}"
@@ -291,11 +294,23 @@ if [ "${INBOUND_RELAY_ENABLE}" = "true" ] || [ "${DMARC_REPORTS_ENABLE}" = "true
# attribute" on every report. They go into the service line itself. # attribute" on every report. They go into the service line itself.
# ${recipient} is expanded by pipe(8), not by the shell: the command # ${recipient} is expanded by pipe(8), not by the shell: the command
# learns which report address the message came to from its argument. # learns which report address the message came to from its argument.
postconf -M 'dmarc-ingest/unix=dmarc-ingest unix - n n - - pipe flags=Rq user=panel argv=/usr/local/bin/dmarc-ingest ${recipient}' # One at a time (the 1 before "pipe"): reports are in no hurry, every run
# writes to the database the panel writes to, and a hundred of them in
# parallel - Postfix's default - is what a flood of reports would start.
postconf -M 'dmarc-ingest/unix=dmarc-ingest unix - n n - 1 pipe flags=Rq user=panel argv=/usr/local/bin/dmarc-ingest ${recipient}'
# One recipient per run, so ${recipient} is one argument. # One recipient per run, so ${recipient} is one argument.
postconf -e "dmarc-ingest_destination_recipient_limit=1" postconf -e "dmarc-ingest_destination_recipient_limit=1"
# pipe(8) starts the command with export_environment only, so what the # pipe(8) starts the command with export_environment only, so what the
# command reads from the container's environment is named there. # command reads from the container's environment is named there.
# export_environment is a list split at whitespace in which Postfix
# expands "$": a path with either would reach the command as another
# path, and it would quietly open a database of its own.
case "${SELFPOST_DATA_DIR:-}${SELFPOST_DB_PATH:-}" in
*[[:space:]]* | *'$'*)
echo "postfix-config: SELFPOST_DATA_DIR and SELFPOST_DB_PATH must not contain whitespace or \$ when DMARC_REPORTS_ENABLE=true" >&2
exit 1
;;
esac
DMARC_INGEST_ENV="DMARC_REPORTS_ENABLE=true" DMARC_INGEST_ENV="DMARC_REPORTS_ENABLE=true"
[ -z "${SELFPOST_DATA_DIR:-}" ] || DMARC_INGEST_ENV="${DMARC_INGEST_ENV} SELFPOST_DATA_DIR=${SELFPOST_DATA_DIR}" [ -z "${SELFPOST_DATA_DIR:-}" ] || DMARC_INGEST_ENV="${DMARC_INGEST_ENV} SELFPOST_DATA_DIR=${SELFPOST_DATA_DIR}"
[ -z "${SELFPOST_DB_PATH:-}" ] || DMARC_INGEST_ENV="${DMARC_INGEST_ENV} SELFPOST_DB_PATH=${SELFPOST_DB_PATH}" [ -z "${SELFPOST_DB_PATH:-}" ] || DMARC_INGEST_ENV="${DMARC_INGEST_ENV} SELFPOST_DB_PATH=${SELFPOST_DB_PATH}"
@@ -345,7 +360,8 @@ if [ "${INBOUND_RELAY_ENABLE}" = "true" ] || [ "${DMARC_REPORTS_ENABLE}" = "true
# would file inbound mail into the outbound send log under the *sender's* # would file inbound mail into the outbound send log under the *sender's*
# domain — polluting Deliveries and letting a forged From: consume that # domain — polluting Deliveries and letting a forged From: consume that
# domain's level-2 rate-limit budget. So smtpd_milters is always overridden # domain's level-2 rate-limit budget. So smtpd_milters is always overridden
# here: either the optional antispam milter, or nothing at all. # here: the optional antispam milter, then - with the inbound relay on - the
# panel's inbound journal milter, or nothing at all.
INBOUND_MILTERS="" INBOUND_MILTERS=""
if [ "${INBOUND_RELAY_ENABLE}" = "true" ] && [ -n "${INBOUND_ANTISPAM_MILTER}" ]; then if [ "${INBOUND_RELAY_ENABLE}" = "true" ] && [ -n "${INBOUND_ANTISPAM_MILTER}" ]; then
case "${INBOUND_ANTISPAM_MILTER}" in case "${INBOUND_ANTISPAM_MILTER}" in
@@ -364,6 +380,22 @@ if [ "${INBOUND_RELAY_ENABLE}" = "true" ] || [ "${DMARC_REPORTS_ENABLE}" = "true
esac esac
INBOUND_MILTERS="{${INBOUND_ANTISPAM_MILTER},default_action=${INBOUND_ANTISPAM_MILTER_ACTION}}" INBOUND_MILTERS="{${INBOUND_ANTISPAM_MILTER},default_action=${INBOUND_ANTISPAM_MILTER_ACTION}}"
fi fi
if [ "${INBOUND_RELAY_ENABLE}" = "true" ]; then
# Last in the chain: Postfix hands a message to its milters one after
# another at end-of-message, so this one reads the verdict headers the
# filter added and sees only what the filter let through. It watches and
# never refuses, and must never hold up someone else's mail: accept when
# it is down. A comma, not a space, between the two - see
# master_cf_set_smtp_inet_o.
case "${INBOUND_JOURNAL_SOCK}" in
/[A-Za-z0-9._/-]*) ;;
*)
echo "FATAL: INBOUND_JOURNAL_MILTER_SOCKET must be an absolute path, got: ${INBOUND_JOURNAL_SOCK}" >&2
exit 1
;;
esac
INBOUND_MILTERS="${INBOUND_MILTERS:+${INBOUND_MILTERS},}{unix:${INBOUND_JOURNAL_SOCK},default_action=accept}"
fi
PORT25_RATE="${INBOUND_RATE_LIMIT_MESSAGES_PER_IP}" PORT25_RATE="${INBOUND_RATE_LIMIT_MESSAGES_PER_IP}"
PORT25_SIZE="${INBOUND_MESSAGE_SIZE_LIMIT}" PORT25_SIZE="${INBOUND_MESSAGE_SIZE_LIMIT}"
@@ -380,8 +412,12 @@ if [ "${INBOUND_RELAY_ENABLE}" = "true" ] || [ "${DMARC_REPORTS_ENABLE}" = "true
fi fi
fi fi
# syslog_name: this listener's lines read postfix/inbound/smtpd, the
# submission ports' postfix/smtpd. The log tailer files a refusal on port 25
# into the inbound journal and has to tell the two apart.
postconf -M "smtp/inet=smtp inet n - n - - smtpd" postconf -M "smtp/inet=smtp inet n - n - - smtpd"
postconf -P \ postconf -P \
"smtp/inet/syslog_name=postfix/inbound" \
"smtp/inet/smtpd_sasl_auth_enable=no" \ "smtp/inet/smtpd_sasl_auth_enable=no" \
"smtp/inet/smtpd_tls_auth_only=no" \ "smtp/inet/smtpd_tls_auth_only=no" \
"smtp/inet/smtpd_sender_login_maps=" \ "smtp/inet/smtpd_sender_login_maps=" \
+10 -4
View File
@@ -2,8 +2,9 @@
# Postfix start wrapper: solves the cold-start race where Postfix would try to # Postfix start wrapper: solves the cold-start race where Postfix would try to
# reach the milter sockets before they are listening. # reach the milter sockets before they are listening.
# #
# It blocks until BOTH milter sockets — OpenDKIM and the panel's journal-milter # It blocks until the milter sockets — OpenDKIM, the panel's journal-milter
# — are present, then execs `postfix start-fg`. If they are not ready within the # and, with the inbound relay on, its inbound journal milter — are present,
# then execs `postfix start-fg`. If they are not ready within the
# timeout it exits non-zero WITHOUT starting Postfix, so supervisord/Docker see # timeout it exits non-zero WITHOUT starting Postfix, so supervisord/Docker see
# an explicit startup failure instead of a relay running blind. # an explicit startup failure instead of a relay running blind.
# #
@@ -16,8 +17,13 @@ JOURNAL_SOCK="${JOURNAL_MILTER_SOCKET:-/run/selfpost/journal.sock}"
TIMEOUT="${MILTER_WAIT_TIMEOUT:-30}" TIMEOUT="${MILTER_WAIT_TIMEOUT:-30}"
INTERVAL=1 INTERVAL=1
SOCKS="$OPENDKIM_SOCK $JOURNAL_SOCK"
if [ "${INBOUND_RELAY_ENABLE:-false}" = "true" ]; then
SOCKS="$SOCKS ${INBOUND_JOURNAL_MILTER_SOCKET:-/run/selfpost/inbound-journal.sock}"
fi
elapsed=0 elapsed=0
for sock in "$OPENDKIM_SOCK" "$JOURNAL_SOCK"; do for sock in $SOCKS; do
while [ ! -S "$sock" ]; do while [ ! -S "$sock" ]; do
if [ "$elapsed" -ge "$TIMEOUT" ]; then if [ "$elapsed" -ge "$TIMEOUT" ]; then
echo "postfix-wrapper: timed out after ${TIMEOUT}s waiting for milter socket $sock" >&2 echo "postfix-wrapper: timed out after ${TIMEOUT}s waiting for milter socket $sock" >&2
@@ -29,5 +35,5 @@ for sock in "$OPENDKIM_SOCK" "$JOURNAL_SOCK"; do
echo "postfix-wrapper: milter socket ready: $sock" echo "postfix-wrapper: milter socket ready: $sock"
done done
echo "postfix-wrapper: both milter sockets ready, starting postfix" echo "postfix-wrapper: milter sockets ready, starting postfix"
exec postfix start-fg exec postfix start-fg
+35 -1
View File
@@ -1,6 +1,7 @@
package main package main
import ( import (
"errors"
"fmt" "fmt"
"io" "io"
"log" "log"
@@ -28,14 +29,47 @@ func runDMARCIngestMode() error {
recipient := ingestRecipient(os.Args[1:]) recipient := ingestRecipient(os.Args[1:])
if err := dmarc.IngestMessage(st, io.LimitReader(os.Stdin, maxIngestStdin), recipient, time.Now().UTC()); err != nil { if err := dmarc.IngestMessage(st, io.LimitReader(os.Stdin, maxIngestStdin), recipient, time.Now().UTC()); err != nil {
if !errors.Is(err, dmarc.ErrRejected) {
return err
}
// Not a report this relay keeps. It is dropped: deferring would have
// Postfix try it again for days, and a bounce would go to an envelope
// sender anyone can write - the report addresses are public.
if incrErr := st.IncrDMARCParseFailures(); incrErr != nil { if incrErr := st.IncrDMARCParseFailures(); incrErr != nil {
log.Printf("dmarc ingest: record failure: %v", incrErr) log.Printf("dmarc ingest: record failure: %v", incrErr)
} }
return err logRejected(err)
} }
return nil return nil
} }
// logRejected leaves the reason in the mail log. Postfix puts the start of
// what a delivery command wrote - about a hundred bytes - into its own line
// for the message ("status=sent (delivered via dmarc-ingest service (...))"),
// so the reason goes out first and bare: no prefix, no date, no recipient,
// all of which that line already has. postlog is no use here: pipe(8) starts
// the command without the group that may write to Postfix's log socket.
func logRejected(reason error) {
fmt.Fprintln(os.Stderr, "dropped: "+logSafe(strings.TrimPrefix(reason.Error(), dmarc.ErrRejected.Error()+": ")))
}
// logSafe makes a reason fit for a log line. Parts of it are the sender's own
// bytes - a header line, a file name inside an archive, a piece of XML - so it
// is cut short and anything that is not printable ASCII becomes "?", whatever
// Postfix would have done with it.
func logSafe(s string) string {
const max = 200
if len(s) > max {
s = s[:max]
}
return strings.Map(func(r rune) rune {
if r < 0x20 || r > 0x7e {
return '?'
}
return r
}, s)
}
// ingestRecipient is the report address the message was delivered to. Postfix's // ingestRecipient is the report address the message was delivered to. Postfix's
// pipe(8) hands it over as an argument (${recipient} in the service's argv, // pipe(8) hands it over as an argument (${recipient} in the service's argv,
// build/postfix-config.sh) - it sets no RECIPIENT variable, that is local(8). // build/postfix-config.sh) - it sets no RECIPIENT variable, that is local(8).
+14
View File
@@ -18,3 +18,17 @@ func TestIngestRecipient(t *testing.T) {
} }
} }
} }
func TestLogSafe(t *testing.T) {
got := logSafe("zip entry \"a\nstatus=bounced\tx\" \x1b[31m é")
if want := "zip entry \"a?status=bounced?x\" ?[31m ?"; got != want {
t.Errorf("logSafe = %q, want %q", got, want)
}
long := make([]byte, 1000)
for i := range long {
long[i] = 'a'
}
if got := logSafe(string(long)); len(got) != 200 {
t.Errorf("logSafe kept %d bytes of 1000, want 200", len(got))
}
}
+11 -2
View File
@@ -24,6 +24,7 @@ type mailStack struct {
Domains *domain.Service Domains *domain.Service
Apps *app.Service Apps *app.Service
Inbound *inbound.Service Inbound *inbound.Service
Lists *inbound.Lists
DMARC *dmarc.Service DMARC *dmarc.Service
pf *postfix.Postfix pf *postfix.Postfix
odk *domain.OpenDKIM odk *domain.OpenDKIM
@@ -40,7 +41,8 @@ func newMailStack(cfg config, st *store.Store) *mailStack {
dmarcSvc := dmarc.NewService(st, pf, cfg.hostname, cfg.dmarcEnabled) dmarcSvc := dmarc.NewService(st, pf, cfg.hostname, cfg.dmarcEnabled)
return &mailStack{ return &mailStack{
Domains: domains, Apps: apps, Inbound: inb, DMARC: dmarcSvc, Domains: domains, Apps: apps, Inbound: inb, DMARC: dmarcSvc,
pf: pf, odk: odk, inboundEnabled: cfg.inboundEnabled, dmarcEnabled: cfg.dmarcEnabled, Lists: inbound.NewLists(st, cfg.antispamDir),
pf: pf, odk: odk, inboundEnabled: cfg.inboundEnabled, dmarcEnabled: cfg.dmarcEnabled,
} }
} }
@@ -57,6 +59,11 @@ func (m *mailStack) Resync() error {
if err := m.Inbound.Resync(); err != nil { if err := m.Inbound.Resync(); err != nil {
return fmt.Errorf("inbound maps resync: %w", err) return fmt.Errorf("inbound maps resync: %w", err)
} }
// The filter's configuration names all six list files; they exist from
// the first start, empty until an entry is added.
if err := m.Lists.Sync(); err != nil {
return fmt.Errorf("filter lists resync: %w", err)
}
} }
if m.dmarcEnabled { if m.dmarcEnabled {
if err := m.DMARC.Resync(); err != nil { if err := m.DMARC.Resync(); err != nil {
@@ -91,7 +98,7 @@ func newPanel(cfg config, st *store.Store) (*web.Server, error) {
// effective config, including a manual override; the panel keeps this // effective config, including a manual override; the panel keeps this
// snapshot for the process lifetime (architecture.md). // snapshot for the process lifetime (architecture.md).
retryPolicy := postfix.LoadRetryPolicy() retryPolicy := postfix.LoadRetryPolicy()
return web.New(st, ms.Domains, ms.Apps, ms.Inbound, ms.DMARC, web.Config{ return web.New(st, ms.Domains, ms.Apps, ms.Inbound, ms.Lists, ms.DMARC, web.Config{
Hostname: cfg.hostname, Hostname: cfg.hostname,
CookieSecure: cfg.cookieSecure, CookieSecure: cfg.cookieSecure,
SubmissionEnabled: cfg.submissionEnabled, SubmissionEnabled: cfg.submissionEnabled,
@@ -114,6 +121,8 @@ func newPanel(cfg config, st *store.Store) (*web.Server, error) {
InboundAntispamAction: cfg.inboundAntispamAction, InboundAntispamAction: cfg.inboundAntispamAction,
DMARCEnabled: cfg.dmarcEnabled, DMARCEnabled: cfg.dmarcEnabled,
SendLogRetentionEnvDefault: cfg.retentionDays, SendLogRetentionEnvDefault: cfg.retentionDays,
InboundSpamLogRetentionEnvDefault: cfg.inboundSpamLogRetentionDays,
}, cfg.setupTokenPath) }, cfg.setupTokenPath)
} }
+28 -9
View File
@@ -15,21 +15,42 @@ import (
// serveJournal opens the journal-milter Unix socket and runs the real milter // serveJournal opens the journal-milter Unix socket and runs the real milter
// (architecture.md § Mail path), recording accepted messages into the send // (architecture.md § Mail path), recording accepted messages into the send
// log. Socket lifecycle (creation, stale cleanup, group permissions) lives // log. Socket lifecycle (creation, stale cleanup, group permissions) lives
// here; the protocol handler lives in internal/milter. // in listenMilter; the protocol handler lives in internal/milter.
func serveJournal(ctx context.Context, cfg config, st *store.Store) error { func serveJournal(ctx context.Context, cfg config, st *store.Store) error {
socketPath := cfg.journalSocket ln, err := listenMilter(cfg.journalSocket)
if err := os.MkdirAll(filepath.Dir(socketPath), 0o755); err != nil { if err != nil {
return err return err
} }
log.Printf("journal-milter listening on %s", cfg.journalSocket)
return milter.Serve(ctx, ln, st)
}
// serveInboundJournal runs the inbound journal milter, the last milter on
// port 25 (internal/milter/inbound.go). It exists only while the inbound relay
// is on: build/postfix-config.sh attaches the socket under the same condition.
func serveInboundJournal(ctx context.Context, cfg config, st *store.Store) error {
ln, err := listenMilter(cfg.inboundJournalSocket)
if err != nil {
return err
}
log.Printf("inbound-journal-milter listening on %s", cfg.inboundJournalSocket)
return milter.ServeInbound(ctx, ln, st, cfg.inboundAntispamMilter != "")
}
// listenMilter opens a milter's Unix socket where Postfix can reach it.
func listenMilter(socketPath string) (net.Listener, error) {
if err := os.MkdirAll(filepath.Dir(socketPath), 0o755); err != nil {
return nil, err
}
// Clear a stale socket left behind by an unclean shutdown, otherwise the // Clear a stale socket left behind by an unclean shutdown, otherwise the
// listen below fails with "address already in use". // listen below fails with "address already in use".
if err := os.Remove(socketPath); err != nil && !errors.Is(err, os.ErrNotExist) { if err := os.Remove(socketPath); err != nil && !errors.Is(err, os.ErrNotExist) {
return err return nil, err
} }
ln, err := net.Listen("unix", socketPath) ln, err := net.Listen("unix", socketPath)
if err != nil { if err != nil {
return err return nil, err
} }
// Postfix (user `postfix`) connects to this socket as a milter and needs // Postfix (user `postfix`) connects to this socket as a milter and needs
// group access. The socket inherits group `selfpost` from the setgid parent // group access. The socket inherits group `selfpost` from the setgid parent
@@ -37,9 +58,7 @@ func serveJournal(ctx context.Context, cfg config, st *store.Store) error {
// it (connecting to a Unix socket needs write permission on the node). // it (connecting to a Unix socket needs write permission on the node).
if err := os.Chmod(socketPath, 0o660); err != nil { if err := os.Chmod(socketPath, 0o660); err != nil {
ln.Close() ln.Close()
return err return nil, err
} }
return ln, nil
log.Printf("journal-milter listening on %s", socketPath)
return milter.Serve(ctx, ln, st)
} }
+51 -4
View File
@@ -23,6 +23,7 @@ import (
"github.com/mixeme/selfpost/internal/backup" "github.com/mixeme/selfpost/internal/backup"
"github.com/mixeme/selfpost/internal/buildinfo" "github.com/mixeme/selfpost/internal/buildinfo"
"github.com/mixeme/selfpost/internal/dnscheck" "github.com/mixeme/selfpost/internal/dnscheck"
"github.com/mixeme/selfpost/internal/inbound"
"github.com/mixeme/selfpost/internal/logtail" "github.com/mixeme/selfpost/internal/logtail"
"github.com/mixeme/selfpost/internal/store" "github.com/mixeme/selfpost/internal/store"
) )
@@ -89,6 +90,9 @@ type config struct {
saslRealm string saslRealm string
postfixDir string postfixDir string
deployRoot string deployRoot string
// Where the filter lists are written as map files for the external spam
// filter; the antispam compose fragment mounts it into the sidecar.
antispamDir string
inboundEnabled bool inboundEnabled bool
// The optional milter on the inbound listener (INBOUND_ANTISPAM_MILTER) and // The optional milter on the inbound listener (INBOUND_ANTISPAM_MILTER) and
@@ -96,7 +100,12 @@ type config struct {
// what attaches it; the panel only says so. See antispamAction. // what attaches it; the panel only says so. See antispamAction.
inboundAntispamMilter string inboundAntispamMilter string
inboundAntispamAction string inboundAntispamAction string
dmarcEnabled bool // The inbound journal milter's socket; the default mirrors
// build/postfix-config.sh. Served only while the inbound relay is on.
inboundJournalSocket string
// Retention of the inbound journal until Server > Settings says otherwise.
inboundSpamLogRetentionDays int
dmarcEnabled bool
} }
func loadConfig() config { func loadConfig() config {
@@ -157,11 +166,16 @@ func loadConfig() config {
saslDBPath: envDefault("SASL_DB_PATH", filepath.Join(dataDir, "sasl", "sasldb2")), saslDBPath: envDefault("SASL_DB_PATH", filepath.Join(dataDir, "sasl", "sasldb2")),
saslRealm: saslRealm(), saslRealm: saslRealm(),
postfixDir: envDefault("POSTFIX_DIR", filepath.Join(dataDir, "postfix")), postfixDir: envDefault("POSTFIX_DIR", filepath.Join(dataDir, "postfix")),
deployRoot: envDefault("SELFPOST_DEPLOY_ROOT", "/selfpost-deploy"),
antispamDir: envDefault("ANTISPAM_DIR", filepath.Join(dataDir, "antispam")),
deployRoot: envDefault("SELFPOST_DEPLOY_ROOT", "/selfpost-deploy"),
// Optional inbound relay (backup-MX / forwarder). Off unless exactly "true". // Optional inbound relay (backup-MX / forwarder). Off unless exactly "true".
inboundEnabled: os.Getenv("INBOUND_RELAY_ENABLE") == "true", inboundEnabled: os.Getenv("INBOUND_RELAY_ENABLE") == "true",
inboundAntispamMilter: os.Getenv("INBOUND_ANTISPAM_MILTER"), inboundAntispamMilter: os.Getenv("INBOUND_ANTISPAM_MILTER"),
inboundAntispamAction: antispamAction(os.Getenv("INBOUND_ANTISPAM_MILTER_ACTION")), inboundAntispamAction: antispamAction(os.Getenv("INBOUND_ANTISPAM_MILTER_ACTION")),
inboundJournalSocket: envDefault("INBOUND_JOURNAL_MILTER_SOCKET", "/run/selfpost/inbound-journal.sock"),
inboundSpamLogRetentionDays: envInt("INBOUND_SPAM_LOG_RETENTION_DAYS", store.InboundSpamLogRetentionDaysDefault),
// Optional DMARC aggregate ingest on port 25. Off unless exactly "true". // Optional DMARC aggregate ingest on port 25. Off unless exactly "true".
dmarcEnabled: os.Getenv("DMARC_REPORTS_ENABLE") == "true", dmarcEnabled: os.Getenv("DMARC_REPORTS_ENABLE") == "true",
} }
@@ -289,12 +303,38 @@ func run() error {
} }
} }
// The filter lists as files, on every start: the spam filter's
// configuration names all six, and a data directory from before the lists
// existed has none. Not fatal - the lists are the filter's business, and
// the panel has mail to relay either way.
if cfg.inboundEnabled && !restored {
if err := inbound.NewLists(st, cfg.antispamDir).Sync(); err != nil {
log.Printf("filter lists bootstrap: %v", err)
}
}
if err := st.EnsureSendLogRetentionDays(cfg.retentionDays); err != nil { if err := st.EnsureSendLogRetentionDays(cfg.retentionDays); err != nil {
return fmt.Errorf("bootstrap send-log retention: %w", err) return fmt.Errorf("bootstrap send-log retention: %w", err)
} }
// With the inbound relay on, the tailer also writes the inbound journal's
// rows for refused mail and prunes that journal.
var inboundTail *logtail.Inbound
if cfg.inboundEnabled {
if err := st.EnsureInboundSpamLogRetentionDays(cfg.inboundSpamLogRetentionDays); err != nil {
return fmt.Errorf("bootstrap inbound journal retention: %w", err)
}
inboundTail = &logtail.Inbound{Store: st, Retention: func() int {
days, err := st.GetInboundSpamLogRetentionDays(cfg.inboundSpamLogRetentionDays)
if err != nil {
log.Printf("inbound journal retention: %v", err)
return cfg.inboundSpamLogRetentionDays
}
return days
}}
}
var wg sync.WaitGroup var wg sync.WaitGroup
errc := make(chan error, 4) errc := make(chan error, 5)
roles := []struct { roles := []struct {
name string name string
@@ -310,7 +350,7 @@ func run() error {
return cfg.retentionDays return cfg.retentionDays
} }
return days return days
}) }, inboundTail)
}}, }},
{"rate-limit-recalc", func(ctx context.Context) error { {"rate-limit-recalc", func(ctx context.Context) error {
runAutoRateLimitRecalc(ctx, cfg, st) runAutoRateLimitRecalc(ctx, cfg, st)
@@ -318,6 +358,13 @@ func run() error {
}}, }},
} }
if cfg.inboundEnabled {
roles = append(roles, struct {
name string
fn func(context.Context) error
}{"inbound-journal-milter", func(ctx context.Context) error { return serveInboundJournal(ctx, cfg, st) }})
}
for _, r := range roles { for _, r := range roles {
wg.Add(1) wg.Add(1)
go func(name string, fn func(context.Context) error) { go func(name string, fn func(context.Context) error) {
+4
View File
@@ -30,6 +30,10 @@ DMARC_REPORTS_ENABLE=false
# INBOUND_ANTISPAM_MILTER=inet:antispam:11332 # INBOUND_ANTISPAM_MILTER=inet:antispam:11332
# INBOUND_ANTISPAM_MILTER_ACTION=accept # INBOUND_ANTISPAM_MILTER_ACTION=accept
# Days the inbound journal keeps a row (7-365) until the panel says otherwise;
# it never holds more than 10 000 rows.
# INBOUND_SPAM_LOG_RETENTION_DAYS=90
# Coarse per-client-IP limit and message size on inbound smtpd (port 25). # Coarse per-client-IP limit and message size on inbound smtpd (port 25).
# INBOUND_RATE_LIMIT_MESSAGES_PER_IP=20 # INBOUND_RATE_LIMIT_MESSAGES_PER_IP=20
# INBOUND_MESSAGE_SIZE_LIMIT=26214400 # INBOUND_MESSAGE_SIZE_LIMIT=26214400
@@ -27,3 +27,10 @@ services:
image: rspamd/rspamd:3.11 image: rspamd/rspamd:3.11
restart: unless-stopped restart: unless-stopped
# No host ports: only SelfPost talks to the milter on the compose network. # No host ports: only SelfPost talks to the milter on the compose network.
volumes:
# The verdict headers SelfPost's inbound journal reads.
- ./antispam/rspamd/milter_headers.conf:/etc/rspamd/local.d/milter_headers.conf:ro
# The allow and deny lists kept in the panel: the rules, and the files
# SelfPost writes them to.
- ./antispam/rspamd/multimap.conf:/etc/rspamd/local.d/multimap.conf:ro
- ./data/antispam:/etc/rspamd/selfpost:ro
@@ -0,0 +1,11 @@
# Mounted into the sidecar as /etc/rspamd/local.d/milter_headers.conf.
#
# rspamd writes its verdict into two headers of every message it lets through:
# the action it chose and the score with the symbols behind it. SelfPost's
# inbound journal reads them (Inbound > Log shows them next to the message),
# and the upstream mailbox server receives them with the mail.
#
# rspamd removes headers of these names that a sender wrote himself before it
# adds its own. Without this file the journal still lists accepted mail, with
# no verdict next to it.
use = ["x-spamd-result", "x-rspamd-action"];
+63
View File
@@ -0,0 +1,63 @@
# Mounted into the sidecar as /etc/rspamd/local.d/multimap.conf.
#
# The allow and deny lists kept in the panel (Inbound > Filter lists). SelfPost
# writes them as six files under data/antispam/, which the fragment mounts at
# /etc/rspamd/selfpost; rspamd rereads a file when it changes, no restart.
#
# Deny wins over allow, whichever entries match. A denied sender is refused
# before anything else is looked at (a prefilter). An allowed one is not
# accepted outright - rspamd would then let whichever of the two rules ran
# first decide - but given a weight no spam score outweighs, which a refusal
# by the deny list still overrides.
#
# A domain entry is that domain, not its subdomains.
#
# SelfPost counts how often an entry decided a message by what these rules
# leave behind: the deny message below and the names of the allow symbols.
# Change either and Inbound > Filter lists shows "never" for every entry.
SELFPOST_DENY_IP {
type = "ip";
map = "/etc/rspamd/selfpost/deny_ip.map";
prefilter = true;
action = "reject";
message = "Refused by this server's deny list";
description = "Client address on the deny list";
}
SELFPOST_DENY_FROM {
type = "from";
map = "/etc/rspamd/selfpost/deny_from.map";
prefilter = true;
action = "reject";
message = "Refused by this server's deny list";
description = "Sender address on the deny list";
}
SELFPOST_DENY_DOMAIN {
type = "from";
filter = "email:domain";
map = "/etc/rspamd/selfpost/deny_domain.map";
prefilter = true;
action = "reject";
message = "Refused by this server's deny list";
description = "Sender domain on the deny list";
}
SELFPOST_ALLOW_IP {
type = "ip";
map = "/etc/rspamd/selfpost/allow_ip.map";
score = -100.0;
description = "Client address on the allow list";
}
SELFPOST_ALLOW_FROM {
type = "from";
map = "/etc/rspamd/selfpost/allow_from.map";
score = -100.0;
description = "Sender address on the allow list";
}
SELFPOST_ALLOW_DOMAIN {
type = "from";
filter = "email:domain";
map = "/etc/rspamd/selfpost/allow_domain.map";
score = -100.0;
description = "Sender domain on the allow list";
}
+2 -1
View File
@@ -21,7 +21,7 @@
services: services:
selfpost: selfpost:
image: ghcr.io/mixeme/selfpost:2.0.0 image: ghcr.io/mixeme/selfpost:2.0.1
restart: unless-stopped restart: unless-stopped
environment: environment:
SELFPOST_HOSTNAME: "${SELFPOST_HOSTNAME:?set the mail/panel hostname, e.g. mail.example.com}" SELFPOST_HOSTNAME: "${SELFPOST_HOSTNAME:?set the mail/panel hostname, e.g. mail.example.com}"
@@ -47,6 +47,7 @@ services:
# How long the send log keeps rows before the background sweep deletes # How long the send log keeps rows before the background sweep deletes
# them — the main driver of /data growth over time. # them — the main driver of /data growth over time.
SEND_LOG_RETENTION_DAYS: "${SEND_LOG_RETENTION_DAYS:-90}" SEND_LOG_RETENTION_DAYS: "${SEND_LOG_RETENTION_DAYS:-90}"
INBOUND_SPAM_LOG_RETENTION_DAYS: "${INBOUND_SPAM_LOG_RETENTION_DAYS:-90}"
# Sliding idle timeout for the panel login session, in days. # Sliding idle timeout for the panel login session, in days.
PANEL_SESSION_IDLE_DAYS: "${PANEL_SESSION_IDLE_DAYS:-7}" PANEL_SESSION_IDLE_DAYS: "${PANEL_SESSION_IDLE_DAYS:-7}"
# Resolvers the panel's PTR/SPF/DKIM/DMARC checks query directly, so they # Resolvers the panel's PTR/SPF/DKIM/DMARC checks query directly, so they
+53 -9
View File
@@ -27,7 +27,7 @@ Managed programs ([build/supervisord.conf](../build/supervisord.conf)):
| Program | User | Priority | Role | | Program | User | Priority | Role |
|---|---|---|---| |---|---|---|---|
| `opendkim` | root → `opendkim` (`UserID` in opendkim.conf) | 100 | DKIM signing milter | | `opendkim` | root → `opendkim` (`UserID` in opendkim.conf) | 100 | DKIM signing milter |
| `panel` | panel | 200 | HTTP UI + journal-milter + log-tailer + rate-limit recalc | | `panel` | panel | 200 | HTTP UI + journal-milter + log-tailer + rate-limit recalc; with the inbound relay on also the inbound journal milter |
| `postfix` | root (wrapper) | 300 | MTA — started only after both milter sockets exist | | `postfix` | root (wrapper) | 300 | MTA — started only after both milter sockets exist |
| `postfix-reload` | root | — | On-demand `postfix reload` (autostart off) | | `postfix-reload` | root | — | On-demand `postfix reload` (autostart off) |
| `cert-reload` | root | 400 | Daily `postfix reload` for renewed TLS certs | | `cert-reload` | root | 400 | Daily `postfix reload` for renewed TLS certs |
@@ -74,10 +74,53 @@ smtp/inet` removes Debian's stock smtpd). Outbound delivery still uses the
`smtpd_tls_security_level=encrypt`. `smtpd_tls_security_level=encrypt`.
- **25/smtp inet** — when `INBOUND_RELAY_ENABLE=true` and/or - **25/smtp inet** — when `INBOUND_RELAY_ENABLE=true` and/or
`DMARC_REPORTS_ENABLE=true`. No SASL, no OpenDKIM, no journal-milter — `DMARC_REPORTS_ENABLE=true`. No SASL, no OpenDKIM, no journal-milter —
`smtpd_milters` is always overridden on this service (to the antispam milter `smtpd_milters` is always overridden on this service, because main.cf's
or to nothing), because main.cf's submission chain would otherwise let a submission chain would otherwise let a signing outage defer relayed mail and
signing outage defer relayed mail and would file inbound mail into the would file inbound mail into the outbound send log under the sender's
outbound send log under the sender's domain. domain. With the inbound relay on the chain is the optional antispam milter,
then the panel's **inbound journal milter** (unix socket
`INBOUND_JOURNAL_MILTER_SOCKET`, default
`/run/selfpost/inbound-journal.sock`, `default_action=accept`); with DMARC
ingest alone it is empty. Postfix hands a message to its milters one after
another at end-of-message, each seeing what the one before added, so the
journal milter sees only mail the filter let through and reads the filter's
verdict headers (`X-Rspamd-Action`, `X-Spamd-Result`) off it. It writes one
`inbound_spam_log` row per recipient at an inbound domain - a DMARC report
address is not one - and never refuses anything
([internal/milter/inbound.go](../internal/milter/inbound.go)). The headers
are read only while a filter is attached; refused mail never reaches it.
Refusals come from `mail.log`: this listener logs as `postfix/inbound`
(`syslog_name`), and the log tailer turns a `reject:` line of
`postfix/inbound/smtpd` (Postfix's own policy) and a `milter-reject` /
`milter-hold` / `milter-discard` line of `cleanup` (the filter's verdict,
counted only for a queue id that listener opened) into a row, dated by the
line, when the recipient is at an inbound domain
([internal/logtail/inbound.go](../internal/logtail/inbound.go)). A line
names one recipient, so a message refused at end-of-message is one row
whatever its recipient count, and a line read twice (the tailer saves its
place once a second; an unclean stop can lose that second) is two rows. The
journal is pruned every six hours: rows
older than the `inbound_spam_log_retention_days` setting, then down to
10 000.
**Filter lists** (`inbound_spam_list`): allow and deny entries for the
external filter - sender address, sender domain, client address or network -
validated and normalised by `internal/inbound` (`lists.go`) and written as
six map files under `/data/antispam/` (`ANTISPAM_DIR`), each atomically, on
every change and on every start with the relay on. SelfPost never consults
them. The sidecar mounts the directory and rspamd's multimap rules
([deploy/antispam/rspamd/multimap.conf](../deploy/antispam/rspamd/multimap.conf))
read them: a deny entry is a prefilter `reject`, an allow entry a symbol
weighing -100. Allow is deliberately not a prefilter `accept`: with two
prefilters rspamd lets the one that ran first decide, and deny then won or
lost by rule order (seen on rspamd 3.11). A refusal by the deny list
overrides any score, so deny wins whichever entries match.
How often an entry decided a message is not stored: `Lists.Usage` reads it
off the journal on each view of Filter lists - the rows a list decided (the
deny rule's reply, the allow rule's symbol, which the journal milter keeps
ahead of its six-symbol cap) matched against the entries as rspamd matches
them. When that stops being good enough is written in
[plans/inbound-antispam-panel.md](plans/inbound-antispam-panel.md) § Filter
list usage.
**Inbound relay** (`INBOUND_RELAY_ENABLE=true`): accepts only **Inbound relay** (`INBOUND_RELAY_ENABLE=true`): accepts only
`relay_domains` + `relay_recipient_maps` (`reject_unauth_destination`, `relay_domains` + `relay_recipient_maps` (`reject_unauth_destination`,
`reject_unlisted_recipient`). Maps under `/data/postfix/` (`relay_domains`, `reject_unlisted_recipient`). Maps under `/data/postfix/` (`relay_domains`,
@@ -531,16 +574,17 @@ full backups also archive the operator's `./certs` PEM files when present.
### Database ### Database
The schema is one embedded baseline, `internal/store/migrations/0001_init.sql`, The schema is an embedded baseline, `internal/store/migrations/0001_init.sql`,
applied by `store.Open` into an empty file; `PRAGMA user_version` is `1` and and the migrations after it (`0002_inbound_spam_log.sql`), applied by
`store.Open` in file order; `PRAGMA user_version` counts the files applied and
`PRAGMA application_id` is `0x53503230` ("SP20"). There is no 1.x migration `PRAGMA application_id` is `0x53503230` ("SP20"). There is no 1.x migration
chain. **2.0 starts from an empty data directory:** a database with a schema chain. **2.0 starts from an empty data directory:** a database with a schema
version but without the 2.x `application_id` — what every 1.x file looks like — version but without the 2.x `application_id` — what every 1.x file looks like —
is refused at start with `ErrForeignSchema` and left untouched, as is one whose is refused at start with `ErrForeignSchema` and left untouched, as is one whose
version is newer than the build knows. A restored 1.x backup fails the same version is newer than the build knows. A restored 1.x backup fails the same
way. Domains are carried across with a domain export, whose format is not the way. Domains are carried across with a domain export, whose format is not the
database. Later schema changes are new files (`0002_*.sql`, …); once `2.0.0` is database. A schema change is a new file; the baseline has not been edited
released the baseline is never edited. History and the per-table list: since `2.0.0`. History and the per-table list:
[schema-migrations.md](schema-migrations.md). [schema-migrations.md](schema-migrations.md).
The tables that carry the panel's model: The tables that carry the panel's model:
+1 -1
View File
@@ -78,7 +78,7 @@
<div class="box"> <div class="box">
<div class="sp-box-head"><span class="sp-no">01</span><h2>Full backup</h2><a class="sp-help" href="help.html#backup" title="Backups, encryption and restore"><i class="ti ti-help-circle"></i></a></div> <div class="sp-box-head"><span class="sp-no">01</span><h2>Full backup</h2><a class="sp-help" href="help.html#backup" title="Backups, encryption and restore"><i class="ti ti-help-circle"></i></a></div>
<div class="sp-box-body"> <div class="sp-box-body">
<p class="mb-3"><code>data/</code> (database, DKIM keys, credentials, the queue), <code>docker-compose.yml</code>, <code>.env</code> and <code>certs/</code> in one archive. Restore into an empty project directory and start the <strong>same SelfPost version</strong>. The reverse-proxy vhost is not included.</p> <p class="mb-3"><code>data/</code> (database, DKIM keys, credentials), <code>docker-compose.yml</code>, <code>.env</code> and <code>certs/</code> in one archive. The Postfix queue is left out: it cannot be copied reliably while the server is running. Restore into an empty project directory and start the <strong>same SelfPost version</strong>. The reverse-proxy vhost is not included.</p>
<form class="sp-form" method="post" action="#"> <form class="sp-form" method="post" action="#">
<div class="field"><label class="checkbox"><input type="checkbox" name="encrypt" checked> Encrypt with a password (<code>.spbk</code>)</label></div> <div class="field"><label class="checkbox"><input type="checkbox" name="encrypt" checked> Encrypt with a password (<code>.spbk</code>)</label></div>
<div class="sp-pair"> <div class="sp-pair">
+1 -1
View File
@@ -37,7 +37,7 @@ SIGNED_OUT = {"sp-signin", "sp-signin-brand", "sp-signin-form", "sp-foot"}
# Selectors allowed to set a width. Everything else gets its width from the # Selectors allowed to set a width. Everything else gets its width from the
# layout it sits in — that is the whole point of the redesign. # layout it sits in — that is the whole point of the redesign.
WIDTH_OK = (".navbar-brand img", ".sp-head .sp-lead", ".sp-head .sp-grow", ".sp-postmark", ".table td.sp-clip", WIDTH_OK = (".navbar-brand img", ".sp-head .sp-lead", ".sp-head .sp-grow", ".sp-postmark", ".table td.sp-clip",
".sp-h-icon", ".sp-signin", ".sp-timeline li::before", ".sp-perf") ".sp-h-icon", ".sp-signin", ".sp-timeline li::before", ".sp-perf", ".sp-sheet")
class Node: class Node:
@@ -114,6 +114,7 @@
<tr><td>error, fail, mismatch, bounced</td><td><span class="tag is-danger is-light">fail</span></td><td class="sp-mono sp-small">tag is-danger is-light</td></tr> <tr><td>error, fail, mismatch, bounced</td><td><span class="tag is-danger is-light">fail</span></td><td class="sp-mono sp-small">tag is-danger is-light</td></tr>
<tr><td>unknown, off, neutral counts</td><td><span class="tag is-light">off</span></td><td class="sp-mono sp-small">tag is-light</td></tr> <tr><td>unknown, off, neutral counts</td><td><span class="tag is-light">off</span></td><td class="sp-mono sp-small">tag is-light</td></tr>
<tr><td>held: quarantine · role: global</td><td><span class="tag is-primary is-light">quarantine</span></td><td class="sp-mono sp-small">tag is-primary is-light</td></tr> <tr><td>held: quarantine · role: global</td><td><span class="tag is-primary is-light">quarantine</span></td><td class="sp-mono sp-small">tag is-primary is-light</td></tr>
<tr><td>a tag that opens a <code>sheet</code></td><td><button type="button" class="tag is-danger is-light" popovertarget="kit-sheet">reject</button><div class="sp-sheet" id="kit-sheet" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-danger is-light">reject</span><h2>sheet</h2><button type="button" popovertarget="kit-sheet" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>What it holds</th><td>What does not fit the row, as name and value</td></tr><tr><th>How it closes</th><td class="sp-mono sp-small">Esc<br>a click outside<br>the cross</td></tr></tbody></table></div></div></div></div></td><td class="sp-mono sp-small">button.tag + sp-sheet (popover)</td></tr>
</tbody> </tbody>
</table></div> </table></div>
</div> </div>
+1 -1
View File
@@ -80,7 +80,7 @@
<div><dt>Last report</dt><dd>2026-09-21 06:12 UTC</dd></div> <div><dt>Last report</dt><dd>2026-09-21 06:12 UTC</dd></div>
<div><dt>This week</dt><dd>14 kept · 0 parse failures</dd></div> <div><dt>This week</dt><dd>14 kept · 0 parse failures</dd></div>
<div><dt>Hosted addresses</dt><dd class="sp-mono sp-small">dmarc-reports+&lt;domain&gt;@mail.example.org</dd></div> <div><dt>Hosted addresses</dt><dd class="sp-mono sp-small">dmarc-reports+&lt;domain&gt;@mail.example.org</dd></div>
<div><dt>Retention</dt><dd>500 reports or 90 days</dd></div> <div><dt>Retention</dt><dd>500 reports a domain or 90 days</dd></div>
</dl> </dl>
</div> </div>
</div> </div>
+6 -2
View File
@@ -97,6 +97,8 @@
<li><a href="#upstream"><i class="ti ti-arrow-forward-up"></i>Upstream</a></li> <li><a href="#upstream"><i class="ti ti-arrow-forward-up"></i>Upstream</a></li>
<li><a href="#recipients"><i class="ti ti-address-book"></i>Valid recipients</a></li> <li><a href="#recipients"><i class="ti ti-address-book"></i>Valid recipients</a></li>
<li><a href="#filter"><i class="ti ti-shield-check"></i>Spam filter</a></li> <li><a href="#filter"><i class="ti ti-shield-check"></i>Spam filter</a></li>
<li><a href="#inbound-log"><i class="ti ti-list-details"></i>Inbound log</a></li>
<li><a href="#filter-lists"><i class="ti ti-filter"></i>Filter lists</a></li>
</ul> </ul>
<p class="menu-label">Server</p> <p class="menu-label">Server</p>
<ul class="menu-list"> <ul class="menu-list">
@@ -132,12 +134,14 @@
<div class="sp-help-topic" id="upstream"><h3>Upstream</h3><p>The upstream is the server that actually holds the mailboxes. Give its host and port — used exactly as written, with no MX lookup — and choose the TLS for the hop to it: Opportunistic uses TLS when the upstream offers it and otherwise sends in the clear, Required holds the mail until a TLS session is possible, Off never uses TLS. While the upstream does not answer, Postfix keeps the mail and retries as for any other mail.</p></div> <div class="sp-help-topic" id="upstream"><h3>Upstream</h3><p>The upstream is the server that actually holds the mailboxes. Give its host and port — used exactly as written, with no MX lookup — and choose the TLS for the hop to it: Opportunistic uses TLS when the upstream offers it and otherwise sends in the clear, Required holds the mail until a TLS session is possible, Off never uses TLS. While the upstream does not answer, Postfix keeps the mail and retries as for any other mail.</p></div>
<div class="sp-help-topic" id="recipients"><h3>Valid recipients</h3><p>Choose whom this domain accepts. Listed addresses only: the relay accepts exactly the addresses you list, one per line or comma-separated, and refuses any other while the sender is still connected, so it never has to send a bounce for them. Any recipient: everything at the domain is accepted and passed on, and an address the upstream then rejects comes back as a bounce from this server to a sender who is often forged — backscatter. Prefer a list, and keep it in step with the mailboxes upstream.</p></div> <div class="sp-help-topic" id="recipients"><h3>Valid recipients</h3><p>Choose whom this domain accepts. Listed addresses only: the relay accepts exactly the addresses you list, one per line or comma-separated, and refuses any other while the sender is still connected, so it never has to send a bounce for them. Any recipient: everything at the domain is accepted and passed on, and an address the upstream then rejects comes back as a bounce from this server to a sender who is often forged — backscatter. Prefer a list, and keep it in step with the mailboxes upstream.</p></div>
<div class="sp-help-topic" id="filter"><h3>Spam filter</h3><p>The spam filter belongs to the server, not to a domain. The server's administrator attaches it in the server's <code>.env</code>, and from then on every inbound domain is screened by it before mail is handed on; the box shows on or off and has nothing to switch. Off means mail goes to the upstream unfiltered. While the filter is down, mail is either accepted unfiltered (the default) or deferred, so the sending server tries again later, as the server's setting says; the box states which.</p></div> <div class="sp-help-topic" id="filter"><h3>Spam filter</h3><p>The spam filter belongs to the server, not to a domain. The server's administrator attaches it in the server's <code>.env</code>, and from then on every inbound domain is screened by it before mail is handed on; the box shows on or off and has nothing to switch. Off means mail goes to the upstream unfiltered. While the filter is down, mail is either accepted unfiltered (the default) or deferred, so the sending server tries again later, as the server's setting says; the box states which.</p></div>
<div class="sp-help-topic" id="inbound-log"><h3>Inbound log</h3><p>One row per recipient of a message offered on port 25 for an inbound domain, with what was decided: <span class="tag is-success is-light">accept</span> — handed on to the upstream; <span class="tag is-danger is-light">reject</span> — refused for good; <span class="tag is-warning is-light">tempfail</span> — the sender was asked to come back later; <span class="tag is-primary is-light">quarantine</span> — held by the filter. <em>Why</em> names who decided. <code>rspamd</code> is the spam filter, with its action, the score against the reject threshold and the heaviest symbols; <code>postfix</code> is the relay's own rule — an unknown recipient, a sender domain that does not exist — with its reply; <code>selfpost</code> means nobody judged the message: it was relayed with no filter attached. Accept says what this server did, not what the upstream did with the message afterwards.<br><br>Accepted mail is recorded as it passes; refused mail is read from <code>mail.log</code>, so its row has no subject, and a message refused after its text was sent is one row however many recipients it had. A refusal to relay for a domain that is not an inbound one is every scanner on the internet and is not recorded. Rows are kept for the number of days set in Settings, and never more than 10 000.</p></div>
<div class="sp-help-topic" id="filter-lists"><h3>Filter lists</h3><p>Two lists for the whole server that correct the spam filter by hand. An entry is a sender address, a sender domain — that domain, not its subdomains — or a client address or network in CIDR form. A sender on the Deny list is refused outright. A sender on the Allow list is given a weight no spam score outweighs, so the filter lets it through. When entries of both lists match one message, <strong>Deny wins</strong>.<br><br>SelfPost does not apply the lists itself: it writes them as files under <code>/data/antispam/</code>, and rspamd rereads a file within seconds of a change. With the spam filter off the lists are kept and can be edited, but nothing reads them. The sender matched is the envelope sender — the one in the log — not the From line a mail program shows. Matched counts the messages in the Inbound log that the entry decided, so it reaches back as far as the log does; never means not once in that time, and such an entry is one to consider removing.</p></div>
</div> </div>
<div class="box"> <div class="box">
<div class="sp-box-head"><span class="sp-no">04</span><h2>Server</h2></div> <div class="sp-box-head"><span class="sp-no">04</span><h2>Server</h2></div>
<div class="sp-help-topic" id="backup"><h3>Backup and moving a domain</h3><p>Full backup downloads one archive of the whole instance: <code>data/</code> (the database, DKIM keys, application credentials and the queue), <code>docker-compose.yml</code>, <code>.env</code> and <code>certs/</code>. It holds secrets, so leave Encrypt with a password ticked: the file is sealed as <code>.spbk</code>, the password is not stored, and without it nothing can be recovered. To restore, unpack it into an empty project directory (decrypt a <code>.spbk</code> first with <code>selfpost-backup -decrypt</code>) and start the same SelfPost version that made it; the panel refuses data from another version. The reverse-proxy configuration and <code>mail.log</code> are not included. One domain moves on its own: Export domain on its settings page writes a <code>.json</code> (or encrypted <code>.spde</code>) with its DKIM key and its applications' passwords, and Import a domain here reads it into another instance, so the published DNS needs no change.</p></div> <div class="sp-help-topic" id="backup"><h3>Backup and moving a domain</h3><p>Full backup downloads one archive of the whole instance: <code>data/</code> (the database, DKIM keys and application credentials), <code>docker-compose.yml</code>, <code>.env</code> and <code>certs/</code>. The Postfix queue is not in it. The backup is taken while the server is running, and Postfix keeps moving messages through the queue, so a copy could hold a message twice or miss it; mail still waiting for delivery is therefore not carried over, and a restored server starts with an empty queue. To keep the queue, stop the container and archive the project directory instead. The private key in <code>certs/</code> has to be readable by the panel: give it to group 998 on the host (<code>chgrp 998</code>, <code>chmod 640</code>). Until then the panel refuses to make a backup and names the file, rather than hand over an archive without the key. It holds secrets, so leave Encrypt with a password ticked: the file is sealed as <code>.spbk</code>, the password is not stored, and without it nothing can be recovered. To restore, unpack it into an empty project directory (decrypt a <code>.spbk</code> first with <code>selfpost-backup -decrypt</code>) and start the same SelfPost version that made it; the panel refuses data from another version. The reverse-proxy configuration and <code>mail.log</code> are not included. One domain moves on its own: Export domain on its settings page writes a <code>.json</code> (or encrypted <code>.spde</code>) with its DKIM key and its applications' passwords, and Import a domain here reads it into another instance, so the published DNS needs no change.</p></div>
<div class="sp-help-topic" id="users"><h3>Users and roles</h3><p>The two roles are a reach, not a rank: every user is an administrator and signs in with a username and password. Global reaches the whole server — Overview, Server, the queue and every domain — and is the only role that adds or deletes domains and users. Domain reaches only the domains assigned to it, outbound and inbound separately (the same name on both lists is two assignments); <em>All</em> includes domains added later, and a domain user needs at least one domain or <em>All</em> on either list. The only global user cannot be demoted or deleted, and you cannot delete the account you are signed in with. A new password set here signs that user out of their other sessions.</p></div> <div class="sp-help-topic" id="users"><h3>Users and roles</h3><p>The two roles are a reach, not a rank: every user is an administrator and signs in with a username and password. Global reaches the whole server — Overview, Server, the queue and every domain — and is the only role that adds or deletes domains and users. Domain reaches only the domains assigned to it, outbound and inbound separately (the same name on both lists is two assignments); <em>All</em> includes domains added later, and a domain user needs at least one domain or <em>All</em> on either list. The only global user cannot be demoted or deleted, and you cannot delete the account you are signed in with. A new password set here signs that user out of their other sessions.</p></div>
<div class="sp-help-topic" id="settings"><h3>Settings</h3><p>Settings holds what is true of the whole instance. Log retention is how many days the Outbound log keeps its rows, from 7 to 365 and 90 by default; a sweep every six hours deletes older ones, and it is the main driver of how fast <code>/data</code> grows. It does not change the rotation of <code>mail.log</code>, which is daily with about two weeks kept. The sending rate limit shown under it is level 1, the per-client-IP ceiling set in <code>.env</code>: changing it needs a restart, and no level-2 limit can exceed it.</p></div> <div class="sp-help-topic" id="settings"><h3>Settings</h3><p>Settings holds what is true of the whole instance. Log retention is how many days the Outbound log keeps its rows, from 7 to 365 and 90 by default; a sweep every six hours deletes older ones, and it is the main driver of how fast <code>/data</code> grows. With the inbound relay on there is a second number, for the Inbound log, with the same range. It does not change the rotation of <code>mail.log</code>, which is daily with about two weeks kept. The sending rate limit shown under it is level 1, the per-client-IP ceiling set in <code>.env</code>: changing it needs a restart, and no level-2 limit can exceed it.</p></div>
</div> </div>
<div class="box"> <div class="box">
<div class="sp-box-head"><span class="sp-no">05</span><h2>Account</h2></div> <div class="sp-box-head"><span class="sp-no">05</span><h2>Account</h2></div>
@@ -74,7 +74,7 @@
</div> </div>
<div class="box"> <div class="box">
<div class="sp-box-head"><span class="sp-no">01</span><h2>Add an entry</h2><a class="sp-help" href="help.html#inbound" title="How the lists are applied"><i class="ti ti-help-circle"></i></a></div> <div class="sp-box-head"><span class="sp-no">01</span><h2>Add an entry</h2><a class="sp-help" href="help.html#filter-lists" title="How the lists are applied"><i class="ti ti-help-circle"></i></a></div>
<div class="sp-box-body"> <div class="sp-box-body">
<form class="sp-form" method="post" action="#"> <form class="sp-form" method="post" action="#">
<div class="columns"> <div class="columns">
@@ -84,7 +84,7 @@
</div> </div>
<div class="column is-3"> <div class="column is-3">
<div class="field"><label class="label" for="etype">Entry is</label> <div class="field"><label class="label" for="etype">Entry is</label>
<div class="control"><div class="select is-fullwidth"><select id="etype" name="entry_type"><option>a sender address</option><option>a sender domain</option><option>a client IP or CIDR</option></select></div></div></div> <div class="control"><div class="select is-fullwidth"><select id="etype" name="entry_type"><option>a sender address</option><option>a sender domain</option><option>a client address or network</option></select></div></div></div>
</div> </div>
<div class="column is-6"> <div class="column is-6">
<div class="field"><label class="label" for="value">Value</label> <div class="field"><label class="label" for="value">Value</label>
@@ -103,11 +103,11 @@
<div class="box"> <div class="box">
<div class="sp-box-head"><span class="sp-no">02</span><h2>Deny</h2><div class="sp-end">3 entries · wins over Allow</div></div> <div class="sp-box-head"><span class="sp-no">02</span><h2>Deny</h2><div class="sp-end">3 entries · wins over Allow</div></div>
<div class="table-container"><table class="table is-fullwidth is-hoverable"> <div class="table-container"><table class="table is-fullwidth is-hoverable">
<thead><tr><th>Value</th><th>Note</th><th>Added</th><th></th></tr></thead> <thead><tr><th>Value</th><th>Kind</th><th>Matched</th><th></th></tr></thead>
<tbody> <tbody>
<tr><td class="sp-mono sp-small">@bulk.example <span class="tag is-light">domain</span></td><td class="sp-muted">newsletter nobody subscribed to</td><td class="sp-muted sp-nowrap sp-small">2026-09-02</td><td class="sp-actions"><a class="has-text-danger" href="#">Remove</a></td></tr> <tr><td class="sp-mono sp-small">bulk.example</td><td><button type="button" class="tag is-light" popovertarget="deny-1">domain</button><div class="sp-sheet" id="deny-1" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-light">domain</span><h2>bulk.example</h2><button type="button" popovertarget="deny-1" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Note</th><td>newsletter nobody subscribed to</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-09-02</td></tr><tr><th>Matched</th><td>41 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-09-21 13:57</td></tr></tbody></table></div></div></div></div></td><td class="sp-nowrap">41</td><td class="sp-actions"><form method="post" action="#"><button type="submit" class="has-text-danger">Remove</button></form></td></tr>
<tr><td class="sp-mono sp-small">198.51.100.0/24 <span class="tag is-light">ip</span></td><td class="sp-muted">dictionary attack on lists.example.org</td><td class="sp-muted sp-nowrap sp-small">2026-08-19</td><td class="sp-actions"><a class="has-text-danger" href="#">Remove</a></td></tr> <tr><td class="sp-mono sp-small">198.51.100.0/24</td><td><button type="button" class="tag is-light" popovertarget="deny-2">ip</button><div class="sp-sheet" id="deny-2" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-light">ip</span><h2>198.51.100.0/24</h2><button type="button" popovertarget="deny-2" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Note</th><td>dictionary attack on lists.example.org</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-08-19</td></tr><tr><th>Matched</th><td>1203 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-09-21 09:12</td></tr></tbody></table></div></div></div></div></td><td class="sp-nowrap">1203</td><td class="sp-actions"><form method="post" action="#"><button type="submit" class="has-text-danger">Remove</button></form></td></tr>
<tr><td class="sp-mono sp-small">win@lottery.example <span class="tag is-light">address</span></td><td class="sp-muted">—</td><td class="sp-muted sp-nowrap sp-small">2026-07-30</td><td class="sp-actions"><a class="has-text-danger" href="#">Remove</a></td></tr> <tr><td class="sp-mono sp-small">win@lottery.example</td><td><button type="button" class="tag is-light" popovertarget="deny-3">address</button><div class="sp-sheet" id="deny-3" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-light">address</span><h2>win@lottery.example</h2><button type="button" popovertarget="deny-3" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Note</th><td>—</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-07-30</td></tr><tr><th>Matched</th><td>never in the last 90 days</td></tr></tbody></table></div></div></div></div></td><td class="sp-muted sp-nowrap">never</td><td class="sp-actions"><form method="post" action="#"><button type="submit" class="has-text-danger">Remove</button></form></td></tr>
</tbody> </tbody>
</table></div> </table></div>
</div> </div>
@@ -116,10 +116,10 @@
<div class="box"> <div class="box">
<div class="sp-box-head"><span class="sp-no">03</span><h2>Allow</h2><div class="sp-end">2 entries</div></div> <div class="sp-box-head"><span class="sp-no">03</span><h2>Allow</h2><div class="sp-end">2 entries</div></div>
<div class="table-container"><table class="table is-fullwidth is-hoverable"> <div class="table-container"><table class="table is-fullwidth is-hoverable">
<thead><tr><th>Value</th><th>Note</th><th>Added</th><th></th></tr></thead> <thead><tr><th>Value</th><th>Kind</th><th>Matched</th><th></th></tr></thead>
<tbody> <tbody>
<tr><td class="sp-mono sp-small">@partner.example <span class="tag is-light">domain</span></td><td class="sp-muted">invoices were scored as spam</td><td class="sp-muted sp-nowrap sp-small">2026-09-11</td><td class="sp-actions"><a class="has-text-danger" href="#">Remove</a></td></tr> <tr><td class="sp-mono sp-small">partner.example</td><td><button type="button" class="tag is-light" popovertarget="allow-1">domain</button><div class="sp-sheet" id="allow-1" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-light">domain</span><h2>partner.example</h2><button type="button" popovertarget="allow-1" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Note</th><td>invoices were scored as spam</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-09-11</td></tr><tr><th>Matched</th><td>7 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-09-19 16:40</td></tr></tbody></table></div></div></div></div></td><td class="sp-nowrap">7</td><td class="sp-actions"><form method="post" action="#"><button type="submit" class="has-text-danger">Remove</button></form></td></tr>
<tr><td class="sp-mono sp-small">noreply@github.example <span class="tag is-light">address</span></td><td class="sp-muted">security advisories</td><td class="sp-muted sp-nowrap sp-small">2026-06-04</td><td class="sp-actions"><a class="has-text-danger" href="#">Remove</a></td></tr> <tr><td class="sp-mono sp-small">noreply@github.example</td><td><button type="button" class="tag is-light" popovertarget="allow-2">address</button><div class="sp-sheet" id="allow-2" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-light">address</span><h2>noreply@github.example</h2><button type="button" popovertarget="allow-2" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Note</th><td>security advisories</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-06-04</td></tr><tr><th>Matched</th><td>18 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-09-21 13:30</td></tr></tbody></table></div></div></div></div></td><td class="sp-nowrap">18</td><td class="sp-actions"><form method="post" action="#"><button type="submit" class="has-text-danger">Remove</button></form></td></tr>
</tbody> </tbody>
</table></div> </table></div>
</div> </div>
@@ -130,10 +130,10 @@
<div class="sp-box-head"><span class="sp-no">04</span><h2>Where the lists go</h2><div class="sp-end"><span class="tag is-success is-light">applied</span></div></div> <div class="sp-box-head"><span class="sp-no">04</span><h2>Where the lists go</h2><div class="sp-end"><span class="tag is-success is-light">applied</span></div></div>
<div class="sp-box-body"> <div class="sp-box-body">
<dl class="sp-facts"> <dl class="sp-facts">
<div><dt>Applied by</dt><dd>rspamd, from map files</dd></div> <div><dt>Applied by</dt><dd>rspamd, from map files it rereads within seconds of a change</dd></div>
<div><dt>Files</dt><dd class="sp-mono sp-small">/data/antispam/allow.map<br>/data/antispam/deny.map</dd></div> <div><dt>Files</dt><dd><span class="sp-mono">/data/antispam/</span> — six, one for each list and kind of entry</dd></div>
<div><dt>Last written</dt><dd>2026-09-11 08:40 UTC</dd></div> <div><dt>Last written</dt><dd>2026-09-11 08:40 UTC</dd></div>
<div><dt>When the filter is off</dt><dd class="sp-muted">The lists are kept and editable, but nothing reads them until <code>INBOUND_ANTISPAM_MILTER</code> is set.</dd></div> <div><dt>When both lists match</dt><dd>Deny wins, whichever entries they are.</dd></div>
</dl> </dl>
</div> </div>
</div> </div>
+10 -9
View File
@@ -69,7 +69,7 @@
<div class="sp-grow"> <div class="sp-grow">
<p class="sp-kicker">Inbound</p> <p class="sp-kicker">Inbound</p>
<h1 class="title is-3">Inbound log</h1> <h1 class="title is-3">Inbound log</h1>
<p class="sp-lead">What port 25 decided about each message, one row per recipient. Accepts come from the inbound journal, rejects from <code>mail.log</code>; the filter's reasons appear when the spam filter is on.</p> <p class="sp-lead">What port 25 decided about each message, one row per recipient. Accepted mail is recorded as it passes, refused mail is read from <code>mail.log</code>; the filter's verdict appears when the spam filter is on. Rows older than 90 days are deleted — change that in <a href="settings.html">Settings</a>.</p>
</div> </div>
</div> </div>
@@ -83,19 +83,20 @@
<input class="input" type="date" name="to" value="2026-09-21" aria-label="To date"> <input class="input" type="date" name="to" value="2026-09-21" aria-label="To date">
<button type="submit" class="button"><i class="ti ti-filter"></i>Filter</button> <button type="submit" class="button"><i class="ti ti-filter"></i>Filter</button>
</form> </form>
<a class="sp-help" href="help.html#inbound-log" title="What the decisions mean"><i class="ti ti-help-circle"></i></a>
</div> </div>
<div class="table-container"><table class="table is-fullwidth is-hoverable"> <div class="table-container"><table class="table is-fullwidth is-hoverable">
<thead><tr><th>Time, UTC</th><th>From · client</th><th>To</th><th>Subject</th><th>Decision</th><th>Why</th><th></th></tr></thead> <thead><tr><th>Time, UTC</th><th>From</th><th>To</th><th>Subject</th><th>Decision</th><th></th></tr></thead>
<tbody> <tbody>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 14:00:18</td><td>t.berg@<wbr>uni.example<br><span class="sp-mono sp-muted sp-small">192.0.2.77</span></td><td>dev@<wbr>lists.example.org</td><td class="sp-clip">Re: [dev] patch series v3</td><td><span class="tag is-success is-light">accept</span></td><td class="sp-muted sp-small">rspamd · score 0.4</td><td class="sp-actions"><a href="in-filter-lists.html">Deny sender</a></td></tr> <tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 14:00:18</td><td>t.berg@<wbr>uni.example<br><span class="sp-mono sp-muted sp-small">192.0.2.77</span></td><td>dev@<wbr>lists.example.org</td><td class="sp-clip">Re: [dev] patch series v3</td><td><button type="button" class="tag is-success is-light" popovertarget="why-1">accept</button><div class="sp-sheet" id="why-1" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-success is-light">accept</span><h2>Why</h2><button type="button" popovertarget="why-1" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Verdict</th><td>no action, score 0.40 / 15.00</td></tr><tr><th>Subject</th><td>Re: [dev] patch series v3</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"><a href="in-filter-lists.html">Deny sender</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:59:02</td><td>invoice@<wbr>partner-billing.example<br><span class="sp-mono sp-muted sp-small">203.0.113.140</span></td><td>owner@<wbr>lists.example.org</td><td class="sp-clip">Outstanding invoice — action required</td><td><span class="tag is-primary is-light">quarantine</span></td><td class="sp-muted sp-small">rspamd · score 9.8 · PHISHING, FORGED_SENDER</td><td class="sp-actions"><a href="in-quarantine-message.html">Review</a></td></tr> <tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:59:02</td><td>invoice@<wbr>partner-billing.example<br><span class="sp-mono sp-muted sp-small">203.0.113.140</span></td><td>owner@<wbr>lists.example.org</td><td class="sp-clip">Outstanding invoice — action required</td><td><button type="button" class="tag is-success is-light" popovertarget="why-2">accept</button><div class="sp-sheet" id="why-2" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-success is-light">accept</span><h2>Why</h2><button type="button" popovertarget="why-2" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Verdict</th><td>add header, score 9.80 / 15.00</td></tr><tr><th>Symbols</th><td class="sp-mono sp-small">PHISHING(5.00)<br>FORGED_SENDER(3.00)<br>R_SPF_FAIL(1.00)</td></tr><tr><th>Subject</th><td>Outstanding invoice — action required</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"><a href="in-filter-lists.html">Deny sender</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:57:41</td><td>promo@<wbr>bulk.example<br><span class="sp-mono sp-muted sp-small">198.51.100.90</span></td><td>sales@<wbr>lists.example.org</td><td class="sp-clip sp-muted">—</td><td><span class="tag is-danger is-light">reject</span></td><td class="sp-muted sp-small">postfix · 550 5.1.1 unknown recipient</td><td class="sp-actions"><a href="in-filter-lists.html">Deny sender</a></td></tr> <tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:57:41</td><td>promo@<wbr>bulk.example<br><span class="sp-mono sp-muted sp-small">198.51.100.90</span></td><td>sales@<wbr>lists.example.org</td><td class="sp-clip sp-muted">—</td><td><button type="button" class="tag is-danger is-light" popovertarget="why-3">reject</button><div class="sp-sheet" id="why-3" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-danger is-light">reject</span><h2>Why</h2><button type="button" popovertarget="why-3" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>postfix</td></tr><tr><th>Reply</th><td class="sp-mono sp-small">550 5.1.1 &lt;sales@lists.example.org&gt;: Recipient address rejected: User unknown in relay recipient table</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:51:30</td><td>billing@<wbr>partner.example<br><span class="sp-mono sp-muted sp-small">198.51.100.12</span></td><td>owner@<wbr>lists.example.org</td><td class="sp-clip">Invoice 2026-0917</td><td><span class="tag is-danger is-light">reject</span></td><td class="sp-muted sp-small">rspamd · score 16.2 · BAYES_SPAM</td><td class="sp-actions"><a href="in-filter-lists.html">Allow sender</a></td></tr> <tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:51:30</td><td>billing@<wbr>partner.example<br><span class="sp-mono sp-muted sp-small">198.51.100.12</span></td><td>owner@<wbr>lists.example.org</td><td class="sp-clip sp-muted">—</td><td><button type="button" class="tag is-danger is-light" popovertarget="why-4">reject</button><div class="sp-sheet" id="why-4" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-danger is-light">reject</span><h2>Why</h2><button type="button" popovertarget="why-4" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Reply</th><td class="sp-mono sp-small">5.7.1 Spam message rejected</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"><a href="in-filter-lists.html">Allow sender</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:44:02</td><td>hr@<wbr>partner.example<br><span class="sp-mono sp-muted sp-small">192.0.2.31</span></td><td>jobs@<wbr>acme.io</td><td class="sp-clip">Application: backend engineer</td><td><span class="tag is-warning is-light">tempfail</span></td><td class="sp-muted sp-small">selfpost · upstream 10.0.4.12 timed out</td><td class="sp-actions"></td></tr> <tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:44:02</td><td>hr@<wbr>partner.example<br><span class="sp-mono sp-muted sp-small">192.0.2.31</span></td><td>jobs@<wbr>acme.io</td><td class="sp-clip sp-muted">—</td><td><button type="button" class="tag is-warning is-light" popovertarget="why-5">tempfail</button><div class="sp-sheet" id="why-5" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-warning is-light">tempfail</span><h2>Why</h2><button type="button" popovertarget="why-5" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Reply</th><td class="sp-mono sp-small">4.7.1 Try again later</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"><a href="in-filter-lists.html">Allow sender</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:30:55</td><td>noreply@<wbr>github.example<br><span class="sp-mono sp-muted sp-small">192.0.2.200</span></td><td>security@<wbr>lists.example.org</td><td class="sp-clip">[advisory] new report</td><td><span class="tag is-success is-light">accept</span></td><td class="sp-muted sp-small">rspamd · allow list</td><td class="sp-actions"></td></tr> <tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:30:55</td><td>noreply@<wbr>github.example<br><span class="sp-mono sp-muted sp-small">192.0.2.200</span></td><td>security@<wbr>lists.example.org</td><td class="sp-clip">[advisory] new report</td><td><button type="button" class="tag is-success is-light" popovertarget="why-6">accept</button><div class="sp-sheet" id="why-6" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-success is-light">accept</span><h2>Why</h2><button type="button" popovertarget="why-6" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Verdict</th><td>no action, score -98.10 / 15.00</td></tr><tr><th>Symbols</th><td class="sp-mono sp-small">SELFPOST_ALLOW_FROM(-100.00)<br>R_DKIM_ALLOW(-0.20)</td></tr><tr><th>Subject</th><td>[advisory] new report</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"><a href="in-filter-lists.html">Deny sender</a></td></tr>
</tbody> </tbody>
</table></div> </table></div>
<div class="sp-box-foot"><span>Page 1 of 38</span><a href="#">Older →</a><span class="sp-end">Allow / Deny sender opens <a href="in-filter-lists.html">Filter lists</a> with the entry filled in · global administrator only</span></div> <div class="sp-box-foot"><span>Page 1 of 38</span><a href="#">Older →</a><span class="sp-end">A decision opens its reason · Allow / Deny sender opens <a href="in-filter-lists.html">Filter lists</a> with the entry filled in</span></div>
</div> </div>
<footer class="sp-foot">SelfPost 1.8.0 · © 2026 Mix · <a href="#">License (AGPL-3.0)</a> · <a href="#">Source</a> · No warranty</footer> <footer class="sp-foot">SelfPost 1.8.0 · © 2026 Mix · <a href="#">License (AGPL-3.0)</a> · <a href="#">Source</a> · No warranty</footer>
@@ -123,7 +123,7 @@
"is-9": [ "is-9": [
"box[Overview]:help×3", "box[Overview]:help×3",
"box[Outbound]:help×6", "box[Outbound]:help×6",
"box[Inbound]:help×4", "box[Inbound]:help×6",
"box[Server]:help×3", "box[Server]:help×3",
"box[Account]:help" "box[Account]:help"
] ]
@@ -82,6 +82,9 @@
<div class="field"><label class="label" for="ret">Keep outbound log rows, days</label> <div class="field"><label class="label" for="ret">Keep outbound log rows, days</label>
<div class="control"><input class="input" id="ret" name="send_log_retention_days" type="number" min="7" max="365" value="30" required></div> <div class="control"><input class="input" id="ret" name="send_log_retention_days" type="number" min="7" max="365" value="30" required></div>
<p class="help">The main driver of <code>/data</code> growth. Does not affect <code>mail.log</code> rotation (fourteen daily files).</p></div> <p class="help">The main driver of <code>/data</code> growth. Does not affect <code>mail.log</code> rotation (fourteen daily files).</p></div>
<div class="field"><label class="label" for="inret">Keep inbound log rows, days</label>
<div class="control"><input class="input" id="inret" name="inbound_spam_log_retention_days" type="number" min="7" max="365" value="90" required></div>
<p class="help">The inbound log also never holds more than 10 000 rows: refused mail is logged at the rate strangers knock on port 25.</p></div>
</div> </div>
</div> </div>
</div> </div>
@@ -12,7 +12,7 @@
<div class="box"> <div class="box">
<div class="sp-box-head"><span class="sp-no">01</span><h2>Full backup</h2><a class="sp-help" href="help.html#backup" title="Backups, encryption and restore"><i class="ti ti-help-circle"></i></a></div> <div class="sp-box-head"><span class="sp-no">01</span><h2>Full backup</h2><a class="sp-help" href="help.html#backup" title="Backups, encryption and restore"><i class="ti ti-help-circle"></i></a></div>
<div class="sp-box-body"> <div class="sp-box-body">
<p class="mb-3"><code>data/</code> (database, DKIM keys, credentials, the queue), <code>docker-compose.yml</code>, <code>.env</code> and <code>certs/</code> in one archive. Restore into an empty project directory and start the <strong>same SelfPost version</strong>. The reverse-proxy vhost is not included.</p> <p class="mb-3"><code>data/</code> (database, DKIM keys, credentials), <code>docker-compose.yml</code>, <code>.env</code> and <code>certs/</code> in one archive. The Postfix queue is left out: it cannot be copied reliably while the server is running. Restore into an empty project directory and start the <strong>same SelfPost version</strong>. The reverse-proxy vhost is not included.</p>
<form class="sp-form" method="post" action="#"> <form class="sp-form" method="post" action="#">
<div class="field"><label class="checkbox"><input type="checkbox" name="encrypt" checked> Encrypt with a password (<code>.spbk</code>)</label></div> <div class="field"><label class="checkbox"><input type="checkbox" name="encrypt" checked> Encrypt with a password (<code>.spbk</code>)</label></div>
<div class="sp-pair"> <div class="sp-pair">
@@ -48,6 +48,7 @@
<tr><td>error, fail, mismatch, bounced</td><td><span class="tag is-danger is-light">fail</span></td><td class="sp-mono sp-small">tag is-danger is-light</td></tr> <tr><td>error, fail, mismatch, bounced</td><td><span class="tag is-danger is-light">fail</span></td><td class="sp-mono sp-small">tag is-danger is-light</td></tr>
<tr><td>unknown, off, neutral counts</td><td><span class="tag is-light">off</span></td><td class="sp-mono sp-small">tag is-light</td></tr> <tr><td>unknown, off, neutral counts</td><td><span class="tag is-light">off</span></td><td class="sp-mono sp-small">tag is-light</td></tr>
<tr><td>held: quarantine · role: global</td><td><span class="tag is-primary is-light">quarantine</span></td><td class="sp-mono sp-small">tag is-primary is-light</td></tr> <tr><td>held: quarantine · role: global</td><td><span class="tag is-primary is-light">quarantine</span></td><td class="sp-mono sp-small">tag is-primary is-light</td></tr>
<tr><td>a tag that opens a <code>sheet</code></td><td><button type="button" class="tag is-danger is-light" popovertarget="kit-sheet">reject</button><div class="sp-sheet" id="kit-sheet" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-danger is-light">reject</span><h2>sheet</h2><button type="button" popovertarget="kit-sheet" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>What it holds</th><td>What does not fit the row, as name and value</td></tr><tr><th>How it closes</th><td class="sp-mono sp-small">Esc<br>a click outside<br>the cross</td></tr></tbody></table></div></div></div></div></td><td class="sp-mono sp-small">button.tag + sp-sheet (popover)</td></tr>
</tbody> </tbody>
</table></div> </table></div>
</div> </div>
@@ -14,7 +14,7 @@
<div><dt>Last report</dt><dd>2026-09-21 06:12 UTC</dd></div> <div><dt>Last report</dt><dd>2026-09-21 06:12 UTC</dd></div>
<div><dt>This week</dt><dd>14 kept · 0 parse failures</dd></div> <div><dt>This week</dt><dd>14 kept · 0 parse failures</dd></div>
<div><dt>Hosted addresses</dt><dd class="sp-mono sp-small">dmarc-reports+&lt;domain&gt;@mail.example.org</dd></div> <div><dt>Hosted addresses</dt><dd class="sp-mono sp-small">dmarc-reports+&lt;domain&gt;@mail.example.org</dd></div>
<div><dt>Retention</dt><dd>500 reports or 90 days</dd></div> <div><dt>Retention</dt><dd>500 reports a domain or 90 days</dd></div>
</dl> </dl>
</div> </div>
</div> </div>
@@ -31,6 +31,8 @@
<li><a href="#upstream"><i class="ti ti-arrow-forward-up"></i>Upstream</a></li> <li><a href="#upstream"><i class="ti ti-arrow-forward-up"></i>Upstream</a></li>
<li><a href="#recipients"><i class="ti ti-address-book"></i>Valid recipients</a></li> <li><a href="#recipients"><i class="ti ti-address-book"></i>Valid recipients</a></li>
<li><a href="#filter"><i class="ti ti-shield-check"></i>Spam filter</a></li> <li><a href="#filter"><i class="ti ti-shield-check"></i>Spam filter</a></li>
<li><a href="#inbound-log"><i class="ti ti-list-details"></i>Inbound log</a></li>
<li><a href="#filter-lists"><i class="ti ti-filter"></i>Filter lists</a></li>
</ul> </ul>
<p class="menu-label">Server</p> <p class="menu-label">Server</p>
<ul class="menu-list"> <ul class="menu-list">
@@ -66,12 +68,14 @@
<div class="sp-help-topic" id="upstream"><h3>Upstream</h3><p>The upstream is the server that actually holds the mailboxes. Give its host and port — used exactly as written, with no MX lookup — and choose the TLS for the hop to it: Opportunistic uses TLS when the upstream offers it and otherwise sends in the clear, Required holds the mail until a TLS session is possible, Off never uses TLS. While the upstream does not answer, Postfix keeps the mail and retries as for any other mail.</p></div> <div class="sp-help-topic" id="upstream"><h3>Upstream</h3><p>The upstream is the server that actually holds the mailboxes. Give its host and port — used exactly as written, with no MX lookup — and choose the TLS for the hop to it: Opportunistic uses TLS when the upstream offers it and otherwise sends in the clear, Required holds the mail until a TLS session is possible, Off never uses TLS. While the upstream does not answer, Postfix keeps the mail and retries as for any other mail.</p></div>
<div class="sp-help-topic" id="recipients"><h3>Valid recipients</h3><p>Choose whom this domain accepts. Listed addresses only: the relay accepts exactly the addresses you list, one per line or comma-separated, and refuses any other while the sender is still connected, so it never has to send a bounce for them. Any recipient: everything at the domain is accepted and passed on, and an address the upstream then rejects comes back as a bounce from this server to a sender who is often forged — backscatter. Prefer a list, and keep it in step with the mailboxes upstream.</p></div> <div class="sp-help-topic" id="recipients"><h3>Valid recipients</h3><p>Choose whom this domain accepts. Listed addresses only: the relay accepts exactly the addresses you list, one per line or comma-separated, and refuses any other while the sender is still connected, so it never has to send a bounce for them. Any recipient: everything at the domain is accepted and passed on, and an address the upstream then rejects comes back as a bounce from this server to a sender who is often forged — backscatter. Prefer a list, and keep it in step with the mailboxes upstream.</p></div>
<div class="sp-help-topic" id="filter"><h3>Spam filter</h3><p>The spam filter belongs to the server, not to a domain. The server's administrator attaches it in the server's <code>.env</code>, and from then on every inbound domain is screened by it before mail is handed on; the box shows on or off and has nothing to switch. Off means mail goes to the upstream unfiltered. While the filter is down, mail is either accepted unfiltered (the default) or deferred, so the sending server tries again later, as the server's setting says; the box states which.</p></div> <div class="sp-help-topic" id="filter"><h3>Spam filter</h3><p>The spam filter belongs to the server, not to a domain. The server's administrator attaches it in the server's <code>.env</code>, and from then on every inbound domain is screened by it before mail is handed on; the box shows on or off and has nothing to switch. Off means mail goes to the upstream unfiltered. While the filter is down, mail is either accepted unfiltered (the default) or deferred, so the sending server tries again later, as the server's setting says; the box states which.</p></div>
<div class="sp-help-topic" id="inbound-log"><h3>Inbound log</h3><p>One row per recipient of a message offered on port 25 for an inbound domain, with what was decided: <span class="tag is-success is-light">accept</span> — handed on to the upstream; <span class="tag is-danger is-light">reject</span> — refused for good; <span class="tag is-warning is-light">tempfail</span> — the sender was asked to come back later; <span class="tag is-primary is-light">quarantine</span> — held by the filter. <em>Why</em> names who decided. <code>rspamd</code> is the spam filter, with its action, the score against the reject threshold and the heaviest symbols; <code>postfix</code> is the relay's own rule — an unknown recipient, a sender domain that does not exist — with its reply; <code>selfpost</code> means nobody judged the message: it was relayed with no filter attached. Accept says what this server did, not what the upstream did with the message afterwards.<br><br>Accepted mail is recorded as it passes; refused mail is read from <code>mail.log</code>, so its row has no subject, and a message refused after its text was sent is one row however many recipients it had. A refusal to relay for a domain that is not an inbound one is every scanner on the internet and is not recorded. Rows are kept for the number of days set in Settings, and never more than 10 000.</p></div>
<div class="sp-help-topic" id="filter-lists"><h3>Filter lists</h3><p>Two lists for the whole server that correct the spam filter by hand. An entry is a sender address, a sender domain — that domain, not its subdomains — or a client address or network in CIDR form. A sender on the Deny list is refused outright. A sender on the Allow list is given a weight no spam score outweighs, so the filter lets it through. When entries of both lists match one message, <strong>Deny wins</strong>.<br><br>SelfPost does not apply the lists itself: it writes them as files under <code>/data/antispam/</code>, and rspamd rereads a file within seconds of a change. With the spam filter off the lists are kept and can be edited, but nothing reads them. The sender matched is the envelope sender — the one in the log — not the From line a mail program shows. Matched counts the messages in the Inbound log that the entry decided, so it reaches back as far as the log does; never means not once in that time, and such an entry is one to consider removing.</p></div>
</div> </div>
<div class="box"> <div class="box">
<div class="sp-box-head"><span class="sp-no">04</span><h2>Server</h2></div> <div class="sp-box-head"><span class="sp-no">04</span><h2>Server</h2></div>
<div class="sp-help-topic" id="backup"><h3>Backup and moving a domain</h3><p>Full backup downloads one archive of the whole instance: <code>data/</code> (the database, DKIM keys, application credentials and the queue), <code>docker-compose.yml</code>, <code>.env</code> and <code>certs/</code>. It holds secrets, so leave Encrypt with a password ticked: the file is sealed as <code>.spbk</code>, the password is not stored, and without it nothing can be recovered. To restore, unpack it into an empty project directory (decrypt a <code>.spbk</code> first with <code>selfpost-backup -decrypt</code>) and start the same SelfPost version that made it; the panel refuses data from another version. The reverse-proxy configuration and <code>mail.log</code> are not included. One domain moves on its own: Export domain on its settings page writes a <code>.json</code> (or encrypted <code>.spde</code>) with its DKIM key and its applications' passwords, and Import a domain here reads it into another instance, so the published DNS needs no change.</p></div> <div class="sp-help-topic" id="backup"><h3>Backup and moving a domain</h3><p>Full backup downloads one archive of the whole instance: <code>data/</code> (the database, DKIM keys and application credentials), <code>docker-compose.yml</code>, <code>.env</code> and <code>certs/</code>. The Postfix queue is not in it. The backup is taken while the server is running, and Postfix keeps moving messages through the queue, so a copy could hold a message twice or miss it; mail still waiting for delivery is therefore not carried over, and a restored server starts with an empty queue. To keep the queue, stop the container and archive the project directory instead. The private key in <code>certs/</code> has to be readable by the panel: give it to group 998 on the host (<code>chgrp 998</code>, <code>chmod 640</code>). Until then the panel refuses to make a backup and names the file, rather than hand over an archive without the key. It holds secrets, so leave Encrypt with a password ticked: the file is sealed as <code>.spbk</code>, the password is not stored, and without it nothing can be recovered. To restore, unpack it into an empty project directory (decrypt a <code>.spbk</code> first with <code>selfpost-backup -decrypt</code>) and start the same SelfPost version that made it; the panel refuses data from another version. The reverse-proxy configuration and <code>mail.log</code> are not included. One domain moves on its own: Export domain on its settings page writes a <code>.json</code> (or encrypted <code>.spde</code>) with its DKIM key and its applications' passwords, and Import a domain here reads it into another instance, so the published DNS needs no change.</p></div>
<div class="sp-help-topic" id="users"><h3>Users and roles</h3><p>The two roles are a reach, not a rank: every user is an administrator and signs in with a username and password. Global reaches the whole server — Overview, Server, the queue and every domain — and is the only role that adds or deletes domains and users. Domain reaches only the domains assigned to it, outbound and inbound separately (the same name on both lists is two assignments); <em>All</em> includes domains added later, and a domain user needs at least one domain or <em>All</em> on either list. The only global user cannot be demoted or deleted, and you cannot delete the account you are signed in with. A new password set here signs that user out of their other sessions.</p></div> <div class="sp-help-topic" id="users"><h3>Users and roles</h3><p>The two roles are a reach, not a rank: every user is an administrator and signs in with a username and password. Global reaches the whole server — Overview, Server, the queue and every domain — and is the only role that adds or deletes domains and users. Domain reaches only the domains assigned to it, outbound and inbound separately (the same name on both lists is two assignments); <em>All</em> includes domains added later, and a domain user needs at least one domain or <em>All</em> on either list. The only global user cannot be demoted or deleted, and you cannot delete the account you are signed in with. A new password set here signs that user out of their other sessions.</p></div>
<div class="sp-help-topic" id="settings"><h3>Settings</h3><p>Settings holds what is true of the whole instance. Log retention is how many days the Outbound log keeps its rows, from 7 to 365 and 90 by default; a sweep every six hours deletes older ones, and it is the main driver of how fast <code>/data</code> grows. It does not change the rotation of <code>mail.log</code>, which is daily with about two weeks kept. The sending rate limit shown under it is level 1, the per-client-IP ceiling set in <code>.env</code>: changing it needs a restart, and no level-2 limit can exceed it.</p></div> <div class="sp-help-topic" id="settings"><h3>Settings</h3><p>Settings holds what is true of the whole instance. Log retention is how many days the Outbound log keeps its rows, from 7 to 365 and 90 by default; a sweep every six hours deletes older ones, and it is the main driver of how fast <code>/data</code> grows. With the inbound relay on there is a second number, for the Inbound log, with the same range. It does not change the rotation of <code>mail.log</code>, which is daily with about two weeks kept. The sending rate limit shown under it is level 1, the per-client-IP ceiling set in <code>.env</code>: changing it needs a restart, and no level-2 limit can exceed it.</p></div>
</div> </div>
<div class="box"> <div class="box">
<div class="sp-box-head"><span class="sp-no">05</span><h2>Account</h2></div> <div class="sp-box-head"><span class="sp-no">05</span><h2>Account</h2></div>
@@ -8,7 +8,7 @@
</div> </div>
<div class="box"> <div class="box">
<div class="sp-box-head"><span class="sp-no">01</span><h2>Add an entry</h2><a class="sp-help" href="help.html#inbound" title="How the lists are applied"><i class="ti ti-help-circle"></i></a></div> <div class="sp-box-head"><span class="sp-no">01</span><h2>Add an entry</h2><a class="sp-help" href="help.html#filter-lists" title="How the lists are applied"><i class="ti ti-help-circle"></i></a></div>
<div class="sp-box-body"> <div class="sp-box-body">
<form class="sp-form" method="post" action="#"> <form class="sp-form" method="post" action="#">
<div class="columns"> <div class="columns">
@@ -18,7 +18,7 @@
</div> </div>
<div class="column is-3"> <div class="column is-3">
<div class="field"><label class="label" for="etype">Entry is</label> <div class="field"><label class="label" for="etype">Entry is</label>
<div class="control"><div class="select is-fullwidth"><select id="etype" name="entry_type"><option>a sender address</option><option>a sender domain</option><option>a client IP or CIDR</option></select></div></div></div> <div class="control"><div class="select is-fullwidth"><select id="etype" name="entry_type"><option>a sender address</option><option>a sender domain</option><option>a client address or network</option></select></div></div></div>
</div> </div>
<div class="column is-6"> <div class="column is-6">
<div class="field"><label class="label" for="value">Value</label> <div class="field"><label class="label" for="value">Value</label>
@@ -37,11 +37,11 @@
<div class="box"> <div class="box">
<div class="sp-box-head"><span class="sp-no">02</span><h2>Deny</h2><div class="sp-end">3 entries · wins over Allow</div></div> <div class="sp-box-head"><span class="sp-no">02</span><h2>Deny</h2><div class="sp-end">3 entries · wins over Allow</div></div>
<div class="table-container"><table class="table is-fullwidth is-hoverable"> <div class="table-container"><table class="table is-fullwidth is-hoverable">
<thead><tr><th>Value</th><th>Note</th><th>Added</th><th></th></tr></thead> <thead><tr><th>Value</th><th>Kind</th><th>Matched</th><th></th></tr></thead>
<tbody> <tbody>
<tr><td class="sp-mono sp-small">@bulk.example <span class="tag is-light">domain</span></td><td class="sp-muted">newsletter nobody subscribed to</td><td class="sp-muted sp-nowrap sp-small">2026-09-02</td><td class="sp-actions"><a class="has-text-danger" href="#">Remove</a></td></tr> <tr><td class="sp-mono sp-small">bulk.example</td><td><button type="button" class="tag is-light" popovertarget="deny-1">domain</button><div class="sp-sheet" id="deny-1" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-light">domain</span><h2>bulk.example</h2><button type="button" popovertarget="deny-1" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Note</th><td>newsletter nobody subscribed to</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-09-02</td></tr><tr><th>Matched</th><td>41 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-09-21 13:57</td></tr></tbody></table></div></div></div></div></td><td class="sp-nowrap">41</td><td class="sp-actions"><form method="post" action="#"><button type="submit" class="has-text-danger">Remove</button></form></td></tr>
<tr><td class="sp-mono sp-small">198.51.100.0/24 <span class="tag is-light">ip</span></td><td class="sp-muted">dictionary attack on lists.example.org</td><td class="sp-muted sp-nowrap sp-small">2026-08-19</td><td class="sp-actions"><a class="has-text-danger" href="#">Remove</a></td></tr> <tr><td class="sp-mono sp-small">198.51.100.0/24</td><td><button type="button" class="tag is-light" popovertarget="deny-2">ip</button><div class="sp-sheet" id="deny-2" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-light">ip</span><h2>198.51.100.0/24</h2><button type="button" popovertarget="deny-2" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Note</th><td>dictionary attack on lists.example.org</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-08-19</td></tr><tr><th>Matched</th><td>1203 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-09-21 09:12</td></tr></tbody></table></div></div></div></div></td><td class="sp-nowrap">1203</td><td class="sp-actions"><form method="post" action="#"><button type="submit" class="has-text-danger">Remove</button></form></td></tr>
<tr><td class="sp-mono sp-small">win@lottery.example <span class="tag is-light">address</span></td><td class="sp-muted">—</td><td class="sp-muted sp-nowrap sp-small">2026-07-30</td><td class="sp-actions"><a class="has-text-danger" href="#">Remove</a></td></tr> <tr><td class="sp-mono sp-small">win@lottery.example</td><td><button type="button" class="tag is-light" popovertarget="deny-3">address</button><div class="sp-sheet" id="deny-3" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-light">address</span><h2>win@lottery.example</h2><button type="button" popovertarget="deny-3" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Note</th><td>—</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-07-30</td></tr><tr><th>Matched</th><td>never in the last 90 days</td></tr></tbody></table></div></div></div></div></td><td class="sp-muted sp-nowrap">never</td><td class="sp-actions"><form method="post" action="#"><button type="submit" class="has-text-danger">Remove</button></form></td></tr>
</tbody> </tbody>
</table></div> </table></div>
</div> </div>
@@ -50,10 +50,10 @@
<div class="box"> <div class="box">
<div class="sp-box-head"><span class="sp-no">03</span><h2>Allow</h2><div class="sp-end">2 entries</div></div> <div class="sp-box-head"><span class="sp-no">03</span><h2>Allow</h2><div class="sp-end">2 entries</div></div>
<div class="table-container"><table class="table is-fullwidth is-hoverable"> <div class="table-container"><table class="table is-fullwidth is-hoverable">
<thead><tr><th>Value</th><th>Note</th><th>Added</th><th></th></tr></thead> <thead><tr><th>Value</th><th>Kind</th><th>Matched</th><th></th></tr></thead>
<tbody> <tbody>
<tr><td class="sp-mono sp-small">@partner.example <span class="tag is-light">domain</span></td><td class="sp-muted">invoices were scored as spam</td><td class="sp-muted sp-nowrap sp-small">2026-09-11</td><td class="sp-actions"><a class="has-text-danger" href="#">Remove</a></td></tr> <tr><td class="sp-mono sp-small">partner.example</td><td><button type="button" class="tag is-light" popovertarget="allow-1">domain</button><div class="sp-sheet" id="allow-1" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-light">domain</span><h2>partner.example</h2><button type="button" popovertarget="allow-1" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Note</th><td>invoices were scored as spam</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-09-11</td></tr><tr><th>Matched</th><td>7 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-09-19 16:40</td></tr></tbody></table></div></div></div></div></td><td class="sp-nowrap">7</td><td class="sp-actions"><form method="post" action="#"><button type="submit" class="has-text-danger">Remove</button></form></td></tr>
<tr><td class="sp-mono sp-small">noreply@github.example <span class="tag is-light">address</span></td><td class="sp-muted">security advisories</td><td class="sp-muted sp-nowrap sp-small">2026-06-04</td><td class="sp-actions"><a class="has-text-danger" href="#">Remove</a></td></tr> <tr><td class="sp-mono sp-small">noreply@github.example</td><td><button type="button" class="tag is-light" popovertarget="allow-2">address</button><div class="sp-sheet" id="allow-2" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-light">address</span><h2>noreply@github.example</h2><button type="button" popovertarget="allow-2" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Note</th><td>security advisories</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-06-04</td></tr><tr><th>Matched</th><td>18 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-09-21 13:30</td></tr></tbody></table></div></div></div></div></td><td class="sp-nowrap">18</td><td class="sp-actions"><form method="post" action="#"><button type="submit" class="has-text-danger">Remove</button></form></td></tr>
</tbody> </tbody>
</table></div> </table></div>
</div> </div>
@@ -64,10 +64,10 @@
<div class="sp-box-head"><span class="sp-no">04</span><h2>Where the lists go</h2><div class="sp-end"><span class="tag is-success is-light">applied</span></div></div> <div class="sp-box-head"><span class="sp-no">04</span><h2>Where the lists go</h2><div class="sp-end"><span class="tag is-success is-light">applied</span></div></div>
<div class="sp-box-body"> <div class="sp-box-body">
<dl class="sp-facts"> <dl class="sp-facts">
<div><dt>Applied by</dt><dd>rspamd, from map files</dd></div> <div><dt>Applied by</dt><dd>rspamd, from map files it rereads within seconds of a change</dd></div>
<div><dt>Files</dt><dd class="sp-mono sp-small">/data/antispam/allow.map<br>/data/antispam/deny.map</dd></div> <div><dt>Files</dt><dd><span class="sp-mono">/data/antispam/</span> — six, one for each list and kind of entry</dd></div>
<div><dt>Last written</dt><dd>2026-09-11 08:40 UTC</dd></div> <div><dt>Last written</dt><dd>2026-09-11 08:40 UTC</dd></div>
<div><dt>When the filter is off</dt><dd class="sp-muted">The lists are kept and editable, but nothing reads them until <code>INBOUND_ANTISPAM_MILTER</code> is set.</dd></div> <div><dt>When both lists match</dt><dd>Deny wins, whichever entries they are.</dd></div>
</dl> </dl>
</div> </div>
</div> </div>
@@ -3,7 +3,7 @@
<div class="sp-grow"> <div class="sp-grow">
<p class="sp-kicker">Inbound</p> <p class="sp-kicker">Inbound</p>
<h1 class="title is-3">Inbound log</h1> <h1 class="title is-3">Inbound log</h1>
<p class="sp-lead">What port 25 decided about each message, one row per recipient. Accepts come from the inbound journal, rejects from <code>mail.log</code>; the filter's reasons appear when the spam filter is on.</p> <p class="sp-lead">What port 25 decided about each message, one row per recipient. Accepted mail is recorded as it passes, refused mail is read from <code>mail.log</code>; the filter's verdict appears when the spam filter is on. Rows older than 90 days are deleted — change that in <a href="settings.html">Settings</a>.</p>
</div> </div>
</div> </div>
@@ -17,17 +17,18 @@
<input class="input" type="date" name="to" value="2026-09-21" aria-label="To date"> <input class="input" type="date" name="to" value="2026-09-21" aria-label="To date">
<button type="submit" class="button"><i class="ti ti-filter"></i>Filter</button> <button type="submit" class="button"><i class="ti ti-filter"></i>Filter</button>
</form> </form>
<a class="sp-help" href="help.html#inbound-log" title="What the decisions mean"><i class="ti ti-help-circle"></i></a>
</div> </div>
<div class="table-container"><table class="table is-fullwidth is-hoverable"> <div class="table-container"><table class="table is-fullwidth is-hoverable">
<thead><tr><th>Time, UTC</th><th>From · client</th><th>To</th><th>Subject</th><th>Decision</th><th>Why</th><th></th></tr></thead> <thead><tr><th>Time, UTC</th><th>From</th><th>To</th><th>Subject</th><th>Decision</th><th></th></tr></thead>
<tbody> <tbody>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 14:00:18</td><td>t.berg@<wbr>uni.example<br><span class="sp-mono sp-muted sp-small">192.0.2.77</span></td><td>dev@<wbr>lists.example.org</td><td class="sp-clip">Re: [dev] patch series v3</td><td><span class="tag is-success is-light">accept</span></td><td class="sp-muted sp-small">rspamd · score 0.4</td><td class="sp-actions"><a href="in-filter-lists.html">Deny sender</a></td></tr> <tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 14:00:18</td><td>t.berg@<wbr>uni.example<br><span class="sp-mono sp-muted sp-small">192.0.2.77</span></td><td>dev@<wbr>lists.example.org</td><td class="sp-clip">Re: [dev] patch series v3</td><td><button type="button" class="tag is-success is-light" popovertarget="why-1">accept</button><div class="sp-sheet" id="why-1" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-success is-light">accept</span><h2>Why</h2><button type="button" popovertarget="why-1" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Verdict</th><td>no action, score 0.40 / 15.00</td></tr><tr><th>Subject</th><td>Re: [dev] patch series v3</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"><a href="in-filter-lists.html">Deny sender</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:59:02</td><td>invoice@<wbr>partner-billing.example<br><span class="sp-mono sp-muted sp-small">203.0.113.140</span></td><td>owner@<wbr>lists.example.org</td><td class="sp-clip">Outstanding invoice — action required</td><td><span class="tag is-primary is-light">quarantine</span></td><td class="sp-muted sp-small">rspamd · score 9.8 · PHISHING, FORGED_SENDER</td><td class="sp-actions"><a href="in-quarantine-message.html">Review</a></td></tr> <tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:59:02</td><td>invoice@<wbr>partner-billing.example<br><span class="sp-mono sp-muted sp-small">203.0.113.140</span></td><td>owner@<wbr>lists.example.org</td><td class="sp-clip">Outstanding invoice — action required</td><td><button type="button" class="tag is-success is-light" popovertarget="why-2">accept</button><div class="sp-sheet" id="why-2" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-success is-light">accept</span><h2>Why</h2><button type="button" popovertarget="why-2" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Verdict</th><td>add header, score 9.80 / 15.00</td></tr><tr><th>Symbols</th><td class="sp-mono sp-small">PHISHING(5.00)<br>FORGED_SENDER(3.00)<br>R_SPF_FAIL(1.00)</td></tr><tr><th>Subject</th><td>Outstanding invoice — action required</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"><a href="in-filter-lists.html">Deny sender</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:57:41</td><td>promo@<wbr>bulk.example<br><span class="sp-mono sp-muted sp-small">198.51.100.90</span></td><td>sales@<wbr>lists.example.org</td><td class="sp-clip sp-muted">—</td><td><span class="tag is-danger is-light">reject</span></td><td class="sp-muted sp-small">postfix · 550 5.1.1 unknown recipient</td><td class="sp-actions"><a href="in-filter-lists.html">Deny sender</a></td></tr> <tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:57:41</td><td>promo@<wbr>bulk.example<br><span class="sp-mono sp-muted sp-small">198.51.100.90</span></td><td>sales@<wbr>lists.example.org</td><td class="sp-clip sp-muted">—</td><td><button type="button" class="tag is-danger is-light" popovertarget="why-3">reject</button><div class="sp-sheet" id="why-3" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-danger is-light">reject</span><h2>Why</h2><button type="button" popovertarget="why-3" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>postfix</td></tr><tr><th>Reply</th><td class="sp-mono sp-small">550 5.1.1 &lt;sales@lists.example.org&gt;: Recipient address rejected: User unknown in relay recipient table</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:51:30</td><td>billing@<wbr>partner.example<br><span class="sp-mono sp-muted sp-small">198.51.100.12</span></td><td>owner@<wbr>lists.example.org</td><td class="sp-clip">Invoice 2026-0917</td><td><span class="tag is-danger is-light">reject</span></td><td class="sp-muted sp-small">rspamd · score 16.2 · BAYES_SPAM</td><td class="sp-actions"><a href="in-filter-lists.html">Allow sender</a></td></tr> <tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:51:30</td><td>billing@<wbr>partner.example<br><span class="sp-mono sp-muted sp-small">198.51.100.12</span></td><td>owner@<wbr>lists.example.org</td><td class="sp-clip sp-muted">—</td><td><button type="button" class="tag is-danger is-light" popovertarget="why-4">reject</button><div class="sp-sheet" id="why-4" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-danger is-light">reject</span><h2>Why</h2><button type="button" popovertarget="why-4" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Reply</th><td class="sp-mono sp-small">5.7.1 Spam message rejected</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"><a href="in-filter-lists.html">Allow sender</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:44:02</td><td>hr@<wbr>partner.example<br><span class="sp-mono sp-muted sp-small">192.0.2.31</span></td><td>jobs@<wbr>acme.io</td><td class="sp-clip">Application: backend engineer</td><td><span class="tag is-warning is-light">tempfail</span></td><td class="sp-muted sp-small">selfpost · upstream 10.0.4.12 timed out</td><td class="sp-actions"></td></tr> <tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:44:02</td><td>hr@<wbr>partner.example<br><span class="sp-mono sp-muted sp-small">192.0.2.31</span></td><td>jobs@<wbr>acme.io</td><td class="sp-clip sp-muted">—</td><td><button type="button" class="tag is-warning is-light" popovertarget="why-5">tempfail</button><div class="sp-sheet" id="why-5" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-warning is-light">tempfail</span><h2>Why</h2><button type="button" popovertarget="why-5" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Reply</th><td class="sp-mono sp-small">4.7.1 Try again later</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"><a href="in-filter-lists.html">Allow sender</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:30:55</td><td>noreply@<wbr>github.example<br><span class="sp-mono sp-muted sp-small">192.0.2.200</span></td><td>security@<wbr>lists.example.org</td><td class="sp-clip">[advisory] new report</td><td><span class="tag is-success is-light">accept</span></td><td class="sp-muted sp-small">rspamd · allow list</td><td class="sp-actions"></td></tr> <tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:30:55</td><td>noreply@<wbr>github.example<br><span class="sp-mono sp-muted sp-small">192.0.2.200</span></td><td>security@<wbr>lists.example.org</td><td class="sp-clip">[advisory] new report</td><td><button type="button" class="tag is-success is-light" popovertarget="why-6">accept</button><div class="sp-sheet" id="why-6" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-success is-light">accept</span><h2>Why</h2><button type="button" popovertarget="why-6" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Verdict</th><td>no action, score -98.10 / 15.00</td></tr><tr><th>Symbols</th><td class="sp-mono sp-small">SELFPOST_ALLOW_FROM(-100.00)<br>R_DKIM_ALLOW(-0.20)</td></tr><tr><th>Subject</th><td>[advisory] new report</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"><a href="in-filter-lists.html">Deny sender</a></td></tr>
</tbody> </tbody>
</table></div> </table></div>
<div class="sp-box-foot"><span>Page 1 of 38</span><a href="#">Older →</a><span class="sp-end">Allow / Deny sender opens <a href="in-filter-lists.html">Filter lists</a> with the entry filled in · global administrator only</span></div> <div class="sp-box-foot"><span>Page 1 of 38</span><a href="#">Older →</a><span class="sp-end">A decision opens its reason · Allow / Deny sender opens <a href="in-filter-lists.html">Filter lists</a> with the entry filled in</span></div>
</div> </div>
@@ -16,6 +16,9 @@
<div class="field"><label class="label" for="ret">Keep outbound log rows, days</label> <div class="field"><label class="label" for="ret">Keep outbound log rows, days</label>
<div class="control"><input class="input" id="ret" name="send_log_retention_days" type="number" min="7" max="365" value="30" required></div> <div class="control"><input class="input" id="ret" name="send_log_retention_days" type="number" min="7" max="365" value="30" required></div>
<p class="help">The main driver of <code>/data</code> growth. Does not affect <code>mail.log</code> rotation (fourteen daily files).</p></div> <p class="help">The main driver of <code>/data</code> growth. Does not affect <code>mail.log</code> rotation (fourteen daily files).</p></div>
<div class="field"><label class="label" for="inret">Keep inbound log rows, days</label>
<div class="control"><input class="input" id="inret" name="inbound_spam_log_retention_days" type="number" min="7" max="365" value="90" required></div>
<p class="help">The inbound log also never holds more than 10 000 rows: refused mail is logged at the rate strangers knock on port 25.</p></div>
</div> </div>
</div> </div>
</div> </div>
@@ -270,3 +270,27 @@ code { background: var(--sp-well); color: var(--sp-ink); border-radius: 3px; pad
.sp-record-head small { margin-left: 0; text-align: left; flex-basis: 100%; } .sp-record-head small { margin-left: 0; text-align: left; flex-basis: 100%; }
.sp-postmark { display: none; } .sp-postmark { display: none; }
} }
/* sheet: what does not fit a table row, opened over the page from the row's
tag. It is the browser's own popover (the tag is a button with
popovertarget), so it needs no script: Esc, a click outside and the close
button all shut it, and focus returns to the tag. It rises from the bottom
edge across the whole window - the one component that is not laid out by the
page, hence the widths - with its content, a table of name and value, in a
column of reading width in the middle. */
.sp-sheet {
inset: auto 0 0 0; width: auto; margin: 0; padding: 0; border: 0; border-top: 3px solid var(--sp-brick);
max-height: 75vh; overflow: auto; background: #fff; color: inherit; box-shadow: 0 -12px 40px rgba(40, 20, 15, .22);
}
.sp-sheet-in { max-width: 44rem; margin: 0 auto; padding: 0 1.25rem; }
.sp-sheet::backdrop { background: rgba(40, 20, 15, .38); }
.sp-sheet-head { display: flex; align-items: center; gap: .7rem; padding: 1rem 0 .8rem; border-bottom: 1px solid var(--sp-rule); }
.sp-sheet-head h2 { font-size: 1.15rem; font-weight: 600; margin-right: auto; }
.sp-sheet-head button { border: 0; background: none; cursor: pointer; font-size: 1.3rem; color: var(--sp-muted); }
.sp-sheet-body { padding: .4rem 0 1.4rem; }
/* The sheet's content is a table of name and value: the name is a label, set like the dt of facts. */
.sp-sheet .table th { font-family: var(--sp-mono); font-size: var(--sp-fs-xs); font-weight: 500; letter-spacing: .08em; text-transform: uppercase; color: var(--sp-muted); white-space: nowrap; vertical-align: top; width: 1%; padding-right: 2rem; }
.sp-sheet .table td { overflow-wrap: anywhere; }
/* A tag that opens a sheet is a button: it keeps the tag's look and says it can
be pressed with the dotted underline links to more detail carry. */
button.tag { border: 0; cursor: pointer; text-decoration: underline dotted; text-underline-offset: 3px; }
+3 -2
View File
@@ -107,7 +107,7 @@ than the default «meaningful step» rule in [§ Commits and release build](#com
2. Git tag `vX.Y.Z` and publish the GitHub Release only on explicit request (see 2. Git tag `vX.Y.Z` and publish the GitHub Release only on explicit request (see
[§ Release image](#release-image)). [§ Release image](#release-image)).
**Planned version cuts** (from pin `2.0.0`; adjust if semver changes mid-track): **Planned version cuts** (from pin `2.0.1`; adjust if semver changes mid-track):
| Stage | ID | Cut | | Stage | ID | Cut |
|---|---|---| |---|---|---|
@@ -124,7 +124,8 @@ than the default «meaningful step» rule in [§ Commits and release build](#com
| 11 | panel-redesign plan: component kit, stages, reorder (docs only, owner-requested cut) | `1.9.4` (PATCH) | | 11 | panel-redesign plan: component kit, stages, reorder (docs only, owner-requested cut) | `1.9.4` (PATCH) |
| 12 | GHCR release fix: keep per-arch images, sweep orphaned versions (CI only) | `1.9.5` (PATCH) | | 12 | GHCR release fix: keep per-arch images, sweep orphaned versions (CI only) | `1.9.5` (PATCH) |
| 13 | panel-redesign | `2.0.0` (MAJOR, no compatibility) | | 13 | panel-redesign | `2.0.0` (MAJOR, no compatibility) |
| 14 | inbound-antispam-panel | `2.1.0` (planned; was `1.10.0`, reordered 2026-10-09) | | 14 | DMARC ingest drops and bounds, full backup from the panel, no queue in a backup | `2.0.1` (PATCH) |
| 15 | inbound-antispam-panel | `2.1.0` (planned; was `1.10.0`, reordered 2026-10-09) |
Docs-only prep (checklists in plans, no product code) uses the same per-step Docs-only prep (checklists in plans, no product code) uses the same per-step
commit rule but **no** version cut until the next product stage ships. commit rule but **no** version cut until the next product stage ships.
+78 -8
View File
@@ -183,7 +183,7 @@ cat ./data/setup-token
#### Fixed image tag #### Fixed image tag
`deploy/docker-compose.yml` pins an explicit version (`ghcr.io/mixeme/selfpost:X.Y.Z`), `deploy/docker-compose.yml` pins an explicit version (`ghcr.io/mixeme/selfpost:X.Y.Z`),
deliberately never `:latest`. The current pin is `2.0.0`. Intermediate deliberately never `:latest`. The current pin is `2.0.1`. Intermediate
CHANGELOG sections (`0.2.0`…`0.6.0`) record development cuts from before that CHANGELOG sections (`0.2.0`…`0.6.0`) record development cuts from before that
image was published. Pinning matters because of the backup version check (see image was published. Pinning matters because of the backup version check (see
[Full backup and restore](#full-backup-and-restore)): the panel binary's [Full backup and restore](#full-backup-and-restore)): the panel binary's
@@ -209,6 +209,8 @@ expected to set; defaults match the code exactly.
| `DMARC_MESSAGE_SIZE_LIMIT` | Maximum report message size in bytes when DMARC ingest is on. | `5242880` (5 MiB) | `.env` | | `DMARC_MESSAGE_SIZE_LIMIT` | Maximum report message size in bytes when DMARC ingest is on. | `5242880` (5 MiB) | `.env` |
| `INBOUND_ANTISPAM_MILTER` | Optional milter on the inbound relay listener only (requires `INBOUND_RELAY_ENABLE=true`; not 465/587 or DMARC-only port 25). Empty = off. Format `inet:host:port` or `unix:/path`. Example with [deploy/antispam/docker-compose.antispam.yml](../deploy/antispam/docker-compose.antispam.yml): `inet:antispam:11332`. The panel reads it to report on the *Spam filter* box of the inbound pages and to check that the filter answers (Server › Health). | *(empty)* | `.env` | | `INBOUND_ANTISPAM_MILTER` | Optional milter on the inbound relay listener only (requires `INBOUND_RELAY_ENABLE=true`; not 465/587 or DMARC-only port 25). Empty = off. Format `inet:host:port` or `unix:/path`. Example with [deploy/antispam/docker-compose.antispam.yml](../deploy/antispam/docker-compose.antispam.yml): `inet:antispam:11332`. The panel reads it to report on the *Spam filter* box of the inbound pages and to check that the filter answers (Server › Health). | *(empty)* | `.env` |
| `INBOUND_ANTISPAM_MILTER_ACTION` | What Postfix does if that milter is down: `accept` (fail-open) or `tempfail` (defer). The *Spam filter* box states which. | `accept` | `.env` | | `INBOUND_ANTISPAM_MILTER_ACTION` | What Postfix does if that milter is down: `accept` (fail-open) or `tempfail` (defer). The *Spam filter* box states which. | `accept` | `.env` |
| `INBOUND_SPAM_LOG_RETENTION_DAYS` | How long the inbound journal keeps a row, 7-365 days; it also never holds more than 10 000 rows. Seeds the setting on the first start with the inbound relay on. | `90` | `.env` |
| `INBOUND_JOURNAL_MILTER_SOCKET` | Unix socket of the panel's inbound journal milter, the last milter on port 25 while the inbound relay is on. It records accepted inbound mail and never refuses any; there is no reason to change it. | `/run/selfpost/inbound-journal.sock` | compose |
| `INBOUND_RATE_LIMIT_MESSAGES_PER_IP` | Coarse per-client-IP cap on inbound smtpd (`smtpd_client_message_rate_limit`). Uses the same window as `RATE_LIMIT_WINDOW_SECONDS`. | `20` | `.env` | | `INBOUND_RATE_LIMIT_MESSAGES_PER_IP` | Coarse per-client-IP cap on inbound smtpd (`smtpd_client_message_rate_limit`). Uses the same window as `RATE_LIMIT_WINDOW_SECONDS`. | `20` | `.env` |
| `INBOUND_MESSAGE_SIZE_LIMIT` | Maximum message size in bytes on inbound smtpd (`message_size_limit`). When both inbound relay and DMARC ingest are on, the shared listener uses the **greater** of this and `DMARC_MESSAGE_SIZE_LIMIT`. | `26214400` (25 MiB) | `.env` | | `INBOUND_MESSAGE_SIZE_LIMIT` | Maximum message size in bytes on inbound smtpd (`message_size_limit`). When both inbound relay and DMARC ingest are on, the shared listener uses the **greater** of this and `DMARC_MESSAGE_SIZE_LIMIT`. | `26214400` (25 MiB) | `.env` |
| `RATE_LIMIT_MESSAGES_PER_IP` | Level-1 backstop: maximum messages one client IP may submit per window (Postfix `smtpd_client_message_rate_limit`). See [Rate limiting — level 1](#rate-limiting--level-1-ip-backstop). | `100` | `.env` | | `RATE_LIMIT_MESSAGES_PER_IP` | Level-1 backstop: maximum messages one client IP may submit per window (Postfix `smtpd_client_message_rate_limit`). See [Rate limiting — level 1](#rate-limiting--level-1-ip-backstop). | `100` | `.env` |
@@ -542,10 +544,14 @@ journal-milter (level 2) is down. There is no per-IP bypass.
### Full backup and restore ### Full backup and restore
**Full backup** is a self-contained project archive: `data/` (SQLite, all **Full backup** is a self-contained project archive: `data/` (SQLite, all
domains' DKIM keys, all applications' SASL credentials, the Postfix queue, domains' DKIM keys, all applications' SASL credentials, `manifest.json` with
`manifest.json` with the version that created it), plus `docker-compose.yml`, the version that created it), plus `docker-compose.yml`, `.env`, and `certs/`
`.env`, and `certs/` from the operator directory next to `./data`. Delivery from the operator directory next to `./data`. Delivery logs under `data/log/`
logs under `data/log/` are excluded. The base compose file mounts the project are excluded, and so is the Postfix queue: the archive is taken while the
instance runs, and a copy of a queue Postfix is working on can hold a message
twice or not at all. Mail still waiting for delivery is therefore not carried
over, and a restored instance starts with an empty queue; to keep the queue,
stop the instance and use `tar` (methods 3 and 4 below). The base compose file mounts the project
directory read-only at `/selfpost-deploy` so the panel and CLI can read those directory read-only at `/selfpost-deploy` so the panel and CLI can read those
deploy files — without that mount, *Full backup* refuses with an error. deploy files — without that mount, *Full backup* refuses with an error.
@@ -556,12 +562,25 @@ instance keeps running, what goes into the archive, and what happens on restore.
**1. Panel — while the container is running (no downtime).** *Server › Backup* **1. Panel — while the container is running (no downtime).** *Server › Backup*
→ *Full backup*. *Encrypt with a password* is ticked; untick it for a plain → *Full backup*. *Encrypt with a password* is ticked; untick it for a plain
archive. Same archive as the CLI below. archive. Same archive as the CLI below. The panel runs as its own user, not as
root, so **the private key in `certs/` has to be readable by it**: give the
file to the container's `selfpost` group, which has the fixed id 998 and holds
both the panel and Postfix —
```sh
chgrp 998 certs/privkey.pem && chmod 640 certs/privkey.pem
```
— and have whatever renews the certificate do the same (a certbot deploy hook,
for one). A key readable by root alone is enough for Postfix, but then the
panel refuses to make a backup and names the file: it does not hand over an
archive without the key.
**2. `selfpost-backup` CLI — while the container is running (no downtime).** **2. `selfpost-backup` CLI — while the container is running (no downtime).**
The panel button and this command call the same code path: a consistent SQLite The panel button and this command call the same code path: a consistent SQLite
snapshot via `VACUUM INTO`, then a gzip tar of `data/` (minus diagnostics) plus snapshot via `VACUUM INTO`, then a gzip tar of `data/` (minus diagnostics) plus
deploy files. deploy files. `docker exec` runs the command as root, so it reads the private
key whatever its permissions.
```sh ```sh
docker exec <container> selfpost-backup > selfpost-backup.tar.gz docker exec <container> selfpost-backup > selfpost-backup.tar.gz
@@ -616,6 +635,8 @@ methods avoid it by stopping first.
| Self-contained archive | Yes | Yes | No — deploy files separate | | Self-contained archive | Yes | Yes | No — deploy files separate |
| SQLite consistency | `VACUUM INTO` snapshot | Safe when stopped | Safe when stopped | | SQLite consistency | `VACUUM INTO` snapshot | Safe when stopped | Safe when stopped |
| `data/log/` | Excluded | Included (optional exclude) | Included (optional exclude) | | `data/log/` | Excluded | Included (optional exclude) | Included (optional exclude) |
| Postfix queue | No | Yes | Yes |
| `certs/` private key | Yes (the panel needs it readable by group 998) | Yes | No — deploy files separate |
| `data/manifest.json` | Written (version stamp) | Absent | Absent | | `data/manifest.json` | Written (version stamp) | Absent | Absent |
| Restore version check | Yes — must match image tag | No | No | | Restore version check | Yes — must match image tag | No | No |
| Post-restore *Resync* | Yes, on first boot | No | No | | Post-restore *Resync* | Yes, on first boot | No | No |
@@ -691,6 +712,15 @@ needed) for the new hostname or proxy, then `docker compose up -d`. The archive
carries `certs/` from the old host — re-issue certificates when the hostname or carries `certs/` from the old host — re-issue certificates when the hostname or
IP changes. Set up the reverse-proxy vhost separately (not in the backup). IP changes. Set up the reverse-proxy vhost separately (not in the backup).
The inbound log and the filter lists are in the database and so in the
backup, and so are the list files under `data/antispam/`; the panel writes
those files again from the database on the first start after a restore. What
the backup does not hold is the spam filter's own side: the `antispam/`
directory with the compose fragment and the rspamd configuration. Like the
reverse-proxy fragments it is part of the installation, not of its state -
copy it from [deploy/antispam/](../deploy/antispam/) again, or the restored
instance runs with no filter.
**Restoring an encrypted (`.spbk`) backup** needs a running container to **Restoring an encrypted (`.spbk`) backup** needs a running container to
decrypt it first — any container with the `selfpost-backup` CLI works; decryption decrypt it first — any container with the `selfpost-backup` CLI works; decryption
does not read `/data` and performs no version check. Start one normally does not read `/data` and performs no version check. Start one normally
@@ -822,6 +852,40 @@ only sees SelfPost. Default action is fail-open (`accept`) so a down sidecar
does not block backup-MX; set `INBOUND_ANTISPAM_MILTER_ACTION=tempfail` to does not block backup-MX; set `INBOUND_ANTISPAM_MILTER_ACTION=tempfail` to
defer instead. defer instead.
The fragment mounts one file into the sidecar,
[deploy/antispam/rspamd/milter_headers.conf](../deploy/antispam/rspamd/milter_headers.conf):
it makes rspamd write its verdict - the action and the score with its symbols
- into two headers of every message it lets through. SelfPost records them
with the message, and the upstream receives them with the mail. Copy the
`antispam/rspamd/` directory next to the fragment; without the file accepted
mail is still recorded, with no verdict.
The same directory holds
[multimap.conf](../deploy/antispam/rspamd/multimap.conf), the rules for the
**allow and deny lists** kept in the panel. SelfPost does not apply the lists
itself: it writes them as six files under `data/antispam/` (`allow_ip.map`,
`allow_from.map`, `allow_domain.map` and the three `deny_` ones), the fragment
mounts that directory into the sidecar, and rspamd rereads a file within
seconds of a change - no restart. An entry is a sender address, a sender
domain (that domain, not its subdomains) or a client address or network. A
denied sender is refused outright. An allowed one is given a weight that no
spam score outweighs, so it is accepted - unless a deny entry matches the same
message: **deny wins**, whichever entries they are. The files are readable by
anyone on the host; they hold addresses, not secrets.
In the panel the lists are **Inbound › Filter lists** (global role), and what
port 25 decided about each message is **Inbound › Log**: accepted mail with the
filter's verdict, refused mail with the reply it was given. A row of the log
leads to the lists with its sender filled in. Beside each entry of a list is
how often it decided a message, counted from the log - so over the days the
log keeps - and *never* marks an entry that may have outlived its use. The
count depends on two things in
[multimap.conf](../deploy/antispam/rspamd/multimap.conf): the message of the
deny rules and the names of the allow symbols; change them and every entry
reads *never*. A user with the *domain* role
sees the log of the inbound domains assigned to them, and not the lists. How
long the log keeps a row is set under **Server › Settings**.
The panel only reports this setting; it does not change it. Every inbound The panel only reports this setting; it does not change it. Every inbound
domain page has a **Spam filter** box — on or off, what happens to mail while domain page has a **Spam filter** box — on or off, what happens to mail while
the filter is down, and that the filter belongs to the server, not to the the filter is down, and that the filter belongs to the server, not to the
@@ -891,7 +955,13 @@ sees the *Ingest* box — whether ingest works, the last report received, report
kept this week and parse failures. Open a domain's roll-up from the list or kept this week and parse failures. Open a domain's roll-up from the list or
from *DMARC reports* in the domain page's menu. A user with the *domain* role from *DMARC reports* in the domain page's menu. A user with the *domain* role
sees the reports of the domains assigned to them, without the *Ingest* box. sees the reports of the domains assigned to them, without the *Ingest* box.
Summaries are pruned (500 kept, 90 days max). Summaries are pruned (500 kept for each domain, 90 days max).
A message that is not a report the relay keeps — unreadable as an aggregate
report, about a domain not sent from here, or delivered to another domain's
report address — is dropped without a retry and without a bounce. It adds one
to *parse failures*, and the reason is on the message's delivery line in the mail log
(*Server › System log*, `dropped: …`).
**Not an open relay.** The inbound smtpd offers no SASL. With DMARC ingest **Not an open relay.** The inbound smtpd offers no SASL. With DMARC ingest
alone, `check_recipient_access` permits only configured report addresses; alone, `check_recipient_access` permits only configured report addresses;
+74 -14
View File
@@ -57,8 +57,9 @@ smtpd_milters = { antispam, inbound-journal }
``` ```
Antispam runs first; inbound-journal runs at end-of-message on the **accept** Antispam runs first; inbound-journal runs at end-of-message on the **accept**
path and reads rspamd-added headers (`X-Rspamd-Action`, `X-Rspamd-Score`, symbol path and reads rspamd-added headers (`X-Rspamd-Action`, and `X-Spamd-Result`
summary when present). for the score and symbols - rspamd has no `X-Rspamd-Score`; the sidecar adds
both through `deploy/antispam/rspamd/milter_headers.conf`).
**Antispam rejects** do not reach end-of-message on the journal milter. v1 also **Antispam rejects** do not reach end-of-message on the journal milter. v1 also
tails `mail.log` for inbound `postfix/smtpd` **`milter-reject`** / **`reject`** tails `mail.log` for inbound `postfix/smtpd` **`milter-reject`** / **`reject`**
@@ -83,6 +84,25 @@ configured.
**Global administrator only** for journal and lists — same as inbound relay. No **Global administrator only** for journal and lists — same as inbound relay. No
domain-admin access in v1. domain-admin access in v1.
*As built (2026-10-11):* the lists are the global role's; the **log follows
the delegation of Inbound** that 2.0 introduced after this was written - a
user with the `domain` role sees the rows of the inbound domains assigned to
them and nothing else, without the *Allow / Deny sender* links. That is what
the accepted `in-log` mockup draws (no "global role" in its head). The owner
was told and can turn it back into a global-only page.
**Inbound › Log accepted by the owner on 2026-10-12** ("Этот экран
утверждаем"), from screenshots of the page built from its fixture, after his
changes to it: the sender column is headed *From*; *Why* is not a column but
a sheet over the page, opened by the decision tag, holding a centred table of
who decided, the verdict and symbols or the reply, and the subject whole
(kept on the sheet at his word: the table clips it); a message Postfix itself
refused offers no *Deny sender*. Inbound › Filter lists was redrawn at his word
the same day - kind in a column of its own, the note on the entry's sheet,
matches counted (§ Filter list usage) - and **accepted by him on 2026-10-12**
("Да"), after a deny entry he added in the panel refused a test message with
the list's reply and the journal recorded it.
### Retention ### Retention
Mirror send-log retention: Mirror send-log retention:
@@ -101,13 +121,53 @@ Mirror send-log retention:
| Entry types | sender address, sender domain (`@domain` suffix), client IP / CIDR | | Entry types | sender address, sender domain (`@domain` suffix), client IP / CIDR |
| Precedence | **deny overrides allow** | | Precedence | **deny overrides allow** |
| Scope | instance-wide (not per inbound domain in v1) | | Scope | instance-wide (not per inbound domain in v1) |
| Where applied | **rspamd map files** under `/data/antispam/` (allow.map, deny.map); documented compose fragment mounts them into the sidecar | | Where applied | **rspamd map files** under `/data/antispam/`, one per list and entry type (`allow_ip.map`, `allow_from.map`, `allow_domain.map`, `deny_…`) - an IP map and a sender map are different map types to rspamd; documented compose fragment mounts them into the sidecar. Deny is a prefilter reject, allow a weight of -100 (as built: two prefilters do not give "deny overrides allow") |
| SelfPost pre-check | **out of v1** — lists live in rspamd only | | SelfPost pre-check | **out of v1** — lists live in rspamd only |
| Edit access | global administrator only | | Edit access | global administrator only |
| When inactive | lists editable in the panel but UI warns that sync applies only when `INBOUND_ANTISPAM_MILTER` is set; `postfix reload` / rspamd reload documented in guide | | When inactive | lists editable in the panel but UI warns that sync applies only when `INBOUND_ANTISPAM_MILTER` is set; `postfix reload` / rspamd reload documented in guide |
List change → atomic map rewrite (same pattern as Postfix maps) → rspamd reload List change → atomic map rewrite (same pattern as Postfix maps). No reload:
via documented operator step or sidecar `SIGHUP` in the compose fragment. rspamd rereads a changed map file by itself (about ten seconds on 3.11).
### Filter list usage (owner, 2026-10-12)
Each entry shows how often it decided a message (*Matched*, or *never*) and,
on its sheet, when it last did - so that entries nothing matches any more can
be removed. **The count is read off the journal on every view of the page;
nothing is stored for it.** A journal row counts for an entry when the list
decided the message - a rejected row whose reply is the deny rule's message,
an accepted row whose verdict carries the allow rule's symbol
(`SELFPOST_ALLOW_IP` / `_FROM` / `_DOMAIN`, which the journal milter always
keeps) - and the entry fits the row's sender or client address the way rspamd
matches it (`inbound.Lists.Usage`).
What follows from counting this way, and is accepted:
- the count reaches back as far as the journal does: its retention (90 days by
default), which is what the sheet says;
- a row counts for every fitting entry of the deciding list, since the journal
does not say which of two the filter met first;
- a message refused at end-of-message is one row whatever its recipient count,
so a count is of messages, roughly.
**When to move to a counter kept with each entry** (a migration adding hits
and last-hit to `inbound_spam_list`, bumped where the journal row is written;
the owner asked for the condition to be written down). Any one of:
1. **The journal is cut by its 10 000-row cap and not by age.** Then it holds
fewer days than its retention, the sheet's "in the last 90 days" is no
longer true, and *never* stops being evidence that an entry is stale. The
sign: the log tailer's prune line keeps reporting rows removed while the
oldest row of Inbound › Log is younger than the retention.
2. **The retention is set short** (towards its minimum of 7 days) for reasons
of its own, privacy or disk. A week without a match does not make an entry
stale.
3. **Counts are wanted that outlive the journal** - since the entry was added.
4. **The page is slow.** A view reads every journal row a list decided and
tries it against every entry. With the cap that is at most 10 000 rows
times the number of entries; past a few hundred entries, measure it.
Until then no migration is spent on it.
### Activation ### Activation
@@ -243,14 +303,14 @@ Target version cut: **`2.1.0`** (MINOR), after `2.0.0`. One commit per step;
[development.md](../development.md) § Plan checklists. [development.md](../development.md) § Plan checklists.
- [x] Agree journal fields, retention, and RBAC (this plan § Decisions) — **Composer** - [x] Agree journal fields, retention, and RBAC (this plan § Decisions) — **Composer**
- [ ] Migration `0002_inbound_spam_log.sql` on the 2.0 baseline — **Composer** - [x] Migration `0002_inbound_spam_log.sql` on the 2.0 baseline — **Composer**
- [ ] Inbound journal-milter + Postfix wiring — **Composer** - [x] Inbound journal-milter + Postfix wiring — **Composer**
- [ ] mail.log tailer: inbound reject rows — **Composer** - [x] mail.log tailer: inbound reject rows — **Composer**
- [ ] List CRUD + validation + atomic rspamd map sync — **Composer** - [x] List CRUD + validation + atomic rspamd map sync — **Composer**
- [ ] Panel: Inbound › Log and Inbound › Filter lists from the kit partials, mockups `in-log` / `in-filter-lists` (add client IP and engine columns mockup-first), evidence pairs — **Composer** - [x] Panel: Inbound › Log and Inbound › Filter lists from the kit partials, mockups `in-log` / `in-filter-lists` (add client IP and engine columns mockup-first), evidence pairs — **Composer**
- [ ] Backup includes `/data/antispam/` — **Composer** - [x] Backup includes `/data/antispam/` — **Composer** (it always did: a backup walks all of `/data`; now held by a test, and the guide says the `antispam/` fragment directory is not in it)
- [ ] Unit + handler tests — **Composer** - [x] Unit + handler tests — **Composer** (written with each step: store, milter, tailer, lists and their usage, view fixtures held to the outlines, handlers for both roles)
- [ ] [guide.md](../guide.md), [architecture.md](../architecture.md) — **Composer** - [x] [guide.md](../guide.md), [architecture.md](../architecture.md) — **Composer** (kept up with each step; [security.md](../security.md) follows the security review)
- [ ] `go vet`, `go test`, e2e if applicable — **Composer** - [x] `go vet`, `go test`, e2e if applicable — **Composer** (on the test server for every step; last on `7192c38`: vet ok, 22 packages ok, e2e ok, which relays, refuses, reads the log page and edits a list through the panel)
- [ ] Technical review of inbound antispam changes — **Opus** - [ ] Technical review of inbound antispam changes — **Opus**
- [ ] Security review inbound antispam path — **Fable** - [ ] Security review inbound antispam path — **Fable**
+39 -2
View File
@@ -30,7 +30,7 @@ in `git log` and [CHANGELOG.md](../CHANGELOG.md).
|---|---|---|---|---| |---|---|---|---|---|
| preflight | Installation check page (`/preflight`) | candidate | — | [plans/preflight.md](plans/preflight.md) | | preflight | Installation check page (`/preflight`) | candidate | — | [plans/preflight.md](plans/preflight.md) |
| contributing | `CONTRIBUTING.md` | candidate | — | — | | contributing | `CONTRIBUTING.md` | candidate | — | — |
| inbound-antispam-panel | Inbound antispam journal + allow/deny lists | agreed | 1/12 | [plans/inbound-antispam-panel.md](plans/inbound-antispam-panel.md) | | inbound-antispam-panel | Inbound antispam journal + allow/deny lists | in progress | 10/12 | [plans/inbound-antispam-panel.md](plans/inbound-antispam-panel.md) |
| inbound-quarantine | Inbound spam quarantine (hold / review / release) | candidate | — | [plans/inbound-quarantine.md](plans/inbound-quarantine.md) | | inbound-quarantine | Inbound spam quarantine (hold / review / release) | candidate | — | [plans/inbound-quarantine.md](plans/inbound-quarantine.md) |
| csrf-tokens | Per-session CSRF tokens on state-changing forms | candidate | — | — | | csrf-tokens | Per-session CSRF tokens on state-changing forms | candidate | — | — |
| template-data-typing | Typed structs for template data instead of `map[string]any` | candidate | — | — | | template-data-typing | Typed structs for template data instead of `map[string]any` | candidate | — | — |
@@ -39,6 +39,7 @@ in `git log` and [CHANGELOG.md](../CHANGELOG.md).
| panel-notifications | E-mail notifications about important events | candidate | — | — | | panel-notifications | E-mail notifications about important events | candidate | — | — |
| delivery-log-storage | Keep a message's `mail.log` lines with its send-log row | candidate | — | — | | delivery-log-storage | Keep a message's `mail.log` lines with its send-log row | candidate | — | — |
| password-reset | Password reset by e-mail | candidate | — | — | | password-reset | Password reset by e-mail | candidate | — | — |
| phone-desktop-mode | A phone browser in "desktop site" mode gets the phone navigation | candidate | — | — |
| panel-redesign | Panel redesign (accepted mockups, design contract) | shipped `2.0.0` | 27/27 | [plans/panel-redesign.md](plans/panel-redesign.md) | | panel-redesign | Panel redesign (accepted mockups, design contract) | shipped `2.0.0` | 27/27 | [plans/panel-redesign.md](plans/panel-redesign.md) |
| schema-squash | Squash SQLite migrations into a 2.x baseline | built in panel-redesign (stage 1) | — | [plans/panel-redesign.md](plans/panel-redesign.md) | | schema-squash | Squash SQLite migrations into a 2.x baseline | built in panel-redesign (stage 1) | — | [plans/panel-redesign.md](plans/panel-redesign.md) |
@@ -47,7 +48,8 @@ development convenience): **panel-redesign** shipped as `2.0.0` (2026-10-11); th
**inbound-antispam-panel** (`2.1.0`), built in the new component kit — **inbound-antispam-panel** (`2.1.0`), built in the new component kit —
styling its two screens once instead of twice. panel-docs shipped in styling its two screens once instead of twice. panel-docs shipped in
[CHANGELOG.md](../CHANGELOG.md) `[1.8.0]`; dmarc-reports in `[1.7.0]` [CHANGELOG.md](../CHANGELOG.md) `[1.8.0]`; dmarc-reports in `[1.7.0]`
(security review of the ingest path pending); domain-stats-auto-ratelimit in (its ingest path first ran in `2.0.0` and was reviewed on 2026-10-11, see
[security.md](security.md)); domain-stats-auto-ratelimit in
`[1.6.0]`; send-log-retention in `[1.5.0]`; inbound-relay in `[1.4.0]`; `[1.6.0]`; send-log-retention in `[1.5.0]`; inbound-relay in `[1.4.0]`;
queue-retries in `[1.3.1]`; the 2026-08-13 full-tree review follow-ups are in queue-retries in `[1.3.1]`; the 2026-08-13 full-tree review follow-ups are in
`[1.3.0]`. Candidates need explicit agreement before they join the queue. `[1.3.0]`. Candidates need explicit agreement before they join the queue.
@@ -405,6 +407,41 @@ goes and today Account saves it without the password.
--- ---
## phone-desktop-mode
**Problem (owner, 2026-10-11):** the panel opened on a phone with the
browser's "desktop site" switched on does not become the desktop panel.
Seen on Server › Health at `selfpost.mixfed.ru`: the page body is laid out in
columns (Machine and Processes side by side, three cards in the next row),
while the navigation is the phone one - not a bar but a stack with every
section open (Outbound, Inbound, Server, the user menu), filling the first
screen before the page starts. The content also sits flush against the left
edge of the screen.
**Goal:** find out why, then decide what the panel should be in that mode -
the desktop panel, or a phone panel that is at least consistent.
**Where to look first (a guess, not checked):** "desktop site" on Android
lays a page out about 980 px wide. Bulma turns the navigation into a bar at
1024 px and puts columns side by side from 769 px, so 980 px falls between
the two: desktop columns under a touch navigation. `panel.css` has its own
breakpoints at 768, 1023 and 1407 px. Check the width the phone really
reports, why every dropdown is open rather than folded behind the burger,
and where the left padding went.
**Boundary:** the navigation and page gutter between 769 and 1023 px, which
is also a tablet in portrait and a narrow desktop window - the same defect is
expected there. A design change is mockup-first
([plans/panel-redesign.md](plans/panel-redesign.md) § Enforcement).
**Done when:** the cause is written down here, and the panel at that width
has one coherent layout, checked on a phone in "desktop site" mode and in a
980 px window.
**Version:** `2.x` PATCH or MINOR; `candidate`.
---
## panel-redesign ## panel-redesign
**Goal:** a panel an operator opens a few times a year and still finds their **Goal:** a panel an operator opens a few times a year and still finds their
+12 -9
View File
@@ -17,8 +17,8 @@ When you add or review a migration, update this file in the same change.
| Field | Value | | Field | Value |
|---|---| |---|---|
| Chain head | `user_version = 1` | | Chain head | `user_version = 2` |
| Files | `0001_init.sql` (the 2.0 baseline) | | Files | `0001_init.sql` (the 2.0 baseline), `0002_inbound_spam_log.sql` |
| File mark | `PRAGMA application_id = 0x53503230` ("SP20") | | File mark | `PRAGMA application_id = 0x53503230` ("SP20") |
| Database file | `/data/selfpost.db` (bind mount) | | Database file | `/data/selfpost.db` (bind mount) |
| Compatibility | 2.x MINOR releases must boot a `2.0.0` data directory. **Nothing is carried over from 1.x.** | | Compatibility | 2.x MINOR releases must boot a `2.0.0` data directory. **Nothing is carried over from 1.x.** |
@@ -60,10 +60,9 @@ What that means in practice:
**Implication:** once `2.0.0` is released, `0001_init.sql` is never edited **Implication:** once `2.0.0` is released, `0001_init.sql` is never edited
again and no file is deleted, renamed or reordered — every 2.x database has again and no file is deleted, renamed or reordered — every 2.x database has
already run it. Until that release the baseline is still being written: already run it. `2.0.0` and `2.0.1` shipped with the baseline alone; the
later steps of the redesign and the features queued behind it (the next one features after them add files, the first being `0002_inbound_spam_log.sql` of
is `0002_inbound_spam_log.sql` of inbound-antispam-panel, `2.1.0`) add files inbound-antispam-panel (`2.1.0`).
after it.
--- ---
@@ -72,13 +71,14 @@ after it.
| Ver | File | Shipped | Kind | Summary | | Ver | File | Shipped | Kind | Summary |
|-----|------|---------|------|---------| |-----|------|---------|------|---------|
| 1 | `0001_init.sql` | 2.0.0 | DDL | The whole 2.0 schema, below | | 1 | `0001_init.sql` | 2.0.0 | DDL | The whole 2.0 schema, below |
| 2 | `0002_inbound_spam_log.sql` | 2.1.0 (unreleased) | DDL | `inbound_spam_log`, `inbound_spam_list` and two indexes; nothing existing changes |
**Kind:** *DDL* — schema only; *data* — `INSERT`/`UPDATE` that must stay correct **Kind:** *DDL* — schema only; *data* — `INSERT`/`UPDATE` that must stay correct
for operators upgrading from older 2.x images. for operators upgrading from older 2.x images.
--- ---
## Schema at head (v1) ## Schema at head (v2)
| Table | Role | | Table | Role |
|-------|------| |-------|------|
@@ -98,9 +98,12 @@ for operators upgrading from older 2.x images.
| `user_inbound_domains` | Inbound domains assigned to a `domain` user — granted separately from the outbound ones | | `user_inbound_domains` | Inbound domains assigned to a `domain` user — granted separately from the outbound ones |
| `dmarc_reports` | Parsed aggregate report summaries | | `dmarc_reports` | Parsed aggregate report summaries |
| `dmarc_report_records` | Per-source rows inside a report | | `dmarc_report_records` | Per-source rows inside a report |
| `inbound_spam_log` | Inbound antispam journal: one row per recipient of a message offered on port 25, with the decision (`accept`, `reject`, `tempfail`, `quarantine`) and who made it (`rspamd`, `postfix`, `selfpost`). Metadata only (v2) |
| `inbound_spam_list` | Allow / deny entries for the external filter (`address`, `domain`, `ip`), instance-wide; unique per list (v2) |
`TestBaselineSchema` (`internal/store/schema_test.go`) pins every table, column `TestBaselineSchema` (`internal/store/schema_test.go`) pins every table, column
and index of this list. and index of this list; `TestOpenMigratesA20Database` opens a database that
holds the baseline alone.
### What changed against the last 1.x schema ### What changed against the last 1.x schema
@@ -129,7 +132,7 @@ for the domain. Nothing refers to a user: deleting one changes no domain.
1. Add `internal/store/migrations/00NN_short_name.sql` (next number). 1. Add `internal/store/migrations/00NN_short_name.sql` (next number).
2. Update the table in [Migration chain](#migration-chain), the snapshot at the 2. Update the table in [Migration chain](#migration-chain), the snapshot at the
top, and the affected rows of [Schema at head](#schema-at-head-v1). top, and the affected rows of [Schema at head](#schema-at-head-v2).
3. Update `TestBaselineSchema` if tables, columns or indexes change, and add a 3. Update `TestBaselineSchema` if tables, columns or indexes change, and add a
test that opens a database at the previous version and migrates it. test that opens a database at the previous version and migrates it.
4. A *data* migration states in a comment which 2.x versions it is written for. 4. A *data* migration states in a comment which 2.x versions it is written for.
+17 -3
View File
@@ -26,7 +26,14 @@ the user form did not end that user's sessions, internal errors reached forms a
(see [Authentication and sessions](#authentication-and-sessions) and [Errors (see [Authentication and sessions](#authentication-and-sessions) and [Errors
shown to users](#errors-shown-to-users)); two deliberate behaviours are shown to users](#errors-shown-to-users)); two deliberate behaviours are
recorded under Accepted risks (upstream of a delegated inbound domain, profile recorded under Accepted risks (upstream of a delegated inbound domain, profile
saved without the password). saved without the password). The 2026-10-11 review (Fable) covered the DMARC
report path from port 25 to the report pages, which ran for the first time in
2.0.0 - the 2026-08-19 pass read code that Postfix could not yet start: no
Critical or High finding; a report's size was bounded only as bytes (now 5 000
records, 255 bytes a field), the retention cap was one for the whole server
(now per domain), the delivery command could run a hundred at once (now one),
and the reason logged for a dropped message carried the sender's bytes as they
came (now cut and reduced to printable ASCII).
(2) **Accepted risks** — (2) **Accepted risks** —
deliberate departures beyond the mandatory, recorded so the decision is not deliberate departures beyond the mandatory, recorded so the decision is not
lost. lost.
@@ -274,8 +281,15 @@ deferred item from the roadmap.
reporters are arbitrary third parties and the mailbox address is published in reporters are arbitrary third parties and the mailbox address is published in
DNS. The ingest already constrains the meaningful part — a report cannot be DNS. The ingest already constrains the meaningful part — a report cannot be
filed under a domain this instance does not host, and the claimed domain is filed under a domain this instance does not host, and the claimed domain is
cross-checked against the `+tag` of the delivery address — and the data is cross-checked against the `+tag` of the delivery address (effective since
advisory statistics only; nothing enforces off it. The trigger to revisit is 2.0.0: before it the recipient never reached the command) — and the data is
advisory statistics only; nothing enforces off it. What a stranger can do is
limited to a domain that chose a SelfPost-hosted address, through that
address: file reports under it, replace one whose reporter and id they know,
and fill that domain's 500 — not another domain's. A message that is not a
report the relay keeps is dropped, not bounced (the envelope sender is the
author's to write); a genuine report the parser cannot read is lost the same
way, with a count and a line in the mail log. The trigger to revisit is
any feature that lets report contents drive an automatic action (policy any feature that lets report contents drive an automatic action (policy
changes, rate limiting, alerting an operator into a config change). changes, rate limiting, alerting an operator into a config change).
- **The application auth-IP allow-list is enforced fail-open.** The check runs - **The application auth-IP allow-list is enforced fail-open.** The check runs
+84 -10
View File
@@ -1,16 +1,16 @@
// Package backup implements SelfPost's full-server backup and the restore // Package backup implements SelfPost's full-server backup and the restore
// version guard (architecture.md § Persistence). A full backup is a // version guard (architecture.md § Persistence). A full backup is a
// gzip-compressed tar of the consolidated persistent state under /data — the // gzip-compressed tar of the consolidated persistent state under /data — the
// SQLite database (as a consistent snapshot), the per-domain DKIM keys, the // SQLite database (as a consistent snapshot), the per-domain DKIM keys and the
// SASL database, and the Postfix queue — plus docker-compose.yml, .env, and // SASL database — plus docker-compose.yml, .env, and certs/ from the
// certs/ from the operator's deploy directory, and a manifest recording the // operator's deploy directory, and a manifest recording the SelfPost version
// SelfPost version that produced it. Postfix delivery logs under log/ are // that produced it. Postfix delivery logs under log/ are excluded
// excluded (diagnostics, not state). // (diagnostics, not state), and so is the Postfix queue (see QueueDir).
// //
// Restore is not a separate code path in the panel: a backup is extracted into // Restore is not a separate code path in the panel: a backup is extracted into
// the operator's project directory (data/, docker-compose.yml, .env, certs/) // the operator's project directory (data/, docker-compose.yml, .env, certs/)
// before first start. The archive carries everything needed to bring the // before first start. The archive carries everything needed to bring the
// instance back — DKIM keys, sasldb2, sender map, queue, and deploy files — // instance back — DKIM keys, sasldb2, sender map, and deploy files —
// so the operator only adjusts hostname or proxy settings on a new host. The // so the operator only adjusts hostname or proxy settings on a new host. The
// restore-specific steps the panel runs are CheckRestore, which refuses to // restore-specific steps the panel runs are CheckRestore, which refuses to
// boot if the manifest's version does not match the running binary so // boot if the manifest's version does not match the running binary so
@@ -28,6 +28,7 @@ import (
"compress/gzip" "compress/gzip"
"database/sql" "database/sql"
"encoding/json" "encoding/json"
"errors"
"fmt" "fmt"
"io" "io"
"io/fs" "io/fs"
@@ -81,6 +82,18 @@ type Params struct {
OnWarn func(string) OnWarn func(string)
} }
// QueueDir is the Postfix queue, relative to the data directory. No backup
// holds it (owner, 2026-10-11). An archive is made while the instance runs,
// and Postfix moves messages between the queue's directories all the time: a
// copy taken then can hold a message twice or not at all, so what it would
// bring back is not the queue. Nor could the panel read it - Postfix keeps
// every queue file readable by its own user alone, which is why the panel's
// backup failed outright from 1.3.0 to 2.0.0. What is not carried over is mail
// still waiting for delivery; a start without a queue directory builds an
// empty one. Whoever needs the queue stops the instance and archives the
// project directory (guide § Taking a full backup).
const QueueDir = "postfix/queue"
// excludedFromArchive lists the data-directory entries a backup never carries. // excludedFromArchive lists the data-directory entries a backup never carries.
// The live database files are replaced by a consistent VACUUM INTO snapshot // The live database files are replaced by a consistent VACUUM INTO snapshot
// written under the canonical name; the setup token is transient bootstrap // written under the canonical name; the setup token is transient bootstrap
@@ -145,7 +158,7 @@ func Create(w io.Writer, p Params) error {
return err return err
} }
if err := addTree(tw, p.DataDir, DataArchivePrefix, excludedFromArchive); err != nil { if err := addTree(tw, p.DataDir, DataArchivePrefix, excludedFromArchive, skipQueue); err != nil {
return err return err
} }
@@ -181,6 +194,52 @@ func validateDeployRoot(deployRoot string) error {
return nil return nil
} }
// skipFunc says whether the entry at path (rel to the root being walked) is
// left out of the archive. A directory left out is not descended into.
type skipFunc func(path, rel string, d fs.DirEntry) bool
// skipQueue leaves the Postfix queue out of every archive.
func skipQueue(_, rel string, _ fs.DirEntry) bool {
return filepath.ToSlash(rel) == QueueDir
}
// Check reads through everything Create would put into the archive, without
// writing one, and returns the first thing that cannot be read. The panel
// streams its archive to the browser, so once Create has begun a failure can
// only cut the download short; this is run first, while a refusal can still be
// shown as a page.
//
// The panel is not root, and nothing is left out on that account: its archive
// is the one selfpost-backup makes, or it is not made. What it most likely
// cannot read is the private key under certs/ - see UnreadableCert.
func Check(p Params) error {
if p.DataDir == "" || p.DBPath == "" {
return fmt.Errorf("backup: DataDir and DBPath are required")
}
if err := ValidateDeployRoot(p.DeployRoot); err != nil {
return err
}
if err := addTree(nil, p.DataDir, DataArchivePrefix, excludedFromArchive, skipQueue); err != nil {
return err
}
return addDeployFiles(nil, p.DeployRoot, nil)
}
// UnreadableCert reports whether err, from Check or Create, is a file under
// the deploy root's certs/ that the caller is not allowed to read, and names
// it relative to the deploy root ("certs/privkey.pem").
func UnreadableCert(err error, deployRoot string) (name string, ok bool) {
var pathErr *fs.PathError
if !errors.As(err, &pathErr) || !errors.Is(err, fs.ErrPermission) {
return "", false
}
rel, relErr := filepath.Rel(filepath.Join(deployRoot, CertsDirName), pathErr.Path)
if relErr != nil || rel == ".." || strings.HasPrefix(filepath.ToSlash(rel), "../") {
return "", false
}
return CertsDirName + "/" + filepath.ToSlash(rel), true
}
func addDeployFiles(tw *tar.Writer, deployRoot string, onWarn func(string)) error { func addDeployFiles(tw *tar.Writer, deployRoot string, onWarn func(string)) error {
for _, name := range []string{ComposeFileName, EnvFileName} { for _, name := range []string{ComposeFileName, EnvFileName} {
src := filepath.Join(deployRoot, name) src := filepath.Join(deployRoot, name)
@@ -206,14 +265,16 @@ func addDeployFiles(tw *tar.Writer, deployRoot string, onWarn func(string)) erro
} }
return fmt.Errorf("backup: stat %s: %w", CertsDirName, err) return fmt.Errorf("backup: stat %s: %w", CertsDirName, err)
} }
return addTree(tw, certsDir, CertsDirName+"/", nil) return addTree(tw, certsDir, CertsDirName+"/", nil, nil)
} }
// addTree walks root and adds every regular file (and directory, to preserve // addTree walks root and adds every regular file (and directory, to preserve
// empty ones and modes) to tw under archivePrefix + path relative to root. // empty ones and modes) to tw under archivePrefix + path relative to root.
// When exclude is non-nil, top-level names relative to root are skipped. // When exclude is non-nil, top-level names relative to root are skipped; skip,
// when non-nil, leaves out whatever it says. With a nil tw nothing is written:
// every file is opened and closed, which is how Check finds what cannot be read.
// Non-regular, non-directory entries (symlinks, sockets) are skipped. // Non-regular, non-directory entries (symlinks, sockets) are skipped.
func addTree(tw *tar.Writer, root, archivePrefix string, exclude map[string]bool) error { func addTree(tw *tar.Writer, root, archivePrefix string, exclude map[string]bool, skip skipFunc) error {
return filepath.WalkDir(root, func(path string, d fs.DirEntry, err error) error { return filepath.WalkDir(root, func(path string, d fs.DirEntry, err error) error {
if err != nil { if err != nil {
return err return err
@@ -236,12 +297,22 @@ func addTree(tw *tar.Writer, root, archivePrefix string, exclude map[string]bool
} }
} }
if skip != nil && skip(path, rel, d) {
if d.IsDir() {
return fs.SkipDir
}
return nil
}
info, err := d.Info() info, err := d.Info()
if err != nil { if err != nil {
return err return err
} }
switch { switch {
case d.IsDir(): case d.IsDir():
if tw == nil {
return nil
}
hdr := &tar.Header{ hdr := &tar.Header{
Typeflag: tar.TypeDir, Typeflag: tar.TypeDir,
Name: name + "/", Name: name + "/",
@@ -282,6 +353,9 @@ func writeTarFile(tw *tar.Writer, name string, mode fs.FileMode, srcPath string)
return fmt.Errorf("backup: open %s: %w", srcPath, err) return fmt.Errorf("backup: open %s: %w", srcPath, err)
} }
defer f.Close() defer f.Close()
if tw == nil {
return nil
}
info, err := f.Stat() info, err := f.Stat()
if err != nil { if err != nil {
+75 -1
View File
@@ -5,7 +5,9 @@ import (
"bytes" "bytes"
"compress/gzip" "compress/gzip"
"encoding/json" "encoding/json"
"fmt"
"io" "io"
"io/fs"
"os" "os"
"path/filepath" "path/filepath"
"strings" "strings"
@@ -40,6 +42,7 @@ func seedDataDir(t *testing.T) (dataDir, dbPath, deployRoot string) {
writeFile(t, filepath.Join(dataDir, "opendkim", "keys", "example.com", "selfpost.private"), "PRIVATE KEY") writeFile(t, filepath.Join(dataDir, "opendkim", "keys", "example.com", "selfpost.private"), "PRIVATE KEY")
writeFile(t, filepath.Join(dataDir, "sasl", "sasldb2"), "SASLDB") writeFile(t, filepath.Join(dataDir, "sasl", "sasldb2"), "SASLDB")
writeFile(t, filepath.Join(dataDir, "postfix", "sender_login_maps"), "@example.com login") writeFile(t, filepath.Join(dataDir, "postfix", "sender_login_maps"), "@example.com login")
writeFile(t, filepath.Join(dataDir, "antispam", "deny_domain.map"), "evil.example\n")
writeFile(t, filepath.Join(dataDir, "postfix", "queue", "deferred", "sample"), "queue-file") writeFile(t, filepath.Join(dataDir, "postfix", "queue", "deferred", "sample"), "queue-file")
// Transient files that must NOT be archived. // Transient files that must NOT be archived.
writeFile(t, filepath.Join(dataDir, "setup-token"), "secret-token") writeFile(t, filepath.Join(dataDir, "setup-token"), "secret-token")
@@ -112,7 +115,10 @@ func TestCreateIncludesStateExcludesTransient(t *testing.T) {
DataArchivePrefix + "opendkim/keys/example.com/selfpost.private", DataArchivePrefix + "opendkim/keys/example.com/selfpost.private",
DataArchivePrefix + "sasl/sasldb2", DataArchivePrefix + "sasl/sasldb2",
DataArchivePrefix + "postfix/sender_login_maps", DataArchivePrefix + "postfix/sender_login_maps",
DataArchivePrefix + "postfix/queue/deferred/sample", // The filter lists as the spam filter reads them. They are rewritten
// from the database on the first start after a restore; carrying them
// means the filter has its lists before the panel is up.
DataArchivePrefix + "antispam/deny_domain.map",
ComposeFileName, ComposeFileName,
EnvFileName, EnvFileName,
CertsDirName + "/fullchain.pem", CertsDirName + "/fullchain.pem",
@@ -129,6 +135,7 @@ func TestCreateIncludesStateExcludesTransient(t *testing.T) {
DataArchivePrefix + "selfpost.db-shm", DataArchivePrefix + "selfpost.db-shm",
DataArchivePrefix + "log/mail.log", DataArchivePrefix + "log/mail.log",
DataArchivePrefix + "log/mail.log.1", DataArchivePrefix + "log/mail.log.1",
DataArchivePrefix + "postfix/queue/deferred/sample",
} { } {
if _, ok := files[name]; ok { if _, ok := files[name]; ok {
t.Errorf("archive should not contain %s", name) t.Errorf("archive should not contain %s", name)
@@ -235,3 +242,70 @@ func TestCheckRestoreWrongFormatRejected(t *testing.T) {
t.Error("CheckRestore wrong format reported a restore") t.Error("CheckRestore wrong format reported a restore")
} }
} }
// No archive holds the Postfix queue; the rest is there, the private key
// included, and Check agrees that it can be made.
func TestCreateLeavesTheQueueOut(t *testing.T) {
dataDir, dbPath, deployRoot := seedDataDir(t)
p := Params{DataDir: dataDir, DBPath: dbPath, Version: "1.2.3", DeployRoot: deployRoot}
if err := Check(p); err != nil {
t.Fatalf("Check: %v", err)
}
var buf bytes.Buffer
if err := Create(&buf, p); err != nil {
t.Fatalf("Create: %v", err)
}
files := readArchive(t, buf.Bytes())
for name := range files {
if strings.HasPrefix(name, DataArchivePrefix+QueueDir) {
t.Errorf("the panel's archive holds %s", name)
}
}
for _, name := range []string{
DataArchivePrefix + "selfpost.db",
DataArchivePrefix + "opendkim/keys/example.com/selfpost.private",
DataArchivePrefix + "sasl/sasldb2",
DataArchivePrefix + "postfix/sender_login_maps",
ComposeFileName, EnvFileName,
CertsDirName + "/fullchain.pem",
CertsDirName + "/privkey.pem",
} {
if _, ok := files[name]; !ok {
t.Errorf("archive missing %s", name)
}
}
}
// A file under certs/ that may not be read is told apart from any other
// failure, so the panel can say what to do about it.
func TestUnreadableCert(t *testing.T) {
root := filepath.Join("srv", "deploy")
denied := func(path string) error {
return fmt.Errorf("backup: open %s: %w", path, &fs.PathError{Op: "open", Path: path, Err: fs.ErrPermission})
}
if name, ok := UnreadableCert(denied(filepath.Join(root, "certs", "privkey.pem")), root); !ok || name != "certs/privkey.pem" {
t.Errorf("certs/privkey.pem: got %q, %v", name, ok)
}
if name, ok := UnreadableCert(denied(filepath.Join(root, ".env")), root); ok {
t.Errorf(".env was taken for a certificate file: %q", name)
}
if name, ok := UnreadableCert(denied(filepath.Join("data", "postfix", "x")), root); ok {
t.Errorf("a data file was taken for a certificate file: %q", name)
}
missing := &fs.PathError{Op: "open", Path: filepath.Join(root, "certs", "privkey.pem"), Err: fs.ErrNotExist}
if _, ok := UnreadableCert(missing, root); ok {
t.Error("a missing file was taken for one that may not be read")
}
}
// Check finds what Create would stumble over, before anything is written.
func TestCheckReportsWhatCannotBeRead(t *testing.T) {
dataDir, dbPath, deployRoot := seedDataDir(t)
if err := os.Remove(filepath.Join(deployRoot, EnvFileName)); err != nil {
t.Fatal(err)
}
if err := Check(Params{DataDir: dataDir, DBPath: dbPath, Version: "1.2.3", DeployRoot: deployRoot}); err == nil {
t.Fatal("Check passed with .env missing from the deploy root")
}
}
+109
View File
@@ -0,0 +1,109 @@
package dmarc
import (
"errors"
"strings"
"testing"
"time"
"github.com/mixeme/selfpost/internal/store"
)
func reportXML(orgName, domain, records string) string {
return `<?xml version="1.0"?><feedback><report_metadata><org_name>` + orgName +
`</org_name><email>a@example.net</email><report_id>1</report_id>` +
`<date_range><begin>1723593600</begin><end>1723680000</end></date_range></report_metadata>` +
`<policy_published><domain>` + domain + `</domain><p>none</p></policy_published>` + records + `</feedback>`
}
const oneRecord = `<record><row><source_ip>203.0.113.10</source_ip><count>3</count>` +
`<policy_evaluated><disposition>none</disposition><dkim>pass</dkim><spf>pass</spf></policy_evaluated></row>` +
`<identifiers><header_from>example.com</header_from></identifiers></record>`
// The XML is a stranger's: what it may make the relay store is bounded.
func TestParseAggregateBoundsWhatAReportHolds(t *testing.T) {
long := strings.Repeat("é", 4000)
got, err := ParseAggregate([]byte(reportXML(long, "example.com", oneRecord)))
if err != nil {
t.Fatal(err)
}
if len(got.Reporter) > maxFieldLen || !strings.HasPrefix(long, got.Reporter) || got.Reporter == "" {
t.Fatalf("reporter name is %d bytes, want the start of the name in at most %d", len(got.Reporter), maxFieldLen)
}
if strings.ContainsRune(got.Reporter, '�') {
t.Fatal("the name was cut in the middle of a character")
}
huge := `<record><row><source_ip>203.0.113.10</source_ip><count>9223372036854775807</count>` +
`<policy_evaluated><dkim>pass</dkim></policy_evaluated></row></record>`
got, err = ParseAggregate([]byte(reportXML("r", "example.com", huge+huge)))
if err != nil {
t.Fatal(err)
}
if got.PassCount != 2*maxRecordCount {
t.Fatalf("pass count = %d, want two records clamped to %d each", got.PassCount, maxRecordCount)
}
if _, err := ParseAggregate([]byte(reportXML("r", "example.com", strings.Repeat(oneRecord, maxReportRecords)))); err != nil {
t.Fatalf("a report of %d records was refused: %v", maxReportRecords, err)
}
if _, err := ParseAggregate([]byte(reportXML("r", "example.com", strings.Repeat(oneRecord, maxReportRecords+1)))); err == nil {
t.Fatalf("a report of %d records was accepted", maxReportRecords+1)
}
}
// A report over the bound is one the relay does not keep: dropped, not retried.
func TestIngestMessageRejectsAReportWithTooManyRecords(t *testing.T) {
st, err := store.Open(t.TempDir() + "/test.db")
if err != nil {
t.Fatal(err)
}
defer st.Close()
if _, err := st.AddDomain("example.com", "selfpost"); err != nil {
t.Fatal(err)
}
xml := reportXML("r", "example.com", strings.Repeat(oneRecord, maxReportRecords+1))
err = IngestMessage(st, strings.NewReader(gzipMessage(t, xml)), "dmarc-reports+example.com@mail.example.com", time.Now().UTC())
if !errors.Is(err, ErrRejected) {
t.Fatalf("err = %v, want ErrRejected", err)
}
}
// Reports filed under one domain do not push another domain's reports out.
func TestIngestKeepsEachDomainsReportsApart(t *testing.T) {
st, err := store.Open(t.TempDir() + "/test.db")
if err != nil {
t.Fatal(err)
}
defer st.Close()
now := time.Now().UTC()
if _, err := st.InsertDMARCReport(store.DMARCReport{
Domain: "quiet.example", Reporter: "genuine", ReportID: "1", PeriodBegin: now, PeriodEnd: now, ReceivedAt: now.Add(-time.Hour),
}); err != nil {
t.Fatal(err)
}
for i := 0; i < store.DMARCReportsMaxKeep+20; i++ {
if _, err := st.InsertDMARCReport(store.DMARCReport{
Domain: "flooded.example", Reporter: "stranger", ReportID: strings.Repeat("x", 1+i), PeriodBegin: now, PeriodEnd: now, ReceivedAt: now,
}); err != nil {
t.Fatal(err)
}
}
if err := st.PruneDMARCReports("flooded.example"); err != nil {
t.Fatal(err)
}
quiet, err := st.ListDMARCReports(store.DMARCReportScope{Domains: []string{"quiet.example"}}, 10)
if err != nil {
t.Fatal(err)
}
if len(quiet) != 1 {
t.Fatalf("the other domain has %d reports after the flood, want its 1", len(quiet))
}
flooded, err := st.ListDMARCReports(store.DMARCReportScope{Domains: []string{"flooded.example"}}, 10000)
if err != nil {
t.Fatal(err)
}
if len(flooded) > store.DMARCReportsMaxKeep {
t.Fatalf("the flooded domain keeps %d reports, want at most %d", len(flooded), store.DMARCReportsMaxKeep)
}
}
+16 -6
View File
@@ -3,6 +3,7 @@ package dmarc
import ( import (
"bytes" "bytes"
"encoding/base64" "encoding/base64"
"errors"
"fmt" "fmt"
"io" "io"
"mime" "mime"
@@ -16,6 +17,15 @@ import (
"github.com/mixeme/selfpost/internal/store" "github.com/mixeme/selfpost/internal/store"
) )
// ErrRejected marks a message that is not a report this relay keeps and never
// will be: it cannot be read as an aggregate report, or it reports on a domain
// that is not sent from here, or it came to another domain's address. Sending
// it again changes nothing, so the caller drops it. Any other error is the
// store's and worth another try.
var ErrRejected = errors.New("dmarc report rejected")
func rejected(err error) error { return fmt.Errorf("%w: %w", ErrRejected, err) }
// IngestMessage parses a raw RFC 5322 message from r and stores any DMARC // IngestMessage parses a raw RFC 5322 message from r and stores any DMARC
// aggregate attachment it finds. // aggregate attachment it finds.
// //
@@ -34,11 +44,11 @@ func IngestMessage(st *store.Store, r io.Reader, recipient string, receivedAt ti
} }
payload, err := extractAggregatePayload(raw) payload, err := extractAggregatePayload(raw)
if err != nil { if err != nil {
return err return rejected(err)
} }
parsed, err := ParseAggregate(payload) parsed, err := ParseAggregate(payload)
if err != nil { if err != nil {
return err return rejected(err)
} }
recipient = strings.ToLower(strings.TrimSpace(recipient)) recipient = strings.ToLower(strings.TrimSpace(recipient))
if err := checkReportDomain(st, parsed.Domain, recipient); err != nil { if err := checkReportDomain(st, parsed.Domain, recipient); err != nil {
@@ -74,7 +84,7 @@ func IngestMessage(st *store.Store, r io.Reader, recipient string, receivedAt ti
if _, err := st.InsertDMARCReport(rep); err != nil { if _, err := st.InsertDMARCReport(rep); err != nil {
return err return err
} }
return st.PruneDMARCReports() return st.PruneDMARCReports(rep.Domain)
} }
// checkReportDomain refuses a report whose policy domain is not one of this // checkReportDomain refuses a report whose policy domain is not one of this
@@ -82,17 +92,17 @@ func IngestMessage(st *store.Store, r io.Reader, recipient string, receivedAt ti
// delivered to. // delivered to.
func checkReportDomain(st *store.Store, domain, recipient string) error { func checkReportDomain(st *store.Store, domain, recipient string) error {
if domain == "" { if domain == "" {
return fmt.Errorf("dmarc report: no policy domain") return rejected(fmt.Errorf("no policy domain"))
} }
if tagged := TaggedDomain(recipient); tagged != "" && tagged != domain { if tagged := TaggedDomain(recipient); tagged != "" && tagged != domain {
return fmt.Errorf("dmarc report: domain %q does not match the address it was sent to (%s)", domain, recipient) return rejected(fmt.Errorf("domain %q does not match the address it was sent to (%s)", domain, recipient))
} }
ok, err := st.DomainExists(domain) ok, err := st.DomainExists(domain)
if err != nil { if err != nil {
return err return err
} }
if !ok { if !ok {
return fmt.Errorf("dmarc report: %q is not a domain configured on this relay", domain) return rejected(fmt.Errorf("%q is not a domain configured on this relay", domain))
} }
return nil return nil
} }
+50 -15
View File
@@ -9,6 +9,7 @@ import (
"io" "io"
"strings" "strings"
"time" "time"
"unicode/utf8"
) )
// ParsedReport is the structured form of a DMARC aggregate XML payload. // ParsedReport is the structured form of a DMARC aggregate XML payload.
@@ -74,6 +75,31 @@ type feedbackXML struct {
} `xml:"record"` } `xml:"record"`
} }
// What a report may hold. The XML comes from anyone who knows a report address,
// and eight megabytes of it (maxAggregateXML) fit some 160 000 empty records or
// one name of eight megabytes - every one of them a row or a table cell later.
// A real report has tens of records, a few thousand from the largest
// receivers, and no field longer than a line.
const (
maxReportRecords = 5000
maxFieldLen = 255
maxRecordCount = 1_000_000_000
)
// field trims a text value of the report and cuts it to maxFieldLen bytes, on
// a character boundary.
func field(s string) string {
s = strings.TrimSpace(s)
if len(s) <= maxFieldLen {
return s
}
s = s[:maxFieldLen]
for len(s) > 0 && !utf8.ValidString(s) {
s = s[:len(s)-1]
}
return s
}
// ParseAggregate decodes gzip- or zip-compressed, or raw, DMARC aggregate XML. // ParseAggregate decodes gzip- or zip-compressed, or raw, DMARC aggregate XML.
func ParseAggregate(raw []byte) (ParsedReport, error) { func ParseAggregate(raw []byte) (ParsedReport, error) {
data, err := decompress(raw) data, err := decompress(raw)
@@ -87,34 +113,43 @@ func ParseAggregate(raw []byte) (ParsedReport, error) {
if doc.Metadata.ReportID == "" || doc.Policy.Domain == "" { if doc.Metadata.ReportID == "" || doc.Policy.Domain == "" {
return ParsedReport{}, fmt.Errorf("dmarc xml: missing report_id or domain") return ParsedReport{}, fmt.Errorf("dmarc xml: missing report_id or domain")
} }
if len(doc.Records) > maxReportRecords {
return ParsedReport{}, fmt.Errorf("dmarc xml: %d records, more than the %d a report may hold", len(doc.Records), maxReportRecords)
}
// Cut, not refused: a domain name is at most 253 bytes, so a longer one
// names nothing this relay sends for and is turned away as such.
out := ParsedReport{ out := ParsedReport{
Reporter: strings.TrimSpace(doc.Metadata.OrgName), Reporter: field(doc.Metadata.OrgName),
ReportID: strings.TrimSpace(doc.Metadata.ReportID), ReportID: field(doc.Metadata.ReportID),
ContactEmail: strings.TrimSpace(doc.Metadata.Email), ContactEmail: field(doc.Metadata.Email),
PeriodBegin: time.Unix(doc.Metadata.DateRange.Begin, 0).UTC(), PeriodBegin: time.Unix(doc.Metadata.DateRange.Begin, 0).UTC(),
PeriodEnd: time.Unix(doc.Metadata.DateRange.End, 0).UTC(), PeriodEnd: time.Unix(doc.Metadata.DateRange.End, 0).UTC(),
Domain: strings.ToLower(strings.TrimSpace(doc.Policy.Domain)), Domain: strings.ToLower(field(doc.Policy.Domain)),
PolicyP: strings.TrimSpace(doc.Policy.P), PolicyP: field(doc.Policy.P),
PolicySP: strings.TrimSpace(doc.Policy.SP), PolicySP: field(doc.Policy.SP),
PolicyPct: doc.Policy.Pct, PolicyPct: doc.Policy.Pct,
PolicyADKIM: strings.TrimSpace(doc.Policy.ADKIM), PolicyADKIM: field(doc.Policy.ADKIM),
PolicyASPF: strings.TrimSpace(doc.Policy.ASPF), PolicyASPF: field(doc.Policy.ASPF),
} }
if out.PolicyPct == 0 { if out.PolicyPct == 0 {
out.PolicyPct = 100 out.PolicyPct = 100
} }
for _, rec := range doc.Records { for _, rec := range doc.Records {
row := ParsedRecord{ row := ParsedRecord{
SourceIP: strings.TrimSpace(rec.Row.SourceIP), SourceIP: field(rec.Row.SourceIP),
Count: rec.Row.Count, Count: rec.Row.Count,
Disposition: strings.TrimSpace(rec.Row.Policy.Disposition), Disposition: field(rec.Row.Policy.Disposition),
SPFResult: strings.TrimSpace(rec.Row.Policy.SPF), SPFResult: field(rec.Row.Policy.SPF),
DKIMResult: strings.TrimSpace(rec.Row.Policy.DKIM), DKIMResult: field(rec.Row.Policy.DKIM),
HeaderFrom: strings.TrimSpace(rec.Identifiers.HeaderFrom), HeaderFrom: field(rec.Identifiers.HeaderFrom),
} }
if row.Count <= 0 { if row.Count <= 0 {
row.Count = 1 row.Count = 1
} }
// Bounded so that the sums of a report cannot wrap around.
if row.Count > maxRecordCount {
row.Count = maxRecordCount
}
if dmarcRecordPasses(row) { if dmarcRecordPasses(row) {
out.PassCount += row.Count out.PassCount += row.Count
} else { } else {
@@ -172,12 +207,12 @@ func unzip(raw []byte) ([]byte, error) {
} }
rc, err := f.Open() rc, err := f.Open()
if err != nil { if err != nil {
return nil, fmt.Errorf("dmarc zip open %s: %w", f.Name, err) return nil, fmt.Errorf("dmarc zip open %q: %w", field(f.Name), err)
} }
data, err := io.ReadAll(io.LimitReader(rc, maxAggregateXML)) data, err := io.ReadAll(io.LimitReader(rc, maxAggregateXML))
rc.Close() rc.Close()
if err != nil { if err != nil {
return nil, fmt.Errorf("dmarc zip read %s: %w", f.Name, err) return nil, fmt.Errorf("dmarc zip read %q: %w", field(f.Name), err)
} }
return data, nil return data, nil
} }
+45 -4
View File
@@ -4,6 +4,7 @@ import (
"archive/zip" "archive/zip"
"bytes" "bytes"
"compress/gzip" "compress/gzip"
"errors"
"testing" "testing"
"time" "time"
@@ -126,8 +127,9 @@ func TestIngestMessageRejectsUnknownDomain(t *testing.T) {
} }
defer st.Close() defer st.Close()
if err := IngestMessage(st, bytes.NewBufferString(gzipMessage(t, sampleXML)), "dmarc-reports@mail.example.com", time.Now().UTC()); err == nil { err = IngestMessage(st, bytes.NewBufferString(gzipMessage(t, sampleXML)), "dmarc-reports@mail.example.com", time.Now().UTC())
t.Fatal("report for an unconfigured domain was accepted") if !errors.Is(err, ErrRejected) {
t.Fatalf("report for an unconfigured domain: err = %v, want ErrRejected", err)
} }
list, err := st.ListDMARCReports(store.DMARCReportScope{AllDomains: true}, 10) list, err := st.ListDMARCReports(store.DMARCReportScope{AllDomains: true}, 10)
if err != nil { if err != nil {
@@ -155,8 +157,47 @@ func TestIngestMessageRejectsTagMismatch(t *testing.T) {
err = IngestMessage(st, bytes.NewBufferString(gzipMessage(t, sampleXML)), err = IngestMessage(st, bytes.NewBufferString(gzipMessage(t, sampleXML)),
"dmarc-reports+other.example@mail.example.com", time.Now().UTC()) "dmarc-reports+other.example@mail.example.com", time.Now().UTC())
if err == nil { if !errors.Is(err, ErrRejected) {
t.Fatal("report was accepted at an address tagged for another domain") t.Fatalf("report at an address tagged for another domain: err = %v, want ErrRejected", err)
}
}
// What cannot be read as a report is rejected, not put off: Postfix would
// offer the same bytes again for days.
func TestIngestMessageRejectsWhatIsNotAReport(t *testing.T) {
st, err := store.Open(t.TempDir() + "/test.db")
if err != nil {
t.Fatal(err)
}
defer st.Close()
if _, err := st.AddDomain("example.com", "selfpost"); err != nil {
t.Fatal(err)
}
for name, msg := range map[string]string{
"plain text": "From: someone@example.net\r\nSubject: hello\r\n\r\nnot a report\r\n",
"no headers": "garbage",
"broken gzip": "From: a@example.net\r\nContent-Type: application/gzip\r\n\r\nnot gzip at all",
} {
err := IngestMessage(st, bytes.NewBufferString(msg), "dmarc-reports+example.com@mail.example.com", time.Now().UTC())
if !errors.Is(err, ErrRejected) {
t.Errorf("%s: err = %v, want ErrRejected", name, err)
}
}
}
// A failing store is not the report's fault: the message is to be tried again.
func TestIngestMessageStoreFailureIsNotARejection(t *testing.T) {
st, err := store.Open(t.TempDir() + "/test.db")
if err != nil {
t.Fatal(err)
}
if _, err := st.AddDomain("example.com", "selfpost"); err != nil {
t.Fatal(err)
}
st.Close()
err = IngestMessage(st, bytes.NewBufferString(gzipMessage(t, sampleXML)), "dmarc-reports+example.com@mail.example.com", time.Now().UTC())
if err == nil || errors.Is(err, ErrRejected) {
t.Fatalf("closed store: err = %v, want an error that is not ErrRejected", err)
} }
} }
+351
View File
@@ -0,0 +1,351 @@
package inbound
import (
"fmt"
"net/netip"
"os"
"path/filepath"
"sort"
"strings"
"time"
"unicode/utf8"
"github.com/mixeme/selfpost/internal/atomicfile"
"github.com/mixeme/selfpost/internal/configsafe"
"github.com/mixeme/selfpost/internal/store"
)
// ListStore is the slice of the store the filter lists need. *store.Store
// satisfies it.
type ListStore interface {
AddSpamListEntry(list, kind, value, note string) (store.SpamListEntry, error)
ListSpamListEntries() ([]store.SpamListEntry, error)
DeleteSpamListEntry(id int64) error
ListSpamListHits(marks ...string) ([]store.SpamListHit, error)
}
// Lists owns the allow and deny lists of the external spam filter: the rows in
// SQLite and the map files the filter reads. SelfPost does not consult the
// lists itself - it writes them out, six files in one directory, one entry a
// line, and the filter (rspamd, deploy/antispam/rspamd/multimap.conf) rereads
// a file when it changes. Which list wins when both match is the filter's
// configuration: deny.
type Lists struct {
store ListStore
dir string
}
// NewLists builds the filter lists over the store and the map directory
// (/data/antispam).
func NewLists(st ListStore, dir string) *Lists {
return &Lists{store: st, dir: dir}
}
// noteMaxRunes caps the operator's comment on an entry.
const noteMaxRunes = 200
// mapFiles names the file of every (list, kind). An IP map and a sender map
// are different things to the filter, and a domain entry is matched against
// another part of the sender than an address, so each has its own file.
var mapFiles = map[[2]string]string{
{store.SpamListAllow, store.SpamEntryIP}: "allow_ip.map",
{store.SpamListAllow, store.SpamEntryAddress}: "allow_from.map",
{store.SpamListAllow, store.SpamEntryDomain}: "allow_domain.map",
{store.SpamListDeny, store.SpamEntryIP}: "deny_ip.map",
{store.SpamListDeny, store.SpamEntryAddress}: "deny_from.map",
{store.SpamListDeny, store.SpamEntryDomain}: "deny_domain.map",
}
// Entries returns every entry of both lists.
func (l *Lists) Entries() ([]store.SpamListEntry, error) {
return l.store.ListSpamListEntries()
}
// Add validates an entry, stores it normalised and rewrites the maps. A
// refusal is a *ValidationError; an entry already on that list is
// store.ErrSpamListEntryExists.
func (l *Lists) Add(list, kind, rawValue, rawNote string) (store.SpamListEntry, error) {
if list != store.SpamListAllow && list != store.SpamListDeny {
return store.SpamListEntry{}, invalid("choose the allow or the deny list")
}
value, err := normalizeListValue(kind, rawValue)
if err != nil {
return store.SpamListEntry{}, err
}
note, err := normalizeNote(rawNote)
if err != nil {
return store.SpamListEntry{}, err
}
e, err := l.store.AddSpamListEntry(list, kind, value, note)
if err != nil {
return store.SpamListEntry{}, err
}
if err := l.Sync(); err != nil {
// An entry the filter never got is worse than no entry: the page
// would show a rule that does nothing.
_ = l.store.DeleteSpamListEntry(e.ID)
return store.SpamListEntry{}, err
}
return e, nil
}
// Delete removes an entry and rewrites the maps.
func (l *Lists) Delete(id int64) error {
if err := l.store.DeleteSpamListEntry(id); err != nil {
return err
}
return l.Sync()
}
// Sync writes all six map files from SQLite, each atomically, so the filter
// reads a whole old file or a whole new one. Every file is written even when
// empty: the filter's configuration names all six.
func (l *Lists) Sync() error {
entries, err := l.store.ListSpamListEntries()
if err != nil {
return err
}
lines := make(map[string][]string, len(mapFiles))
for _, name := range mapFiles {
lines[name] = nil
}
for _, e := range entries {
name, ok := mapFiles[[2]string{e.List, e.Kind}]
if !ok {
return fmt.Errorf("filter list entry %d: unknown list %q or kind %q", e.ID, e.List, e.Kind)
}
// Values were validated on the way in; this is the writer's own guard
// against a row that got into the table some other way. In a map file a
// newline ends the entry, whitespace starts its value and # a comment.
if err := configsafe.Token("filter list entry", e.Value, " \t\r\n#"); err != nil {
return err
}
lines[name] = append(lines[name], e.Value)
}
// The filter runs under another user in another container: the directory
// and the files are readable by anyone. They hold addresses, not secrets.
if err := os.MkdirAll(l.dir, 0o755); err != nil {
return fmt.Errorf("create %s: %w", l.dir, err)
}
for name, vals := range lines {
sort.Strings(vals)
var b strings.Builder
b.WriteString("# Written by SelfPost (Inbound > Filter lists). Do not edit: the next change in the panel replaces this file.\n")
for _, v := range vals {
b.WriteString(v)
b.WriteByte('\n')
}
if err := atomicfile.Write(filepath.Join(l.dir, name), []byte(b.String()), 0o644); err != nil {
return err
}
}
return nil
}
// LastWritten is when the maps were last written: the newest modification
// time of the six files, the zero time while none exists (the filter lists
// have never been touched, or the directory is gone).
func (l *Lists) LastWritten() time.Time {
var newest time.Time
for _, name := range mapFiles {
if fi, err := os.Stat(filepath.Join(l.dir, name)); err == nil && fi.ModTime().After(newest) {
newest = fi.ModTime()
}
}
return newest
}
// normalizeListValue validates one entry and returns the form it is stored and
// written in: lower case for an address or a domain, the canonical text of an
// address or a network for an IP.
func normalizeListValue(kind, raw string) (string, error) {
v := strings.ToLower(strings.TrimSpace(raw))
switch kind {
case store.SpamEntryAddress:
if v == "" {
return "", invalid("sender address is required")
}
if len(v) > 254 {
return "", invalid("sender address must be at most 254 characters")
}
at := strings.LastIndexByte(v, '@')
if at <= 0 || at >= len(v)-1 {
return "", invalid("%q is not a valid email address", v)
}
local, domain := v[:at], v[at+1:]
if local[0] == '.' || local[len(local)-1] == '.' || len(local) > 64 {
return "", invalid("%q: invalid local part", v)
}
for i := 0; i < len(local); i++ {
c := local[i]
lower := c >= 'a' && c <= 'z'
digit := c >= '0' && c <= '9'
// '=' for the rewritten senders of forwarders (SRS0=…).
if !lower && !digit && !strings.ContainsRune("._+-=", rune(c)) {
return "", invalid("%q: local part contains invalid characters", v)
}
}
if err := checkDomain(domain); err != nil {
return "", err
}
return v, nil
case store.SpamEntryDomain:
// "@example.com" is how a domain is often written in such a list.
v = strings.TrimPrefix(v, "@")
if err := checkDomain(v); err != nil {
return "", err
}
return v, nil
case store.SpamEntryIP:
if v == "" {
return "", invalid("address or network is required")
}
if strings.Contains(v, "/") {
p, err := netip.ParsePrefix(v)
if err != nil {
return "", invalid("%q is not a network in CIDR form (e.g. 192.0.2.0/24)", v)
}
if p.Addr().Is4In6() {
return "", invalid("%q: write an IPv4 network as IPv4 (e.g. 192.0.2.0/24)", v)
}
if p.Bits() == 0 {
return "", invalid("%q is every address there is", v)
}
return p.Masked().String(), nil
}
a, err := netip.ParseAddr(v)
if err != nil || a.Zone() != "" {
return "", invalid("%q is not an IP address or a network in CIDR form", v)
}
return a.Unmap().String(), nil
default:
return "", invalid("choose what the entry is: a sender address, a sender domain or a client address")
}
}
// normalizeNote trims the operator's comment and refuses what a single line of
// text cannot be.
func normalizeNote(raw string) (string, error) {
note := strings.TrimSpace(raw)
if utf8.RuneCountInString(note) > noteMaxRunes {
return "", invalid("note must be at most %d characters", noteMaxRunes)
}
for _, r := range note {
if r < 0x20 || r == 0x7f || r == utf8.RuneError {
return "", invalid("note must be one line of plain text")
}
}
return note, nil
}
// What a list leaves in the journal when it decides a message - the two ends
// of deploy/antispam/rspamd/multimap.conf this package depends on. A deny rule
// refuses with a message, which the log tailer files as the reply of a
// rejected row. An allow rule adds a symbol named after the kind of entry,
// which the journal milter keeps in the verdict of an accepted row.
const (
denyReplyMark = "this server's deny list"
allowMark = "SELFPOST_ALLOW_"
)
var allowSymbol = map[string]string{
store.SpamEntryIP: "SELFPOST_ALLOW_IP(",
store.SpamEntryAddress: "SELFPOST_ALLOW_FROM(",
store.SpamEntryDomain: "SELFPOST_ALLOW_DOMAIN(",
}
// Usage is how often an entry decided a message, as far back as the journal
// reaches, and when it last did. The zero value is an entry that never did.
type Usage struct {
Hits int
Last time.Time
}
// Usage counts, for every entry, the journal rows it decided. Nothing is
// stored for this: the journal already says which list decided a message and
// who the message was from, and an entry is matched against that the way the
// filter matches it - an address whole, a domain as the sender's domain, an
// address or network against the client's. So the count reaches back as far
// as the journal does (its retention, or less when it is full), and a row
// counts for every entry of the deciding list that fits it: the journal does
// not say which of two fitting entries the filter met first.
//
// A counter kept with each entry would not depend on the journal; the plan
// (docs/plans/inbound-antispam-panel.md, Filter list usage) says when that is
// worth a migration.
func (l *Lists) Usage() (map[int64]Usage, error) {
entries, err := l.store.ListSpamListEntries()
if err != nil {
return nil, err
}
out := make(map[int64]Usage, len(entries))
if len(entries) == 0 {
return out, nil
}
hits, err := l.store.ListSpamListHits(denyReplyMark, allowMark)
if err != nil {
return nil, err
}
type matcher struct {
id int64
list string
kind string
value string
prefix netip.Prefix
}
matchers := make([]matcher, 0, len(entries))
for _, e := range entries {
m := matcher{id: e.ID, list: e.List, kind: e.Kind, value: e.Value}
if e.Kind == store.SpamEntryIP {
if p, err := netip.ParsePrefix(e.Value); err == nil {
m.prefix = p
} else if a, err := netip.ParseAddr(e.Value); err == nil {
m.prefix = netip.PrefixFrom(a, a.BitLen())
} else {
continue // not an entry this package wrote
}
}
matchers = append(matchers, m)
}
for _, h := range hits {
from := strings.ToLower(h.From)
domain := ""
if i := strings.LastIndexByte(from, '@'); i >= 0 {
domain = from[i+1:]
}
client, clientErr := netip.ParseAddr(h.ClientIP)
client = client.Unmap()
denied := h.Decision != store.SpamDecisionAccept && strings.Contains(h.Detail, denyReplyMark)
for _, m := range matchers {
switch m.list {
case store.SpamListDeny:
if !denied {
continue
}
case store.SpamListAllow:
if h.Decision != store.SpamDecisionAccept || !strings.Contains(h.Detail, allowSymbol[m.kind]) {
continue
}
}
fits := false
switch m.kind {
case store.SpamEntryAddress:
fits = from == m.value
case store.SpamEntryDomain:
fits = domain == m.value
case store.SpamEntryIP:
fits = clientErr == nil && m.prefix.Contains(client)
}
if !fits {
continue
}
u := out[m.id]
u.Hits++
if h.CreatedAt.After(u.Last) {
u.Last = h.CreatedAt
}
out[m.id] = u
}
}
return out, nil
}
+370
View File
@@ -0,0 +1,370 @@
package inbound
import (
"errors"
"os"
"path/filepath"
"strings"
"testing"
"time"
"github.com/mixeme/selfpost/internal/store"
)
func testLists(t *testing.T) (*Lists, *store.Store, string) {
t.Helper()
st, err := store.Open(filepath.Join(t.TempDir(), "test.db"))
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { st.Close() })
dir := filepath.Join(t.TempDir(), "antispam")
return NewLists(st, dir), st, dir
}
// mapLines reads a map file without its comment line.
func mapLines(t *testing.T, dir, name string) []string {
t.Helper()
raw, err := os.ReadFile(filepath.Join(dir, name))
if err != nil {
t.Fatalf("read %s: %v", name, err)
}
var out []string
for _, line := range strings.Split(strings.TrimRight(string(raw), "\n"), "\n") {
if !strings.HasPrefix(line, "#") {
out = append(out, line)
}
}
return out
}
func TestNormalizeListValue(t *testing.T) {
for _, c := range []struct{ kind, raw, want string }{
{store.SpamEntryAddress, " Billing@Vendor.Example ", "billing@vendor.example"},
{store.SpamEntryAddress, "srs0=ab12=cd=example.org=user@forwarder.example", "srs0=ab12=cd=example.org=user@forwarder.example"},
{store.SpamEntryDomain, "@Evil.Example", "evil.example"},
{store.SpamEntryDomain, "sub.evil.example", "sub.evil.example"},
{store.SpamEntryIP, "192.0.2.7", "192.0.2.7"},
{store.SpamEntryIP, "192.0.2.77/24", "192.0.2.0/24"},
{store.SpamEntryIP, "::ffff:192.0.2.7", "192.0.2.7"},
{store.SpamEntryIP, "2001:DB8:0:0::1", "2001:db8::1"},
{store.SpamEntryIP, "2001:db8:1::5/48", "2001:db8:1::/48"},
} {
got, err := normalizeListValue(c.kind, c.raw)
if err != nil || got != c.want {
t.Errorf("normalizeListValue(%s, %q) = %q, %v; want %q", c.kind, c.raw, got, err, c.want)
}
}
for _, c := range []struct{ kind, raw string }{
{store.SpamEntryAddress, ""},
{store.SpamEntryAddress, "no-at-sign"},
{store.SpamEntryAddress, "@example.com"},
{store.SpamEntryAddress, "a@localhost"},
{store.SpamEntryAddress, "a b@example.com"},
{store.SpamEntryAddress, "a#b@example.com"},
{store.SpamEntryAddress, "a\nb@example.com"},
{store.SpamEntryAddress, ".a@example.com"},
{store.SpamEntryDomain, ""},
{store.SpamEntryDomain, "example"},
{store.SpamEntryDomain, "exa mple.com"},
{store.SpamEntryDomain, "example.com # all of it"},
{store.SpamEntryDomain, "user@example.com"},
{store.SpamEntryIP, ""},
{store.SpamEntryIP, "example.com"},
{store.SpamEntryIP, "192.0.2.300"},
{store.SpamEntryIP, "192.0.2.0/33"},
{store.SpamEntryIP, "0.0.0.0/0"},
{store.SpamEntryIP, "::/0"},
{store.SpamEntryIP, "fe80::1%eth0"},
{store.SpamEntryIP, "::ffff:192.0.2.0/120"},
{store.SpamEntryIP, "192.0.2.7 192.0.2.8"},
{"subject", "pills"},
{"", "x"},
} {
got, err := normalizeListValue(c.kind, c.raw)
var ve *ValidationError
if err == nil || !errors.As(err, &ve) {
t.Errorf("normalizeListValue(%q, %q) = %q, %v; want a ValidationError", c.kind, c.raw, got, err)
}
}
}
func TestListsAddWritesTheMaps(t *testing.T) {
l, _, dir := testLists(t)
for _, e := range []struct{ list, kind, value string }{
{store.SpamListAllow, store.SpamEntryAddress, "Billing@Vendor.Example"},
{store.SpamListAllow, store.SpamEntryAddress, "accounts@vendor.example"},
{store.SpamListDeny, store.SpamEntryDomain, "@evil.example"},
{store.SpamListDeny, store.SpamEntryIP, "198.51.100.9/24"},
// The same value on both lists is two entries: deny wins in the filter.
{store.SpamListDeny, store.SpamEntryAddress, "billing@vendor.example"},
} {
if _, err := l.Add(e.list, e.kind, e.value, " why "); err != nil {
t.Fatalf("Add(%s, %s, %q): %v", e.list, e.kind, e.value, err)
}
}
want := map[string]string{
"allow_from.map": "accounts@vendor.example billing@vendor.example",
"allow_domain.map": "",
"allow_ip.map": "",
"deny_from.map": "billing@vendor.example",
"deny_domain.map": "evil.example",
"deny_ip.map": "198.51.100.0/24",
}
for name, lines := range want {
if got := strings.Join(mapLines(t, dir, name), " "); got != lines {
t.Errorf("%s = %q, want %q", name, got, lines)
}
// The filter is another user in another container.
if info, err := os.Stat(filepath.Join(dir, name)); err != nil {
t.Errorf("stat %s: %v", name, err)
} else if info.Mode().Perm()&0o004 == 0 && os.PathSeparator == '/' {
t.Errorf("%s is mode %o, not readable by the filter", name, info.Mode().Perm())
}
}
if files, _ := os.ReadDir(dir); len(files) != len(want) {
t.Errorf("the directory holds %d files, want %d (no temp file left behind)", len(files), len(want))
}
entries, err := l.Entries()
if err != nil || len(entries) != 5 {
t.Fatalf("Entries = %d, %v", len(entries), err)
}
if entries[0].Note != "why" {
t.Errorf("note = %q, want it trimmed", entries[0].Note)
}
}
func TestListsRefusals(t *testing.T) {
l, _, dir := testLists(t)
if _, err := l.Add(store.SpamListAllow, store.SpamEntryIP, "192.0.2.7", ""); err != nil {
t.Fatalf("Add: %v", err)
}
// Written differently, the same entry.
if _, err := l.Add(store.SpamListAllow, store.SpamEntryIP, "::ffff:192.0.2.7", ""); !errors.Is(err, store.ErrSpamListEntryExists) {
t.Errorf("the same address again: %v, want ErrSpamListEntryExists", err)
}
var ve *ValidationError
for name, add := range map[string]func() error{
"an unknown list": func() error { _, err := l.Add("block", store.SpamEntryIP, "192.0.2.8", ""); return err },
"a bad value": func() error { _, err := l.Add(store.SpamListDeny, store.SpamEntryIP, "nope", ""); return err },
"a long note": func() error {
_, err := l.Add(store.SpamListDeny, store.SpamEntryIP, "192.0.2.8", strings.Repeat("x", noteMaxRunes+1))
return err
},
"a note of two lines": func() error {
_, err := l.Add(store.SpamListDeny, store.SpamEntryIP, "192.0.2.8", "one\ntwo")
return err
},
} {
if err := add(); !errors.As(err, &ve) {
t.Errorf("%s: %v, want a ValidationError", name, err)
}
}
if got := strings.Join(mapLines(t, dir, "deny_ip.map"), " "); got != "" {
t.Errorf("deny_ip.map = %q after refusals only", got)
}
if entries, _ := l.Entries(); len(entries) != 1 {
t.Errorf("entries = %d, want the first one alone", len(entries))
}
}
func TestListsDelete(t *testing.T) {
l, _, dir := testLists(t)
e, err := l.Add(store.SpamListDeny, store.SpamEntryDomain, "evil.example", "")
if err != nil {
t.Fatalf("Add: %v", err)
}
if err := l.Delete(e.ID); err != nil {
t.Fatalf("Delete: %v", err)
}
if got := mapLines(t, dir, "deny_domain.map"); len(got) != 0 {
t.Errorf("deny_domain.map = %v after the delete", got)
}
if err := l.Delete(e.ID); !errors.Is(err, store.ErrSpamListEntryNotFound) {
t.Errorf("a second delete: %v", err)
}
}
// An entry the filter never got must not stay on the page.
func TestListsAddRollsBackWhenTheMapsCannotBeWritten(t *testing.T) {
st, err := store.Open(filepath.Join(t.TempDir(), "test.db"))
if err != nil {
t.Fatal(err)
}
defer st.Close()
// A file where the directory should be.
blocked := filepath.Join(t.TempDir(), "antispam")
if err := os.WriteFile(blocked, nil, 0o600); err != nil {
t.Fatal(err)
}
l := NewLists(st, blocked)
if _, err := l.Add(store.SpamListDeny, store.SpamEntryDomain, "evil.example", ""); err == nil {
t.Fatal("Add succeeded with nowhere to write the maps")
}
if entries, _ := l.Entries(); len(entries) != 0 {
t.Errorf("the entry stayed: %+v", entries)
}
}
// The writer's own guard: a row that got into the table some other way is not
// written into a map.
func TestListsSyncRefusesAnUnsafeRow(t *testing.T) {
l, st, dir := testLists(t)
if _, err := st.AddSpamListEntry(store.SpamListDeny, store.SpamEntryDomain, "evil.example\ngood.example", ""); err != nil {
t.Fatalf("AddSpamListEntry: %v", err)
}
if err := l.Sync(); err == nil {
t.Fatal("Sync wrote a value with a newline in it")
}
if _, err := os.Stat(filepath.Join(dir, "deny_domain.map")); err == nil {
t.Error("a map file was written")
}
}
// With nothing on the lists all six files are there, empty.
func TestListsSyncWritesEmptyMaps(t *testing.T) {
l, _, dir := testLists(t)
if err := l.Sync(); err != nil {
t.Fatalf("Sync: %v", err)
}
for _, name := range mapFiles {
if got := mapLines(t, dir, name); len(got) != 0 {
t.Errorf("%s = %v", name, got)
}
}
}
func TestListsLastWritten(t *testing.T) {
l, _, dir := testLists(t)
if got := l.LastWritten(); !got.IsZero() {
t.Fatalf("LastWritten before anything was written = %v", got)
}
if err := l.Sync(); err != nil {
t.Fatal(err)
}
if got := l.LastWritten(); got.IsZero() || time.Since(got) > time.Minute {
t.Errorf("LastWritten after a sync = %v", got)
}
// The newest of the six counts, whichever it is.
old := time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC)
for _, name := range mapFiles {
if err := os.Chtimes(filepath.Join(dir, name), old, old); err != nil {
t.Fatal(err)
}
}
newest := old.Add(48 * time.Hour)
if err := os.Chtimes(filepath.Join(dir, "deny_domain.map"), newest, newest); err != nil {
t.Fatal(err)
}
if got := l.LastWritten(); !got.Equal(newest) {
t.Errorf("LastWritten = %v, want %v", got, newest)
}
// A file that is not one of the six does not count.
other := newest.Add(time.Hour)
if err := os.WriteFile(filepath.Join(dir, "notes.txt"), nil, 0o644); err != nil {
t.Fatal(err)
}
if err := os.Chtimes(filepath.Join(dir, "notes.txt"), other, other); err != nil {
t.Fatal(err)
}
if got := l.LastWritten(); !got.Equal(newest) {
t.Errorf("LastWritten counts a stray file: %v", got)
}
}
// How often an entry decided a message is read off the journal: the list that
// decided, and whether the entry fits who the message was from.
func TestListsUsage(t *testing.T) {
l, st, _ := testLists(t)
add := func(list, kind, value string) int64 {
t.Helper()
e, err := l.Add(list, kind, value, "")
if err != nil {
t.Fatalf("Add(%s, %s, %s): %v", list, kind, value, err)
}
return e.ID
}
denyDomain := add(store.SpamListDeny, store.SpamEntryDomain, "evil.example")
denyAddr := add(store.SpamListDeny, store.SpamEntryAddress, "spammer@evil.example")
denyNet := add(store.SpamListDeny, store.SpamEntryIP, "198.51.100.0/24")
denyV6 := add(store.SpamListDeny, store.SpamEntryIP, "2001:db8::/32")
allowAddr := add(store.SpamListAllow, store.SpamEntryAddress, "billing@vendor.example")
allowDomain := add(store.SpamListAllow, store.SpamEntryDomain, "vendor.example")
allowIP := add(store.SpamListAllow, store.SpamEntryIP, "203.0.113.5")
unused := add(store.SpamListAllow, store.SpamEntryDomain, "quiet.example")
const denied = "5.7.1 Refused by this server's deny list"
at := time.Date(2026, 10, 1, 0, 0, 0, 0, time.UTC)
hour := 0
row := func(from, ip, decision, engine, detail string) {
t.Helper()
hour++
err := st.InsertInboundSpam(store.InboundSpamEntry{
InboundDomain: "in.example.com", To: "a@in.example.com", From: from, ClientIP: ip,
Decision: decision, Engine: engine, Detail: detail, At: at.Add(time.Duration(hour) * time.Hour),
})
if err != nil {
t.Fatal(err)
}
}
// hour 1: the address and its domain are both on the deny list, and both fit.
row("Spammer@Evil.Example", "192.0.2.9", store.SpamDecisionReject, store.SpamEngineRspamd, denied)
// hour 2: another address of the domain.
row("other@evil.example", "192.0.2.9", store.SpamDecisionReject, store.SpamEngineRspamd, denied)
// hour 3, 4: the networks, v4 (as Postfix may log it, mapped) and v6.
row("x@elsewhere.example", "::ffff:198.51.100.77", store.SpamDecisionReject, store.SpamEngineRspamd, denied)
row("x@elsewhere.example", "2001:db8::1", store.SpamDecisionReject, store.SpamEngineRspamd, denied)
// hour 5: a subdomain is not the domain.
row("x@sub.evil.example", "192.0.2.9", store.SpamDecisionReject, store.SpamEngineRspamd, denied)
// hour 6: allowed by its address and its domain - two symbols, two entries.
row("billing@vendor.example", "192.0.2.50", store.SpamDecisionAccept, store.SpamEngineRspamd,
"no action, score -199.00 / 15.00; SELFPOST_ALLOW_DOMAIN(-100.00) SELFPOST_ALLOW_FROM(-100.00)")
// hour 7: allowed by its domain alone.
row("accounts@vendor.example", "192.0.2.50", store.SpamDecisionAccept, store.SpamEngineRspamd,
"no action, score -99.00 / 15.00; SELFPOST_ALLOW_DOMAIN(-100.00)")
// hour 8: allowed by the client address; the sender's domain is not why.
row("x@quiet.example", "203.0.113.5", store.SpamDecisionAccept, store.SpamEngineRspamd,
"no action, score -99.00 / 15.00; SELFPOST_ALLOW_IP(-100.00)")
// Nobody's: the filter's own refusal, an accept with no list symbol, a
// refusal by Postfix, and a sender who spells the mark in his own address.
row("spammer@evil.example", "192.0.2.9", store.SpamDecisionReject, store.SpamEngineRspamd, "5.7.1 Spam message rejected")
row("billing@vendor.example", "192.0.2.50", store.SpamDecisionAccept, store.SpamEngineRspamd, "no action, score 0.10 / 15.00")
row("spammer@evil.example", "198.51.100.1", store.SpamDecisionReject, store.SpamEnginePostfix, "550 5.1.1 <nobody@in.example.com>: this server's deny list")
row("billing@vendor.example", "203.0.113.5", store.SpamDecisionAccept, store.SpamEngineSelfPost, "")
usage, err := l.Usage()
if err != nil {
t.Fatalf("Usage: %v", err)
}
for name, c := range map[string]struct {
id int64
hits int
last int // hour
}{
"deny domain": {denyDomain, 2, 2}, "deny address": {denyAddr, 1, 1},
"deny network": {denyNet, 1, 3}, "deny v6 network": {denyV6, 1, 4},
"allow address": {allowAddr, 1, 6}, "allow domain": {allowDomain, 2, 7}, "allow ip": {allowIP, 1, 8},
"unused": {unused, 0, 0},
} {
got := usage[c.id]
wantLast := time.Time{}
if c.hits > 0 {
wantLast = at.Add(time.Duration(c.last) * time.Hour)
}
if got.Hits != c.hits || !got.Last.Equal(wantLast) {
t.Errorf("%s: %d hits, last %v; want %d, %v", name, got.Hits, got.Last, c.hits, wantLast)
}
}
}
func TestListsUsageWithNoEntries(t *testing.T) {
l, _, _ := testLists(t)
if usage, err := l.Usage(); err != nil || len(usage) != 0 {
t.Errorf("Usage = %v, %v", usage, err)
}
}
+231
View File
@@ -0,0 +1,231 @@
package logtail
import (
"context"
"log"
"regexp"
"strings"
"time"
"github.com/mixeme/selfpost/internal/store"
)
// InboundStore is what the tailer needs of the inbound antispam journal:
// writing the rows for refused mail, knowing which domains are inbound, and
// pruning. *store.Store satisfies it.
type InboundStore interface {
InsertInboundSpam(e store.InboundSpamEntry) error
IsInboundDomain(name string) (bool, error)
PruneInboundSpam(cutoff time.Time, keep int) (int64, error)
}
// Inbound turns on the inbound half of the tailer: journal rows for mail that
// port 25 refused, and the journal's retention. Run takes nil when the inbound
// relay is off.
type Inbound struct {
Store InboundStore
Retention RetentionDays
}
// The inbound journal milter writes a row for mail that was accepted; mail
// that was refused never reaches it, and what is known about it is one line
// of mail.log:
//
// postfix/inbound/smtpd[41]: NOQUEUE: reject: RCPT from mx.example.net[203.0.113.9]: 550 5.1.1 <nobody@in.example.com>: Recipient address rejected: User unknown in relay recipient table; from=<a@example.net> to=<nobody@in.example.com> proto=ESMTP helo=<mx.example.net>
// postfix/cleanup[52]: 4A1B2C3D: milter-reject: END-OF-MESSAGE from mx.example.net[203.0.113.9]: 5.7.1 Spam message rejected; from=<a@example.net> to=<b@in.example.com> proto=ESMTP helo=<mx.example.net>
//
// The first is Postfix's own policy, written by the port-25 smtpd, which
// build/postfix-config.sh names postfix/inbound so that its lines are told
// from those of the submission ports. The second is the filter's verdict at
// end-of-message, which cleanup(8) writes under one name for every listener -
// so it counts only for a queue id the port-25 smtpd was seen to open
// (inboundClientRe).
//
// Both are matched from the head of the line, after the timestamp and host
// name: the tail of such a line is the sender's own words (HELO name,
// addresses), and a pattern that could start anywhere would let him write
// journal rows.
var (
inboundRejectRe = regexp.MustCompile(`^\S+ postfix/(inbound/smtpd|cleanup)\[\d+\]: ([0-9A-Za-z]+): (reject|milter-reject|milter-hold|milter-discard): \S+ from [^\[\s]*\[([0-9A-Fa-f:.]+)\]: (.*); from=<([^>]*)> to=<([^>]*)> proto=\S+(?: helo=<[^>]*>)?$`)
inboundClientRe = regexp.MustCompile(`^\S+ postfix/inbound/smtpd\[\d+\]: ([0-9A-Za-z]+): client=`)
)
// A row holds one line of reason; Postfix's reply text carries the far end's
// bytes and is cut and reduced to printable ASCII like any of them.
const inboundDetailMax = 255
// inboundQueueIDs is how many port-25 queue ids are remembered while waiting
// for cleanup's verdict line on them. A message is in that state for the
// seconds its DATA takes; the cap only bounds a flood.
const inboundQueueIDs = 4096
// inboundJournal is the per-run state of the inbound half.
type inboundJournal struct {
st InboundStore
// port25 holds the queue ids the port-25 smtpd opened and cleanup has not
// reported on yet; order is their arrival, for dropping the oldest.
port25 map[string]struct{}
order []string
}
func newInboundJournal(st InboundStore) *inboundJournal {
return &inboundJournal{st: st, port25: make(map[string]struct{})}
}
// handle looks at one mail.log line and writes a journal row when it is a
// refusal of mail for an inbound domain. Errors are logged; the tailer goes on.
func (j *inboundJournal) handle(line string, now time.Time) {
at, rest := lineTime(line, now)
if m := inboundClientRe.FindStringSubmatch(rest); m != nil {
j.remember(m[1])
return
}
m := inboundRejectRe.FindStringSubmatch(rest)
if m == nil {
return
}
program, queueID, kind, clientIP, reason, from, to := m[1], m[2], m[3], m[4], m[5], m[6], m[7]
if program == "cleanup" {
if _, ok := j.port25[queueID]; !ok {
return // a submission-port message, or one opened before this run
}
delete(j.port25, queueID)
}
// Only mail for an inbound domain is the relay's. "Relay access denied"
// for a stranger's domain is every scanner on the internet, and a line
// with no recipient (size limit at MAIL) belongs to no domain.
domain := ""
if i := strings.LastIndexByte(to, '@'); i >= 0 {
domain = strings.ToLower(to[i+1:])
}
if domain == "" {
return
}
ok, err := j.st.IsInboundDomain(domain)
if err != nil {
log.Printf("log-tailer: inbound journal: look up %s: %v", domain, err)
return
}
if !ok {
return
}
e := store.InboundSpamEntry{
InboundDomain: domain,
ClientIP: clientIP,
From: from,
To: to,
Decision: store.SpamDecisionReject,
Engine: store.SpamEngineRspamd,
Detail: printable(reason, inboundDetailMax),
At: at,
}
switch {
case kind == "reject":
e.Engine = store.SpamEnginePostfix
case kind == "milter-hold":
e.Decision = store.SpamDecisionQuarantine
}
// "450 4.7.1 …" from Postfix, "4.7.1 …" from a milter: either way a reply
// that starts with 4 asks the sender to come back.
if e.Decision == store.SpamDecisionReject && strings.HasPrefix(reason, "4") {
e.Decision = store.SpamDecisionTempfail
}
if err := j.st.InsertInboundSpam(e); err != nil {
log.Printf("log-tailer: inbound journal: record %s -> %s: %v", from, to, err)
}
}
func (j *inboundJournal) remember(queueID string) {
if _, ok := j.port25[queueID]; ok {
return
}
if len(j.order) >= inboundQueueIDs {
delete(j.port25, j.order[0])
j.order = j.order[1:]
}
j.port25[queueID] = struct{}{}
j.order = append(j.order, queueID)
}
// printable cuts s to max bytes and replaces everything outside printable
// ASCII.
func printable(s string, max int) string {
if len(s) > max {
s = s[:max]
}
b := []byte(s)
for i, c := range b {
if c < 0x20 || c > 0x7e {
b[i] = '?'
}
}
return string(b)
}
// lineTime reads the time at the head of a mail.log line and returns it with
// the rest of the line. A refusal is dated by its line and not by when the
// tailer got to it: after the panel was down the two are hours apart. The
// syslog format carries neither year nor zone - the year is taken from now
// (a stamp that would then lie in the future is last year's), the zone is the
// container's, which is what postlogd wrote it in. A line with no readable
// stamp is dated now.
func lineTime(line string, now time.Time) (time.Time, string) {
if m := isoStampRe.FindStringSubmatch(line); m != nil {
rest := strings.TrimSpace(line[len(m[0]):])
if t, err := time.Parse(time.RFC3339Nano, strings.TrimSpace(m[0])); err == nil {
return t.UTC(), rest
}
return now, rest
}
if m := syslogStampRe.FindStringSubmatch(line); m != nil {
rest := strings.TrimSpace(line[len(m[0]):])
stamp := strings.Join(strings.Fields(m[1]), " ")
t, err := time.ParseInLocation("Jan 2 15:04:05", stamp, time.Local)
if err != nil {
return now, rest
}
t = t.AddDate(now.In(time.Local).Year(), 0, 0)
if t.After(now.Add(24 * time.Hour)) {
t = t.AddDate(-1, 0, 0)
}
return t.UTC(), rest
}
return now, line
}
// The journal keeps at most this many rows whatever their age: refusals are
// written at the rate strangers knock on port 25.
const inboundSpamLogMaxRows = 10000
// inboundRetentionLoop prunes the inbound journal at once and then on the
// cadence of the send log.
func inboundRetentionLoop(ctx context.Context, in *Inbound) {
prune := func() {
days := in.Retention()
if days <= 0 {
days = store.InboundSpamLogRetentionDaysDefault
}
n, err := in.Store.PruneInboundSpam(time.Now().UTC().AddDate(0, 0, -days), inboundSpamLogMaxRows)
if err != nil {
log.Printf("log-tailer: inbound journal prune: %v", err)
return
}
if n > 0 {
log.Printf("log-tailer: pruned %d inbound journal rows (older than %d days or beyond %d rows)", n, days, inboundSpamLogMaxRows)
}
}
prune()
t := time.NewTicker(retentionInterval)
defer t.Stop()
for {
select {
case <-ctx.Done():
return
case <-t.C:
prune()
}
}
}
+218
View File
@@ -0,0 +1,218 @@
package logtail
import (
"context"
"errors"
"strings"
"testing"
"time"
"github.com/mixeme/selfpost/internal/store"
)
type fakeInboundStore struct {
domains map[string]bool
rows []store.InboundSpamEntry
lookupErr error
pruned []int
}
func (f *fakeInboundStore) InsertInboundSpam(e store.InboundSpamEntry) error {
f.rows = append(f.rows, e)
return nil
}
func (f *fakeInboundStore) IsInboundDomain(name string) (bool, error) {
if f.lookupErr != nil {
return false, f.lookupErr
}
return f.domains[name], nil
}
func (f *fakeInboundStore) PruneInboundSpam(_ time.Time, keep int) (int64, error) {
f.pruned = append(f.pruned, keep)
return 0, nil
}
func inboundFixture() (*inboundJournal, *fakeInboundStore) {
st := &fakeInboundStore{domains: map[string]bool{"in.example.com": true}}
return newInboundJournal(st), st
}
var tailNow = time.Date(2026, 10, 11, 18, 0, 0, 0, time.UTC)
const (
head = "2026-10-11T12:00:05.123456+00:00 mail "
unknownRcp = head + "postfix/inbound/smtpd[41]: NOQUEUE: reject: RCPT from mx.example.net[203.0.113.9]: 550 5.1.1 <nobody@in.example.com>: Recipient address rejected: User unknown in relay recipient table; from=<a@example.net> to=<nobody@in.example.com> proto=ESMTP helo=<mx.example.net>"
)
// Postfix's own refusal of a recipient at an inbound domain is a row, dated by
// its line.
func TestInboundRejectByPostfix(t *testing.T) {
j, st := inboundFixture()
j.handle(unknownRcp, tailNow)
if len(st.rows) != 1 {
t.Fatalf("rows = %+v", st.rows)
}
want := store.InboundSpamEntry{
InboundDomain: "in.example.com", ClientIP: "203.0.113.9",
From: "a@example.net", To: "nobody@in.example.com",
Decision: store.SpamDecisionReject, Engine: store.SpamEnginePostfix,
Detail: "550 5.1.1 <nobody@in.example.com>: Recipient address rejected: User unknown in relay recipient table",
At: time.Date(2026, 10, 11, 12, 0, 5, 123456000, time.UTC),
}
if st.rows[0] != want {
t.Errorf("row = %+v\nwant %+v", st.rows[0], want)
}
}
func TestInboundTempfailByPostfix(t *testing.T) {
j, st := inboundFixture()
j.handle(head+"postfix/inbound/smtpd[41]: NOQUEUE: reject: RCPT from unknown[2001:db8::9]: 450 4.1.8 <a@nx.example>: Sender address rejected: Domain not found; from=<a@nx.example> to=<B@IN.example.com> proto=ESMTP helo=<x>", tailNow)
if len(st.rows) != 1 || st.rows[0].Decision != store.SpamDecisionTempfail || st.rows[0].ClientIP != "2001:db8::9" || st.rows[0].InboundDomain != "in.example.com" {
t.Errorf("rows = %+v", st.rows)
}
}
// The filter's verdict is written by cleanup, under one name for every
// listener: it is the journal's only for a queue id port 25 opened.
func TestInboundMilterVerdictNeedsAPort25QueueID(t *testing.T) {
const verdict = "postfix/cleanup[52]: 4A1B2C3D: milter-reject: END-OF-MESSAGE from mx.example.net[203.0.113.9]: 5.7.1 Spam message rejected; from=<a@example.net> to=<b@in.example.com> proto=ESMTP helo=<mx.example.net>"
j, st := inboundFixture()
// Opened by a submission port: postfix/smtpd, not postfix/inbound/smtpd.
j.handle(head+"postfix/smtpd[20]: 4A1B2C3D: client=app.example.com[203.0.113.4], sasl_method=PLAIN, sasl_username=app", tailNow)
j.handle(head+verdict, tailNow)
if len(st.rows) != 0 {
t.Fatalf("a submission-port verdict was journalled: %+v", st.rows)
}
j.handle(head+"postfix/inbound/smtpd[41]: 4A1B2C3D: client=mx.example.net[203.0.113.9]", tailNow)
j.handle(head+verdict, tailNow)
if len(st.rows) != 1 {
t.Fatalf("rows = %+v", st.rows)
}
got := st.rows[0]
if got.Decision != store.SpamDecisionReject || got.Engine != store.SpamEngineRspamd || got.Detail != "5.7.1 Spam message rejected" {
t.Errorf("row = %+v", got)
}
// One verdict per message: the id is forgotten with it.
j.handle(head+verdict, tailNow)
if len(st.rows) != 1 {
t.Errorf("the same queue id was journalled twice")
}
}
func TestInboundMilterDecisions(t *testing.T) {
for kind, c := range map[string]struct{ reply, decision string }{
"milter-reject": {"4.7.1 Try again later", store.SpamDecisionTempfail},
"milter-hold": {"milter triggers HOLD action", store.SpamDecisionQuarantine},
"milter-discard": {"milter triggers DISCARD action", store.SpamDecisionReject},
} {
j, st := inboundFixture()
j.handle(head+"postfix/inbound/smtpd[41]: Q1: client=mx.example.net[203.0.113.9]", tailNow)
j.handle(head+"postfix/cleanup[52]: Q1: "+kind+": END-OF-MESSAGE from mx.example.net[203.0.113.9]: "+c.reply+"; from=<a@example.net> to=<b@in.example.com> proto=ESMTP helo=<mx.example.net>", tailNow)
if len(st.rows) != 1 || st.rows[0].Decision != c.decision || st.rows[0].Engine != store.SpamEngineRspamd {
t.Errorf("%s: rows = %+v", kind, st.rows)
}
}
}
// What is not the relay's mail, or is not a refusal at all, leaves no row.
func TestInboundIgnores(t *testing.T) {
for name, line := range map[string]string{
"another domain (relay access denied)": head + "postfix/inbound/smtpd[41]: NOQUEUE: reject: RCPT from x[203.0.113.9]: 554 5.7.1 <c@elsewhere.example>: Relay access denied; from=<a@example.net> to=<c@elsewhere.example> proto=ESMTP helo=<x>",
"no recipient (size limit at MAIL)": head + "postfix/inbound/smtpd[41]: NOQUEUE: reject: MAIL from x[203.0.113.9]: 552 5.3.4 Message size exceeds fixed limit; proto=ESMTP helo=<x>",
"a submission port": head + "postfix/smtpd[20]: NOQUEUE: reject: RCPT from x[203.0.113.9]: 553 5.7.1 <b@in.example.com>: Sender address rejected: not owned by user app; from=<a@example.net> to=<b@in.example.com> proto=ESMTP helo=<x>",
"a delivery line": head + "postfix/smtp[26]: 41E862C00D9E: to=<b@in.example.com>, relay=mx[203.0.113.9]:25, dsn=2.0.0, status=sent (250 OK)",
"a connect line": head + "postfix/inbound/smtpd[41]: connect from mx.example.net[203.0.113.9]",
// The HELO name is the sender's; a line cannot be started from inside one.
"a refusal spelled inside another line": head + "postfix/inbound/smtpd[41]: warning: hostname x does not resolve: mail postfix/inbound/smtpd[1]: NOQUEUE: reject: RCPT from x[203.0.113.9]: 550 no; from=<a@example.net> to=<b@in.example.com> proto=ESMTP helo=<x>",
"an address with a bracket in it": head + "postfix/inbound/smtpd[41]: NOQUEUE: reject: RCPT from x[203.0.113.9]: 550 no; from=<a> to=<v@in.example.com> proto=X helo=<y>@example.net> to=<b@elsewhere.example> proto=ESMTP helo=<x>",
} {
j, st := inboundFixture()
j.handle(line, tailNow)
if len(st.rows) != 0 {
t.Errorf("%s: journalled %+v", name, st.rows)
}
}
}
func TestInboundDetailIsCutAndPrintable(t *testing.T) {
j, st := inboundFixture()
j.handle(head+"postfix/inbound/smtpd[41]: NOQUEUE: reject: RCPT from x[203.0.113.9]: 550 5.7.1 caf\xc3\xa9 \x1b[31m"+strings.Repeat("x", 400)+"; from=<a@example.net> to=<b@in.example.com> proto=ESMTP helo=<x>", tailNow)
if len(st.rows) != 1 {
t.Fatalf("rows = %+v", st.rows)
}
d := st.rows[0].Detail
if len(d) != inboundDetailMax || !strings.HasPrefix(d, "550 5.7.1 caf?? ?[31mxxx") {
t.Errorf("detail (%d bytes) = %q", len(d), d)
}
}
func TestInboundLookupFailureLeavesNoRow(t *testing.T) {
j, st := inboundFixture()
st.lookupErr = errors.New("boom")
j.handle(unknownRcp, tailNow)
if len(st.rows) != 0 {
t.Errorf("rows = %+v", st.rows)
}
}
// The queue ids waiting for a verdict are bounded.
func TestInboundQueueIDsAreBounded(t *testing.T) {
j, _ := inboundFixture()
for i := 0; i < inboundQueueIDs+10; i++ {
j.remember("Q" + strings.Repeat("A", i%7) + string(rune('0'+i%10)) + time.Duration(i).String())
}
if len(j.port25) != inboundQueueIDs || len(j.order) != inboundQueueIDs {
t.Errorf("remembered %d ids (%d in order), want %d", len(j.port25), len(j.order), inboundQueueIDs)
}
}
func TestLineTime(t *testing.T) {
now := time.Date(2026, 1, 2, 10, 0, 0, 0, time.UTC)
for _, c := range []struct {
line string
want time.Time
rest string
}{
{"2026-01-02T09:00:00.5+03:00 mail postfix/x: y", time.Date(2026, 1, 2, 6, 0, 0, 500000000, time.UTC), "mail postfix/x: y"},
{"Jan 2 09:30:00 mail postfix/x: y", time.Date(2026, 1, 2, 9, 30, 0, 0, time.Local).UTC(), "mail postfix/x: y"},
// December read in January is last year's.
{"Dec 31 23:59:00 mail postfix/x: y", time.Date(2025, 12, 31, 23, 59, 0, 0, time.Local).UTC(), "mail postfix/x: y"},
{"no stamp here", now, "no stamp here"},
} {
got, rest := lineTime(c.line, now)
if !got.Equal(c.want) || rest != c.rest {
t.Errorf("lineTime(%q) = %v, %q; want %v, %q", c.line, got, rest, c.want, c.rest)
}
}
}
func TestInboundRetentionPrunesAtStart(t *testing.T) {
old := retentionInterval
retentionInterval = time.Hour
defer func() { retentionInterval = old }()
st := &fakeInboundStore{}
ctx, cancel := contextWithCancel()
done := make(chan struct{})
go func() {
inboundRetentionLoop(ctx, &Inbound{Store: st, Retention: func() int { return 0 }})
close(done)
}()
deadline := time.Now().Add(2 * time.Second)
for len(st.pruned) == 0 && time.Now().Before(deadline) {
time.Sleep(10 * time.Millisecond)
}
cancel()
<-done
if len(st.pruned) != 1 || st.pruned[0] != inboundSpamLogMaxRows {
t.Errorf("prune calls = %v", st.pruned)
}
}
func contextWithCancel() (context.Context, context.CancelFunc) {
return context.WithCancel(context.Background())
}
+12 -2
View File
@@ -107,9 +107,16 @@ func parseDelivery(line string) (queueID, recipient, status string, ok bool) {
// Run follows path and updates send-log statuses until ctx is cancelled, while // Run follows path and updates send-log statuses until ctx is cancelled, while
// a background sweep prunes rows older than retention(). Reading resumes at // a background sweep prunes rows older than retention(). Reading resumes at
// the offset the previous run persisted, so a restart parses the delivery lines // the offset the previous run persisted, so a restart parses the delivery lines
// written while the panel was down. It returns nil on a clean shutdown. // written while the panel was down. With inbound set it also writes the
func Run(ctx context.Context, path string, st StatusStore, retention RetentionDays) error { // inbound journal's rows for refused mail and prunes that journal
// (inbound.go). It returns nil on a clean shutdown.
func Run(ctx context.Context, path string, st StatusStore, retention RetentionDays, inbound *Inbound) error {
go retentionLoop(ctx, st, retention) go retentionLoop(ctx, st, retention)
var journal *inboundJournal
if inbound != nil {
go inboundRetentionLoop(ctx, inbound)
journal = newInboundJournal(inbound.Store)
}
// The reconcile sweep must not run against a backlog the tailer has not // The reconcile sweep must not run against a backlog the tailer has not
// read yet: on a restart the log holds the very lines that resolve the rows // read yet: on a restart the log holds the very lines that resolve the rows
@@ -119,6 +126,9 @@ func Run(ctx context.Context, path string, st StatusStore, retention RetentionDa
go reconcileLoop(ctx, st, caughtUp) go reconcileLoop(ctx, st, caughtUp)
return follow(ctx, path, &tracker{st: st, path: path}, caughtUp, func(line string) { return follow(ctx, path, &tracker{st: st, path: path}, caughtUp, func(line string) {
if journal != nil {
journal.handle(line, time.Now().UTC())
}
queueID, recipient, status, ok := parseDelivery(line) queueID, recipient, status, ok := parseDelivery(line)
if !ok { if !ok {
return return
+2 -2
View File
@@ -212,7 +212,7 @@ func TestFollowTailsAndRotates(t *testing.T) {
cs := &captureStore{} cs := &captureStore{}
ctx, cancel := context.WithCancel(context.Background()) ctx, cancel := context.WithCancel(context.Background())
done := make(chan error, 1) done := make(chan error, 1)
go func() { done <- Run(ctx, path, cs, func() int { return 90 }) }() go func() { done <- Run(ctx, path, cs, func() int { return 90 }, nil) }()
// Give follow() time to open at EOF (it seeks to end immediately on start, // Give follow() time to open at EOF (it seeks to end immediately on start,
// so the seed line above is ignored), then append a delivery line. // so the seed line above is ignored), then append a delivery line.
@@ -295,7 +295,7 @@ func startRun(t *testing.T, path string, cs *captureStore) func() {
t.Helper() t.Helper()
ctx, cancel := context.WithCancel(context.Background()) ctx, cancel := context.WithCancel(context.Background())
done := make(chan error, 1) done := make(chan error, 1)
go func() { done <- Run(ctx, path, cs, func() int { return 90 }) }() go func() { done <- Run(ctx, path, cs, func() int { return 90 }, nil) }()
// follow() opens and seeks on start; give it a moment before the caller // follow() opens and seeks on start; give it a moment before the caller
// appends, so the append is not raced by the initial open. // appends, so the append is not raced by the initial open.
time.Sleep(50 * time.Millisecond) time.Sleep(50 * time.Millisecond)
+246
View File
@@ -0,0 +1,246 @@
package milter
import (
"context"
"log"
"net"
"net/textproto"
"regexp"
"strings"
"github.com/emersion/go-milter"
"github.com/mixeme/selfpost/internal/mailhdr"
"github.com/mixeme/selfpost/internal/store"
)
// InboundStore is what the inbound journal milter needs: the journal itself
// and the list of inbound domains, since port 25 also takes mail that is not
// the relay's. *store.Store satisfies it; tests substitute a fake.
type InboundStore interface {
InsertInboundSpam(e store.InboundSpamEntry) error
IsInboundDomain(name string) (bool, error)
}
// inboundSession is the inbound journal milter: the last milter on port 25,
// after the optional external filter. It writes one journal row per recipient
// of a message that reached end-of-message - that is, one the filter did not
// refuse; refusals never get this far and are read from mail.log instead.
//
// Like the send-log milter it only watches: every callback continues or
// accepts, and Postfix attaches the socket with default_action=accept, so
// neither an error here nor the panel being down holds up inbound mail.
type inboundSession struct {
milter.NoOpMilter
rec InboundStore
// filter says whether an external filter is attached in front of this
// milter. Without one the verdict headers below can only have been written
// by the sender, and are not read.
filter bool
clientIP string // once per connection
from string
rcpts []string
subject string
action string // X-Rspamd-Action
result string // X-Spamd-Result
seen bool // any verdict header of the filter
}
func (s *inboundSession) Connect(host, family string, port uint16, addr net.IP, m *milter.Modifier) (milter.Response, error) {
if addr != nil {
s.clientIP = addr.String()
}
return milter.RespContinue, nil
}
// MailFrom starts a message: a connection may carry several.
func (s *inboundSession) MailFrom(from string, m *milter.Modifier) (milter.Response, error) {
s.reset()
s.from = cleanAddress(from)
return milter.RespContinue, nil
}
func (s *inboundSession) RcptTo(rcpt string, m *milter.Modifier) (milter.Response, error) {
s.rcpts = append(s.rcpts, cleanAddress(rcpt))
return milter.RespContinue, nil
}
// Header keeps the first Subject and the filter's verdict. Postfix hands the
// message to its milters one after another at end-of-message, each seeing what
// the one before added, which is how the filter's headers arrive here. The
// filter replaces headers of these names, so the last one read is its own.
func (s *inboundSession) Header(name, value string, m *milter.Modifier) (milter.Response, error) {
switch textproto.CanonicalMIMEHeaderKey(name) {
case "Subject":
if s.subject == "" {
s.subject = mailhdr.DecodeSubject(value)
}
case "X-Rspamd-Action":
if s.filter {
s.action, s.seen = value, true
}
case "X-Spamd-Result":
if s.filter {
s.result, s.seen = value, true
}
}
return milter.RespContinue, nil
}
// Body fires at end-of-message: the message is about to be queued for its
// upstream.
func (s *inboundSession) Body(m *milter.Modifier) (milter.Response, error) {
s.record()
s.reset()
return milter.RespAccept, nil
}
func (s *inboundSession) Abort(m *milter.Modifier) error {
s.reset()
return nil
}
func (s *inboundSession) reset() {
s.from, s.rcpts, s.subject = "", nil, ""
s.action, s.result, s.seen = "", "", false
}
// record writes one row per recipient at an inbound domain. Other recipients
// of port 25 (a DMARC report address at the server's own name) are not the
// relay's mail and get no row. Failures are logged, never propagated.
func (s *inboundSession) record() {
e := store.InboundSpamEntry{
ClientIP: s.clientIP,
From: s.from,
Subject: s.subject,
Decision: store.SpamDecisionAccept,
Engine: store.SpamEngineSelfPost,
}
if s.seen {
action, detail := rspamdVerdict(s.action, s.result)
e.Engine, e.Detail = store.SpamEngineRspamd, detail
if action == "quarantine" {
e.Decision = store.SpamDecisionQuarantine
}
}
for _, to := range s.rcpts {
domain := domainOf(to)
ok, err := s.rec.IsInboundDomain(domain)
if err != nil {
log.Printf("inbound-journal: look up %s: %v", domain, err)
continue
}
if !ok {
continue
}
e.InboundDomain, e.To = domain, to
if err := s.rec.InsertInboundSpam(e); err != nil {
log.Printf("inbound-journal: record %s -> %s: %v", s.from, to, err)
}
}
}
// What the journal keeps of a verdict: enough to see why, on one line.
const (
verdictMaxSymbols = 6
verdictMaxBytes = 255
)
// ListSymbolPrefix starts the names of the symbols the filter lists' rules add
// (deploy/antispam/rspamd/multimap.conf): SELFPOST_ALLOW_IP, _FROM, _DOMAIN.
const ListSymbolPrefix = "SELFPOST_"
var (
// "default: False [7.20 / 15.00]" - the score and the reject threshold.
rspamdScore = regexp.MustCompile(`\[(-?[0-9]+(?:\.[0-9]+)?) / (-?[0-9]+(?:\.[0-9]+)?)\]`)
// "BAYES_SPAM(5.10)[100.00%]" - a symbol and its weight.
rspamdSymbol = regexp.MustCompile(`([A-Z][A-Z0-9_]{1,63})\((-?[0-9]+(?:\.[0-9]+)?)\)`)
)
// rspamdVerdict reduces the filter's headers to an action and a one-line
// summary: the action, the score against the threshold, and the first few
// symbols that carried weight. It reads only what it recognises - the headers
// are the filter's when it is up, but the sender's own when it is not (it is
// attached fail-open), so nothing of them is kept verbatim except through
// these patterns.
func rspamdVerdict(actionHeader, resultHeader string) (action, detail string) {
action = strings.ToLower(strings.TrimSpace(actionHeader))
for _, r := range action {
if (r < 'a' || r > 'z') && r != ' ' {
action = ""
break
}
}
if len(action) > 32 {
action = ""
}
var parts []string
if action != "" {
parts = append(parts, action)
}
if m := rspamdScore.FindStringSubmatch(resultHeader); m != nil {
parts = append(parts, "score "+m[1]+" / "+m[2])
}
detail = strings.Join(parts, ", ")
// The filter lists' own symbols come first and are kept whatever else there
// is: Filter lists counts how often an entry decided by finding them here,
// and rspamd writes symbols heaviest first, which puts an allow entry's
// -100 last, behind any six others.
var symbols, rest []string
for _, m := range rspamdSymbol.FindAllStringSubmatch(resultHeader, -1) {
if strings.Trim(m[2], "-0.") == "" { // weight zero: informational
continue
}
if strings.HasPrefix(m[1], ListSymbolPrefix) {
if len(symbols) < verdictMaxSymbols {
symbols = append(symbols, m[1]+"("+m[2]+")")
}
continue
}
rest = append(rest, m[1]+"("+m[2]+")")
}
for _, sym := range rest {
if len(symbols) == verdictMaxSymbols {
break
}
symbols = append(symbols, sym)
}
if len(symbols) > 0 {
if detail != "" {
detail += "; "
}
detail += strings.Join(symbols, " ")
}
if len(detail) > verdictMaxBytes {
detail = detail[:verdictMaxBytes]
}
return action, detail
}
// ServeInbound runs the inbound journal milter on ln until ctx is cancelled.
// filter says whether an external filter is attached in front of it
// (INBOUND_ANTISPAM_MILTER). It returns nil on a clean shutdown.
func ServeInbound(ctx context.Context, ln net.Listener, rec InboundStore, filter bool) error {
srv := &milter.Server{
NewMilter: func() milter.Milter { return &inboundSession{rec: rec, filter: filter} },
Actions: 0, // read-only
Protocol: milter.OptNoBody, // headers and end-of-message, not the body
}
go func() {
<-ctx.Done()
_ = srv.Close()
}()
if err := srv.Serve(ln); err != nil {
if ctx.Err() != nil {
return nil
}
return err
}
return nil
}
+236
View File
@@ -0,0 +1,236 @@
package milter
import (
"errors"
"net"
"strings"
"testing"
"github.com/emersion/go-milter"
"github.com/mixeme/selfpost/internal/store"
)
// fakeInbound is the journal and the inbound-domain list of one test.
type fakeInbound struct {
domains map[string]bool
rows []store.InboundSpamEntry
insertErr error
lookupErr error
}
func (f *fakeInbound) InsertInboundSpam(e store.InboundSpamEntry) error {
if f.insertErr != nil {
return f.insertErr
}
f.rows = append(f.rows, e)
return nil
}
func (f *fakeInbound) IsInboundDomain(name string) (bool, error) {
if f.lookupErr != nil {
return false, f.lookupErr
}
return f.domains[name], nil
}
// A header of the kind the documented rspamd sidecar adds.
const spamdResult = "default: False [7.20 / 15.00];\n\tBAYES_SPAM(5.10)[100.00%];\n\tMIME_GOOD(-0.10)[text/plain];\n\tARC_NA(0.00)[];\n\tRCVD_COUNT_ZERO(0.00)[0];\n\tR_DKIM_NA(2.20)[]"
// message replays one message through s: envelope, headers in order, end.
func message(t *testing.T, s *inboundSession, from string, rcpts []string, headers [][2]string) milter.Response {
t.Helper()
if _, err := s.MailFrom(from, mods(nil)); err != nil {
t.Fatalf("MailFrom: %v", err)
}
for _, r := range rcpts {
if _, err := s.RcptTo(r, mods(nil)); err != nil {
t.Fatalf("RcptTo: %v", err)
}
}
for _, h := range headers {
if _, err := s.Header(h[0], h[1], mods(nil)); err != nil {
t.Fatalf("Header: %v", err)
}
}
resp, err := s.Body(mods(map[string]string{"i": "ABC123"}))
if err != nil {
t.Fatalf("Body: %v", err)
}
return resp
}
func inboundSessionFor(t *testing.T, rec InboundStore, filter bool) *inboundSession {
t.Helper()
s := &inboundSession{rec: rec, filter: filter}
if _, err := s.Connect("mx.example.net", "tcp4", 0, net.ParseIP("203.0.113.7"), mods(nil)); err != nil {
t.Fatalf("Connect: %v", err)
}
return s
}
// With no filter attached an accepted message is a plain row, one per
// recipient, decided by nobody.
func TestInboundRecordsAcceptedMessage(t *testing.T) {
rec := &fakeInbound{domains: map[string]bool{"in.example.com": true}}
s := inboundSessionFor(t, rec, false)
resp := message(t, s, "<sender@example.net> SIZE=1200",
[]string{"<a@in.example.com>", "b@IN.example.com"},
[][2]string{{"Subject", "=?utf-8?Q?Sch=C3=B6n?="}})
if resp != milter.RespAccept {
t.Errorf("end-of-message response = %v, want accept", resp)
}
if len(rec.rows) != 2 {
t.Fatalf("rows = %+v, want one per recipient", rec.rows)
}
want := store.InboundSpamEntry{
InboundDomain: "in.example.com", ClientIP: "203.0.113.7",
From: "sender@example.net", To: "a@in.example.com", Subject: "Schön",
Decision: store.SpamDecisionAccept, Engine: store.SpamEngineSelfPost,
}
if rec.rows[0] != want {
t.Errorf("row = %+v\nwant %+v", rec.rows[0], want)
}
if rec.rows[1].To != "b@IN.example.com" || rec.rows[1].InboundDomain != "in.example.com" {
t.Errorf("second row = %+v", rec.rows[1])
}
}
// Port 25 also takes DMARC reports for the server's own name. They are not
// the relay's mail and the journal says nothing about them.
func TestInboundSkipsRecipientsOutsideInboundDomains(t *testing.T) {
rec := &fakeInbound{domains: map[string]bool{"in.example.com": true}}
s := inboundSessionFor(t, rec, false)
message(t, s, "noreply-dmarc@google.com",
[]string{"dmarc-example.com@mail.example.org", "a@in.example.com"}, nil)
if len(rec.rows) != 1 || rec.rows[0].To != "a@in.example.com" {
t.Errorf("rows = %+v, want the inbound recipient alone", rec.rows)
}
}
// With the filter attached its verdict headers become the engine and detail.
func TestInboundReadsTheFiltersVerdict(t *testing.T) {
rec := &fakeInbound{domains: map[string]bool{"in.example.com": true}}
s := inboundSessionFor(t, rec, true)
message(t, s, "sender@example.net", []string{"a@in.example.com"}, [][2]string{
{"Subject", "Cheap pills"},
{"X-Rspamd-Action", "add header"},
{"X-Spamd-Result", spamdResult},
})
if len(rec.rows) != 1 {
t.Fatalf("rows = %+v", rec.rows)
}
got := rec.rows[0]
if got.Engine != store.SpamEngineRspamd || got.Decision != store.SpamDecisionAccept {
t.Errorf("engine %q, decision %q", got.Engine, got.Decision)
}
if want := "add header, score 7.20 / 15.00; BAYES_SPAM(5.10) MIME_GOOD(-0.10) R_DKIM_NA(2.20)"; got.Detail != want {
t.Errorf("detail = %q\nwant %q", got.Detail, want)
}
}
func TestInboundQuarantineAction(t *testing.T) {
rec := &fakeInbound{domains: map[string]bool{"in.example.com": true}}
s := inboundSessionFor(t, rec, true)
message(t, s, "sender@example.net", []string{"a@in.example.com"},
[][2]string{{"x-rspamd-action", "quarantine"}})
if len(rec.rows) != 1 || rec.rows[0].Decision != store.SpamDecisionQuarantine || rec.rows[0].Detail != "quarantine" {
t.Errorf("rows = %+v", rec.rows)
}
}
// Without a filter in front, a verdict header is the sender's own writing.
func TestInboundIgnoresVerdictHeadersWithoutAFilter(t *testing.T) {
rec := &fakeInbound{domains: map[string]bool{"in.example.com": true}}
s := inboundSessionFor(t, rec, false)
message(t, s, "sender@example.net", []string{"a@in.example.com"}, [][2]string{
{"X-Rspamd-Action", "no action"},
{"X-Spamd-Result", spamdResult},
})
if len(rec.rows) != 1 || rec.rows[0].Engine != store.SpamEngineSelfPost || rec.rows[0].Detail != "" {
t.Errorf("rows = %+v", rec.rows)
}
}
// The filter is attached fail-open, so with it down the headers are again the
// sender's: nothing of them reaches the journal except through the patterns.
func TestRspamdVerdictKeepsOnlyWhatItRecognises(t *testing.T) {
action, detail := rspamdVerdict("<script>alert(1)</script>", "default: False [1 / 15]; <b>HTML</b> lower_case(9.9) \x1b[31m")
if action != "" {
t.Errorf("action = %q, want it dropped", action)
}
if detail != "score 1 / 15" {
t.Errorf("detail = %q", detail)
}
var many strings.Builder
for i := 0; i < 400; i++ {
many.WriteString("SYMBOL_WITH_A_LONG_NAME_" + strings.Repeat("X", 30) + "(1.00)[];")
}
if _, detail := rspamdVerdict("reject", many.String()); len(detail) > verdictMaxBytes || strings.Count(detail, "(") > verdictMaxSymbols {
t.Errorf("detail of %d bytes with %d symbols", len(detail), strings.Count(detail, "("))
}
if action, _ := rspamdVerdict(strings.Repeat("a", 33), ""); action != "" {
t.Errorf("an action of 33 letters was kept")
}
}
// A second message on the same connection starts clean.
func TestInboundResetsBetweenMessages(t *testing.T) {
rec := &fakeInbound{domains: map[string]bool{"in.example.com": true}}
s := inboundSessionFor(t, rec, true)
message(t, s, "one@example.net", []string{"a@in.example.com"}, [][2]string{
{"Subject", "First"}, {"X-Rspamd-Action", "add header"},
})
message(t, s, "two@example.net", []string{"b@in.example.com"}, nil)
if len(rec.rows) != 2 {
t.Fatalf("rows = %+v", rec.rows)
}
want := store.InboundSpamEntry{
InboundDomain: "in.example.com", ClientIP: "203.0.113.7",
From: "two@example.net", To: "b@in.example.com",
Decision: store.SpamDecisionAccept, Engine: store.SpamEngineSelfPost,
}
if rec.rows[1] != want {
t.Errorf("second row = %+v\nwant %+v", rec.rows[1], want)
}
// An aborted transaction leaves nothing behind for the next one either.
if _, err := s.MailFrom("three@example.net", mods(nil)); err != nil {
t.Fatal(err)
}
if _, err := s.RcptTo("c@in.example.com", mods(nil)); err != nil {
t.Fatal(err)
}
if err := s.Abort(mods(nil)); err != nil {
t.Fatal(err)
}
if len(rec.rows) != 2 || len(s.rcpts) != 0 {
t.Errorf("after an abort: rows %d, pending recipients %v", len(rec.rows), s.rcpts)
}
}
// The journal failing is never the mail's problem.
func TestInboundAcceptsWhenTheStoreFails(t *testing.T) {
for name, rec := range map[string]*fakeInbound{
"insert": {domains: map[string]bool{"in.example.com": true}, insertErr: errors.New("boom")},
"lookup": {lookupErr: errors.New("boom")},
} {
s := inboundSessionFor(t, rec, false)
if resp := message(t, s, "sender@example.net", []string{"a@in.example.com"}, nil); resp != milter.RespAccept {
t.Errorf("%s failure: response = %v, want accept", name, resp)
}
}
}
// rspamd writes symbols heaviest first, so an allow entry's -100 comes last.
// Filter lists counts an entry's matches by finding that symbol in the journal:
// it is kept whatever else there is.
func TestRspamdVerdictKeepsTheListSymbols(t *testing.T) {
result := "default: False [-91.00 / 15.00]; SYM_A(3.00)[]; SYM_B(2.00)[]; SYM_C(1.50)[]; SYM_D(1.00)[]; SYM_E(0.80)[]; SYM_F(0.50)[]; SYM_G(0.20)[]; ARC_NA(0.00)[]; SELFPOST_ALLOW_FROM(-100.00)[billing@vendor.example]"
_, detail := rspamdVerdict("no action", result)
want := "no action, score -91.00 / 15.00; SELFPOST_ALLOW_FROM(-100.00) SYM_A(3.00) SYM_B(2.00) SYM_C(1.50) SYM_D(1.00) SYM_E(0.80)"
if detail != want {
t.Errorf("detail = %q\nwant %q", detail, want)
}
}
+9 -5
View File
@@ -131,14 +131,18 @@ func (s *Store) InsertDMARCReport(rep DMARCReport) (int64, error) {
return id, nil return id, nil
} }
// PruneDMARCReports enforces count and age caps. // PruneDMARCReports enforces the age cap for every domain and the count cap
func (s *Store) PruneDMARCReports() error { // for domain. The count is kept per domain on purpose: reports are accepted
// from anyone who knows a domain's report address, and with one cap for the
// whole instance a stranger filing reports under one domain would push every
// other domain's reports out.
func (s *Store) PruneDMARCReports(domain string) error {
cutoff := time.Now().UTC().AddDate(0, 0, -DMARCReportsMaxAgeDays).Format(time.RFC3339) cutoff := time.Now().UTC().AddDate(0, 0, -DMARCReportsMaxAgeDays).Format(time.RFC3339)
if _, err := s.db.Exec(`DELETE FROM dmarc_reports WHERE received_at < ?`, cutoff); err != nil { if _, err := s.db.Exec(`DELETE FROM dmarc_reports WHERE received_at < ?`, cutoff); err != nil {
return fmt.Errorf("prune dmarc by age: %w", err) return fmt.Errorf("prune dmarc by age: %w", err)
} }
var count int var count int
if err := s.db.QueryRow(`SELECT COUNT(*) FROM dmarc_reports`).Scan(&count); err != nil { if err := s.db.QueryRow(`SELECT COUNT(*) FROM dmarc_reports WHERE domain = ?`, domain).Scan(&count); err != nil {
return fmt.Errorf("count dmarc reports: %w", err) return fmt.Errorf("count dmarc reports: %w", err)
} }
if count <= DMARCReportsMaxKeep { if count <= DMARCReportsMaxKeep {
@@ -147,8 +151,8 @@ func (s *Store) PruneDMARCReports() error {
excess := count - DMARCReportsMaxKeep excess := count - DMARCReportsMaxKeep
_, err := s.db.Exec(` _, err := s.db.Exec(`
DELETE FROM dmarc_reports WHERE id IN ( DELETE FROM dmarc_reports WHERE id IN (
SELECT id FROM dmarc_reports ORDER BY received_at ASC LIMIT ? SELECT id FROM dmarc_reports WHERE domain = ? ORDER BY received_at ASC LIMIT ?
)`, excess) )`, domain, excess)
if err != nil { if err != nil {
return fmt.Errorf("prune dmarc by count: %w", err) return fmt.Errorf("prune dmarc by count: %w", err)
} }
+1 -1
View File
@@ -58,7 +58,7 @@ func TestPruneDMARCReports(t *testing.T) {
t.Fatal(err) t.Fatal(err)
} }
} }
if err := st.PruneDMARCReports(); err != nil { if err := st.PruneDMARCReports("example.com"); err != nil {
t.Fatal(err) t.Fatal(err)
} }
list, _ := st.ListDMARCReports(DMARCReportScope{AllDomains: true}, 100) list, _ := st.ListDMARCReports(DMARCReportScope{AllDomains: true}, 100)
+351
View File
@@ -0,0 +1,351 @@
package store
import (
"database/sql"
"errors"
"fmt"
"strings"
"time"
)
// Decisions of the inbound antispam journal. Kept in sync with the CHECK
// constraint of inbound_spam_log.
const (
SpamDecisionAccept = "accept"
SpamDecisionReject = "reject"
SpamDecisionTempfail = "tempfail"
SpamDecisionQuarantine = "quarantine"
)
// Who made a decision of the inbound antispam journal. Kept in sync with the
// CHECK constraint of inbound_spam_log.
const (
SpamEngineRspamd = "rspamd" // the external filter
SpamEnginePostfix = "postfix" // Postfix's own policy: unknown recipient, size, rate
SpamEngineSelfPost = "selfpost" // nobody: relayed with no filter attached
)
// InboundSpamEntry is one row of the inbound antispam journal: what happened
// to a message offered on port 25 for one recipient. Metadata only - the
// message itself is relayed, not stored. Subject and Detail may be empty.
type InboundSpamEntry struct {
InboundDomain string
ClientIP string
From string
To string
Subject string
Decision string
Engine string
Detail string
// At is when it happened; zero means now. The log tailer dates a refusal
// by its mail.log line, which it may read long after.
At time.Time
}
// InsertInboundSpam records one journal row. It is called from the inbound
// journal milter's hot path and from the log tailer, so it returns any error
// for the caller to log; neither of them may hold up mail over it.
func (s *Store) InsertInboundSpam(e InboundSpamEntry) error {
at := e.At
if at.IsZero() {
at = time.Now()
}
_, err := s.db.Exec(
`INSERT INTO inbound_spam_log
(inbound_domain, client_ip, from_addr, to_addr, subject, decision, engine, detail, created_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)`,
e.InboundDomain, e.ClientIP, e.From, e.To, e.Subject, e.Decision, e.Engine, e.Detail,
at.UTC().Format(time.RFC3339),
)
if err != nil {
return fmt.Errorf("insert inbound_spam_log: %w", err)
}
return nil
}
// PruneInboundSpam deletes journal rows older than cutoff and then, if more
// than keep rows are left, the oldest of those. It returns how many went.
func (s *Store) PruneInboundSpam(cutoff time.Time, keep int) (int64, error) {
res, err := s.db.Exec("DELETE FROM inbound_spam_log WHERE created_at < ?", cutoff.UTC().Format(time.RFC3339))
if err != nil {
return 0, fmt.Errorf("prune inbound_spam_log by age: %w", err)
}
n, _ := res.RowsAffected()
res, err = s.db.Exec(
`DELETE FROM inbound_spam_log WHERE id <=
(SELECT id FROM inbound_spam_log ORDER BY id DESC LIMIT 1 OFFSET ?)`, keep)
if err != nil {
return n, fmt.Errorf("prune inbound_spam_log by count: %w", err)
}
m, _ := res.RowsAffected()
return n + m, nil
}
// InboundSpamRow is one row of the inbound antispam journal as it is read
// back for the panel: the entry as it was written, with its id and when it
// happened.
type InboundSpamRow struct {
ID int64
InboundDomain string
ClientIP string
From string
To string
Subject string
Decision string
Engine string
Detail string
CreatedAt time.Time
}
// InboundSpamFilter narrows QueryInboundSpam and CountInboundSpam. Like
// SendLogFilter it carries two kinds of narrowing that behave in opposite ways
// on purpose.
//
// Domain, Decision, Since and Until are the operator's own filters, chosen in
// the UI: an empty field matches everything. Since and Until are days, both
// inclusive and read as UTC days whatever the time of day they carry.
//
// Domains is the authorization scope, which no query parameter may widen: the
// exhaustive set of inbound domain names the caller may read, applied as an IN
// constraint. nil lifts it (the global role reads the whole journal, including
// rows of a domain deleted since); an empty set is a caller who reaches
// nothing and gets nothing. Unlike SendLogFilter, forgetting the scope is
// therefore the permissive case - the caller states "all" by leaving it out -
// so every caller that serves someone other than the global role must set it.
type InboundSpamFilter struct {
Domain string
Decision string
Since time.Time
Until time.Time
Domains []string
}
// QueryInboundSpam returns journal rows matching filter, newest first, for
// the Inbound log's server-side pagination. A refusal is dated by the mail.log
// line the tailer read it from, which can be older than rows written since, so
// the order is by time and not by id.
func (s *Store) QueryInboundSpam(filter InboundSpamFilter, limit, offset int) ([]InboundSpamRow, error) {
where, args := inboundSpamWhere(filter)
args = append(args, limit, offset)
rows, err := s.db.Query(
`SELECT id, inbound_domain, client_ip, from_addr, to_addr, subject, decision, engine, detail, created_at
FROM inbound_spam_log`+where+`
ORDER BY created_at DESC, id DESC
LIMIT ? OFFSET ?`,
args...,
)
if err != nil {
return nil, fmt.Errorf("query inbound_spam_log: %w", err)
}
defer rows.Close()
var out []InboundSpamRow
for rows.Next() {
var (
row InboundSpamRow
created string
)
if err := rows.Scan(&row.ID, &row.InboundDomain, &row.ClientIP, &row.From, &row.To,
&row.Subject, &row.Decision, &row.Engine, &row.Detail, &created); err != nil {
return nil, fmt.Errorf("scan inbound_spam_log row: %w", err)
}
row.CreatedAt, _ = time.Parse(time.RFC3339, created)
out = append(out, row)
}
return out, rows.Err()
}
// CountInboundSpam returns how many journal rows match filter, for the page
// numbers under the Inbound log.
func (s *Store) CountInboundSpam(filter InboundSpamFilter) (int64, error) {
where, args := inboundSpamWhere(filter)
var n int64
if err := s.db.QueryRow(`SELECT COUNT(*) FROM inbound_spam_log`+where, args...).Scan(&n); err != nil {
return 0, fmt.Errorf("count inbound_spam_log: %w", err)
}
return n, nil
}
func inboundSpamWhere(f InboundSpamFilter) (string, []any) {
var clauses []string
var args []any
if f.Domains != nil {
if len(f.Domains) == 0 {
return " WHERE 1 = 0", nil
}
marks := make([]string, len(f.Domains))
for i, name := range f.Domains {
marks[i] = "?"
args = append(args, name)
}
clauses = append(clauses, "inbound_domain IN ("+strings.Join(marks, ", ")+")")
}
if f.Domain != "" {
clauses = append(clauses, "inbound_domain = ?")
args = append(args, f.Domain)
}
if f.Decision != "" {
clauses = append(clauses, "decision = ?")
args = append(args, f.Decision)
}
// created_at is RFC3339 UTC, so comparing it as text against the same
// format is comparing moments: from midnight of the first day up to, not
// including, midnight after the last.
if !f.Since.IsZero() {
y, m, d := f.Since.UTC().Date()
clauses = append(clauses, "created_at >= ?")
args = append(args, time.Date(y, m, d, 0, 0, 0, 0, time.UTC).Format(time.RFC3339))
}
if !f.Until.IsZero() {
y, m, d := f.Until.UTC().Date()
clauses = append(clauses, "created_at < ?")
args = append(args, time.Date(y, m, d+1, 0, 0, 0, 0, time.UTC).Format(time.RFC3339))
}
if len(clauses) == 0 {
return "", nil
}
return " WHERE " + strings.Join(clauses, " AND "), args
}
// IsInboundDomain reports whether name is a configured inbound domain. Port 25
// also takes mail that is not the relay's - DMARC reports for the server's own
// name - and the journal keeps rows for inbound domains only.
func (s *Store) IsInboundDomain(name string) (bool, error) {
var one int
err := s.db.QueryRow("SELECT 1 FROM inbound_domains WHERE name = ?", strings.ToLower(name)).Scan(&one)
if errors.Is(err, sql.ErrNoRows) {
return false, nil
}
if err != nil {
return false, fmt.Errorf("look up inbound domain: %w", err)
}
return true, nil
}
// Lists and entry kinds of the filter lists. Kept in sync with the CHECK
// constraints of inbound_spam_list.
const (
SpamListAllow = "allow"
SpamListDeny = "deny"
SpamEntryAddress = "address" // sender address
SpamEntryDomain = "domain" // sender domain
SpamEntryIP = "ip" // client address or network
)
// ErrSpamListEntryExists is returned when the entry is already on that list.
var ErrSpamListEntryExists = errors.New("filter list entry already exists")
// ErrSpamListEntryNotFound is returned when a filter list entry id does not exist.
var ErrSpamListEntryNotFound = errors.New("filter list entry not found")
// SpamListEntry is one allow or deny entry for the external filter. Value is
// stored normalised by the caller (internal/inbound), which is what makes the
// uniqueness of an entry on its list mean anything.
type SpamListEntry struct {
ID int64
List string
Kind string
Value string
Note string
CreatedAt time.Time
}
// AddSpamListEntry inserts an entry. The caller must have validated it.
func (s *Store) AddSpamListEntry(list, kind, value, note string) (SpamListEntry, error) {
now := time.Now().UTC()
res, err := s.db.Exec(
"INSERT INTO inbound_spam_list (list_type, entry_type, value, note, created_at) VALUES (?, ?, ?, ?, ?)",
list, kind, value, note, now.Format(time.RFC3339),
)
if err != nil {
if isUniqueViolation(err) {
return SpamListEntry{}, ErrSpamListEntryExists
}
return SpamListEntry{}, fmt.Errorf("insert inbound_spam_list: %w", err)
}
id, err := res.LastInsertId()
if err != nil {
return SpamListEntry{}, fmt.Errorf("filter list entry id: %w", err)
}
return SpamListEntry{ID: id, List: list, Kind: kind, Value: value, Note: note, CreatedAt: now.Truncate(time.Second)}, nil
}
// ListSpamListEntries returns every entry of both lists: by list, then kind,
// then value.
func (s *Store) ListSpamListEntries() ([]SpamListEntry, error) {
rows, err := s.db.Query(
"SELECT id, list_type, entry_type, value, note, created_at FROM inbound_spam_list ORDER BY list_type, entry_type, value")
if err != nil {
return nil, fmt.Errorf("list inbound_spam_list: %w", err)
}
defer rows.Close()
var out []SpamListEntry
for rows.Next() {
var e SpamListEntry
var created string
if err := rows.Scan(&e.ID, &e.List, &e.Kind, &e.Value, &e.Note, &created); err != nil {
return nil, fmt.Errorf("scan inbound_spam_list: %w", err)
}
e.CreatedAt, _ = time.Parse(time.RFC3339, created)
out = append(out, e)
}
return out, rows.Err()
}
// DeleteSpamListEntry removes an entry by id.
func (s *Store) DeleteSpamListEntry(id int64) error {
res, err := s.db.Exec("DELETE FROM inbound_spam_list WHERE id = ?", id)
if err != nil {
return fmt.Errorf("delete inbound_spam_list: %w", err)
}
if n, _ := res.RowsAffected(); n == 0 {
return ErrSpamListEntryNotFound
}
return nil
}
// SpamListHit is what the journal holds about one message a filter list
// decided: who it was from, which server offered it, when, and the detail that
// says which list it was.
type SpamListHit struct {
ClientIP string
From string
Decision string
Detail string
CreatedAt time.Time
}
// ListSpamListHits returns the journal rows of the filter in whose detail one
// of the marks occurs - the marks the filter lists leave (internal/inbound).
// Which entry a row belongs to is for the caller to work out; here the rows
// are only narrowed, so that a page view does not read the whole journal.
func (s *Store) ListSpamListHits(marks ...string) ([]SpamListHit, error) {
if len(marks) == 0 {
return nil, nil
}
likes := make([]string, len(marks))
args := []any{SpamEngineRspamd}
for i, m := range marks {
likes[i] = "instr(detail, ?) > 0"
args = append(args, m)
}
rows, err := s.db.Query(
`SELECT client_ip, from_addr, decision, detail, created_at FROM inbound_spam_log
WHERE engine = ? AND (`+strings.Join(likes, " OR ")+`)`, args...)
if err != nil {
return nil, fmt.Errorf("query filter list hits: %w", err)
}
defer rows.Close()
var out []SpamListHit
for rows.Next() {
var h SpamListHit
var created string
if err := rows.Scan(&h.ClientIP, &h.From, &h.Decision, &h.Detail, &created); err != nil {
return nil, fmt.Errorf("scan filter list hit: %w", err)
}
h.CreatedAt, _ = time.Parse(time.RFC3339, created)
out = append(out, h)
}
return out, rows.Err()
}
+286
View File
@@ -0,0 +1,286 @@
package store
import (
"testing"
"time"
)
func TestInsertInboundSpam(t *testing.T) {
st := openTestStore(t)
e := InboundSpamEntry{
InboundDomain: "in.example.com", ClientIP: "203.0.113.7",
From: "sender@example.net", To: "a@in.example.com", Subject: "Hello",
Decision: SpamDecisionAccept, Engine: SpamEngineRspamd, Detail: "no action, score 0.10 / 15.00",
}
if err := st.InsertInboundSpam(e); err != nil {
t.Fatalf("InsertInboundSpam: %v", err)
}
var got InboundSpamEntry
var created string
err := st.db.QueryRow(
"SELECT inbound_domain, client_ip, from_addr, to_addr, subject, decision, engine, detail, created_at FROM inbound_spam_log",
).Scan(&got.InboundDomain, &got.ClientIP, &got.From, &got.To, &got.Subject, &got.Decision, &got.Engine, &got.Detail, &created)
if err != nil {
t.Fatalf("read back: %v", err)
}
if got != e {
t.Errorf("row = %+v\nwant %+v", got, e)
}
if at, err := time.Parse(time.RFC3339, created); err != nil || time.Since(at) > time.Minute {
t.Errorf("created_at = %q (%v)", created, err)
}
// Every decision and engine the code names is one the table takes.
for _, d := range []string{SpamDecisionAccept, SpamDecisionReject, SpamDecisionTempfail, SpamDecisionQuarantine} {
for _, eng := range []string{SpamEngineRspamd, SpamEnginePostfix, SpamEngineSelfPost} {
e.Decision, e.Engine = d, eng
if err := st.InsertInboundSpam(e); err != nil {
t.Errorf("decision %s by %s: %v", d, eng, err)
}
}
}
e.Decision = "bounce"
if err := st.InsertInboundSpam(e); err == nil {
t.Error("an unknown decision was stored")
}
}
func TestIsInboundDomain(t *testing.T) {
st := openTestStore(t)
if _, err := st.AddInboundDomain("in.example.com"); err != nil {
t.Fatalf("AddInboundDomain: %v", err)
}
for name, want := range map[string]bool{
"in.example.com": true, "IN.Example.COM": true,
"example.com": false, "": false, "other.example.com": false,
} {
if got, err := st.IsInboundDomain(name); err != nil || got != want {
t.Errorf("IsInboundDomain(%q) = %v, %v; want %v", name, got, err, want)
}
}
}
func TestInsertInboundSpamKeepsItsOwnTime(t *testing.T) {
st := openTestStore(t)
at := time.Date(2026, 10, 1, 9, 30, 0, 0, time.FixedZone("msk", 3*3600))
err := st.InsertInboundSpam(InboundSpamEntry{
InboundDomain: "in.example.com", ClientIP: "203.0.113.7", To: "a@in.example.com",
Decision: SpamDecisionReject, Engine: SpamEnginePostfix, At: at,
})
if err != nil {
t.Fatalf("InsertInboundSpam: %v", err)
}
var created string
if err := st.db.QueryRow("SELECT created_at FROM inbound_spam_log").Scan(&created); err != nil || created != "2026-10-01T06:30:00Z" {
t.Errorf("created_at = %q, %v", created, err)
}
}
func TestPruneInboundSpam(t *testing.T) {
st := openTestStore(t)
now := time.Now().UTC()
add := func(to string, age time.Duration) {
t.Helper()
err := st.InsertInboundSpam(InboundSpamEntry{
InboundDomain: "in.example.com", ClientIP: "203.0.113.7", To: to,
Decision: SpamDecisionReject, Engine: SpamEnginePostfix, At: now.Add(-age),
})
if err != nil {
t.Fatalf("InsertInboundSpam: %v", err)
}
}
left := func() string {
t.Helper()
rows, err := st.db.Query("SELECT to_addr FROM inbound_spam_log ORDER BY id")
if err != nil {
t.Fatal(err)
}
defer rows.Close()
var out string
for rows.Next() {
var to string
if err := rows.Scan(&to); err != nil {
t.Fatal(err)
}
out += to + " "
}
return out
}
add("old", 100*24*time.Hour)
add("a", 3*time.Hour)
add("b", 2*time.Hour)
add("c", time.Hour)
// Age first: nothing else goes while the rest fits.
if n, err := st.PruneInboundSpam(now.AddDate(0, 0, -90), 10); err != nil || n != 1 {
t.Fatalf("prune by age = %d, %v", n, err)
}
if got := left(); got != "a b c " {
t.Fatalf("left = %q", got)
}
// Then the cap, oldest first.
if n, err := st.PruneInboundSpam(now.AddDate(0, 0, -90), 2); err != nil || n != 1 {
t.Fatalf("prune by count = %d, %v", n, err)
}
if got := left(); got != "b c " {
t.Fatalf("left = %q", got)
}
if n, err := st.PruneInboundSpam(now.AddDate(0, 0, -90), 2); err != nil || n != 0 {
t.Errorf("a second prune removed %d, %v", n, err)
}
}
func TestInboundSpamLogRetentionDays(t *testing.T) {
st := openTestStore(t)
if days, err := st.GetInboundSpamLogRetentionDays(30); err != nil || days != 30 {
t.Errorf("unset, env 30: %d, %v", days, err)
}
if days, err := st.GetInboundSpamLogRetentionDays(3); err != nil || days != InboundSpamLogRetentionDaysDefault {
t.Errorf("unset, env out of range: %d, %v", days, err)
}
if err := st.EnsureInboundSpamLogRetentionDays(30); err != nil {
t.Fatalf("Ensure: %v", err)
}
if err := st.EnsureInboundSpamLogRetentionDays(60); err != nil {
t.Fatalf("Ensure again: %v", err)
}
if days, _ := st.GetInboundSpamLogRetentionDays(90); days != 30 {
t.Errorf("after Ensure(30), Ensure(60): %d, want the first seed kept", days)
}
if err := st.SetInboundSpamLogRetentionDays(6); err == nil {
t.Error("6 days was accepted")
}
if err := st.SetInboundSpamLogRetentionDays(365); err != nil {
t.Errorf("365 days: %v", err)
}
if days, _ := st.GetInboundSpamLogRetentionDays(90); days != 365 {
t.Errorf("after Set(365): %d", days)
}
}
func TestQueryInboundSpamFilters(t *testing.T) {
st := openTestStore(t)
day := func(d, h int) time.Time { return time.Date(2026, 9, d, h, 0, 0, 0, time.UTC) }
add := func(domain, to, decision string, at time.Time) {
t.Helper()
err := st.InsertInboundSpam(InboundSpamEntry{
InboundDomain: domain, ClientIP: "203.0.113.7", From: "s@example.net", To: to,
Subject: "S " + to, Decision: decision, Engine: SpamEngineRspamd, Detail: "d", At: at,
})
if err != nil {
t.Fatalf("InsertInboundSpam: %v", err)
}
}
add("a.example", "1@a.example", SpamDecisionAccept, day(20, 23))
add("a.example", "2@a.example", SpamDecisionReject, day(21, 0))
add("b.example", "3@b.example", SpamDecisionAccept, day(21, 12))
add("b.example", "4@b.example", SpamDecisionTempfail, day(22, 0))
// Written last but dated earliest: the tailer reads a refusal after the fact.
add("a.example", "5@a.example", SpamDecisionReject, day(19, 8))
to := func(f InboundSpamFilter) string {
t.Helper()
rows, err := st.QueryInboundSpam(f, 50, 0)
if err != nil {
t.Fatalf("QueryInboundSpam(%+v): %v", f, err)
}
n, err := st.CountInboundSpam(f)
if err != nil || n != int64(len(rows)) {
t.Fatalf("CountInboundSpam(%+v) = %d, %v; the query returned %d", f, n, err, len(rows))
}
var out string
for _, r := range rows {
out += r.To[:1]
}
return out
}
// Newest first by time, not by the order of writing.
if got := to(InboundSpamFilter{}); got != "43215" {
t.Errorf("no filter = %q, want 43215", got)
}
if got := to(InboundSpamFilter{Domain: "a.example"}); got != "215" {
t.Errorf("domain = %q, want 215", got)
}
if got := to(InboundSpamFilter{Decision: SpamDecisionReject}); got != "25" {
t.Errorf("decision = %q, want 25", got)
}
// Days are whole UTC days and both ends are in: 21 st is 00:00 up to 23:59.
if got := to(InboundSpamFilter{Since: day(21, 15), Until: day(21, 3)}); got != "32" {
t.Errorf("one day = %q, want 32", got)
}
if got := to(InboundSpamFilter{Since: day(22, 0)}); got != "4" {
t.Errorf("since = %q, want 4", got)
}
if got := to(InboundSpamFilter{Until: day(20, 0)}); got != "15" {
t.Errorf("until = %q, want 15", got)
}
// A time in another zone is read as the UTC day it falls on.
msk := time.FixedZone("msk", 3*3600)
if got := to(InboundSpamFilter{Since: time.Date(2026, 9, 22, 2, 0, 0, 0, msk)}); got != "432" {
t.Errorf("since in another zone = %q, want 432 (it is the 21st in UTC)", got)
}
if got := to(InboundSpamFilter{Domain: "b.example", Decision: SpamDecisionAccept, Since: day(21, 0), Until: day(21, 0)}); got != "3" {
t.Errorf("all filters = %q, want 3", got)
}
// The scope: nil is everything, a set is its domains, an empty set nothing,
// and a chosen domain outside the set is nothing too.
if got := to(InboundSpamFilter{Domains: []string{"b.example"}}); got != "43" {
t.Errorf("scope = %q, want 43", got)
}
if got := to(InboundSpamFilter{Domains: []string{}}); got != "" {
t.Errorf("empty scope = %q, want nothing", got)
}
if got := to(InboundSpamFilter{Domains: []string{"b.example"}, Domain: "a.example"}); got != "" {
t.Errorf("a chosen domain outside the scope = %q, want nothing", got)
}
// Paging.
rows, err := st.QueryInboundSpam(InboundSpamFilter{}, 2, 2)
if err != nil || len(rows) != 2 || rows[0].To != "2@a.example" || rows[1].To != "1@a.example" {
t.Errorf("page 2 of 2 rows = %+v, %v", rows, err)
}
if rows[0].ID == 0 || !rows[0].CreatedAt.Equal(day(21, 0)) || rows[0].Subject != "S 2@a.example" ||
rows[0].Engine != SpamEngineRspamd || rows[0].Detail != "d" || rows[0].ClientIP != "203.0.113.7" || rows[0].From != "s@example.net" {
t.Errorf("a row read back = %+v", rows[0])
}
// A value is data, not SQL.
if got := to(InboundSpamFilter{Domain: "a.example' OR '1'='1"}); got != "" {
t.Errorf("injected domain = %q", got)
}
}
func TestListSpamListHits(t *testing.T) {
st := openTestStore(t)
add := func(engine, decision, detail string) {
t.Helper()
err := st.InsertInboundSpam(InboundSpamEntry{
InboundDomain: "in.example.com", ClientIP: "203.0.113.7", From: "a@example.net", To: "a@in.example.com",
Decision: decision, Engine: engine, Detail: detail, At: time.Date(2026, 10, 1, 9, 0, 0, 0, time.UTC),
})
if err != nil {
t.Fatal(err)
}
}
add(SpamEngineRspamd, SpamDecisionReject, "5.7.1 Refused by this server's deny list")
add(SpamEngineRspamd, SpamDecisionAccept, "no action, score -99.00 / 15.00; SELFPOST_ALLOW_IP(-100.00)")
add(SpamEngineRspamd, SpamDecisionReject, "5.7.1 Spam message rejected")
add(SpamEnginePostfix, SpamDecisionReject, "550 5.1.1 this server's deny list")
// a mark with the characters LIKE would treat as wildcards is text
add(SpamEngineRspamd, SpamDecisionAccept, "no action; SELFPOSTXALLOWX(1.00)")
hits, err := st.ListSpamListHits("this server's deny list", "SELFPOST_ALLOW_")
if err != nil || len(hits) != 2 {
t.Fatalf("hits = %+v, %v; want the two rows a list decided", hits, err)
}
h := hits[0]
if h.From != "a@example.net" || h.ClientIP != "203.0.113.7" || h.Decision != SpamDecisionReject || h.CreatedAt.Hour() != 9 {
t.Errorf("hit = %+v", h)
}
if hits, err := st.ListSpamListHits(); err != nil || hits != nil {
t.Errorf("no marks: %v, %v", hits, err)
}
}
@@ -0,0 +1,40 @@
-- Inbound antispam journal and filter lists (inbound-antispam-panel, 2.1.0).
-- The first migration after the 2.0 baseline: DDL only, so a 2.0.x database
-- gains two empty tables and nothing in it changes.
-- What happened to a message offered on port 25, one row per recipient.
-- Metadata only - SelfPost relays inbound mail and stores none of it. Accepted
-- messages are written by the inbound journal milter, refused ones by the
-- mail.log tailer, which is why a row can lack a subject (refused before DATA)
-- and why engine says who decided: the external filter, Postfix's own policy,
-- or nobody (selfpost: relayed with no filter attached).
CREATE TABLE inbound_spam_log (
id INTEGER PRIMARY KEY AUTOINCREMENT,
inbound_domain TEXT NOT NULL,
client_ip TEXT NOT NULL,
from_addr TEXT NOT NULL,
to_addr TEXT NOT NULL,
subject TEXT NOT NULL,
decision TEXT NOT NULL CHECK (decision IN ('accept', 'reject', 'tempfail', 'quarantine')),
engine TEXT NOT NULL CHECK (engine IN ('rspamd', 'postfix', 'selfpost')),
detail TEXT NOT NULL,
created_at TEXT NOT NULL
);
-- Retention prunes by age; the Log page lists newest first, for the whole
-- server or for one domain.
CREATE INDEX idx_inbound_spam_log_created_at ON inbound_spam_log (created_at);
CREATE INDEX idx_inbound_spam_log_domain_created_at ON inbound_spam_log (inbound_domain, created_at);
-- Allow / deny entries for the external filter, instance-wide. The panel
-- writes them out as map files under /data/antispam/; SelfPost itself does not
-- consult them. value is stored normalised, so the same entry cannot be added
-- twice to one list.
CREATE TABLE inbound_spam_list (
id INTEGER PRIMARY KEY AUTOINCREMENT,
list_type TEXT NOT NULL CHECK (list_type IN ('allow', 'deny')),
entry_type TEXT NOT NULL CHECK (entry_type IN ('address', 'domain', 'ip')),
value TEXT NOT NULL,
note TEXT NOT NULL DEFAULT '',
created_at TEXT NOT NULL,
UNIQUE (list_type, entry_type, value)
);
+95 -2
View File
@@ -55,6 +55,8 @@ func TestBaselineSchema(t *testing.T) {
"user_inbound_domains": "user_id inbound_domain_id", "user_inbound_domains": "user_id inbound_domain_id",
"dmarc_reports": "id domain reporter report_id period_begin period_end received_at contact_email policy_p policy_sp policy_pct policy_adkim policy_aspf pass_count fail_count recipient", "dmarc_reports": "id domain reporter report_id period_begin period_end received_at contact_email policy_p policy_sp policy_pct policy_adkim policy_aspf pass_count fail_count recipient",
"dmarc_report_records": "id report_row_id source_ip count disposition spf_result dkim_result header_from", "dmarc_report_records": "id report_row_id source_ip count disposition spf_result dkim_result header_from",
"inbound_spam_log": "id inbound_domain client_ip from_addr to_addr subject decision engine detail created_at",
"inbound_spam_list": "id list_type entry_type value note created_at",
} }
rows, err := st.db.Query("SELECT name FROM sqlite_master WHERE type = 'table' AND name NOT LIKE 'sqlite_%' ORDER BY name") rows, err := st.db.Query("SELECT name FROM sqlite_master WHERE type = 'table' AND name NOT LIKE 'sqlite_%' ORDER BY name")
if err != nil { if err != nil {
@@ -99,6 +101,7 @@ func TestBaselineSchema(t *testing.T) {
rows.Close() rows.Close()
wantIdx := []string{ wantIdx := []string{
"idx_dmarc_report_records_report", "idx_dmarc_reports_domain", "idx_dmarc_reports_received", "idx_dmarc_report_records_report", "idx_dmarc_reports_domain", "idx_dmarc_reports_received",
"idx_inbound_spam_log_created_at", "idx_inbound_spam_log_domain_created_at",
"idx_send_log_app_login_created_at", "idx_send_log_created_at", "idx_send_log_domain", "idx_send_log_queue_id", "idx_send_log_app_login_created_at", "idx_send_log_created_at", "idx_send_log_domain", "idx_send_log_queue_id",
"idx_sessions_expires_at", "idx_sessions_expires_at",
} }
@@ -108,8 +111,8 @@ func TestBaselineSchema(t *testing.T) {
} }
var version, appID int64 var version, appID int64
if err := st.db.QueryRow("PRAGMA user_version").Scan(&version); err != nil || version != 1 { if err := st.db.QueryRow("PRAGMA user_version").Scan(&version); err != nil || version != 2 {
t.Errorf("user_version = %d, %v; want 1", version, err) t.Errorf("user_version = %d, %v; want 2", version, err)
} }
if err := st.db.QueryRow("PRAGMA application_id").Scan(&appID); err != nil || appID != applicationID { if err := st.db.QueryRow("PRAGMA application_id").Scan(&appID); err != nil || appID != applicationID {
t.Errorf("application_id = %#x, %v; want %#x", appID, err, applicationID) t.Errorf("application_id = %#x, %v; want %#x", appID, err, applicationID)
@@ -156,6 +159,96 @@ func TestReopenKeepsTheDatabase(t *testing.T) {
} }
} }
// A 2.0.x data directory - the baseline and nothing after it - is brought to
// the head by opening it, with what it holds left alone.
func TestOpenMigratesA20Database(t *testing.T) {
path := filepath.Join(t.TempDir(), "test.db")
baseline, err := migrationsFS.ReadFile("migrations/0001_init.sql")
if err != nil {
t.Fatalf("read the baseline: %v", err)
}
db, err := sql.Open("sqlite", "file:"+path)
if err != nil {
t.Fatalf("create a 2.0 db: %v", err)
}
for _, stmt := range []string{
string(baseline),
"PRAGMA user_version = 1",
"INSERT INTO users (username, password_hash, role, created_at) VALUES ('admin', 'hash', 'global', '2026-10-11T00:00:00Z')",
} {
if _, err := db.Exec(stmt); err != nil {
t.Fatalf("prepare a 2.0 db: %v", err)
}
}
db.Close()
st, err := Open(path)
if err != nil {
t.Fatalf("open a 2.0 database: %v", err)
}
defer st.Close()
var version int
if err := st.db.QueryRow("PRAGMA user_version").Scan(&version); err != nil || version != 2 {
t.Errorf("user_version = %d, %v; want 2", version, err)
}
if ok, err := st.UserExists(); err != nil || !ok {
t.Errorf("the user of the 2.0 database: exists = %v, %v", ok, err)
}
for _, table := range []string{"inbound_spam_log", "inbound_spam_list"} {
if len(tableColumns(t, st, table)) == 0 {
t.Errorf("table %s was not created", table)
}
}
}
// What the antispam tables refuse: a decision, engine or list the panel does
// not know, and the same entry twice in one list.
func TestInboundSpamConstraints(t *testing.T) {
st := openTestStore(t)
logRow := func(decision, engine string) error {
_, err := st.db.Exec(
"INSERT INTO inbound_spam_log (inbound_domain, client_ip, from_addr, to_addr, subject, decision, engine, detail, created_at)"+
" VALUES ('in.example.com', '192.0.2.1', 'a@example.net', 'b@in.example.com', '', ?, ?, '', '2026-10-11T00:00:00Z')",
decision, engine)
return err
}
if err := logRow("reject", "postfix"); err != nil {
t.Fatalf("a reject by postfix: %v", err)
}
if err := logRow("bounce", "postfix"); err == nil {
t.Error("decision bounce was accepted")
}
if err := logRow("accept", "spamassassin"); err == nil {
t.Error("engine spamassassin was accepted")
}
entry := func(list, kind, value string) error {
_, err := st.db.Exec(
"INSERT INTO inbound_spam_list (list_type, entry_type, value, created_at) VALUES (?, ?, ?, '2026-10-11T00:00:00Z')",
list, kind, value)
return err
}
if err := entry("allow", "address", "billing@vendor.example"); err != nil {
t.Fatalf("an allow entry: %v", err)
}
if err := entry("allow", "address", "billing@vendor.example"); err == nil {
t.Error("the same entry was added to one list twice")
}
if err := entry("deny", "address", "billing@vendor.example"); err != nil {
t.Errorf("the same value on the other list: %v", err)
}
if err := entry("block", "address", "x@example.net"); err == nil {
t.Error("list type block was accepted")
}
if err := entry("deny", "subject", "pills"); err == nil {
t.Error("entry type subject was accepted")
}
var note string
if err := st.db.QueryRow("SELECT note FROM inbound_spam_list WHERE list_type = 'allow'").Scan(&note); err != nil || note != "" {
t.Errorf("note of an entry added without one = %q, %v", note, err)
}
}
// A 1.x data directory must be refused, loudly. Its user_version (10 at the // A 1.x data directory must be refused, loudly. Its user_version (10 at the
// last 1.x release) would otherwise read as "the baseline is already applied", // last 1.x release) would otherwise read as "the baseline is already applied",
// and the panel would start on tables it does not know. // and the panel would start on tables it does not know.
+66
View File
@@ -77,6 +77,72 @@ func (s *Store) EnsureSendLogRetentionDays(envDefault int) error {
return s.SetSendLogRetentionDays(sendLogRetentionFallback(envDefault)) return s.SetSendLogRetentionDays(sendLogRetentionFallback(envDefault))
} }
const (
// InboundSpamLogRetentionDaysKey is the settings table key for the
// inbound antispam journal's retention. Same range as the send log's.
InboundSpamLogRetentionDaysKey = "inbound_spam_log_retention_days"
InboundSpamLogRetentionDaysMin = 7
InboundSpamLogRetentionDaysMax = 365
// InboundSpamLogRetentionDaysDefault matches
// INBOUND_SPAM_LOG_RETENTION_DAYS when unset.
InboundSpamLogRetentionDaysDefault = 90
)
// ErrInboundSpamLogRetentionDaysOutOfRange is returned when the inbound
// journal's retention is outside the allowed panel range.
var ErrInboundSpamLogRetentionDaysOutOfRange = errors.New("inbound journal retention days out of range")
// ValidateInboundSpamLogRetentionDays checks the panel-allowed window.
func ValidateInboundSpamLogRetentionDays(days int) error {
if days < InboundSpamLogRetentionDaysMin || days > InboundSpamLogRetentionDaysMax {
return fmt.Errorf("%w: must be between %d and %d days", ErrInboundSpamLogRetentionDaysOutOfRange, InboundSpamLogRetentionDaysMin, InboundSpamLogRetentionDaysMax)
}
return nil
}
func inboundSpamLogRetentionFallback(envDefault int) int {
if ValidateInboundSpamLogRetentionDays(envDefault) == nil {
return envDefault
}
return InboundSpamLogRetentionDaysDefault
}
// GetInboundSpamLogRetentionDays returns the effective retention window of
// the inbound journal. When the setting is missing or invalid, envDefault is
// used (INBOUND_SPAM_LOG_RETENTION_DAYS).
func (s *Store) GetInboundSpamLogRetentionDays(envDefault int) (int, error) {
raw, err := s.GetSetting(InboundSpamLogRetentionDaysKey)
if err != nil {
return 0, err
}
days, err := strconv.Atoi(raw)
if err != nil || ValidateInboundSpamLogRetentionDays(days) != nil {
return inboundSpamLogRetentionFallback(envDefault), nil
}
return days, nil
}
// SetInboundSpamLogRetentionDays persists the inbound journal's retention.
func (s *Store) SetInboundSpamLogRetentionDays(days int) error {
if err := ValidateInboundSpamLogRetentionDays(days); err != nil {
return err
}
return s.SetSetting(InboundSpamLogRetentionDaysKey, strconv.Itoa(days))
}
// EnsureInboundSpamLogRetentionDays seeds the setting from envDefault when it
// has never been written.
func (s *Store) EnsureInboundSpamLogRetentionDays(envDefault int) error {
raw, err := s.GetSetting(InboundSpamLogRetentionDaysKey)
if err != nil {
return err
}
if raw != "" {
return nil
}
return s.SetInboundSpamLogRetentionDays(inboundSpamLogRetentionFallback(envDefault))
}
// GetSetting returns a settings value or empty string when missing. // GetSetting returns a settings value or empty string when missing.
func (s *Store) GetSetting(key string) (string, error) { func (s *Store) GetSetting(key string) (string, error) {
var value string var value string
+3
View File
@@ -56,6 +56,9 @@ var routes = struct{ always, inbound, dmarc []string }{
"POST /inbound/domains/{id}/upstream", "POST /inbound/domains/{id}/upstream",
"POST /inbound/domains/{id}/recipients", "POST /inbound/domains/{id}/recipients",
"GET /inbound/domains/{id}/delete", "POST /inbound/domains/{id}/delete", "GET /inbound/domains/{id}/delete", "POST /inbound/domains/{id}/delete",
"GET /inbound/log",
"GET /inbound/filter-lists", "POST /inbound/filter-lists",
"POST /inbound/filter-lists/{id}/delete",
}, },
dmarc: []string{ dmarc: []string{
"GET /outbound/dmarc", "GET /outbound/dmarc/domains/{id}", "GET /outbound/dmarc/reports/{id}", "GET /outbound/dmarc", "GET /outbound/dmarc/domains/{id}", "GET /outbound/dmarc/reports/{id}",
+6
View File
@@ -50,6 +50,9 @@ type Config struct {
// SendLogRetentionEnvDefault is SEND_LOG_RETENTION_DAYS at panel start; used // SendLogRetentionEnvDefault is SEND_LOG_RETENTION_DAYS at panel start; used
// as bootstrap and fallback when the settings row is missing or invalid. // as bootstrap and fallback when the settings row is missing or invalid.
SendLogRetentionEnvDefault int SendLogRetentionEnvDefault int
// InboundSpamLogRetentionEnvDefault is INBOUND_SPAM_LOG_RETENTION_DAYS at
// panel start, the same bootstrap and fallback for the inbound journal.
InboundSpamLogRetentionEnvDefault int
} }
// Handlers holds dependencies for authenticated panel routes. // Handlers holds dependencies for authenticated panel routes.
@@ -58,6 +61,7 @@ type Handlers struct {
domains *domain.Service domains *domain.Service
apps *app.Service apps *app.Service
inbound *inbound.Service inbound *inbound.Service
lists *inbound.Lists
dmarc *dmarc.Service dmarc *dmarc.Service
cfg Config cfg Config
view *view.Engine view *view.Engine
@@ -72,6 +76,7 @@ func New(
domains *domain.Service, domains *domain.Service,
apps *app.Service, apps *app.Service,
inboundSvc *inbound.Service, inboundSvc *inbound.Service,
lists *inbound.Lists,
dmarcSvc *dmarc.Service, dmarcSvc *dmarc.Service,
cfg Config, cfg Config,
v *view.Engine, v *view.Engine,
@@ -84,6 +89,7 @@ func New(
domains: domains, domains: domains,
apps: apps, apps: apps,
inbound: inboundSvc, inbound: inboundSvc,
lists: lists,
dmarc: dmarcSvc, dmarc: dmarcSvc,
cfg: cfg, cfg: cfg,
view: v, view: v,
+32 -9
View File
@@ -70,11 +70,30 @@ func (h *Handlers) HandleBackup(w http.ResponseWriter, r *http.Request) {
h.renderBackupPageWith(w, r, http.StatusBadRequest, "", pwErr) h.renderBackupPageWith(w, r, http.StatusBadRequest, "", pwErr)
return return
} }
params := backup.Params{
DataDir: h.cfg.DataDir,
DBPath: h.cfg.DBPath,
Version: h.cfg.Version,
DeployRoot: h.cfg.DeployRoot,
OnWarn: func(msg string) {
logf("panel: full backup: %s", msg)
},
}
if err := backup.ValidateDeployRoot(h.cfg.DeployRoot); err != nil { if err := backup.ValidateDeployRoot(h.cfg.DeployRoot); err != nil {
logf("panel: full backup: %v", err) logf("panel: full backup: %v", err)
h.renderBackupPageWith(w, r, http.StatusBadRequest, "", deployBackupErr(err)) h.renderBackupPageWith(w, r, http.StatusBadRequest, "", deployBackupErr(err))
return return
} }
// Everything is read through once before the first byte goes out. Past the
// headers a failure can only cut the download short - which is what the
// button did from 1.3.0 to 2.0.0, with a 200 and ten bytes, because the
// queue was not the panel's to read. Nothing is skipped to make the archive
// possible: it is the one selfpost-backup makes, or it is refused.
if err := backup.Check(params); err != nil {
logf("panel: full backup: %v", err)
h.renderBackupPageWith(w, r, http.StatusInternalServerError, "", unreadableBackupErr(err, h.cfg.DeployRoot))
return
}
stamp := time.Now().UTC().Format("20060102-150405") stamp := time.Now().UTC().Format("20060102-150405")
filename := fmt.Sprintf("selfpost-backup-%s.tar.gz", stamp) filename := fmt.Sprintf("selfpost-backup-%s.tar.gz", stamp)
@@ -109,15 +128,7 @@ func (h *Handlers) HandleBackup(w http.ResponseWriter, r *http.Request) {
sink = env sink = env
} }
if err := backup.Create(sink, backup.Params{ if err := backup.Create(sink, params); err != nil {
DataDir: h.cfg.DataDir,
DBPath: h.cfg.DBPath,
Version: h.cfg.Version,
DeployRoot: h.cfg.DeployRoot,
OnWarn: func(msg string) {
logf("panel: full backup: %s", msg)
},
}); err != nil {
logf("panel: full backup failed: %v", err) logf("panel: full backup failed: %v", err)
return return
} }
@@ -319,6 +330,18 @@ func decryptErrorMessage(err error) string {
} }
} }
// unreadableBackupErr phrases what backup.Check found. The usual case is the
// private key: the panel runs as its own user, and a key readable by root
// alone is not its to read. The way out is on the host, so the message says
// what to do there (guide § Taking a full backup).
func unreadableBackupErr(err error, deployRoot string) string {
if name, ok := backup.UnreadableCert(err, deployRoot); ok {
return "The panel is not allowed to read " + name + ", and a backup without it would not bring the server back. " +
"On the host, give the file to group 998 and make it group-readable (chgrp 998, chmod 640), then try again. Nothing was downloaded."
}
return "Could not read everything a backup holds. Nothing was downloaded; the reason is in the panel's log."
}
// deployBackupErr phrases a pre-flight backup failure for the operator. // deployBackupErr phrases a pre-flight backup failure for the operator.
func deployBackupErr(err error) string { func deployBackupErr(err error) string {
if strings.Contains(err.Error(), "DeployRoot") || strings.Contains(err.Error(), "deploy root") { if strings.Contains(err.Error(), "DeployRoot") || strings.Contains(err.Error(), "deploy root") {
@@ -49,7 +49,7 @@ func TestDMARCHubShowsIngestAndRecentReports(t *testing.T) {
body := getBody(t, s.h.HandleDMARCList, "/outbound/dmarc") body := getBody(t, s.h.HandleDMARCList, "/outbound/dmarc")
has(t, "hub", body, has(t, "hub", body,
`<h1 class="title is-3">DMARC reports</h1>`, `<h2>Ingest</h2>`, `<span class="tag is-success is-light">ok</span>`, `<h1 class="title is-3">DMARC reports</h1>`, `<h2>Ingest</h2>`, `<span class="tag is-success is-light">ok</span>`,
`1 kept · 0 parse failures`, `Hosted addresses`, `dmarc-reports&#43;&lt;domain&gt;@mail.example.org`, `500 reports or 90 days`, `1 kept · 0 parse failures`, `Hosted addresses`, `dmarc-reports&#43;&lt;domain&gt;@mail.example.org`, `500 reports a domain or 90 days`,
`<h2>Recent reports</h2>`, `<a href="/outbound/dmarc/domains/1">example.org</a>`, `Outlook.com`, `<h2>Recent reports</h2>`, `<a href="/outbound/dmarc/domains/1">example.org</a>`, `Outlook.com`,
"<td>1\u00a0903</td>", `<span class="tag is-warning is-light">7</span>`, `href="/outbound/dmarc/reports/`+idStr(s.reportID)+`">View</a>`) "<td>1\u00a0903</td>", `<span class="tag is-warning is-light">7</span>`, `href="/outbound/dmarc/reports/`+idStr(s.reportID)+`">View</a>`)
lacks(t, "hub", body, `None yet`) lacks(t, "hub", body, `None yet`)
@@ -0,0 +1,247 @@
package handlers
import (
"errors"
"net/http"
"strconv"
"time"
"github.com/mixeme/selfpost/internal/inbound"
"github.com/mixeme/selfpost/internal/store"
"github.com/mixeme/selfpost/internal/web/view"
)
// Inbound › Log and Inbound › Filter lists. The log is for everyone who reaches
// an inbound domain, with the rows of their domains only (requireInbound); the
// lists are instance-wide and so the global role's (requireInboundGlobal), and
// answer 404 to anyone else, like the rest of what only that role may do.
const inboundLogPageSize = 50
// logDay is the form of a day in the log's date filters and in <input
// type="date">.
const logDay = "2006-01-02"
// HandleInboundLog renders the Inbound log: the journal of port 25's decisions,
// filtered server-side by domain, decision and day, a page at a time. A filter
// value that is not one the page offers is dropped rather than refused, and the
// domains a domain administrator may read are decided here, before the query
// runs - the dropdown only offers them.
func (h *Handlers) HandleInboundLog(w http.ResponseWriter, r *http.Request) {
p, ok := h.requireInbound(w, r)
if !ok {
return
}
assigned, err := h.assignedInboundDomains(p)
if err != nil {
logf("panel: inbound log: %v", err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
names := make([]string, 0, len(assigned))
offered := make(map[string]bool, len(assigned))
for _, d := range assigned {
names = append(names, d.Name)
offered[d.Name] = true
}
q := r.URL.Query()
form := view.InLogFilter{}
query := store.InboundSpamFilter{}
if d := q.Get("domain"); offered[d] {
form.Domain, query.Domain = d, d
}
switch d := q.Get("decision"); d {
case store.SpamDecisionAccept, store.SpamDecisionReject, store.SpamDecisionTempfail, store.SpamDecisionQuarantine:
form.Decision, query.Decision = d, d
}
if t, err := time.Parse(logDay, q.Get("from")); err == nil {
form.From, query.Since = t.Format(logDay), t
}
if t, err := time.Parse(logDay, q.Get("to")); err == nil {
form.To, query.Until = t.Format(logDay), t
}
// A global administrator reads the whole journal, including rows left
// behind by a domain that has since been deleted; anyone else only what is
// assigned to them, however empty that is.
if !p.IsGlobal() {
query.Domains = names
}
total, err := h.store.CountInboundSpam(query)
if err != nil {
logf("panel: inbound log: %v", err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
lastPage := 1
if total > 0 {
lastPage = int((total + inboundLogPageSize - 1) / inboundLogPageSize)
}
page := min(parsePage(q.Get("p")), lastPage)
rows, err := h.store.QueryInboundSpam(query, inboundLogPageSize, (page-1)*inboundLogPageSize)
if err != nil {
logf("panel: inbound log: %v", err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
out := view.NewInLog(h.shellMeta(r), h.inboundSpamLogRetentionDays(), names, form)
table := make([]view.InLogRow, len(rows))
for i, row := range rows {
table[i] = out.NewRow(view.InLogRowInput{
ID: row.ID, At: row.CreatedAt, From: row.From, Client: row.ClientIP, To: row.To, Subject: row.Subject,
Decision: row.Decision, Engine: row.Engine, Detail: row.Detail,
})
}
h.view.Render(w, http.StatusOK, "in-log", out.WithRows(table, page, lastPage))
}
// inboundSpamLogRetentionDays returns the effective inbound journal retention
// window.
func (h *Handlers) inboundSpamLogRetentionDays() int {
days, err := h.store.GetInboundSpamLogRetentionDays(h.cfg.InboundSpamLogRetentionEnvDefault)
if err != nil {
logf("panel: inbound log retention: %v", err)
if err := store.ValidateInboundSpamLogRetentionDays(h.cfg.InboundSpamLogRetentionEnvDefault); err == nil {
return h.cfg.InboundSpamLogRetentionEnvDefault
}
return store.InboundSpamLogRetentionDaysDefault
}
return days
}
// ---- Filter lists
func (h *Handlers) requireFilterLists(w http.ResponseWriter, r *http.Request) bool {
if _, ok := h.requireInboundGlobal(w, r); !ok {
return false
}
if h.lists == nil {
http.NotFound(w, r)
return false
}
return true
}
// filterListForm is what the add form shows: the list and kind it offers (only
// values it knows are kept) and the text of the two fields.
type filterListForm struct {
list, kind, value, note string
}
// HandleFilterLists shows the lists. The query fills the add form in - the
// Inbound log's Allow / Deny sender links lead here - and only a list and a
// kind the form offers are taken; the value is just put in the field.
func (h *Handlers) HandleFilterLists(w http.ResponseWriter, r *http.Request) {
if !h.requireFilterLists(w, r) {
return
}
q := r.URL.Query()
form := filterListForm{value: q.Get("value")}
switch l := q.Get("list"); l {
case store.SpamListAllow, store.SpamListDeny:
form.list = l
}
switch k := q.Get("kind"); k {
case store.SpamEntryAddress, store.SpamEntryDomain, store.SpamEntryIP:
form.kind = k
}
h.renderFilterLists(w, r, http.StatusOK, filterListsFlash(r), "", form)
}
func filterListsFlash(r *http.Request) string {
switch {
case r.URL.Query().Has("added"):
return "Entry added."
case r.URL.Query().Has("removed"):
return "Entry removed."
}
return ""
}
func (h *Handlers) renderFilterLists(w http.ResponseWriter, r *http.Request, status int, flash, formErr string, form filterListForm) {
entries, err := h.lists.Entries()
if err != nil {
logf("panel: filter lists: %v", err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
// How often each entry decided a message is counted from the journal. A
// failure there is not a reason to hide the lists: they are shown without
// the counts' worth - every entry "never" - and the log says why.
usage, err := h.lists.Usage()
if err != nil {
logf("panel: filter lists: usage: %v", err)
}
on, _, _ := h.inboundFilter()
in := make([]view.FilterEntryInput, len(entries))
for i, e := range entries {
in[i] = view.FilterEntryInput{ID: e.ID, List: e.List, Kind: e.Kind, Value: e.Value, Note: e.Note, Added: e.CreatedAt,
Hits: usage[e.ID].Hits, LastHit: usage[e.ID].Last}
}
page := view.NewFilterLists(h.shellMeta(r), view.FilterListsInput{
FilterOn: on, LastWritten: h.lists.LastWritten(), RetentionDays: h.inboundSpamLogRetentionDays()}).
WithEntries(in).
WithResult(flash, formErr)
// A blank list or kind leaves the form on its first choice.
if form.list == "" {
form.list = store.SpamListDeny
}
if form.kind == "" {
form.kind = store.SpamEntryAddress
}
h.view.Render(w, status, "in-filter-lists", page.WithForm(form.list, form.kind, form.value, form.note))
}
// HandleFilterListAdd adds an entry and rewrites the map files. A refusal - an
// entry the lists do not take, or one already on the list - shows the page again
// with the message and what was typed.
func (h *Handlers) HandleFilterListAdd(w http.ResponseWriter, r *http.Request) {
if !h.requireFilterLists(w, r) {
return
}
if err := r.ParseForm(); err != nil {
h.renderFilterLists(w, r, http.StatusBadRequest, "", "Invalid form submission.", filterListForm{})
return
}
form := filterListForm{
list: r.PostFormValue("list_type"), kind: r.PostFormValue("entry_type"),
value: r.PostFormValue("value"), note: r.PostFormValue("note"),
}
_, err := h.lists.Add(form.list, form.kind, form.value, form.note)
var refused *inbound.ValidationError
switch {
case err == nil:
http.Redirect(w, r, "/inbound/filter-lists?added=1", http.StatusSeeOther)
case errors.Is(err, store.ErrSpamListEntryExists):
h.renderFilterLists(w, r, http.StatusUnprocessableEntity, "", "That entry is already on the list.", form)
case errors.As(err, &refused):
h.renderFilterLists(w, r, http.StatusUnprocessableEntity, "", refused.Error(), form)
default:
logf("panel: add filter list entry: %v", err)
h.renderFilterLists(w, r, http.StatusInternalServerError, "", "Could not add the entry. Check the logs and try again.", form)
}
}
// HandleFilterListDelete removes an entry and rewrites the map files.
func (h *Handlers) HandleFilterListDelete(w http.ResponseWriter, r *http.Request) {
if !h.requireFilterLists(w, r) {
return
}
id, err := strconv.ParseInt(r.PathValue("id"), 10, 64)
if err != nil || id <= 0 {
http.NotFound(w, r)
return
}
if err := h.lists.Delete(id); err != nil {
if errors.Is(err, store.ErrSpamListEntryNotFound) {
http.NotFound(w, r)
return
}
logf("panel: delete filter list entry %d: %v", id, err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
http.Redirect(w, r, "/inbound/filter-lists?removed=1", http.StatusSeeOther)
}
@@ -0,0 +1,674 @@
package handlers
import (
"fmt"
"net/http"
"net/http/httptest"
"net/url"
"os"
"path/filepath"
"strings"
"testing"
"time"
"github.com/mixeme/selfpost/internal/inbound"
"github.com/mixeme/selfpost/internal/store"
"github.com/mixeme/selfpost/internal/web/auth"
)
// Inbound › Log and Inbound › Filter lists, rendered from the real journal and
// lists by the handlers. Their markup is held to the mockups by the guards in
// internal/web/view; what is checked here is what the handlers read and who may.
// inboundLogStand is an inbound relay with two domains, a journal row for each
// and the lists over a directory the test can read the map files from.
type inboundLogStand struct {
h *Handlers
st *store.Store
dir string
a, b store.InboundDomain
}
func newInboundLogStand(t *testing.T) inboundLogStand {
t.Helper()
h, st := inboundHandlers(t)
s := inboundLogStand{h: h, st: st, dir: filepath.Join(t.TempDir(), "antispam")}
h.lists = inbound.NewLists(st, s.dir)
var err error
if s.a, err = st.AddInboundDomain("a.example.com"); err != nil {
t.Fatal(err)
}
if s.b, err = st.AddInboundDomain("b.example.com"); err != nil {
t.Fatal(err)
}
s.row("a.example.com", "alice@a.example.com", "Hello A", store.SpamDecisionAccept, store.SpamEngineRspamd)
s.row("b.example.com", "bob@b.example.com", "Hello B", store.SpamDecisionAccept, store.SpamEngineRspamd)
return s
}
func (s inboundLogStand) row(domain, to, subject, decision, engine string) {
s.rowAt(domain, to, subject, decision, engine, time.Time{})
}
func (s inboundLogStand) rowAt(domain, to, subject, decision, engine string, at time.Time) {
err := s.st.InsertInboundSpam(store.InboundSpamEntry{
InboundDomain: domain, ClientIP: "203.0.113.9", From: "sender@example.net", To: to, Subject: subject,
Decision: decision, Engine: engine, Detail: "detail of " + subject, At: at,
})
if err != nil {
panic(err)
}
}
// get runs a handler over a GET as p.
func (s inboundLogStand) get(h http.HandlerFunc, target string, p auth.Principal) *httptest.ResponseRecorder {
rec := httptest.NewRecorder()
h(rec, auth.RequestWithPrincipal(httptest.NewRequest(http.MethodGet, target, nil), p))
return rec
}
func (s inboundLogStand) owner(t *testing.T, username string, domains ...store.InboundDomain) auth.Principal {
t.Helper()
var ids []int64
for _, d := range domains {
ids = append(ids, d.ID)
}
return userWith(t, s.st, username, store.RoleDomain, store.Reach{InboundDomainIDs: ids})
}
// The global role reads the whole journal, is offered every domain, and is given
// the way to correct a decision and to change how long rows are kept.
func TestInboundLogForTheGlobalRole(t *testing.T) {
s := newInboundLogStand(t)
s.row("a.example.com", "carol@a.example.com", "Quarantined", store.SpamDecisionQuarantine, store.SpamEngineRspamd)
if err := s.st.SetInboundSpamLogRetentionDays(30); err != nil {
t.Fatal(err)
}
rec := s.get(s.h.HandleInboundLog, "/inbound/log", globalPrincipal)
if rec.Code != http.StatusOK {
t.Fatalf("GET /inbound/log = %d:\n%s", rec.Code, rec.Body.String())
}
body := rec.Body.String()
has(t, "log", body,
`<h1 class="title is-3">Inbound log</h1>`,
`Rows older than 30 days are deleted — change that in <a href="/server/settings">Settings</a>.`,
`alice@<wbr>a.example.com`, `bob@<wbr>b.example.com`, `Hello A`, `Hello B`,
`<option value="a.example.com">a.example.com</option>`, `<option value="b.example.com">b.example.com</option>`,
`class="tag is-success is-light" popovertarget="why-`, `class="tag is-primary is-light" popovertarget="why-`,
`<span class="sp-mono sp-muted sp-small">203.0.113.9</span>`,
`<tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Verdict</th><td>detail of Hello A</td></tr>`,
`<a href="/inbound/filter-lists?list=deny&amp;kind=address&amp;value=sender%40example.net">Deny sender</a>`,
`<a href="/inbound/filter-lists?list=allow&amp;kind=address&amp;value=sender%40example.net">Allow sender</a>`,
`Page 1 of 1`, `Allow / Deny sender opens <a href="/inbound/filter-lists">Filter lists</a>`,
)
// Newest first: the last row written is the first on the page.
if strings.Index(body, `Quarantined`) > strings.Index(body, `Hello B`) || strings.Index(body, `Hello B`) > strings.Index(body, `Hello A`) {
t.Error("the rows are not newest first")
}
lacks(t, "log", body, `data-poll`, `hx-get`)
}
// A domain administrator reads the rows of the inbound domains assigned to them,
// is offered those domains alone, and is not given the lists or the setting.
func TestInboundLogForADomainAdministrator(t *testing.T) {
s := newInboundLogStand(t)
p := s.owner(t, "owns-a", s.a)
body := s.get(s.h.HandleInboundLog, "/inbound/log", p).Body.String()
has(t, "log", body, `Hello A`, `<option value="a.example.com">a.example.com</option>`, `Rows older than 90 days are deleted.`)
lacks(t, "log", body, `Hello B`, `b.example.com`, `href="/server/settings"`, `/inbound/filter-lists`,
`Deny sender`, `Allow sender`, `Allow / Deny`)
// A domain filter outside their reach is dropped, not honoured.
body = s.get(s.h.HandleInboundLog, "/inbound/log?domain=b.example.com", p).Body.String()
has(t, "log", body, `Hello A`)
lacks(t, "log", body, `Hello B`, `b.example.com`)
// One with All sees every domain, one with none sees no log at all.
all := userWith(t, s.st, "owns-all", store.RoleDomain, store.Reach{AllInbound: true})
body = s.get(s.h.HandleInboundLog, "/inbound/log", all).Body.String()
has(t, "log", body, `Hello A`, `Hello B`)
lacks(t, "log", body, `Deny sender`)
outboundOnly := userWith(t, s.st, "outbound-only", store.RoleDomain, store.Reach{AllDomains: true})
if rec := s.get(s.h.HandleInboundLog, "/inbound/log", outboundOnly); rec.Code != http.StatusNotFound {
t.Errorf("an outbound-only administrator got %d, want 404", rec.Code)
}
}
// A domain administrator whose assigned domain has been deleted since has nothing
// to read, not everything.
func TestInboundLogOfADeletedDomainIsEmptyForItsAdministrator(t *testing.T) {
s := newInboundLogStand(t)
p := s.owner(t, "owns-a", s.a)
both := s.owner(t, "owns-both", s.a, s.b)
if err := s.st.DeleteInboundDomain(s.a.ID); err != nil {
t.Fatal(err)
}
body := s.get(s.h.HandleInboundLog, "/inbound/log", p).Body.String()
has(t, "log", body, `No messages logged yet.`)
lacks(t, "log", body, `Hello A`, `Hello B`)
body = s.get(s.h.HandleInboundLog, "/inbound/log", both).Body.String()
has(t, "log", body, `Hello B`)
lacks(t, "log", body, `Hello A`)
// The global role still reads what the deleted domain left in the journal.
has(t, "log", s.get(s.h.HandleInboundLog, "/inbound/log", globalPrincipal).Body.String(), `Hello A`, `Hello B`)
}
func TestInboundLogFilters(t *testing.T) {
s := newInboundLogStand(t)
day := func(d, h int) time.Time { return time.Date(2026, 9, d, h, 0, 0, 0, time.UTC) }
s.rowAt("a.example.com", "r1@a.example.com", "Rejected on the 10th", store.SpamDecisionReject, store.SpamEnginePostfix, day(10, 23))
s.rowAt("a.example.com", "r2@a.example.com", "Rejected on the 11th", store.SpamDecisionReject, store.SpamEnginePostfix, day(11, 0))
s.rowAt("b.example.com", "r3@b.example.com", "Tempfail on the 11th", store.SpamDecisionTempfail, store.SpamEngineRspamd, day(11, 23))
s.rowAt("a.example.com", "r4@a.example.com", "Accepted on the 12th", store.SpamDecisionAccept, store.SpamEngineRspamd, day(12, 0))
rows := func(query string) string {
t.Helper()
rec := s.get(s.h.HandleInboundLog, "/inbound/log"+query, globalPrincipal)
if rec.Code != http.StatusOK {
t.Fatalf("GET /inbound/log%s = %d", query, rec.Code)
}
var got []string
body := rec.Body.String()
for _, n := range []string{"10th", "11th", "12th"} {
for _, kind := range []string{"Rejected", "Tempfail", "Accepted"} {
if strings.Contains(body, kind+" on the "+n) {
got = append(got, kind[:1]+n[:2])
}
}
}
return strings.Join(got, " ")
}
for query, want := range map[string]string{
"?decision=reject": "R10 R11",
"?domain=b.example.com": "T11",
"?from=2026-09-11&to=2026-09-11": "R11 T11",
"?from=2026-09-12": "A12",
"?to=2026-09-10": "R10",
"?domain=a.example.com&decision=reject&from=2026-09-11": "R11",
// A value that is not one the page offers is ignored.
"?decision=bounce": "R10 R11 T11 A12",
"?from=yesterday&to=2026-13-45": "R10 R11 T11 A12",
"?domain=nobody.example.com": "R10 R11 T11 A12",
"?decision=reject%27+OR+%271%27%3D%271": "R10 R11 T11 A12",
"?domain=a.example.com&domain=b.example.com": "R10 R11 A12",
} {
if got := rows(query); got != want {
t.Errorf("GET /inbound/log%s shows %q, want %q", query, got, want)
}
}
// The choice made is the one shown, and the page leads on with it.
body := s.get(s.h.HandleInboundLog, "/inbound/log?domain=a.example.com&decision=reject&from=2026-09-11&to=2026-09-12", globalPrincipal).Body.String()
has(t, "log", body, `<option value="a.example.com" selected>`, `<option value="reject" selected>`,
`name="from" value="2026-09-11"`, `name="to" value="2026-09-12"`)
body = s.get(s.h.HandleInboundLog, "/inbound/log?decision=quarantine", globalPrincipal).Body.String()
has(t, "log", body, `No messages match these filters.`)
}
func TestInboundLogPages(t *testing.T) {
s := newInboundLogStand(t)
for i := 0; i < inboundLogPageSize; i++ {
s.row("a.example.com", fmt.Sprintf("u%d@a.example.com", i), fmt.Sprintf("Message %02d", i), store.SpamDecisionAccept, store.SpamEngineRspamd)
}
// 51 rows with the one the stand wrote: 50 and 1.
first := s.get(s.h.HandleInboundLog, "/inbound/log?domain=a.example.com", globalPrincipal).Body.String()
has(t, "page 1", first, `Page 1 of 2`, `<a href="/inbound/log?domain=a.example.com&amp;p=2">Older →</a>`, `Message 49`, `Message 00`)
lacks(t, "page 1", first, `← Newer`, `Hello A`)
if got := strings.Count(first, `<tr><td`); got != inboundLogPageSize {
t.Errorf("page 1 has %d rows, want %d", got, inboundLogPageSize)
}
second := s.get(s.h.HandleInboundLog, "/inbound/log?domain=a.example.com&p=2", globalPrincipal).Body.String()
has(t, "page 2", second, `Page 2 of 2`, `<a href="/inbound/log?domain=a.example.com">← Newer</a>`, `Hello A`)
lacks(t, "page 2", second, `Older →`, `Message 49`, `Message 00`)
if got := strings.Count(second, `<tr><td`); got != 1 {
t.Errorf("page 2 has %d rows, want 1", got)
}
// A page past the end, or nonsense, is the last page, or the first.
has(t, "page 9", s.get(s.h.HandleInboundLog, "/inbound/log?domain=a.example.com&p=9", globalPrincipal).Body.String(), `Page 2 of 2`, `Hello A`)
has(t, "page x", s.get(s.h.HandleInboundLog, "/inbound/log?domain=a.example.com&p=x", globalPrincipal).Body.String(), `Page 1 of 2`)
}
func TestInboundLogEmptyAndOff(t *testing.T) {
h, _ := inboundHandlers(t)
s := inboundLogStand{h: h}
body := s.get(h.HandleInboundLog, "/inbound/log", globalPrincipal).Body.String()
has(t, "empty log", body, `No messages logged yet.`)
lacks(t, "empty log", body, `<table`, `Page 1`)
h.cfg.InboundEnabled = false
if rec := s.get(h.HandleInboundLog, "/inbound/log", globalPrincipal); rec.Code != http.StatusNotFound {
t.Errorf("with the inbound relay off GET /inbound/log = %d, want 404", rec.Code)
}
}
// What a sender or a server wrote is text on the page.
func TestInboundLogEscapesWhatSendersWrote(t *testing.T) {
s := newInboundLogStand(t)
err := s.st.InsertInboundSpam(store.InboundSpamEntry{
InboundDomain: "a.example.com", ClientIP: "203.0.113.9", From: `"><img src=x onerror=alert(1)>@e.example`,
To: "x@a.example.com", Subject: `<script>alert(3)</script>`, Decision: store.SpamDecisionReject,
Engine: store.SpamEnginePostfix, Detail: `550 <b>no</b>`,
})
if err != nil {
t.Fatal(err)
}
body := s.get(s.h.HandleInboundLog, "/inbound/log", globalPrincipal).Body.String()
lacks(t, "log", body, `<script>alert`, `<img src=x`, `<b>no</b>`)
has(t, "log", body, `&lt;script&gt;alert(3)&lt;/script&gt;`)
}
// ---- Filter lists
func (s inboundLogStand) post(h http.HandlerFunc, target string, form url.Values, p auth.Principal) *httptest.ResponseRecorder {
req := httptest.NewRequest(http.MethodPost, target, strings.NewReader(form.Encode()))
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
if rest, ok := strings.CutPrefix(target, "/inbound/filter-lists/"); ok {
id, _, _ := strings.Cut(rest, "/")
req.SetPathValue("id", id)
}
rec := httptest.NewRecorder()
h(rec, auth.RequestWithPrincipal(req, p))
return rec
}
func (s inboundLogStand) mapFile(t *testing.T, name string) string {
t.Helper()
b, err := os.ReadFile(filepath.Join(s.dir, name))
if err != nil {
t.Fatalf("read %s: %v", name, err)
}
return string(b)
}
func entry(list, kind, value, note string) url.Values {
return url.Values{"list_type": {list}, "entry_type": {kind}, "value": {value}, "note": {note}}
}
// The lists are the global role's: everything about them is 404 to a domain
// administrator and while the inbound relay is off, and nothing is written.
func TestFilterListsAreForTheGlobalRoleOnly(t *testing.T) {
s := newInboundLogStand(t)
e, err := s.h.lists.Add("deny", "domain", "bulk.example", "")
if err != nil {
t.Fatal(err)
}
ops := s.owner(t, "owns-a", s.a)
all := userWith(t, s.st, "owns-all", store.RoleDomain, store.Reach{AllInbound: true, AllDomains: true})
for _, p := range []auth.Principal{ops, all} {
if rec := s.get(s.h.HandleFilterLists, "/inbound/filter-lists", p); rec.Code != http.StatusNotFound {
t.Errorf("GET as %s = %d, want 404", p.Username, rec.Code)
}
if rec := s.post(s.h.HandleFilterListAdd, "/inbound/filter-lists", entry("deny", "domain", "evil.example", ""), p); rec.Code != http.StatusNotFound {
t.Errorf("POST as %s = %d, want 404", p.Username, rec.Code)
}
if rec := s.post(s.h.HandleFilterListDelete, "/inbound/filter-lists/"+itoa(e.ID)+"/delete", nil, p); rec.Code != http.StatusNotFound {
t.Errorf("POST delete as %s = %d, want 404", p.Username, rec.Code)
}
}
if list, _ := s.h.lists.Entries(); len(list) != 1 {
t.Errorf("a domain administrator changed the lists: %+v", list)
}
if got := s.mapFile(t, "deny_domain.map"); strings.Contains(got, "evil.example") || !strings.Contains(got, "bulk.example") {
t.Errorf("deny_domain.map = %q", got)
}
s.h.cfg.InboundEnabled = false
if rec := s.get(s.h.HandleFilterLists, "/inbound/filter-lists", globalPrincipal); rec.Code != http.StatusNotFound {
t.Errorf("with the inbound relay off GET = %d, want 404", rec.Code)
}
if rec := s.post(s.h.HandleFilterListAdd, "/inbound/filter-lists", entry("deny", "domain", "evil.example", ""), globalPrincipal); rec.Code != http.StatusNotFound {
t.Errorf("with the inbound relay off POST = %d, want 404", rec.Code)
}
if rec := s.post(s.h.HandleFilterListDelete, "/inbound/filter-lists/"+itoa(e.ID)+"/delete", nil, globalPrincipal); rec.Code != http.StatusNotFound {
t.Errorf("with the inbound relay off POST delete = %d, want 404", rec.Code)
}
// Nothing to write the lists to is nothing to show.
s.h.cfg.InboundEnabled, s.h.lists = true, nil
if rec := s.get(s.h.HandleFilterLists, "/inbound/filter-lists", globalPrincipal); rec.Code != http.StatusNotFound {
t.Errorf("without lists GET = %d, want 404", rec.Code)
}
}
func TestFilterListsAddAndRemove(t *testing.T) {
s := newInboundLogStand(t)
empty := s.get(s.h.HandleFilterLists, "/inbound/filter-lists", globalPrincipal).Body.String()
has(t, "empty lists", empty, `<h1 class="title is-3">Filter lists</h1>`, `<h2>Deny</h2><div class="sp-end">0 entries · wins over Allow</div>`,
`<h2>Allow</h2><div class="sp-end">0 entries</div>`, `No sender is turned away by hand.`, `No sender is let through by hand.`)
lacks(t, "empty lists", empty, `<table`)
// The domain is normalised, written to the file of its list and kind, and the
// page shows it.
rec := s.post(s.h.HandleFilterListAdd, "/inbound/filter-lists", entry("deny", "domain", " @Bulk.Example ", " newsletter "), globalPrincipal)
if rec.Code != http.StatusSeeOther || rec.Header().Get("Location") != "/inbound/filter-lists?added=1" {
t.Fatalf("add = %d to %q:\n%s", rec.Code, rec.Header().Get("Location"), rec.Body.String())
}
if got := s.mapFile(t, "deny_domain.map"); !strings.Contains(got, "\nbulk.example\n") {
t.Errorf("deny_domain.map = %q", got)
}
if got := s.mapFile(t, "allow_domain.map"); strings.Contains(got, "bulk.example") {
t.Errorf("allow_domain.map = %q", got)
}
rec = s.post(s.h.HandleFilterListAdd, "/inbound/filter-lists", entry("allow", "ip", "198.51.100.77/24", ""), globalPrincipal)
if rec.Code != http.StatusSeeOther {
t.Fatalf("add ip = %d:\n%s", rec.Code, rec.Body.String())
}
if got := s.mapFile(t, "allow_ip.map"); !strings.Contains(got, "\n198.51.100.0/24\n") {
t.Errorf("allow_ip.map = %q", got)
}
page := s.get(s.h.HandleFilterLists, "/inbound/filter-lists?added=1", globalPrincipal).Body.String()
has(t, "lists", page,
`<div class="notification is-success is-light">`, `Entry added.`,
`<h2>Deny</h2><div class="sp-end">1 entry · wins over Allow</div>`, `<h2>Allow</h2><div class="sp-end">1 entry</div>`,
`<td class="sp-mono sp-small">bulk.example</td><td><button type="button" class="tag is-light" popovertarget="entry-`,
`<tr><th>Note</th><td>newsletter</td></tr><tr><th>Added</th><td class="sp-mono sp-small">`+time.Now().UTC().Format("2006-01-02")+`</td></tr><tr><th>Matched</th><td>never in the last 90 days</td></tr>`,
`<td class="sp-mono sp-small">198.51.100.0/24</td>`, `<tr><th>Note</th><td>—</td></tr>`,
`<td class="sp-muted sp-nowrap">never</td>`,
)
lacks(t, "lists", page, `No sender is turned away`)
// Remove the deny entry: gone from the page and from the file, the allow
// entry untouched.
entries, _ := s.h.lists.Entries()
var denyID int64
for _, e := range entries {
if e.List == "deny" {
denyID = e.ID
}
}
has(t, "lists", page, `action="/inbound/filter-lists/`+itoa(denyID)+`/delete"`)
rec = s.post(s.h.HandleFilterListDelete, "/inbound/filter-lists/"+itoa(denyID)+"/delete", nil, globalPrincipal)
if rec.Code != http.StatusSeeOther || rec.Header().Get("Location") != "/inbound/filter-lists?removed=1" {
t.Fatalf("delete = %d to %q", rec.Code, rec.Header().Get("Location"))
}
if got := s.mapFile(t, "deny_domain.map"); strings.Contains(got, "bulk.example") {
t.Errorf("deny_domain.map still holds the entry: %q", got)
}
if got := s.mapFile(t, "allow_ip.map"); !strings.Contains(got, "198.51.100.0/24") {
t.Errorf("allow_ip.map lost its entry: %q", got)
}
page = s.get(s.h.HandleFilterLists, "/inbound/filter-lists?removed=1", globalPrincipal).Body.String()
has(t, "lists", page, `Entry removed.`, `<h2>Deny</h2><div class="sp-end">0 entries`)
lacks(t, "lists", page, `bulk.example <span`)
// An entry that is not there, or an id that is not one.
for _, id := range []string{itoa(denyID), "9999", "x", "0", "-1"} {
if rec := s.post(s.h.HandleFilterListDelete, "/inbound/filter-lists/"+id+"/delete", nil, globalPrincipal); rec.Code != http.StatusNotFound {
t.Errorf("delete %q = %d, want 404", id, rec.Code)
}
}
}
// A refusal comes back with 422 on the same page, the message between the head
// and the form and what was typed kept in it; nothing is written.
func TestFilterListsRefusals(t *testing.T) {
s := newInboundLogStand(t)
if rec := s.post(s.h.HandleFilterListAdd, "/inbound/filter-lists", entry("deny", "address", "win@lottery.example", ""), globalPrincipal); rec.Code != http.StatusSeeOther {
t.Fatalf("add = %d", rec.Code)
}
before := s.mapFile(t, "deny_from.map")
for _, c := range []struct {
name string
form url.Values
want string
}{
{"duplicate", entry("deny", "address", "WIN@lottery.example", "again"), "That entry is already on the list."},
{"not an address", entry("deny", "address", "not-an-address", ""), "is not a valid email address"},
{"not a network", entry("allow", "ip", "192.0.2.0/99", "kept note"), "is not a network in CIDR form"},
{"no kind", entry("deny", "everything", "x", ""), "choose what the entry is"},
{"no list", entry("maybe", "domain", "x.example", ""), "choose the allow or the deny list"},
{"empty", entry("deny", "domain", " ", ""), "domain is required"},
{"long note", entry("deny", "domain", "x.example", strings.Repeat("n", 201)), "note must be at most 200 characters"},
} {
rec := s.post(s.h.HandleFilterListAdd, "/inbound/filter-lists", c.form, globalPrincipal)
if rec.Code != http.StatusUnprocessableEntity {
t.Errorf("%s: %d, want 422:\n%s", c.name, rec.Code, rec.Body.String())
continue
}
body := rec.Body.String()
has(t, c.name, body, `<div class="notification is-danger is-light">`, c.want, `value="`+c.form.Get("value")+`"`)
if c.form.Get("list_type") == "allow" || c.form.Get("list_type") == "deny" {
has(t, c.name, body, `<option value="`+c.form.Get("list_type")+`" selected>`)
}
if c.form.Get("note") != "" && len(c.form.Get("note")) < 100 {
has(t, c.name, body, `value="`+c.form.Get("note")+`"`)
}
}
if got := s.mapFile(t, "deny_from.map"); got != before {
t.Errorf("a refused entry changed deny_from.map:\n%s", got)
}
if e, _ := s.h.lists.Entries(); len(e) != 1 {
t.Errorf("a refused entry was stored: %+v", e)
}
// What was typed is escaped when it comes back.
rec := s.post(s.h.HandleFilterListAdd, "/inbound/filter-lists", entry("deny", "address", `"><script>1</script>`, `<i>x</i>`), globalPrincipal)
if rec.Code != http.StatusUnprocessableEntity {
t.Fatalf("hostile entry = %d", rec.Code)
}
lacks(t, "refused form", rec.Body.String(), `<script>1`, `<i>x</i>`)
}
// The log's Allow / Deny sender links fill the form in. Only a list and a kind
// the form offers are taken; the value is text.
func TestFilterListsPrefillFromTheLog(t *testing.T) {
s := newInboundLogStand(t)
get := func(query string) string {
rec := s.get(s.h.HandleFilterLists, "/inbound/filter-lists"+query, globalPrincipal)
if rec.Code != http.StatusOK {
t.Fatalf("GET %s = %d", query, rec.Code)
}
return rec.Body.String()
}
body := get("?list=allow&kind=address&value=billing%40partner.example")
has(t, "prefilled", body, `<option value="allow" selected>`, `<option value="address" selected>`,
`id="value" name="value" value="billing@partner.example"`)
lacks(t, "prefilled", body, `<option value="deny" selected>`)
body = get("?list=allow&kind=domain&value=partner.example")
has(t, "prefilled", body, `<option value="allow" selected>`, `<option value="domain" selected>`, `value="partner.example"`)
// Unknown choices leave the form on its first; the value is still put in.
body = get("?list=everything&kind=whatever&value=x%40y.example")
has(t, "unknown choices", body, `<option value="deny" selected>`, `<option value="address" selected>`, `value="x@y.example"`)
// The value cannot leave its attribute.
body = get(`?value=` + url.QueryEscape(`"><script>alert(1)</script>`))
lacks(t, "hostile value", body, `<script>alert`)
has(t, "hostile value", body, `value="&#34;&gt;&lt;script&gt;alert(1)&lt;/script&gt;"`)
// Nothing in the query is a flash.
lacks(t, "plain", get(""), `notification`)
}
// The log's links and the form agree: what the log links to is what the form
// accepts, for every kind of sender it can name.
func TestFilterListsAcceptWhatTheLogLinksTo(t *testing.T) {
s := newInboundLogStand(t)
log := s.get(s.h.HandleInboundLog, "/inbound/log", globalPrincipal).Body.String()
has(t, "log", log, `href="/inbound/filter-lists?list=deny&amp;kind=address&amp;value=sender%40example.net"`)
body := s.get(s.h.HandleFilterLists, "/inbound/filter-lists?list=deny&kind=address&value=sender%40example.net", globalPrincipal).Body.String()
has(t, "form", body, `value="sender@example.net"`)
if rec := s.post(s.h.HandleFilterListAdd, "/inbound/filter-lists", entry("deny", "address", "sender@example.net", ""), globalPrincipal); rec.Code != http.StatusSeeOther {
t.Errorf("the entry the log leads to was refused: %d", rec.Code)
}
}
// Box 04 says whether anything reads the lists and when they were last written.
func TestFilterListsWhereTheyGo(t *testing.T) {
s := newInboundLogStand(t)
// Filter off: the lists are kept, the box says nothing reads them, and the
// files have never been written.
body := s.get(s.h.HandleFilterLists, "/inbound/filter-lists", globalPrincipal).Body.String()
has(t, "filter off", body, `<h2>Where the lists go</h2>`, `<span class="tag is-light">not applied</span>`,
`Nothing — the spam filter is off. The lists are kept and can be edited; nothing reads them until <code>INBOUND_ANTISPAM_MILTER</code> is set.`,
`<dt>Last written</dt><dd>never</dd>`, `Deny wins, whichever entries they are.`)
lacks(t, "filter off", body, `is-success is-light">applied`, `rspamd, from map files`)
// Filter on, and the files written.
s.h.cfg.InboundAntispamMilter = "inet:antispam:11332"
if err := s.h.lists.Sync(); err != nil {
t.Fatal(err)
}
stamp := time.Date(2026, 9, 11, 8, 40, 0, 0, time.UTC)
for _, name := range []string{"allow_ip.map", "allow_from.map", "allow_domain.map", "deny_ip.map", "deny_from.map", "deny_domain.map"} {
if err := os.Chtimes(filepath.Join(s.dir, name), stamp, stamp); err != nil {
t.Fatal(err)
}
}
body = s.get(s.h.HandleFilterLists, "/inbound/filter-lists", globalPrincipal).Body.String()
has(t, "filter on", body, `<span class="tag is-success is-light">applied</span>`,
`rspamd, from map files it rereads within seconds of a change`, `<dt>Last written</dt><dd>2026-09-11 08:40 UTC</dd>`,
`/data/antispam/`)
lacks(t, "filter on", body, `not applied`, `Nothing — the spam filter is off`, `INBOUND_ANTISPAM_MILTER`)
}
// ---- Settings and Help
func settingsWithInbound(t *testing.T, on bool) *Handlers {
t.Helper()
h, _ := settingsServer(t)
h.cfg.InboundEnabled = on
h.cfg.InboundSpamLogRetentionEnvDefault = 90
return h
}
func TestSettingsShowsAndSavesTheInboundRetention(t *testing.T) {
h := settingsWithInbound(t, true)
body := getBody(t, h.HandleServerSettings, "/server/settings")
has(t, "settings", body, `Keep outbound log rows, days`, `Keep inbound log rows, days`,
`name="inbound_spam_log_retention_days"`, `value="90"`,
`The inbound log also never holds more than 10 000 rows`)
if err := h.store.SetInboundSpamLogRetentionDays(45); err != nil {
t.Fatal(err)
}
has(t, "settings", getBody(t, h.HandleServerSettings, "/server/settings"), `name="inbound_spam_log_retention_days" type="number" min="7" max="365" step="1" value="45"`)
rec := postFormAs(h.HandleServerSettings, globalPrincipal, "/server/settings", nil,
url.Values{"send_log_retention_days": {"120"}, "inbound_spam_log_retention_days": {"200"}})
if rec.Code != http.StatusSeeOther || rec.Header().Get("Location") != "/server/settings?saved=1" {
t.Fatalf("POST /server/settings = %d to %q:\n%s", rec.Code, rec.Header().Get("Location"), rec.Body.String())
}
if got, _ := h.store.GetSendLogRetentionDays(90); got != 120 {
t.Errorf("outbound retention = %d, want 120", got)
}
if got, _ := h.store.GetInboundSpamLogRetentionDays(90); got != 200 {
t.Errorf("inbound retention = %d, want 200", got)
}
// The Inbound log's lead reads the same setting.
if got := h.inboundSpamLogRetentionDays(); got != 200 {
t.Errorf("inboundSpamLogRetentionDays = %d, want 200", got)
}
}
func TestSettingsRefuseAnInboundRetentionOutOfRange(t *testing.T) {
h := settingsWithInbound(t, true)
for _, bad := range []string{"3", "366", "nine", ""} {
rec := postFormAs(h.HandleServerSettings, globalPrincipal, "/server/settings", nil,
url.Values{"send_log_retention_days": {"120"}, "inbound_spam_log_retention_days": {bad}})
if rec.Code != http.StatusBadRequest {
t.Fatalf("POST (inbound %q) = %d, want 400:\n%s", bad, rec.Code, rec.Body.String())
}
if bad == "3" && !strings.Contains(rec.Body.String(), "between 7 and 365") {
t.Errorf("no range in the refusal:\n%s", rec.Body.String())
}
if bad == "nine" && !strings.Contains(rec.Body.String(), "Inbound log retention must be a whole number of days.") {
t.Errorf("no sentence in the refusal:\n%s", rec.Body.String())
}
// The outbound value that was fine is shown as typed, the bad one as it was.
has(t, "refused form", rec.Body.String(), `name="send_log_retention_days" type="number" min="7" max="365" step="1" value="120"`)
}
// Nothing was saved, not even the outbound value that was fine.
if got, _ := h.store.GetSendLogRetentionDays(90); got != 90 {
t.Errorf("a refused form changed the outbound retention to %d", got)
}
if got, _ := h.store.GetInboundSpamLogRetentionDays(90); got != 90 {
t.Errorf("a refused form changed the inbound retention to %d", got)
}
// A bad outbound value refuses the whole form too.
rec := postFormAs(h.HandleServerSettings, globalPrincipal, "/server/settings", nil,
url.Values{"send_log_retention_days": {"3"}, "inbound_spam_log_retention_days": {"200"}})
if rec.Code != http.StatusBadRequest {
t.Fatalf("POST (outbound 3) = %d", rec.Code)
}
if got, _ := h.store.GetInboundSpamLogRetentionDays(90); got != 90 {
t.Errorf("the inbound retention was saved by a refused form: %d", got)
}
}
// With the relay off there is no inbound journal to keep: the field is not on the
// page and a value posted for it is not read.
func TestSettingsHaveNoInboundRetentionWithTheRelayOff(t *testing.T) {
h := settingsWithInbound(t, false)
body := getBody(t, h.HandleServerSettings, "/server/settings")
has(t, "settings", body, `Keep outbound log rows, days`)
lacks(t, "settings", body, `inbound_spam_log_retention_days`, `Keep inbound log rows`)
rec := postFormAs(h.HandleServerSettings, globalPrincipal, "/server/settings", nil,
url.Values{"send_log_retention_days": {"120"}, "inbound_spam_log_retention_days": {"3"}})
if rec.Code != http.StatusSeeOther {
t.Fatalf("POST = %d:\n%s", rec.Code, rec.Body.String())
}
if got, _ := h.store.GetSendLogRetentionDays(90); got != 120 {
t.Errorf("outbound retention = %d, want 120", got)
}
if raw, _ := h.store.GetSetting(store.InboundSpamLogRetentionDaysKey); raw != "" {
t.Errorf("an inbound retention was saved with the relay off: %q", raw)
}
}
func TestHelpHasTheInboundLogAndFilterListsTopics(t *testing.T) {
h, _ := inboundHandlers(t)
rec := send(h.HandleHelp, &globalPrincipal, "GET", "/help", nil, nil)
has(t, "help", rec.Body.String(),
`id="inbound-log"`, `id="filter-lists"`, `href="#inbound-log"`, `href="#filter-lists"`,
`With the inbound relay on there is a second number, for the Inbound log, with the same range.`)
// Both pages' help links land on them.
s := inboundLogStand{h: h, st: h.store, dir: filepath.Join(t.TempDir(), "antispam")}
h.lists = inbound.NewLists(h.store, s.dir)
has(t, "log", s.get(h.HandleInboundLog, "/inbound/log", globalPrincipal).Body.String(), `href="/help#inbound-log"`)
has(t, "lists", s.get(h.HandleFilterLists, "/inbound/filter-lists", globalPrincipal).Body.String(), `href="/help#filter-lists"`)
}
// An entry's count comes from the journal: a row counts for the entry that
// fits it on the list that decided it.
func TestFilterListsCountMatchesFromTheJournal(t *testing.T) {
s := newInboundLogStand(t)
for _, e := range [][3]string{{"deny", "domain", "bulk.example"}, {"allow", "address", "billing@vendor.example"}, {"deny", "ip", "203.0.113.0/24"}} {
if _, err := s.h.lists.Add(e[0], e[1], e[2], ""); err != nil {
t.Fatalf("Add %v: %v", e, err)
}
}
at := time.Date(2026, 10, 1, 9, 30, 0, 0, time.UTC)
for i, row := range []store.InboundSpamEntry{
{From: "promo@bulk.example", ClientIP: "198.51.100.9", Decision: store.SpamDecisionReject, Engine: store.SpamEngineRspamd, Detail: "5.7.1 Refused by this server's deny list"},
{From: "other@BULK.example", ClientIP: "198.51.100.9", Decision: store.SpamDecisionReject, Engine: store.SpamEngineRspamd, Detail: "5.7.1 Refused by this server's deny list"},
{From: "billing@vendor.example", ClientIP: "192.0.2.5", Decision: store.SpamDecisionAccept, Engine: store.SpamEngineRspamd, Detail: "no action, score -99.00 / 15.00; SELFPOST_ALLOW_FROM(-100.00)"},
// refused by the filter for its own reasons: nobody's match
{From: "promo@bulk.example", ClientIP: "198.51.100.9", Decision: store.SpamDecisionReject, Engine: store.SpamEngineRspamd, Detail: "5.7.1 Spam message rejected"},
} {
row.InboundDomain, row.To, row.At = "a.example.com", "alice@a.example.com", at.Add(time.Duration(i)*time.Hour)
if err := s.st.InsertInboundSpam(row); err != nil {
t.Fatal(err)
}
}
page := s.get(s.h.HandleFilterLists, "/inbound/filter-lists", globalPrincipal).Body.String()
has(t, "lists", page,
`<tr><th>Matched</th><td>2 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-10-01 10:30</td></tr>`,
`<td class="sp-nowrap">2</td>`,
`<tr><th>Matched</th><td>1 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-10-01 11:30</td></tr>`,
`<td class="sp-nowrap">1</td>`,
// the network nobody came from
`<td class="sp-muted sp-nowrap">never</td>`,
)
if got := strings.Count(page, `<td class="sp-muted sp-nowrap">never</td>`); got != 1 {
t.Errorf("%d entries say never, want the one network", got)
}
}
+50 -8
View File
@@ -10,7 +10,8 @@ import (
) )
// Server › Settings holds what is true of the whole instance, whoever is // Server › Settings holds what is true of the whole instance, whoever is
// signed in: log retention and the read-only view of the rate limits. A // signed in: log retention (the outbound log's, and the inbound log's while the
// inbound relay is on) and the read-only view of the rate limits. A
// user's own name, e-mail, password and DMARC default are on Account // user's own name, e-mail, password and DMARC default are on Account
// (handlers_account.go). Before placing a setting here, ask whose it is. // (handlers_account.go). Before placing a setting here, ask whose it is.
@@ -21,7 +22,7 @@ func (h *Handlers) HandleServerSettings(w http.ResponseWriter, r *http.Request)
} }
switch r.Method { switch r.Method {
case http.MethodGet: case http.MethodGet:
h.renderServerSettings(w, r, http.StatusOK, "", h.sendLogRetentionDays()) h.renderServerSettings(w, r, http.StatusOK, "", h.currentRetentions())
case http.MethodPost: case http.MethodPost:
h.submitServerSettings(w, r) h.submitServerSettings(w, r)
default: default:
@@ -30,8 +31,26 @@ func (h *Handlers) HandleServerSettings(w http.ResponseWriter, r *http.Request)
} }
} }
func (h *Handlers) renderServerSettings(w http.ResponseWriter, r *http.Request, status int, formErr string, formRetention int) { // retentions are the two journal retentions the Settings form shows: the
page := view.NewSettings(h.shellMeta(r), strconv.Itoa(formRetention), view.FormatRate(h.l1Messages(), h.l1Window())) // outbound log's, and the inbound log's while the inbound relay is on.
type retentions struct {
outbound, inbound int
}
func (h *Handlers) currentRetentions() retentions {
cur := retentions{outbound: h.sendLogRetentionDays()}
if h.cfg.InboundEnabled {
cur.inbound = h.inboundSpamLogRetentionDays()
}
return cur
}
func (h *Handlers) renderServerSettings(w http.ResponseWriter, r *http.Request, status int, formErr string, form retentions) {
page := view.NewSettings(h.shellMeta(r), strconv.Itoa(form.outbound), view.FormatRate(h.l1Messages(), h.l1Window()))
// The inbound field exists while there is an inbound journal to keep.
if h.cfg.InboundEnabled {
page.WithInbound(strconv.Itoa(form.inbound))
}
flash := "" flash := ""
if r.URL.Query().Has("saved") { if r.URL.Query().Has("saved") {
flash = "Settings saved." flash = "Settings saved."
@@ -41,7 +60,7 @@ func (h *Handlers) renderServerSettings(w http.ResponseWriter, r *http.Request,
} }
func (h *Handlers) submitServerSettings(w http.ResponseWriter, r *http.Request) { func (h *Handlers) submitServerSettings(w http.ResponseWriter, r *http.Request) {
current := h.sendLogRetentionDays() current := h.currentRetentions()
if err := r.ParseForm(); err != nil { if err := r.ParseForm(); err != nil {
h.renderServerSettings(w, r, http.StatusBadRequest, "Invalid form submission.", current) h.renderServerSettings(w, r, http.StatusBadRequest, "Invalid form submission.", current)
return return
@@ -51,17 +70,40 @@ func (h *Handlers) submitServerSettings(w http.ResponseWriter, r *http.Request)
h.renderServerSettings(w, r, http.StatusBadRequest, "Send log retention must be a whole number of days.", current) h.renderServerSettings(w, r, http.StatusBadRequest, "Send log retention must be a whole number of days.", current)
return return
} }
form := retentions{outbound: days, inbound: current.inbound}
if err := store.ValidateSendLogRetentionDays(days); err != nil { if err := store.ValidateSendLogRetentionDays(days); err != nil {
h.renderServerSettings(w, r, http.StatusBadRequest, err.Error(), days) h.renderServerSettings(w, r, http.StatusBadRequest, err.Error(), form)
return return
} }
if h.cfg.InboundEnabled {
inboundDays, err := strconv.Atoi(strings.TrimSpace(r.PostFormValue("inbound_spam_log_retention_days")))
if err != nil {
h.renderServerSettings(w, r, http.StatusBadRequest, "Inbound log retention must be a whole number of days.", form)
return
}
form.inbound = inboundDays
if err := store.ValidateInboundSpamLogRetentionDays(inboundDays); err != nil {
h.renderServerSettings(w, r, http.StatusBadRequest, err.Error(), form)
return
}
}
if err := h.store.SetSendLogRetentionDays(days); err != nil { if err := h.store.SetSendLogRetentionDays(days); err != nil {
logf("panel: settings: set send-log retention failed: %v", err) logf("panel: settings: set send-log retention failed: %v", err)
h.renderServerSettings(w, r, http.StatusInternalServerError, h.renderServerSettings(w, r, http.StatusInternalServerError,
"Could not save send log retention. Please check the logs and try again.", days) "Could not save send log retention. Please check the logs and try again.", form)
return return
} }
logf("panel: instance settings updated (send-log retention: %d days)", days) if h.cfg.InboundEnabled {
if err := h.store.SetInboundSpamLogRetentionDays(form.inbound); err != nil {
logf("panel: settings: set inbound-log retention failed: %v", err)
h.renderServerSettings(w, r, http.StatusInternalServerError,
"Could not save inbound log retention. Please check the logs and try again.", form)
return
}
logf("panel: instance settings updated (send-log retention: %d days, inbound-log retention: %d days)", days, form.inbound)
} else {
logf("panel: instance settings updated (send-log retention: %d days)", days)
}
http.Redirect(w, r, "/server/settings?saved=1", http.StatusSeeOther) http.Redirect(w, r, "/server/settings?saved=1", http.StatusSeeOther)
} }
+12
View File
@@ -56,6 +56,10 @@ type Settings struct {
Retention Box Retention Box
RetentionDays string RetentionDays string
// Inbound is whether the inbound relay is on, which gives the retention box
// its second field; InboundRetentionDays is what that field shows.
Inbound bool
InboundRetentionDays string
RateLimits Box RateLimits Box
RateFacts []Fact RateFacts []Fact
@@ -86,6 +90,14 @@ func NewSettings(m Meta, retention, level1 string) *Settings {
} }
} }
// WithInbound adds the retention of the Inbound log to the Log retention box.
// days is what the field shows, like retention in NewSettings. The field exists
// only while the inbound relay is on: with it off there is no journal to keep.
func (s *Settings) WithInbound(days string) *Settings {
s.Inbound, s.InboundRetentionDays = true, days
return s
}
// WithResult shows the result of the last action, or its refusal, between the // WithResult shows the result of the last action, or its refusal, between the
// head and the form. // head and the form.
func (s *Settings) WithResult(flash, formErr string) *Settings { func (s *Settings) WithResult(flash, formErr string) *Settings {
+29 -1
View File
@@ -105,18 +105,41 @@ type Tag struct {
Label string Label string
} }
// Sheet is what does not fit a table row, opened over the page from the row's
// tag (partials "sheet_tag" and "sheet"; used by the Inbound log, where it says
// why a message was decided the way it was). It is the browser's own popover:
// the tag is a button that names the sheet by ID, so ID is unique on the page.
// Its content is a table of name and value.
type Sheet struct {
ID string
Tag Tag
Title string
Rows []SheetRow
}
// SheetRow is one name and its value in a sheet. A value of several lines is
// one line each; Mono is for what is read as written - a reply, a list of
// symbols.
type SheetRow struct {
Label string
Lines []string
Mono bool
}
// statusClass is the one mapping of status to Bulma tag classes — box 03 of the // statusClass is the one mapping of status to Bulma tag classes — box 03 of the
// kit page. A page passes the status it knows (a DKIM check says ok, a message // kit page. A page passes the status it knows (a DKIM check says ok, a message
// says deferred) and never picks a colour. // says deferred) and never picks a colour.
var statusClass = map[string]string{ var statusClass = map[string]string{
"ok": "tag is-success is-light", "published": "tag is-success is-light", "ok": "tag is-success is-light", "published": "tag is-success is-light",
"running": "tag is-success is-light", "sent": "tag is-success is-light", "running": "tag is-success is-light", "sent": "tag is-success is-light",
"accept": "tag is-success is-light",
"warn": "tag is-warning is-light", "deferred": "tag is-warning is-light", "warn": "tag is-warning is-light", "deferred": "tag is-warning is-light",
"quiet": "tag is-warning is-light", "quiet": "tag is-warning is-light", "tempfail": "tag is-warning is-light",
"error": "tag is-danger is-light", "fail": "tag is-danger is-light", "error": "tag is-danger is-light", "fail": "tag is-danger is-light",
"mismatch": "tag is-danger is-light", "bounced": "tag is-danger is-light", "mismatch": "tag is-danger is-light", "bounced": "tag is-danger is-light",
"reject": "tag is-danger is-light",
"quarantine": "tag is-primary is-light", "held": "tag is-primary is-light", "quarantine": "tag is-primary is-light", "held": "tag is-primary is-light",
"global": "tag is-primary is-light", "global": "tag is-primary is-light",
@@ -503,6 +526,9 @@ type Shell struct {
ShowDMARC bool ShowDMARC bool
ShowInbound bool ShowInbound bool
ShowServer bool ShowServer bool
// ShowFilterLists is the Filter lists entry of Inbound: the lists are the
// instance's, so only the global role has them.
ShowFilterLists bool
} }
// Frame is what layout.html is executed with: the shell and the page's own // Frame is what layout.html is executed with: the shell and the page's own
@@ -523,6 +549,8 @@ func (e *Engine) shell(m Meta, copyright, sourceURL string) Shell {
ShowQueue: m.IsGlobal, ShowQueue: m.IsGlobal,
ShowOutbound: m.IsGlobal || m.HasOutbound, ShowOutbound: m.IsGlobal || m.HasOutbound,
ShowInbound: e.inboundEnabled && (m.IsGlobal || m.HasInbound), ShowInbound: e.inboundEnabled && (m.IsGlobal || m.HasInbound),
ShowFilterLists: e.inboundEnabled && m.IsGlobal,
} }
s.ShowDMARC = e.dmarcEnabled && s.ShowOutbound s.ShowDMARC = e.dmarcEnabled && s.ShowOutbound
switch { switch {
+7 -6
View File
@@ -71,7 +71,8 @@ func TestStatusTagMapsEveryStatusOfTheKitTable(t *testing.T) {
"ok": "tag is-success is-light", "published": "tag is-success is-light", "ok": "tag is-success is-light", "published": "tag is-success is-light",
"running": "tag is-success is-light", "sent": "tag is-success is-light", "running": "tag is-success is-light", "sent": "tag is-success is-light",
"warn": "tag is-warning is-light", "deferred": "tag is-warning is-light", "warn": "tag is-warning is-light", "deferred": "tag is-warning is-light",
"quiet": "tag is-warning is-light", "quiet": "tag is-warning is-light",
"accept": "tag is-success is-light", "tempfail": "tag is-warning is-light", "reject": "tag is-danger is-light",
"error": "tag is-danger is-light", "fail": "tag is-danger is-light", "error": "tag is-danger is-light", "fail": "tag is-danger is-light",
"mismatch": "tag is-danger is-light", "bounced": "tag is-danger is-light", "mismatch": "tag is-danger is-light", "bounced": "tag is-danger is-light",
"quarantine": "tag is-primary is-light", "global": "tag is-primary is-light", "quarantine": "tag is-primary is-light", "global": "tag is-primary is-light",
@@ -413,7 +414,7 @@ func TestShellVisibilityFollowsRoleAndFlags(t *testing.T) {
wantIn bool wantIn bool
}{ }{
{"global, everything on", on, Meta{IsGlobal: true}, {"global, everything on", on, Meta{IsGlobal: true},
Shell{Home: "/overview", ShowOverview: true, ShowOutbound: true, ShowQueue: true, ShowDMARC: true, ShowInbound: true, ShowServer: true}, true}, Shell{Home: "/overview", ShowOverview: true, ShowOutbound: true, ShowQueue: true, ShowDMARC: true, ShowInbound: true, ShowServer: true, ShowFilterLists: true}, true},
{"global, inbound and DMARC off", off, Meta{IsGlobal: true}, {"global, inbound and DMARC off", off, Meta{IsGlobal: true},
Shell{Home: "/overview", ShowOverview: true, ShowOutbound: true, ShowQueue: true, ShowServer: true}, true}, Shell{Home: "/overview", ShowOverview: true, ShowOutbound: true, ShowQueue: true, ShowServer: true}, true},
{"domain admin with both reaches", on, Meta{HasOutbound: true, HasInbound: true}, {"domain admin with both reaches", on, Meta{HasOutbound: true, HasInbound: true},
@@ -446,19 +447,19 @@ func renderShell(t *testing.T, e *Engine, m Meta) string {
return rec.Body.String() return rec.Body.String()
} }
// The menu has the 2.0 entries and none of the ones that are not in 2.0 (plan // The menu has the entries that are built and none of the ones that are not (plan
// § Routes): no Inbound Log, Queue, Quarantine or Filter lists, no Preflight. // § Routes): no Inbound Queue or Quarantine, no Preflight.
func TestMenuHoldsTheTwoDotZeroEntries(t *testing.T) { func TestMenuHoldsTheTwoDotZeroEntries(t *testing.T) {
out := renderShell(t, kitEngine(t), Meta{Title: "t", User: "admin", IsGlobal: true}) out := renderShell(t, kitEngine(t), Meta{Title: "t", User: "admin", IsGlobal: true})
for _, href := range []string{ for _, href := range []string{
"/overview", "/outbound/domains", "/outbound/log", "/outbound/queue", "/outbound/dmarc", "/overview", "/outbound/domains", "/outbound/log", "/outbound/queue", "/outbound/dmarc",
"/inbound/domains", "/server/health", "/server/log", "/server/backup", "/server/users", "/server/settings", "/inbound/domains", "/inbound/log", "/inbound/filter-lists", "/server/health", "/server/log", "/server/backup", "/server/users", "/server/settings",
"/account", "/help", "/license", "/account", "/help", "/license",
} { } {
mustContain(t, out, `href="`+href+`"`) mustContain(t, out, `href="`+href+`"`)
} }
for _, gone := range []string{ for _, gone := range []string{
"/inbound/log", "/inbound/queue", "/inbound/quarantine", "/inbound/filter-lists", "/server/preflight", "/inbound/queue", "/inbound/quarantine", "/server/preflight",
`href="/status"`, `href="/domains"`, `href="/users"`, `href="/settings"`, `href="/status"`, `href="/domains"`, `href="/users"`, `href="/settings"`,
} { } {
mustNotContain(t, out, gone) mustNotContain(t, out, gone)
+1 -1
View File
@@ -113,7 +113,7 @@ func NewDMARCHub(m Meta, in IngestInput) *DMARCHub {
last, last,
{Label: "This week", Value: Plain(FormatCount(int64(in.KeptThisWeek)) + " kept · " + plural(int64(in.ParseFailures), "parse failure", "parse failures"))}, {Label: "This week", Value: Plain(FormatCount(int64(in.KeptThisWeek)) + " kept · " + plural(int64(in.ParseFailures), "parse failure", "parse failures"))},
{Label: "Hosted addresses", Value: Plain(in.Hosted), Mono: true, Small: true}, {Label: "Hosted addresses", Value: Plain(in.Hosted), Mono: true, Small: true},
{Label: "Retention", Value: Plain(FormatCount(int64(in.RetentionMax)) + " reports or " + plural(int64(in.RetentionDays), "day", "days"))}, {Label: "Retention", Value: Plain(FormatCount(int64(in.RetentionMax)) + " reports a domain or " + plural(int64(in.RetentionDays), "day", "days"))},
}, },
Recent: Box{No: "02", Title: "Recent reports"}, Recent: Box{No: "02", Title: "Recent reports"},
Empty: EmptyState{Icon: "ti-report-analytics", Text: Plain("No reports yet.")}, Empty: EmptyState{Icon: "ti-report-analytics", Text: Plain("No reports yet.")},
+1 -1
View File
@@ -12,7 +12,7 @@ func TestDMARCHubShowsIngestAndTheLatestReports(t *testing.T) {
pageHas(t, "DMARC reports", out, pageHas(t, "DMARC reports", out,
`<h1 class="title is-3">DMARC reports</h1>`, `<code>rua=</code>`, `<h1 class="title is-3">DMARC reports</h1>`, `<code>rua=</code>`,
`<h2>Ingest</h2>`, `<span class="tag is-success is-light">ok</span>`, `<h2>Ingest</h2>`, `<span class="tag is-success is-light">ok</span>`,
`<dd>2026-09-21 06:12 UTC</dd>`, `<dd>14 kept · 0 parse failures</dd>`, `<dt>Hosted addresses</dt>`, `<dd class="sp-mono sp-small">dmarc-reports&#43;&lt;domain&gt;@mail.example.org</dd>`, `<dd>500 reports or 90 days</dd>`, `<dd>2026-09-21 06:12 UTC</dd>`, `<dd>14 kept · 0 parse failures</dd>`, `<dt>Hosted addresses</dt>`, `<dd class="sp-mono sp-small">dmarc-reports&#43;&lt;domain&gt;@mail.example.org</dd>`, `<dd>500 reports a domain or 90 days</dd>`,
`<th>Received, UTC</th>`, `09-21 06:12`, `<a href="/outbound/dmarc/domains/1">example.org</a>`, `<th>Received, UTC</th>`, `09-21 06:12`, `<a href="/outbound/dmarc/domains/1">example.org</a>`,
"<td>1 903</td>", `<span class="tag is-warning is-light">7</span>`, `<td>0</td><td class="sp-actions"><a href="/outbound/dmarc/reports/41">View</a>`, "<td>1 903</td>", `<span class="tag is-warning is-light">7</span>`, `<td>0</td><td class="sp-actions"><a href="/outbound/dmarc/reports/41">View</a>`,
) )
+50 -1
View File
@@ -38,6 +38,8 @@ var pageFixtures = map[string]func() any{
"in-domains": func() any { return inDomainsFixture() }, "in-domains": func() any { return inDomainsFixture() },
"in-domain": func() any { return inDomainFixture() }, "in-domain": func() any { return inDomainFixture() },
"in-domain-delete": func() any { return inDomainDeleteFixture() }, "in-domain-delete": func() any { return inDomainDeleteFixture() },
"in-log": func() any { return inLogFixture() },
"in-filter-lists": func() any { return inFilterListsFixture() },
"out-log": func() any { return outLogFixture() }, "out-log": func() any { return outLogFixture() },
"out-message": func() any { return outMessageFixture() }, "out-message": func() any { return outMessageFixture() },
@@ -131,7 +133,7 @@ func accountFixture() *Account {
// settingsFixture is the mockup's Settings: thirty days of log, a limit of 600 // settingsFixture is the mockup's Settings: thirty days of log, a limit of 600
// messages an hour. // messages an hour.
func settingsFixture() *Settings { func settingsFixture() *Settings {
return NewSettings(admin(), "30", FormatRate(600, 3600)) return NewSettings(admin(), "30", FormatRate(600, 3600)).WithInbound("90")
} }
// outDomainsFixture is the mockup's Outbound domains: three domains, the first // outDomainsFixture is the mockup's Outbound domains: three domains, the first
@@ -411,3 +413,50 @@ func userFixture() *UserForm {
func userDeleteFixture() *UserDelete { func userDeleteFixture() *UserDelete {
return NewUserDelete(admin(), UserDeleteInput{ID: 3, Username: "acme-ops", AllOutbound: true, Inbound: []string{"acme.io"}}) return NewUserDelete(admin(), UserDeleteInput{ID: 3, Username: "acme-ops", AllOutbound: true, Inbound: []string{"acme.io"}})
} }
// inLogFixture is the mockup's Inbound log: six rows of a week's mail, as the
// global role sees them - an accepted message scored as spam and one let through
// by an Allow entry, refusals by Postfix and by the filter, and a tempfail.
func inLogFixture() *InLog {
p := NewInLog(admin(), 90, []string{"lists.example.org", "acme.io"},
InLogFilter{From: "2026-09-14", To: "2026-09-21"})
var id int64
row := func(clock, from, client, to, subject, decision, engine, detail string) InLogRow {
at, err := time.Parse("2006-01-02 15:04:05", "2026-09-21 "+clock)
if err != nil {
panic(err)
}
id++
return p.NewRow(InLogRowInput{ID: id, At: at, From: from, Client: client, To: to, Subject: subject,
Decision: decision, Engine: engine, Detail: detail})
}
return p.WithRows([]InLogRow{
row("14:00:18", "t.berg@uni.example", "192.0.2.77", "dev@lists.example.org", "Re: [dev] patch series v3",
"accept", "rspamd", "no action, score 0.40 / 15.00"),
row("13:59:02", "invoice@partner-billing.example", "203.0.113.140", "owner@lists.example.org", "Outstanding invoice — action required",
"accept", "rspamd", "add header, score 9.80 / 15.00; PHISHING(5.00) FORGED_SENDER(3.00) R_SPF_FAIL(1.00)"),
row("13:57:41", "promo@bulk.example", "198.51.100.90", "sales@lists.example.org", "",
"reject", "postfix", "550 5.1.1 <sales@lists.example.org>: Recipient address rejected: User unknown in relay recipient table"),
row("13:51:30", "billing@partner.example", "198.51.100.12", "owner@lists.example.org", "",
"reject", "rspamd", "5.7.1 Spam message rejected"),
row("13:44:02", "hr@partner.example", "192.0.2.31", "jobs@acme.io", "",
"tempfail", "rspamd", "4.7.1 Try again later"),
row("13:30:55", "noreply@github.example", "192.0.2.200", "security@lists.example.org", "[advisory] new report",
"accept", "rspamd", "no action, score -98.10 / 15.00; SELFPOST_ALLOW_FROM(-100.00) R_DKIM_ALLOW(-0.20)"),
}, 1, 38)
}
// inFilterListsFixture is the mockup's Filter lists: three senders denied, two
// allowed, and a filter that reads them.
func inFilterListsFixture() *FilterLists {
day := func(m time.Month, d int) time.Time { return time.Date(2026, m, d, 0, 0, 0, 0, time.UTC) }
at := func(d, h, m int) time.Time { return time.Date(2026, 9, d, h, m, 0, 0, time.UTC) }
return NewFilterLists(admin(), FilterListsInput{FilterOn: true, LastWritten: time.Date(2026, 9, 11, 8, 40, 0, 0, time.UTC), RetentionDays: 90}).
WithEntries([]FilterEntryInput{
{ID: 1, List: "deny", Kind: "domain", Value: "bulk.example", Note: "newsletter nobody subscribed to", Added: day(9, 2), Hits: 41, LastHit: at(21, 13, 57)},
{ID: 2, List: "deny", Kind: "ip", Value: "198.51.100.0/24", Note: "dictionary attack on lists.example.org", Added: day(8, 19), Hits: 1203, LastHit: at(21, 9, 12)},
{ID: 3, List: "deny", Kind: "address", Value: "win@lottery.example", Added: day(7, 30)},
{ID: 4, List: "allow", Kind: "domain", Value: "partner.example", Note: "invoices were scored as spam", Added: day(9, 11), Hits: 7, LastHit: at(19, 16, 40)},
{ID: 5, List: "allow", Kind: "address", Value: "noreply@github.example", Note: "security advisories", Added: day(6, 4), Hits: 18, LastHit: at(21, 13, 30)},
})
}
-2
View File
@@ -46,8 +46,6 @@ const mockupDir = "../../../docs/assets/panel-redesign/panel"
// with its own plan (panel-redesign.md § Room for roadmap items). They are the // with its own plan (panel-redesign.md § Room for roadmap items). They are the
// only keys of outlines.json the finished panel may lack. // only keys of outlines.json the finished panel may lack.
var deferredScreens = map[string]string{ var deferredScreens = map[string]string{
"in-log": "inbound-antispam-panel, 2.1.0",
"in-filter-lists": "inbound-antispam-panel, 2.1.0",
"in-queue": "queue as a table / inbound queue, undecided", "in-queue": "queue as a table / inbound queue, undecided",
"in-quarantine": "inbound-quarantine, candidate", "in-quarantine": "inbound-quarantine, candidate",
"in-quarantine-message": "inbound-quarantine, candidate", "in-quarantine-message": "inbound-quarantine, candidate",
+30 -3
View File
@@ -52,6 +52,9 @@ const (
HelpUpstream = "upstream" HelpUpstream = "upstream"
HelpRecipients = "recipients" HelpRecipients = "recipients"
HelpFilter = "filter" HelpFilter = "filter"
// HelpInboundLog and HelpFilterLists: the plain "log" is the Outbound log's.
HelpInboundLog = "inbound-log"
HelpFilterLists = "filter-lists"
// Server // Server
HelpBackup = "backup" HelpBackup = "backup"
HelpUsers = "users" HelpUsers = "users"
@@ -109,6 +112,8 @@ func NewHelp(m Meta, inbound bool) *Help {
{Label: "Upstream", Icon: "ti-arrow-forward-up", Href: "#" + HelpUpstream}, {Label: "Upstream", Icon: "ti-arrow-forward-up", Href: "#" + HelpUpstream},
{Label: "Valid recipients", Icon: "ti-address-book", Href: "#" + HelpRecipients}, {Label: "Valid recipients", Icon: "ti-address-book", Href: "#" + HelpRecipients},
{Label: "Spam filter", Icon: "ti-shield-check", Href: "#" + HelpFilter}, {Label: "Spam filter", Icon: "ti-shield-check", Href: "#" + HelpFilter},
{Label: "Inbound log", Icon: "ti-list-details", Href: "#" + HelpInboundLog},
{Label: "Filter lists", Icon: "ti-filter", Href: "#" + HelpFilterLists},
}) })
} }
if m.IsGlobal { if m.IsGlobal {
@@ -184,13 +189,35 @@ var inboundTopics = []HelpTopic{
Rich("The spam filter belongs to the server, not to a domain. The server's administrator attaches it in the server's ", Code(".env"), Rich("The spam filter belongs to the server, not to a domain. The server's administrator attaches it in the server's ", Code(".env"),
", and from then on every inbound domain is screened by it before mail is handed on; the box shows on or off and has nothing to switch. Off means mail goes to the upstream unfiltered. While the filter is down, mail is either accepted unfiltered (the default) or deferred, so the sending server tries again later, as the server's setting says; the box states which."), ", and from then on every inbound domain is screened by it before mail is handed on; the box shows on or off and has nothing to switch. Off means mail goes to the upstream unfiltered. While the filter is down, mail is either accepted unfiltered (the default) or deferred, so the sending server tries again later, as the server's setting says; the box states which."),
}}, }},
{ID: HelpInboundLog, Title: "Inbound log", Body: []Text{
Rich("One row per recipient of a message offered on port 25 for an inbound domain, with what was decided: ",
TagOf("accept", ""), " — handed on to the upstream; ", TagOf("reject", ""), " — refused for good; ",
TagOf("tempfail", ""), " — the sender was asked to come back later; ", TagOf("quarantine", ""), " — held by the filter. ",
Em("Why"), " names who decided. ", Code("rspamd"), " is the spam filter, with its action, the score against the reject threshold and the heaviest symbols; ",
Code("postfix"), " is the relay's own rule — an unknown recipient, a sender domain that does not exist — with its reply; ",
Code("selfpost"), " means nobody judged the message: it was relayed with no filter attached. Accept says what this server did, not what the upstream did with the message afterwards.",
Br(), Br(),
"Accepted mail is recorded as it passes; refused mail is read from ", Code("mail.log"),
", so its row has no subject, and a message refused after its text was sent is one row however many recipients it had. A refusal to relay for a domain that is not an inbound one is every scanner on the internet and is not recorded. Rows are kept for the number of days set in Settings, and never more than 10 000."),
}},
{ID: HelpFilterLists, Title: "Filter lists", Body: []Text{
Rich("Two lists for the whole server that correct the spam filter by hand. An entry is a sender address, a sender domain — that domain, not its subdomains — or a client address or network in CIDR form. A sender on the Deny list is refused outright. A sender on the Allow list is given a weight no spam score outweighs, so the filter lets it through. When entries of both lists match one message, ",
Strong("Deny wins"), ".",
Br(), Br(),
"SelfPost does not apply the lists itself: it writes them as files under ", Code("/data/antispam/"),
", and rspamd rereads a file within seconds of a change. With the spam filter off the lists are kept and can be edited, but nothing reads them. The sender matched is the envelope sender — the one in the log — not the From line a mail program shows. Matched counts the messages in the Inbound log that the entry decided, so it reaches back as far as the log does; never means not once in that time, and such an entry is one to consider removing."),
}},
} }
var serverTopics = []HelpTopic{ var serverTopics = []HelpTopic{
{ID: HelpBackup, Title: "Backup and moving a domain", Body: []Text{ {ID: HelpBackup, Title: "Backup and moving a domain", Body: []Text{
Rich("Full backup downloads one archive of the whole instance: ", Code("data/"), " (the database, DKIM keys, application credentials and the queue), ", Rich("Full backup downloads one archive of the whole instance: ", Code("data/"), " (the database, DKIM keys and application credentials), ",
Code("docker-compose.yml"), ", ", Code(".env"), " and ", Code("certs/"), Code("docker-compose.yml"), ", ", Code(".env"), " and ", Code("certs/"),
". It holds secrets, so leave Encrypt with a password ticked: the file is sealed as ", Code(".spbk"), ". The Postfix queue is not in it. The backup is taken while the server is running, and Postfix keeps moving messages through the queue, "+
"so a copy could hold a message twice or miss it; mail still waiting for delivery is therefore not carried over, and a restored server "+
"starts with an empty queue. To keep the queue, stop the container and archive the project directory instead. The private key in ", Code("certs/"),
" has to be readable by the panel: give it to group 998 on the host (", Code("chgrp 998"), ", ", Code("chmod 640"),
"). Until then the panel refuses to make a backup and names the file, rather than hand over an archive without the key. It holds secrets, so leave Encrypt with a password ticked: the file is sealed as ", Code(".spbk"),
", the password is not stored, and without it nothing can be recovered. To restore, unpack it into an empty project directory (decrypt a ", ", the password is not stored, and without it nothing can be recovered. To restore, unpack it into an empty project directory (decrypt a ",
Code(".spbk"), " first with ", Code("selfpost-backup -decrypt"), Code(".spbk"), " first with ", Code("selfpost-backup -decrypt"),
") and start the same SelfPost version that made it; the panel refuses data from another version. The reverse-proxy configuration and ", ") and start the same SelfPost version that made it; the panel refuses data from another version. The reverse-proxy configuration and ",
@@ -204,7 +231,7 @@ var serverTopics = []HelpTopic{
}}, }},
{ID: HelpSettings, Title: "Settings", Body: []Text{ {ID: HelpSettings, Title: "Settings", Body: []Text{
Rich("Settings holds what is true of the whole instance. Log retention is how many days the Outbound log keeps its rows, from 7 to 365 and 90 by default; a sweep every six hours deletes older ones, and it is the main driver of how fast ", Rich("Settings holds what is true of the whole instance. Log retention is how many days the Outbound log keeps its rows, from 7 to 365 and 90 by default; a sweep every six hours deletes older ones, and it is the main driver of how fast ",
Code("/data"), " grows. It does not change the rotation of ", Code("mail.log"), Code("/data"), " grows. With the inbound relay on there is a second number, for the Inbound log, with the same range. It does not change the rotation of ", Code("mail.log"),
", which is daily with about two weeks kept. The sending rate limit shown under it is level 1, the per-client-IP ceiling set in ", ", which is daily with about two weeks kept. The sending rate limit shown under it is level 1, the per-client-IP ceiling set in ",
Code(".env"), ": changing it needs a restart, and no level-2 limit can exceed it."), Code(".env"), ": changing it needs a restart, and no level-2 limit can exceed it."),
}}, }},
+27
View File
@@ -164,3 +164,30 @@ func TestHelpLinksStayInsideWhatTheirViewerIsShown(t *testing.T) {
landsOn("DMARC hub", hub.Recent.Help) landsOn("DMARC hub", hub.Recent.Help)
} }
} }
// The Inbound log and Filter lists have their topics, after the spam filter's, in
// the menu and on the page, for everyone who is shown Inbound; the Settings topic
// mentions the second retention.
func TestHelpHasTheInboundLogAndFilterListsTopics(t *testing.T) {
out := renderSignedIn(t, "help", NewHelp(admin(), true))
pageHas(t, "Help", out,
`<li><a href="#inbound-log"><i class="ti ti-list-details"></i>Inbound log</a></li>`,
`<li><a href="#filter-lists"><i class="ti ti-filter"></i>Filter lists</a></li>`,
`<div class="sp-help-topic" id="inbound-log"><h3>Inbound log</h3>`,
`<span class="tag is-primary is-light">quarantine</span> — held by the filter. <em>Why</em> names who decided.`,
`a message refused after its text was sent is one row however many recipients it had.`,
`and never more than 10 000.`,
`<div class="sp-help-topic" id="filter-lists"><h3>Filter lists</h3>`,
`<strong>Deny wins</strong>.`, `files under <code>/data/antispam/</code>`,
`With the inbound relay on there is a second number, for the Inbound log, with the same range.`)
for _, v := range helpViewers {
out := renderSignedIn(t, "help", NewHelp(v.meta, v.inbound))
shown := strings.Contains(out, `id="inbound-log"`)
if want := v.inbound && (v.meta.IsGlobal || v.meta.HasInbound); shown != want {
t.Errorf("%s: inbound log topic shown = %v, want %v", v.name, shown, want)
}
}
if HelpInboundLog != "inbound-log" || HelpFilterLists != "filter-lists" {
t.Errorf("anchors %q and %q", HelpInboundLog, HelpFilterLists)
}
}
+1 -1
View File
@@ -19,7 +19,7 @@ func TestInDomainShowsTheMockupsBoxes(t *testing.T) {
`<h2>Spam filter</h2>`, `href="#filter"`, `href="/inbound/domains/1/delete"`, `Delete domain`, `<h2>Spam filter</h2>`, `href="#filter"`, `href="/inbound/domains/1/delete"`, `Delete domain`,
) )
// What has no source in this build is not drawn. // What has no source in this build is not drawn.
pageLacks(t, "Inbound domain", out, `rspamd`, `quarantine`, `Quarantine`, `Log for this domain`, `/inbound/log`, `/inbound/quarantine`, `Allow and deny`) pageLacks(t, "Inbound domain", out, `rspamd`, `quarantine`, `Quarantine`, `Log for this domain`, `/inbound/quarantine`, `Allow and deny`)
} }
// The Spam filter box states what the server is set up with, and what of it the // The Spam filter box states what the server is set up with, and what of it the
+202
View File
@@ -0,0 +1,202 @@
package view
// The data of Inbound › Filter lists (templates/in_filter_lists.html, mockup
// in-filter-lists.html): the allow and deny lists that correct the external spam
// filter by hand. The lists are instance-wide, so the page is the global role's.
//
// A page is a typed struct that embeds Meta, built by a constructor and filled
// by With… methods; the handler reads the lists, the view decides what is said
// and how it is worded.
import (
"net/url"
"strconv"
"time"
)
const inboundFilterLists = "/inbound/filter-lists"
// FilterListsHref is the address of the page with the add form filled in: list
// is "allow" or "deny", kind "address", "domain" or "ip". The Inbound log links
// its rows here.
func FilterListsHref(list, kind, value string) string {
return inboundFilterLists + "?list=" + url.QueryEscape(list) + "&kind=" + url.QueryEscape(kind) + "&value=" + url.QueryEscape(value)
}
// FilterLists is the data of the page: the form that adds an entry, the two
// lists side by side and what happens to them after that.
type FilterLists struct {
Meta
Head Head
Flash *Flash
Add Box
Action string
ListOptions []Option
KindOptions []Option
// List, Kind, Value and Note are what the form shows: blank, prefilled from
// the Inbound log, or what was typed into a form that was refused.
List string
Kind string
Value string
Note string
Deny FilterList
Allow FilterList
Where Box
WhereFacts []Fact
retentionDays int
}
// FilterList is one of the two boxes of entries.
type FilterList struct {
Box Box
Rows []FilterRow
Empty EmptyState
}
// FilterRow is one entry of a list: the value, the kind of entry as a tag that
// opens the entry's sheet (the note, the day it was added, how often it decided
// a message and when last), how often it decided, and the way to remove it.
// Matched is empty for an entry that never did; the table then says "never".
type FilterRow struct {
Value string
Entry Sheet
Matched string
Remove RowAction
}
// FilterEntryInput is what the page knows of one entry of a list. Hits and
// LastHit are counted from the Inbound log, so they reach back as far as it
// does.
type FilterEntryInput struct {
ID int64
List string
Kind string
Value string
Note string
Added time.Time
Hits int
LastHit time.Time
}
// FilterListsInput is the state of the filter the lists are written for: whether
// a filter is attached at all, and when the map files it reads were last written.
type FilterListsInput struct {
FilterOn bool
LastWritten time.Time
// RetentionDays is how long the Inbound log keeps a row: how far back an
// entry's matches are counted.
RetentionDays int
}
// NewFilterLists builds the page. The form starts on the Deny list for a sender
// address, the commonest correction.
func NewFilterLists(m Meta, in FilterListsInput) *FilterLists {
m.Title, m.Section, m.Page = "Filter lists", "inbound", "filter-lists"
p := &FilterLists{
Meta: m,
Head: Head{
Kicker: "Inbound · global role",
Title: "Filter lists",
Lead: Plain("Correct the spam filter by hand: senders it must always let through, and senders it must always turn away. One pair of lists for the whole instance."),
},
Add: Box{No: "01", Title: "Add an entry", Help: topicLink(HelpFilterLists, "How the lists are applied")},
Action: inboundFilterLists,
ListOptions: []Option{
{Value: "deny", Label: "Deny — always turn away"},
{Value: "allow", Label: "Allow — always let through"},
},
KindOptions: []Option{
{Value: "address", Label: "a sender address"},
{Value: "domain", Label: "a sender domain"},
{Value: "ip", Label: "a client address or network"},
},
List: "deny",
Kind: "address",
Deny: FilterList{Box: Box{No: "02", Title: "Deny"}, Empty: EmptyState{Icon: "ti-filter", Text: Plain("No sender is turned away by hand.")}},
Allow: FilterList{Box: Box{No: "03", Title: "Allow"}, Empty: EmptyState{Icon: "ti-filter", Text: Plain("No sender is let through by hand.")}},
Where: Box{No: "04", Title: "Where the lists go"},
}
p.Deny.Box.End = Plain(plural(0, "entry", "entries") + " · wins over Allow")
p.Allow.Box.End = Plain(plural(0, "entry", "entries"))
p.retentionDays = in.RetentionDays
p.setWhere(in)
return p
}
// setWhere writes box 04: the tag that says whether anything reads the lists,
// and the four facts about where they go. With no filter attached the lists are
// still kept and editable, and the box says so instead of promising an effect.
func (p *FilterLists) setWhere(in FilterListsInput) {
applied := Plain("rspamd, from map files it rereads within seconds of a change")
p.Where.End = Rich(TagOf("ok", "applied"))
if !in.FilterOn {
p.Where.End = Rich(TagOf("off", "not applied"))
applied = Rich("Nothing — the spam filter is off. The lists are kept and can be edited; nothing reads them until ",
Code("INBOUND_ANTISPAM_MILTER"), " is set.")
}
written := "never"
if !in.LastWritten.IsZero() {
written = in.LastWritten.UTC().Format("2006-01-02 15:04") + " UTC"
}
p.WhereFacts = []Fact{
{Label: "Applied by", Value: applied},
{Label: "Files", Value: Rich(Mono("/data/antispam/"), " — six, one for each list and kind of entry")},
{Label: "Last written", Value: Plain(written)},
{Label: "When both lists match", Value: Plain("Deny wins, whichever entries they are.")},
}
}
// WithEntries sets the two tables, each counted in its box head.
func (p *FilterLists) WithEntries(entries []FilterEntryInput) *FilterLists {
p.Deny.Rows, p.Allow.Rows = nil, nil
for _, e := range entries {
id := strconv.FormatInt(e.ID, 10)
window := "in the last " + plural(int64(p.retentionDays), "day", "days")
sheet := []SheetRow{
{Label: "Note", Lines: []string{orDash(e.Note)}},
{Label: "Added", Lines: []string{e.Added.UTC().Format("2006-01-02")}, Mono: true},
}
row := FilterRow{
Value: e.Value,
Remove: RowAction{Label: "Remove", Post: inboundFilterLists + "/" + id + "/delete", Danger: true},
}
if e.Hits > 0 {
row.Matched = strconv.Itoa(e.Hits)
sheet = append(sheet,
SheetRow{Label: "Matched", Lines: []string{row.Matched + " " + window}},
SheetRow{Label: "Last match", Lines: []string{e.LastHit.UTC().Format("2006-01-02 15:04")}, Mono: true})
} else {
sheet = append(sheet, SheetRow{Label: "Matched", Lines: []string{"never " + window}})
}
row.Entry = Sheet{ID: "entry-" + id, Tag: Tag{Label: e.Kind}, Title: e.Value, Rows: sheet}
switch e.List {
case "deny":
p.Deny.Rows = append(p.Deny.Rows, row)
case "allow":
p.Allow.Rows = append(p.Allow.Rows, row)
}
}
p.Deny.Box.End = Plain(plural(int64(len(p.Deny.Rows)), "entry", "entries") + " · wins over Allow")
p.Allow.Box.End = Plain(plural(int64(len(p.Allow.Rows)), "entry", "entries"))
return p
}
// WithForm sets what the add form shows. The handler passes only what it has
// checked to be a list and a kind the form offers, or what was submitted to a
// form it is showing again.
func (p *FilterLists) WithForm(list, kind, value, note string) *FilterLists {
p.List, p.Kind, p.Value, p.Note = list, kind, value, note
return p
}
// WithResult shows the result of the last action, or the refusal of the add
// form, between the head and the first box.
func (p *FilterLists) WithResult(flash, formErr string) *FilterLists {
p.Flash = resultFlash(flash, formErr)
return p
}
+231
View File
@@ -0,0 +1,231 @@
package view
// The data of the Inbound log (templates/in_log.html, mockup in-log.html): what
// port 25 decided about each message offered for an inbound domain, one row per
// recipient. Unlike the Outbound log it does not poll: a decision does not change
// after it is made, and the page is read to find one row, not watched.
//
// A page is a typed struct that embeds Meta, built by a constructor and filled
// by With… methods; the handler reads the journal, the view decides what is
// said and how it is worded.
import (
"net/url"
"strconv"
"strings"
"time"
)
const inboundLog = "/inbound/log"
// InLogFilter is what the filters of the Inbound log are set to, as the strings
// the form sends: a domain name, a decision and two days (2006-01-02). Empty
// means no narrowing.
type InLogFilter struct {
Domain string
Decision string
From string
To string
}
func (f InLogFilter) active() bool { return f != InLogFilter{} }
// query is the query string of a page of the log: the filters and the page, each
// left out when it says nothing, so the first unfiltered page is the bare path.
func (f InLogFilter) query(page int) string {
q := url.Values{}
for name, v := range map[string]string{"domain": f.Domain, "decision": f.Decision, "from": f.From, "to": f.To} {
if v != "" {
q.Set(name, v)
}
}
if page > 1 {
q.Set("p", strconv.Itoa(page))
}
if len(q) == 0 {
return ""
}
return "?" + q.Encode()
}
// InLogHref is the address of a page of the Inbound log.
func InLogHref(f InLogFilter, page int) string { return inboundLog + f.query(page) }
// InLog is the data of the Inbound log: the decisions about mail for the inbound
// domains the viewer reaches, filtered by domain, decision and day, a page at a
// time.
type InLog struct {
Meta
Head Head
Messages Box
Rows []InLogRow
Empty EmptyState
Foot Foot
filter InLogFilter
}
// InLogRow is one recipient of one message: when it was decided, who the mail
// was from (and the address of the server that offered it) and for, its subject,
// the decision as a tag that opens the sheet saying who decided and why, and —
// for the global role, who alone edits the filter lists — the way to correct
// the decision.
type InLogRow struct {
Time string
From string
Client string
To string
Subject string
Why Sheet
Action *Anchor
}
// InLogRowInput is what the journal holds about one row. Subject, Detail and
// From may be empty: refused mail is read from mail.log, which has no subject,
// and a filter that said nothing has no detail.
type InLogRowInput struct {
ID int64
At time.Time
From string
Client string
To string
Subject string
Decision string
Engine string
Detail string
}
// NewInLog builds the log for a viewer. domains are the choices of the domain
// filter (only the inbound domains the viewer reaches) and filter is what is in
// force. retentionDays is how long rows are kept; the global role, who can
// change it and who edits the lists, is read from m.
func NewInLog(m Meta, retentionDays int, domains []string, filter InLogFilter) *InLog {
m.Title, m.Section, m.Page = "Inbound log", "inbound", "log"
lead := Rich("What port 25 decided about each message, one row per recipient. Accepted mail is recorded as it passes, refused mail is read from ",
Code("mail.log"), "; the filter's verdict appears when the spam filter is on. Rows older than "+plural(int64(retentionDays), "day", "days")+" are deleted")
if m.IsGlobal {
lead = append(lead, Inline{Text: " — change that in "}, Link("/server/settings", "Settings"))
}
lead = append(lead, Inline{Text: "."})
domainOptions := []Option{{Value: "", Label: "All domains"}}
for _, d := range domains {
domainOptions = append(domainOptions, Option{Value: d, Label: d})
}
decisionOptions := []Option{{Value: "", Label: "Any decision"}}
for _, d := range []string{"accept", "reject", "tempfail", "quarantine"} {
decisionOptions = append(decisionOptions, Option{Value: d, Label: d})
}
p := &InLog{
Meta: m,
Head: Head{Kicker: "Inbound", Title: "Inbound log", Lead: lead},
Messages: Box{No: "01", Title: "Messages", Help: topicLink(HelpInboundLog, "What the decisions mean"), Filter: &Filter{
Action: inboundLog,
Selects: []Select{
{Name: "domain", Label: "Domain", Options: domainOptions, Selected: filter.Domain},
{Name: "decision", Label: "Decision", Options: decisionOptions, Selected: filter.Decision},
},
Dates: []DateInput{
{Name: "from", Label: "From date", Value: filter.From},
{Name: "to", Label: "To date", Value: filter.To},
},
}},
Empty: EmptyState{Icon: "ti-list-details", Text: Plain("No messages logged yet.")},
filter: filter,
}
if filter.active() {
p.Empty.Text = Plain("No messages match these filters.")
}
return p
}
// NewRow words one row of the journal for the viewer of the page.
func (p *InLog) NewRow(in InLogRowInput) InLogRow {
r := InLogRow{
Time: FormatLogTime(in.At), From: orDash(in.From), Client: in.Client, To: in.To, Subject: in.Subject,
Why: Sheet{
ID: "why-" + strconv.FormatInt(in.ID, 10), Tag: Tag{Status: in.Decision}, Title: "Why",
Rows: whyRows(in.Engine, in.Detail, in.Subject),
},
}
if p.IsGlobal {
r.Action = senderAction(in.Decision, in.Engine, in.From)
}
return r
}
// whyRows is the content of a row's sheet: who decided, what they said, and
// the subject whole, which the table clips. What was said has one of two
// shapes, told apart by how it starts. The inbound journal milter writes the
// filter's verdict on mail that passed - an action and a score, then the
// symbols after a semicolon. The log tailer writes the reply a refusal was
// given, which starts with its code.
func whyRows(engine, detail, subject string) []SheetRow {
rows := []SheetRow{{Label: "Decided by", Lines: []string{engine}}}
switch {
case detail == "":
case detail[0] >= '0' && detail[0] <= '9':
rows = append(rows, SheetRow{Label: "Reply", Lines: []string{detail}, Mono: true})
default:
verdict, symbols, _ := strings.Cut(detail, "; ")
rows = append(rows, SheetRow{Label: "Verdict", Lines: []string{verdict}})
if symbols != "" {
rows = append(rows, SheetRow{Label: "Symbols", Lines: strings.Fields(symbols), Mono: true})
}
}
if subject != "" {
rows = append(rows, SheetRow{Label: "Subject", Lines: []string{subject}})
}
return rows
}
// senderAction is the way to correct a decision from its row, if there is one:
// a message that got through is the one to deny its sender for, and a message
// the filter turned away, put off or held is the one to allow its sender for.
// A refusal by Postfix's own policy (an unknown recipient, say) offers nothing:
// the message is refused already and the lists would not change what Postfix
// thinks of the recipient. A row with no sender has nobody to list.
func senderAction(decision, engine, from string) *Anchor {
if from == "" {
return nil
}
list := ""
switch {
case decision == "accept":
list = "deny"
case engine == "rspamd":
list = "allow"
default:
return nil
}
label := "Deny sender"
if list == "allow" {
label = "Allow sender"
}
return &Anchor{Href: FilterListsHref(list, "address", from), Label: label}
}
// WithRows sets the table for page of last, and the foot under it: where the
// viewer is, and the way to the newer and older rows. For the global role the
// foot also says where the sender links lead.
func (p *InLog) WithRows(rows []InLogRow, page, last int) *InLog {
p.Rows = rows
if len(rows) == 0 {
return p
}
text := Rich("Page " + strconv.Itoa(page) + " of " + strconv.Itoa(last))
if page > 1 {
text = append(text, Inline{Text: " · "}, Link(InLogHref(p.filter, page-1), "← Newer"))
}
p.Foot.Text = text
if page < last {
p.Foot.Link = &Anchor{Href: InLogHref(p.filter, page+1), Label: "Older →"}
}
if p.IsGlobal {
p.Foot.End = Rich("A decision opens its reason · Allow / Deny sender opens ", Link(inboundFilterLists, "Filter lists"), " with the entry filled in")
} else {
p.Foot.End = Plain("A decision opens its reason")
}
return p
}
+301
View File
@@ -0,0 +1,301 @@
package view
import (
"strings"
"testing"
"time"
)
// ---- The log
// The log is the whole of what the journal holds about the viewer's inbound mail:
// the filters offer what the viewer reaches, the head says how long rows live and
// where to change it, and the global role is given the way to correct a decision.
func TestInLogShowsTheRowsAndTheirFilters(t *testing.T) {
out := renderSignedIn(t, "in-log", inLogFixture())
pageHas(t, "Inbound log", out,
`<h1 class="title is-3">Inbound log</h1>`,
`refused mail is read from <code>mail.log</code>; the filter&#39;s verdict appears when the spam filter is on. Rows older than 90 days are deleted — change that in <a href="/server/settings">Settings</a>.`,
`<form class="sp-end" method="get" action="/inbound/log">`,
`<select name="domain" aria-label="Domain"><option value="" selected>All domains</option><option value="lists.example.org">lists.example.org</option><option value="acme.io">acme.io</option></select>`,
`<select name="decision" aria-label="Decision"><option value="" selected>Any decision</option><option value="accept">accept</option><option value="reject">reject</option><option value="tempfail">tempfail</option><option value="quarantine">quarantine</option></select>`,
`<input class="input" type="date" name="from" value="2026-09-14" aria-label="From date">`,
`<input class="input" type="date" name="to" value="2026-09-21" aria-label="To date">`,
`href="/help#inbound-log" title="What the decisions mean"`,
`<th>Time, UTC</th><th>From</th><th>To</th><th>Subject</th><th>Decision</th><th></th></tr>`,
`09-21 14:00:18`,
`t.berg@<wbr>uni.example<br><span class="sp-mono sp-muted sp-small">192.0.2.77</span>`,
`dev@<wbr>lists.example.org`, `title="Re: [dev] patch series v3"`,
`<td class="sp-clip sp-muted">—</td>`,
// the decision is a tag that opens the sheet with who decided and why
`<button type="button" class="tag is-success is-light" popovertarget="why-1">accept</button><div class="sp-sheet" id="why-1" popover>`,
`<div class="sp-sheet-head"><span class="tag is-success is-light">accept</span><h2>Why</h2><button type="button" popovertarget="why-1" popovertargetaction="hide" aria-label="Close">`,
`<button type="button" class="tag is-danger is-light" popovertarget="why-3">reject</button>`,
`<button type="button" class="tag is-warning is-light" popovertarget="why-5">tempfail</button>`,
`<tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Verdict</th><td>no action, score 0.40 / 15.00</td></tr><tr><th>Subject</th><td>Re: [dev] patch series v3</td></tr>`,
`<tr><th>Verdict</th><td>add header, score 9.80 / 15.00</td></tr><tr><th>Symbols</th><td class="sp-mono sp-small">PHISHING(5.00)<br>FORGED_SENDER(3.00)<br>R_SPF_FAIL(1.00)</td></tr>`,
`<tr><th>Decided by</th><td>postfix</td></tr><tr><th>Reply</th><td class="sp-mono sp-small">550 5.1.1 &lt;sales@lists.example.org&gt;: Recipient address rejected: User unknown in relay recipient table</td></tr></tbody>`,
`Page 1 of 38`, `<a href="/inbound/log?from=2026-09-14&amp;p=2&amp;to=2026-09-21">Older →</a>`,
`A decision opens its reason · Allow / Deny sender opens <a href="/inbound/filter-lists">Filter lists</a> with the entry filled in`,
)
pageLacks(t, "Inbound log", out, `← Newer`, `data-poll`, `hx-get`, `<th>Why</th>`)
// A message Postfix itself refused offers no way into the lists: one Deny
// for each of the three accepted messages, one Allow for each the filter
// refused or put off.
if deny, allow := strings.Count(out, `>Deny sender</a>`), strings.Count(out, `>Allow sender</a>`); deny != 3 || allow != 2 {
t.Errorf("the fixture's six rows offer Deny %d times and Allow %d times, want 3 and 2", deny, allow)
}
// Without the global role there is nowhere to change the retention, and
// nowhere to send a sender.
domain := NewInLog(Meta{User: "ops", HasInbound: true}, 7, []string{"acme.io"}, InLogFilter{})
out = renderSignedIn(t, "in-log", domain.WithRows([]InLogRow{domain.NewRow(InLogRowInput{
At: time.Date(2026, 9, 21, 14, 0, 0, 0, time.UTC), From: "a@example.net", To: "b@acme.io",
Decision: "accept", Engine: "rspamd"})}, 1, 1))
pageHas(t, "Inbound log", out, `Rows older than 7 days are deleted.`, `rspamd</td>`)
pageLacks(t, "Inbound log", out, `/server/settings`, `/inbound/filter-lists`, `Deny sender`, `Allow sender`, `Allow / Deny`)
}
// The lead's day count reads right for one day too, though the setting's range
// starts at seven: the word follows the number, not the other way round.
func TestInLogLeadCountsDays(t *testing.T) {
out := renderSignedIn(t, "in-log", NewInLog(Meta{User: "ops", HasInbound: true}, 1, nil, InLogFilter{}))
pageHas(t, "Inbound log", out, `Rows older than 1 day are deleted.`)
}
// The filter in force is the one selected, and the paging and the empty state
// carry and name it.
func TestInLogCarriesItsFilters(t *testing.T) {
f := InLogFilter{Domain: "acme.io", Decision: "reject", From: "2026-09-01", To: "2026-09-30"}
p := NewInLog(admin(), 90, []string{"lists.example.org", "acme.io"}, f)
p.WithRows([]InLogRow{p.NewRow(InLogRowInput{At: time.Date(2026, 9, 2, 1, 2, 3, 0, time.UTC),
From: "a@example.net", To: "b@acme.io", Decision: "reject", Engine: "postfix"})}, 3, 5)
out := renderSignedIn(t, "in-log", p)
pageHas(t, "Inbound log", out,
`<option value="acme.io" selected>acme.io</option>`, `<option value="reject" selected>reject</option>`,
`name="from" value="2026-09-01"`, `name="to" value="2026-09-30"`,
`Page 3 of 5 · <a href="/inbound/log?decision=reject&amp;domain=acme.io&amp;from=2026-09-01&amp;p=2&amp;to=2026-09-30">← Newer</a>`,
`<a href="/inbound/log?decision=reject&amp;domain=acme.io&amp;from=2026-09-01&amp;p=4&amp;to=2026-09-30">Older →</a>`,
)
if got := InLogHref(InLogFilter{}, 1); got != "/inbound/log" {
t.Errorf("the first unfiltered page is %q", got)
}
if got := InLogHref(InLogFilter{Domain: "a b&c"}, 2); got != "/inbound/log?domain=a+b%26c&p=2" {
t.Errorf("a filter is not encoded: %q", got)
}
out = renderSignedIn(t, "in-log", NewInLog(admin(), 90, nil, f).WithRows(nil, 1, 1))
pageHas(t, "Inbound log", out, `No messages match these filters.`)
out = renderSignedIn(t, "in-log", NewInLog(admin(), 90, nil, InLogFilter{}).WithRows(nil, 1, 1))
pageHas(t, "Inbound log", out, `No messages logged yet.`)
pageLacks(t, "Inbound log", out, `<table`, `Page 1`, `sp-box-foot`)
}
// The last page has nothing older.
func TestInLogLastPage(t *testing.T) {
p := NewInLog(admin(), 90, nil, InLogFilter{})
row := p.NewRow(InLogRowInput{At: time.Now(), From: "a@example.net", To: "b@acme.io", Decision: "accept", Engine: "selfpost"})
out := renderSignedIn(t, "in-log", p.WithRows([]InLogRow{row}, 2, 2))
pageHas(t, "Inbound log", out, `Page 2 of 2`, `← Newer</a>`)
pageLacks(t, "Inbound log", out, `Older →`)
}
// A row says who decided, and what they said when they said something; a message
// refused before its sender was known has no sender to show or to list.
func TestInLogRowWording(t *testing.T) {
p := NewInLog(admin(), 90, nil, InLogFilter{})
at := time.Date(2026, 9, 21, 14, 0, 18, 0, time.UTC)
r := p.NewRow(InLogRowInput{At: at, From: "a@example.net", To: "b@acme.io", Decision: "accept", Engine: "selfpost"})
if len(r.Why.Rows) != 1 || r.Why.Rows[0].Lines[0] != "selfpost" || r.Time != "09-21 14:00:18" || r.Client != "" {
t.Errorf("row = %+v", r)
}
r = p.NewRow(InLogRowInput{ID: 7, At: at, To: "b@acme.io", Decision: "reject", Engine: "postfix", Detail: "550 5.1.1 no"})
if r.Why.ID != "why-7" || r.Why.Tag.Status != "reject" || len(r.Why.Rows) != 2 || r.Why.Rows[1].Label != "Reply" || r.From != "—" || r.Action != nil {
t.Errorf("row = %+v", r)
}
out := renderSignedIn(t, "in-log", p.WithRows([]InLogRow{r}, 1, 1))
pageHas(t, "Inbound log", out, `<td>—</td>`)
pageLacks(t, "Inbound log", out, `<br><span class="sp-mono sp-muted sp-small"></span>`)
}
// Which decisions lead to which list, for the global role: the sender of mail
// that got through can be denied, the sender of mail the filter turned away or
// put off can be allowed, and a refusal that is Postfix's own offers neither.
func TestInLogSenderAction(t *testing.T) {
for _, c := range []struct {
decision, engine, from, label, href string
}{
{"accept", "rspamd", "a@example.net", "Deny sender", "/inbound/filter-lists?list=deny&kind=address&value=a%40example.net"},
{"accept", "selfpost", "a@example.net", "Deny sender", "/inbound/filter-lists?list=deny&kind=address&value=a%40example.net"},
{"reject", "rspamd", "a+b@example.net", "Allow sender", "/inbound/filter-lists?list=allow&kind=address&value=a%2Bb%40example.net"},
{"tempfail", "rspamd", "a@example.net", "Allow sender", "/inbound/filter-lists?list=allow&kind=address&value=a%40example.net"},
{"quarantine", "rspamd", "a@example.net", "Allow sender", "/inbound/filter-lists?list=allow&kind=address&value=a%40example.net"},
{"reject", "postfix", "a@example.net", "", ""},
{"tempfail", "postfix", "a@example.net", "", ""},
{"reject", "selfpost", "a@example.net", "", ""},
{"accept", "rspamd", "", "", ""},
{"reject", "rspamd", "", "", ""},
} {
got := senderAction(c.decision, c.engine, c.from)
switch {
case c.label == "" && got != nil:
t.Errorf("%s by %s from %q is offered %+v", c.decision, c.engine, c.from, *got)
case c.label != "" && (got == nil || got.Label != c.label || got.Href != c.href):
t.Errorf("%s by %s from %q is offered %+v, want %s -> %s", c.decision, c.engine, c.from, got, c.label, c.href)
}
}
// The address in the link is data: it cannot leave the attribute.
p := NewInLog(admin(), 90, nil, InLogFilter{})
row := p.NewRow(InLogRowInput{At: time.Now(), From: `"><script>1</script>@x`, To: "b@acme.io", Decision: "accept", Engine: "rspamd"})
out := renderSignedIn(t, "in-log", p.WithRows([]InLogRow{row}, 1, 1))
pageLacks(t, "Inbound log", out, `<script>1`)
pageHas(t, "Inbound log", out, `value=%22%3E%3Cscript%3E1%3C%2Fscript%3E%40x`)
}
// What a sender wrote is text on the page: an address, a subject or a reply from
// a server is never markup.
func TestInLogEscapesWhatSendersWrote(t *testing.T) {
p := NewInLog(admin(), 90, nil, InLogFilter{})
row := p.NewRow(InLogRowInput{At: time.Now(), From: `<img src=x onerror=alert(1)>@e.example`, Client: `<b>1</b>`,
To: `<script>alert(2)</script>@acme.io`, Subject: `<script>alert(3)</script> "quoted"`,
Decision: "reject", Engine: "rspamd", Detail: `<i>5.7.1</i>`})
out := renderSignedIn(t, "in-log", p.WithRows([]InLogRow{row}, 1, 1))
pageLacks(t, "Inbound log", out, `<script>alert`, `<img src=x`, `<b>1</b>`, `<i>5.7.1</i>`)
pageHas(t, "Inbound log", out, `&lt;script&gt;alert(3)&lt;/script&gt;`, `&lt;i&gt;5.7.1&lt;/i&gt;`)
}
// ---- Filter lists
// The page is the add form, the two lists with their counts, and where the lists
// go - drawn as the mockup draws it.
func TestFilterListsShowsTheFormTheListsAndWhereTheyGo(t *testing.T) {
out := renderSignedIn(t, "in-filter-lists", inFilterListsFixture())
pageHas(t, "Filter lists", out,
`<p class="sp-kicker">Inbound · global role</p>`, `<h1 class="title is-3">Filter lists</h1>`,
`<form class="sp-form" method="post" action="/inbound/filter-lists">`,
`<select id="list" name="list_type">`, `<option value="deny" selected>Deny — always turn away</option>`,
`<option value="allow">Allow — always let through</option>`,
`<select id="etype" name="entry_type">`, `<option value="address" selected>a sender address</option>`,
`<option value="domain">a sender domain</option>`, `<option value="ip">a client address or network</option>`,
`<input class="input sp-mono" id="value" name="value" value="" placeholder="promo@bulk.example"`,
`<input class="input" id="note" name="note" value=""`, `Add entry`,
`href="/help#filter-lists" title="How the lists are applied"`,
`<h2>Deny</h2><div class="sp-end">3 entries · wins over Allow</div>`,
`<h2>Allow</h2><div class="sp-end">2 entries</div>`,
`<thead><tr><th>Value</th><th>Kind</th><th>Matched</th><th></th></tr></thead>`,
// the kind is a tag that opens the entry's sheet; the count is beside it
`<td class="sp-mono sp-small">bulk.example</td><td><button type="button" class="tag is-light" popovertarget="entry-1">domain</button><div class="sp-sheet" id="entry-1" popover>`,
`<div class="sp-sheet-head"><span class="tag is-light">domain</span><h2>bulk.example</h2>`,
`<tr><th>Note</th><td>newsletter nobody subscribed to</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-09-02</td></tr><tr><th>Matched</th><td>41 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-09-21 13:57</td></tr></tbody>`,
`</div></td><td class="sp-nowrap">41</td><td class="sp-actions">`,
`<td class="sp-nowrap">1203</td>`,
// an entry that never decided anything says so, in the row and on its sheet
`<td class="sp-mono sp-small">win@lottery.example</td><td><button type="button" class="tag is-light" popovertarget="entry-3">address</button>`,
`<tr><th>Note</th><td>—</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-07-30</td></tr><tr><th>Matched</th><td>never in the last 90 days</td></tr></tbody>`,
`</div></td><td class="sp-muted sp-nowrap">never</td><td class="sp-actions">`,
`<form method="post" action="/inbound/filter-lists/3/delete"><button type="submit" class="has-text-danger">Remove</button></form>`,
`<h2>Where the lists go</h2><div class="sp-end"><span class="tag is-success is-light">applied</span></div>`,
`rspamd, from map files it rereads within seconds of a change`,
`<span class="sp-mono">/data/antispam/</span> — six, one for each list and kind of entry`,
`2026-09-11 08:40 UTC`, `Deny wins, whichever entries they are.`,
)
if got := strings.Count(out, `<table class="table is-fullwidth is-hoverable">`); got != 2 {
t.Errorf("%d list tables, want the two lists", got)
}
pageLacks(t, "Filter lists", out, `<th>Note</th><th>`, `<th>Added</th><th>`)
}
// With no filter attached the lists are still the operator's to edit, and the
// box says nothing reads them; a list with no entries says so in its box.
func TestFilterListsOffAndEmpty(t *testing.T) {
p := NewFilterLists(admin(), FilterListsInput{}).WithEntries([]FilterEntryInput{
{ID: 9, List: "allow", Kind: "ip", Value: "192.0.2.1", Added: time.Date(2026, 9, 1, 0, 0, 0, 0, time.UTC)},
})
out := renderSignedIn(t, "in-filter-lists", p)
pageHas(t, "Filter lists", out,
`<span class="tag is-light">not applied</span>`,
`Nothing — the spam filter is off. The lists are kept and can be edited; nothing reads them until <code>INBOUND_ANTISPAM_MILTER</code> is set.`,
`<dt>Last written</dt><dd>never</dd>`,
`<h2>Deny</h2><div class="sp-end">0 entries · wins over Allow</div>`,
`<h2>Allow</h2><div class="sp-end">1 entry</div>`,
`No sender is turned away by hand.`,
)
pageLacks(t, "Filter lists", out, `rspamd, from map files`, `is-success is-light">applied`, `No sender is let through by hand.`)
if got := strings.Count(out, `<table class="table is-fullwidth is-hoverable">`); got != 1 {
t.Errorf("%d list tables, want the one list that has entries", got)
}
}
// The form shows what it is told to: prefilled from the log, or what was typed
// into a form that was refused, with the refusal between the head and the form.
func TestFilterListsFormAndRefusal(t *testing.T) {
p := inFilterListsFixture().
WithForm("allow", "domain", `x"><script>1</script>`, `a "note"`).
WithResult("", "That entry is already on the list.")
out := renderSignedIn(t, "in-filter-lists", p)
pageHas(t, "Filter lists", out,
`<option value="allow" selected>Allow — always let through</option>`, `<option value="domain" selected>a sender domain</option>`,
`value="x&#34;&gt;&lt;script&gt;1&lt;/script&gt;"`, `value="a &#34;note&#34;"`,
`<div class="notification is-danger is-light">`, `That entry is already on the list.`)
pageLacks(t, "Filter lists", out, `<script>1`, `<option value="deny" selected>`)
if strings.Index(out, `class="notification`) < strings.Index(out, `class="sp-head"`) ||
strings.Index(out, `class="notification`) > strings.Index(out, `<form class="sp-form"`) {
t.Error("the refusal is not between the head and the form")
}
out = renderSignedIn(t, "in-filter-lists", inFilterListsFixture().WithResult("Entry added.", ""))
pageHas(t, "Filter lists", out, `<div class="notification is-success is-light">`, `Entry added.`)
}
// What a person typed into a list is text on the page.
func TestFilterListsEscapesTheEntries(t *testing.T) {
p := NewFilterLists(admin(), FilterListsInput{FilterOn: true}).WithEntries([]FilterEntryInput{
{ID: 1, List: "deny", Kind: "address", Value: `<img src=x>@a.example`, Note: `<script>alert(1)</script>`, Added: time.Now()},
})
out := renderSignedIn(t, "in-filter-lists", p)
pageLacks(t, "Filter lists", out, `<img src=x>`, `<script>alert`)
}
// The menu leads to both pages, the lists for the global role only.
func TestInboundMenuHasLogAndFilterLists(t *testing.T) {
out := renderSignedIn(t, "in-log", inLogFixture())
pageHas(t, "Inbound log", out,
`<a class="navbar-item is-active" href="/inbound/log"><i class="ti ti-list-details"></i>Log</a>`,
`<a class="navbar-item" href="/inbound/filter-lists"><i class="ti ti-filter"></i>Filter lists</a>`,
`<a href="/inbound/log" aria-current="page"><i class="ti ti-list-details"></i>Log</a><a href="/inbound/filter-lists"><i class="ti ti-filter"></i>Filter lists</a>`)
out = renderSignedIn(t, "in-filter-lists", inFilterListsFixture())
pageHas(t, "Filter lists", out,
`<a class="navbar-item is-active" href="/inbound/filter-lists">`, `<a href="/inbound/filter-lists" aria-current="page">`)
domain := NewInLog(Meta{User: "ops", HasInbound: true}, 90, nil, InLogFilter{}).WithRows(nil, 1, 1)
out = renderSignedIn(t, "in-log", domain)
pageHas(t, "Inbound log", out, `<a class="navbar-item is-active" href="/inbound/log">`, `<a href="/inbound/log" aria-current="page">`)
pageLacks(t, "Inbound log", out, `/inbound/filter-lists`)
}
// What a sheet says, from the two shapes a row's detail has: the filter's
// verdict on mail that passed, and the reply a refusal was given.
func TestWhyRows(t *testing.T) {
flat := func(rows []SheetRow) string {
var out []string
for _, r := range rows {
out = append(out, r.Label+"="+strings.Join(r.Lines, "|"))
}
return strings.Join(out, "; ")
}
for _, c := range []struct{ engine, detail, subject, want string }{
{"selfpost", "", "", "Decided by=selfpost"},
{"selfpost", "", "Hello", "Decided by=selfpost; Subject=Hello"},
{"rspamd", "no action, score 0.40 / 15.00", "Hi", "Decided by=rspamd; Verdict=no action, score 0.40 / 15.00; Subject=Hi"},
{"rspamd", "add header, score 9.80 / 15.00; A(5.00) B(-0.10)", "", "Decided by=rspamd; Verdict=add header, score 9.80 / 15.00; Symbols=A(5.00)|B(-0.10)"},
// a verdict with symbols and no action or score
{"rspamd", "A(5.00)", "", "Decided by=rspamd; Verdict=A(5.00)"},
{"rspamd", "5.7.1 Spam message rejected", "", "Decided by=rspamd; Reply=5.7.1 Spam message rejected"},
{"postfix", "450 4.1.8 <a@nx.example>: Sender address rejected; see x", "", "Decided by=postfix; Reply=450 4.1.8 <a@nx.example>: Sender address rejected; see x"},
} {
if got := flat(whyRows(c.engine, c.detail, c.subject)); got != c.want {
t.Errorf("whyRows(%q, %q, %q)\n got: %s\nwant: %s", c.engine, c.detail, c.subject, got, c.want)
}
}
}
+8 -3
View File
@@ -21,6 +21,7 @@ type Kit struct {
Postmarks []Postmark Postmarks []Postmark
StatusTag Box StatusTag Box
StatusRows []KitStatusRow StatusRows []KitStatusRow
Sheet Sheet
BoxHead Box BoxHead Box
BoxFoot Foot BoxFoot Foot
DangerZone Box DangerZone Box
@@ -91,10 +92,14 @@ func KitPage() *Kit {
}, },
StatusTag: Box{No: "03", Title: "status_tag", End: Plain("One mapping, one function")}, StatusTag: Box{No: "03", Title: "status_tag", End: Plain("One mapping, one function")},
Sheet: Sheet{ID: "kit-sheet", Tag: Tag{Status: "reject"}, Title: "sheet", Rows: []SheetRow{
{Label: "What it holds", Lines: []string{"What does not fit the row, as name and value"}},
{Label: "How it closes", Lines: []string{"Esc", "a click outside", "the cross"}, Mono: true},
}},
StatusRows: []KitStatusRow{ StatusRows: []KitStatusRow{
{"ok, published, running, sent", Tag{Status: "ok"}}, {"ok, published, running, sent, accept", Tag{Status: "ok"}},
{"warn, deferred, quiet", Tag{Status: "warn"}}, {"warn, deferred, quiet, tempfail", Tag{Status: "warn"}},
{"error, fail, mismatch, bounced", Tag{Status: "fail"}}, {"error, fail, mismatch, bounced, reject", Tag{Status: "fail"}},
{"unknown, off, neutral counts", Tag{Status: "off"}}, {"unknown, off, neutral counts", Tag{Status: "off"}},
{"held: quarantine · role: global", Tag{Status: "quarantine"}}, {"held: quarantine · role: global", Tag{Status: "quarantine"}},
}, },
+20
View File
@@ -402,3 +402,23 @@ func TestOverviewFragmentIsTheBoxAndTheHeadOutOfBand(t *testing.T) {
} }
} }
} }
// With the inbound relay on the retention box has a second field, for the Inbound
// log; with it off the field is not there, since there is no journal to keep.
func TestSettingsHasTheInboundRetentionOnlyWithTheRelay(t *testing.T) {
out := renderSignedIn(t, "settings", settingsFixture())
pageHas(t, "Settings", out,
`for="inbound_spam_log_retention_days">Keep inbound log rows, days</label>`,
`name="inbound_spam_log_retention_days" type="number" min="7" max="365" step="1" value="90" required`,
`The inbound log also never holds more than 10 000 rows: refused mail is logged at the rate strangers knock on port 25.`,
`name="send_log_retention_days"`)
if got := strings.Count(out, `<form class="sp-form"`); got != 1 {
t.Errorf("Settings has %d forms, want 1", got)
}
pageLacks(t, "Settings", renderSignedIn(t, "settings", NewSettings(admin(), "30", "600 / h")),
`inbound_spam_log_retention_days`, `Keep inbound log rows`, `10 000 rows`)
// A refused value comes back as typed.
out = renderSignedIn(t, "settings", NewSettings(admin(), "30", "600 / h").WithInbound("3").WithResult("", "Inbound log retention must be between 7 and 365."))
pageHas(t, "Settings", out, `value="3"`, `between 7 and 365`)
}
+24
View File
@@ -327,3 +327,27 @@ code { background: var(--sp-well); color: var(--sp-ink); border-radius: 3px; pad
.sp-record-head small { margin-left: 0; text-align: left; flex-basis: 100%; } .sp-record-head small { margin-left: 0; text-align: left; flex-basis: 100%; }
.sp-postmark { display: none; } .sp-postmark { display: none; }
} }
/* sheet: what does not fit a table row, opened over the page from the row's
tag. It is the browser's own popover (the tag is a button with
popovertarget), so it needs no script: Esc, a click outside and the close
button all shut it, and focus returns to the tag. It rises from the bottom
edge across the whole window - the one component that is not laid out by the
page, hence the widths - with its content, a table of name and value, in a
column of reading width in the middle. */
.sp-sheet {
inset: auto 0 0 0; width: auto; margin: 0; padding: 0; border: 0; border-top: 3px solid var(--sp-brick);
max-height: 75vh; overflow: auto; background: #fff; color: inherit; box-shadow: 0 -12px 40px rgba(40, 20, 15, .22);
}
.sp-sheet-in { max-width: 44rem; margin: 0 auto; padding: 0 1.25rem; }
.sp-sheet::backdrop { background: rgba(40, 20, 15, .38); }
.sp-sheet-head { display: flex; align-items: center; gap: .7rem; padding: 1rem 0 .8rem; border-bottom: 1px solid var(--sp-rule); }
.sp-sheet-head h2 { font-size: 1.15rem; font-weight: 600; margin-right: auto; }
.sp-sheet-head button { border: 0; background: none; cursor: pointer; font-size: 1.3rem; color: var(--sp-muted); }
.sp-sheet-body { padding: .4rem 0 1.4rem; }
/* The sheet's content is a table of name and value: the name is a label, set like the dt of facts. */
.sp-sheet .table th { font-family: var(--sp-mono); font-size: var(--sp-fs-xs); font-weight: 500; letter-spacing: .08em; text-transform: uppercase; color: var(--sp-muted); white-space: nowrap; vertical-align: top; width: 1%; padding-right: 2rem; }
.sp-sheet .table td { overflow-wrap: anywhere; }
/* A tag that opens a sheet is a button: it keeps the tag's look and says it can
be pressed with the dotted underline links to more detail carry. */
button.tag { border: 0; cursor: pointer; text-decoration: underline dotted; text-underline-offset: 3px; }
+1 -1
View File
@@ -13,7 +13,7 @@
<div class="column is-6"> <div class="column is-6">
{{template "box_open" .Full}} {{template "box_open" .Full}}
{{template "body_open"}} {{template "body_open"}}
<p class="mb-3"><code>data/</code> (database, DKIM keys, credentials, the queue), <code>docker-compose.yml</code>, <code>.env</code> and <code>certs/</code> in one archive. Restore into an empty project directory and start the <strong>same SelfPost version</strong>. The reverse-proxy vhost is not included.</p> <p class="mb-3"><code>data/</code> (database, DKIM keys, credentials), <code>docker-compose.yml</code>, <code>.env</code> and <code>certs/</code> in one archive. The Postfix queue is left out: it cannot be copied reliably while the server is running. Restore into an empty project directory and start the <strong>same SelfPost version</strong>. The reverse-proxy vhost is not included.</p>
<form class="sp-form" method="post" action="{{.FullAction}}"> <form class="sp-form" method="post" action="{{.FullAction}}">
<div class="field"><label class="checkbox"><input type="checkbox" name="encrypt" value="1"{{if .Encrypt}} checked{{end}} data-encrypt-toggle> Encrypt with a password (<code>.spbk</code>)</label></div> <div class="field"><label class="checkbox"><input type="checkbox" name="encrypt" value="1"{{if .Encrypt}} checked{{end}} data-encrypt-toggle> Encrypt with a password (<code>.spbk</code>)</label></div>
<div data-encrypt-fields> <div data-encrypt-fields>
@@ -27,6 +27,13 @@
mapping of status to Bulma classes; the label defaults to the status. */}} mapping of status to Bulma classes; the label defaults to the status. */}}
{{define "tag"}}<span class="{{status_tag .Status}}">{{if .Label}}{{.Label}}{{else}}{{.Status}}{{end}}</span>{{end}} {{define "tag"}}<span class="{{status_tag .Status}}">{{if .Label}}{{.Label}}{{else}}{{.Status}}{{end}}</span>{{end}}
{{/* sheet_tag is a status tag that opens a sheet (input: Sheet): the tag as a
button, and the sheet it names right after it. sheet is what opens: the tag
again, a title, the cross that closes it, and a table of name and value.
The browser's popover does the rest - no script. */}}
{{define "sheet_tag"}}<button type="button" class="{{status_tag .Tag.Status}}" popovertarget="{{.ID}}">{{if .Tag.Label}}{{.Tag.Label}}{{else}}{{.Tag.Status}}{{end}}</button>{{template "sheet" .}}{{end}}
{{define "sheet"}}<div class="sp-sheet" id="{{.ID}}" popover><div class="sp-sheet-in"><div class="sp-sheet-head">{{template "tag" .Tag}}<h2>{{.Title}}</h2><button type="button" popovertarget="{{.ID}}" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody>{{range .Rows}}<tr><th>{{.Label}}</th><td{{if .Mono}} class="sp-mono sp-small"{{end}}>{{range $i, $line := .Lines}}{{if $i}}<br>{{end}}{{$line}}{{end}}</td></tr>{{end}}</tbody></table></div></div></div></div>{{end}}
{{/* kicker is the upper-case mono micro-label on its own (input: string). {{/* kicker is the upper-case mono micro-label on its own (input: string).
page_head writes its own kicker as the line above the title. */}} page_head writes its own kicker as the line above the title. */}}
{{define "kicker"}}<span class="sp-kicker">{{.}}</span>{{end}} {{define "kicker"}}<span class="sp-kicker">{{.}}</span>{{end}}
@@ -0,0 +1,64 @@
{{/* Inbound > Filter lists (mockup in-filter-lists.html): the allow and deny
lists that correct the spam filter by hand, for the whole instance, and
where they go. Global role only. The form adds an entry; each row of a list
removes its own, says how often the entry decided a message, and opens a
sheet with the entry's note from the tag of its kind. */}}
{{define "content"}}
{{template "page_head" .Head}}
{{if .Flash}}{{template "flash" .Flash}}{{end}}
{{template "box_open" .Add}}
{{template "body_open"}}
<form class="sp-form" method="post" action="/inbound/filter-lists">
<div class="columns">
<div class="column is-3">
<div class="field"><label class="label" for="list">List</label>
<div class="control"><div class="select is-fullwidth"><select id="list" name="list_type">{{$list := .List}}{{range .ListOptions}}
<option value="{{.Value}}"{{if eq .Value $list}} selected{{end}}>{{.Label}}</option>{{end}}
</select></div></div></div>
</div>
<div class="column is-3">
<div class="field"><label class="label" for="etype">Entry is</label>
<div class="control"><div class="select is-fullwidth"><select id="etype" name="entry_type">{{$kind := .Kind}}{{range .KindOptions}}
<option value="{{.Value}}"{{if eq .Value $kind}} selected{{end}}>{{.Label}}</option>{{end}}
</select></div></div></div>
</div>
<div class="column is-6">
<div class="field"><label class="label" for="value">Value</label>
<div class="control"><input class="input sp-mono" id="value" name="value" value="{{.Value}}" placeholder="promo@bulk.example" autocomplete="off" autocapitalize="none" spellcheck="false" required></div></div>
</div>
</div>
<div class="field"><label class="label" for="note">Note</label>
<div class="control"><input class="input" id="note" name="note" value="{{.Note}}" placeholder="Why — for whoever reads this list next year"></div></div>
<button type="submit" class="button is-primary"><i class="ti ti-plus"></i>Add entry</button>
</form>
{{template "body_close"}}
{{template "box_close"}}
<div class="columns">
<div class="column is-6">
{{template "filter_list" .Deny}}
</div>
<div class="column is-6">
{{template "filter_list" .Allow}}
</div>
</div>
{{template "box_open" .Where}}
{{template "body_open"}}
{{template "facts" .WhereFacts}}
{{template "body_close"}}
{{template "box_close"}}
{{end}}
{{/* filter_list is one of the two boxes of entries (input: FilterList): the
table, or what is absent when the list is empty. */}}
{{define "filter_list"}}{{template "box_open" .Box}}
{{if .Rows}}<div class="table-container"><table class="table is-fullwidth is-hoverable">
<thead><tr><th>Value</th><th>Kind</th><th>Matched</th><th></th></tr></thead>
<tbody>{{range .Rows}}
<tr><td class="sp-mono sp-small">{{.Value}}</td><td>{{template "sheet_tag" .Entry}}</td>{{if .Matched}}<td class="sp-nowrap">{{.Matched}}</td>{{else}}<td class="sp-muted sp-nowrap">never</td>{{end}}<td class="sp-actions">{{template "row_action" .Remove}}</td></tr>{{end}}
</tbody>
</table></div>
{{else}}{{template "empty_state" .Empty}}
{{end}}{{template "box_close"}}{{end}}
+23
View File
@@ -0,0 +1,23 @@
{{/* Inbound log (mockup in-log.html): what port 25 decided about each message
offered for an inbound domain, one row per recipient, newest first,
filtered by domain, decision and day. It does not poll. The decision is a
tag that opens a sheet with who decided and why. The last cell is the way
to correct a decision in Filter lists, and only the global role, who alone
edits the lists, is given it. */}}
{{define "content"}}
{{template "page_head" .Head}}
{{template "box_open" .Messages}}
{{if .Rows}}
<div class="table-container"><table class="table is-fullwidth is-hoverable">
<thead><tr><th>Time, UTC</th><th>From</th><th>To</th><th>Subject</th><th>Decision</th><th></th></tr></thead>
<tbody>{{range .Rows}}
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">{{.Time}}</td><td>{{wbr_at .From}}{{if .Client}}<br><span class="sp-mono sp-muted sp-small">{{.Client}}</span>{{end}}</td><td>{{wbr_at .To}}</td>{{if .Subject}}<td class="sp-clip" title="{{.Subject}}">{{.Subject}}</td>{{else}}<td class="sp-clip sp-muted">—</td>{{end}}<td>{{template "sheet_tag" .Why}}</td><td class="sp-actions">{{if .Action}}<a href="{{.Action.Href}}">{{.Action.Label}}</a>{{end}}</td></tr>{{end}}
</tbody>
</table></div>
{{template "box_foot" .Foot}}
{{else}}
{{template "empty_state" .Empty}}
{{end}}
{{template "box_close"}}
{{end}}
+1
View File
@@ -31,6 +31,7 @@
<thead><tr><th>Status</th><th>Tag</th><th>Bulma classes</th></tr></thead> <thead><tr><th>Status</th><th>Tag</th><th>Bulma classes</th></tr></thead>
<tbody>{{range .StatusRows}} <tbody>{{range .StatusRows}}
<tr><td>{{.Words}}</td><td>{{template "tag" .Tag}}</td><td class="sp-mono sp-small">{{status_tag .Tag.Status}}</td></tr>{{end}} <tr><td>{{.Words}}</td><td>{{template "tag" .Tag}}</td><td class="sp-mono sp-small">{{status_tag .Tag.Status}}</td></tr>{{end}}
<tr><td>a tag that opens a <code>sheet</code></td><td>{{template "sheet_tag" .Sheet}}</td><td class="sp-mono sp-small">button.tag + sp-sheet (popover)</td></tr>
</tbody> </tbody>
</table></div> </table></div>
{{template "box_close"}} {{template "box_close"}}
+4 -2
View File
@@ -104,7 +104,9 @@
<a class="navbar-link" href="/inbound/domains">Inbound</a> <a class="navbar-link" href="/inbound/domains">Inbound</a>
<div class="navbar-dropdown"> <div class="navbar-dropdown">
<a class="navbar-item{{if and (eq .Section "inbound") (eq .Page "domains")}} is-active{{end}}" href="/inbound/domains"><i class="ti ti-world-download"></i>Domains</a> <a class="navbar-item{{if and (eq .Section "inbound") (eq .Page "domains")}} is-active{{end}}" href="/inbound/domains"><i class="ti ti-world-download"></i>Domains</a>
</div> <a class="navbar-item{{if and (eq .Section "inbound") (eq .Page "log")}} is-active{{end}}" href="/inbound/log"><i class="ti ti-list-details"></i>Log</a>
{{if .ShowFilterLists}} <a class="navbar-item{{if and (eq .Section "inbound") (eq .Page "filter-lists")}} is-active{{end}}" href="/inbound/filter-lists"><i class="ti ti-filter"></i>Filter lists</a>
{{end}} </div>
</div> </div>
{{end}}{{if .ShowServer}} <div class="navbar-item has-dropdown is-hoverable{{if eq .Section "server"}} sp-current{{end}}"> {{end}}{{if .ShowServer}} <div class="navbar-item has-dropdown is-hoverable{{if eq .Section "server"}} sp-current{{end}}">
<a class="navbar-link" href="/server/health">Server</a> <a class="navbar-link" href="/server/health">Server</a>
@@ -143,6 +145,6 @@
pages of the current group, the current one marked. A page outside every pages of the current group, the current one marked. A page outside every
group (the user menu, the kit page's neighbours) has none. */}} group (the user menu, the kit page's neighbours) has none. */}}
{{define "subnav"}}{{if and .ShowOutbound (eq .Section "outbound")}}<div class="sp-subnav"><div class="container"><a href="/outbound/domains"{{if eq .Page "domains"}} aria-current="page"{{end}}><i class="ti ti-world-upload"></i>Domains</a><a href="/outbound/log"{{if eq .Page "log"}} aria-current="page"{{end}}><i class="ti ti-list-details"></i>Log</a>{{if .ShowQueue}}<a href="/outbound/queue"{{if eq .Page "queue"}} aria-current="page"{{end}}><i class="ti ti-stack-2"></i>Queue</a>{{end}}{{if .ShowDMARC}}<a href="/outbound/dmarc"{{if eq .Page "dmarc"}} aria-current="page"{{end}}><i class="ti ti-report-analytics"></i>DMARC reports</a>{{end}}</div></div> {{define "subnav"}}{{if and .ShowOutbound (eq .Section "outbound")}}<div class="sp-subnav"><div class="container"><a href="/outbound/domains"{{if eq .Page "domains"}} aria-current="page"{{end}}><i class="ti ti-world-upload"></i>Domains</a><a href="/outbound/log"{{if eq .Page "log"}} aria-current="page"{{end}}><i class="ti ti-list-details"></i>Log</a>{{if .ShowQueue}}<a href="/outbound/queue"{{if eq .Page "queue"}} aria-current="page"{{end}}><i class="ti ti-stack-2"></i>Queue</a>{{end}}{{if .ShowDMARC}}<a href="/outbound/dmarc"{{if eq .Page "dmarc"}} aria-current="page"{{end}}><i class="ti ti-report-analytics"></i>DMARC reports</a>{{end}}</div></div>
{{else if and .ShowInbound (eq .Section "inbound")}}<div class="sp-subnav"><div class="container"><a href="/inbound/domains"{{if eq .Page "domains"}} aria-current="page"{{end}}><i class="ti ti-world-download"></i>Domains</a></div></div> {{else if and .ShowInbound (eq .Section "inbound")}}<div class="sp-subnav"><div class="container"><a href="/inbound/domains"{{if eq .Page "domains"}} aria-current="page"{{end}}><i class="ti ti-world-download"></i>Domains</a><a href="/inbound/log"{{if eq .Page "log"}} aria-current="page"{{end}}><i class="ti ti-list-details"></i>Log</a>{{if .ShowFilterLists}}<a href="/inbound/filter-lists"{{if eq .Page "filter-lists"}} aria-current="page"{{end}}><i class="ti ti-filter"></i>Filter lists</a>{{end}}</div></div>
{{else if and .ShowServer (eq .Section "server")}}<div class="sp-subnav"><div class="container"><a href="/server/health"{{if eq .Page "health"}} aria-current="page"{{end}}><i class="ti ti-heartbeat"></i>Health</a><a href="/server/log"{{if eq .Page "log"}} aria-current="page"{{end}}><i class="ti ti-file-text"></i>System log</a><a href="/server/backup"{{if eq .Page "backup"}} aria-current="page"{{end}}><i class="ti ti-archive"></i>Backup</a><a href="/server/users"{{if eq .Page "users"}} aria-current="page"{{end}}><i class="ti ti-users"></i>Users</a><a href="/server/settings"{{if eq .Page "settings"}} aria-current="page"{{end}}><i class="ti ti-settings"></i>Settings</a></div></div> {{else if and .ShowServer (eq .Section "server")}}<div class="sp-subnav"><div class="container"><a href="/server/health"{{if eq .Page "health"}} aria-current="page"{{end}}><i class="ti ti-heartbeat"></i>Health</a><a href="/server/log"{{if eq .Page "log"}} aria-current="page"{{end}}><i class="ti ti-file-text"></i>System log</a><a href="/server/backup"{{if eq .Page "backup"}} aria-current="page"{{end}}><i class="ti ti-archive"></i>Backup</a><a href="/server/users"{{if eq .Page "users"}} aria-current="page"{{end}}><i class="ti ti-users"></i>Users</a><a href="/server/settings"{{if eq .Page "settings"}} aria-current="page"{{end}}><i class="ti ti-settings"></i>Settings</a></div></div>
{{end}}{{end}} {{end}}{{end}}
@@ -13,6 +13,10 @@
<div class="field"><label class="label" for="send_log_retention_days">Keep outbound log rows, days</label> <div class="field"><label class="label" for="send_log_retention_days">Keep outbound log rows, days</label>
<div class="control"><input class="input" id="send_log_retention_days" name="send_log_retention_days" type="number" min="7" max="365" step="1" value="{{.RetentionDays}}" required></div> <div class="control"><input class="input" id="send_log_retention_days" name="send_log_retention_days" type="number" min="7" max="365" step="1" value="{{.RetentionDays}}" required></div>
<p class="help">The main driver of <code>/data</code> growth. Does not affect <code>mail.log</code> rotation (fourteen daily files).</p></div> <p class="help">The main driver of <code>/data</code> growth. Does not affect <code>mail.log</code> rotation (fourteen daily files).</p></div>
{{if .Inbound}}<div class="field"><label class="label" for="inbound_spam_log_retention_days">Keep inbound log rows, days</label>
<div class="control"><input class="input" id="inbound_spam_log_retention_days" name="inbound_spam_log_retention_days" type="number" min="7" max="365" step="1" value="{{.InboundRetentionDays}}" required></div>
<p class="help">The inbound log also never holds more than 10 000 rows: refused mail is logged at the rate strangers knock on port 25.</p></div>
{{end}}
{{template "body_close"}} {{template "body_close"}}
{{template "box_close"}} {{template "box_close"}}
</div> </div>
+2
View File
@@ -54,6 +54,8 @@ var pageFiles = map[string][]string{
"in-domains": {"templates/in_domains.html"}, "in-domains": {"templates/in_domains.html"},
"in-domain": {"templates/in_domain.html"}, "in-domain": {"templates/in_domain.html"},
"in-domain-delete": {"templates/in_domain_delete.html"}, "in-domain-delete": {"templates/in_domain_delete.html"},
"in-log": {"templates/in_log.html"},
"in-filter-lists": {"templates/in_filter_lists.html"},
"out-log": {"templates/out_log.html", "templates/out_log_rows.html"}, "out-log": {"templates/out_log.html", "templates/out_log_rows.html"},
"out-message": {"templates/out_message.html"}, "out-message": {"templates/out_message.html"},
"dmarc": {"templates/dmarc.html"}, "dmarc": {"templates/dmarc.html"},
+14 -2
View File
@@ -97,6 +97,9 @@ type Config struct {
DMARCEnabled bool DMARCEnabled bool
// SendLogRetentionEnvDefault is SEND_LOG_RETENTION_DAYS at panel start. // SendLogRetentionEnvDefault is SEND_LOG_RETENTION_DAYS at panel start.
SendLogRetentionEnvDefault int SendLogRetentionEnvDefault int
// InboundSpamLogRetentionEnvDefault is INBOUND_SPAM_LOG_RETENTION_DAYS at
// panel start.
InboundSpamLogRetentionEnvDefault int
} }
// Server is the panel HTTP application. // Server is the panel HTTP application.
@@ -111,7 +114,7 @@ type Server struct {
// that owns DKIM keys and the OpenDKIM tables (architecture.md § OpenDKIM); // that owns DKIM keys and the OpenDKIM tables (architecture.md § OpenDKIM);
// apps owns application SASL accounts and the Postfix sender map // apps owns application SASL accounts and the Postfix sender map
// (architecture.md § Mail path). // (architecture.md § Mail path).
func New(st *store.Store, domains *domain.Service, apps *app.Service, inboundSvc *inbound.Service, dmarcSvc *dmarc.Service, cfg Config, setupTokenPath string) (*Server, error) { func New(st *store.Store, domains *domain.Service, apps *app.Service, inboundSvc *inbound.Service, lists *inbound.Lists, dmarcSvc *dmarc.Service, cfg Config, setupTokenPath string) (*Server, error) {
v, err := view.New(cfg.Version) v, err := view.New(cfg.Version)
if err != nil { if err != nil {
return nil, err return nil, err
@@ -124,7 +127,7 @@ func New(st *store.Store, domains *domain.Service, apps *app.Service, inboundSvc
SessionIdleDays: cfg.SessionIdleDays, SessionIdleDays: cfg.SessionIdleDays,
TrustedProxyCIDRs: cfg.TrustedProxyCIDRs, TrustedProxyCIDRs: cfg.TrustedProxyCIDRs,
}, v, setupTokenPath) }, v, setupTokenPath)
h := handlers.New(st, domains, apps, inboundSvc, dmarcSvc, handlers.Config{ h := handlers.New(st, domains, apps, inboundSvc, lists, dmarcSvc, handlers.Config{
Hostname: cfg.Hostname, Hostname: cfg.Hostname,
SubmissionEnabled: cfg.SubmissionEnabled, SubmissionEnabled: cfg.SubmissionEnabled,
MailLogPath: cfg.MailLogPath, MailLogPath: cfg.MailLogPath,
@@ -143,6 +146,8 @@ func New(st *store.Store, domains *domain.Service, apps *app.Service, inboundSvc
InboundAntispamAction: cfg.InboundAntispamAction, InboundAntispamAction: cfg.InboundAntispamAction,
DMARCEnabled: cfg.DMARCEnabled, DMARCEnabled: cfg.DMARCEnabled,
SendLogRetentionEnvDefault: cfg.SendLogRetentionEnvDefault, SendLogRetentionEnvDefault: cfg.SendLogRetentionEnvDefault,
InboundSpamLogRetentionEnvDefault: cfg.InboundSpamLogRetentionEnvDefault,
}, v, dnscheck.New(cfg.DNSResolvers), &health.MachineSampler{}, a) }, v, dnscheck.New(cfg.DNSResolvers), &health.MachineSampler{}, a)
return &Server{cfg: cfg, auth: a, handlers: h}, nil return &Server{cfg: cfg, auth: a, handlers: h}, nil
} }
@@ -240,6 +245,13 @@ func (s *Server) muxes() (public, authed *http.ServeMux) {
authed.HandleFunc("POST /inbound/domains/{id}/recipients", h.HandleInboundRecipients) authed.HandleFunc("POST /inbound/domains/{id}/recipients", h.HandleInboundRecipients)
authed.HandleFunc("GET /inbound/domains/{id}/delete", h.HandleInboundDeleteConfirm) authed.HandleFunc("GET /inbound/domains/{id}/delete", h.HandleInboundDeleteConfirm)
authed.HandleFunc("POST /inbound/domains/{id}/delete", h.HandleInboundDelete) authed.HandleFunc("POST /inbound/domains/{id}/delete", h.HandleInboundDelete)
// The log is for everyone who reaches an inbound domain (scoped to
// theirs); the lists are the instance's, so the handlers keep the
// global role's check (requireInboundGlobal) and answer 404 to the rest.
authed.HandleFunc("GET /inbound/log", h.HandleInboundLog)
authed.HandleFunc("GET /inbound/filter-lists", h.HandleFilterLists)
authed.HandleFunc("POST /inbound/filter-lists", h.HandleFilterListAdd)
authed.HandleFunc("POST /inbound/filter-lists/{id}/delete", h.HandleFilterListDelete)
} }
// Server: everything under /server/ is for the global role, so the subtree // Server: everything under /server/ is for the global role, so the subtree
+26 -2
View File
@@ -70,6 +70,12 @@ const (
recipient = "rcpt@sink.e2e.test" recipient = "rcpt@sink.e2e.test"
) )
// The one administrator of the run.
const (
adminUser = "e2eadmin"
adminPassword = "correct-horse-battery-staple"
)
// scenario carries state forward between the ordered subtests of TestE2E: // scenario carries state forward between the ordered subtests of TestE2E:
// each later step needs something an earlier one produced (the domain id, the // each later step needs something an earlier one produced (the domain id, the
// DKIM record, the application credentials, an observed client IP). // DKIM record, the application credentials, an observed client IP).
@@ -131,10 +137,10 @@ func TestE2E(t *testing.T) {
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
if err := p.setup(token, "e2eadmin", "correct-horse-battery-staple"); err != nil { if err := p.setup(token, adminUser, adminPassword); err != nil {
t.Fatalf("setup: %v", err) t.Fatalf("setup: %v", err)
} }
if err := p.login("e2eadmin", "correct-horse-battery-staple"); err != nil { if err := p.login(adminUser, adminPassword); err != nil {
t.Fatalf("login: %v", err) t.Fatalf("login: %v", err)
} }
sc.panel = p sc.panel = p
@@ -268,6 +274,24 @@ func TestE2E(t *testing.T) {
run("negative_level1_ratelimit", func(t *testing.T) { run("negative_level1_ratelimit", func(t *testing.T) {
testLevel1RateLimit(t, sc) testLevel1RateLimit(t, sc)
}) })
// From here on the installation is no longer the one the run started: it
// is restored from its own backup, then switched to accept mail on port 25.
run("full_backup_restore", func(t *testing.T) {
testFullBackupRestore(t, sc)
})
run("enable_port25_features", func(t *testing.T) {
testEnablePort25Features(t, sc)
})
run("dmarc_report_reaches_the_panel", func(t *testing.T) {
testDMARCReportReachesThePanel(t, sc)
})
run("dmarc_junk_is_dropped", func(t *testing.T) {
testDMARCJunkIsDropped(t, sc)
})
run("inbound_relay_delivers_and_refuses", func(t *testing.T) {
testInboundRelay(t, sc)
})
} }
// trimZoneSuffix strips the "e2e.test" suffix a fully-qualified DKIM record // trimZoneSuffix strips the "e2e.test" suffix a fully-qualified DKIM record
+81
View File
@@ -0,0 +1,81 @@
package e2e
import (
"fmt"
"net"
"net/smtp"
"time"
)
// port25Addr is where compose.override.yml publishes port 25: no TLS required,
// no AUTH offered - a stranger's mail server talking to this one.
const port25Addr = "127.0.0.1:20025"
// deliverPort25 is one transaction on port 25 the way another mail server makes
// it: HELO, MAIL, RCPT, DATA with message as it stands. The result says at
// which stage it was refused, like attemptSend's.
func deliverPort25(from, to, message string) sendResult {
var res sendResult
conn, err := net.DialTimeout("tcp", port25Addr, 10*time.Second)
if err != nil {
res.dialErr = fmt.Errorf("dial %s: %w", port25Addr, err)
return res
}
c, err := smtp.NewClient(conn, selfpostHostname)
if err != nil {
res.dialErr = fmt.Errorf("smtp handshake: %w", err)
_ = conn.Close()
return res
}
defer c.Close()
if err := c.Hello("mx.stranger.e2e.test"); err != nil {
res.dialErr = err
return res
}
if err := c.Mail(from); err != nil {
res.mailErr = err
_ = c.Quit()
return res
}
if err := c.Rcpt(to); err != nil {
res.rcptErr = err
_ = c.Quit()
return res
}
w, err := c.Data()
if err != nil {
res.dataErr = err
_ = c.Quit()
return res
}
if _, err := w.Write([]byte(message)); err != nil {
res.dataErr = err
} else if err := w.Close(); err != nil {
res.dataErr = err
}
_ = c.Quit()
return res
}
// waitForPort25Ready polls port 25 until Postfix greets on it. A bare connect
// is not enough: Docker's proxy accepts on the published port whether or not
// anything listens behind it.
func waitForPort25Ready(timeout time.Duration) error {
return waitFor("port 25 to greet", timeout, 500*time.Millisecond, func() (bool, error) {
conn, err := net.DialTimeout("tcp", port25Addr, 2*time.Second)
if err != nil {
return false, err
}
defer conn.Close()
_ = conn.SetReadDeadline(time.Now().Add(3 * time.Second))
buf := make([]byte, 4)
n, err := conn.Read(buf)
if err != nil {
return false, err
}
if string(buf[:n]) != "220 " {
return false, fmt.Errorf("greeting starts with %q", buf[:n])
}
return true, nil
})
}
+10 -6
View File
@@ -34,7 +34,7 @@ type port25Case struct {
// flags=, user= and argv= are arguments of pipe(8); set as -o parameters they // flags=, user= and argv= are arguments of pipe(8); set as -o parameters they
// are dropped, and a bare "pipe" dies on every report with "fatal: missing // are dropped, and a bare "pipe" dies on every report with "fatal: missing
// user= command-line attribute" - the state of 1.7.0 through 1.9.5. // user= command-line attribute" - the state of 1.7.0 through 1.9.5.
const ingestService = "dmarc-ingest unix - n n - - pipe flags=Rq user=panel argv=/usr/local/bin/dmarc-ingest ${recipient}" const ingestService = "dmarc-ingest unix - n n - 1 pipe flags=Rq user=panel argv=/usr/local/bin/dmarc-ingest ${recipient}"
var port25Cases = []port25Case{ var port25Cases = []port25Case{
{ {
@@ -47,9 +47,13 @@ var port25Cases = []port25Case{
{param: "smtpd_sasl_auth_enable", contains: "no", wantExact: true}, {param: "smtpd_sasl_auth_enable", contains: "no", wantExact: true},
{param: "smtpd_relay_restrictions", contains: "reject_unauth_destination", wantExact: true}, {param: "smtpd_relay_restrictions", contains: "reject_unauth_destination", wantExact: true},
{param: "smtpd_recipient_restrictions", contains: "reject_unauth_destination,reject_unlisted_recipient", wantExact: true}, {param: "smtpd_recipient_restrictions", contains: "reject_unauth_destination,reject_unlisted_recipient", wantExact: true},
// No antispam milter configured: the submission chain (strict // The log tailer tells this listener's refusals from the
// OpenDKIM + journal-milter) must not reach inbound mail. // submission ports' by the name they are logged under.
{param: "smtpd_milters", contains: "", wantExact: true}, {param: "syslog_name", contains: "postfix/inbound", wantExact: true},
// No antispam milter configured: the inbound journal milter
// alone. The submission chain (strict OpenDKIM + journal-milter)
// must not reach inbound mail.
{param: "smtpd_milters", contains: "{unix:/run/selfpost/inbound-journal.sock,default_action=accept}", wantExact: true},
}, },
}, },
{ {
@@ -72,7 +76,7 @@ var port25Cases = []port25Case{
}, },
check: []optionCheck{ check: []optionCheck{
{param: "smtpd_recipient_restrictions", contains: "check_recipient_access,texthash:/data/postfix/dmarc_recipients,reject_unauth_destination,reject_unlisted_recipient", wantExact: true}, {param: "smtpd_recipient_restrictions", contains: "check_recipient_access,texthash:/data/postfix/dmarc_recipients,reject_unauth_destination,reject_unlisted_recipient", wantExact: true},
{param: "smtpd_milters", contains: "", wantExact: true}, {param: "smtpd_milters", contains: "{unix:/run/selfpost/inbound-journal.sock,default_action=accept}", wantExact: true},
}, },
// Both on: the report addresses sit on a relay domain, and the relay's // Both on: the report addresses sit on a relay domain, and the relay's
// unlisted-recipient check runs after the restriction list. Unless their // unlisted-recipient check runs after the restriction list. Unless their
@@ -94,7 +98,7 @@ var port25Cases = []port25Case{
check: []optionCheck{ check: []optionCheck{
// Exact equality also covers the absence of OpenDKIM/journal-milter // Exact equality also covers the absence of OpenDKIM/journal-milter
// sockets — anything else there would fail the comparison. // sockets — anything else there would fail the comparison.
{param: "smtpd_milters", contains: "{inet:antispam:11332,default_action=accept}", wantExact: true}, {param: "smtpd_milters", contains: "{inet:antispam:11332,default_action=accept},{unix:/run/selfpost/inbound-journal.sock,default_action=accept}", wantExact: true},
}, },
}, },
} }
+421
View File
@@ -0,0 +1,421 @@
package e2e
import (
"bytes"
"compress/gzip"
"encoding/base64"
"fmt"
"net/url"
"regexp"
"strings"
"testing"
"time"
)
// The stand starts with both port-25 features off, as a new installation does.
// The last part of the run switches them on and uses them from outside, the
// way a stranger's mail server would: until 2.0.1 the suite only read back the
// generated configuration, and DMARC ingest shipped in 1.7.0 with a delivery
// service Postfix could not start - nothing ever sent a report through it.
// testEnablePort25Features recreates the container with the inbound relay and
// DMARC ingest on, over the same /data.
func testEnablePort25Features(t *testing.T, sc *scenario) {
h.env = append(h.env, "INBOUND_RELAY_ENABLE=true", "DMARC_REPORTS_ENABLE=true")
if err := h.recreateSelfpost(); err != nil {
t.Fatalf("recreate selfpost with the port-25 features on: %v", err)
}
if err := waitForPanelReady(); err != nil {
t.Fatalf("panel after enabling the port-25 features: %v\n%s", err, h.logs("selfpost"))
}
if err := waitForSMTPSReady(60 * time.Second); err != nil {
t.Fatal(err)
}
if err := waitForPort25Ready(60 * time.Second); err != nil {
t.Fatalf("%v\n%s", err, h.logs("selfpost"))
}
if err := checkSupervisorProcesses(h); err != nil {
t.Fatal(err)
}
// Nothing in the container's own output may complain about the generated
// configuration: the broken delivery service of 1.7.0-1.9.5 was announced
// there at every start ("unmatched service_name/type") and went unread.
logs := h.logs("selfpost")
for _, bad := range []string{"unmatched service_name", "postconf: warning", "postconf: fatal"} {
if strings.Contains(logs, bad) {
t.Fatalf("the container log has %q with the port-25 features on:\n%s", bad, logs)
}
}
resp, err := sc.panel.status()
if err != nil {
t.Fatal(err)
}
if resp.Request.URL.Path != "/overview" {
t.Fatalf("the session did not survive the recreate: landed on %s", resp.Request.URL.Path)
}
}
const reportReporter = "reporter.e2e.test"
// aggregateXML is a DMARC aggregate report on domain, as a receiver sends it.
func aggregateXML(domain, reportID string) string {
return `<?xml version="1.0"?>
<feedback>
<report_metadata>
<org_name>` + reportReporter + `</org_name>
<email>noreply@` + reportReporter + `</email>
<report_id>` + reportID + `</report_id>
<date_range><begin>1723593600</begin><end>1723680000</end></date_range>
</report_metadata>
<policy_published>
<domain>` + domain + `</domain>
<p>none</p><sp>none</sp><pct>100</pct>
<adkim>r</adkim><aspf>r</aspf>
</policy_published>
<record>
<row>
<source_ip>203.0.113.10</source_ip>
<count>7</count>
<policy_evaluated>
<disposition>none</disposition>
<dkim>pass</dkim><spf>pass</spf>
</policy_evaluated>
</row>
<identifiers><header_from>` + domain + `</header_from></identifiers>
</record>
</feedback>
`
}
// reportMessage is the mail a receiver sends a report in: one gzip attachment,
// base64 in 76-column lines.
func reportMessage(t *testing.T, to, xml string) string {
t.Helper()
var gz bytes.Buffer
zw := gzip.NewWriter(&gz)
if _, err := zw.Write([]byte(xml)); err != nil {
t.Fatal(err)
}
if err := zw.Close(); err != nil {
t.Fatal(err)
}
encoded := base64.StdEncoding.EncodeToString(gz.Bytes())
var b strings.Builder
fmt.Fprintf(&b, "From: noreply@%s\r\nTo: %s\r\nSubject: Report domain: %s\r\nDate: %s\r\nMIME-Version: 1.0\r\n",
reportReporter, to, senderDomain, time.Now().UTC().Format(time.RFC1123Z))
b.WriteString("Content-Type: application/gzip; name=\"report.xml.gz\"\r\nContent-Transfer-Encoding: base64\r\n\r\n")
for len(encoded) > 76 {
b.WriteString(encoded[:76] + "\r\n")
encoded = encoded[76:]
}
b.WriteString(encoded + "\r\n")
return b.String()
}
// testDMARCReportReachesThePanel gives the sending domain a SelfPost-hosted
// report address and mails a report to it on port 25. It has to show up under
// Outbound > DMARC reports.
func testDMARCReportReachesThePanel(t *testing.T, sc *scenario) {
address := "dmarc-reports+" + senderDomain + "@" + selfpostHostname
resp, body, err := sc.panel.postForm("/outbound/domains/"+sc.domainID+"/settings/reports", url.Values{"dmarc_rua_mode": {"hosted"}})
if err != nil {
t.Fatal(err)
}
if resp.StatusCode != 200 || resp.Request.URL.Path != "/outbound/domains/"+sc.domainID {
t.Fatalf("choosing the hosted report address: status %d, landed on %s:\n%.800s", resp.StatusCode, resp.Request.URL.Path, body)
}
// An address nobody chose is refused before DATA, hosted form or not.
res := deliverPort25("noreply@"+reportReporter, "dmarc-reports+nobody.e2e.test@"+selfpostHostname, "Subject: x\r\n\r\nx\r\n")
if res.rcptErr == nil {
t.Fatalf("port 25 accepted a report address no domain uses (first error: %v)", res.firstErr())
}
res = deliverPort25("noreply@"+reportReporter, address, reportMessage(t, address, aggregateXML(senderDomain, "e2e-report-1")))
if !res.ok() {
t.Fatalf("port 25 did not accept a report at %s: %v", address, res.firstErr())
}
if err := waitFor("the report to appear under Outbound > DMARC reports", 30*time.Second, 500*time.Millisecond, func() (bool, error) {
resp, body, err := sc.panel.get("/outbound/dmarc")
if err != nil {
return false, err
}
if resp.StatusCode != 200 {
return false, fmt.Errorf("GET /outbound/dmarc = %d", resp.StatusCode)
}
if strings.Contains(body, reportReporter) {
return true, nil
}
return false, fmt.Errorf("no report from %s on the page yet; mail log:\n%s", reportReporter, mailLogTail(h, "dmarc-ingest"))
}); err != nil {
t.Fatal(err)
}
}
// testDMARCJunkIsDropped mails two things that are not reports this relay
// keeps to the same address: a report on somebody else's domain, and a message
// with no report in it. Each is accepted, dropped at once with the reason in
// the mail log, and counted; neither stays in the queue to be tried again and
// neither becomes a report.
func testDMARCJunkIsDropped(t *testing.T, sc *scenario) {
address := "dmarc-reports+" + senderDomain + "@" + selfpostHostname
junk := map[string]string{
"a report on another domain": reportMessage(t, address, aggregateXML("stranger.e2e.test", "e2e-report-2")),
"a message with no report": "From: someone@stranger.e2e.test\r\nTo: " + address + "\r\nSubject: hello\r\n\r\nnot a report\r\n",
}
for what, msg := range junk {
if res := deliverPort25("someone@stranger.e2e.test", address, msg); !res.ok() {
t.Fatalf("%s was refused on port 25 instead of being accepted and dropped: %v", what, res.firstErr())
}
}
dropped := regexp.MustCompile(`status=sent \(delivered via dmarc-ingest service \(dropped: [^)]`)
if err := waitFor("both messages to be dropped", 30*time.Second, 500*time.Millisecond, func() (bool, error) {
log := mailLogTail(h, "dmarc-ingest")
if n := len(dropped.FindAllString(log, -1)); n != len(junk) {
return false, fmt.Errorf("%d of %d dropped so far; mail log:\n%s", n, len(junk), log)
}
return true, nil
}); err != nil {
t.Fatal(err)
}
queue, err := h.execIn("selfpost", "postqueue", "-p")
if err != nil {
t.Fatalf("postqueue -p: %v\n%s", err, queue)
}
if !strings.Contains(queue, "Mail queue is empty") {
t.Fatalf("a dropped message is still in the queue:\n%s", queue)
}
_, body, err := sc.panel.get("/outbound/dmarc")
if err != nil {
t.Fatal(err)
}
if !strings.Contains(body, "2 parse failures") {
t.Fatalf("the DMARC page does not count the two dropped messages as parse failures:\n%s", body)
}
if strings.Contains(body, "stranger.e2e.test") {
t.Fatal("the report on another domain was stored")
}
}
// mailLogTail returns the lines of the end of the mail log that contain filter.
func mailLogTail(s *stack, filter string) string {
out, err := s.execIn("selfpost", "tail", "-n", "400", "/data/log/mail.log")
if err != nil {
return "(mail log unreadable: " + err.Error() + ")"
}
var b strings.Builder
for _, line := range strings.Split(out, "\n") {
if strings.Contains(line, filter) {
b.WriteString(line + "\n")
}
}
return b.String()
}
const (
inboundDomain = "in.e2e.test"
inboundRecipient = "user@" + inboundDomain
)
// testInboundRelay sets up an inbound domain with the sink as its upstream and
// one listed recipient, then talks to port 25 as a stranger: mail for the
// listed recipient reaches the upstream, mail for anyone else at the domain
// and for a domain that is not configured is refused before DATA.
func testInboundRelay(t *testing.T, sc *scenario) {
resp, body, err := sc.panel.postForm("/inbound/domains", url.Values{"name": {inboundDomain}})
if err != nil {
t.Fatal(err)
}
m := regexp.MustCompile(`^/inbound/domains/(\d+)$`).FindStringSubmatch(resp.Request.URL.Path)
if m == nil {
t.Fatalf("add inbound domain: landed on %s:\n%.800s", resp.Request.URL.Path, body)
}
base := "/inbound/domains/" + m[1]
// The sink by address: an upstream named by an IP is delivered to as it
// stands, with no MX lookup in between.
resp, body, err = sc.panel.postForm(base+"/upstream", url.Values{"host": {"10.77.0.11"}, "port": {"25"}, "tls_mode": {"none"}})
if err != nil {
t.Fatal(err)
}
if resp.StatusCode != 200 || resp.Request.URL.Path != base {
t.Fatalf("set the upstream: status %d, landed on %s:\n%.800s", resp.StatusCode, resp.Request.URL.Path, body)
}
resp, body, err = sc.panel.postForm(base+"/recipients", url.Values{"recipient_mode": {"list"}, "addresses": {inboundRecipient}})
if err != nil {
t.Fatal(err)
}
if resp.StatusCode != 200 || resp.Request.URL.Path != base {
t.Fatalf("set the recipients: status %d, landed on %s:\n%.800s", resp.StatusCode, resp.Request.URL.Path, body)
}
message := func(to, token string) string {
return fmt.Sprintf("From: someone@stranger.e2e.test\r\nTo: %s\r\nSubject: e2e %s\r\nDate: %s\r\n\r\nmail for the inbound domain\r\n",
to, token, time.Now().UTC().Format(time.RFC1123Z))
}
token := uniqueToken("inbound")
if res := deliverPort25("someone@stranger.e2e.test", inboundRecipient, message(inboundRecipient, token)); !res.ok() {
t.Fatalf("port 25 did not accept mail for %s: %v", inboundRecipient, res.firstErr())
}
if _, err := findSinkMessage(h.stageDir, token, 30*time.Second); err != nil {
t.Fatalf("mail for the inbound domain never reached its upstream: %v\n%s", err, mailLogTail(h, "postfix/"))
}
// The filter lists exist as files from the first start with the relay on,
// readable by a filter that is another user in another container. Nothing
// is on them until someone adds an entry.
maps, err := h.execIn("selfpost", "sh", "-c", `cd /data/antispam && stat -c "%a %n" . *.map && { grep -vc "^#" *.map; true; }`)
if err != nil {
t.Fatalf("list /data/antispam: %v\n%s", err, maps)
}
for _, want := range []string{"755 .",
"644 allow_domain.map", "644 allow_from.map", "644 allow_ip.map",
"644 deny_domain.map", "644 deny_from.map", "644 deny_ip.map",
"allow_ip.map:0", "deny_from.map:0",
} {
if !strings.Contains(maps, want) {
t.Fatalf("/data/antispam: no %q in\n%s", want, maps)
}
}
// The inbound journal milter saw it go through, and the Inbound log shows it.
// No filter is attached on the stand: the row is an accept by nobody, so its
// reason is the engine alone.
logRows := func(want string) []string {
t.Helper()
var rows []string
var body string
for deadline := time.Now().Add(15 * time.Second); ; time.Sleep(time.Second) {
resp, b, err := sc.panel.get("/inbound/log")
if err != nil || resp.StatusCode != 200 {
t.Fatalf("GET /inbound/log: status %v, %v\n%.800s", resp, err, b)
}
body = b
rows = rows[:0]
for _, row := range strings.Split(body, "<tr><td")[1:] {
// The row's own end is the last one: the sheet in its
// decision cell holds a table of its own.
rows = append(rows, row[:strings.LastIndex(row, "</tr>")])
}
if strings.Contains(body, want) {
return rows
}
if time.Now().After(deadline) {
t.Fatalf("no row with %q in the Inbound log:\n%.3000s\n%s", want, body, mailLogTail(h, "postfix/"))
}
}
}
// rowWith finds the one row that holds all of wants.
rowWith := func(rows []string, wants ...string) string {
t.Helper()
for _, row := range rows {
ok := true
for _, w := range wants {
ok = ok && strings.Contains(row, w)
}
if ok {
return row
}
}
t.Fatalf("no row of the Inbound log holds all of %q in\n%s", wants, strings.Join(rows, "\n"))
return ""
}
rows := logRows("e2e " + token)
rowWith(rows,
"someone@<wbr>stranger.e2e.test", "user@<wbr>"+inboundDomain, "e2e "+token,
`class="tag is-success is-light" popovertarget="why-`, `<th>Decided by</th><td>selfpost</td>`,
// the way to correct it, which only the global role is given
`href="/inbound/filter-lists?list=deny&amp;kind=address&amp;value=someone%40stranger.e2e.test">Deny sender</a>`)
// A report mailed to the server's own name earlier in the run came through
// the same listener and is not the relay's mail.
if len(rows) != 1 {
t.Fatalf("the Inbound log holds %d rows, want the relayed message alone:\n%s", len(rows), strings.Join(rows, "\n"))
}
for _, to := range []string{"nobody@" + inboundDomain, "someone@elsewhere.e2e.test"} {
res := deliverPort25("someone@stranger.e2e.test", to, message(to, uniqueToken("refused")))
if res.rcptErr == nil {
t.Fatalf("port 25 accepted the recipient %s (first error: %v)", to, res.firstErr())
}
}
// The refusal of an unknown recipient at the inbound domain is read from
// the mail log into the journal; the refusal to relay for a stranger's
// domain is not the relay's mail and is not.
rows = logRows("nobody@<wbr>" + inboundDomain)
refused := rowWith(rows,
"someone@<wbr>stranger.e2e.test", "nobody@<wbr>"+inboundDomain, `<td class="sp-clip sp-muted">—</td>`,
`class="tag is-danger is-light" popovertarget="why-`, `<th>Decided by</th><td>postfix</td>`,
`<th>Reply</th><td class="sp-mono sp-small">550 5.1.1 `)
// A message Postfix itself refused offers no way into the lists.
if strings.Contains(refused, "sender</a>") {
t.Fatalf("the refused row offers a way into the filter lists:\n%s", refused)
}
if len(rows) != 2 {
t.Fatalf("the Inbound log holds %d rows, want the relayed message and one refusal:\n%s", len(rows), strings.Join(rows, "\n"))
}
// The filter lists: an entry added in the panel is in the map file the
// filter reads, and one removed is gone from it. Nothing reads the file on
// this stand (no filter is attached), so the file is the whole effect.
const spammer = "spammer.e2e.test"
denyDomain := func() string {
t.Helper()
out, err := h.execIn("selfpost", "cat", "/data/antispam/deny_domain.map")
if err != nil {
t.Fatalf("read deny_domain.map: %v\n%s", err, out)
}
return out
}
resp, body, err = sc.panel.postForm("/inbound/filter-lists", url.Values{
"list_type": {"deny"}, "entry_type": {"domain"}, "value": {" @" + strings.ToUpper(spammer)}, "note": {"e2e"}})
if err != nil {
t.Fatal(err)
}
if resp.StatusCode != 200 || resp.Request.URL.Path != "/inbound/filter-lists" || !strings.Contains(body, "Entry added.") {
t.Fatalf("add a filter list entry: status %d, landed on %s:\n%.800s", resp.StatusCode, resp.Request.URL.Path, body)
}
if got := denyDomain(); !strings.Contains(got, "\n"+spammer+"\n") {
t.Fatalf("deny_domain.map after adding %s:\n%s", spammer, got)
}
if got, err := h.execIn("selfpost", "sh", "-c", `cd /data/antispam && { grep -vc "^#" *.map; true; }`); err != nil ||
strings.Count(got, ":0") != 5 || !strings.Contains(got, "deny_domain.map:1") {
t.Fatalf("the entry is in other files than deny_domain.map (%v):\n%s", err, got)
}
// The same entry again is refused on the page, and the file still holds it once.
resp, body, err = sc.panel.postForm("/inbound/filter-lists", url.Values{
"list_type": {"deny"}, "entry_type": {"domain"}, "value": {spammer}})
if err != nil {
t.Fatal(err)
}
if resp.StatusCode != 422 || !strings.Contains(body, "That entry is already on the list.") {
t.Fatalf("add the same entry again: status %d:\n%.800s", resp.StatusCode, body)
}
if got := denyDomain(); strings.Count(got, spammer) != 1 {
t.Fatalf("deny_domain.map after a duplicate:\n%s", got)
}
resp, body, err = sc.panel.get("/inbound/filter-lists")
if err != nil || resp.StatusCode != 200 {
t.Fatalf("GET /inbound/filter-lists: status %v, %v", resp, err)
}
m = regexp.MustCompile(`action="(/inbound/filter-lists/\d+/delete)"`).FindStringSubmatch(body)
if m == nil || !strings.Contains(body, spammer) {
t.Fatalf("the list does not show %s with a Remove:\n%.2000s", spammer, body)
}
resp, body, err = sc.panel.postForm(m[1], nil)
if err != nil {
t.Fatal(err)
}
if resp.StatusCode != 200 || resp.Request.URL.Path != "/inbound/filter-lists" || !strings.Contains(body, "Entry removed.") {
t.Fatalf("remove the entry: status %d, landed on %s:\n%.800s", resp.StatusCode, resp.Request.URL.Path, body)
}
if got := denyDomain(); strings.Contains(got, spammer) {
t.Fatalf("deny_domain.map after removing %s:\n%s", spammer, got)
}
}
+198
View File
@@ -0,0 +1,198 @@
package e2e
import (
"archive/tar"
"bytes"
"compress/gzip"
"fmt"
"io"
"net/url"
"os"
"path/filepath"
"strings"
"testing"
"time"
)
// testFullBackupRestore takes a full backup from the panel, throws the
// installation away and brings it back the way the guide tells an operator to
// (§ Full backup and restore): unpack the archive into an empty project
// directory, start the same image. What has to come back is what an operator
// cannot recreate - the administrator, the domain with its DKIM key, the
// application with its password - and the proof is the same one the first
// send gave: a message signed with the restored key verifies against the
// record that was published before the backup.
func testFullBackupRestore(t *testing.T, sc *scenario) {
// A private key the panel may not read - the state a server is in until
// its operator gives the key to group 998. The panel does not make an
// archive without it: it refuses, names the file and says what to do.
keyPath := filepath.Join(h.stageDir, "certs", "privkey.pem")
if err := os.Chmod(keyPath, 0o600); err != nil {
t.Fatal(err)
}
resp, body, err := sc.panel.postForm("/server/backup", url.Values{})
if err != nil {
t.Fatalf("full backup with an unreadable key: %v", err)
}
if resp.StatusCode != 500 || !strings.Contains(body, "not allowed to read certs/privkey.pem") || !strings.Contains(body, "chgrp 998") {
t.Fatalf("full backup with an unreadable key: status %d, want a refusal that names the file and the way out:\n%.800s", resp.StatusCode, body)
}
if err := os.Chmod(keyPath, 0o644); err != nil {
t.Fatal(err)
}
resp, archive, err := sc.panel.postForm("/server/backup", url.Values{})
if err != nil {
t.Fatalf("full backup: %v", err)
}
if ct := resp.Header.Get("Content-Type"); resp.StatusCode != 200 || ct != "application/gzip" {
t.Fatalf("full backup: status %d, Content-Type %q, want an application/gzip download:\n%.600s", resp.StatusCode, ct, archive)
}
// A download that broke off half way is still a 200: the panel streams the
// archive and can only log what went wrong. Read it through once while the
// container that wrote it can still say why.
if _, err := untar(t.TempDir(), []byte(archive)); err != nil {
t.Fatalf("the backup the panel sent (%d bytes) is not a complete archive: %v\n%s", len(archive), err, logTail(h.logs("selfpost"), "backup"))
}
// The installation goes: containers down, then everything the archive is
// supposed to bring back. The fake zone and the sink's directory are the
// outside world and stay.
h.reclaimData()
h.down()
for _, name := range []string{"data", "certs", "docker-compose.yml", ".env"} {
if err := removeAllBestEffort(filepath.Join(h.stageDir, name)); err != nil {
t.Fatalf("remove %s before restore: %v", name, err)
}
}
names, err := untar(h.stageDir, []byte(archive))
if err != nil {
t.Fatalf("unpack the backup into the empty project directory: %v", err)
}
for _, want := range []string{"docker-compose.yml", ".env", "data/manifest.json", "data/selfpost.db", "certs/fullchain.pem", "certs/privkey.pem"} {
if !names[want] {
t.Fatalf("the backup has no %s (it has %d entries)", want, len(names))
}
}
// No backup holds the Postfix queue (guide § Full backup and restore): the
// restored instance has to build an empty one.
for name := range names {
if name == "data/postfix/queue" || strings.HasPrefix(name, "data/postfix/queue/") {
t.Fatalf("the backup holds %s", name)
}
}
if err := h.up(); err != nil {
t.Fatalf("start the restored installation: %v", err)
}
if err := waitForPanelReady(); err != nil {
t.Fatalf("the restored installation did not come up: %v\n%s", err, h.logs("selfpost"))
}
if err := waitForSMTPSReady(60 * time.Second); err != nil {
t.Fatal(err)
}
// The manifest guards the first start only and is removed by it.
if err := waitFor("manifest.json to be removed by the first start", 30*time.Second, 500*time.Millisecond, func() (bool, error) {
out, _ := h.execIn("selfpost", "sh", "-c", "test -e /data/manifest.json && echo present || echo gone")
return strings.TrimSpace(out) == "gone", fmt.Errorf("manifest.json is %s", strings.TrimSpace(out))
}); err != nil {
t.Fatal(err)
}
p, err := newPanelClient()
if err != nil {
t.Fatal(err)
}
if err := p.login(adminUser, adminPassword); err != nil {
t.Fatalf("the administrator cannot sign in after the restore: %v", err)
}
sc.panel = p
_, value, err := p.dkimRecord(sc.domainID)
if err != nil {
t.Fatalf("the domain is not there after the restore: %v", err)
}
if value != sc.dkimValue {
t.Fatalf("the domain came back with another DKIM key:\n before %s\n after %s", sc.dkimValue, value)
}
token := uniqueToken("restored")
res := attemptSend(sendAttempt{
authLogin: sc.appLogin, authPassword: sc.appPassword,
from: "alerts@" + senderDomain, to: recipient,
subject: "e2e " + token, body: "sent by the restored installation",
})
if !res.ok() {
t.Fatalf("the application cannot send after the restore: %v", res.firstErr())
}
raw, err := findSinkMessage(h.stageDir, token, 30*time.Second)
if err != nil {
t.Fatalf("the restored installation did not deliver: %v", err)
}
if err := verifyDKIM(raw, senderDomain); err != nil {
t.Fatalf("mail signed after the restore does not verify against the record published before it: %v", err)
}
}
// untar unpacks a .tar.gz into dir and returns the names it held. Only
// directories and regular files are expected in a backup; anything else, or a
// name that leaves dir, is an error rather than something to skip quietly.
func untar(dir string, archive []byte) (map[string]bool, error) {
zr, err := gzip.NewReader(bytes.NewReader(archive))
if err != nil {
return nil, err
}
names := map[string]bool{}
tr := tar.NewReader(zr)
for {
hdr, err := tr.Next()
if err == io.EOF {
return names, nil
}
if err != nil {
return nil, err
}
name := filepath.ToSlash(filepath.Clean(hdr.Name))
if name == ".." || strings.HasPrefix(name, "../") || filepath.IsAbs(hdr.Name) {
return nil, fmt.Errorf("entry %q leaves the directory", hdr.Name)
}
target := filepath.Join(dir, filepath.FromSlash(name))
switch hdr.Typeflag {
case tar.TypeDir:
if err := os.MkdirAll(target, 0o755); err != nil {
return nil, err
}
case tar.TypeReg:
if err := os.MkdirAll(filepath.Dir(target), 0o755); err != nil {
return nil, err
}
f, err := os.OpenFile(target, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, hdr.FileInfo().Mode().Perm())
if err != nil {
return nil, err
}
if _, err := io.Copy(f, tr); err != nil {
f.Close()
return nil, err
}
if err := f.Close(); err != nil {
return nil, err
}
default:
return nil, fmt.Errorf("entry %q has type %q, neither a directory nor a regular file", hdr.Name, hdr.Typeflag)
}
names[name] = true
}
}
// logTail returns the lines of log that contain filter.
func logTail(log, filter string) string {
var b strings.Builder
for _, line := range strings.Split(log, "\n") {
if strings.Contains(line, filter) {
b.WriteString(line + "\n")
}
}
return b.String()
}
+13
View File
@@ -24,6 +24,10 @@ type stack struct {
baseComposeFile string baseComposeFile string
overrideFile string overrideFile string
stageDir string stageDir string
// env is added to compose's own environment. The shipped compose file takes
// the optional features from there (${INBOUND_RELAY_ENABLE:-false} and the
// like), so this is how a later part of the run switches them on.
env []string
} }
func newStack() (*stack, error) { func newStack() (*stack, error) {
@@ -64,6 +68,7 @@ func (s *stack) compose(args ...string) ([]byte, error) {
// looked at (that mapping only reaches the container, not compose's own // looked at (that mapping only reaches the container, not compose's own
// variable interpolation), so it has to be set here too. // variable interpolation), so it has to be set here too.
cmd.Env = append(os.Environ(), "SELFPOST_HOSTNAME="+selfpostHostname) cmd.Env = append(os.Environ(), "SELFPOST_HOSTNAME="+selfpostHostname)
cmd.Env = append(cmd.Env, s.env...)
var out bytes.Buffer var out bytes.Buffer
cmd.Stdout = &out cmd.Stdout = &out
cmd.Stderr = &out cmd.Stderr = &out
@@ -102,6 +107,14 @@ func (s *stack) restartSelfpost() error {
return err return err
} }
// recreateSelfpost replaces the selfpost container with one started from the
// same /data and the environment as it is now (see env), leaving the DNS and
// sink sidecars and their state alone.
func (s *stack) recreateSelfpost() error {
_, err := s.compose("up", "-d", "--force-recreate", "--no-deps", "selfpost")
return err
}
func (s *stack) down() { func (s *stack) down() {
_, _ = s.compose("down", "-v", "--remove-orphans") _, _ = s.compose("down", "-v", "--remove-orphans")
} }
+28 -1
View File
@@ -39,6 +39,9 @@ func prepareStage(s *stack) error {
); err != nil { ); err != nil {
return err return err
} }
if err := writeDeployFiles(s); err != nil {
return err
}
corefile, err := os.ReadFile(filepath.Join("dns", "Corefile")) corefile, err := os.ReadFile(filepath.Join("dns", "Corefile"))
if err != nil { if err != nil {
return fmt.Errorf("read Corefile: %w", err) return fmt.Errorf("read Corefile: %w", err)
@@ -49,6 +52,22 @@ func prepareStage(s *stack) error {
return writeZone(s.stageDir, nil) return writeZone(s.stageDir, nil)
} }
// writeDeployFiles puts docker-compose.yml and .env into the stage directory.
// The stage directory is what the container sees as /selfpost-deploy, the
// operator's project directory, and a full backup refuses to run unless both
// files are there to be archived. compose reads neither: the stand names its
// files with -f, and the .env repeats what compose is given anyway.
func writeDeployFiles(s *stack) error {
compose, err := os.ReadFile(s.baseComposeFile)
if err != nil {
return fmt.Errorf("read the shipped compose file: %w", err)
}
if err := os.WriteFile(filepath.Join(s.stageDir, "docker-compose.yml"), compose, 0o644); err != nil {
return err
}
return os.WriteFile(filepath.Join(s.stageDir, ".env"), []byte("SELFPOST_HOSTNAME="+selfpostHostname+"\n"), 0o644)
}
// removeAllBestEffort deletes path; if a previous run left container-UID files // removeAllBestEffort deletes path; if a previous run left container-UID files
// on the bind mount, a root alpine one-shot removes them first. // on the bind mount, a root alpine one-shot removes them first.
func removeAllBestEffort(path string) error { func removeAllBestEffort(path string) error {
@@ -94,5 +113,13 @@ func writeSelfSignedCert(certPath, keyPath string) error {
return err return err
} }
keyPEM := pem.EncodeToMemory(&pem.Block{Type: "RSA PRIVATE KEY", Bytes: x509.MarshalPKCS1PrivateKey(key)}) keyPEM := pem.EncodeToMemory(&pem.Block{Type: "RSA PRIVATE KEY", Bytes: x509.MarshalPKCS1PrivateKey(key)})
return os.WriteFile(keyPath, keyPEM, 0o600) // Readable by everyone, which a real key must never be. On a server the
// operator gives the key to the container's selfpost group (998) so the
// panel can put it into a full backup; the runner of this suite is not
// always root and cannot hand a file to a group it is not in. The key is
// thrown away with the stage directory.
if err := os.WriteFile(keyPath, keyPEM, 0o644); err != nil {
return err
}
return os.Chmod(keyPath, 0o644)
} }