127 Commits
Author SHA1 Message Date
mixandClaude Opus 5.5 d20009c0f0 sasldb2 changed through a copy renamed over it, never in place
test / test (push) Canceled after 0s
Cyrus SASL 2.1.28 opens sasldb2 with no Berkeley DB environment, so with
no locks (sasldb/db_berkeley.c, berkeleydb_open(): db_create(mbdb, NULL,
0)); saslpasswd2 puts and deletes in the file itself (_sasldb_putdata()),
while smtpd opens and closes it at every AUTH (_sasldb_getdata()) and the
backup copies it whenever it runs. Either could read a half-written page:
a spurious 535, or a torn sasldb2 in a backup whose restore breaks every
application's AUTH (assumptions C-S3, C-Q3).

SASLDB.Set and Delete now run saslpasswd2 on sasldb2.new - a byte copy of
the database, or no file when there is none yet (saslpasswd2 creates it,
DB_CREATE) - then chmod 0640, fsync, rename over sasldb2; a failed run
removes the copy and leaves the database; a copy a killed panel left is
removed first. A mutex keeps two changes of the panel from copying the
same old file. The backup needs no lock after all (the plan said one): it
reads each file through one descriptor, and a rename leaves that file
whole; the walk now leaves out a file that vanished between listing and
open (a writer's temporary file), while the SQLite snapshot's absence
stays an error. CHANGELOG also gains the entries of f2ad699 (supervision),
written then without them.

Tests: the copy holds the database's bytes, saslpasswd2 never gets the
live path, a reader of the old file keeps reading it (Linux), mode 0640,
nothing left behind; a failed change leaves the database; the first
account creates it. Stand (2026-10-15, make e2e, exit 0): new
sasldb_changed_by_rename - the inode of sasldb2 changes across one
password change (it cannot be the old one's: both exist at the copy),
then 35 AUTHs of app1 from three clients during 104 password changes of
another application, none refused; afterwards 640 panel selfpost, no copy
left in /data/sasl, the last password authenticates. That cannot prove a
race absent; it guards the mechanism. (A first run compared the inode
after 15 changes and found it equal: Linux reuses a freed inode number.)
go vet and go test ./... pass on the server (24 packages).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 00:19:03 +03:00
mixandClaude Opus 5.5 f2ad699473 supervision as supervisord does it: Postfix stopped, give-ups FATAL, reload's exit read
test / test (push) Canceled after 0s
C-P42. postfix-script's start-fg runs master as its child and master
calls setsid() (Postfix 3.7.11, conf/postfix-script; src/master/master.c);
supervisord signals the pid alone (supervisor 4.2.5, process.py,
Subprocess.kill()). A stop of program postfix left master running - a
restart then found "already running" - and a container stop cut Postfix
off with SIGKILL. The wrapper now stays the parent of "postfix start-fg"
and on TERM/INT runs "postfix stop", asking again while start-fg is still
in its checks (master -t). stopwaitsecs=15.

C-V6. The wrapper's give-up on the milter sockets came after the default
startsecs, so it was EXITED and restarted for ever. startsecs is now
MILTER_WAIT_TIMEOUT + 10, exported by the entrypoint as POSTFIX_STARTSECS
and expanded by supervisord (options.py, processes_from_section(); a
non-number MILTER_WAIT_TIMEOUT stops the start): a give-up is "Exited too
quickly", BACKOFF, FATAL after startretries, crashexit. For those seconds
after each start the program is STARTING (the healthcheck's 90 s start
period covers it; the sender check counts no downtime then).

C-V5. logrotate had startsecs=0, so a refused config restarted it every
second for ever. startsecs=1: FATAL after startretries; crashexit reads
the event's processname and leaves logrotate's FATAL on Server > Health.

B-U2. "supervisorctl start" returns when a startsecs=0 program is RUNNING,
not when it ends, and mapped "already started" to success. The panel now
speaks supervisord's XML-RPC itself (internal/supervisor, RunOnce):
startProcess without waiting, getProcessInfo until it leaves
STARTING/RUNNING, exitstatus 0 or an error; a run in progress is waited
out and one of its own follows. A failed reload while program postfix is
not RUNNING is logged, not an error: the next master reads the files.
C-V10: postfix-reload rests EXITED after a run; the e2e process check
takes STOPPED or EXITED. C-P34: the dead "postfix set-permissions" branch
goes - postfix check's create-missing builds the queue tree.

Stand (2026-10-14, make e2e, exit 0; MILTER_WAIT_TIMEOUT=10 on the stand):
new supervision_stops_postfix_and_gives_up - supervisorctl restart postfix
writes postfix-script's "stopping the Postfix mail system" and leaves one
master; with the panel's sockets removed, start answers "postfix: ERROR
(spawn error)" (finish() sets spawnerr "Exited too quickly", onwait()
reads it first) and Postfix comes up once the panel is back; a
group-writable logrotate config ends FATAL with the container up; the
panel's Reload configuration answers only after mail.log has "refreshing
the Postfix mail system", postfix-reload EXITED. session_survives_restart
sees the stop line for the container's restart. Measured: master's own
"terminating on signal 15" never reaches mail.log (master TERMs its group,
postlogd included, before it logs). go vet and go test ./... pass on the
server (24 packages; new unit tests of RunOnce against a fake supervisord).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 23:49:25 +03:00
mixandClaude Opus 5.5 ee37d16dc7 audit: stage 2c - seven lines built, ticked
31c2b50, 55aef6e, 0ed03b2 (two lines), c8f6697, 463d4ff, e174537 built
the lines without ticking them. Roadmap 38/81.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 23:13:27 +03:00
mixandClaude Opus 5.5 463d4ff3fe log tailer: the rotated mail.log read until postlogd lets it go
test / test (push) Canceled after 0s
postlogd opens maillog_file once per process (Postfix 3.7.11,
src/postlogd/postlogd.c, pre_jail_init()) and leaves it only when it
exits: at "postfix reload" master closes the service's status pipe
(src/master/master_service.c, master_restart_service() ->
master_status_cleanup()) and dgram_server_abort() exits on reading its end
(src/master/dgram_server.c). logrotate renames and creates, then runs the
reload (build/logrotate-mail.conf postrotate; logrotate 3.21.0,
logrotate.c, rotateLogSet(), after rotateSingleLog()), so lines go to the
renamed file in between, and the tailer, already on the new one, never
read them (assumptions A-33). follow() now keeps the renamed file open as a
rotatedTail and reads it on each tick for rotatedGrace (a minute, far
longer than the gap); its places are never saved as the resume point, so a
restart still reads the rotated file through readRotated. A truncated file
is not kept.

The RFC 3339 stamp goes (A-12): it was written for a maillog_file_format
said to arrive in Postfix 3.9; there is none in 3.7.11 nor in the 3.12
snapshot of 2026-09-25 (proto/postconf.proto; msg_logger.c,
msg_logger_print() keeps strftime("%b %d %H:%M:%S ")). Every fixture now
carries the stamp Postfix writes; logtail's TestMain fixes time.Local to
UTC, since the stamp is local time and this PC is UTC+3.

Tests: TestFollowReadsTheRotatedFileAfterTheSwitch (Linux: a line appended
to the renamed file after the switch reaches the store). Stand
(2026-10-14): go vet and go test ./... pass on the server (23 packages);
make e2e exit 0, delivery_log_survives_rotation included. The tested tree
was this commit plus the previous one; the CHANGELOG entry was written
after the run, and a comment's logrotate citation corrected
(rotateLogSet runs postrotate, not rotateSingleLog).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 23:12:37 +03:00
mixandClaude Opus 5.5 e174537512 postfix config: port 25's size limit for cleanup too, no local delivery
Port 25 says it takes INBOUND_MESSAGE_SIZE_LIMIT (25 MiB by default) with
smtpd -o message_size_limit, but cleanup writes the queue file under its
own limit, main.cf's - never set, so DEF_MESSAGE_LIMIT, 10240000 (Postfix
3.7.11, src/cleanup/cleanup_init.c, cleanup_post_jail(), set_file_limit();
src/global/mail_params.h). A larger message went through DATA and was
refused 552 (assumptions C-P31). cleanup now gets port 25's limit as a
master.cf -o when it is larger than main.cf's; 465 and 587 keep main.cf's
in smtpd's own DATA/BDAT checks. /etc/postfix outlives a restart, so the
override is removed with postconf -PX first on every start.

An empty mydestination left one local destination: an address literal of
an interface the container listens on (src/global/resolve_local.c,
resolve_local(), own_inet_addr()). local_transport is now
"error:local mail delivery is disabled", as STANDARD_CONFIGURATION_README's
null client has it (assumptions C-P8).

Stand (2026-10-14, make e2e, exit 0): inbound_relay_delivers_and_refuses
sends a 12 MB message through port 25 and finds it at the upstream, and
counts it among the Inbound log's rows; new address_literal_is_not_local -
probe@[127.0.0.1] from an application is refused at RCPT "550 5.1.1 ...
local mail delivery is disabled" (smtpd's check_rcpt_maps() rejects what
resolves to the error transport; before, local(8) took it). go vet and go
test ./... pass on the server (23 packages). Tested together with the next
commit (the log tailer); the two touch no common file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 23:11:20 +03:00
mixandClaude Opus 5.5 c8f669714e journal milter: recipients as Postfix logs them, once each
test / test (push) Canceled after 0s
The row's recipient has to be the address Postfix writes in to=<..> or
orig_to=<..>, or the delivery line finds no row and the sweep closes it as
bounced. Postfix 3.7.11 hands the milter the RCPT argument as the client
wrote it (smtpd.c, milter_argv(); extract_addr() leaves it), go-milter
v0.4.1 trims "<>" and drops the ESMTP parameters (session.go, case
CodeRcpt) - so cleanAddress's cut at the first blank broke a quoted local
part (B-G5). Postfix keeps the address in internal form (tok822: quotes
off, backslash escapes taken) and logs it external
(info_log_addr_form.c, quote_822_local(): quoted again only when not a
dot-string); recipientAsLogged does the same, "ab"@x -> ab@x, "a b"@x
stays. A recipient given twice in any case is one row: cleanup drops the
duplicate, comparing folded (cleanup_out_recipient.c, been_here(),
BH_FLAG_FOLD; A-31). A rewritten recipient is matched by orig_to=, which
log_adhoc() writes only when it differs (A-9). The store's
case-insensitive match stays, now with its reason from the source (D-S20).

Stand (2026-10-14, make e2e, exit 0): new recipients_as_postfix_logs_them
- RCPT TO "x y"@, "ab"@ and "X Y"@ sink.e2e.test give two rows, "x
y"@sink.e2e.test and ab@sink.e2e.test, both sent. go vet and go test ./...
pass on the server; a unit test drives the milter through go-milter's own
client.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 20:30:47 +03:00
mixandClaude Opus 5.5 0ed03b2094 log tailer: expiry, Postfix's time values and refusal lines as Postfix writes them
test / test (push) Canceled after 0s
Expiry (assumptions A-8). Postfix 3.7.11 logs the end of a message's queue
lifetime once per message, "<id>: from=<..>, status=expired, returned to
sender" (src/qmgr/qmgr_active.c, qmgr_active_done_25_generic()), with no
line per recipient; the tailer waited for "to=..., status=expired", which
no Postfix writes, and the sweep closed only queued rows, so such rows
stayed deferred. The tailer now reads qmgr's line (ExpireMessage: each
recipient's newest open row of the id becomes bounced); the sweep closes a
deferred row too when Postfix no longer holds the message, and counts as
missed only a final line (sent, bounced, expiry). The queue listing's '#'
(forced expiry, showq_compat.c showq_message()) is read as a flag (B-PQ1).

Time values (B-PQ10, B-PQ11): conv_time() ported - one number, one
lower-case unit, the parameter's default unit when none (get_def_time_unit:
d for the lifetimes, h for delay_warning_time), the rest ignored; postconf
-h -x expands $name; bounce_queue_lifetime capped at maximal_queue_lifetime
as qmgr_post_init() caps it, so LongestLifetime is the queue lifetime.

Refusals (A-21, A-23, A-24, A-15): smtpd_whatsup() ends the NOQUEUE line
with sasl_username=<login> on an authenticated session; the line is now
matched to its row by client, login and sender, and a line without a login
is no application's. The client= line's login ends at the comma. The
"IPv6:" trim goes (namaddr is built from state->addr). A row recovered from
cleanup's line says that line names the last recipient only.

Stand (2026-10-14, make e2e, exit 0): new expired_message_is_closed -
defer_transports=smtp makes the retry service defer the message, postsuper
-e and postqueue -i force its expiry; mail.log has "<id>:
from=<alerts@sender.e2e.test>, status=force-expired, returned to sender"
and the row goes deferred -> bounced; refusals_show_postfixs_line passes
with the login match. go vet and go test ./... pass on the server.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 20:21:51 +03:00
mixandClaude Opus 5.5 55aef6e721 sender check: the sender map read and looked up as Postfix does it
Postfix 3.7.11 takes the owners of the first key mail_addr_find() finds -
the address, then without its extension, then @domain, each tried only
when the one before is absent (src/global/mail_addr_find.c,
mail_addr_find_opt()) - and compares the login with them by
strcasecmp_utf8() (src/smtpd/smtpd_check.c,
reject_auth_sender_login_mismatch()). The check accepted a From: address
when any of the three keys named the application: an application holding
@domain could write an address listed for another application of the
domain, which Postfix refuses it in the envelope, and have it signed
(assumptions B-PS1).

The table is now read as dict_thash_open() reads it: continuation lines,
owners split on commas and blanks (B-PS4), the key ending at the first
blank outside quotes, keys folded; a key given twice makes Postfix refuse
the whole table (no DICT_FLAG_DUP_* flag), and the check then defers too.
The writer folds keys, so two cases of one address cannot be two lines.

e2e on the stand (2026-10-14, make e2e, exit 0): the wildcard application
is refused billing@ - listed for app-list - in From: (550 5.7.1), and in
the envelope by Postfix at RCPT ("553 5.7.1 <billing@...>: Sender address
rejected: not owned by user"). go vet and go test ./... pass on the
server (23 packages).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 20:03:34 +03:00
mixandClaude Opus 5.5 31c2b50457 antispam: rspamd strips a sender's own verdict for every client
test / test (push) Canceled after 0s
rspamd 3.11.1's milter_headers skips its verdict routines - removes none
of the sender's X-Spamd-Result / X-Rspamd-Action, adds none - for a client
it counts as local (loopback, a unix socket, local_addrs: the private
ranges) or one that authenticated (skip_local, skip_authenticated, both on
by default; milter_headers.lua skip_wanted(), lua_ip.c lua_ip_is_local()).
The custom routine that signs X-SelfPost-Filter is never put through
skip_wanted(), so it signed the sender's own verdict, and Inbound > Log
showed it as rspamd's. Every sender is private when Docker's proxy relays
port 25 from the bridge gateway. Both options set to false.

Tried 2026-10-14 in a throwaway rspamd/rspamd:3.11.1-3 on the test server
(rspamc --ucl, a message carrying its own verdict headers): old config,
from 10.0.0.5 and as an authenticated user, the milter reply removed and
added neither and added the signature; new config, both removed and
rspamd's own added, as from 203.0.113.9.

Also: the sender rules take the envelope only (extract_from = "smtp"; by
default a bounce's From: header is used - tried: a bounce whose From:
domain is denied was refused before, is not now; a denied sender is
refused either way); the image pinned to 3.11.1-3, the build read (3.11 and
3.11.1 move with each rebuild); the prefilter lines no longer read go.
architecture.md and security.md say so. Assumptions C-R7, B-R1, C-R4,
C-R12, C-R5.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 19:48:07 +03:00
mixandClaude Opus 5.5 a658da11f7 audit: stage 2c - the answers, and what they make to fix
test / test (push) Canceled after 0s
docs/assumptions-2026-10-14.md gets its second half: every entry of the
inventory settled in the source of the version the image runs, with the
citation now beside it in the code. 559 verdicts (the documents' claims
that repeat one another answered once): 415 hold, 77 hold in part, 55 do
not hold, 12 need a measurement on the stand.

The plan's heuristics line is done; what the sources contradict becomes
fourteen lines of stage 2c, worst first: rspamd signs a forged verdict for
a client it counts as local; the sender check takes any key of the sender
map where Postfix takes the first; expiry, the forced-expire flag and
Postfix's time units; refusals and recipients matched by what Postfix
really writes; postlogd after a rename; inbound mail over 10 MB refused by
cleanup; master outside supervisord's reach; sasldb written in place;
Health showing a socket file and a certificate Postfix does not serve; the
Origin check refusing every POST where Sec-Fetch-Site is missing; two
processes migrating at once; certbot's symlinks; the documents; and the
measurements. Roadmap 31/81.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 19:41:07 +03:00
mixandClaude Opus 5.5 599149bf83 build, deploy, inbound, dmarc, backup, e2e: cite the source of what they rely on (assumptions part C)
Comments only, no behaviour changed. Each main.cf/master.cf parameter the
build sets, the wrapper's and the reload hook's reliance on Postfix 3.7.11
(postfix-script, master.5, postconf.5, cleanup_init.c, smtpd_peer.c,
README_FILES), supervisor 4.2.5 (process.py, options.py, events.rst),
logrotate 3.21.0, OpenDKIM's config, Cyrus SASL's sasldb, rspamd 3.11.1
(milter_headers.lua, multimap, the map watcher), Docker Compose, CoreDNS
and certbot now say where that is written. Measurements already there stay
measurements ("date not recorded" where none was); hand-composed fixtures
say so.

Contradictions left for their own commits: C-P31 cleanup's size limit,
C-P42 master outside supervisor's signal, C-V5/V6 startsecs, C-V10,
C-S3/Q3 sasldb written in place, C-Q4/A1 certbot's symlinks, C-R7 rspamd
skips local clients but the signature routine does not, C-P8, C-R4, C-R12,
C-M7, C-E2, C-K8/K9, C-D1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 19:39:21 +03:00
mixandClaude Opus 5.5 ad3d834392 store, web: cite the source of what they rely on (assumptions part D)
Comments only, no behaviour changed. modernc.org/sqlite v1.60.1 (the
_pragma list on every connection, busy_timeout first; extended result
codes; a deferred BEGIN; multi-statement Exec) and SQLite 3.53.4's
documentation (transactions, WAL, foreign keys, NOCASE, upsert, julianday,
partial indexes); htmx 2.0.4 src/htmx.js; the Fetch and HTML standards for
Sec-Fetch-Site, Origin, SameSite, __Host-, CSP and defer order; Go's
html/template. Comments the source contradicts are corrected - the driver
does not serialise writes (SetMaxOpenConns(1) does), htmx:load fires per
swapped element, no-store does not keep a page out of history.

Left for their own commits: D-S3 a migration two processes can both run,
D-S20 recipient case, D-B3 Origin: null under Referrer-Policy: no-referrer,
D-H5/H6/H7/H9 (audit M11, stage 4).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 19:36:26 +03:00
mixandClaude Opus 5.5 b113e72c71 milter, postfix, app, domain: cite the source of what they rely on (assumptions part B)
Comments only, no behaviour changed. go-milter v0.4.1 (session.go: one
session per connection, replaced after a non-continue reply; macros per
stage), Postfix 3.7.11 (milter8.c, smtpd_check.c
reject_auth_sender_login_mismatch(), mail_addr_find.c, showq_compat.c,
mail_conf_time.c, mail_params.h), Cyrus SASL 2.1.28 (saslpasswd.c,
plugins/sasldb.c: deleting a missing login is not an error), OpenDKIM
2.11.0~beta2 (opendkim-db.c, opendkim-genkey), Berkeley DB 5.3 db_dump,
rspamd 3.11.0. Comments that claimed what the source denies are corrected
(sasldb keeps the password as given, not hashed; an embedded newline is
kept, not truncated; rspamd sorts symbols by absolute weight); fixtures say
seen or composed. The sender check's From: parsing gets no citations: it is
replaced by the port of OpenDKIM's dkim_mail_parse next.

Contradictions left for their own commits: B-G5 cleanAddress, B-PS1 first
key wins, B-PQ1 the forced-expire flag, B-PQ10/PQ11 Postfix's time units.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 19:36:12 +03:00
mixandClaude Opus 5.5 ce363f48dd logtail, health, panel: cite the source of what they rely on (assumptions part A)
Comments only, no behaviour changed. Each place the log tailer, Health and
cmd/panel rely on another program now says where that is written, for the
version the image runs: Postfix 3.7.11 (log_adhoc(), msg_logger_print(),
smtpd_whatsup(), cleanup_milter.c, qmgr_active.c, postlogd.c, showq.c,
pipe.c), logrotate 3.21.0, supervisor 4.2.5, proc(5). Measurements stay
measurements and say where and when; test fixtures say whether they were
seen or composed.

Where the source contradicts the code the comment does not bless it but
names the register entry (A-8 expiry, A-12 the ISO stamp no Postfix writes,
A-15 the last recipient, A-21/A-23 sasl_username, A-33 postlogd's old
inode, A-38 a socket file is not a listener, A-65 the certificate served);
those are fixed in their own commits.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 19:36:11 +03:00
mixandClaude Opus 5.5 a8b6276b0d audit: stage 2c - the inventory of what SelfPost assumes of other programs
test / test (push) Canceled after 0s
docs/assumptions-2026-10-14.md: four read-only readings of the tree at
0f984ff (log tailer/Health/cmd/panel; milters/postfix/app/domain;
build/deploy/inbound/DMARC/backup/e2e; store/web and the claims of
architecture, security and guide), each place rated by what justifies it
today. 621 entries: 3 cited, 25 measured, 535 guessed, 58 with a reason to
think them false - among them expiry (Postfix logs status=expired once per
message, without to=, so expired rows stay deferred), the sender check
accepting a From: any key lists while Postfix takes the first key found,
cleanup capping inbound mail at main.cf's 10240000 bytes, and the
milter-reject line carrying sasl_username after all.

Sources read: the Debian bookworm packages the image installs (apt-get
source), go-milter v0.4.1 and modernc v1.60.1 from the module cache.
Next: the answers, entry by entry.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 19:03:22 +03:00
mixandClaude Opus 5.5 811055d4c0 development: the rule's example cites the function Postfix really has
The example comment named check_sender_login() and strcasecmp; Postfix
3.7.11 compares the login in reject_auth_sender_login_mismatch() with
strcasecmp_utf8 (src/smtpd/smtpd_check.c:3913, 3936). Found by the
heuristics inventory - the rule's own example was written from memory.
Says where the Debian sources are fetched from and kept.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 18:58:04 +03:00
mixandClaude Opus 5.5 0f984ff8d8 development: code relying on another component cites its source; stage 2c: the heuristics review
test / test (push) Canceled after 0s
Owner, 2026-10-14: "нам нужно полное ревью кода на предмет всей этой
эвристики. Её не должно там, где можно посмотреть код и ответить точно и
однозначно", and "в коде таких наших блоков должно быть обоснование со
ссылкой на код компонента".

- development.md § Code that relies on another component's behaviour: the
  rule, with the form of the comment (component, the image's version, file,
  function; or a document's URL and section; or "measured on the stand,
  date" said as such), and fixtures saying whether they were seen or
  composed.
- Audit stage 2c: the line "what was taken on trust" becomes the heuristics
  review - an inventory over the whole tree written down as
  docs/assumptions-2026-10-14.md, then each answered from the source of the
  version the image runs, the comment added, the code fixed where it
  contradicts. Plan still 66 lines.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 18:44:19 +03:00
mixandClaude Opus 5.5 3c8628074f release: 2.5.1: audit stage 2b - the log tailer and what the panel shows of a refusal
test / test (push) Canceled after 0s
A log rotated while the panel was down is read to its end; the deferred
counter remembers a place in the log, not a second; a refused message's
page shows what Postfix wrote; a refusal the panel did not get to write is
written from mail.log; downtime is estimated (≈) when supervisord is
silent; the Sender check card counts connections and messages apart (the
owner's decision); a message the sweep closed says the panel closed it.
Plus the findings of both reviews of the stage.

Rolled out to the test server as 2.5.0-post.e1ddf12 (the same code):
migration 0009 ran on the live database - schema 9, the new columns in
place, 68/68 send-log rows still tied to their domain, integrity ok.

CHANGELOG cut; pin 2.5.1 in deploy/docker-compose.yml, README (with the
local-build note), guide; development.md "from pin"; roadmap row and the
recommended order; stage 2b marked shipped in the plan. No tag.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 18:43:20 +03:00
mixandClaude Opus 5.5 e1ddf12744 audit: stage 2c - what was taken on trust, read in Postfix's source
test / test (push) Canceled after 0s
Owner, 2026-10-14, after the rule "do not guess how code works when its
documentation or source can be read": "Есть ли ещё у нас места, где мы
угадывали?". The inventory found four: qmgr's expiry against the queue
lifetime (a margin in queued_over_lifetime), postlogd after a rename (the
rotation window), the stale smtpd after a reload (measured, not
understood), and the NOQUEUE line's fields (two fixtures carry a
sasl_username Postfix does not write, one presented as seen on the stand).
A line in stage 2c reads Postfix 3.7.11 for them, records what is read,
fixes the fixtures and the code where they disagree. Plan 66 lines.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 18:38:56 +03:00
mixandClaude Opus 5.5 af130e268a findings of the reviews of audit stage 2b
Technical review (Opus) and security review (Fable) of 5be109b..5154bba;
reports in the session's scratchpad. Nothing above low from security; one
bug and three risks from the technical review. Migration 0009 is not
deployed, so it was extended.

- Bug: two refusals of one second swapped their NOQUEUE lines (each line
  went to the newest lineless row). AttachRefusalLine now matches the
  client address too (send_log.client_ip, written by the journal milter -
  Postfix's line names the client and the sender, never the login: the
  sasl_username part of the pattern never matched and could be fed by HELO,
  security F1/F2), takes the nearest row in time and the oldest of equals,
  and a line already on a row goes on no other (security F6).
- Waiting lines and recoveries lived only in memory while the saved offset
  moved past them: a stop within 30 s lost them. The tracker now never saves
  an offset past the oldest waiting item in its file (tracker.hold,
  refusalLines.oldest) - for a recovery, the client= line that names its
  login - so they are read again; the "once" guards make that harmless.
- A recovered row whose login the tailer did not see was nobody's: its
  domain_id falls back to the envelope domain's.
- The fingerprint's file identity is the inode alone: the device number can
  change across a host reboot and make the same log look rotated.
- A refused row carries its queue id; UpdateStatus's "newest row" now skips
  refused rows, so a later message with that id does not flip it to sent.
- The alive mark: its age is read first and kept from then on, so asking
  supervisord for 30 s cannot make it stale; the entrypoint removes it with
  the sockets - a container restart is not the check's downtime, and a mark
  from before would have counted the whole outage as "≈".
- Security F3: the sender check prints no ";" of the client's in the From:
  text of its reply, and the tailer reads the envelope after the first ";"
  - Postfix's own text.
- A refused row with a queue id and no stored lines says so (it was told the
  mail.log text of 2.2.0); the fact's em dash is the house hyphen.
- Docs: architecture (client address, no login), guide § Health (warns on
  either count), help topics log and sender-check (design 1ca87f2).
- e2e: refusals_show_postfixs_line wants the refusal's own sender in the
  line; refusal_recovered_from_the_log waits past the tailer's 30 s and
  checks that a refusal the panel wrote got no recovered twin.

Not changed, written down: review 9 (mail.log.1 read to its end at that
moment; postlogd may still write to it until the reload), 10 (a log under
512 bytes has no fingerprint, so places fall back to time briefly after a
rotation), 6 (a 451 at MAIL FROM of another milter reads the same - the
client and sender must match too now), security F5/F12 (info). Review 12:
the refused and closed-by-the-panel variants of out-message were shown to
the owner as pictures of the real templates, not drawn in out-message.html:
the guard ties every drawn screen to a page of the engine, and a variant
has the page's outline.

Evidence on the test server for this tree: go vet clean, go test ./... 23
packages ok, make e2e exit 0 (refusal_recovered_from_the_log 70 s).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 18:38:22 +03:00
mixandClaude Opus 5.5 1ca87f2d1a design: help wording after the review of stage 2b
Topic log: a message's lines are found by queue id - and for a message
refused before it had one, Postfix's one line about the refusal (the
review: "found by queue id" no longer holds for MAIL FROM refusals).
Topic sender-check: the ≈ sentence said "at most what Postfix was up",
which reads backwards; it is the time since the panel's last sign of life,
"which may be more than Postfix was up". build.py, check.py: 0
violations. help.go follows in the next commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 18:28:52 +03:00
mixandClaude Opus 5.5 5dcff8c2b9 audit: stage 2c - the sender check reads From: as OpenDKIM does (owner's decision)
test / test (push) Canceled after 0s
Owner, 2026-10-14, after reading OpenDKIM's source together
(dkim_mail_parse, called in mlfi_eoh on the first From:; SenderHeaders is
unset in build/opendkim.conf): "Да" to porting the parser so that the check
decides by the address OpenDKIM will take. The scan of the display name and
comments for another sender of the server - a guard against two parsers
parting ways, and the reason L8 is half closed - then goes.

New stage 2c (three lines) after 2.5.1 and before stage 3: the port from
the version the image runs, checked against OpenDKIM on the stand with
generated lines, then docs and wording. Stages 3 and 4 move to 2.5.3 and
2.5.4 (development.md's planned cuts). Plan 65 lines, roadmap 30/65.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 18:19:34 +03:00
mixandClaude Opus 5.5 57142b21b5 security: L8's remaining probe accepted by the owner
test / test (push) Canceled after 0s
Owner, 2026-10-14, asked whether to refuse every foreign address in a
display name to close what L8 leaves (refused or passed still tells an
application which domains are hosted here): "Ответ 1" - leave it. The
probe is a trusted application of this server guessing names one at a
time, as it already had to before L8, when the reply said in words what the
line itself had named. security.md (Mail path and accepted risks, with the
trigger to revisit: untrusted tenants, or the display-name rule decided
anew) and the audit plan record it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 18:06:05 +03:00
mixandClaude Opus 5.5 5154bba61c docs: stage 2b of the audit - the log tailer, refusals, the Sender check card
test / test (push) Canceled after 0s
architecture.md
- § Log tailer: the fingerprint carries the inode; a log rotated while the
  panel was down is read from mail.log.1 to its end, or the gap is logged;
  a refusal's line (the queue id of a From: refusal, the NOQUEUE line put on
  a MAIL FROM refusal's row) and the refusal written from mail.log when the
  panel had not.
- § Sender check, what it costs: supervisord asked for half a minute, then
  the estimate and ≈; connections and messages in two columns; the count's
  saved place in the log.

guide.md
- § Outbound log: a refused message's page shows Postfix's line(s); rows
  from before, and a row written from mail.log, say so.
- § Overview and Health: the two columns and ≈.

schema-migrations row 9 was written with the migration. Audit plan line
ticked: stage 2b's lines are all done. Roadmap 30/62.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 17:19:05 +03:00
mixandClaude Opus 5.5 415334c2cc message page: a row the sweep closed says the panel closed it (stage 2b)
The History step of every bounced row said the receiving server refused
the message permanently or Postfix gave up, "the reason is in the delivery
log below" - for a row the reconcile sweep closed (reason "reconcile")
neither is known and the log has nothing (review of stage 1). Such a row's
step is now "Closed by the panel - Postfix no longer held the message and
mail.log never said what became of it ... rather than show a delivery it
cannot evidence." The help topic log names the case (design df96690).
Pictured for the owner with the real templates (dev/shots); the variant has
no mockup screen of its own, as the refused one.

Test: the page of a row the sweep closed says "Closed by the panel" and not
"the receiving server refused".

CHANGELOG [Unreleased], audit plan line ticked, roadmap 29/62.
Evidence: the message-page tests of ./internal/web/handlers ok locally; the
server run follows before the push.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 17:18:15 +03:00
mixandClaude Opus 5.5 ac985d03a8 sender check: connections and messages counted apart; downtime estimated when supervisord is silent (stage 2b)
Following the design commit df96690.

- Two columns (owner, 2026-10-14): sender_check_days.messages (migration
  0009, not deployed) holds the messages deferred mid-message, from
  cleanup's warning; deferred stays the connections smtpd turned away.
  AddSenderCheckDeferred takes which; the counter passes cleanup's as a
  message. The card has "Connections turned away" and "Messages deferred";
  it warns on either within 24 hours (one mark, as before).
- Supervisord silent at start: postfixStarted asks again every 2 s for 30 s
  (postfixState: started, running, known). With no answer the downtime is
  the time since the alive mark - an upper bound - booked with estimated = 1
  (sender_check_days.estimated, MAX over a day); the card prefixes such a
  period's time with "≈ " and its note says what ≈ means. Before, such a
  start counted nothing and the number the owner decides by read low.
- Help topic sender-check and the log topic word for word as the mockup
  (help.go).

Tests: store (connections and messages apart, a period with an estimated
day); logtail (cleanup's warnings are messages); cmd/panel (supervisord
silent: the time since the mark, estimated); view fixture and the Health
page at four columns with ≈; e2e reads the row's two counts (panel down:
connections; panel hung: messages). Schema at 0009 with the columns.

schema-migrations row 9, CHANGELOG [Unreleased], audit plan lines ticked
(two), roadmap 28/62.
Evidence: go vet ./... clean; go test of store, view, cmd/panel ok locally,
logtail but for the Windows-only test; the server run follows before the
push.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 17:16:02 +03:00
mixandClaude Opus 5.5 df96690ba6 design: the Sender check card counts connections and messages apart; ≈ for an estimate
Owner, 2026-10-14 ("Может просто показывать и то, и то и не смешивать?"):
the figure that was "Messages deferred" is two - connections Postfix turned
away while the panel did not answer (refused from the connection on, before
a message is offered), and messages deferred in the middle, when the panel
stopped answering after a message had begun (cleanup's case, counted since
the M13 fix). The card has a column for each.

The audit's stage 2b also marks a day whose downtime is an estimate - a
start at which supervisord could not say since when Postfix ran: such a
period's time is shown with ≈, and the text under the table says it is at
most what is shown.

Help: topic sender-check says what each column counts and what ≈ means;
topic log names the bounced row the panel closed itself (the sweep, when
Postfix no longer held the message and mail.log never said what became of
it). build.py, check.py: 0 violations, outlines unchanged. The code follows
in the next commits; until then the help text guard fails - not pushed in
between.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 17:11:23 +03:00
mixandClaude Opus 5.5 65d916f26a logtail: a refusal the panel did not get to write is written from mail.log (stage 2b)
test / test (push) Canceled after 0s
The sender check answers 550 5.7.1 and writes the refused row through a
buffer a moment later; a panel stopped in between (a rollout, a kill) left
an application refused and a log that did not know it. A refusal at MAIL
FROM needs nothing: the journal milter writes its row before it answers.

- logtail.refusalLines remembers each queue id's login from smtpd's
  "<id>: client=..., sasl_username=<login>" (4096 ids at most) and, on
  cleanup's "<id>: milter-reject: END-OF-MESSAGE ...: 5.7.1 From: <line> -
  <why>; from=<...> to=<...>", waits 30 s and hands it to
  store.RecoverRefusal - an INSERT ... WHERE NOT EXISTS a refused row of that
  queue id within two minutes: the row the panel wrote, or one recovered
  before, means nothing is written. It is read from the saved offset on, so
  a kill is covered as well as a clean stop - which a flush of the buffer at
  shutdown would not.
- Migration 0009 (not deployed yet) gains send_log.recovered; the row is
  written with the line's time, the login's domain id, reason "from", the
  From: line as the check printed it (to the last " - " before ";").
- The message page: a fact "Written: from mail.log after a restart: the
  panel stopped before it wrote the refusal".

Tests: store (not written when the panel wrote it; written once however
often); logtail (the login from client=, the From: line with a " - " in it,
nothing before the wait is over); view (the fact). e2e
refusal_recovered_from_the_log: a From: refusal, the panel stopped, its row
deleted and the tailer's saved place moved before the message's first line
(python3 in the container, as the panel's user - a kill cannot be timed to
that moment from outside); started again, the row is back, recovered, once.

Also fixed: handlers TestMessagePageOfARefusedMessage still expected the
old sentence; it failed on the server after e166a2e, whose run showed only
the schema failures - now it wants the new one.

schema-migrations row 9, CHANGELOG [Unreleased], audit plan line ticked,
roadmap 26/62.
Evidence on the test server for this tree but the handler-test fix: make e2e
exit 0 (the new case 37 s); go test 22/23 ok, the 23rd that test - fixed,
passes locally; the full run follows before the push.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 17:09:05 +03:00
mixandClaude Opus 5.5 e166a2ebd6 refused messages: the page shows what Postfix wrote (stage 2b, 0009)
The Delivery log box of every refused row said "This message never reached
the queue, so Postfix wrote no delivery lines for it" - untrue: Postfix
wrote a line for each. The owner, 2026-10-13: the box is required on every
message page. Pictured for him first (dev/shots, the refused variant of
out-message: wrong From with its queue id lines, rate-limited with the
NOQUEUE line); the variant has no mockup screen of its own - the guard ties
screens to pages, and its outline is out-message's.

- Wrong From: cleanup gives the message a queue id before the headers are
  judged; the sender check reads {i} at end-of-headers and writes it on the
  refused row. Its lines - stored under the id like any other's - are on the
  page, but only those from 10 minutes before to 1 minute after the refusal
  (store.SendLogLines for a rejected row): Postfix frees the id at once, and
  a later message with it lends nothing. No mail.log fallback for a refused
  row, for the same reason.
- Rate-limited and wrong client: refused at MAIL FROM, no queue id. Migration
  0009 adds send_log.refusal_line; logtail.refusalLines matches smtpd's
  "NOQUEUE: milter-reject: MAIL from ...; from=<...>" (not port 25's, not
  CONNECT/EHLO) and store.AttachRefusalLine puts it on the newest refused row
  of that sender (and login, when the line names it) within two minutes; a
  line whose row is not written yet waits up to 30 s (retried at each pass).
- The page: the queue id fact says "given, the message was not queued" or
  "none - refused at MAIL FROM, before one is given"; the box's caption is
  "The line from mail.log for this refusal" for the latter; a row from
  before says the line was not kept. LogLineLevel colours a refusal by its
  reply: 5xx red, 4xx amber (IPv6 clients included). The id in the fact is
  not muted as in the picture: the kit has no muted inline.

Tests: store (attach by sender, login, time; not a queued row, not one an
hour off; a refused row's lines are its own minutes); logtail (attach, wait
and retry, port 25 and other lines ignored); milter (the refused row carries
{i}); view (facts, caption, line levels); handlers (the line on the page,
the sentence without it). e2e refusals_show_postfixs_line: a client-address
refusal made there, and the pages of a wrong From, a rate-limited and a
wrong client refusal each show Postfix's line.

schema-migrations row 9, CHANGELOG [Unreleased], audit plan line ticked,
roadmap 25/62.
Evidence on the test server for this tree: make e2e exit 0 (the new case
in 5.4 s); go test ./... ok but for two schema tests, fixed after the run
(the schema at 0009) - store tests ok locally; the full run follows before
the push.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 16:56:29 +03:00
mixandClaude Opus 5.5 c452606175 logtail: the deferred counter remembers a place in the log, not a second (stage 2b)
test / test (push) Canceled after 0s
The counter passed over, while catching up after a start, every warning no
later than the last one counted - by time; mail.log stamps whole seconds,
so a second warning of that second, later in the file, was lost.

- The tailer gives each line its place (Place: the file's fingerprint and
  the offset just past the line): follow tracks the bytes the reader hands
  over, readRotated likewise in mail.log.1. Run hands tr.line to the
  counter.
- store.AddSenderCheckDeferred(at, n, place) counts, moves the
  last-deferral mark (never back - an upsert WHERE) and saves the place
  (settings sender_check_counted_place) in one transaction.
- Catching up, a warning in the file of the last one counted is passed over
  by place (end <= counted end); in another file - a rotated log read again
  - by time, as before. The fingerprint carries the inode, so the place
  stays true across a rotation.

Tests: two warnings of one second, an unclean stop between them, all lines
read again - each counted once; the store saves the place with the count
and keeps the latest deferral time. Existing counter tests pass a zero
place (time rule).

CHANGELOG [Unreleased], audit plan line ticked, roadmap 24/62.
Evidence: go vet ./... clean; go test ./internal/store, ./internal/logtail ok
(but for the Windows-only test); the server run follows before the push.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 16:28:13 +03:00
mixandClaude Opus 5.5 4678015bbd logtail: a log rotated while the panel was down is read to its end (stage 2b)
At start, a saved offset whose fingerprint was not the current file's meant
"start on the new file": what Postfix wrote between the panel's stop and the
rotation was never read - delivery statuses, stored lines, refusals, inbound
journal rows, the deferred counter.

- tracker.resume marks the case (behind, prev); follow calls readRotated
  before the new file: it opens mail.log.1 (delaycompress leaves it plain),
  checks by fingerprint that it is the file the place belongs to, reads it
  from the saved offset to its end through handle (catching up: the
  once-protections apply) and flushes the stored lines.
- Not found, not that file, or the offset past its end (rotated twice):
  logged, with the saved place's time - from when nothing was read.
- The fingerprint gains the file's identity (device.inode, fileid_unix.go;
  empty elsewhere): two logs that begin alike are two files - the
  review-2026-08-followups row on collisions. A fingerprint saved before
  (hash alone) still matches by its hash (sameLog), so the first start after
  the upgrade resumes where it was.
- store.LogtailState reads back updated_at.

Tests: stop, append, rotate, append, start - both halves read, once each,
the earlier line not again; a mail.log.1 that is another file is not read
from the place; sameLog with an old and a new fingerprint.

CHANGELOG [Unreleased], audit plan line ticked, roadmap 23/62.
Evidence: GOOS=linux go vet ./internal/logtail clean; go test ./internal/logtail
ok on Windows but for TestFollowTailsAndRotates; the server run follows
before the push.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 16:25:29 +03:00
mixandClaude Opus 5.5 5be109b871 release: 2.5.0: audit stages 1 and 2 - the delivery log and the tenant boundary
test / test (push) Canceled after 0s
Stage 1: a cut queue listing no longer bounces held mail; the reconcile
sweep closes only queued rows, by id, marked "reconcile", and its fate is
measured (delivery_log_days, migration 0007); delivery lines are read at
the head of the line; a renewed TLS certificate is served without a
restart; Health counts the message a hung panel deferred.
Stage 2: logins folded to lower case and unique without regard to case;
the last global administrator kept inside the transaction, users saved
whole; a domain import held to the forms' rules; log rows by domain id
(migration 0008); the 550 reply names no other domain; which key OpenDKIM
signs with, tested. Plus the findings of both reviews of each stage.

Rolled out to the test server as 2.4.2-post.ef18ef2 (the same code):
migrations 0007 and 0008 ran on the live database - schema 8, 68/68 send-log
and 10/10 inbound rows tied to their domain, integrity ok.

CHANGELOG cut; pin 2.5.0 in deploy/docker-compose.yml, README (with the
local-build note), guide; development.md "from pin"; roadmap row and the
recommended order; stages 1 and 2 marked shipped in the plan. No tag (the
owner's command only).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 16:22:04 +03:00
mixandClaude Opus 5.5 ef18ef21a0 findings of the reviews of audit stage 2
test / test (push) Canceled after 0s
Technical review (Opus) and security review (Fable) of a59e3d4..cb5a4a2;
reports in the session's scratchpad. Nothing critical, high or medium; no
outright bug. Fixed:

- F2, F3 (0008 backfill, before it runs on the live database): rows are
  filled by the rule new rows follow - the application's domain, then the
  name's - and only by a domain and application that existed when the row
  was written (created_at); the inbound log likewise. Test: the migration's
  UPDATEs re-run on prepared rows (before the domain, after, null sender,
  application gone, a stranger's name). Dry run on a copy of the live
  database: 68/68 and 10/10 rows filled, none disagreeing with its login's
  domain, integrity ok.
- V2, F4: an application's statistics and level-2 count are its login's
  rows of its domain (appByLogin), and StatsWindow starts no earlier than
  the entity's creation; the message page's "may send as" only for a login
  still of the row's domain. Test: a login taken again in another domain
  starts at zero.
- F7: MAIL FROM:<> is held to the domain limit of its login
  (ApplicationByLogin fills DomainName).
- V3: an imported DKIM selector is held to DNS labels (validate.Selector).
- F5, F6: an imported auto limit's numbers are dropped and recalculated
  right after the import; its mode is stored trimmed.
- V6: the store refuses a role that is neither global nor domain.
- F9: SaveUser keeps the stored hash when no new password is given (the
  form no longer writes back the hash it read). F10: a reach naming a
  domain deleted meanwhile is ErrNoSuchDomain, answered on the form.
- V4, F12: from-lines-other.txt - lines naming another sender ({other}) in
  a quoted string, a quoted pair, capitals, a comment, an encoded word, a
  quoted local part; the unit test wants each refused, the e2e
  (lines_naming_another_sender_never_signed_for_it) wants each refused and
  none signed for it. The passed list gained a.{DOMAIN}, a comment after
  the angle address, a quoted local part, and "b@ {other}" - passed by
  design (security.md), and the e2e shows it signed for the own domain.
- F13: e2e login_in_other_capitals_does_not_authenticate - AUTH with the
  login upper-cased gets 535.

Not changed, written down: V1/F1 - L8 is half closed (refused or passed
still tells an application which domains are hosted here); security.md's
accepted risks and the CHANGELOG say so, and the owner is asked whether to
refuse every foreign address in a display name (changes the 2.4.0 rule).
F11 - deleting a domain rewrites its rows' domain_id (seconds at 1e5 rows),
accepted at this size. F8 (test cannot tell write-first from read-first;
the reviewer verified it with two stores, 30/30), V5, V7, P2: noted.

Evidence on the test server for this tree: go vet clean, go test ./... 23
packages ok, make e2e exit 0 with the three new cases.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 16:17:57 +03:00
mixandClaude Opus 5.5 8b53d63160 audit: the owner's decision on the Sender check columns (stage 2b)
test / test (push) Canceled after 0s
Owner, 2026-10-14: show connections turned away and messages deferred
mid-message apart, not mixed. The stage 2b step now says: smtpd's warning
to the existing column (Connections turned away), cleanup's to a new column
of sender_check_days in the stage's migration (Messages deferred), both on
the card, the help topic saying what each counts; mockup first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 15:57:53 +03:00
mixandClaude Opus 5.5 cb5a4a22e1 docs: stage 2 of the audit - roles, the guard, logs by id, import, logins
test / test (push) Canceled after 0s
security.md
- § Roles: the last-global guard is inside the transaction that deletes or
  demotes (write, then count); setup is one statement; a user is saved in
  one transaction. A log row, outbound or inbound, belongs to a domain by
  id; a domain added again starts with an empty log. The Inbound › Log row of
  the table says by id.
- § Mail path, sender check: the agreement of the check and OpenDKIM is
  tested (the shared list of passed From: lines); the refusal does not say
  which other sender a line names.
- Accepted risks: "a row of the inbound log belongs to a domain by name"
  struck (L12).

guide.md
- Domain export and import: the file is checked like the forms before
  anything is created; a login keeps its case and a case clash is refused.
- The application form: a new login is stored in lower case.

schema-migrations.md was updated with 0008 in its commits. Audit plan line
ticked; stage 2's lines are all done. Roadmap 22/62.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 15:45:46 +03:00
mixandClaude Opus 5.5 5747f0b25c sender check: the reply names no other domain (L8); which key signs, tested (stage 2)
L8: a From: line naming another sender of the server was refused with
"it names another sender of this server, <domain>" - a confirmation, to
whoever tries names, that the domain is hosted here. The reason now stops
before the domain; the line as written stays in the reply (the client's own
text) and on the refused message's page (header_from). Unit test: the reason
of every refusal names none of the server's other domains; e2e: the reason
of each sender-check refusal does not contain other.e2e.test.

Which key OpenDKIM signs with (owner, 2026-10-13): the check and OpenDKIM
read From: with two parsers, and their agreement was argued, never run.
internal/milter/testdata/from-lines-passed.txt holds the lines the check
passes for an application - bare and bracketed, quoted names with a comma,
escaped quotes and an address inside, an extension, three encoded charsets,
two folded lines, comments (nested too), a stranger's address in the name -
as Go-quoted strings with {domain}. The unit test wants each passed; the new
e2e case passed_from_lines_signed_for_their_domain_only sends each as app1
and wants every DKIM signature at the sink to be for the sender domain,
with one valid. All fifteen were. "(the shop) a@domain" - a comment before
the address, valid RFC 5322 - is refused by the check ("it must be one
address"); strict, not a hole, so it is not in the file. Groups are refused
too.

Audit plan lines ticked (two) with what was found, CHANGELOG [Unreleased],
roadmap 21/62.
Evidence on the test server for this tree (with a618ffb): go vet clean, go
test ./... 23 packages ok, make e2e exit 0 - the new case passed in 4 s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 15:44:57 +03:00
mixandClaude Opus 5.5 a618ffb0e4 send log: rows belong to a domain by id (audit L12, 0008)
A domain user's log, the message page, a domain's statistics, the level-2
count and the notifier's refusal count picked send-log rows by domain name,
so a domain deleted and re-added under its old name showed its earlier mail
to the next assignee and went on counting it.

Migration 0008 (not run on the live database yet, so extended here):
send_log.domain_id and inbound_spam_log.inbound_domain_id, REFERENCES ...
ON DELETE SET NULL, filled once by name from the registry as it is at
migration time, each with an (id, created_at) index.

- Writes: InsertQueued / InsertRejected set domain_id to the domain of the
  row's login (loginDomainID) - the journal milter and the sender check's
  refusal rows both go through them; the inbound journal sets
  inbound_domain_id from the inbound domain's name at insert.
- Reads: SendLogFilter.DomainIDs (was Domains, by name) and
  InboundSpamFilter.DomainIDs are the scope; the global role still reads
  everything. The message page checks row.DomainID against the user's
  domains. DomainSendStats and CountMessages keep their name argument and
  select domain_id = (SELECT id FROM domains WHERE name = ?). The notifier
  joins send_log.domain_id to the domain and its application.
- DMARC reports stay by name on purpose (a report is about the DNS name).

Tests: store - scope by ids; a re-added domain's log, level-2 count and
statistics hold its own row only, the global log both; inbound scope by
ids. Handlers - the new assignee gets 404 for the old row's page and does
not see it in the log, the global role sees both. Tests that wrote rows for
a login that is no application now register it (a row is a domain's through
its application). Schema at 0008 with the two columns and indexes.

schema-migrations row 8, CHANGELOG [Unreleased], audit plan line ticked,
roadmap 19/62.
Evidence: go vet ./... clean; go test ./... ok locally but for the two
Windows-only tests; the server run follows before the push.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 15:37:26 +03:00
mixandClaude Opus 5.5 e469034477 import: the export file held to the forms' rules before anything is created (audit M10)
test / test (push) Canceled after 0s
domain.Import trusted the file for everything but the domain name, the
selector and the password: DMARCRua was not validate.Email'ed, client IPs
not parsed, a rate limit's mode, window and count not checked - a window of
0 counted nothing, a garbage IP made the application refuse every client.

handlers.checkDomainImport runs in HandleImportDomain after the domain name
check and before Import, and refuses the file with "Invalid <what> in export
file: <the form's own text>": the report address (validate.Email), each
rate limit (parseRateLimitMode; auto - checkAutoMultiplier, 0 meaning the
default as an empty field does; manual - checkManualLimit, which the form
now uses too: count > 0, <= the level-1 backstop, window > 0; both 0 is an
export without a limit of its own), each application's client addresses
and the legacy ones on its rate limit (parseIPList, written back as the
form stores them; allow-list on with none is refused). Login, sending
addresses and password stay checked in app.ImportApplication.

Tests: a good export passes and its addresses come out normalised; nine
broken fields each refused with their message; a refused file posted to the
handler creates no domain.

CHANGELOG [Unreleased], audit plan line ticked, roadmap 18/62.
Evidence: go vet clean; the import, rate-limit and application tests of
./internal/web/handlers ok; the server run follows before the push.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 15:28:29 +03:00
mixandClaude Opus 5.5 f9e75de8d1 store: users saved in one transaction, the last global kept inside it (audit M7, M9)
M7: DeleteUser counted the global users and then deleted; the handler's
demotion and CreateGlobalUser checked and then wrote. MaxOpenConns(1)
serialises statements, not pairs: two globals deleting or demoting each
other at once could leave none. Now:
- DeleteUser: a transaction that deletes first (taking the write lock) and
  then requires a global left (requireAGlobal) - ErrLastGlobal rolls back.
- SaveUser: likewise for a demotion, the count inside the transaction after
  the write. The handler's early count stays only to answer before hashing a
  password; SaveUser decides, and ErrLastGlobal maps to the same message.
- CreateGlobalUser: INSERT ... SELECT ... WHERE NOT EXISTS (SELECT 1 FROM
  users), one statement.

M9: submitUserUpdate made five writes in a row (UpdateUser, sessions,
SetUserRole, ClearUserReach with its error only logged, SetUserReach).
store.SaveUser(UserChange) writes name, hash, e-mail, role and reach in one
transaction (a global user's reach cleared); sessions end after it commits.
The create path had the same flaw - user, reach and then e-mail in three
writes - and now goes through AddUser, one transaction; CreateUser stays as
a wrapper for its many test callers. SetUserRole is gone; writeUserReach is
the reach write inside a transaction.

Tests: store - two globals demoting, and deleting, each other from two
goroutines: exactly one refused, one global left; a refused SaveUser (taken
name, a reach to no domain, nothing assigned) changes nothing; promoted to
global loses the reach; AddUser with a bad reach leaves no user; four setups
at once create one user. Handlers (the users pages, direct): the whole
change saved, back to domain with a reach; the last global not demoted and
the typed rename not written; a domain user with nothing assigned refused,
nothing changed; create with e-mail and reach.

CHANGELOG [Unreleased], audit plan lines ticked (two), roadmap 17/62.
Evidence: go vet ./... clean; go test ./internal/store ok; the users tests of
./internal/web/handlers ok; the server run follows before the push.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 15:25:41 +03:00
mixandClaude Opus 5.5 60c7b85f84 findings of the reviews of audit stage 1
test / test (push) Canceled after 0s
Technical review (Opus) and security review (Fable) of 9efc9c7..a59e3d4;
reports in the session's scratchpad. Fixed:

- B1 (verified bug): store.UpdateStatus read before writing in its
  transaction; a commit by another connection in between (the dmarc-ingest
  process opens the same file) failed it at once with "database is locked"
  and the delivery line was lost. The first statement is now a write.
- R3: a delivery line updated every row of its queue id and recipient; it
  now updates only the newest (Postfix reuses short ids). The sweep closes
  by row id (QueuedDelivery.ID, CloseUnreported(id)).
- R4: a contradicted sweep keeps a mark, reason "reconcile-contradicted",
  instead of losing which row it was.
- R1: contradicted cannot see a lost line (never read late). New counter
  reconcile_missed: at close, the sweep looks for the row's delivery line
  in mail.log and mail.log.1 (QueueLines). 0007 edited - it has run on no
  database but throwaway stands.
- R2: queued_over_lifetime cannot measure "what removing the sweep would
  leave" while the sweep runs; its text now says rows stuck despite the
  sweep, and the rule uses closed - missed - contradicted.
- cert-reload (R5, R6, R7, security medium/low): both files copied to .new
  and the copies checked as a pair before both are renamed; acts on the copy
  only when postconf says Postfix reads it; a failed reload is retried next
  pass; umask 077; says so when the mounted files cannot be read. The e2e
  case asserts Postfix reads the copy, the copy equals the mount, the key is
  600 root.
- Security low: a recipient with a quoted local part - the journal cut
  "a b"@x at the space, deliveryRe cut "a>b"@x at the ">" - matched no
  delivery line and was closed by the sweep. cleanAddress ends a bracketed
  address at the first ">" outside quotes; deliveryRe takes the recipient
  up to the ">, " before relay= (test fixtures given the relay= every real
  delivery line has).
- Security low: postqueue -p's error carries at most 512 bytes of output.
- Nits: addDeliveryLogDay refuses a column not on its list; the truncation
  test checks its message.
- To stage 2b (page wording, mockup-first): the history step of a bounced
  row says the receiving server refused it, false for a row the sweep
  closed. To the owner when stage 2b is reached: since M13 the "deferred"
  figure counts whole messages too, which "Connections turned away" would
  not name.

Found while verifying: the e2e's main positive path failed once with "553
not owned by user app1" - an smtpd idle from before the panel's reload took
the connection in the reload's second. Measured on the stand (one-off
probe, not committed): sending 0, 10, 40, 110 s after the save (two seconds
at the soonest) always met an smtpd started after the reload. The e2e now
waits for the sender map after each application it creates
(waitForSenderMap: AUTH, MAIL, RCPT, RSET - no message, no row), and the
level-1 case uses it instead of its own loop.

Docs: architecture § Queue reconcile, schema-migrations (rows 7 and 8,
chain head 8), CHANGELOG, audit plan (stage 1 note, the measured window,
a new stage 2b line: 62 lines), roadmap 15/62.

Evidence on the test server for this tree: go vet clean, go test ./... 23
packages ok, make e2e exit 0 (renewed certificate, hung panel, level 1 at
49/60).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 15:21:52 +03:00
mixandClaude Opus 5.5 f51f5db245 app: logins folded to lower case, unique without regard to case (audit H4, 0008)
Postfix's reject_sender_login_mismatch compares the SASL login with the
sender map without regard to case - seen on the e2e stand this time (a
one-off probe, not committed: map rewritten to "APP1", client authenticated
as "app1", RCPT accepted). The panel stored logins as typed under a BINARY
UNIQUE, so "Ops" and "ops" were two applications to the panel and one sender
to Postfix: a tenant could send with another tenant's envelope sender.

- app.foldLogin: Service.Create lower-cases the login (owner: fold, do not
  refuse); CreateWithSettings rolls back the stored (folded) login.
- ImportApplication keeps the login as written: folding it would rename the
  SASL account and break the imported application's working credentials;
  the index below refuses a case clash there too.
- Migration 0008 (DDL): CREATE UNIQUE INDEX idx_applications_login_nocase ON
  applications (login COLLATE NOCASE). The column's own BINARY UNIQUE stays
  (SQLite cannot change a collation without a table rebuild, which a
  migration transaction cannot do with foreign keys on). Checked on the live
  installation first: 4 logins, no case clash; "Quieter3C" keeps its case.
  0008 is not run anywhere yet; send_log.domain_id joins it later in stage 2.

Tests: "Ops" created as ops, SASL account and map binding in lower case,
OPS in another domain and an imported oPs refused; the store refuses ops
beside Ops; schema at user_version 8 with the index.

CHANGELOG [Unreleased], audit plan line ticked, roadmap 15/61.
Evidence on the test server for this tree (with the probe): go vet clean,
go test ./... 23 packages ok, make e2e exit 0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 14:55:24 +03:00
mixandClaude Opus 5.5 a59e3d4d61 docs: stage 1 of the audit - log tailer, reconcile, sender check, guide
test / test (push) Canceled after 0s
architecture.md
- § Log tailer: the rotation drain does not close the gap entirely
  (postlogd writes to the renamed file until the reload reaches it); the fix
  waits for the measurement. Delivery lines read at the head of the line
  (M12).
- § Queue reconcile: reason "reconcile", the 4 MiB cap, the conditional
  close; the measurement (delivery_log_days, its four counters) and the
  decision rule written before the data.
- § Sender check, what it costs: cleanup meets a check that hangs mid-message,
  with the warning seen on the stand; counted once per message.

guide.md
- § Outbound log: `bounced` is the panel's word when Postfix no longer holds
  a message and the log never said; it is counted.
- § Overview and Health, Sender check: the message a hung panel deferred is
  counted too.

Audit plan line ticked, roadmap 14/61. Stage 1's code items are done
(the two deferred tailer fixes wait for the data, as planned).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 14:42:02 +03:00
mixandClaude Opus 5.5 db36aa2c0e logtail: count the message a hung sender check defers in cleanup (audit M13)
Seen on the e2e stand, the panel frozen with SIGSTOP once DATA was
answered: the client gets 451 4.7.1 after 30 s, and the warning that names
the socket is cleanup's, not smtpd's:

  postfix/cleanup[1339]: warning: milter unix:/run/selfpost/sender.sock:
    can't read SMFIC_HEADER reply packet header: Connection timed out
  postfix/cleanup[1339]: 7BCAF3C75DA0: milter-reject: END-OF-MESSAGE ...
    4.7.1 Service unavailable - try again later; ...

The counter read smtpd lines only, so Health's "messages deferred" missed
the one case that tells a hung panel from a stopped one. milterLineRe now
takes smtpd and cleanup; cleanup's "milter unix:<socket>:" warning counts a
message each time (one cleanup process serves message after message, and the
first warning ends the message); smtpd keeps once per connection. The
composed fixture is replaced by the seen lines, with two messages through one
cleanup process and OpenDKIM hung in cleanup (not counted).

e2e panel_hung_mid_message_is_counted: freezes the panel after DATA's 354,
wants a 4xx at the end of DATA, logs what mail.log says, and waits for
Health's today count to grow. kill through sh -c: the image has no kill(1).

CHANGELOG [Unreleased], audit plan line ticked, roadmap 13/61.
Evidence on the test server for this tree (with the next commit's docs):
go vet clean, go test ./... 23 packages ok, make e2e exit 0 -
renewed_certificate_picked_up 1.2 s, panel_hung_mid_message_is_counted
31.7 s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 14:42:01 +03:00
mixandClaude Opus 5.5 7af37654a1 cert-reload: renew Postfix's copy of a renewed certificate (audit H2)
postfix-config.sh copies the mounted certificate and key into
/etc/postfix/tls-internal at start (root-owned, key 0600) and Postfix reads
the copy; cert-reload only ran `postfix reload` daily, which re-read the
stale copy, so a certificate renewed while the container ran expired on
465/587 with the old one.

postfix-cert-reload.sh now, every TLS_RELOAD_INTERVAL_SECONDS: compares the
mount with the copy (cmp); when they differ and the certificate and key
belong together (openssl public keys equal - a renewer writes them one
after the other), writes each file beside its target, root-owned with the
start's modes, renames it over the target, and reloads Postfix. Nothing
changed - no reload. No copy (files missing at start) - the old periodic
reload. Never exits non-zero.

e2e renewed_certificate_picked_up: writes a new certificate into the
stand's mount and waits (30 s) for port 465 to present its serial;
compose.override.yml sets TLS_RELOAD_INTERVAL_SECONDS=2.
supervisord.conf comment, architecture.md (process table, § Postfix TLS
certs, the env knob), guide (after the proxies: picked up within a day).
CHANGELOG [Unreleased], audit plan line ticked, roadmap 12/61.

Evidence: GOOS=linux go vet in test/e2e clean; the server run of the stage
follows before the push.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 14:25:20 +03:00
mixandClaude Opus 5.5 63520f1678 logtail: deliveryRe anchored at the head of the line (audit M12)
The pattern that writes a row's status matched anywhere in a line: a
port-25 reject line ending helo=<ID: to=<victim@...>, status=sent> parsed
as (ID, victim, sent). parseDelivery now takes the stamp off with
SplitTimestamp and matches `^host program[pid]: <id>: to=<...>, ...
status=`, as queueLineRe and the inbound patterns do.

Tests: the probed HELO line, the same text in a quoted sender on a qmgr
line, the text after an smtpd line's queue id - none parse; a syslog-stamped
line as Postfix 3.7 writes it parses. The existing cases unchanged.

CHANGELOG [Unreleased], audit plan line ticked, roadmap 11/61.
Evidence: go test ./internal/logtail ok but for TestFollowTailsAndRotates
(Windows); the server run of the stage follows before the push.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 14:22:03 +03:00
mixandClaude Opus 5.5 ffefa9d6df delivery log: measure the sweep before deciding its fate (audit H3, 0007)
Migration 0007 (DDL only): delivery_log_days, a row per UTC day, kept a
year, shown nowhere - read off a copy of the database after a month by the
decision rule written in the audit plan.

- reconcile_closed: the sweep adds the rows it actually closed.
- reconcile_contradicted: store.UpdateStatus, in one transaction, counts the
  rows with reason "reconcile" a delivery line now speaks of, writes the
  log's status and clears the reason - so a re-read line counts once.
- tailer_blind_seconds: follow() tracks stretches in which the file is gone
  or reopening fails (logtail.blindness); stored when a stretch ends and
  every minute while it lasts; under 3 s not counted (the rotation's tick
  between rename and create loses nothing).
- queued_over_lifetime: every sweep overwrites the day's level with the rows
  queued past LongestLifetime + MaximalBackoff (a message Postfix still
  works on stays queued until its first retry after the lifetime).
- retentionLoop prunes days older than a year.

Tests: store (sums per day, the level overwritten, under a second nothing,
prune; a contradicted sweep counted once and its reason cleared; count of
queued rows); logtail (the sweep counts what it closed; countStuck records
the level; blindness - a one-tick gap not counted, 75 s stored as 60 + 15
from where the last part ended). Schema tests at user_version 7.

schema-migrations.md: chain head 7 (it still said 5 since 2.4.0), row 7.
CHANGELOG [Unreleased], audit plan line ticked, roadmap 10/61.
Evidence: go vet ./... clean; go test ./internal/store ok, ./internal/logtail
ok but for TestFollowTailsAndRotates (Windows); the server run of the stage
follows before the push.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 14:20:44 +03:00
mixandClaude Opus 5.5 9adb44845c logtail: the sweep closes only queued rows, with reason "reconcile" (audit H3)
reconcile() closed a row through UpdateStatus (WHERE queue_id AND to_addr),
which could overwrite a "sent" the tailer wrote between the listing and the
close, and rewrite an old row of a reused queue id. store.CloseUnreported
sets status bounced and reason "reconcile" (store.ReasonReconcile; the
0006 column, no schema change) WHERE ... AND status = 'queued'; the sweep
counts rows actually closed.

Tests: store - a queued row is closed with the reason, a sent row is left
alone (0 rows); logtail - the sweep's close goes through CloseUnreported.

CHANGELOG [Unreleased], audit plan line ticked, roadmap 9/61.
Evidence: go vet ./... clean; go test ./internal/store ok, ./internal/logtail
ok but for TestFollowTailsAndRotates (Windows rename of an open file, fails
here always); the server run of the stage follows before the push.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 14:16:26 +03:00
mixandClaude Opus 5.5 dfe10f6c27 postfix: a truncated postqueue -p listing is no listing for the sweep (audit H3)
QueueIDs parsed Queue()'s text, which caps postqueue -p at 4 MiB and
appends a note, and returned the ids before the cut with no error: the
reconcile sweep then closed every queued row past the cut as bounced while
Postfix still held the mail. listQueue() now returns the text and whether
it was cut; QueueIDs refuses a cut listing with an error (reconcile already
treats an error as no information), the way postqueueJSON does for the
notifier. Queue() keeps showing the cut listing with its note.

postqueueP is a var so tests run a fake postqueue (the test binary as a
helper process): 5 MiB -> QueueIDs errors, Queue ends with the note; 1000
bytes -> ids read.

CHANGELOG [Unreleased], audit plan line ticked, roadmap 8/61.
Evidence: go test ./internal/postfix ok (Windows); the server run of the
stage follows before the push.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 14:13:34 +03:00
mixandClaude Opus 5.5 9efc9c7005 release: 2.4.2: audit stage 0, release hygiene
test / test (push) Canceled after 0s
Stage 0 of docs/audit-2026-10-12.md, 7/7: the release image says which
release it is (H1), the e2e module is compiled on every push (M20),
test/ is out of the build context (L33), the documents say the 2.x image
is not published and show the local build (M6), dependencies raised,
staticcheck clean, the negative e2e cases held to their own refusals
(which showed the level-1 rate-limit case had never seen the limit).

CHANGELOG cut; pin 2.4.2 in deploy/docker-compose.yml, README (with the
local-build note), guide; development.md "from pin" and the planned cuts
of the audit's stages (2.5.0 ... 2.6.0); roadmap row in progress and the
recommended order; stage 0 marked shipped in the plan. No tag (the
owner's command only).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 14:11:25 +03:00
mixandClaude Opus 5.5 96425dd515 e2e: a negative case passes only on its own refusal (audit stage 0)
sendResult.refusedAt(stage, code, text) and replyIs(err, code, text):
nil only for the server's reply with that code and text at that stage;
a reset, a timeout or another reply is described for the failure.

- level 1: only anvil's "450 4.7.1 Error: too much mail from", only after
  a message of the loop was accepted. Held to that, it failed twice on
  the stand, each time for a real reason:
  1. waitForSMTPSReady took a TCP connect for readiness; Docker's port
     proxy accepts before Postfix listens and then resets - the reset
     once taken for the limit. It now waits for the TLS 220 greeting.
  2. the first message then got "553 ... not owned by user l1app": an
     smtpd idle from before the panel's postfix reload took the
     connection with the old sender_login_maps. The level-1 limit had
     never been what the case saw. The first message now waits (15 s at
     most) for the map, failing at once on any other refusal.
  Now: "level-1 limit tripped on attempt 51/60: 450 4.7.1 Error: too
  much mail from 10.77.0.1".
- level 2: 451 4.7.1 at MAIL, and both MAIL FROM of one connection.
- sender mismatch: 553 at RCPT, "not owned by user".
- no AUTH and the relay case: 554 "Client host rejected: Access
  denied" at RCPT; the relay case's comment claimed
  reject_unauth_destination, which on 465 comes after the client
  restriction - port 25 holds "554 Relay access denied".
- port 25: 550 5.1.1 for unknown recipients and report addresses.
- panel down: a 4xx reply (textproto), not just text starting with 4.

The product side (a second after an application is saved when its first
message can get 553) is written in the audit plan, not changed. Audit plan
line ticked with what was found, roadmap 7/61. No CHANGELOG line: tests only.

Evidence: make e2e on the test server for this tree, exit 0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 14:10:47 +03:00
mixandClaude Opus 5.5 8b9798884a staticcheck: the four items of the audit (stage 0)
test / test (push) Canceled after 0s
- handlers/authz.go: domainIDSet removed, nothing called it (U1000).
- handlers_outbound_pages_test.go: two refusals whose status code was
  assigned and never read (SA4006) now check it is 400 - a custom DMARC
  report mode without an address, export passwords that differ.
- view/pages_test.go: !strings.Contains for a missing box (S1003).
- Audit plan line ticked, roadmap 6/61. No CHANGELOG line: nothing an
  operator sees changes.

staticcheck ./... clean afterwards. Evidence on the test server: see the
previous commit (same tree run).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 13:55:12 +03:00
mixandClaude Opus 5.5 80bab92c8c deps: modernc.org/sqlite 1.60.1, x/crypto 0.58.0, x/sys 0.49.0 (audit stage 0)
- go get ...@latest + go mod tidy: sqlite 1.53.0 -> 1.60.1 (libc
  1.77.1, memory 1.12.1), x/crypto 0.54.0 -> 0.58.0, x/sys 0.47.0 ->
  0.49.0, go-isatty 0.0.24. The go line reads 1.26.0 (tidy's spelling of
  1.26); no dependency asks for a newer Go.
- development.md: golang:1.26-bookworm floats to the newest patch, and
  the standard library's security fixes arrive that way - rebuild the image
  after each Go patch release; dependencies are raised by hand.
- CHANGELOG [Unreleased], audit plan line ticked, roadmap 5/61.

Evidence on the test server (Go 1.26.5) for this tree plus the next
commit's staticcheck fixes: go vet clean, go test ./... 23 packages ok,
make e2e exit 0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 13:55:12 +03:00
mixandClaude Opus 5.5 937b41ddb7 docs: say the 2.x image is not published; show the local build (audit M6)
README, guide and deploy/docker-compose.yml pinned
ghcr.io/mixeme/selfpost:2.4.1, which does not exist: ghcr.io holds 1.0.0
and 1.3.0 only. The owner decided (2026-10-12) that old tags and images
stay untouched and the pin points forward, made true by the next
publication. Until then:

- README Quick start and guide Fixed image tag say the 2.x line is not on
  ghcr.io and show `docker build ... --build-arg VERSION=2.4.1 -t
  ghcr.io/mixeme/selfpost:2.4.1 .`; with the image present locally,
  docker run and Compose (pull_policy missing) use it without pulling.
- compose header says the same in two lines.
- development.md release steps: bump the version in those notes at each
  cut, remove them at the first publication of the line.
- CHANGELOG [Unreleased], audit plan line ticked, roadmap 4/61.

Documents only; no test reads the pin.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 13:42:53 +03:00
mixandClaude Opus 5.5 cc2ca3c968 ci: compile the e2e on every push; test/ out of the build context (audit M20, L33)
test / test (push) Canceled after 0s
- test.yml: after go test, `go vet ./...` and `go test -c -o /dev/null .`
  in test/e2e - compiled and linked, not run. Not `go test -run xxx` as
  the plan said: TestMain would still run and bring up the stand. The Go
  cache now keys on test/e2e/go.sum too.
- .dockerignore: test (e2e leftovers in test/e2e/.stage - DKIM keys,
  sasldb2 - no longer reach the builder; the sink image is built from
  test/e2e/sink as its own context, so it is unaffected).
- development.md (test.yml line), CHANGELOG [Unreleased], audit plan
  lines ticked, roadmap 3/61.

Evidence on the test server for this tree: make e2e (no
SELFPOST_E2E_VERSION, so the "e2e" default path) exit 0, including
image_stamped_with_version and the logrotate-mode build; in test/e2e
go vet ./... and go test -c -o /dev/null . clean; docker build of
build/Dockerfile with a file under test/e2e/.stage succeeds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 13:24:48 +03:00
mixandClaude Opus 5.5 70b4eb5585 release: stamp the image with the release's version (audit H1)
test / test (push) Canceled after 0s
The e2e gate rebuilt selfpost:e2e through compose.override.yml with its
own VERSION=e2e, and that rebuild is the image release.yml pushed: every
published image printed "e2e", and the backup/restore version check
compared a backup's version with "e2e".

- compose.override.yml: VERSION: ${SELFPOST_E2E_VERSION:-e2e}.
- release.yml: the e2e step gets SELFPOST_E2E_VERSION = the release's
  X.Y.Z; a new step before the push checks panel -version and
  selfpost-backup -version of selfpost:e2e against it.
- e2e: image_stamped_with_version checks both binaries against
  expectedVersion() (the variable, or "e2e" when unset).
- development.md, CHANGELOG [Unreleased], audit plan line ticked,
  roadmap 1/61.

Evidence: make e2e on the test server for this tree with
SELFPOST_E2E_VERSION=2.4.2-stamp - exit 0, TestE2E/image_stamped_with_version
PASS, and `docker run --rm --entrypoint <bin> selfpost:e2e -version` prints
2.4.2-stamp for panel and selfpost-backup. GOOS=linux go vet in test/e2e
clean. Published images are left as they are.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 13:18:20 +03:00
mixandClaude Opus 5.5 86665f08e6 audit: decisions of 2026-10-13/14 on what 2.4.0 left undone
test / test (push) Canceled after 0s
Agreed with the owner, one question at a time; documents only, no code.

From from-header-binding's Not done list, each a step of the audit plan:
the Sender check column says "Connections turned away"; a stand test of
which key OpenDKIM signs with; Health connects to a milter socket instead
of stat-ing it; the cards layout orders a phone by the box's number;
a refusal the panel did not write is recovered from mail.log; a log
rotated while the panel was down is read to its end; the deferred counter
keeps a place in the log; downtime is counted when supervisord is silent.
"Today, UTC" became the roadmap item local-time (agreed): one instance
zone, no labels beside a time, days counted in that zone, with the
conditions for per-user zones and offsets, and the trap of mail.log's own
stamps written down.

The six steps on the log tailer are split off stage 1 into stage 2b, a
cut of its own (2.5.1); stages 3 and 4 move to 2.5.2 and 2.5.3.

Also: negative e2e cases pass only on their own refusal (stage 0);
notifications are owed per recipient (stage 4); a DNS outage is unknown,
not wrong (M14, M15 had no step); DMARC reports tested where they come in.
review-2026-08-followups is closed, its rows placed in stages or dropped
with a reason. Certificate warnings by lifetime: left as is, with the
condition for return.

The audit's checklist is 61 lines (was 48).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 07:29:13 +03:00
mixandClaude Opus 5.5 c3d99b1f15 roadmap: decisions of 2026-10-13 on the candidates
test / test (push) Canceled after 0s
Agreed with the owner, one question at a time; documents only, no code.

- csrf-tokens -> origin-strict (agreed): no tokens, the origin check
  refuses a request with neither header. security.md's ADR records the
  decision and when tokens are built after all.
- structured-logging, contributing: dropped, each with what brings it
  back (new roadmap section "Dropped, with the condition for return").
- template-data-typing: done by 2.0.0; its last map (healthChecks) is a
  step of the audit's stage 4.
- preflight -> health-setup-checks (certificate name, Postfix's greeting,
  trusted proxy as lines on Health) and test-email, both agreed. The port
  check is dropped with its reason; plans/preflight.md removed.
- password-reset: agreed; a reset link goes only to a confirmed e-mail,
  so the item builds confirmation first.
- phone-desktop-mode: agreed, after stages 1 and 2 of the audit.
- backup-reauth (agreed): the account password before a backup or a
  domain export, out of review-2026-08-followups.
- from-header-binding, Not done: the Delivery log box is required on a
  refused message's page, with the line Postfix wrote, for all three
  kinds of refusal (audit stage 1); the Outbound log gets a filter by
  status and by day (audit stage 4).

The audit's checklist is 48 lines (was 45).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 19:13:04 +03:00
mixandClaude Fable 5.1 b9353db507 audit: the plan as a checklist, with how to resume it
test / test (push) Canceled after 0s
45 steps as '- [ ]' lines per development.md § Plan checklists; roadmap
progress 0/45; a resume note naming the active stage and the working rules.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-12 23:21:30 +03:00
mixandClaude Fable 5.1 7fdf09839f audit: the four items below the line are decisions, and say so
test / test (push) Canceled after 0s
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-12 23:18:56 +03:00
mixandClaude Fable 5.1 6d9273f418 roadmap: audit-2026-10-12 agreed, stage 0 first
test / test (push) Canceled after 0s
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-12 23:15:49 +03:00
mixandClaude Fable 5.1 e3bcafb6d9 audit: decisions of 2026-10-12 written into the plan
Stage 0 then 1+2 agreed; H4 folds logins; H3 measured without a screen
(delivery_log_days, decision rule, no card); L5 validated at save; L12 the
send log gets domain_id; L8 the reply names no other domain; L9 local
comment and test only; stage 5 taken whole. Migrations renumbered
0007-0009.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-12 23:15:34 +03:00
mixandClaude Fable 5.1 2a96008803 release: 2.4.1: code audit of 2.4.0 and its plan
test / test (push) Canceled after 0s
Docs only. docs/audit-2026-10-12.md: findings of a full-tree audit (four
high, twenty-one medium, the rest), doc drift, test gaps, what was checked
and found sound, and a six-stage plan; roadmap index row; CHANGELOG cut and
pin bump. Old tags and images stay as they are (owner): fixes go forward.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-12 22:57:08 +03:00
mixandClaude Opus 5.5 8f226bd82f release: 2.4.0: an application signs only as itself
test / test (push) Canceled after 0s
from-header-binding (docs/plans/from-header-binding.md, 9/9), at the owner's
word (2026-10-12, "Да").

  - A message's From: line is held to what its envelope sender is held to:
    any address of the application's domain, or the addresses of its list,
    with or without a +extension. Anything else is refused with 550 5.7.1 -
    another registered domain, the server's own name, a domain this server
    does not sign for. Until now an application could have mail signed with
    another domain's key by writing that domain into From:.
  - The check is the panel's, first in the milter chain and strict: while
    the panel does not answer, Postfix defers mail.
  - Server > Health counts what that costs - the panel's downtime under a
    running Postfix and the submissions turned away - and is laid out anew,
    boxes numbered by importance.
  - A refused row of the Outbound log says why in its tag.
  - Migration 0006.

Upgrading: mail whose From: is not the application's own stops being
accepted - another of your domains, an address off an application's list, a
visitor's address a form puts there. The guide says what to do about each.
A stopped panel now stops submission.

Not done, written into the plan at the owner's word: the Delivery log box on
a refused message's page, the log's status filter, "messages" that are
connections, and what was reasoned about rather than run - OpenDKIM's own
parser above all.

Cut: CHANGELOG 2.4.0 written (nothing had been put under [Unreleased]
while building); image pin 2.4.0 in compose, README, guide; version table,
roadmap, plan status. security.md: a malformed packet on a milter socket,
as an accepted risk. No tag: the owner tags when he decides to, so there is
no image of this version until then.

On the test server, on c817f08: go vet ./... ok, go test ./... ok
(23 packages), make e2e ok. This commit changes documents and the pin only.

Full detail: CHANGELOG.md (2.4.0).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 22:00:29 +03:00
mixandClaude Opus 5.5 c817f0833c from-header-binding: what the reviews had left as found, and address extensions
test / test (push) Canceled after 0s
The owner on the three things the reviews' commit left as found: can they be
fixed? They can.

- Downtime that crossed a UTC midnight was all booked on the day of the
  restart: it is cut at the midnights between (byDay).
- A message with the null envelope sender - a bounce an application writes
  itself - left a refusal row without a domain, which no domain
  administrator would see: the row takes the domain the login is bound to.
- A log read again after an unclean stop counted its deferrals again: while
  the tailer catches up, a warning no later than the last one counted is
  passed over. The mark is kept to the fraction of a second the log gives.

And on the check being stricter than Postfix for an address with an
extension (invoices+x@ passed in the envelope, was refused in the header):
it makes Postfix's second try now, the address without its extension.
recipient_delimiter was the distribution's default; it is set in
postfix-config.sh, since the check depends on it. e2e sends one through.

Test server, this tree: go vet ok, go test 23 packages ok, make e2e ok (339 s).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 21:29:17 +03:00
mixandClaude Opus 5.5 89fd0a6116 from-header-binding: findings of the technical and security reviews
test / test (push) Canceled after 0s
Both reviews were of c171fb5, read-only, given the code and questions.

Found by both, and the one that would have refused ordinary mail from the
first day: a folded From: line - what a mail library makes of a long display
name - did not parse and was refused with 550. The line is unfolded before it
is read. e2e sends one through Postfix now.

Technical (Opus)
- A display name in a charset Go does not convert (windows-1251, koi8-r) was
  refused: such a name is passed through unread, only the address is judged.
- The statistics ran between binding the socket and serving it, with a
  supervisorctl that has no timeout: they are kept beside the check now and
  nothing of them is waited for. A supervisorctl that fails is logged.
- The count of deferred mail followed smtpd's reject lines and so missed a
  check that stops answering in the middle of a message, where cleanup
  writes them: it is taken at Postfix's warning, the line that names the
  socket. The test line for that case had been invented with the wrong
  process; it is marked as composed from Postfix's source.
- A card whose figures could not be read showed zeros and "ok": dashes and
  "unknown".
- entrypoint removes the sockets a killed container left, which satisfied
  the wrapper's wait before the check was listening.
- Outside quoted strings the line may hold one "@" only - a rule that needs
  no knowledge of OpenDKIM's parser.
- Comments that still spoke of two sockets, or of a session replaced after a
  refusal; the limits of the cards layout on a phone, said where it is defined.

Security (Fable; no Critical or High)
- Another sender's name in an encoded display name passed the second look:
  the decoded name is looked at as well.
- A client could arm the deferred counter by sending the socket's name where
  Postfix quotes it in a warning: the two warnings are matched from their
  first word.
- A refused message wrote a row per recipient: one row, naming the first
  recipient and how many more.
- Docs: the check's lookup is stricter than Postfix's for address extensions
  and the case of logins, rather than "cannot disagree"; what a display name
  looks like to a person is not promised; the timeouts and the 454 at AUTH
  stated as they are; "messages" are connections turned away.

Left as found, in the plan: downtime that crosses midnight is booked on the
day of the restart; a message with the null envelope sender leaves a row
without a domain; a log read again after an unclean stop counts again.

Test server, this tree: go vet ok, go test 23 packages ok, make e2e ok (330 s).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 20:06:02 +03:00
mixandClaude Opus 5.5 c171fb5bc7 from-header-binding: the sender check, what it costs, its pages; guide, architecture, security
test / test (push) Canceled after 0s
A message's From: line is held against the application that sends it, under
the rule Postfix holds the envelope sender to. OpenDKIM chooses the key by
that line, so until now an application could have mail signed as another
domain of the server or as the server itself.

The check (internal/milter/sender.go)
- A role of the panel on a socket of its own (SENDER_CHECK_SOCKET), first in
  smtpd_milters and strict: default_action=tempfail. The journal milter keeps
  its socket and stays fail-open. postfix-wrapper waits for the new socket.
- It reads no database: what an application may send as comes from Postfix's
  own sender_login_maps, what the server signs for from OpenDKIM's
  SigningTable. Both are read again when they change; one that cannot be
  read defers mail.
- One From: field, one address, allowed by the envelope's lookup; then no
  other name of the SigningTable may appear anywhere in the raw line.
  Refused: 550 5.7.1 naming the line, reduced to printable ASCII.
- A refusal is a send-log row with reason "from" and the header address
  (migration 0006), written by another goroutine and not waited for. The
  journal's refusals record their reason too (limit, client), and the
  notifier takes only a limit's.

What it costs (docs/plans/from-header-binding.md, Statistics)
- The role marks itself alive every 2 s; at start the gap since the mark is
  downtime, counted from Postfix's start if later and not at all when
  supervisord says Postfix is not running.
- The log tailer counts the connections Postfix turned away: a warning that
  names the socket, then milter-reject "Service unavailable" from the same
  smtpd. Seen on the stand: the client is refused from CONNECT on and leaves
  at AUTH (454), so no MAIL line is ever written.
- sender_check_days, a row per UTC day, kept a year.

Pages
- Health: the card "Sender check", the socket's row, and the cards layout -
  boxes numbered by importance in two stacks read in rows; the left stack
  polls, the three readings of the right one are swapped in by id.
- Outbound log: a refused row says why in its tag (wrong From, wrong client,
  rate-limited); the message's page shows the From: line and what the
  application may send as.
- Help: topic "Sender check", word for word the mockup's.

Found by the stand, fixed here: the names for the second look came from the
sender bindings, and a domain with a key and no application yet was not
among them.

e2e: negative_journal_milter_fail_open is gone - it proved that mail passes
with the panel down, which is no longer so. In its place
sender_check_holds_the_from_line (another domain's From:, the hostname's, a
stranger's, an address off the list, another sender in the display name -
refused; own addresses - delivered) and panel_down_defers_mail (4xx with the
panel stopped, the container alive, mail again after, downtime and one
deferred message on Health, the card warning).

Test server, this tree before the last cosmetic edits (a fixture row, a
fact's size, docs): go vet ok, go test 23 packages ok, make e2e ok (337 s).
Locally after them: internal/web/view ok.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 19:35:43 +03:00
mixandClaude Opus 5.5 8542e673a8 design: Sender check in the menu of Help
The topic was added with the Health card; its entry in the page's menu was
not. check.py: 0 violations, outlines unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 19:35:43 +03:00
mixandClaude Opus 5.5 cbff3e7e33 design: sender check on Server > Health, Health re-laid, a refusal's reason in the log
Accepted by the owner on 2026-10-12 from screenshots, one change at a time.

- health: a card "Sender check" (today / 30 days / 12 months: panel not
  answering, messages deferred) and a row for its socket.
- health: the boxes are numbered by importance and laid out as `cards` -
  two stacks, odd numbers on the left and even on the right, each closed
  up. His words on the row layout: too many holes; and on stacked columns
  numbered down each side: the third box was below the fold while the
  fourth was not. On a phone the boxes come in the order of their numbers.
- theme.css, components: the `cards` layout; the first pair of stacks of
  the components page is laid out so.
- out-log: a row refused for its From: line, tagged "wrong From".
- help: topic "Sender check"; "rejected" in the Outbound log topic names
  the three reasons.

outlines.json rewritten (check.py --write): the structure of health changed,
and help has one more topic under Outbound. check.py: 0 violations.

The pages of internal/web follow in their own commit; until then the
outline guard fails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 18:46:11 +03:00
mixandClaude Opus 5.5 972516d1c4 from-header-binding: agreed, taken into work
The design questions were closed with the owner on 2026-10-12, one at a
time; the plan records what he chose and what I had proposed instead.

- The rule: From: is held to what the envelope is held to - any address of
  the application's domain, or the addresses of its list. Anything else is
  refused, a domain this server knows nothing of included (I had proposed
  to refuse only names the server signs for).
- The check is the panel's and strict: while the panel does not answer,
  Postfix defers mail. It gets a socket of its own so that the journal
  stays fail-open, and reads Postfix's own file of sender bindings.
- Server > Health counts the panel's downtime under a running Postfix and
  the mail deferred for it, to decide later whether the check needs a
  process of its own.
- A refusal is 550 5.7.1, and its reason is the row's status tag.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 18:46:11 +03:00
mixandClaude Opus 5.5 ce7b09a65c release: 2.3.0: the panel writes to its users
test / test (push) Canceled after 0s
panel-notifications (docs/plans/panel-notifications.md, 10/10), at the
owner's word (2026-10-12, "Да").

  - Four things are reported by e-mail, each once: a sending domain's
    DNS record that does not match, a level-2 rate limit refusing mail,
    the TLS certificate about to expire, a message more than three days
    in the queue. Global users are told all of it, domain users about
    their own domains.
  - Sent from postmaster@<SELFPOST_HOSTNAME> and signed with a DKIM key
    of the server's own name, which the panel generates.
  - Account has the user's switch, Server > Settings the instance's and
    the two DNS records to publish, Server > Health their state.
  - An account e-mail may now be at a public mail host.
  - Migrations 0004 and 0005.

Upgrading: publish the two records Server > Settings shows for the
hostname (a DKIM key and "v=spf1 a -all"); until then notifications are
sent and may be filed as spam. Every user with an e-mail starts with
notifications on. non_smtpd_milters now names OpenDKIM.

Accepted and recorded in security.md: an application can have a message
signed as the server's own name by writing it into From: (roadmap
candidate from-header-binding).

Cut: CHANGELOG [Unreleased] -> [2.3.0]; image pin 2.3.0 in compose,
README, guide; version table, roadmap, plan status. No tag: the owner
tags when he decides to (2026-10-11), so there is no image of this
version until then.

On the test server, on f09b60a: go vet ./... ok, go test ./... ok
(23 packages), make e2e ok.

Full detail: CHANGELOG.md (2.3.0).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 17:08:37 +03:00
mixandClaude Opus 5.5 f09b60aaa2 panel-notifications: findings of the technical and security reviews
test / test (push) Canceled after 0s
Neither review (read-only agents, Opus and Fable, over 972a9f8..400aa76) has
a Critical finding. Both found the same serious one, and more:

- The notification about refused mail was built on what a rejected row says
  of itself. Its domain is the one in MAIL FROM, which the client typed and
  Postfix has not yet held against the login when the milter refuses: an
  application over its limit could have had the panel write a message per
  made-up domain, to another tenant's users among others. And the same
  status is written when an application is refused for its client address,
  which is no limit. Refusals are now counted by the domain the login is
  registered under, only for logins that exist and only while a limit is set
  on the login or its domain; one subject per sending domain. They are read
  through a partial index of their own (migration 0005): the query walked
  the whole send log every minute.
- Account and user e-mails were checked with the DMARC report validator,
  which refuses Gmail and the like - the owner's own kind of address. A new
  validate.AccountEmail accepts them and refuses a local part beginning "-".
- "Said" now means sent: what arises while nobody can be told, or while
  Postfix does not take the message, is told at a later check. Its ending
  with nobody left to tell is simply over.
- Two DNS checks must be 45 minutes apart as well as two: the last check
  before a restart and the first after it were minutes apart.
- The notifier's own mail waiting in the queue, and its bounce coming back
  to postmaster@ this server, are not the queue's subject. A listing cut at
  its size limit or with a line too long to read is no listing; postqueue -j
  gets 30 seconds.
- The tables must name the selector the panel shows, not only have the line.
- An a/mx lookup that did not answer inside the SPF check made the record
  "not authorising"; it is "cannot tell" now, for the pages too.
- A ceiling of 60 messages an hour; a pass stops between domains when the
  process is asked to stop; sendmail gets a WaitDelay; days are rounded as a
  person counts; a failed read of the switch shows it on, not off.

Recorded as accepted in security.md, with a roadmap candidate
(from-header-binding): an application can have a message signed as the
server's own name - or, as always, as another registered domain - by writing
it into the From: header, which nothing holds against the login.

Left as found: a recipient whose copy failed while another's went out is not
tried again; a certificate that lives under 14 days is reported at each
renewal.

On the test server, for this tree: go vet clean, go test ./... 23 packages
ok, make e2e ok (326 s).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 17:00:01 +03:00
mixandClaude Opus 5.5 400aa76737 panel-notifications: the start puts the server's name into OpenDKIM's tables
test / test (push) Canceled after 0s
Found on the test server after the rollout of b905f3e: the SigningTable had
the three sending domains and no line for selfpost.mixfed.ru. The tables are
rebuilt when a domain is added or deleted, and nothing had done either since
the upgrade - notifications would have gone out unsigned until something did.
The e2e stand did not show it: it adds a domain right after its start.

The notifier now asks at its start whether the table on disk has the line
and rebuilds the tables once if it does not (domain.Service.EnsureServerSigned).
A failure is logged and costs the signature, not the message.

Checked on Windows: go vet clean; go test ./internal/domain passes apart from
TestWriteLoadPrivateKeyRoundtrip, which fails there on file modes and always
did; the new test holds one rebuild, none the second time, and that a longer
name is not taken for the server's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 16:27:34 +03:00
mixandClaude Opus 5.5 b905f3e560 panel-notifications: Account, Server > Settings, Server > Health, help; e2e
test / test (push) Canceled after 0s
The pages of the mockups accepted in 87fa0a4, built from the kit's partials
(written by a Sonnet agent into the working tree, reviewed here):

- Account: "Send me notifications" in the Profile form, saved with the
  profile; an unticked box is off.
- Server > Settings: the instance switch (box 03), and box 04 Notification
  mail with the DKIM and SPF records of the server's own name, drawn with the
  record partial of a domain's DNS page and checked in DNS. Sending rate
  limits are box 02 in the left column. A record that is not in place is
  tagged as a warning, not a failure: notifications are sent without it.
- Server > Health: card 06 Notification mail with the state of each record
  and a link to Settings, beside Configuration (07); part of the polled
  fragment; Re-check DNS forces it too. Overview does not run this check.
- help.go: the Notifications topic and the clauses in Settings and Your
  account, word for word with help.html.

If the server's key cannot be read the pages still render: the DKIM record
is left out and SPF is shown unchecked.

e2e: the administrator gets a mailbox behind the sink and the server's DKIM
record from Settings is published; the level-2 refusal of the next step must
then arrive as a notification from postmaster@mail.e2e.test, signed as that
name with the key Settings shows, and must not be a row of the Outbound log.
The rotation step now waits for Postfix to be running: it follows a step that
restarts the container, and logrotate's `postfix reload` failed in between.

On the test server, for this tree: go vet clean, go test ./... 23 packages
ok, make e2e ok (147 s); the notification reached the sink 28 s after the
refusal.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 16:21:48 +03:00
mixandClaude Opus 5.5 c9027381e5 panel-notifications: guide, architecture, security; refusals looked for every minute
guide.md gains Notifications: the four things, when each is said and said
again, who is told, the two switches, and the two DNS records of the server's
own name that make the mail arrive. architecture.md: the notifier as the
fifth role, its readings, state, recipients and sending; the server's own key
in OpenDKIM's tables; non_smtpd_milters. security.md: the panel submits mail
of its own - how, to whom, and what text from outside can reach a body.

That text - a queued message's sender, a DNS finding - is reduced to
printable ASCII and cut before it is printed. Refused mail is looked for
every minute instead of every five: it is one indexed query, and a refusal is
mail not being sent now. The image's note on the panel's postdrop membership
no longer calls it read-side only.

go test ./internal/notify passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 16:00:06 +03:00
mixandClaude Opus 5.5 50fd6b859f panel-notifications: the notifier, its mail and the server's own DKIM key
internal/notify applies the owner's rules. Each subject has a row of state,
so a thing is said once and a restart says nothing again:

- DNS of a sending domain: said after two hourly checks in a row found a
  record wrong, said to be over after two found it right. A lookup that did
  not answer moves nothing; a warning (no DMARC record) is not a subject.
- TLS certificate: at 14 days and at 3, per certificate.
- Queue: a message older than three days, once; once more when none is.
- Level-2 limit: from the rejected rows of the send log, every five minutes;
  at most once a day per domain and application.

Global users are told everything, domain users about their domains; one
message per recipient. With the instance switch off nothing is checked and no
state moves. A message that could not be handed to Postfix is tried again at
the next check.

Mail leaves by sendmail(1) as postmaster@<hostname>, addresses after "--",
no shell. For it to be believed the server's own name is signed like a
sending domain: domain.Service keeps a key for it and a line in OpenDKIM's
tables without a registry row, and non_smtpd_milters now names OpenDKIM. A
sending domain of the same name shares its key with the server.
dnscheck.ServerMail checks the name's DKIM and SPF records;
postfix.QueueMessages reads postqueue -j for arrival times.

The notifier is a sixth role of the panel process. Nothing in the panel
shows or switches it yet.

Checked on Windows: go vet clean; go test passes for notify, store, postfix,
dnscheck, cmd, and domain apart from TestWriteLoadPrivateKeyRoundtrip, which
fails there on file modes and always did. Not yet run in a container: whether
the unprivileged panel may submit is for the e2e step.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 15:56:13 +03:00
mixandClaude Opus 5.5 06ad174fb9 panel-notifications: schema and store
Migration 0004_notifications.sql, DDL only: users.notify, a user's own
switch, on by default; notify_state, one row per subject the panel has
written about (dns:<domain>, cert, queue, limit:<domain>:<login>), so that
each thing is said once and a restart says nothing again.

Store: the instance switch as a settings key, on unless turned off;
SetUserNotify; NotifyRecipients - global users for everything, domain users
for the domains on their list or all of them, only with an e-mail and their
switch on, a shared mailbox once; the state's get, set, delete and list by
prefix; LimitRefusalsSince, the mail refused under a level-2 limit per domain
and application.

Nothing is sent yet. Checked on Windows: go vet clean, go test
./internal/store passes with the schema test at user_version 4. The page
guards fail from the design commit 87fa0a4 until the pages are built.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 15:48:56 +03:00
mixandClaude Opus 5.5 87fa0a4381 design: notifications on Account, Server > Settings and Server > Health
Accepted by the owner on 2026-10-12 from screenshots, after two changes of
his: DKIM for the server's own name as well as SPF ("we need the best
deliverability for these messages"), and the records themselves under
Settings with only their state on Health.

- account: a checkbox "Send me notifications" in the Profile box, under the
  e-mail it writes to. No box of its own, no choice per event.
- settings: box 03 Notifications with the instance switch; box 04
  Notification mail with the two DNS records of the server's own name, DKIM
  and SPF, drawn with the record component of a domain's DNS page. Sending
  rate limits move to the left column as box 02.
- health: card 06 Notification mail - the state of each record and a link to
  Settings - beside Configuration, now 07, each half the width.
- help: a Notifications topic (what is sent, when, to whom, what makes it
  arrive); Settings and Your account each gain a clause.

outlines.json rewritten with check.py --write: the structure of settings and
health changed. build.py: 35 pages; check.py: 32 screens, 52 sp- classes,
0 violations. The plan records both of his changes; step 1 of 10.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 15:45:06 +03:00
mixandClaude Opus 5.5 eff8010d39 panel-notifications: agreed, taken into work
The owner, 2026-10-12: "Делаем". Roadmap status agreed, 0/10; planned cut
2.3.0. The first step is the mockups, shown to him before any code.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 15:27:20 +03:00
mixandClaude Opus 5.5 394eb67c4c panel-notifications: the design questions, closed with the owner
test / test (push) Canceled after 0s
Discussed on 2026-10-12, one question at a time; still a candidate.
plans/panel-notifications.md records the answers:

- Four events in the first version: a domain's DNS record stops matching,
  a level-2 limit refuses mail, the certificate is about to expire, a
  message is in the queue for more than three days (his number; the roadmap
  said one).
- From postmaster@<hostname>, not from a sending domain: with several the
  choice is arbitrary, and a certificate is no domain's subject. An SPF
  record for the hostname makes such mail pass; Server > Health will check it.
- One checkbox per user instead of a choice per event; one instance switch.
- Hourly checks; a DNS mismatch is sent after two checks in a row; "cleared"
  messages for DNS and the queue only; certificate at 14 and 3 days; a limit
  at most once a day.

roadmap.md follows, and no longer says the screens are drawn: the box was
taken out of the mockups on 2026-10-10.

Docs only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 15:05:33 +03:00
mixandClaude Opus 5.5 972a9f825c release: 2.2.0: a message's page keeps its delivery log
test / test (push) Canceled after 0s
delivery-log-storage (docs/plans/delivery-log-storage.md, 8/8), at the
owner's word (2026-10-12, "Да").

  - The log tailer stores what Postfix writes about queued outbound
    messages; a message's page reads it from the database, so the
    Delivery log box no longer empties when mail.log rotates. The lines
    live as long as the send-log row and go with it.
  - Mail that came in on port 25 is not stored.
  - Migration 0003 (send_log_lines), DDL only.

What an operator should know: the stored lines are 1-2 KB a message
and are in a full backup, which mail.log is not (security.md). A
message accepted before this version shows its lines only while they
are in the current file.

Cut: CHANGELOG [Unreleased] -> [2.2.0]; image pin 2.2.0 in compose,
README, guide; version table, roadmap, plan status. No tag: the owner
tags when he decides to (2026-10-11), so there is no image of this
version until then.

On the test server, on a990e7c: go vet ./... ok, go test ./... ok
(22 packages), make e2e ok.

Full detail: CHANGELOG.md (2.2.0).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 11:59:10 +03:00
mixandClaude Opus 5.5 a990e7cc23 delivery-log-storage: findings of the technical and security reviews
test / test (push) Canceled after 0s
Neither review (read-only agents, Opus and Fable, over ae06d8b..815236a) has
a Critical or High finding. What they found and what was done:

- A stranger on port 25 could grow the table: every queue-id line of inbound
  mail was stored and pruned an hour or more later, his own message-id among
  them. The tailer now leaves out the lines of a queue id that
  postfix/inbound/smtpd opened, until qmgr removes the message or another
  listener opens one under the id; 4096 such ids are remembered.
- A commit, so a disk sync, per stored line on the connection the journal
  milter shares, and a backlog written line by line. Lines are now written in
  one transaction per pass over the log, 500 at most, before the read offset
  is saved (InsertSendLogLines; follow() calls the keeper's flush).
- The prune asked the send log once per line and ran hourly. It asks once per
  queue id and runs at start and with the six-hourly sweep; with port-25 mail
  left out there is little without a row to wait for.
- The 4096-byte cut could split a character; it keeps characters whole.
- security.md did not say that a backup now carries these lines, and the plan
  said the time window excludes a reused queue id where it only limits the
  chance to the row's queue lifetime. Both are corrected; the collision is
  recorded as an accepted risk with its trigger.
- Tests that did not prove their comments: the prune is now tested with the
  sweep, a rejected row next to the prune, batching and the change from
  catching up to following, port-25 mail left out and its id reused.

Left as found and written down in the plan: lines the tailer never reads
(a file rotated away while the panel was down), and a message in the queue
across the upgrade, which shows only its later lines.

On the test server, for this tree: go vet clean, go test ./... 22 packages
ok, make e2e ok.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 10:41:04 +03:00
mixandClaude Opus 5.5 815236a607 delivery-log-storage: e2e holds the page to its lines after a rotation
New step delivery_log_survives_rotation: the page of the first delivered
message must list the line that opened its queue id and its delivery line,
then mail.log is rotated with logrotate -f, the current file is checked to
hold no delivery line, and the page must list both lines still. Before this
feature the page searched the current file and would show none.

The step runs after negative_level1_ratelimit: placed right after the first
delivery it rotated away the "connect from" line the level-2 step reads from
the current log, and that step failed.

On the test server, for this tree: go vet clean, go test ./... 22 packages
ok, make e2e ok (126 s).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 10:25:21 +03:00
mixandClaude Opus 5.5 a6845ae948 delivery-log-storage: architecture and guide
architecture.md § Log tailer gains "Stored lines": what is kept and why the
queue id is read from the head of a line, why the tailer does not check for a
row before writing, the one prune rule that is the table's retention, how a
row is given its lines when Postfix reuses queue ids, the write-once rule
while catching up, and the size. QueueLines is described as what it is now,
the fallback for rows without stored lines. guide.md: the message page keeps
its lines after rotation; retention also bounds the kept lines, which are in
a full backup.

Docs only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 10:15:47 +03:00
mixandClaude Opus 5.5 1e6836b3c1 delivery-log-storage: a message's page reads its stored lines
The Delivery log box on a message's page was filled by searching the current
mail.log on every view; that file is rotated daily, so the box was empty for
anything not sent today. deliveryLog now asks the store for the row's lines -
its queue id, dated within the row's queue lifetime plus a day
(RetryPolicy.LongestLifetime) - and shows the latest 200, oldest first.

When nothing is stored the page still searches the current file, which is
what shows a message accepted before 2.2.0. When that finds nothing either,
the note no longer blames rotation for the ordinary case: it says lines are
kept with the row since 2.2.0. The Outbound log help topic in help.go follows
the mockup changed in fc184ec, word for word.

Checked on Windows: go vet clean; go test ./internal/web/... ./internal/postfix
./internal/store pass, the guards among them. New test: lines shown from the
table when mail.log holds none of them, without another queue id's lines and
without those of an earlier message under the same id.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 10:15:26 +03:00
mixandClaude Opus 5.5 fc184ecc01 design: the Outbound log help topic says the lines are kept
The topic said a message's lines are found in mail.log by queue id and that an
old message may show none, because that file is rotated. With
delivery-log-storage the lines are kept with the row: they stay on the page
for as long as the row does and go with it. A message accepted before 2.2.0
shows them only while they are in the current file.

Mockup source and its built page only; help.go follows with the page.
build.py: 35 pages; check.py: 32 screens, 52 sp- classes, 0 violations.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 10:12:10 +03:00
mixandClaude Opus 5.5 81df131217 delivery-log-storage: the tailer keeps the lines of queued messages
Every mail.log line headed by a queue id is written to send_log_lines as the
tailer reads it, dated by the line's own stamp. The id is taken from the head
of the line only - the tail is the far end's text - and has to look like one:
"NOQUEUE:" and "warning:" stand in the same place.

The tailer does not check for a send-log row first: a message's first lines
come before the journal milter writes it. Lines that never get a row are
removed by the prune, which deletes lines older than an hour whose queue id
no row carries. It runs at start, after each send-log sweep and hourly.

While the tailer catches up at start a line is written only if it is not
there already, as the inbound journal does: its place is saved every few
seconds, so an unclean stop means reading some lines twice.

Checked on Windows: go vet clean; go test ./internal/logtail passes except
TestFollowTailsAndRotates, which fails there on renaming an open file and
always did. New tests: which line heads are queue ids, a line dated by its
stamp, no second copy after a rewound offset, the prune cadence and cutoff.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 10:11:28 +03:00
mixandClaude Opus 5.5 3d17206063 delivery-log-storage: the table for a message's log lines
Migration 0003_send_log_lines.sql, DDL only: queue id, the line's own time,
the line. No reference to send_log, because a message's first lines are
logged before the journal milter writes its row and one queue id is several
rows.

Store methods: InsertSendLogLine; InsertSendLogLineOnce for lines the tailer
reads twice after an unclean stop; SendLogLines, which gives a row the lines
of its queue id dated from an hour before the row to the queue lifetime after
it, so a later message under a reused id is not shown; PruneSendLogLines,
which deletes lines whose queue id has no send-log row - the whole of the
table's retention. A line is cut to 4096 bytes.

Nothing writes or reads the table yet. Checked on Windows: go vet and
go test ./internal/store pass, with the schema test pinning the new table,
its index and user_version 3.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 10:08:56 +03:00
mixandClaude Opus 5.5 4468aa8dd7 delivery-log-storage: agreed, plan and checklist
The owner agreed the item on 2026-10-12 (option 1 of the roadmap entry: store
the lines). plans/delivery-log-storage.md records what that means in detail:
every mail.log line headed by a queue id goes into send_log_lines with the
line's own time; a line is a row's when the id matches and its time lies
within the row's queue lifetime, which is the bound against Postfix reusing
ids; a line whose id has no send-log row is deleted after an hour, which is
both the retention and the clean-up of inbound and bounce lines; the page
reads the table and falls back to the current file for rows older than it.
Eight checklist steps, cut 2.2.0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 10:05:08 +03:00
mixandClaude Opus 5.5 977d63a2ea roadmap: inbound quarantine is an idea, not a candidate
The owner, 2026-10-12: take quarantine out of the main plan, it is an idea
to think about. Its index row and section leave roadmap.md; a new closing
section, "Ideas, not on the roadmap", keeps a paragraph and the link to the
notes in plans/inbound-quarantine.md. panel-notifications no longer lists
"mail is held in quarantine" among its events. The plans that called it a
candidate say what it is now. The drawn screens stay unbuilt, as they were.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 10:04:44 +03:00
mixandClaude Opus 5.5 ae06d8b47e release: 2.1.0: Inbound log and filter lists; a milter security fix
test / test (push) Canceled after 0s
inbound-antispam-panel (docs/plans/inbound-antispam-panel.md, 12/12),
at the owner's word (2026-10-12, "Да").

  - Inbound mail is recorded: accepted mail by a second, read-only
    milter on port 25 with the spam filter's verdict, refused mail from
    the mail log. The filter signs what it judged, and a verdict is
    shown only next to its signature.
  - Inbound > Log, with a sheet over the page that says who decided
    and why; Inbound > Filter lists, allow and deny entries written as
    rspamd maps, each with how often it decided a message.
  - Migration 0002 - the first after the 2.0 baseline.

Security, outside the feature and in every release since 1.9.0: the
level-2 rate limit and an application's client IP list held only for
the first message of a connection. Fixed; e2e sends two MAIL FROM on
one connection.

Upgrading an installation with the antispam sidecar: copy
deploy/antispam/ again (the fragment mounts two new things) and
restart the sidecar. Port 25 now logs as postfix/inbound.

Cut: CHANGELOG [Unreleased] -> [2.1.0]; image pin 2.1.0 in compose,
README, guide; version table, schema-migrations, roadmap, plan status.
No tag: the owner tags when he decides to (2026-10-11), so there is no
image of this version until then.

On the test server, on 1f44e40: go vet ./... ok, go test ./... ok
(22 packages), make e2e ok.

Full detail: CHANGELOG.md (2.1.0).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 09:12:16 +03:00
mixandClaude Opus 5.5 1f44e40382 inbound-antispam-panel: the filter reads its signing key for every message
test / test (push) Canceled after 0s
The routine kept the key after its first read. SelfPost makes a new key
when the file is lost or damaged, or a backup brings back another, and
rspamd would then have gone on signing with the old one - every verdict
unrecorded - until someone restarted the sidecar. The file is one short
line; reading it per message costs nothing worth keeping a copy for.

The guide says what never needs doing: nothing of rspamd's is cleared
after a change of the lists or the key; a restart is for a change of
its own configuration only.

Prompted by the owner's question whether rspamd's cache needs clearing
(2026-10-12).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 08:42:16 +03:00
mixandClaude Opus 5.5 d398f488e2 inbound-antispam-panel: the filter signs what it judged
test / test (push) Canceled after 0s
Closes what the security review left as an accepted risk: with the
spam filter down - it is attached fail-open - nothing removes verdict
headers a sender wrote himself, and the inbound journal recorded his
verdict as the filter's. The owner asked whether rspamd could add a
header that proves the others are ours (2026-10-12) and agreed to a
signature: a fixed mark could be written by a sender as easily as the
verdict.

  - inbound.FilterKey: a key of 32 random bytes, 64 hex characters, in
    /data/antispam/filter.key, made on the first start with the relay
    on. 0644 like the list files beside it: the filter is another user
    in another container.
  - deploy/antispam/rspamd/milter_headers.conf: a custom routine reads
    the key through the directory the sidecar already mounts and adds
    X-SelfPost-Filter, the HMAC-SHA256 of the message's queue id, hex;
    a header of that name from the sender is removed.
  - internal/milter/inbound.go: the verdict headers are kept as they
    are read, and at end-of-message - when the queue id is known - they
    count only if the signature fits (FilterSignature, hmac.Equal). No
    key, no queue id, no signature or a wrong one: the row is that of
    mail nobody judged (accept, selfpost, no detail).

The signature goes on to the upstream with the mail and is of no use
there: it fits one queue id.

Tried in a throwaway rspamd 3.11 container on the test server before
the Go side was written: the routine loads, signs
(2aa1ca6d... for key 0123...cdef and queue id 4A1B2C3D5E, the value
python's hmac gives and TestFilterSignatureIsWhatTheFilterWrites
pins), removes a sender's header of the same name, and adds nothing
without a queue id.

Tests: signed, unsigned, made-up, another message's, another key's,
the sender's followed by the real one, the next message on the
connection, no queue id; the key made once, read after, replaced when
damaged, left alone by the lists. e2e: filter.key is there, 644, 65
bytes.

docs/security.md: the accepted risk is replaced by a narrower one (the
key is readable on the host); Mail path describes the signature.

Checked on the test server, uncommitted tree: go vet ./... ok,
go test ./... ok (22 packages), make e2e ok.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 08:34:29 +03:00
mixandClaude Opus 5.5 89774ab847 inbound-antispam-panel: three things the reviews left as found
test / test (push) Canceled after 0s
The owner asked for them to be put right (2026-10-12).

  - A refusal is not logged twice after an unclean stop. The tailer
    saves its place every five seconds, so on start it reads some
    lines again; for the lines it reads on start a row is written only
    if the same row is not there already (store.InsertInboundSpamOnce:
    same moment, client, sender, recipient, decision, engine, detail).
    Lines read as they are written are always new, so two identical
    refusals in one second stay two rows.
  - The Inbound log offers Deny / Allow sender only for a sender the
    lists would take (inbound.Listable): an envelope sender may be
    spelt in ways the form refuses, and the link led to a refusal.
  - A network of one address is stored as the address: 192.0.2.7/32
    and 192.0.2.7 were two entries of one list.

Checked on the test server, uncommitted tree: go vet ./... ok,
go test ./... ok (22 packages), make e2e ok.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 08:17:23 +03:00
mixandClaude Opus 5.5 c731a98a0f inbound-antispam-panel: findings of the technical and security reviews
test / test (push) Canceled after 0s
Steps 11 and 12 of docs/plans/inbound-antispam-panel.md: 12/12. Two
read-only reviews of d65d632..HEAD by different models (Opus,
technical; Fable, security), each given the code and questions, not
conclusions. Nothing Critical or High in the feature; the High the
technical review found outside it is the commit before this one.

Log tailer (internal/logtail/inbound.go):
  - the reply of a refusal ends at the first "; from=<", Postfix's own,
    and the HELO name after it is whatever is left. Taking the longest
    reply let a HELO name that spelt "; from=<..> to=<..> proto=.."
    supply the row's sender, recipient and reply - and so make a
    refusal read as the deny list's;
  - an address with a quote, a bracket or a space is not kept: a quoted
    sender could otherwise supply the recipient;
  - the journal is cut to its row cap every 500 rows written, not only
    every six hours;
  - a queue id is forgotten when the message is queued or the id is
    opened by another listener, and a stale place in the queue of
    arrivals no longer drops a live id;
  - milter-hold lines are not read: the held message reaches the
    journal milter, which writes its row - it was written twice.
Lists (internal/inbound/lists.go): a change of the table and the
rewrite of the files are one step under a mutex; a failed rewrite puts
a removed entry back and rewrites after taking a new one out.
Store: the row cap removes the oldest by time, as the page orders them.
Page: on a sheet a refusal's text is a Reply whatever it starts with.

Help and the guide say that an allow entry for a sender address trusts
an address anyone can claim (design commit before this one).

docs/security.md: the two reviews; the roles table gains Inbound > Log
and Filter lists; Mail path describes the milters, the tailer and the
lists; four accepted risks - the filter's verdict while it is down, an
allow entry by sender address, the list files' mode, log rows held by
domain name. The plan's stale lines (size and rate refusals, who reads
the journal) are corrected.

Left as found, told to the owner: the journal's rows after an unclean
stop may be written twice (the tailer saves its place every five
seconds); a "Deny sender" link is offered for an address the form then
refuses; 1.2.3.4 and 1.2.3.4/32 are two entries.

Checked on the test server, uncommitted tree: go vet ./... ok,
go test ./... ok (22 packages), make e2e ok.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 07:51:50 +03:00
mixandClaude Opus 5.5 5f86457848 milter: the level-2 limit and the client IP list hold for every message
Security fix, outside the inbound antispam feature. Found by its
technical review (Opus, 2026-10-12), reproduced on the test stand
before the fix: over an application's level-2 limit, two MAIL FROM on
one connection were answered 451 and then accepted.

Cause. Postfix introduces a client to a milter once per connection.
The milter library (go-milter 0.4.1, session.go) replaces the milter's
session after any reply that is not "continue" - a refusal, or the
accept that used to end a message. The new session never saw Connect,
so it had no client address, and both checks passed a message without
one: overLimit returned early ("no client IP to key on") and
authIPAllowed did the same. So every message after the first on a
connection, and every MAIL FROM after a refused one, skipped the
level-2 limit and an application's client IP list. In the code since
1.9.0 for the list and longer for the limit.

Fix.
  - build/postfix-config.sh: milter_mail_macros carries {client_addr},
    so the address comes with every MAIL FROM; MailFrom reads it (both
    spellings of IPv6) in the journal milter and the inbound journal
    milter.
  - overLimit no longer asks for a client address: a ceiling is on a
    domain or an application.
  - authIPAllowed refuses an application that has a client IP list when
    no address is known.
  - Body replies "continue" in both milters, which lets the message
    through as "accept" did and keeps the session.

The inbound journal milter had the same cause with a smaller effect:
rows of a connection's later messages had no client address.

In the same file, the checks of INBOUND_ANTISPAM_MILTER and
INBOUND_JOURNAL_MILTER_SOCKET refuse any character outside their set
first: a glob's "*" matched anything after the first characters.

Tests: a session that never saw Connect is judged by the macro, a
listed and an unlisted address, no address, not an address; the limit
with no client address; the inbound milter answers only "continue".
test/e2e sends two MAIL FROM on one connection over the limit and
wants both refused.

Checked on the test server: e2e red before the fix on exactly that
check, green after; go vet ./... ok, go test ./... ok (22 packages).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 07:51:50 +03:00
mixandClaude Opus 5.5 00feb7c1c7 design: help, an allow entry by sender address trusts a claimed address
From the security review of 2026-10-12: the allow list matches the
envelope sender, which nobody authenticates. Help's Filter lists topic
says so and points to client addresses and networks.

check.py: 32 screens, 0 violations.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 07:51:49 +03:00
mixandClaude Opus 5.5 d2a778edc0 inbound-antispam-panel: Filter lists accepted; steps 8-10 closed
test / test (push) Canceled after 0s
The owner accepted Inbound > Filter lists on 2026-10-12 ("Да"), after
a deny entry he added refused my test message on the live test server
with the list's own reply (554 5.7.1 Refused by this server's deny
list), mail.log carried the milter-reject line the count reads and the
journal its row.

Steps 8-10 of the plan were done with each earlier step and are ticked
with where. 10/12: the technical and the security review remain.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 07:22:09 +03:00
mixandClaude Opus 5.5 7192c3861e inbound-antispam-panel: Filter lists, kind column, matches, note on a sheet
Follows the design commit before it (owner's words quoted there).

The lists' tables are Value, Kind, Matched and Remove. The kind tag
opens the entry's sheet: note, day added, how often it decided a
message in the days the log keeps, and the last match.

Matches are read off the journal on each view, nothing is stored
(inbound.Lists.Usage, store.ListSpamListHits). A row counts for an
entry when the list decided the message - a rejected row whose reply is
the deny rule's message, an accepted row whose verdict carries the
allow rule's symbol for that kind - and the entry fits the row's sender
or client address the way rspamd matches it: an address whole, a domain
as the sender's domain, a network containing the client (an
IPv4-mapped address as IPv4). A failure to count shows the lists with
"never" and logs why.

For that the journal milter keeps the lists' own symbols ahead of its
six-symbol cap: rspamd writes symbols heaviest first, so an allow
entry's -100 came last and was cut behind any six others.

The plan gains "Filter list usage": what counting from the journal
implies, and - the owner asked for it in writing - the conditions under
which a counter kept with each entry becomes worth a migration (the
journal cut by its row cap and not by age; a short retention; counts
that must outlive the journal; a slow page). multimap.conf says which
of its texts SelfPost reads back.

Tests: Usage over every kind on both lists and the rows that are
nobody's; the milter's symbol order; the store query; the page and its
sheets; a handler test from journal rows to the numbers on the page.

Seen: the fixture page with a sheet open, headless Edge at 1360 px -
the same as the accepted drawing.

Checked on the test server, uncommitted tree: go vet ./... ok,
go test ./... ok (22 packages), make e2e ok.

Not seen in a real run: a count above zero. It needs an entry added in
the panel, which I cannot sign in to, and then mail that the list
decides; in particular the deny rule's message arriving in mail.log as
the reply of a milter-reject line is what the trial with rspamc showed,
not yet what Postfix logged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 01:08:26 +03:00
mixandClaude Opus 5.5 e0ccff257d design: in-filter-lists, kind column, matches, the note on a sheet
Owner, 2026-10-12, on Inbound > Filter lists as built: "Note хочется
вынести в лист, подсказку, раскрывающийся блок. Тип в отдельную колонку.
Ещё полезно будет дать статистику использования правила, чтобы удалять
устаревшие." Drawn, then accepted ("Да") together with counting the
matches from the Inbound log.

  - Kind is a column of its own; the tag no longer wraps under the
    value.
  - Note and Added leave the table for the entry's sheet, opened by the
    kind tag (of his three - sheet, tooltip, disclosure - the sheet: it
    exists and works under a finger).
  - Matched: how often the entry decided a message, or "never". The
    sheet adds the window ("in the last 90 days") and the last match.
    A fifth column for the last match did not fit half the page width.

Help, Filter lists: what Matched counts and what "never" suggests.

check.py: 32 screens, 0 violations; outlines.json unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 00:54:04 +03:00
mixandClaude Opus 5.5 c1da784193 inbound-antispam-panel: the backup holds the lists; Inbound log accepted
Step 7/12 of docs/plans/inbound-antispam-panel.md. A full backup walks
all of /data, so data/antispam/ was in it from the day the directory
existed; TestCreateIncludesStateExcludesTransient now holds that. On
the first start after a restore the maps are written again from the
database (mailStack.Resync, already there). The guide says what is not
in a backup: the antispam/ directory with the compose fragment and the
rspamd configuration, which is installation and not state.

The plan records the owner's acceptance of Inbound > Log (2026-10-12,
"Этот экран утверждаем") and what he changed on the way to it.

go test ./internal/backup/ ok.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 00:36:59 +03:00
mixandClaude Opus 5.5 9d330bed77 inbound-antispam-panel: Inbound log, Why as a sheet; no Deny on a Postfix reject
Follows the two design commits before it (owner's decisions quoted
there).

Kit: a new component, Sheet / SheetRow (components.go), partials
sheet_tag and sheet (components.html), its rules in panel.css as in
theme.css, and its sample on the kit page next to status_tag. It is
the browser's popover: no script.

Inbound log: the Why column is gone; the decision tag opens a sheet
with who decided, then either the filter's verdict and its symbols one
a line (a detail the journal milter wrote) or the reply (a detail the
tailer read from mail.log - it starts with its code), and the subject
whole. The sheet's id is the journal row's. A row Postfix itself
refused no longer links to the filter lists.

Tests: the page fixture and view tests for the sheet's markup and
content (TestWhyRows), the link rule, three Deny and two Allow on the
mockup's six rows; e2e reads "Decided by" and "Reply" from the sheet
and wants no sender link on the refused row.

Seen: the fixture page with a sheet open, headless Edge at 1360 px -
the same as the accepted drawing.

Checked on the test server, uncommitted tree: go vet ./... ok,
go test ./... ok (22 packages), make e2e ok.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 00:23:51 +03:00
mixandClaude Opus 5.5 970c8ca8da design: the sheet on the components page is a table; no empty Subject
The sample sheet on the components page still held facts; it shows the
table of name and value the accepted sheet is. In in-log a refused
message's sheet has no Subject row - there is no subject to show whole.

check.py: 32 screens, 0 violations.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 00:13:14 +03:00
mixandClaude Opus 5.5 19156814b8 design: in-log, Why is a sheet over the page; no Deny on a Postfix reject
Owner, 2026-10-11/12, on the Inbound log as built:
  - "Why сделать не столбцом, а листом поверх." Opened by the decision
    tag ("Тег."), drawn first, then: its content as a table ("В лист
    может быть таблицей?" - "Да. Но хорошо бы ещё её отцентрировать."),
    with a narrow column of names and only what the row does not show.
  - "Почему Deny у того, кто rejected?" - "Убирай": a message Postfix
    itself refused offers no Deny sender. Accepted mail offers Deny,
    mail the filter refused offers Allow.

A new component, sheet (theme.css, shown on the components page next to
status_tag): the browser's own popover, so no script - the tag is a
button with popovertarget; Esc, a click outside and the cross close it.
It rises from the bottom edge across the window; its content is a table
of name and value in a 44rem column in the middle. A tag that opens a
sheet carries a dotted underline to tell it from a plain tag.

in-log: the Why column is gone and rows are one height. A sheet holds
who decided, the verdict and the symbols (or the reply), and the
subject whole.

Guard: check.py WIDTH_OK gains ".sp-sheet" - the one component laid out
by the window and not by the page. The owner accepted the drawing with
that said.

check.py: 32 screens, 52 sp- classes, 0 violations; outlines.json
unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 00:12:04 +03:00
mixandClaude Opus 5.5 d2b97b3963 inbound-antispam-panel: Inbound log, the sender column is headed From
Follows the design commit before it.

go test ./internal/web/view/ ok.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 23:59:47 +03:00
mixandClaude Opus 5.5 d4efdb483d design: in-log, the sender column is headed From
Owner, 2026-10-11, of "From · client": "Client в названии столбца
зачем? Мне кажется, оно лишним." The client address stays under the
sender, in mono and muted; an address of that shape needs no label.

check.py: 32 screens, 0 violations; outlines.json unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 23:59:18 +03:00
mixandClaude Opus 5.5 56c5dfee40 inbound-antispam-panel: Inbound > Log and Inbound > Filter lists
Step 6/12 of docs/plans/inbound-antispam-panel.md. Written by a Sonnet
subagent from the mockups, reviewed and corrected by me.

GET /inbound/log - the journal a page at a time (50), filtered by
domain, decision and day; QueryInboundSpam / CountInboundSpam with the
viewer's inbound domains as the scope. The global role reads all of it
and gets "Deny sender" / "Allow sender" links into the lists; a user
with the domain role reads the rows of the inbound domains assigned to
them, without the links. The plan said global-only for the journal;
the accepted mockup and 2.0's delegation of Inbound say otherwise -
recorded in the plan as built.

GET, POST /inbound/filter-lists and POST .../{id}/delete - global role,
404 otherwise. Add form (prefilled from the query), the two lists with
Remove as the kit's row action, and "Where the lists go" with the
filter's state and when the maps were last written
(Lists.LastWritten). A refused or duplicate entry is 422 with the
message and what was typed.

Server > Settings: "Keep inbound log rows, days" with the relay on.
Help: Inbound log, Filter lists. Navigation: Log for everyone who sees
Inbound, Filter lists for the global role (Shell.ShowFilterLists).
Status tags accept / reject / tempfail join the kit's mapping.

Tests: page fixtures for both screens, held to outlines.json; view and
handler tests (both roles, filters, paging, relay off, add / duplicate
/ invalid / delete, map file content, settings field, help). e2e reads
the journal from the page instead of the database and adds and removes
a deny entry through the panel, reading deny_domain.map after each.

Seen: both pages and Settings rendered from their fixtures and compared
with the mockups side by side (headless Edge, 1360 px) - same layout
and wording.

Checked on the test server, uncommitted tree: go vet ./... ok,
go test ./... ok (22 packages), make e2e ok.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 23:50:34 +03:00
mixandClaude Opus 5.5 82d4a209be design: in-log and in-filter-lists leave the deferred screens
Guards and mockups only, ahead of the pages (inbound-antispam-panel
6/12). The owner agreed the feature ("Следующая по дорожной карте -
inbound-antispam-panel" / "Да", 2026-10-11) and started this step
("Начинай"), so the two screens are no longer "drawn but not built":

  - guard_panel_test.go: in-log and in-filter-lists removed from
    deferredScreens. From here TestLegacyRatchetMatchesTheEngine wants
    the engine to have both pages - red until the next commit.
  - guard_routes_test.go: the four routes join routes.inbound, so they
    must exist with the relay on and answer 404 with it off.

Two mockup details brought to what the kit can draw, found when the
pages were built from it:
  - help: the two new topics are one paragraph with a break in it, as
    every topic is (panel.css has no gap between paragraphs of a topic);
  - in-filter-lists: the directory in "Files" is mono at the text size -
    the kit's inline code has no small variant.

check.py: 32 screens, 0 violations; outlines.json unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 23:50:33 +03:00
mixandClaude Opus 5.5 737611a07e design: Inbound log and Filter lists as built; inbound retention; Help
Mockup-first step of inbound-antispam-panel 6/12. The two screens were
drawn for the redesign before the feature existed; this brings them to
what steps 3-5 built.

in-log: the lead says where rows come from and how long they are kept;
a help link in the box head; Why is the engine and the detail as the
journal holds them (rspamd: action, score / threshold, weighted
symbols; postfix and a refusing filter: the reply text). No row
invents what the journal cannot hold: the "selfpost - upstream timed
out" tempfail and the Review link to a quarantine page that does not
exist are gone, a refused message has no subject, an allow-listed one
shows SELFPOST_ALLOW_FROM(-100.00).

in-filter-lists: Remove is a POST - the kit's row action, not a link;
a domain entry is shown as stored, without "@"; "Where the lists go"
names the directory and its six files instead of allow.map / deny.map,
says rspamd rereads them by itself and that Deny wins.

settings: "Keep inbound log rows, days" in the Log retention box.

help: two topics under Inbound, Inbound log and Filter lists, and a
sentence in Settings.

outlines.json rewritten: Help's Inbound box has six topics, was four.
check.py: 32 screens, 0 violations.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 23:18:57 +03:00
mixandClaude Opus 5.5 eea40bcd49 inbound-antispam-panel: filter lists, written as rspamd maps
Step 5/12 of docs/plans/inbound-antispam-panel.md.

Store: AddSpamListEntry, ListSpamListEntries, DeleteSpamListEntry.
internal/inbound/lists.go validates and normalises an entry - sender
address, sender domain, client address or network (canonical text, an
IPv4-mapped address as IPv4, /0 refused) - and writes the lists as six
map files under /data/antispam (ANTISPAM_DIR), each atomically, 0644 in
a 0755 directory: the filter is another user in another container. All
six are written on every change and on every start with the relay on,
empty ones included. An entry whose maps could not be written is taken
back out of the table. The writer refuses a stored value with
whitespace, a newline or # in it.

The plan's allow.map / deny.map became one file per list and entry
type: to rspamd an IP map and a sender map are different map types.

deploy/antispam/rspamd/multimap.conf holds the rules; the fragment
mounts it and ./data/antispam into the sidecar. Deny is a prefilter
reject, allow a symbol weighing -100. Tried in a throwaway rspamd 3.11
container on the test server:
  - with allow as a prefilter accept, an address on both lists was
    accepted and an IP on both was refused - the rule that ran first
    decided; with the weight, deny won in all five mixed cases;
  - IPv4, IPv6, address (any case), domain (not its subdomains) match;
  - a rewritten file is reread in about ten seconds, no restart;
  - files that appear after rspamd started are loaded the same way.

No page edits the lists yet (step 6).

Checked on the test server, uncommitted tree: go vet ./... ok,
go test ./... ok (22 packages), make e2e ok - with the relay on
/data/antispam is 755 and holds the six maps, 644, empty.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 23:07:10 +03:00
mixandClaude Opus 5.5 27fd355e53 inbound-antispam-panel: refused inbound mail from mail.log; retention
Step 4/12 of docs/plans/inbound-antispam-panel.md.

The log tailer writes an inbound_spam_log row for mail port 25 refused
(internal/logtail/inbound.go): a "reject:" line of the port-25 smtpd is
Postfix's own policy; a milter-reject / milter-hold / milter-discard
line of cleanup is the filter's verdict. Only a recipient at an inbound
domain is a row, so "Relay access denied" for a stranger's domain is
not. The row is dated by its log line; the reply text is cut to 255
bytes of printable ASCII; a reply starting with 4 is a tempfail.

To tell port 25 from 465 / 587 the listener now logs as
postfix/inbound (syslog_name). cleanup logs under one name for every
listener, so its verdict counts only for a queue id the port-25 smtpd
was seen to open (4096 remembered). Lines are matched from the head,
after timestamp and host: their tail is the sender's own text.

Retention: PruneInboundSpam every six hours - older than the
inbound_spam_log_retention_days setting (seeded from
INBOUND_SPAM_LOG_RETENTION_DAYS, 90, range 7-365), then down to 10 000
rows. The setting has no field in the panel yet.

Known limits, in architecture.md: a log line names one recipient, so a
message refused at end-of-message is one row; a line re-read after an
unclean stop would be a second row.

Checked on the test server, uncommitted tree: go vet ./... ok,
go test ./... ok (22 packages), make e2e ok - the unknown recipient at
the inbound domain refused on port 25 has its row (reject by postfix),
the stranger's domain has none.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 22:42:25 +03:00
mixandClaude Opus 5.5 94e94490af roadmap: phone in "desktop site" mode gets the phone navigation
A candidate, recorded at the owner's word (2026-10-11, screenshot of
Server > Health on selfpost.mixfed.ru): the body is laid out in columns
while the navigation is the phone stack with every section open. The
item states what was seen and a first guess at the cause (980 px falls
between Bulma's 769 and 1024 px breakpoints); nothing is investigated
yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 22:42:08 +03:00
mixandClaude Opus 5.5 28ec784ae2 inbound-antispam-panel: inbound journal milter on port 25
Step 3/12 of docs/plans/inbound-antispam-panel.md.

The panel serves a second milter while the inbound relay is on
(internal/milter/inbound.go, socket INBOUND_JOURNAL_MILTER_SOCKET,
default /run/selfpost/inbound-journal.sock). build/postfix-config.sh
puts it last on port 25, after the optional antispam milter, with
default_action=accept; postfix-wrapper.sh waits for its socket. At
end-of-message it writes one inbound_spam_log row per recipient at an
inbound domain - other port-25 recipients (DMARC report addresses) get
none - and always accepts. With a filter attached it reads the verdict
from X-Rspamd-Action and X-Spamd-Result (action, score, up to six
weighted symbols, only what the patterns recognise, 255 bytes); without
one those headers are the sender's and are not read. The plan named
X-Rspamd-Score, which rspamd does not have.

The antispam compose fragment mounts rspamd/milter_headers.conf into
the sidecar so that it adds the two headers (checked in rspamd 3.11:
both routines exist and remove a sender's own headers of those names).

Store: InsertInboundSpam, IsInboundDomain.

Checked on the test server, uncommitted tree: go vet ./... ok,
go test ./... ok (22 packages), make e2e ok - the relayed message of
inbound_relay_delivers_and_refuses has its journal row (accept by
selfpost) and the DMARC report sent through the same listener has none.

Not yet seen in a real run: a verdict written by rspamd reaching this
milter. The e2e stand has no filter, and the test server's only inbound
domain has no upstream, so it accepts no mail.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 22:18:14 +03:00
mixandClaude Opus 5.5 a3cc52d6d4 inbound-antispam-panel: migration 0002, journal and filter-list tables
Step 2/12 of docs/plans/inbound-antispam-panel.md, the first migration
after the 2.0 baseline (2.0.1 was cut before it, as agreed).

0002_inbound_spam_log.sql adds inbound_spam_log (one row per recipient:
domain, client IP, envelope, subject, decision, engine, detail) with
indexes on created_at and (inbound_domain, created_at), and
inbound_spam_list (allow / deny entries, unique per list). DDL only;
decision, engine, list_type and entry_type carry CHECKs, as the
baseline does for its own enumerations. Nothing reads or writes the
tables yet.

Tests: TestBaselineSchema pins both tables and user_version 2;
TestOpenMigratesA20Database opens a file holding the baseline alone and
finds its user kept; TestInboundSpamConstraints.

go test ./internal/store/ ok; go vet ./... ok.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 22:05:20 +03:00
mixandClaude Opus 5.5 d65d632505 release: 2.0.1: DMARC ingest drops and bounds; Full backup from the panel
test / test (push) Canceled after 0s
A patch on 2.0.0, all of it found on the test server after the cut:
  - a message that is not a report this relay keeps is dropped at once
    instead of being retried for days;
  - a report is bounded before it is stored, and reports are kept per
    domain (security review of the ingest path, nothing Critical or High);
  - Full backup from the panel works (it had answered 200 with ten bytes
    since 1.3.0);
  - no backup holds the Postfix queue, and the panel says why.

Cut: CHANGELOG [Unreleased] -> [2.0.1]; image pin 2.0.1 in compose,
README, guide; version table.

On the test server, on dd0fefd: go vet ./... ok, go test ./... ok
(22 packages), make e2e ok.

Full detail: CHANGELOG.md (2.0.1).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 21:14:00 +03:00
mixandClaude Opus 5.5 dd0fefdcb7 backup: the panel's archive is the command's, private key included
Owner, 2026-10-11: the private key is needed, and a backup from the
panel and one from the console have to be the same.

The commit that made the panel's backup work skipped a file under
certs/ the panel could not open - on the test server the private key,
which the certificate hook leaves to root and Postfix's group. That gave
an archive with a certificate and no key, and two kinds of archive.

- Params.Unprivileged is gone. Nothing is left out on account of who
  makes the archive; the queue is out for everyone (previous commit).
- When the panel cannot read a file under certs/, backup.Check says so
  before the download and the panel refuses with what to do on the host:
  chgrp 998 certs/privkey.pem && chmod 640. backup.UnreadableCert tells
  that case from any other failure.
- 998 is the container's selfpost group, which holds the panel and
  Postfix. Its id was whatever groupadd gave; the Dockerfile now fixes
  it at the value it has always had, because an operator addresses it
  by number from the host.
- Guide: method 1 is the same archive as method 2, with the permission
  the key needs and a note for whatever renews the certificate. Help and
  the box follow the design commit before this one.
- e2e: the restore scenario first makes the key unreadable and expects
  the refusal naming the file and the way out, then takes the backup and
  expects the key in it; nothing is put back by hand before the restart.

  go vet ./internal/backup/ ./internal/web/handlers/: ok
  go test ./internal/backup/ ./cmd/... ./internal/web/...: ok
  make e2e on the test server: ok

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 20:43:10 +03:00
mixandClaude Opus 5.5 75ebbf6a57 design: the panel's backup needs the private key readable, and says so
Owner, 2026-10-11: the key is needed and the panel's archive must be the console's. The box no longer says a certificate file may be left out; Help says the key has to be readable by the panel (group 998 on the host) and that the panel refuses, naming the file, until it is.

check.py: 0 violations; outlines.json unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 20:43:10 +03:00
mixandClaude Opus 5.5 ede3783734 panel-redesign: Full backup and Help say why the queue is left out
Follows the design commit before it: the paragraph of the Full backup box and the Backup topic of Help.

  go test -count=1 ./internal/web/...: ok
  Backup page at 1280: no overflow, the box holds the longer paragraph.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 20:16:30 +03:00
mixandClaude Opus 5.5 d70371a50c design: Full backup and Help say why the queue is left out
Owner, 2026-10-11: the user has to be told why. The box gives the reason in a sentence - the queue cannot be copied reliably while the server is running; the Help topic explains it (Postfix keeps moving messages, a copy could hold one twice or miss it), what follows (waiting mail is not carried over, a restored server starts with an empty queue) and what to do to keep the queue (stop the container, archive the project directory).

check.py: 0 violations; outlines.json unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 20:16:30 +03:00
mixandClaude Opus 5.5 21e961cb29 backup: no full backup holds the Postfix queue
Owner, 2026-10-11: the queue goes out of the command's archive too.

The commit that made the panel's backup work left the queue in the
archive of selfpost-backup, which runs as root and can read it. But that
copy was never the queue: the archive is made while the instance runs,
Postfix moves messages between the queue's directories all the time, and
a message on its way could be in the archive twice or not at all. One
rule now - QueueDir is left out of every archive - and
Params.Unprivileged is only about a certs/ file the panel may not open.

What is not carried over is mail still waiting for delivery; a restored
instance builds an empty queue (the e2e restore scenario starts one).
To keep the queue: stop the instance and tar the project directory, as
the guide's methods 3 and 4 already describe.

Guide: the description of a full backup, methods 1 and 2 and the
comparison table. Help follows the design commit before this one.

  go vet ./internal/backup/: ok
  go test ./internal/backup/ ./cmd/... ./internal/web/...: ok

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 20:06:41 +03:00
mixandClaude Opus 5.5 82f212ab39 design: Help says no backup holds the Postfix queue
Owner, 2026-10-11: the queue is left out of the command's archive as well as the panel's. The Help topic no longer says selfpost-backup takes it; the command still takes the private key the panel cannot read.

check.py: 0 violations; outlines.json unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 20:06:41 +03:00
mixandClaude Opus 5.5 b55b33ae92 e2e: full backup from the panel, restore into an empty directory
After the rate-limit checks the run takes a full backup through the
panel, reads the archive through while the container can still say what
went wrong, then removes data/, certs/, docker-compose.yml and .env,
unpacks the archive into the project directory, puts the private key
back (the panel's archive comes without it and without the queue, both
asserted), and starts the stand again.

What has to hold: manifest.json is consumed by the first start, the
administrator signs in, the domain shows the same DKIM value, and a
message sent with the application's old password reaches the sink and
verifies against the record published before the backup.

The port-25 scenarios then run on the restored installation.

make e2e on the test server: ok.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 19:55:19 +03:00
mixandClaude Opus 5.5 8c9a71e6a0 backup: the panel's full backup works, without the queue
Found by the e2e restore scenario of the next commit: the Full backup
button answered 200 with ten bytes. The archive includes the Postfix
queue, Postfix keeps every queue file and directory readable by its own
user alone (not configurable), and the panel runs as its own user - the
walk stopped at /data/postfix/queue/active with "permission denied"
after the headers were out. So since the queue moved under /data in
1.3.0. Confirmed on the test server: selfpost-backup as root makes the
whole archive, the same command as user panel fails the same way.

Owner, 2026-10-11: the panel's archive comes without the queue.

- backup.Params.Unprivileged: the queue is left out, and so is a file
  under certs/ that cannot be opened (the private key on the test
  server is root and Postfix's group only), named through OnWarn. The
  selfpost-backup command leaves it unset and takes both, as before.
- backup.Check reads through everything Create would archive, writing
  nothing. The panel runs it before the first byte: a failure is a
  message on the page, not a download cut short.
- The box, Help (design commit before this one) and the guide say what
  the panel's archive leaves out; the guide's comparison table has rows
  for the queue and the private key, and tells to put the key back
  before starting a restored instance.

  go vet ./internal/backup/ ./internal/web/handlers/: ok
  go test ./internal/backup/ ./cmd/... ./internal/web/...: ok
  make e2e on the test server, with the restore scenario: ok

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 19:55:18 +03:00
mixandClaude Opus 5.5 646ec1e149 design: Full backup says what the panel's archive leaves out
Owner, 2026-10-11: the archive made from the panel comes without the Postfix queue (variant 1). The Full backup box and the Help topic say so, and name the other thing the panel may not read - a certificate file, usually the private key - and the command that takes both.

check.py: 0 violations; outlines.json unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 19:55:18 +03:00
mixandClaude Opus 5.5 8496b289ec e2e: the run ends on port 25, with a report, junk and inbound mail
Until now the suite only read back the configuration generated for the
two optional port-25 features. DMARC ingest shipped in 1.7.0 with a
delivery service Postfix could not start, and nothing sent a report
through it until a real one arrived on the test server on 2026-10-11.

The last steps recreate the container with INBOUND_RELAY_ENABLE and
DMARC_REPORTS_ENABLE on, over the same /data, and talk to port 25 as a
stranger's mail server:

- the container log has no postconf warning, the session survives;
- a domain takes the SelfPost-hosted report address; a gzip report
  mailed to it appears under Outbound > DMARC reports; an address nobody
  chose is refused at RCPT;
- a report on another domain and a message with no report are accepted,
  dropped with "dropped: ..." on their delivery lines, counted as two
  parse failures, absent from the queue and from the reports;
- an inbound domain with the sink as upstream and one listed recipient:
  mail for it reaches the sink, another recipient and another domain are
  refused at RCPT.

The stage directory now holds docker-compose.yml and .env, as an
operator's project directory does (a full backup needs them; the
restore scenario that uses it is not in this commit - the panel's full
backup turned out not to work, see the next one).

make e2e on the test server: ok.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 17:40:47 +03:00
mixandClaude Opus 5.5 4dd46ea355 dmarc: a report is bounded, kept per domain, and ingested one at a time
Security review of the DMARC report path (Fable, 2026-10-11), which ran
for the first time in 2.0.0. No Critical or High finding. Fixed:

- Medium: a report was bounded only as 8 MiB of XML - some 160 000
  records or one name of megabytes, all of it rows and table cells. At
  most 5 000 records (more is ErrRejected, dropped), text fields cut to
  255 bytes on a character boundary, a record's count capped.
- Medium: the cap of 500 reports was for the whole server, so a stranger
  filing under one domain's address evicted every other domain's
  reports. PruneDMARCReports counts per domain.
- Low: the pipe service ran up to default_process_limit (100) commands
  at once against the panel's database; maxproc is 1.
- Low: the reason logged for a dropped message carried the sender's
  bytes (header line, zip entry name) as they came; cut to 200 bytes and
  reduced to printable ASCII, zip entry names quoted.
- Low: SELFPOST_DATA_DIR / SELFPOST_DB_PATH with whitespace or "$" would
  reach the command through export_environment as another path; refused
  at start when DMARC ingest is on.

Left as they are, told to the owner: the parse-failure count is a
lifetime figure shown under "This week"; a report larger than the
command reads is dropped as unreadable; nested multipart is not searched.

The hub's Retention line follows the design commit before this one.
security.md: the review, and the accepted-risk entry says what a
stranger can and cannot do now. The roadmap no longer calls the review
pending.

  go vet ./cmd/... ./internal/dmarc/ ./internal/store/: ok
  go test ./cmd/... ./internal/dmarc/ ./internal/store/ ./internal/web/...: ok
  make e2e on the test server (with the port-25 scenarios of the next
  commit): ok

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 17:40:31 +03:00
mixandClaude Opus 5.5 505260e227 design: the DMARC hub says reports are kept 500 a domain
The retention cap is counted per domain (the security commit after this one); the Ingest box says so: 500 reports a domain or 90 days.

check.py: 0 violations; outlines.json unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 17:40:30 +03:00
mixandClaude Opus 5.5 09d9625f32 dmarc-ingest: what is not a report this relay keeps is dropped, not retried
Owner, 2026-10-11: foreign and malformed reports are turned away.

The command answered every failure with EX_TEMPFAIL, so Postfix kept a
message that could never be stored and offered it again for days, each
attempt adding to the parse-failure count. IngestMessage now marks those
failures (unreadable as an aggregate report, a domain not sent from
here, another domain's report address) with ErrRejected; the command
counts one failure, writes the reason where Postfix puts it on the delivery line of the mail log
and reports the delivery as done. No bounce: the envelope sender of such
a message is whatever its author wrote, and the report addresses are
public. A failing store is still EX_TEMPFAIL.

  go vet ./cmd/... ./internal/dmarc/: ok
  go test ./cmd/panel/ ./internal/dmarc/ ./internal/web/...: ok

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 16:28:12 +03:00
274 changed files with 29881 additions and 1721 deletions
+3
View File
@@ -4,6 +4,9 @@
.gitattributes
dev
docs
# test/e2e/.stage holds what an e2e run leaves behind (DKIM keys, sasldb2);
# the e2e's own sink image is built from test/e2e/sink as its own context.
test
bin
*.exe
README.md
+19
View File
@@ -89,8 +89,27 @@ jobs:
cache-dependency-path: test/e2e/go.sum
- name: e2e (gates publishing — see docs/development.md)
# The e2e builds selfpost:e2e again through compose.override.yml, and
# that rebuild is the image pushed below. Without the version here it
# was stamped "e2e" (audit 2026-10-12, H1); the suite now checks the
# stamp against this variable.
env:
SELFPOST_E2E_VERSION: ${{ needs.prepare.outputs.version }}
run: cd test/e2e && go test -v -timeout 20m ./...
- name: Check the version stamp of the image about to be pushed
env:
WANT: ${{ needs.prepare.outputs.version }}
run: |
set -euo pipefail
for bin in panel selfpost-backup; do
got=$(docker run --rm --entrypoint "$bin" selfpost:e2e -version)
if [ "$got" != "$WANT" ]; then
echo "::error::selfpost:e2e $bin -version = '$got', want '$WANT'; refusing to push"
exit 1
fi
done
- name: Log in to ghcr.io
uses: docker/login-action@v3
with:
+13
View File
@@ -24,6 +24,9 @@ jobs:
with:
go-version: "1.26"
cache: true
cache-dependency-path: |
go.sum
test/e2e/go.sum
- name: Format
run: |
@@ -39,3 +42,13 @@ jobs:
- name: Test
run: go test ./...
# The e2e module runs only in release.yml, after a release is published;
# a type error there would first show up then. Compile it here without
# running it: `go test -run` would still run TestMain, which brings up
# the whole stand.
- name: Compile the e2e
working-directory: test/e2e
run: |
go vet ./...
go test -c -o /dev/null .
+685
View File
@@ -5,6 +5,691 @@ Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); version
## [Unreleased]
### Security
- **A sender on a private address could have his own spam verdict signed.**
rspamd's header module by default leaves a message's verdict headers
alone - removes none, adds none - when the client counts as local (a
private range, which every client has when Docker's proxy relays port 25)
or authenticated, while the routine that signs a verdict for SelfPost runs
for every message. Such a sender's `X-Spamd-Result` and `X-Rspamd-Action`
then reached Inbound > Log as rspamd's. `milter_headers.conf` now turns
both exceptions off; tried against rspamd 3.11.1 before and after
(audit 2026-10-12, stage 2c; assumptions C-R7). Copy the new
`antispam/rspamd/` files next to your compose fragment.
- **An address listed for one application could be put in `From:` by
another of its domain.** The sender check accepted a `From:` address when
any of its keys in the sender map - the address, the address without its
extension, the domain - named the application; Postfix takes the first
key it finds and asks only its owners. An application set to any address
of a domain could therefore write in `From:` an address listed for
another application of that domain - which Postfix refuses it in the
envelope - and have it signed. The check now reads the map as Postfix
does: the first key found decides, logins are compared without regard to
case, owners may be separated by blanks or continue on the next line, and
a key written twice makes the check defer, as it makes Postfix refuse the
whole table (assumptions B-PS1, B-PS4).
### Fixed
- **A message Postfix gave up on no longer stays "deferred" for ever.**
Postfix records the end of a message's queue lifetime in one line for the
whole message, without the recipients; the panel waited for a line per
recipient that Postfix never writes, and the reconcile sweep closed only
`queued` rows, so such rows said `deferred` until they were pruned. The
log tailer now reads that line and closes the message's rows as
`bounced`; the sweep also closes a `deferred` row whose message Postfix no
longer holds (which covers the rows left so far, and `postsuper -d`); a
message forced to expire (`postsuper -e`, `#` in the queue listing) is
still counted as held (audit 2026-10-12, stage 2c; assumptions A-8,
B-PQ1).
- **Lines Postfix writes while `mail.log` is being rotated are read.**
Postfix's log writer keeps the old file open until the reload that
follows the rotation; the log tailer switched to the new file at once,
so a delivery written in between never reached its row, which the sweep
later closed as `bounced`. The tailer now reads the rotated file for a
minute more (assumptions A-33).
- **Postfix's time values are read as Postfix reads them.** A bare number
took seconds, so `maximal_queue_lifetime = 3` was three seconds to the
panel and three days to Postfix; `1h7m` was 67 minutes to the panel and
one hour to Postfix. A value now takes the parameter's own default unit
and one unit only; `$name` references are expanded; a
`bounce_queue_lifetime` longer than `maximal_queue_lifetime` is cut to it,
as Postfix's queue manager does (assumptions B-PQ10, B-PQ11).
- **A refusal's line goes on the right application's row.** Postfix's line
about a refusal at `MAIL FROM` was matched to its row by client address,
sender and time; two applications behind one address sending as one
sender could take each other's line. The line carries the login after
all, and the match now uses it. A login read from smtpd's `client=` line
no longer keeps a trailing comma when the client also gave `AUTH=` on
`MAIL FROM`. A refusal written from `mail.log` after a restart says that
Postfix's line names the last recipient only (assumptions A-15, A-21,
A-23).
- **A delivery reaches the row of every recipient Postfix delivers to.** A
quoted local part with a blank in it (`"a b"@x`) was cut at the blank, a
needlessly quoted one (`"ab"@x`) kept quotes Postfix does not log, a
recipient Postfix had rewritten was looked for under its new address, and
a recipient given twice made two rows of which Postfix reports one - each
row then waited for a line that never named it, and the sweep closed it
as `bounced`. The row now holds the recipient as Postfix logs it, once,
and is found under `orig_to=` when Postfix rewrote it (assumptions B-G5,
A-9, A-31, D-S20).
- **Inbound mail between 10 MB and the port's limit is accepted.** Port 25
said it took messages up to `INBOUND_MESSAGE_SIZE_LIMIT` (25 MiB by
default), but Postfix writes the message to its queue under a limit of
its own, main.cf's, never set and so 10240000 bytes: a larger message was
taken through `DATA` and then refused with 552. That limit now follows
port 25's when it is larger; 465 and 587 keep theirs (assumptions
C-P31).
- **Mail to an address literal of the container is refused, not delivered
locally.** An empty `mydestination` left one way to local delivery:
`user@[127.0.0.1]` and the container's other addresses. Local delivery is
now Postfix's refusal, and Postfix refuses such a recipient at `RCPT TO`:
`550 5.1.1 ... local mail delivery is disabled` (assumptions C-P8).
- **A bounce is judged by the filter lists as the panel counts it.**
rspamd's sender rules took a bounce's `From:` header when the envelope
sender was empty, so a bounce could be refused or allowed by an entry the
Filter lists page never showed as used. They now look at the envelope
sender only (assumptions C-R4).
- **Stopping or restarting the container stops Postfix as Postfix means it
to be stopped.** Supervisord's stop signal reached only the shell that
started Postfix, never Postfix's master process: a restart of the program
found Postfix "already running", and a container stop cut Postfix off at
the end. The start wrapper now runs `postfix stop` (assumptions C-P42).
- **Postfix that cannot start takes the container down instead of waiting
for ever.** When the milter sockets never appear, the wrapper gives up
after `MILTER_WAIT_TIMEOUT`; that exit was taken for an ordinary one and
the wrapper started again, endlessly. It now counts as a failed start,
and once supervisord's three retries are spent the container exits for
Docker to restart it. For `MILTER_WAIT_TIMEOUT` plus ten seconds after
each start, Server > Health shows Postfix as starting (assumptions C-V6).
- **A logrotate configuration it would ignore ends in a fault shown on
Server > Health,** where logrotate was restarted every second for ever;
the container stays up, mail is not affected (assumptions C-V5).
- **A failed Postfix reload is reported.** The panel took a reload as done
as soon as it was started, and a reload already running as its own. It now
waits for `postfix reload` to finish and shows its failure on the page that
saved; a reload while Postfix is still starting is left to the start
(assumptions B-U2).
- **Changing an application's password no longer rewrites the live SASL
database.** `saslpasswd2` wrote into the file Postfix reads at every
`AUTH` and the backup copies, with no lock between them: an application
signing in at that moment could be refused, and a backup taken then could
carry a damaged database whose restore would refuse every application.
The change is now made to a copy that replaces the file whole (assumptions
C-S3, C-Q3).
### Changed
- **The rspamd image is pinned to one build**, `rspamd/rspamd:3.11.1-3`:
`3.11` and `3.11.1` move with every rebuild (assumptions C-R12). The rules
lose their `prefilter = true` lines, which rspamd 3.11.1 does not read.
## [2.5.1] - 2026-10-14
Stage 2b of the 2026-10-12 audit: the log tailer and what the panel shows of
a refusal. Migration `0009` keeps Postfix's line about a refusal on its row,
the client a refusal came from, a mark on rows written from `mail.log`, and
the Sender check's two counts and its estimate mark.
### Fixed
- **A log rotated while the panel was down is read to its end.** On start,
a saved place in a file that had been rotated meant "start on the new
file", and what Postfix wrote between the panel's stop and the rotation -
deliveries, refusals, deferrals - was never read. The tailer now reads the
rotated file (`mail.log.1`) from the saved place to its end first; when
that file is not the one the place belongs to (rotated twice), it says so
in the log, with the time from which nothing was read. The log's
fingerprint now includes the file's inode, so two files that begin alike
are still told apart (audit 2026-10-12, stage 2b).
- **A second deferral of the same second is counted after a restart.** The
Sender check's count of deferred mail remembered the time of the last
line it counted, to pass over lines read again after an unclean stop;
`mail.log` stamps whole seconds, so a second warning of that second was
passed over too. It now remembers the place in the log, saved with the
count in one transaction (audit 2026-10-12, stage 2b).
- **A refused message's page shows what Postfix wrote.** Its Delivery log
box said Postfix wrote nothing about it, which was untrue. A message
refused for its `From:` line had a queue id, given before its headers were
judged; the row now keeps it and the page shows the message's lines, the
refusal among them - only those of the minutes around the refusal, since
Postfix frees the id at once. A message refused at `MAIL FROM` (a level-2
limit, a client address off the list) has no queue id; the log tailer puts
Postfix's `NOQUEUE: milter-reject` line on its row (migration `0009`) and
the page shows it. A row refused before this says the line was not kept.
A refusal's line is red for a permanent reply, amber for a temporary one
(audit 2026-10-12, stage 2b).
- **A message the panel closed itself says so.** The history of a `bounced`
row read "the receiving server refused the message permanently, or
Postfix gave up ... The reason is in the delivery log below" also for a
row the reconcile sweep closed, where neither is known and the log has
nothing. Such a row's step is now "Closed by the panel", with why.
- **The Sender check card counts connections and messages apart.** What was
one column, "Messages deferred", was two things: connections Postfix
turned away while the panel did not answer - refused before any message
is offered, so a client trying three times is three - and messages
deferred in the middle, when the panel stopped answering after a message
had begun. The card now has a column for each (the owner's decision).
- **A start at which supervisord does not answer still counts the downtime.**
The panel asks supervisord once at start since when Postfix runs; when it
did not answer, that start's downtime was not counted at all. It now asks
for half a minute, and then takes the time since its last sign of life -
an upper bound - and the card marks such a period with ≈.
- **A refusal the panel did not get to write is written from the log.** The
sender check answers 550 and writes the row a moment later; a panel that
stopped in between - a rollout, a crash - left an application refused and
a log that did not know it. When the log tailer reads cleanup's refusal
line and, after giving the panel half a minute, finds no row for that
queue id, it writes the row from the line, marked on its page as written
from `mail.log` after a restart. A refusal at `MAIL FROM` needs no such
thing: its row is written before Postfix is answered.
## [2.5.0] - 2026-10-14
Stages 1 and 2 of the 2026-10-12 audit: mail the delivery log showed wrong,
and the boundary between tenants. Two migrations: `0007` adds the delivery
log's own measurement, `0008` makes application logins unique without regard
to case and ties every log row to its domain by id (filled once on upgrade).
### Fixed
- **A queue too large to list no longer closes its own messages.** The
reconcile sweep asks `postqueue -p` which messages Postfix still holds and
closes every `queued` row that is not among them. The listing is capped at
4 MiB, and the ids past the cap were read as gone: during a large deferred
backlog - the outage the sweep exists for - mail Postfix still held was
marked `bounced`. A listing over the cap now tells the sweep nothing, as it
already did for the notification about mail waiting too long; the Mail
queue page still shows the cut listing with its note (audit 2026-10-12, H3).
- **The sweep closes only what is still queued, and says it did.** Its close
was the tailer's plain status update, so a `sent` the log wrote an instant
earlier could be overwritten with `bounced`, and a reused queue id could
rewrite an old row of the same recipient. The sweep now closes a row by its
id and only while it is `queued`, and the row it closes carries the reason
`reconcile`; a delivery line updates only the newest row of its queue id
and recipient (audit 2026-10-12, H3).
- **A recipient with a quoted local part is followed to its delivery.** The
journal cut `"a b"@example.net` at its space, and the log tailer cut
`"a>b"@example.net` at its `>`: the row matched no delivery line and the
sweep closed it as bounced. Both now take the whole address.
- **A stranger's HELO name no longer reads as a delivery result.** The
pattern that turns a `mail.log` line into a status matched anywhere in the
line, so a port-25 client whose HELO name read
`<id>: to=<address>, status=sent` could mark the matching row delivered. It
is now matched only at the head of the line, after the host and the
program, as the other patterns of the log tailer are (audit 2026-10-12,
M12).
- **A renewed TLS certificate is served without a restart.** Postfix reads a
copy of the mounted certificate made when the container starts, and the
daily reload re-read that copy, not the renewed files: a certificate renewed
while the container ran expired on 465 and 587 at the old one's end. The
daily job now compares the mount with the copy and, when the certificate and
key belong together, renews the copy and reloads Postfix; when nothing
changed it no longer reloads (audit 2026-10-12, H2).
- **Health counts mail deferred by a panel that hangs mid-message.** The
figure "messages deferred" on the Sender check card read only smtpd's
lines, but once a message's DATA has begun it is cleanup that talks to the
check, and its warning was never counted - the very case that tells a hung
panel from a stopped one. Seen on the e2e stand with the panel frozen, and
now counted, once per message (audit 2026-10-12, M13).
- **Application logins are one name whatever their case.** Postfix compares
the SASL login with the sender map without regard to case, the panel did
not: `Ops` in one domain and `ops` in another were two applications to the
panel and one sender to Postfix, so a tenant could send with another
tenant's envelope sender. A new login is now stored in lower case, and
migration `0008` refuses two logins that differ only in case. A login
stored earlier, or imported from another instance, keeps its case, since
its client uses it (audit 2026-10-12, H4).
- **The last global administrator stays.** Deleting or demoting a global
administrator counted the others first and then wrote, so two
administrators deleting or demoting each other at the same moment could
both succeed and leave nobody able to open Server. The count is now taken
in the same transaction as the write; first-user setup is one statement
(audit 2026-10-12, M7).
- **A user is saved whole or not at all.** The users form wrote the name,
the password, the role and the assignments one after another; a failure
part way left a user renamed with the old role, or a domain administrator
with nothing assigned. Saving and creating a user are now one transaction
each, the e-mail included (audit 2026-10-12, M9).
- **The users form keeps a password changed meanwhile, and says when a
domain it offered is gone.** Saving a user wrote back the password hash
read when the form was opened, which undid a password the user changed on
Account in between; a domain deleted while the form was open ended in an
error page.
- **A domain import is checked like the forms.** The import trusted the file
for the report address, the rate limits and the client addresses: a limit
with a window of no seconds counted nothing, and an address that was not
one made the application refuse every client without a word. The file is
now held to the forms' rules before anything is created, and refused with
the form's own message, naming the field and the application; the DKIM
selector is held to DNS labels; an auto limit's numbers are worked out on
this server right after the import rather than taken from the file
(audit 2026-10-12, M10).
- **An application's numbers are its own.** Statistics, the level-2 count
and an auto limit's recalculation counted every row of the login, so an
application created with the login of a deleted one - in another domain,
even - started with the old one's traffic. They now count the login's rows
of its domain, from the application's creation on. A message with an empty
envelope sender (`MAIL FROM:<>`) is held to its domain's level-2 limit by
its login, like the application's other mail.
- **A domain added again starts with an empty log.** Send-log rows belonged
to a domain by its name, so a domain deleted and added again showed its
earlier mail to whoever it was assigned to next, and its statistics and
level-2 count went on counting it. A row now belongs to the domain of the
application that sent it, by id (migration `0008`, filled once from the
registry); the rows of a deleted domain are the global role's alone. The
inbound log follows the same rule (audit 2026-10-12, L12).
- **A refusal no longer names the other domain.** A `From:` line that named
another sender of the server was refused with that sender's domain in the
reply. The reply now stops at "it names another sender of this server";
the line as written stays in the reply and on the refused message's page.
That such a line is refused at all still tells the sending application
that the domain is hosted here (security.md, accepted risks) (audit
2026-10-12, L8).
### Added
- **The delivery log measures its own guesses** (migration `0007`). A table
`delivery_log_days` counts, per UTC day, the rows the reconcile sweep
closed, those of them whose delivery line was in `mail.log` all along (the
log tailer missed it), those a delivery line read later contradicted (the
log's status then replaces the sweep's, and the row is marked
`reconcile-contradicted`), the seconds the log tailer could not read
`mail.log`, and the rows still queued past Postfix's queue lifetime. No page
shows it: it is read off a copy of the database after a month, to decide
by a rule written in the audit whether the sweep stays (audit 2026-10-12,
H3).
## [2.4.2] - 2026-10-14
Release hygiene: stage 0 of the 2026-10-12 audit. Nothing the panel or the
mail path does changes; the release workflow, the tests, the dependencies and
the documents do.
### Fixed
- **A release image says which release it is.** The e2e gate in
`release.yml` rebuilt the image it was about to push with its own
`VERSION=e2e`, so every published image printed `e2e` for `-version` and in
the panel's footer, and the backup/restore version check compared a
backup's version with `e2e` - any backup restored into any release (audit
2026-10-12, H1). The gate now builds with the release's version
(`SELFPOST_E2E_VERSION`), the e2e checks `panel -version` and
`selfpost-backup -version` against it, and `release.yml` checks the stamp
again on the image right before pushing it. Images already published are
left as they are.
- **The documents no longer promise an image that is not there.** README,
the guide and the compose file pinned `ghcr.io/mixeme/selfpost:2.4.1`, but
no 2.x image has been published (ghcr.io holds `1.0.0` and `1.3.0`). They
now say so and show how to build the image locally under the pinned name,
which Compose and `docker run` then use without pulling (audit 2026-10-12,
M6). The pin stays: the next publication makes it true.
- **Dependencies raised**: `modernc.org/sqlite` 1.53.0 → 1.60.1 (with
`modernc.org/libc` 1.77.1), `golang.org/x/crypto` 0.54.0 → 0.58.0,
`golang.org/x/sys` 0.47.0 → 0.49.0. The three published `x/crypto` fixes
were in code SelfPost does not call (audit 2026-10-12). development.md says
the image is rebuilt after each Go patch release, which is where the
standard library's security fixes come from.
- **The e2e module is compiled on every push** (`test.yml`: `go vet` and
`go test -c` in `test/e2e`), so a type error there no longer first shows
up in `release.yml` after a release is published (audit 2026-10-12, M20).
- **`test/` is out of the image's build context** (`.dockerignore`): a
developer build after an e2e run sent the stand's leftovers - DKIM keys,
`sasldb2` - to the builder (audit 2026-10-12, L33).
## [2.4.1] - 2026-10-12
Documentation only; no product code changes.
### Added
- **A full-tree code audit** at `2.4.0`, with its plan of work:
[docs/audit-2026-10-12.md](docs/audit-2026-10-12.md). Six zones read file
by file against what `architecture.md` promises; what the readers could not
settle from source was settled on the test server. Four high findings - the
published release images are stamped `VERSION=e2e` because the e2e gate
rebuilds the image it is about to push, so the backup version check has
never checked anything; a renewed TLS certificate is not picked up while
the container runs; the queue-reconcile sweep can mark delivered mail
`bounced` while the log tailer is blind; application logins are
case-sensitive to the panel and case-insensitive to Postfix - twenty-one
medium, thirty-four low, the doc drift and the test gaps, and what was
checked and found sound. The plan is six stages, each a cut of its own;
nothing in it is agreed yet. The published `1.x` tags and images stay as
they are (owner, 2026-10-12): fixes go forward only.
## [2.4.0] - 2026-10-12
### Security
- **An application signs only as itself.** OpenDKIM chooses the key by the
`From:` header, and Postfix held only the envelope sender against the
login: an application of one sending domain could write another registered
domain, or the server's own name, into `From:` and have the message signed
with that name's key (security review of `2.3.0`, where it was recorded as
an accepted risk). The panel now holds the header to what the envelope is
held to - any address of the application's domain, or the addresses of its
list, with or without a `+extension` - and refuses anything else with
`550 5.7.1`, a domain this server does not sign for included
([plans/from-header-binding.md](docs/plans/from-header-binding.md)).
- **The check is strict.** It is the first milter of the chain, on a socket
of its own (`SENDER_CHECK_SOCKET`), attached with
`default_action=tempfail`: while the panel does not answer, Postfix defers
mail and accepts nothing unchecked. The journal milter keeps its socket and
stays fail-open.
### Added
- **Server › Health counts what the strict check costs** in a card of its
own, *Sender check*: for how long the panel did not answer while Postfix
was running, and how many submissions Postfix turned away because of it -
today, in the last 30 days and in the last 12 months (UTC days, kept a
year). The card warns when mail was deferred in the last 24 hours. The
numbers are there to decide whether the check needs a process of its own.
- **A refused row of the Outbound log says why**, in its tag: `wrong From`,
`wrong client` (the application's client IP list) or `rate-limited`; a row
refused before this version says `rejected`. A message refused for its
`From:` line shows on its page what the line said and what the application
may send as. Help has a *Sender check* topic.
- Migration `0006_sender_check.sql` (DDL only): `send_log.reason`,
`send_log.header_from`, `sender_check_days`.
### Changed
- **Mail with a `From:` the application is not bound to is refused.** What
stops working on upgrade: an application sending with the `From:` of
another of your domains (give that domain an application); an application
with a list whose envelope and `From:` addresses differ (add the second to
the list); a form that puts its visitor's address into `From:` (use
`Reply-To:`). The line is read strictly: one `From:` field, one address,
one `@` outside a quoted display name; a name with a comma or brackets has
to be quoted. Guide § Who an application may send as.
- **With the panel down, mail is deferred, not accepted.** Until now a
stopped panel cost the send log and the level-2 limits and nothing else.
- **Server › Health is laid out as two stacks of boxes numbered by
importance**, read left to right and top to bottom, each stack closed up:
Processes, Milter sockets, Sender check, TLS certificate, Hostname and
reverse DNS, Notification mail, Machine, Configuration. On a phone they
come in that order.
- A refused message is one row of the log, naming its first recipient and
how many more there were.
- `recipient_delimiter` is set to `+` by the image instead of being left to
the distribution's default; the start removes milter sockets a killed
container left behind.
## [2.3.0] - 2026-10-12
### Added
- **Schema for notifications** (migration `0004_notifications.sql`, DDL only):
a user's own switch, `users.notify`, on for everyone; `notify_state`, what
the panel has already written to its users about. The instance switch is a
settings key, on unless turned off
([plans/panel-notifications.md](docs/plans/panel-notifications.md)).
- **The panel writes to its users when something needs them.** Four things,
each said once: a sending domain's DKIM, SPF or DMARC record stops matching
(after two hourly checks in a row, and once more when it matches again); a
level-2 rate limit refuses mail (at most once a day per sending domain); the TLS
certificate is 14 days from expiry, and 3; a message has waited in the
queue for more than three days (and once more when none has). Global users
are told all of it, domain users about their own domains. Sent from
`postmaster@<SELFPOST_HOSTNAME>` by local submission, one message per
recipient; not rows of the Outbound log and counted against no limit.
- **The server's own name is signed.** The panel generates a DKIM key for
`SELFPOST_HOSTNAME` and OpenDKIM signs mail submitted locally as that name
(`non_smtpd_milters`), so that notifications arrive. The name is not a
sending domain: no application can use it as an envelope sender.
- **Notifications in the panel.** Account has *Send me notifications* beside
the e-mail; Server › Settings has the switch for everyone and the two DNS
records of the server's own name, DKIM and SPF, with the values to publish;
Server › Health has a *Notification mail* card with the state of each. Help
has a Notifications topic. Every user starts with notifications on, and so
does the instance.
### Changed
- **An account e-mail may be at a public mail host.** Account and Server ›
Users refused addresses at Gmail, Outlook, Proton and the like, a rule that
belongs to DMARC report addresses. A user's own address is now where the
panel writes to them.
- **A lookup that does not answer is not an SPF mismatch.** When the address
lookup behind an `a` or `mx` mechanism timed out, a domain's SPF record was
shown as not authorising this server. It is now a warning that the check
could not tell.
### Security
- Technical and security reviews of 2026-10-12 of the notifications, before
release: no Critical finding. The notification about refused mail was built
on the domain a client writes in `MAIL FROM`; it is now built on the
registry. Recorded in [security.md](docs/security.md) as accepted, with a
roadmap candidate (from-header-binding): an application can have a message
signed as another registered domain, or as the server's own name, by
writing it into `From:` - the first part of which has always been so.
## [2.2.0] - 2026-10-12
### Added
- **Schema: `send_log_lines`** (migration `0003_send_log_lines.sql`, DDL only).
The table that will hold what Postfix wrote about a message for as long as
its send-log row lives, with the store methods to write a line, to write it
once, to read the lines of a row and to prune
([plans/delivery-log-storage.md](docs/plans/delivery-log-storage.md)).
- **The log tailer keeps what Postfix writes about queued messages.** A
`mail.log` line headed by a queue id is stored with the line's own time, one
transaction per pass over the log; after an unclean stop a line read twice
is stored once. Mail that came in on port 25 is left out: a stranger's
message never has a send-log row, and what he can make Postfix log is not
written to the database. Lines whose queue id has no send-log row - the row
was pruned, or the message never had one (bounces) - are deleted once they
are an hour old, at start and with every six-hourly send-log sweep.
### Changed
- **A message's page keeps its delivery log after `mail.log` rotates.** The
*Delivery log* box on Outbound › Log › a message was filled by searching
the current `mail.log`, which is rotated daily, so it was empty for anything
not sent today while the row itself lives for ninety days. The page now
reads the lines the tailer stored: they are there for as long as the row
is. Postfix reuses queue ids, so a row is given only the lines dated within
its own queue lifetime. A message accepted before this version shows its
lines only while they are still in the current file, as before; the note
shown when there are none and the *Outbound log* help topic say so.
### Security
- **A full backup now carries the log lines of messages in the send log** -
client addresses, SASL logins, message ids, receivers' replies - where
`mail.log` itself is not in a backup. Recorded in
[security.md](docs/security.md) with the accepted risk of a queue-id
collision showing one message's lines on another's page. Technical and
security reviews of 2026-10-12: no Critical or High finding.
## [2.1.0] - 2026-10-12
### Security
- **The level-2 rate limit and an application's client IP list held only for
the first message of a connection.** After a refused `MAIL FROM`, a second
one on the same connection was accepted, and so was every message after the
first that a connection carried: an application over its limit could go on
sending, and one restricted to certain client addresses could send from any
other with a retry. In the code since `1.9.0` (the client IP list) and
earlier (the limit); found by the technical review of 2026-10-12 and
reproduced on the test stand. Postfix now gives the panel's milters the
client address with every message, the limit no longer depends on it, and an
application with a client IP list is refused when no address is known.
- Reviews of the inbound antispam feature before its release (technical and
security, 2026-10-12; nothing Critical or High in the feature itself). Fixed:
a client's HELO name could supply the sender, recipient and reply of an
inbound log row; the log's row cap was applied only every six hours; rows of
a connection's later messages had no client address; two changes of the
filter lists at once could leave the filter's files behind the page; a held
message was logged twice; after an unclean stop of the panel the refusals of
its last seconds were logged twice; with the spam filter down, a sender's
own verdict headers were recorded as the filter's. Recorded as accepted in
[docs/security.md](docs/security.md): an allow entry for a sender address
trusts an address anyone can claim; the list files are readable by anyone on
the host.
- `INBOUND_ANTISPAM_MILTER` and `INBOUND_JOURNAL_MILTER_SOCKET` are checked
character by character at start; the check looked at their first characters
only.
### Changed
- **Port 25 logs as `postfix/inbound`.** Its lines in the mail log (Server ›
System log) now read `postfix/inbound/smtpd[…]` where they read
`postfix/smtpd[…]`; the submission ports keep `postfix/smtpd`. This is how a
refusal on port 25 is told from one on 465 / 587.
### Added
- Schema: migration `0002_inbound_spam_log.sql`, the first after the 2.0
baseline - two empty tables for the inbound antispam journal and the filter
lists (inbound-antispam-panel). A 2.0.x database is migrated when the panel
starts.
- **Inbound mail is recorded.** With the inbound relay on, the panel runs a
second milter, the last one on port 25, and writes a row for every recipient
of a message that was accepted for an inbound domain: client address,
envelope, subject and - with a spam filter attached - the filter's action,
score and heaviest symbols. It only watches: it refuses nothing and Postfix
accepts mail when it is down.
- **Refused inbound mail is recorded too**, from the mail log: a recipient
Postfix refused at an inbound domain (unknown user, sender domain not
found), and a message the spam filter rejected, deferred, held or
discarded. The row is dated by its log line and carries the reply text. A
refusal to relay for a domain that is not an inbound one is not recorded.
- The inbound journal is pruned every six hours: rows older than 90 days
(`INBOUND_SPAM_LOG_RETENTION_DAYS`, 7-365) and anything beyond 10 000 rows.
- The antispam compose fragment mounts `antispam/rspamd/milter_headers.conf`
into the rspamd sidecar, which makes it write its verdict into
`X-Rspamd-Action` and `X-Spamd-Result`. **Copy `deploy/antispam/rspamd/`
next to the fragment before starting it** - Docker creates a directory in
place of a missing file and the sidecar then fails to read its
configuration.
- **Allow and deny lists for the spam filter**, kept by SelfPost and applied
by rspamd: sender addresses, sender domains, client addresses and networks.
They are written as six files under `data/antispam/`, which the fragment
mounts into the sidecar together with `antispam/rspamd/multimap.conf`;
rspamd rereads a changed file by itself. Deny wins over allow. The files
are there from the first start with the inbound relay on.
- **Inbound › Log**: the journal as a page - time, sender and client address,
recipient, subject and decision, filtered by domain, decision and day. The
decision opens a sheet over the page with who decided and why: the filter's
verdict and symbols, or the reply a refusal was given. A user with the
*domain* role sees the rows of the inbound domains assigned to them. For the
global role a row links to the filter lists with the sender filled in:
*Deny sender* on accepted mail, *Allow sender* on mail the filter refused -
unless the sender is spelt in a way the lists do not take.
- **Inbound › Filter lists** (global role): add and remove allow and deny
entries; the page says where the lists go and whether the filter is on to
read them. Each entry shows how often it decided a message - *Matched*, or
*never* - and its kind opens a sheet with the note, the day it was added and
the last match, so that entries nothing matches any more can be found and
removed. The count is read from the inbound log and reaches back as far as
the log does. A network of one address (`192.0.2.7/32`) is stored as the
address, so it cannot be on a list twice.
- **Server › Settings**: *Keep inbound log rows, days*, shown with the inbound
relay on.
- Help: *Inbound log* and *Filter lists*.
- New setting `ANTISPAM_DIR` (default `/data/antispam`).
- **The spam filter signs what it judged.** rspamd adds `X-SelfPost-Filter` to
every message it lets through - an HMAC of the message's queue id under a
key SelfPost creates in `data/antispam/filter.key` - and the inbound log
shows a verdict only next to a signature that fits. Nothing to configure,
but the sidecar needs the new `antispam/rspamd/milter_headers.conf` and a
restart to read it; until then accepted mail is logged as *selfpost*, with
no verdict.
- A full backup carries the inbound log, the filter lists and their files
under `data/antispam/`. It does not carry the `antispam/` directory with the
compose fragment and the rspamd configuration - copy that from
`deploy/antispam/` when restoring.
- New setting `INBOUND_JOURNAL_MILTER_SOCKET` (default
`/run/selfpost/inbound-journal.sock`).
## [2.0.1] - 2026-10-11
### Changed
- **A message that is not a report this relay keeps is dropped at once.**
One that cannot be read as an aggregate report, reports on a domain not
sent from here, or came to another domain's report address used to be put
off and offered again by Postfix for days, counted as a parse failure each
time. It is now accepted and discarded - no retry, and no bounce, since the
envelope sender of such a message is whatever its author wrote. It counts
as one parse failure and the reason is in the mail log, on the message's
own delivery line (Server › System log, `dropped: …`). Only a failure of
the database is still retried.
- **A full backup no longer holds the Postfix queue**, from the panel or from
`selfpost-backup`. It is taken while the instance runs, and a copy of a
queue Postfix is working on can hold a message twice or not at all. Mail
still waiting for delivery is not carried over; a restored instance starts
with an empty queue. To keep the queue, stop the instance and `tar` the
project directory.
- DMARC reports are kept 500 **for each domain** (and 90 days), not 500 for
the whole server - see Security.
### Fixed
- **Full backup from the panel works.** Since the Postfix queue moved under
`/data` in 1.3.0 the button answered with a 200 and ten bytes: the archive
includes the queue, Postfix keeps the queue readable by its own user alone,
and the panel is not that user ("open /data/postfix/queue/active:
permission denied" in the log). `selfpost-backup` run with `docker exec`
was not affected - it runs as root. The queue is no longer part of a backup
(see Changed), so the two archives are the same again. Everything is read
through before the download starts, so a failure is a message on the page
and not a file cut short. One thing the panel may still be unable to read
is the private key in `certs/`: it then refuses and says what to do -
`chgrp 998 certs/privkey.pem && chmod 640 certs/privkey.pem` on the host,
998 being the container's `selfpost` group, whose id is now fixed in the
image.
### Security
Review of the DMARC report path (Fable, 2026-10-11), which started to run for
the first time in 2.0.0. Nothing Critical or High.
- **A report is bounded before it is stored.** The XML comes from anyone who
knows a report address, and its 8 MiB limit left room for some 160 000
records or a single name of megabytes - rows in the database and cells of
the report page. A report holds at most 5 000 records (a larger one is
dropped), every text field is cut to 255 bytes, and a record's count is
capped so the sums cannot wrap.
- **One domain's reports cannot push out another's.** The cap of 500 was for
the whole server, so a stranger filing reports under one domain's address
could evict every other domain's history. It is counted per domain now.
- The delivery command runs one at a time, not up to a hundred in parallel.
- The reason written to the mail log for a dropped message is cut to 200
bytes and anything outside printable ASCII is replaced: parts of it are the
sender's own bytes (a header line, a file name inside an archive).
- `SELFPOST_DATA_DIR` / `SELFPOST_DB_PATH` with whitespace or `$` are refused
at start when DMARC ingest is on, instead of handing the delivery command
another path and so an empty database of its own.
### Added
- e2e: the run now ends on port 25 with both optional features on. A DMARC
report is mailed to a domain's hosted address and has to appear under
Outbound › DMARC reports; a report on another domain and a message with no
report are accepted and dropped; mail for an inbound domain reaches its
upstream and other recipients are refused. Until now the suite only read
back the generated configuration of these features.
- e2e: full backup and restore. The run takes a backup from the panel, throws
the installation away, unpacks the archive into the empty project directory
and starts it: the administrator signs in, the domain has the same DKIM key,
and mail sent by the application verifies against the record published
before the backup.
## [2.0.0] - 2026-10-11
The panel redesign. A breaking release: every panel URL changed and the
+11 -1
View File
@@ -106,13 +106,23 @@ One container, panel at `http://127.0.0.1:8080` — no reverse proxy, no TLS
files, no compose files. Good for clicking through the UI on your machine;
outbound mail will not reach the real internet without DNS, PTR, and port 25.
> **The 2.x line is not published on ghcr.io yet** (the newest image there is
> `1.3.0`). Until it is, build the image from this checkout under the name the
> commands below and `deploy/docker-compose.yml` use; Docker then takes the
> local image and pulls nothing:
>
> ```sh
> docker build -f build/Dockerfile --build-arg VERSION=2.5.1 \
> -t ghcr.io/mixeme/selfpost:2.5.1 .
> ```
```sh
docker run --rm -d --name selfpost-try \
-p 127.0.0.1:8080:8080 \
-e SELFPOST_HOSTNAME=mail.local.test \
-e PANEL_COOKIE_SECURE=false \
-v selfpost-try-data:/data \
ghcr.io/mixeme/selfpost:2.0.0
ghcr.io/mixeme/selfpost:2.5.1
```
**Get the setup URL** (pick one):
+22 -2
View File
@@ -36,6 +36,16 @@ ENV DEBIAN_FRONTEND=noninteractive
# Preseed Postfix so its install is non-interactive and yields a working
# main.cf. The real relay configuration is generated by the panel.
# (postfix 3.7.11-0+deb12u1, debian/templates: postfix/main_mailer_type and
# postfix/mailname are the questions the install asks.)
# What else arrives through Depends, and is used here: openssl, by
# ca-certificates ("Depends: openssl (>= 1.1.1)", packages.debian.org/bookworm/
# ca-certificates) - postfix-cert-reload.sh; python3, by supervisor (supervisor
# 4.2.5-1, debian/control, ${python3:Depends}) - crashexit.py. sasl2-bin brings
# saslpasswd2 and libsasl2-modules the PLAIN and LOGIN plugins; the sasldb
# auxprop plugin is in libsasl2-modules-db, a Depends of libsasl2-2
# (cyrus-sasl2 2.1.28+dfsg-10, debian/control and debian/*.install). db-util
# ships /usr/bin/db_dump (packages.debian.org/bookworm/amd64/db-util/filelist).
RUN echo "postfix postfix/mailname string localhost" | debconf-set-selections \
&& echo "postfix postfix/main_mailer_type string Internet Site" | debconf-set-selections \
&& apt-get update \
@@ -62,7 +72,12 @@ RUN useradd --system --no-create-home --shell /usr/sbin/nologin panel
# entrypoint.sh) — lets OpenDKIM read the panel-owned keys and lets Postfix read
# the sasldb2/sender map, and lets the panel reach the supervisor control socket
# to signal OpenDKIM/Postfix reloads without any process running as root.
RUN groupadd --system selfpost \
# The id is fixed, and it is the one the group has always had: an operator
# gives the TLS private key on the host to this group by number, so that the
# panel can put it into a full backup (guide § Taking a full backup).
# groupadd(8) refuses a --gid already in use, so a package that took 998 would
# fail this build rather than share the group.
RUN groupadd --system --gid 998 selfpost \
&& usermod -aG selfpost panel \
&& usermod -aG selfpost opendkim \
&& usermod -aG selfpost postfix
@@ -73,9 +88,14 @@ RUN groupadd --system selfpost \
# with `no-new-privileges`, which disables setgid transitions, so the panel
# would always see "Permission denied". Making `panel` a real member of postdrop
# grants the same access without relying on a setgid escalation the hardening
# deliberately forbids. postdrop membership is read-side only: it does not let
# deliberately forbids. The same membership is what lets the panel submit its
# own notifications with sendmail(1) - postdrop, too, is setgid and would fail
# to write the maildrop directory (internal/notify/mailer.go). It does not let
# the panel bypass any Postfix restriction that a local user does not already
# have through the world-executable sendmail.
# (Postfix 3.7.11, conf/postfix-files: postdrop and postqueue are
# root:$setgid_group 2755. Linux, Documentation/userspace-api/no_new_privs.rst:
# with no_new_privs, execve() no longer changes the gid for a setgid file.)
RUN usermod -aG postdrop panel
# Runtime directories: milter sockets and the consolidated persistent root.
+37 -12
View File
@@ -1,18 +1,32 @@
#!/usr/bin/env python3
"""Supervisor event listener: bring the container down on unrecoverable failure.
If any managed program exhausts its restart retries and enters FATAL, this
listener signals supervisord (its parent) to terminate, so the whole container
exits and Docker's `restart` policy can recreate it cleanly — rather than
lingering "alive" with a dead Postfix/OpenDKIM/panel.
If a managed program of the mail path exhausts its restart retries and
enters FATAL, this listener signals supervisord (its parent) to terminate, so
the whole container exits and Docker's `restart` policy starts it again —
rather than lingering "alive" with a dead Postfix/OpenDKIM/panel. logrotate is
not on the mail path: its FATAL (a config it would ignore) stays on
Server > Health, and mail keeps flowing (assumptions C-V5).
Communication with supervisord uses the event listener protocol over stdin and
stdout, so stdout must carry only protocol tokens.
supervisor 4.2.5, docs/events.rst, "Event Listener Notification Protocol": the
listener writes READY\\n, reads one header line of key:value tokens whose len
is the payload's length, reads the payload, and answers "RESULT 2\\nOK".
A PROCESS_STATE_FATAL payload is a token set, "processname:cat groupname:cat
from_state:BACKOFF" (docs/events.rst; supervisor/events.py,
ProcessStateEvent.payload()).
supervisord starts the listener as its own child, and on SIGTERM it shuts down
(docs/running.rst, "Signals"); it is PID 1 here (build/entrypoint.sh, exec).
"""
import os
import signal
import sys
# Programs whose FATAL does not take the container down (build/supervisord.conf).
SPARED = {"logrotate"}
def write_stdout(s):
sys.stdout.write(s)
@@ -29,16 +43,27 @@ def main():
return
headers = dict(pair.split(":", 1) for pair in line.split())
payload_len = int(headers.get("len", 0))
if payload_len:
sys.stdin.read(payload_len)
payload = sys.stdin.read(payload_len) if payload_len else ""
tokens = dict(
pair.split(":", 1) for pair in payload.split() if ":" in pair
)
name = tokens.get("processname", "")
# We only subscribe to PROCESS_STATE_FATAL, so any event means a managed
# program can no longer be restarted. Take the container down.
sys.stderr.write(
"crashexit: a managed process entered FATAL; shutting down container\n"
)
sys.stderr.flush()
os.kill(os.getppid(), signal.SIGTERM)
# program can no longer be restarted.
if name in SPARED:
sys.stderr.write(
"crashexit: %s entered FATAL; it is not on the mail path, the "
"container stays up\n" % name
)
sys.stderr.flush()
else:
sys.stderr.write(
"crashexit: %s entered FATAL; shutting down container\n"
% (name or "a managed process")
)
sys.stderr.flush()
os.kill(os.getppid(), signal.SIGTERM)
write_stdout("RESULT 2\nOK")
+52 -8
View File
@@ -62,6 +62,8 @@ esac
# trees under /data as other users. Go's testing.TempDir is 0700, and a bare
# chown would leave that mode in place — opendkim then cannot read KeyTable and
# the container crash-loops (e2e TestHostnameGate/valid_hostname_starts).
# (Go: testing.TempDir uses os.MkdirTemp, src/os/tempfile.go: "created with
# mode 0o700".)
chown panel:panel /data
chmod 755 /data
# Restored backups or previously-created state may contain panel-owned files
@@ -76,7 +78,8 @@ find /data -mindepth 1 -maxdepth 1 ! -user panel ! -name log ! -name postfix -ex
# tree on every start so it is correct whether /data is fresh, restarted, or
# just restored from a backup:
# - group `selfpost` + setgid on directories so anything the panel creates
# inherits the shared group OpenDKIM traverses;
# inherits the shared group OpenDKIM traverses (inode(7), "The
# set-group-ID bit": files created in such a directory take its group);
# - private keys and tables group-readable (0640);
# - both table files present (empty is fine) BEFORE OpenDKIM starts, so the
# daemon comes up cleanly with no domains yet.
@@ -112,6 +115,15 @@ for f in relay_domains transport relay_recipients tls_policy; do
done
chown panel:selfpost /data/postfix
chmod 2750 /data/postfix
# Filter lists for the external spam filter (written by the panel when
# INBOUND_RELAY_ENABLE=true, read by the antispam sidecar, which mounts this
# directory). The sidecar runs under a user of its own in another container,
# so the directory and its files are readable by anyone: they hold sender
# addresses and networks, no secrets.
mkdir -p /data/antispam
chown panel:selfpost /data/antispam
chmod 0755 /data/antispam
chown panel:selfpost /data/postfix/sender_login_maps
chmod 0640 /data/postfix/sender_login_maps
@@ -126,6 +138,11 @@ chmod 0640 /data/postfix/sender_login_maps
# directory keeps the shared group on anything created inside it later, and
# 2750 keeps it group-traversable but not group-writable — logrotate refuses to
# rotate a log whose directory is writable by a non-root group.
# Postfix 3.7.11: src/util/logwriter.c, logwriter_open_or_die(), creates the
# file 0644, under master's umask(077) (src/master/master.c) - 0600. logrotate
# 3.21.0, logrotate.c, findNeedRotating(): "skipping ... because parent
# directory has insecure permissions" when the directory's group is not root
# and has write permission.
mkdir -p /data/log
[ -e /data/log/mail.log ] || : > /data/log/mail.log
chown -R postfix:selfpost /data/log
@@ -133,8 +150,8 @@ chmod 2750 /data/log
find /data/log -type f -exec chmod 0640 {} +
# Milter socket directories. Postfix (user `postfix`) must actually CONNECT to
# both milter sockets — OpenDKIM's and the panel's journal-milter — not just
# probe them at start-up. The sockets are created by the opendkim and panel
# the milter sockets — OpenDKIM's and the panel's — not just probe them at
# start-up. The sockets are created by the opendkim and panel
# users respectively, so bridge them to `postfix` through the shared `selfpost`
# group: group-owned + setgid dirs mean each socket created inside inherits
# group `selfpost`, and group-traversable (2750) lets postfix reach it. Without
@@ -144,6 +161,15 @@ mkdir -p /run/opendkim /run/selfpost
chown opendkim:selfpost /run/opendkim
chown panel:selfpost /run/selfpost
chmod 2750 /run/opendkim /run/selfpost
# A container that was killed leaves its sockets behind, and a file nobody
# listens on would satisfy postfix-wrapper.sh's wait: Postfix would start
# before the sender check does and defer mail until it is there. The check's
# alive mark goes too: it tells the downtime of the panel restarting inside a
# running container, and a restart of the whole container is not the check's
# downtime (guide § Health) - a mark left from before would count the whole
# outage when supervisord cannot say since when Postfix runs.
# (test(1), -S: true for any socket file, whether or not anything listens.)
rm -f /run/opendkim/*.sock /run/selfpost/*.sock /run/selfpost/*.alive
# Generate the outbound-relay Postfix configuration from the environment (spec
# 5). Kept out of the image build so cert paths, rate limits, hostname and the
@@ -151,11 +177,12 @@ chmod 2750 /run/opendkim /run/selfpost
# start the same way the /data normalisation above is.
/usr/local/bin/postfix-config.sh
# Initialise the persistent queue tree on first start or after restore. postfix
# set-permissions reads queue_directory from main.cf (set by postfix-config.sh).
if [ ! -d /data/postfix/queue/active ]; then
postfix set-permissions
fi
# The queue tree, on a first start or after a restore, is already built here:
# postfix-config.sh ends with "postfix check", whose check-fatal runs
# "post-install create-missing" (Postfix 3.7.11, conf/postfix-script;
# conf/postfix-files, "$queue_directory/active:d:$mail_owner:-:700:ucr").
# A "postfix set-permissions" for a missing active/ stood here and never ran
# (assumptions C-P34).
chown -R postfix:postfix /data/postfix/queue
# The recursive chown above would leave public/ and maildrop/ in group postfix.
# Postfix keeps those two in its setgid group: postqueue and postdrop are setgid
@@ -167,6 +194,23 @@ chgrp postdrop /data/postfix/queue/public /data/postfix/queue/maildrop
# It would also leave the queue directory itself and pid/ owned by postfix.
# Postfix keeps those two owned by root (master, which runs as root, is the only
# writer) and logs "warning: not owned by root" for each at every start.
# (Postfix 3.7.11, conf/postfix-files: $queue_directory root 755, maildrop
# $setgid_group 730, public $setgid_group 710, pid root 755, the other queue
# directories $mail_owner 700; conf/postfix-script, "check-warn": "not owned by
# root".)
chown root /data/postfix/queue /data/postfix/queue/pid
# [program:postfix]'s startsecs (build/supervisord.conf): longer than the
# wrapper's wait for the milter sockets, so its exit at the end of the wait is
# one supervisord counts as too quick. A value that is not a number stops the
# start here.
wait_timeout="${MILTER_WAIT_TIMEOUT:-30}"
case "$wait_timeout" in
'' | *[!0-9]*)
echo "entrypoint: MILTER_WAIT_TIMEOUT must be a whole number of seconds, got '$wait_timeout'" >&2
exit 1
;;
esac
export POSTFIX_STARTSECS=$((wait_timeout + 10))
exec /usr/bin/supervisord -c /etc/supervisor/supervisord.conf
+19 -2
View File
@@ -7,18 +7,35 @@
# before the reload is lost. `create` (rather than `nocreate`) matters here
# beyond timing: a postlogd-triggered recreate lands the file at 0600 owned by
# postfix, which the unprivileged panel process cannot read — confirmed on a
# live container — so logrotate must be the one to create it.
# live container (date not recorded), and so in the source: Postfix 3.7.11,
# src/util/logwriter.c, logwriter_open_or_die(), creates it 0644 under master's
# umask(077), src/master/master.c — so logrotate must be the one to create it.
# The reload: Postfix 3.7.11, conf/postfix-script, "logrotate" (mv, then kill
# -HUP master); master_conf.c, master_config() -> master_restart_service() ->
# master_status_cleanup(), on which postlogd exits (src/master/dgram_server.c,
# dgram_server_abort() on MASTER_STATUS_FD), and the next postlogd opens the
# path anew (src/postlogd/postlogd.c, logwriter_open_or_die()).
#
# logrotate itself only rotates once the configured "daily" period has elapsed
# (tracked in /var/lib/logrotate/status), so it is safe to invoke this more
# often than daily — polling merely bounds how late a legitimate rotation runs.
# (logrotate 3.21.0-1: debian/rules, --with-state-file-path=/var/lib/logrotate/
# status. That file is in the container's own filesystem: a recreated container
# starts without it, and logrotate.c, newState(), records a log it has no state
# for as rotated today, so the first rotation after a recreate comes a day
# later.)
set -eu
CONFIG=/etc/logrotate.d/mail
INTERVAL="${LOGROTATE_INTERVAL_SECONDS:-21600}"
# logrotate refuses configs writable by group or others and exits 0 while
# ignoring them — fail here so supervisord reports the fault.
# ignoring them — fail here so supervisord reports the fault. (logrotate
# 3.21.0, config.c, readConfigFile(): as root, mode & 0022 gives "Ignoring ...
# because it is writable by group or others." and returns 0, not an error; a
# config not owned by root is ignored the same way. "Potentially dangerous
# mode" is only a warning there, for any mode & 07533 other than 0400 - this
# loop chooses to treat it as fatal too.)
logrotate_config_ok() {
mode=$(stat -c '%a' "$CONFIG")
mode=${mode#0}
+3
View File
@@ -4,6 +4,9 @@
# build/postfix-config.sh (maillog_file) and build/entrypoint.sh: postlogd
# writes it as user postfix, the unprivileged panel reads it through the shared
# selfpost group, hence create 0640 postfix selfpost rather than 0644 root root.
# (logrotate 3.21.0, logrotate.8: "create mode owner group" makes the new log
# right after the rotation; "delaycompress" leaves the previous log, .1,
# uncompressed until the next cycle.)
/data/log/mail.log {
daily
rotate 14
+18 -3
View File
@@ -3,11 +3,16 @@
# Signing is strictly per-domain: each sending domain has its own key pair and
# selector. The panel generates keys under /data/opendkim/keys/<domain>/ and
# keeps KeyTable/SigningTable in sync as domains are added or removed, then asks
# supervisord to send this process SIGUSR1 to reload the tables.
# supervisord to send this process SIGUSR1 to reload the tables. (OpenDKIM
# 2.11.0~beta2, opendkim/opendkim.8.in: on SIGUSR1 the configuration file is
# re-read; opendkim/opendkim.c, dkimf_config_reload() -> dkimf_config_load(),
# opens SigningTable and KeyTable again with dkimf_db_open().)
#
# Both table files are created (empty) by entrypoint.sh before OpenDKIM starts,
# so the daemon comes up cleanly on a fresh /data with no domains yet and simply
# signs nothing until the first domain is added.
# signs nothing until the first domain is added. (opendkim/opendkim-db.c,
# dkimf_db_open(): a file or refile table is read line by line; an empty file
# is an empty table, not an error.)
Syslog no
UMask 007
@@ -15,15 +20,25 @@ Mode s
# SigningTable uses refile: so the "*@example.com" left-hand patterns match any
# local-part for a domain; KeyTable maps each domain to its key and selector.
# (opendkim/opendkim.conf.5.in, SigningTable: with "refile" the keys are
# wildcard patterns matched against the From: address, in file order.)
KeyTable /data/opendkim/KeyTable
SigningTable refile:/data/opendkim/SigningTable
# The private keys are owned by `panel` and read by `opendkim` through the shared
# `selfpost` group, so they are deliberately group-readable (mode 0640). That is
# safe on this single-tenant, private bind mount, but it trips OpenDKIM's default
# key-safety check, so the check is disabled here by design.
# key-safety check, so the check is disabled here by design. (opendkim/
# opendkim.c, dkimf_checkfsnode(): a key file must be owned by root or by the
# executing uid, and a group-readable one must be in a group of no other user -
# `panel` owns these and `selfpost` has three members.)
RequireSafeKeys no
# UMask 007 above leaves the socket group read-write (opendkim.conf.5.in,
# UMask: it applies to the socket); its group is selfpost by the setgid
# /run/opendkim (build/entrypoint.sh). UserID: the process takes the user's
# groups (opendkim.conf.5.in, UserID); with Background no and -f
# (build/supervisord.conf, opendkim.8.in) the supervised pid is OpenDKIM itself.
Socket local:/run/opendkim/opendkim.sock
PidFile /run/opendkim/opendkim.pid
UserID opendkim
+82 -17
View File
@@ -1,25 +1,90 @@
#!/bin/sh
# Periodic `postfix reload` so refreshed TLS certificates are picked up (spec
# 5.2 p.4). The reverse-proxy renews the PEM files in the read-only mount every
# few months; Postfix only re-reads them on reload. A simple daily reload is
# more than enough (a day of staleness is harmless) and far simpler than an
# inotify watcher — the spec explicitly prefers this.
# Picks up a renewed TLS certificate while the container runs (spec 5.2 p.4;
# audit 2026-10-12, H2).
#
# Runs under supervisord as root, so it can reload Postfix directly. It sleeps
# first, then reloads in a loop: no reload at container start (the wrapper is
# still bringing Postfix up then) and none until at least one interval has
# passed. A reload is harmless when nothing changed.
set -eu
# Postfix does not read the mounted files: postfix-config.sh copies them at
# start into /etc/postfix/tls-internal, because Postfix wants the key owned by
# root with mode 0600, which a read-only bind mount owned by the host's user
# cannot give. A renewal of the mounted files therefore reaches Postfix only
# through a new copy. Every interval this compares the mounted files with the
# copy; when they differ it copies both beside their targets, checks that the
# copies - not the mounted files, which a renewer may be rewriting - are a
# certificate and the key that belongs to it, renames both over the targets
# and reloads Postfix. When nothing changed it does nothing.
# (Postfix 3.7.11: smtpd loads the certificate in pre_jail_init(), once per
# process, src/smtpd/smtpd.c; "postfix reload" is SIGHUP to master,
# conf/postfix-script, after which the old smtpd processes exit on the closed
# status pipe, src/master/master_conf.c, master_config() ->
# master_restart_service(), and new ones load the files again.)
#
# Runs under supervisord as root. It sleeps first: at container start the
# wrapper is still bringing Postfix up. It never exits non-zero, so a failure
# does not trip the crashexit listener; it is logged and tried again.
set -u
umask 077 # the key's copy is never readable by others, not even for an instant
INTERVAL="${TLS_RELOAD_INTERVAL_SECONDS:-86400}"
SRC_CERT="${TLS_CERT_FILE:-/etc/postfix/tls/fullchain.pem}"
SRC_KEY="${TLS_KEY_FILE:-/etc/postfix/tls/privkey.pem}"
DIR=/etc/postfix/tls-internal
CERT="$DIR/fullchain.pem"
KEY="$DIR/privkey.pem"
PENDING=0 # a copy was renewed and Postfix has not been reloaded since
reload() {
if postfix reload; then
echo "cert-reload: postfix reloaded ($1)"
PENDING=0
else
echo "cert-reload: postfix reload failed, will try again after ${INTERVAL}s" >&2
fi
}
# same_pair says whether a certificate file and a private key file belong
# together.
same_pair() {
cert_pub=$(openssl x509 -noout -pubkey -in "$1" 2>/dev/null) || return 1
key_pub=$(openssl pkey -pubout -in "$2" 2>/dev/null) || return 1
[ -n "$cert_pub" ] && [ "$cert_pub" = "$key_pub" ]
}
refresh() {
if [ "$PENDING" = 1 ]; then
reload "renewed certificate, earlier reload had failed"
fi
if [ "$(postconf -h smtpd_tls_cert_file 2>/dev/null)" != "$CERT" ]; then
# Postfix reads the mounted files themselves (they were missing at
# start, so no copy was made): a reload is what picks up a change.
reload "periodic TLS refresh"
return
fi
if [ ! -r "$SRC_CERT" ] || [ ! -r "$SRC_KEY" ]; then
echo "cert-reload: cannot read ${SRC_CERT} or ${SRC_KEY}; Postfix keeps the certificate it has" >&2
return
fi
if cmp -s "$SRC_CERT" "$CERT" && cmp -s "$SRC_KEY" "$KEY"; then
return
fi
rm -f "$CERT.new" "$KEY.new"
if ! cp "$SRC_CERT" "$CERT.new" || ! cp "$SRC_KEY" "$KEY.new"; then
rm -f "$CERT.new" "$KEY.new"
echo "cert-reload: copying the renewed certificate failed, will try again after ${INTERVAL}s" >&2
return
fi
if ! same_pair "$CERT.new" "$KEY.new"; then
rm -f "$CERT.new" "$KEY.new"
echo "cert-reload: the mounted certificate and key differ from Postfix's copy but do not belong together; will look again after ${INTERVAL}s" >&2
return
fi
chown root:root "$CERT.new" "$KEY.new"
chmod 0644 "$CERT.new"
chmod 0600 "$KEY.new"
mv -f "$KEY.new" "$KEY" && mv -f "$CERT.new" "$CERT"
PENDING=1
reload "renewed certificate"
}
while true; do
sleep "${INTERVAL}"
if postfix reload; then
echo "cert-reload: postfix reloaded (periodic TLS refresh)"
else
# Never exit non-zero: a transient reload failure must not trip the
# crashexit listener and take the container down. Log and retry next cycle.
echo "cert-reload: postfix reload failed, will retry after ${INTERVAL}s" >&2
fi
refresh
done
+230 -24
View File
@@ -23,6 +23,19 @@ set -eu
# splits a restriction list on commas as well as on whitespace, so every value
# passed here is written with commas only; a value with whitespace is a bug in
# this script and stops the container.
#
# Postfix 3.7.11, man/man5/master.5, "-o name=value (short form)", NOTE 2: no
# whitespace in a short-form value (the long form "-o { name = value }" would
# keep it; this script does not write it). src/postconf/postconf_edit.c,
# pcf_edit_master(): "-Pe does not accept whitespace in parameter value".
# src/smtpd/smtpd_check.c, smtpd_check_parse(), and src/milter/milter.c,
# milter_new(), split their lists on CHARS_COMMA_SP: commas or whitespace.
#
# master.5, SYNTAX: a line that starts with whitespace continues a logical
# line, so the "-o" line awk inserts below the header belongs to the service.
# A later postconf -F/-P re-parses a matched entry with its continuation lines
# (postconf_edit.c, pcf_gobble_cf_line()) and prints it whole again, keeping
# these options.
master_cf_set_smtp_inet_o() {
param="$1"
value="$2"
@@ -62,11 +75,19 @@ master_cf_set_smtp_inet_o() {
# sasldb2 accounts are looked up under. The panel creates accounts under realm
# $SELFPOST_HOSTNAME (SASL_REALM), so myhostname MUST match or authentication
# fails. Fall back to the container hostname only outside a real deployment.
# Postfix 3.7.11, src/xsasl/xsasl_cyrus_server.c, xsasl_cyrus_server_create():
# sasl_server_new() gets var_myhostname as serverFQDN; Cyrus SASL 2.1.28,
# plugins/sasldb.c, sasldb_auxprop_lookup(): with no user realm the account is
# looked up under serverFQDN.
HOSTNAME_VALUE="${SELFPOST_HOSTNAME:-$(hostname -f 2>/dev/null || hostname)}"
# TLS material supplied by the reverse-proxy through a read-only bind mount. The
# relay requires TLS on 465; if these files are absent the master still starts
# but TLS handshakes on 465 fail until they appear.
# but TLS handshakes on 465 fail until they appear. Postfix 3.7.11,
# src/smtpd/smtpd.c: pre_jail_init() only warns when the TLS context cannot be
# made, and smtpd_proto() then drops each wrapper-mode connection ("Wrapper-mode
# request dropped ... TLS context initialization failed"); every new smtpd
# process loads the files again in pre_jail_init().
#
# Postfix insists the private key is root-owned and mode 0600. The bind mount
# is often :ro and owned by the host user (e2e TempDir / CI runner UID), so
@@ -93,13 +114,18 @@ fi
RATE_MSGS="${RATE_LIMIT_MESSAGES_PER_IP:-100}"
RATE_WINDOW="${RATE_LIMIT_WINDOW_SECONDS:-3600}"
# Milter sockets: OpenDKIM (signing, strict) and the panel journal-milter
# (monitoring, fail-open). Fixed container paths, matched by postfix-wrapper.sh.
# Milter sockets: OpenDKIM (signing, strict), the panel's sender check
# (strict) and its journal-milter (monitoring, fail-open). Fixed container
# paths, matched by postfix-wrapper.sh.
OPENDKIM_SOCK="${OPENDKIM_SOCKET:-/run/opendkim/opendkim.sock}"
JOURNAL_SOCK="${JOURNAL_MILTER_SOCKET:-/run/selfpost/journal.sock}"
# The panel's sender check (the From: line against the login, strict).
SENDER_SOCK="${SENDER_CHECK_SOCKET:-/run/selfpost/sender.sock}"
# Persistent panel-managed sender map; texthash needs no postmap, so the
# unprivileged panel can rewrite it and just ask for a reload.
# unprivileged panel can rewrite it and just ask for a reload. Postfix 3.7.11,
# html/DATABASE_README.html, "texthash": no postmap(1), and the file is read
# once by each process - a reload makes the new processes read it again.
SENDER_LOGIN_MAPS="${POSTFIX_SENDER_LOGIN_MAPS:-/data/postfix/sender_login_maps}"
# Transit mail queue under /data so deferred/active messages survive container
# recreate (architecture.md § Persistence). Distinct from sender_login_maps.
@@ -117,6 +143,9 @@ SUBMISSION_ENABLE="${SUBMISSION_ENABLE:-false}"
INBOUND_RELAY_ENABLE="${INBOUND_RELAY_ENABLE:-false}"
INBOUND_ANTISPAM_MILTER="${INBOUND_ANTISPAM_MILTER:-}"
INBOUND_ANTISPAM_MILTER_ACTION="${INBOUND_ANTISPAM_MILTER_ACTION:-accept}"
# The panel's inbound journal milter (monitoring, fail-open), on port 25 only
# and only while the inbound relay is on. Matched by postfix-wrapper.sh.
INBOUND_JOURNAL_SOCK="${INBOUND_JOURNAL_MILTER_SOCKET:-/run/selfpost/inbound-journal.sock}"
INBOUND_RATE_LIMIT_MESSAGES_PER_IP="${INBOUND_RATE_LIMIT_MESSAGES_PER_IP:-20}"
INBOUND_MESSAGE_SIZE_LIMIT="${INBOUND_MESSAGE_SIZE_LIMIT:-26214400}"
RELAY_DOMAINS_MAP="${POSTFIX_RELAY_DOMAINS:-/data/postfix/relay_domains}"
@@ -141,10 +170,22 @@ DMARC_RELAY_DOMAINS_MAP="${POSTFIX_DMARC_RELAY_DOMAINS:-/data/postfix/dmarc_rela
# directory and the file with the ownership postlogd writes and the panel reads.
MAIL_LOG_PATH="${MAIL_LOG:-/data/log/mail.log}"
# /etc/postfix outlives a restart of the container, so a cleanup limit set for
# port 25 before is taken back here and set again below when it still applies.
postconf -PX "cleanup/unix/message_size_limit" 2>/dev/null || true
# --- main.cf -----------------------------------------------------------------
# maillog_file lives under the persistent /data bind mount (architecture.md §
# Log tailer). Postfix's default maillog_file_prefixes are only /var and
# /dev/stdout — without /data, `postfix check` rejects the path.
# inet_protocols=all in a bridge network without IPv6: Postfix 3.7.11,
# src/util/inet_proto.c, inet_proto_init(), keeps IPv6 when socket(PF_INET6)
# succeeds, and src/smtp/smtp_connect.c, smtp_connect_inet(), goes on to
# the next address when a connection fails - an AAAA that cannot be reached
# costs one failed attempt, then IPv4. Postfix
# 3.7.11, src/global/mail_params.h, DEF_MAILLOG_FILE_PFXS "/var, /dev/stdout";
# src/global/maillog_client.c, maillog_client_init(): fatal "does not match any
# prefix", in every Postfix program that logs, postfix(1) itself included.
postconf -e \
"myhostname=${HOSTNAME_VALUE}" \
"maillog_file=${MAIL_LOG_PATH}" \
@@ -154,17 +195,26 @@ postconf -e \
"inet_interfaces=all" \
"inet_protocols=all"
# postlogd is mandatory whenever maillog_file is set (MAILLOG_README). Debian's
# postlogd is mandatory whenever maillog_file is set (Postfix 3.7.11,
# html/MAILLOG_README.html, "Configuring logging to file"; conf/postfix-script,
# "check-fatal": "missing 'postlog' service in master.cf"). Debian's
# stock master.cf usually has it; pin it explicitly so a stripped/upgraded
# image cannot lose the service.
postconf -M "postlog/unix-dgram=postlog unix-dgram n - n - 1 postlogd"
# This is an outbound relay: no local delivery, no per-user aliases. Empty
# these so a misfiled recipient never gets delivered locally.
# these so a misfiled recipient never gets delivered locally. An empty
# mydestination is not enough: an address literal of an interface the
# container listens on - x@[127.0.0.1] - is local too (Postfix 3.7.11,
# src/global/resolve_local.c, resolve_local(), own_inet_addr()), so local(8)
# itself is replaced by a refusal, as Postfix's null-client example does
# (README_FILES/STANDARD_CONFIGURATION_README, "local_transport = error:local
# mail delivery is disabled"; assumptions C-P8).
postconf -e \
"local_recipient_maps=" \
"alias_maps=" \
"alias_database="
"alias_database=" \
"local_transport=error:local mail delivery is disabled"
# Outbound delivery: straight to the recipient MX, opportunistic TLS.
postconf -e \
@@ -172,7 +222,10 @@ postconf -e \
"smtp_tls_note_starttls_offer=yes"
# TLS server material shared by every inbound service. auth_only guarantees
# credentials are never accepted before TLS is up on any port.
# credentials are never accepted before TLS is up on any port. Postfix 3.7.11,
# src/smtpd/smtpd_sasl_proto.c, smtpd_sasl_auth_cmd(): "504 5.5.4 Encryption
# required" while auth_only is set and TLS is not up; src/smtpd/smtpd.c,
# pre_jail_init(): wrappermode implies enforce, and enforce implies auth_only.
postconf -e \
"smtpd_tls_cert_file=${TLS_CERT}" \
"smtpd_tls_key_file=${TLS_KEY}" \
@@ -185,6 +238,8 @@ postconf -e \
# uses for sender_login_maps is the BARE login the panel writes into the map;
# the sasldb2 lookup still resolves because Postfix hands Cyrus $myhostname as
# the server realm, which equals the realm the accounts were created under.
# (Postfix 3.7.11, src/smtpd/smtpd_sasl_glue.c, smtpd_sasl_activate(): an empty
# smtpd_sasl_local_domain is a NULL user realm; see HOSTNAME_VALUE above.)
postconf -e \
"smtpd_sasl_auth_enable=yes" \
"smtpd_sasl_type=cyrus" \
@@ -194,17 +249,30 @@ postconf -e \
"smtpd_sasl_tls_security_options=noanonymous" \
"broken_sasl_auth_clients=yes"
# Sender binding (the critical anti-spoofing control). texthash resolves the
# full address first, then the "@domain" wildcard, so both address modes work
# from the same map.
# Sender binding (the critical anti-spoofing control). smtpd looks the sender
# up in this map with mail_addr_find(), the order being smtpd's and not
# texthash's: the full address, the address without its extension, then the
# "@domain" wildcard - so both address modes work from the same map (Postfix
# 3.7.11, src/smtpd/smtpd_check.c, reject_auth_sender_login_mismatch();
# src/global/mail_addr_find.c, mail_addr_find_opt() with MA_FIND_DEFAULT).
#
# recipient_delimiter is said here and not left to the distribution's main.cf:
# it is what lets an application listed as invoices@ send as invoices+march@,
# and the panel's sender check makes the same second try for the From: line
# (extensionDelimiter in internal/milter/sender.go) - the two must agree.
# (mail_addr_find_opt() strips the extension at var_rcpt_delim,
# strip_addr_internal(), for its second try.)
postconf -e \
"smtpd_sender_login_maps=texthash:${SENDER_LOGIN_MAPS}" \
"recipient_delimiter=+" \
"queue_directory=${QUEUE_DIR}"
# Restrictions: authenticated clients only, no relay to foreign destinations,
# and every authenticated sender address must be owned by its login. NO
# permit_mynetworks anywhere — authorisation is by credentials, never by
# network. This is what makes an open relay impossible.
# network. This is what makes an open relay impossible. (Postfix 3.7.11,
# src/smtpd/smtpd_check.c, smtpd_check_rcpt(): the relay and recipient lists
# are evaluated in turn and the first REJECT ends them.)
postconf -e \
"smtpd_helo_required=yes" \
"smtpd_relay_restrictions=permit_sasl_authenticated, reject_unauth_destination" \
@@ -212,7 +280,11 @@ postconf -e \
"smtpd_sender_restrictions=reject_sender_login_mismatch, permit"
# Level-1 rate limit by client IP. Backstop that keeps working even if the
# journal-milter (level 2) is down.
# journal-milter (level 2) is down. Postfix 3.7.11, src/smtpd/smtpd.c,
# mail_cmd(): "450 4.7.1 Error: too much mail from", except for clients in
# smtpd_client_event_limit_exceptions (default $mynetworks; Debian's debconf
# default, debian/templates "postfix/mynetworks", is loopback only).
# anvil_rate_time_unit is one global window for every anvil limit (postconf(5)).
postconf -e \
"smtpd_client_message_rate_limit=${RATE_MSGS}" \
"anvil_rate_time_unit=${RATE_WINDOW}s"
@@ -221,11 +293,38 @@ postconf -e \
# if it is unreachable, defer rather than send unsigned). The journal-milter is
# monitoring only and is fail-open (default_action=accept): its failure must
# never block the relay. Per-milter settings use Postfix 3.0+ brace syntax.
#
# The sender check comes first and is strict as well: it holds the From: line
# against the login, which is what OpenDKIM then chooses the key by, so a
# message it could not judge must not reach the signer. While the panel does
# not answer on that socket, mail is deferred (docs/security.md).
#
# non_smtpd_milters is the chain for mail submitted locally with sendmail(1) -
# the panel's own notifications, sent as postmaster@ the hostname. OpenDKIM
# signs them with the key of the server's own name; the journal-milter is not
# on this chain, since they are not an application's mail. Bounces Postfix
# generates itself do not pass it (internal_mail_filter_classes is empty).
#
# Postfix 3.7.11: html/MILTER_README.html, "Different settings for different
# Milter applications" (the braces, Postfix 3.0 and later) and "Signing
# internally-generated bounce messages"; src/milter/milter.c, milter_*_event()
# and milter_message(): milters are asked in list order and the first non-empty
# reply ends the round; src/global/mail_params.h, DEF_INT_FILT_CLASSES
# INT_FILT_CLASS_NONE.
postconf -e \
"milter_protocol=6" \
"milter_default_action=tempfail" \
"smtpd_milters={ unix:${OPENDKIM_SOCK}, default_action=tempfail }, { unix:${JOURNAL_SOCK}, default_action=accept }" \
"non_smtpd_milters="
"smtpd_milters={ unix:${SENDER_SOCK}, default_action=tempfail }, { unix:${OPENDKIM_SOCK}, default_action=tempfail }, { unix:${JOURNAL_SOCK}, default_action=accept }" \
"non_smtpd_milters={ unix:${OPENDKIM_SOCK}, default_action=tempfail }"
# What Postfix tells a milter with MAIL FROM: its defaults plus the client's
# address. The panel's milters judge every message by that address (an
# application's client-address list) and record it (the inbound journal), and
# the address given once at connect is not there for a connection's later
# messages - see MailFrom in internal/milter/milter.go. The defaults: Postfix
# 3.7.11, src/global/mail_params.h, DEF_MILT_MAIL_MACROS.
postconf -e \
"milter_mail_macros=i {auth_type} {auth_authen} {auth_author} {mail_addr} {mail_host} {mail_mailer} {client_addr}"
# Bounded milter timeouts: a *hung* milter (socket accepts but never replies)
# must fail open just like a crash, not stall mail acceptance until the Postfix
@@ -233,7 +332,8 @@ postconf -e \
# journal-milter hang then resolves to accept and an OpenDKIM hang to tempfail,
# but within seconds rather than minutes. Values are well above any healthy
# response time (signing/DB insert are sub-second), so they never fire in normal
# operation.
# operation. The defaults: Postfix 3.7.11, src/global/mail_params.h,
# DEF_MILT_CONN_TIME "30s", DEF_MILT_CMD_TIME "30s", DEF_MILT_MSG_TIME "300s".
postconf -e \
"milter_connect_timeout=${MILTER_CONNECT_TIMEOUT:-15s}" \
"milter_command_timeout=${MILTER_COMMAND_TIMEOUT:-15s}" \
@@ -243,6 +343,10 @@ postconf -e \
# smtps (465, implicit/wrapper TLS) — the primary, always-on submission
# service. chroot=n so smtpd can read the sasldb2 and sender map under /data
# and the Cyrus config outside any chroot.
# The names smtps, submission and smtp need no /etc/services: Postfix 3.7.11,
# src/global/mail_params.h, DEF_KNOWN_TCP_PORTS "lmtp=24, smtp=25,
# smtps=submissions=465, submission=587" (src/util/find_inet.c,
# find_inet_port() via filter_known_tcp_port()).
postconf -M "smtps/inet=smtps inet n - n - - smtpd"
postconf -P \
"smtps/inet/smtpd_tls_wrappermode=yes" \
@@ -252,7 +356,10 @@ postconf -P \
# submission (587, STARTTLS) — optional. Same SASL/milter/limits; the only
# difference is TLS is negotiated via STARTTLS, so require encryption before
# auth. Added only when SUBMISSION_ENABLE=true, otherwise removed so a restart
# after disabling it does not leave the port listening.
# after disabling it does not leave the port listening. (Postfix 3.7.11,
# src/smtpd/smtpd.c, pre_jail_init(): smtpd_tls_security_level=encrypt sets
# enforce_tls, which implies smtpd_tls_auth_only; src/postconf/postconf_edit.c,
# pcf_edit_master(): -MX drops the matched entry, continuation lines included.)
if [ "${SUBMISSION_ENABLE}" = "true" ]; then
postconf -M "submission/inet=submission inet n - n - - smtpd"
postconf -P \
@@ -264,7 +371,9 @@ else
fi
# --- inbound smtpd on port 25 (optional inbound relay and/or DMARC ingest) -----
# Debian's stock master.cf enables smtp/inet. When both flags are off, remove
# Debian's stock master.cf enables smtp/inet (postfix 3.7.11-0+deb12u1,
# debian/patches/40_chroot_by_default.diff: "smtp inet n - y - - smtpd",
# uncommented). When both flags are off, remove
# that listener so port 25 is not an inbound smtpd (outbound delivery uses
# smtp/unix). When either is on: no SASL, no OpenDKIM; relay accepts only
# configured domains/recipients; DMARC accepts only allow-listed report addresses.
@@ -289,13 +398,35 @@ if [ "${INBOUND_RELAY_ENABLE}" = "true" ] || [ "${DMARC_REPORTS_ENABLE}" = "true
# postconf -P refuses them ("unmatched service_name/type") and leaves a
# bare "pipe", which dies with "fatal: missing user= command-line
# attribute" on every report. They go into the service line itself.
# (Seen in 1.7.0 through 1.9.5, date not recorded. Postfix 3.7.11:
# src/postconf/postconf_master.c, pcf_edit_master_param(), -P writes
# "-o name=value" and nothing else; postconf_edit.c warns "unmatched
# service_name/type" for a service master.cf does not have;
# src/pipe/pipe.c, get_service_attr(): "missing user= command-line
# attribute".)
# ${recipient} is expanded by pipe(8), not by the shell: the command
# learns which report address the message came to from its argument.
postconf -M 'dmarc-ingest/unix=dmarc-ingest unix - n n - - pipe flags=Rq user=panel argv=/usr/local/bin/dmarc-ingest ${recipient}'
# learns which report address the message came to from its argument
# (man/man8/pipe.8, ${recipient}: as many arguments as recipients).
# One at a time (the 1 before "pipe"): reports are in no hurry, every run
# writes to the database the panel writes to, and a hundred of them in
# parallel - Postfix's default - is what a flood of reports would start.
postconf -M 'dmarc-ingest/unix=dmarc-ingest unix - n n - 1 pipe flags=Rq user=panel argv=/usr/local/bin/dmarc-ingest ${recipient}'
# One recipient per run, so ${recipient} is one argument.
postconf -e "dmarc-ingest_destination_recipient_limit=1"
# pipe(8) starts the command with export_environment only, so what the
# command reads from the container's environment is named there.
# export_environment is a list split at whitespace in which Postfix
# expands "$": a path with either would reach the command as another
# path, and it would quietly open a database of its own. (Postfix 3.7.11,
# man/man8/pipe.8, export_environment; src/global/mail_params.h,
# DEF_EXPORT_ENVIRON "TZ MAIL_CONFIG LANG"; src/global/clean_env.c. The
# parameter is main.cf's, so every Postfix process exports these too.)
case "${SELFPOST_DATA_DIR:-}${SELFPOST_DB_PATH:-}" in
*[[:space:]]* | *'$'*)
echo "postfix-config: SELFPOST_DATA_DIR and SELFPOST_DB_PATH must not contain whitespace or \$ when DMARC_REPORTS_ENABLE=true" >&2
exit 1
;;
esac
DMARC_INGEST_ENV="DMARC_REPORTS_ENABLE=true"
[ -z "${SELFPOST_DATA_DIR:-}" ] || DMARC_INGEST_ENV="${DMARC_INGEST_ENV} SELFPOST_DATA_DIR=${SELFPOST_DATA_DIR}"
[ -z "${SELFPOST_DB_PATH:-}" ] || DMARC_INGEST_ENV="${DMARC_INGEST_ENV} SELFPOST_DB_PATH=${SELFPOST_DB_PATH}"
@@ -308,7 +439,8 @@ if [ "${INBOUND_RELAY_ENABLE}" = "true" ] || [ "${DMARC_REPORTS_ENABLE}" = "true
TRANSPORT_SETTING="texthash:${DMARC_TRANSPORT_MAP}"
fi
# Comma, not a space, between check_recipient_access and its table: see
# master_cf_set_smtp_inet_o.
# master_cf_set_smtp_inet_o (master.5, NOTE 2: "or use commas instead of
# spaces").
RECIPIENT_RESTRICTIONS="check_recipient_access,texthash:${DMARC_RECIPIENTS_MAP},${RECIPIENT_RESTRICTIONS}"
else
postconf -MX "dmarc-ingest/unix" 2>/dev/null || true
@@ -326,10 +458,17 @@ if [ "${INBOUND_RELAY_ENABLE}" = "true" ] || [ "${DMARC_REPORTS_ENABLE}" = "true
# relay domain too (the server's own name), so their table is a recipient
# table as well; without it they are "User unknown in relay recipient
# table" and no report is ever accepted while both features are on.
# (Postfix 3.7.11, src/smtpd/smtpd_check.c, smtpd_check_rcpt(): after
# the lists, check_recipient_rcpt_maps() runs unless the status is
# REJECT - an OK from check_recipient_access does not skip it.)
RELAY_RECIPIENT_SETTING="texthash:${RELAY_RECIPIENTS_MAP}"
if [ "${DMARC_REPORTS_ENABLE}" = "true" ]; then
RELAY_RECIPIENT_SETTING="${RELAY_RECIPIENT_SETTING} texthash:${DMARC_RECIPIENTS_MAP}"
fi
# The panel writes the same "[host]:port" into transport and
# tls_policy: the policy lookup key is the verbatim next-hop, port and
# brackets included (Postfix 3.7.11, README_FILES/TLS_README, "TLS
# policy table" note: ":smtp", ":25" and no port are different keys).
postconf -e \
"relay_recipient_maps=${RELAY_RECIPIENT_SETTING}" \
"smtp_tls_policy_maps=texthash:${TLS_POLICY_MAP}"
@@ -345,10 +484,21 @@ if [ "${INBOUND_RELAY_ENABLE}" = "true" ] || [ "${DMARC_REPORTS_ENABLE}" = "true
# would file inbound mail into the outbound send log under the *sender's*
# domain — polluting Deliveries and letting a forged From: consume that
# domain's level-2 rate-limit budget. So smtpd_milters is always overridden
# here: either the optional antispam milter, or nothing at all.
# here: the optional antispam milter, then - with the inbound relay on - the
# panel's inbound journal milter, or nothing at all. (Postfix 3.7.11,
# man/man5/master.5: "-o" overrides the main.cf parameter for this service;
# src/smtpd/smtpd.c hands its own milter list to cleanup, milter_send(),
# and src/cleanup/cleanup_milter.c, cleanup_milter_receive(), uses it.)
INBOUND_MILTERS=""
if [ "${INBOUND_RELAY_ENABLE}" = "true" ] && [ -n "${INBOUND_ANTISPAM_MILTER}" ]; then
# A glob's "*" matches anything, so the shape alone would pass
# "inet:a:1},{unix:/elsewhere": characters outside the set are refused
# first, then the shape is checked.
case "${INBOUND_ANTISPAM_MILTER}" in
*[!A-Za-z0-9._:/-]*)
echo "FATAL: INBOUND_ANTISPAM_MILTER must be inet:host:port or unix:/path, got: ${INBOUND_ANTISPAM_MILTER}" >&2
exit 1
;;
inet:[A-Za-z0-9._-]*:[0-9]* | unix:/[A-Za-z0-9._/-]* ) ;;
*)
echo "FATAL: INBOUND_ANTISPAM_MILTER must be inet:host:port or unix:/path, got: ${INBOUND_ANTISPAM_MILTER}" >&2
@@ -364,6 +514,31 @@ if [ "${INBOUND_RELAY_ENABLE}" = "true" ] || [ "${DMARC_REPORTS_ENABLE}" = "true
esac
INBOUND_MILTERS="{${INBOUND_ANTISPAM_MILTER},default_action=${INBOUND_ANTISPAM_MILTER_ACTION}}"
fi
if [ "${INBOUND_RELAY_ENABLE}" = "true" ]; then
# Last in the chain: Postfix hands a message to its milters one after
# another at end-of-message, so this one reads the verdict headers the
# filter added and sees only what the filter let through. It watches and
# never refuses, and must never hold up someone else's mail: accept when
# it is down. A comma, not a space, between the two - see
# master_cf_set_smtp_inet_o.
# Postfix 3.7.11: src/milter/milter.c, milter_message(), sends the
# message to each milter in list order; src/milter/milter8.c,
# milter8_event(), applies a header a milter adds (SMFIR_ADDHEADER) to
# the queue file at once, and the next milter8_message() reads that
# file with rec_get(), which follows the added records.
case "${INBOUND_JOURNAL_SOCK}" in
*[!A-Za-z0-9._/-]*)
echo "FATAL: INBOUND_JOURNAL_MILTER_SOCKET must be an absolute path, got: ${INBOUND_JOURNAL_SOCK}" >&2
exit 1
;;
/[A-Za-z0-9._/-]*) ;;
*)
echo "FATAL: INBOUND_JOURNAL_MILTER_SOCKET must be an absolute path, got: ${INBOUND_JOURNAL_SOCK}" >&2
exit 1
;;
esac
INBOUND_MILTERS="${INBOUND_MILTERS:+${INBOUND_MILTERS},}{unix:${INBOUND_JOURNAL_SOCK},default_action=accept}"
fi
PORT25_RATE="${INBOUND_RATE_LIMIT_MESSAGES_PER_IP}"
PORT25_SIZE="${INBOUND_MESSAGE_SIZE_LIMIT}"
@@ -380,8 +555,15 @@ if [ "${INBOUND_RELAY_ENABLE}" = "true" ] || [ "${DMARC_REPORTS_ENABLE}" = "true
fi
fi
# syslog_name: this listener's lines read postfix/inbound/smtpd, the
# submission ports' postfix/smtpd. The log tailer files a refusal on port 25
# into the inbound journal and has to tell the two apart. (Postfix 3.7.11,
# src/global/mail_task.c, mail_task(): the log tag is syslog_name "/"
# program name. Only smtpd carries it: cleanup and the rest are other
# services and log as postfix/.)
postconf -M "smtp/inet=smtp inet n - n - - smtpd"
postconf -P \
"smtp/inet/syslog_name=postfix/inbound" \
"smtp/inet/smtpd_sasl_auth_enable=no" \
"smtp/inet/smtpd_tls_auth_only=no" \
"smtp/inet/smtpd_sender_login_maps=" \
@@ -392,6 +574,18 @@ if [ "${INBOUND_RELAY_ENABLE}" = "true" ] || [ "${DMARC_REPORTS_ENABLE}" = "true
"smtp/inet/message_size_limit=${PORT25_SIZE}"
master_cf_set_smtp_inet_o smtpd_recipient_restrictions "${RECIPIENT_RESTRICTIONS}"
master_cf_set_smtp_inet_o smtpd_milters "${INBOUND_MILTERS}"
# smtpd's -o limit is not the whole of it: cleanup writes the queue file
# under its own message_size_limit, main.cf's, which is never set here and
# so is 10240000 (Postfix 3.7.11, src/cleanup/cleanup_init.c,
# cleanup_post_jail(), set_file_limit(); src/global/mail_params.h,
# DEF_MESSAGE_LIMIT), and a larger message is refused 552 after DATA
# (assumptions C-P31). cleanup gets port 25's limit when it is larger;
# smtpd on 465 and 587 still holds its own, main.cf's (src/smtpd/smtpd.c,
# the DATA and BDAT size checks).
if [ "${PORT25_SIZE}" -gt "$(postconf -h message_size_limit)" ]; then
postconf -P "cleanup/unix/message_size_limit=${PORT25_SIZE}"
fi
else
postconf -MX "smtp/inet" 2>/dev/null || true
postconf -MX "dmarc-ingest/unix" 2>/dev/null || true
@@ -409,13 +603,23 @@ fi
# Inside a container the chroot buys little (the container is the isolation
# boundary) and breaks DNS/TLS trust-store access, so turn it off uniformly. Our
# own smtps/submission services are already n; this covers the delivery agents
# and the rest.
# and the rest. (postfix 3.7.11-0+deb12u1: debian/patches/40_chroot_by_default.diff
# sets chroot y; debian/configure-instance.sh copies etc/resolv.conf and the
# like into the chroot, and nothing runs it in this container. chroot(2) also
# needs CAP_SYS_CHROOT, which deploy/docker-compose.yml drops with cap_drop:
# ALL - capabilities(7).)
postconf -F "*/*/chroot=n"
# --- Cyrus SASL app config for smtpd -----------------------------------------
# Tells the Cyrus library (invoked by smtpd via smtpd_sasl_path=smtpd) to verify
# passwords straight from the panel-maintained sasldb2. PLAIN/LOGIN
# only — both are safe because TLS is mandatory before auth on every port.
# The file is found by a Debian patch, not by upstream Postfix: postfix
# 3.7.11-0+deb12u1, debian/patches/07_sasl_config.diff, xsasl_getconfpath() in
# src/xsasl/xsasl_cyrus_log.c gives Cyrus "$config_directory/sasl:/usr/lib/sasl",
# and smtpd_sasl_path=smtpd names smtpd.conf there. The keys: Cyrus SASL
# 2.1.28, doc/legacy/options.html (pwcheck_method, auxprop_plugin,
# sasldb_path, mech_list).
mkdir -p /etc/postfix/sasl
cat > /etc/postfix/sasl/smtpd.conf <<EOF
pwcheck_method: auxprop
@@ -425,7 +629,9 @@ mech_list: PLAIN LOGIN
EOF
# Validate the generated configuration; fail loudly if postconf produced
# anything Postfix rejects, before the wrapper tries to start it.
# anything Postfix rejects, before the wrapper tries to start it. (Postfix
# 3.7.11, conf/postfix-script, "check": "check-fatal || exit 1"; check-fatal
# also runs "post-install create-missing", which builds the queue tree.)
set +e
check_out=$(postfix check 2>&1)
ec=$?
+63 -9
View File
@@ -2,22 +2,36 @@
# Postfix start wrapper: solves the cold-start race where Postfix would try to
# reach the milter sockets before they are listening.
#
# It blocks until BOTH milter sockets — OpenDKIM and the panel's journal-milter
# — are present, then execs `postfix start-fg`. If they are not ready within the
# timeout it exits non-zero WITHOUT starting Postfix, so supervisord/Docker see
# an explicit startup failure instead of a relay running blind.
# It blocks until the milter sockets — OpenDKIM, the panel's sender check and
# journal-milter and, with the inbound relay on, its inbound journal milter —
# are present, then runs `postfix start-fg` and stays its parent. If they are
# not ready within the timeout it exits non-zero WITHOUT starting Postfix.
# supervisord counts that exit as one that came too quickly - the program's
# startsecs is the timeout plus ten seconds (build/entrypoint.sh) - so after
# startretries it is FATAL and crashexit takes the container down, instead of
# a relay running blind (supervisor 4.2.5, supervisor/process.py,
# Subprocess.finish(): an exit before startsecs is "Exited too quickly" and
# goes to BACKOFF; transition(): BACKOFF past startretries is FATAL;
# assumptions C-V6).
#
# This handles cold start only. Runtime milter failures after a successful start
# are governed by fail-open (milter_default_action).
# This handles cold start only. Once Postfix runs, a milter that does not
# answer costs what its default_action says (build/postfix-config.sh): mail is
# deferred for OpenDKIM and the sender check, and passes the journal-milter.
set -eu
OPENDKIM_SOCK="${OPENDKIM_SOCKET:-/run/opendkim/opendkim.sock}"
JOURNAL_SOCK="${JOURNAL_MILTER_SOCKET:-/run/selfpost/journal.sock}"
SENDER_SOCK="${SENDER_CHECK_SOCKET:-/run/selfpost/sender.sock}"
TIMEOUT="${MILTER_WAIT_TIMEOUT:-30}"
INTERVAL=1
SOCKS="$OPENDKIM_SOCK $JOURNAL_SOCK $SENDER_SOCK"
if [ "${INBOUND_RELAY_ENABLE:-false}" = "true" ]; then
SOCKS="$SOCKS ${INBOUND_JOURNAL_MILTER_SOCKET:-/run/selfpost/inbound-journal.sock}"
fi
elapsed=0
for sock in "$OPENDKIM_SOCK" "$JOURNAL_SOCK"; do
for sock in $SOCKS; do
while [ ! -S "$sock" ]; do
if [ "$elapsed" -ge "$TIMEOUT" ]; then
echo "postfix-wrapper: timed out after ${TIMEOUT}s waiting for milter socket $sock" >&2
@@ -29,5 +43,45 @@ for sock in "$OPENDKIM_SOCK" "$JOURNAL_SOCK"; do
echo "postfix-wrapper: milter socket ready: $sock"
done
echo "postfix-wrapper: both milter sockets ready, starting postfix"
exec postfix start-fg
echo "postfix-wrapper: milter sockets ready, starting postfix"
# Postfix 3.7.11: postfix(1) execs conf/postfix-script (src/postfix/postfix.c),
# whose "start-fg" execs master only as PID 1 and otherwise runs "master -s" as
# its child; master then makes itself a session leader (src/master/master.c,
# setsid()). A signal supervisord sends this program never reaches master
# (supervisor 4.2.5, supervisor/process.py, Subprocess.kill(): to the pid
# alone without stopasgroup, and master is not in its group anyway): before,
# a stop left master running - a restart of the program then found Postfix
# "already running", and a stop of the container cut Postfix off with
# SIGKILL (assumptions C-P42). So this shell stays, and on TERM or INT runs
# "postfix stop" (conf/postfix-script, "stop": TERM to master, then waits up
# to five seconds and kills its group; src/master/master_sig.c,
# master_sigdeath(): "terminating on signal 15").
#
# "postfix start-fg" spends its first seconds in its checks before master
# runs; a stop asked then would find nothing to stop, so it is asked again
# until the child is gone. "master -t" exits 0, silently, while no master
# holds the lock (src/master/master.c, test_lock).
daemon_directory=$(postconf -h daemon_directory)
child=
stop_postfix() {
trap '' TERM INT
while [ -n "$child" ] && kill -0 "$child" 2>/dev/null; do
if "$daemon_directory/master" -t 2>/dev/null; then
sleep 1
else
postfix stop || true
fi
done
exit 0
}
trap stop_postfix TERM INT
postfix start-fg &
child=$!
# A trapped signal makes wait return at once, the trap is taken right after
# (POSIX, Shell Command Language, 2.11 Signals and Error Handling) and exits.
# Otherwise wait returns when Postfix ended on its own: its status is this
# program's, and supervisord restarts it.
status=0
wait "$child" || status=$?
echo "postfix-wrapper: postfix exited with status $status" >&2
exit "$status"
+54 -14
View File
@@ -1,12 +1,22 @@
; SelfPost process supervision.
;
; Start ORDER is enforced by priority=: OpenDKIM, then the panel (which opens
; the journal-milter socket), then the Postfix wrapper — which additionally
; blocks until both milter sockets are ready before starting Postfix.
; the sender check's and the journal-milter's sockets), then the Postfix
; wrapper — which additionally blocks until the milter sockets are ready
; before starting Postfix. (supervisor 4.2.5, docs/configuration.rst,
; [program:x] "priority": "the start and shutdown ordering"; supervisor/
; supervisord.py, runforever(), transitions every group in that order in one
; pass, with no wait for a program to be ready.)
;
; If any managed process exhausts its restart retries (FATAL), the crashexit
; event listener brings the whole container down so Docker's restart policy can
; recreate it cleanly, rather than lingering with a dead component.
; If a managed process of the mail path exhausts its restart retries (FATAL),
; the crashexit event listener brings the whole container down so Docker's
; restart policy starts it again (the same container: docs.docker.com, "Start
; containers automatically", restart policies), rather than lingering with a
; dead component. A program reaches FATAL only through BACKOFF, by exits that
; come before its startsecs (supervisor 4.2.5, supervisor/process.py,
; Subprocess.finish() and transition()): an exit after it is EXITED and simply
; restarted, for ever. logrotate's FATAL is left on Server > Health
; (build/crashexit.py).
[supervisord]
nodaemon=true
@@ -21,7 +31,9 @@ loglevel=info
; OpenDKIM a reload signal after it rewrites the DKIM tables. It is still
; unreachable by world; only root and the two SelfPost services are in the
; group. No HTTP auth is configured because access is gated by socket
; permissions, not the network.
; permissions, not the network. (supervisor 4.2.5, docs/configuration.rst,
; [unix_http_server]: chmod and chown set the socket's mode and owner; without
; username and password no authentication is asked.)
[unix_http_server]
file=/run/supervisor.sock
chown=root:selfpost
@@ -44,6 +56,10 @@ stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
; user=panel also gives the panel its supplementary groups, selfpost and
; postdrop (build/Dockerfile): supervisor 4.2.5, supervisor/options.py,
; drop_privileges(), os.setgroups() from the user's memberships in
; grp.getgrall().
[program:panel]
command=/usr/local/bin/panel
user=panel
@@ -56,12 +72,22 @@ stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
; startsecs is MILTER_WAIT_TIMEOUT plus ten seconds (build/entrypoint.sh): the
; wrapper's exit when the milter sockets never come counts as too quick, and
; so does a Postfix that will not start. The price: for that long after each
; start the program is STARTING, not RUNNING (supervisor 4.2.5,
; supervisor/options.py, processes_from_section(): startsecs is expanded like
; any value, %(ENV_x)s from supervisord's environment; assumptions C-V6).
; stopwaitsecs covers "postfix stop", which waits five seconds for master
; before it kills (build/postfix-wrapper.sh).
[program:postfix]
command=/usr/local/bin/postfix-wrapper.sh
priority=300
autostart=true
autorestart=true
startsecs=%(ENV_POSTFIX_STARTSECS)s
startretries=3
stopwaitsecs=15
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
@@ -75,7 +101,12 @@ stderr_logfile_maxbytes=0
; supervisord — over the group-accessible control socket — to run this program,
; which executes the canonical `postfix reload` as root. autostart is off (it
; only runs on demand) and a fast, clean exit is expected, so it never trips the
; crashexit listener.
; crashexit listener. (start-fg: see build/postfix-wrapper.sh. supervisor
; 4.2.5, supervisor/process.py, Subprocess.finish(): with startsecs=0 no exit
; counts as "too quickly", so even a failed run ends EXITED - never BACKOFF,
; never FATAL - with its exit status kept, which the panel reads
; (internal/supervisor, RunOnce). Its resting state is STOPPED until the first
; run, EXITED after it; assumptions C-V10.)
[program:postfix-reload]
command=/usr/sbin/postfix reload
autostart=false
@@ -88,8 +119,9 @@ stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
; Periodic TLS-certificate refresh. Runs a daily `postfix reload`
; so certificates the reverse-proxy renews in the read-only mount are picked up.
; TLS-certificate refresh. Once a day it compares the certificate the
; reverse-proxy renews in the read-only mount with the copy Postfix reads, and
; when they differ copies it again and runs `postfix reload`.
; Long-running (it loops), runs as root so it can reload Postfix, and never exits
; non-zero, so it neither trips the crashexit listener nor needs restarting.
[program:cert-reload]
@@ -103,21 +135,29 @@ stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
; Periodic logrotate for /data/log/mail.log (daily, 7-14 files kept in the
; image). Runs as root so logrotate can read/rotate the log. Exits non-zero when
; the config is group/other-writable (logrotate would ignore it silently);
; autorestart surfaces BACKOFF on the Status page.
; Periodic logrotate for /data/log/mail.log (daily, 14 files kept). Runs as
; root so logrotate can read/rotate the log. Exits non-zero at once when the
; config is group/other-writable (logrotate would ignore it silently): with
; startsecs=1 that exit is too quick; once startretries are spent the program
; is FATAL, and Server > Health shows it. With startsecs=0 it was restarted
; every second for ever, RUNNING and EXITED in turn (supervisor 4.2.5,
; supervisor/process.py, Subprocess.finish(); assumptions C-V5). crashexit
; leaves this FATAL alone: logrotate is not on the mail path.
[program:logrotate]
command=/usr/local/bin/logrotate-loop.sh
priority=400
autostart=true
autorestart=true
startsecs=0
startsecs=1
startretries=3
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
; supervisor 4.2.5, supervisor/process.py: a program enters FATAL only from
; BACKOFF, once startretries are spent; a stop asked of supervisord ends in
; STOPPED (Subprocess.finish(), self.killing), which this listener never sees.
[eventlistener:crashexit]
command=/usr/local/bin/crashexit.py
events=PROCESS_STATE_FATAL
+48 -1
View File
@@ -1,6 +1,7 @@
package main
import (
"errors"
"fmt"
"io"
"log"
@@ -28,17 +29,63 @@ func runDMARCIngestMode() error {
recipient := ingestRecipient(os.Args[1:])
if err := dmarc.IngestMessage(st, io.LimitReader(os.Stdin, maxIngestStdin), recipient, time.Now().UTC()); err != nil {
if !errors.Is(err, dmarc.ErrRejected) {
return err
}
// Not a report this relay keeps. It is dropped: deferring would have
// Postfix try it again for days, and a bounce would go to an envelope
// sender anyone can write - the report addresses are public. Postfix
// 3.7.11: exit 0 is PIPE_STAT_OK, logged "status=sent (delivered via
// ... service ...)" (src/global/pipe_command.c, pipe_command();
// src/pipe/pipe.c, eval_command_status()); a 4.x.x status defers until
// maximal_queue_lifetime and a 5.x.x one bounces, either way a notice
// to the sender (src/qmgr/qmgr_active.c, src/global/bounce.c).
if incrErr := st.IncrDMARCParseFailures(); incrErr != nil {
log.Printf("dmarc ingest: record failure: %v", incrErr)
}
return err
logRejected(err)
}
return nil
}
// logRejected leaves the reason in the mail log. Postfix puts the start of
// what a delivery command wrote - about a hundred bytes - into its own line
// for the message ("status=sent (delivered via dmarc-ingest service (...))"),
// so the reason goes out first and bare: no prefix, no date, no recipient,
// all of which that line already has. Postfix 3.7.11, src/pipe/pipe.c,
// eval_command_status(): " (%.100s)" of the output, which
// src/global/pipe_command.c takes from stdout and stderr together. postlog is
// no use here: pipe(8) starts the command without the group that may write to
// Postfix's log socket (src/util/set_ugid.c, set_ugid(): setgroups() to the
// user's one group). Whether Debian's postlog is set-gid, which would bring
// the group with it, is not recorded - assumptions A-43.
func logRejected(reason error) {
fmt.Fprintln(os.Stderr, "dropped: "+logSafe(strings.TrimPrefix(reason.Error(), dmarc.ErrRejected.Error()+": ")))
}
// logSafe makes a reason fit for a log line. Parts of it are the sender's own
// bytes - a header line, a file name inside an archive, a piece of XML - so it
// is cut short and anything that is not printable ASCII becomes "?", whatever
// Postfix would have done with it.
func logSafe(s string) string {
const max = 200
if len(s) > max {
s = s[:max]
}
return strings.Map(func(r rune) rune {
if r < 0x20 || r > 0x7e {
return '?'
}
return r
}, s)
}
// ingestRecipient is the report address the message was delivered to. Postfix's
// pipe(8) hands it over as an argument (${recipient} in the service's argv,
// build/postfix-config.sh) - it sets no RECIPIENT variable, that is local(8).
// Postfix 3.7.11, src/pipe/pipe.c, deliver_message(): pipe_command() gets only
// CA_PIPE_CMD_EXPORT(export_environment), no extra environment;
// src/local/command.c sets "RECIPIENT".
// Without it the check that a report came to its own domain's address is
// skipped.
func ingestRecipient(args []string) string {
+14
View File
@@ -18,3 +18,17 @@ func TestIngestRecipient(t *testing.T) {
}
}
}
func TestLogSafe(t *testing.T) {
got := logSafe("zip entry \"a\nstatus=bounced\tx\" \x1b[31m é")
if want := "zip entry \"a?status=bounced?x\" ?[31m ?"; got != want {
t.Errorf("logSafe = %q, want %q", got, want)
}
long := make([]byte, 1000)
for i := range long {
long[i] = 'a'
}
if got := logSafe(string(long)); len(got) != 200 {
t.Errorf("logSafe kept %d bytes of 1000, want 200", len(got))
}
}
+3
View File
@@ -32,6 +32,7 @@ var documentedInternal = []string{
"SELFPOST_SETUP_TOKEN_FILE",
"PANEL_HTTP_ADDR",
"JOURNAL_MILTER_SOCKET",
"SENDER_CHECK_SOCKET",
"MAIL_LOG",
"PANEL_COOKIE_SECURE",
"OPENDKIM_SOCKET",
@@ -70,6 +71,7 @@ var loadConfigKeys = []string{
"SELFPOST_DATA_DIR",
"PANEL_HTTP_ADDR",
"JOURNAL_MILTER_SOCKET",
"SENDER_CHECK_SOCKET",
"MAIL_LOG",
"SEND_LOG_RETENTION_DAYS",
"SELFPOST_DB_PATH",
@@ -104,6 +106,7 @@ var buildScriptKeys = []string{
"RATE_LIMIT_WINDOW_SECONDS",
"OPENDKIM_SOCKET",
"JOURNAL_MILTER_SOCKET",
"SENDER_CHECK_SOCKET",
"MAIL_LOG",
"POSTFIX_SENDER_LOGIN_MAPS",
"POSTFIX_QUEUE_DIR",
+18 -3
View File
@@ -24,6 +24,7 @@ type mailStack struct {
Domains *domain.Service
Apps *app.Service
Inbound *inbound.Service
Lists *inbound.Lists
DMARC *dmarc.Service
pf *postfix.Postfix
odk *domain.OpenDKIM
@@ -36,11 +37,15 @@ func newMailStack(cfg config, st *store.Store) *mailStack {
odk := domain.NewOpenDKIM(cfg.opendkimDir)
apps := app.NewService(st, app.NewSASLDB(cfg.saslDBPath, cfg.saslRealm), pf)
domains := domain.NewService(st, odk, apps, cfg.dkimSelectorDef)
// The panel's notifications are sent as postmaster@ the server's own name,
// and signed as it.
domains.SetServerName(cfg.hostname)
inb := inbound.NewService(st, pf)
dmarcSvc := dmarc.NewService(st, pf, cfg.hostname, cfg.dmarcEnabled)
return &mailStack{
Domains: domains, Apps: apps, Inbound: inb, DMARC: dmarcSvc,
pf: pf, odk: odk, inboundEnabled: cfg.inboundEnabled, dmarcEnabled: cfg.dmarcEnabled,
Lists: inbound.NewLists(st, cfg.antispamDir),
pf: pf, odk: odk, inboundEnabled: cfg.inboundEnabled, dmarcEnabled: cfg.dmarcEnabled,
}
}
@@ -57,6 +62,11 @@ func (m *mailStack) Resync() error {
if err := m.Inbound.Resync(); err != nil {
return fmt.Errorf("inbound maps resync: %w", err)
}
// The filter's configuration names all six list files; they exist from
// the first start, empty until an entry is added.
if err := m.Lists.Sync(); err != nil {
return fmt.Errorf("filter lists resync: %w", err)
}
}
if m.dmarcEnabled {
if err := m.DMARC.Resync(); err != nil {
@@ -89,9 +99,11 @@ func newPanel(cfg config, st *store.Store) (*web.Server, error) {
ms := newMailStack(cfg, st)
// postfix-config.sh has already run (entrypoint). postconf -h is the
// effective config, including a manual override; the panel keeps this
// snapshot for the process lifetime (architecture.md).
// snapshot for the process lifetime (architecture.md). Postfix 3.7.11,
// postconf(1): without -d or -n it shows main.cf's value or the default,
// and -h drops the "name = " label; values are not $-expanded without -x.
retryPolicy := postfix.LoadRetryPolicy()
return web.New(st, ms.Domains, ms.Apps, ms.Inbound, ms.DMARC, web.Config{
return web.New(st, ms.Domains, ms.Apps, ms.Inbound, ms.Lists, ms.DMARC, web.Config{
Hostname: cfg.hostname,
CookieSecure: cfg.cookieSecure,
SubmissionEnabled: cfg.submissionEnabled,
@@ -104,6 +116,7 @@ func newPanel(cfg config, st *store.Store) (*web.Server, error) {
TLSCertFile: cfg.tlsCertFile,
OpenDKIMSocket: cfg.opendkimSocket,
JournalSocket: cfg.journalSocket,
SenderSocket: cfg.senderSocket,
SessionIdleDays: cfg.sessionIdleDays,
DNSResolvers: cfg.dnsResolvers,
RateLimitMessagesPerIP: cfg.rateLimitMessagesPerIP,
@@ -114,6 +127,8 @@ func newPanel(cfg config, st *store.Store) (*web.Server, error) {
InboundAntispamAction: cfg.inboundAntispamAction,
DMARCEnabled: cfg.dmarcEnabled,
SendLogRetentionEnvDefault: cfg.retentionDays,
InboundSpamLogRetentionEnvDefault: cfg.inboundSpamLogRetentionDays,
}, cfg.setupTokenPath)
}
+46 -11
View File
@@ -8,6 +8,7 @@ import (
"os"
"path/filepath"
"github.com/mixeme/selfpost/internal/inbound"
"github.com/mixeme/selfpost/internal/milter"
"github.com/mixeme/selfpost/internal/store"
)
@@ -15,31 +16,65 @@ import (
// serveJournal opens the journal-milter Unix socket and runs the real milter
// (architecture.md § Mail path), recording accepted messages into the send
// log. Socket lifecycle (creation, stale cleanup, group permissions) lives
// here; the protocol handler lives in internal/milter.
// in listenMilter; the protocol handler lives in internal/milter.
func serveJournal(ctx context.Context, cfg config, st *store.Store) error {
socketPath := cfg.journalSocket
if err := os.MkdirAll(filepath.Dir(socketPath), 0o755); err != nil {
ln, err := listenMilter(cfg.journalSocket)
if err != nil {
return err
}
// Clear a stale socket left behind by an unclean shutdown, otherwise the
// listen below fails with "address already in use".
if err := os.Remove(socketPath); err != nil && !errors.Is(err, os.ErrNotExist) {
log.Printf("journal-milter listening on %s", cfg.journalSocket)
return milter.Serve(ctx, ln, st)
}
// serveInboundJournal runs the inbound journal milter, the last milter on
// port 25 (internal/milter/inbound.go). It exists only while the inbound relay
// is on: build/postfix-config.sh attaches the socket under the same condition.
func serveInboundJournal(ctx context.Context, cfg config, st *store.Store) error {
ln, err := listenMilter(cfg.inboundJournalSocket)
if err != nil {
return err
}
// With a filter attached, the key it signs with. Without the key no
// verdict can be told from a sender's own, so none is read: the journal
// still records the mail, and the log says why it shows no verdicts.
key := ""
if cfg.inboundAntispamMilter != "" {
if key, err = inbound.FilterKey(cfg.antispamDir); err != nil {
log.Printf("inbound-journal-milter: no key for the filter's signature, verdicts will not be recorded: %v", err)
key = ""
}
}
log.Printf("inbound-journal-milter listening on %s", cfg.inboundJournalSocket)
return milter.ServeInbound(ctx, ln, st, key)
}
// listenMilter opens a milter's Unix socket where Postfix can reach it.
func listenMilter(socketPath string) (net.Listener, error) {
if err := os.MkdirAll(filepath.Dir(socketPath), 0o755); err != nil {
return nil, err
}
// Clear a stale socket left behind by an unclean shutdown, otherwise the
// listen below fails with "address already in use" (unix(7): the socket
// file "must be deleted by the caller when it is no longer needed"; bind(2)
// EADDRINUSE).
if err := os.Remove(socketPath); err != nil && !errors.Is(err, os.ErrNotExist) {
return nil, err
}
ln, err := net.Listen("unix", socketPath)
if err != nil {
return err
return nil, err
}
// Postfix (user `postfix`) connects to this socket as a milter and needs
// group access. The socket inherits group `selfpost` from the setgid parent
// dir entrypoint.sh prepares; make it group read/write so postfix can reach
// it (connecting to a Unix socket needs write permission on the node).
// unix(7): "connecting to a stream socket object requires write
// permission on that socket"; inode(7): a file made in a set-group-ID
// directory takes the directory's group.
if err := os.Chmod(socketPath, 0o660); err != nil {
ln.Close()
return err
return nil, err
}
log.Printf("journal-milter listening on %s", socketPath)
return milter.Serve(ctx, ln, st)
return ln, nil
}
+78 -6
View File
@@ -23,6 +23,7 @@ import (
"github.com/mixeme/selfpost/internal/backup"
"github.com/mixeme/selfpost/internal/buildinfo"
"github.com/mixeme/selfpost/internal/dnscheck"
"github.com/mixeme/selfpost/internal/inbound"
"github.com/mixeme/selfpost/internal/logtail"
"github.com/mixeme/selfpost/internal/store"
)
@@ -33,7 +34,12 @@ func main() {
log.SetPrefix("dmarc-ingest: ")
if err := runDMARCIngestMode(); err != nil {
log.Printf("ingest failed: %v", err)
os.Exit(75) // EX_TEMPFAIL — ask Postfix to defer
// EX_TEMPFAIL — ask Postfix to defer. Postfix 3.7.11,
// src/global/pipe_command.c, pipe_command(): a sysexits code maps
// to its DSN (src/global/sys_exits.c: 75 is "4.3.0") and a 4 defers,
// unless the output starts with a "D.S.N text" of its own - the
// log prefix above makes sure it does not.
os.Exit(75)
}
return
}
@@ -59,6 +65,8 @@ func main() {
type config struct {
httpAddr string
journalSocket string
// The sender check's socket; the default mirrors build/postfix-config.sh.
senderSocket string
mailLog string
retentionDays int
@@ -89,6 +97,9 @@ type config struct {
saslRealm string
postfixDir string
deployRoot string
// Where the filter lists are written as map files for the external spam
// filter; the antispam compose fragment mounts it into the sidecar.
antispamDir string
inboundEnabled bool
// The optional milter on the inbound listener (INBOUND_ANTISPAM_MILTER) and
@@ -96,7 +107,12 @@ type config struct {
// what attaches it; the panel only says so. See antispamAction.
inboundAntispamMilter string
inboundAntispamAction string
dmarcEnabled bool
// The inbound journal milter's socket; the default mirrors
// build/postfix-config.sh. Served only while the inbound relay is on.
inboundJournalSocket string
// Retention of the inbound journal until Server > Settings says otherwise.
inboundSpamLogRetentionDays int
dmarcEnabled bool
}
func loadConfig() config {
@@ -104,6 +120,7 @@ func loadConfig() config {
return config{
httpAddr: envDefault("PANEL_HTTP_ADDR", ":8080"),
journalSocket: envDefault("JOURNAL_MILTER_SOCKET", "/run/selfpost/journal.sock"),
senderSocket: envDefault("SENDER_CHECK_SOCKET", "/run/selfpost/sender.sock"),
// Postfix's delivery log, under /data so the lines that resolve a
// "queued" send-log row outlive the container. The default must match
// maillog_file in build/postfix-config.sh.
@@ -157,11 +174,16 @@ func loadConfig() config {
saslDBPath: envDefault("SASL_DB_PATH", filepath.Join(dataDir, "sasl", "sasldb2")),
saslRealm: saslRealm(),
postfixDir: envDefault("POSTFIX_DIR", filepath.Join(dataDir, "postfix")),
deployRoot: envDefault("SELFPOST_DEPLOY_ROOT", "/selfpost-deploy"),
antispamDir: envDefault("ANTISPAM_DIR", filepath.Join(dataDir, "antispam")),
deployRoot: envDefault("SELFPOST_DEPLOY_ROOT", "/selfpost-deploy"),
// Optional inbound relay (backup-MX / forwarder). Off unless exactly "true".
inboundEnabled: os.Getenv("INBOUND_RELAY_ENABLE") == "true",
inboundAntispamMilter: os.Getenv("INBOUND_ANTISPAM_MILTER"),
inboundAntispamAction: antispamAction(os.Getenv("INBOUND_ANTISPAM_MILTER_ACTION")),
inboundJournalSocket: envDefault("INBOUND_JOURNAL_MILTER_SOCKET", "/run/selfpost/inbound-journal.sock"),
inboundSpamLogRetentionDays: envInt("INBOUND_SPAM_LOG_RETENTION_DAYS", store.InboundSpamLogRetentionDaysDefault),
// Optional DMARC aggregate ingest on port 25. Off unless exactly "true".
dmarcEnabled: os.Getenv("DMARC_REPORTS_ENABLE") == "true",
}
@@ -183,7 +205,14 @@ func antispamAction(raw string) string {
// saslRealm chooses the realm new SASL accounts live under. It mirrors the
// hostname Postfix's SASL layer uses so a client authenticating with a bare
// login resolves to the right account.
// login resolves to the right account. Postfix 3.7.11,
// src/xsasl/xsasl_cyrus_server.c, xsasl_cyrus_server_create(): serverFQDN is
// $myhostname (= SELFPOST_HOSTNAME, build/postfix-config.sh) and user_realm
// NULL while smtpd_sasl_local_domain is empty; Cyrus SASL 2.1.28,
// plugins/sasldb.c, sasldb_auxprop_lookup(), and common/plugin_common.c,
// _plug_parseuser(): a login
// without "@" is then looked up in the realm serverFQDN. A SASL_REALM other
// than SELFPOST_HOSTNAME does not follow that - assumptions A-48.
func saslRealm() string {
if r := os.Getenv("SASL_REALM"); r != "" {
return r
@@ -289,12 +318,46 @@ func run() error {
}
}
// The filter lists as files, on every start: the spam filter's
// configuration names all six, and a data directory from before the lists
// existed has none. Not fatal - the lists are the filter's business, and
// the panel has mail to relay either way.
if cfg.inboundEnabled && !restored {
if err := inbound.NewLists(st, cfg.antispamDir).Sync(); err != nil {
log.Printf("filter lists bootstrap: %v", err)
}
}
// The key the filter signs with has to be in the shared directory before
// the filter reads it, whether or not a filter is attached yet.
if cfg.inboundEnabled {
if _, err := inbound.FilterKey(cfg.antispamDir); err != nil {
log.Printf("filter key bootstrap: %v", err)
}
}
if err := st.EnsureSendLogRetentionDays(cfg.retentionDays); err != nil {
return fmt.Errorf("bootstrap send-log retention: %w", err)
}
// With the inbound relay on, the tailer also writes the inbound journal's
// rows for refused mail and prunes that journal.
var inboundTail *logtail.Inbound
if cfg.inboundEnabled {
if err := st.EnsureInboundSpamLogRetentionDays(cfg.inboundSpamLogRetentionDays); err != nil {
return fmt.Errorf("bootstrap inbound journal retention: %w", err)
}
inboundTail = &logtail.Inbound{Store: st, Retention: func() int {
days, err := st.GetInboundSpamLogRetentionDays(cfg.inboundSpamLogRetentionDays)
if err != nil {
log.Printf("inbound journal retention: %v", err)
return cfg.inboundSpamLogRetentionDays
}
return days
}}
}
var wg sync.WaitGroup
errc := make(chan error, 4)
// Room for every role to report a failure without blocking: seven at most.
errc := make(chan error, 7)
roles := []struct {
name string
@@ -302,6 +365,7 @@ func run() error {
}{
{"http", func(ctx context.Context) error { return serveHTTP(ctx, cfg, st) }},
{"journal-milter", func(ctx context.Context) error { return serveJournal(ctx, cfg, st) }},
{"sender-check", func(ctx context.Context) error { return serveSenderCheck(ctx, cfg, st) }},
{"log-tailer", func(ctx context.Context) error {
return logtail.Run(ctx, cfg.mailLog, st, func() int {
days, err := st.GetSendLogRetentionDays(cfg.retentionDays)
@@ -310,12 +374,20 @@ func run() error {
return cfg.retentionDays
}
return days
})
}, inboundTail, &logtail.SenderCheck{Store: st, Socket: cfg.senderSocket})
}},
{"rate-limit-recalc", func(ctx context.Context) error {
runAutoRateLimitRecalc(ctx, cfg, st)
return nil
}},
{"notifier", func(ctx context.Context) error { return runNotifier(ctx, cfg, st) }},
}
if cfg.inboundEnabled {
roles = append(roles, struct {
name string
fn func(context.Context) error
}{"inbound-journal-milter", func(ctx context.Context) error { return serveInboundJournal(ctx, cfg, st) }})
}
for _, r := range roles {
+59
View File
@@ -0,0 +1,59 @@
package main
import (
"context"
"log"
"github.com/mixeme/selfpost/internal/dnscheck"
"github.com/mixeme/selfpost/internal/health"
"github.com/mixeme/selfpost/internal/notify"
"github.com/mixeme/selfpost/internal/postfix"
"github.com/mixeme/selfpost/internal/store"
)
// runNotifier writes to the panel's users when something needs them
// (architecture.md § Notifications), until ctx is cancelled. It reads what the
// pages read - a domain's published records against the key this server signs
// with, the certificate file, Postfix's queue - with a DNS checker of its own,
// so that its hourly look is never a page's cached one.
func runNotifier(ctx context.Context, cfg config, st *store.Store) error {
ms := newMailStack(cfg, st)
dns := dnscheck.New(cfg.dnsResolvers)
// OpenDKIM's tables are rebuilt when a domain is added or deleted. On the
// first start of a version that signs the server's own name they do not
// have its line yet. A failure here costs the signature, not the message:
// OpenDKIM 2.11.0~beta2, opendkim/opendkim.c, mlfi_eoh(): with no
// SigningTable match and Mode s (build/opendkim.conf) the message is
// accepted unsigned. A line whose key the KeyTable lacks or that does not
// load is another matter - dkimf_apply_signtable() < 0 there answers
// SMFIS_TEMPFAIL (assumptions A-47).
if rebuilt, err := ms.Domains.EnsureServerSigned(); err != nil {
log.Printf("notifier: sign as %s: %v", cfg.hostname, err)
} else if rebuilt {
log.Printf("notifier: OpenDKIM now signs mail of %s", cfg.hostname)
}
n := notify.New(st, notify.Checks{
Domain: func(d store.Domain) (dnscheck.Domain, error) {
rec, err := ms.Domains.DKIMRecord(d)
if err != nil {
return dnscheck.Domain{}, err
}
// The server's own addresses are cached for a minute: one lookup
// for a pass over all the domains.
srv := dns.Server(cfg.hostname, false)
return dns.Domain(dnscheck.Query{
Name: d.Name,
Selector: d.DKIMSelector,
ExpectedDKIM: rec.Value,
Hostname: srv.Hostname,
ServerIPs: srv.IPs,
DMARCReportEmail: d.DMARCRua,
}, true), nil
},
Certificate: func() health.Certificate { return health.CheckCertificate(cfg.tlsCertFile) },
Queue: postfix.QueueMessages,
}, notify.NewMailer(cfg.hostname), cfg.hostname)
return n.Run(ctx)
}
+198
View File
@@ -0,0 +1,198 @@
package main
import (
"context"
"log"
"os"
"time"
"github.com/mixeme/selfpost/internal/domain"
"github.com/mixeme/selfpost/internal/health"
"github.com/mixeme/selfpost/internal/milter"
"github.com/mixeme/selfpost/internal/postfix"
"github.com/mixeme/selfpost/internal/store"
)
// aliveEvery is how often the sender check marks itself alive. The gap a
// restart finds since the last mark is the check's downtime, so the mark's
// age is the error of that figure.
const aliveEvery = 2 * time.Second
// serveSenderCheck runs the sender check (internal/milter/sender.go) on a
// socket of its own. Postfix attaches that socket strictly
// (default_action=tempfail in build/postfix-config.sh; Postfix 3.7.11,
// src/milter/milter8.c, milter8_comm_error(): "451 4.7.1 Service unavailable -
// try again later"), so while this role is not serving, mail is deferred:
// what that costs is counted here and
// by the log tailer, and shown on Server > Health
// (docs/plans/from-header-binding.md § Statistics).
func serveSenderCheck(ctx context.Context, cfg config, st *store.Store) error {
ln, err := listenMilter(cfg.senderSocket)
if err != nil {
return err
}
log.Printf("sender-check listening on %s", cfg.senderSocket)
// Statistics are the database's business and supervisord's, and the check
// is neither: they are kept beside it, and nothing of them is waited for
// before it serves.
now := time.Now()
mark := cfg.senderSocket + ".alive"
// The mark's age is read before the mark is kept again: asking supervisord
// may take half a minute, and the mark must not go stale meanwhile, or a
// restart in that half minute would count it as downtime.
var markAt time.Time
if fi, err := os.Stat(mark); err == nil {
markAt = fi.ModTime()
}
go keepAlive(ctx, mark, st)
go func() {
if err := st.StartSenderCheckStats(now); err != nil {
log.Printf("sender-check: statistics: %v", err)
}
if down, estimated := senderCheckDowntime(markAt, now, postfixStarted); down > 0 {
if estimated {
log.Printf("sender-check: was not answering for at most %s (supervisord did not say since when Postfix ran)", down.Round(time.Second))
} else {
log.Printf("sender-check: was not answering for %s while Postfix was running", down.Round(time.Second))
}
for _, part := range byDay(now.Add(-down), now) {
if err := st.AddSenderCheckDown(part.day, part.down, estimated); err != nil {
log.Printf("sender-check: statistics: %v", err)
}
}
}
}()
// The two tables the mail path itself goes by: whom Postfix lets send as
// what, and what OpenDKIM signs.
bind := milter.NewBindings(postfix.New(cfg.postfixDir).SenderLoginMapsPath())
signers := milter.NewSigners(domain.NewOpenDKIM(cfg.opendkimDir).SigningTablePath())
return milter.ServeSender(ctx, ln, bind, signers, st)
}
// postfixState is what supervisord said about Postfix: since when it runs, if
// it does; known is false when supervisord did not answer.
type postfixState struct {
started time.Time
running, known bool
}
// senderCheckDowntime is for how long the check was not serving before now
// while Postfix was: the time since the mark was last set (markAt; zero
// when there is none), or since
// Postfix started if that was later. No mark means the container is new, and
// a Postfix that is not running means mail was stopped for another reason -
// a rollout, a reboot - and neither is the check's downtime. When supervisord
// could not say, the time since the mark is taken and estimated is true: an
// upper bound, since Postfix may not have been running all of it (audit
// 2026-10-12, stage 2b - until then such a start counted nothing).
func senderCheckDowntime(markAt, now time.Time, ask func() postfixState) (down time.Duration, estimated bool) {
if markAt.IsZero() {
return 0, false
}
from := markAt
p := ask()
switch {
case !p.known:
estimated = true
case !p.running:
return 0, false
case p.started.After(from):
from = p.started
}
if d := now.Sub(from); d > 0 {
return d, estimated
}
return 0, false
}
// dayPart is the part of a stretch of downtime that fell on one UTC day.
type dayPart struct {
day time.Time
down time.Duration
}
// byDay cuts the time from 'from' to 'to' at the UTC midnights between them,
// so that a night's outage is not all booked on the morning it ended.
func byDay(from, to time.Time) []dayPart {
from, to = from.UTC(), to.UTC()
var parts []dayPart
for from.Before(to) {
next := time.Date(from.Year(), from.Month(), from.Day(), 0, 0, 0, 0, time.UTC).AddDate(0, 0, 1)
if next.After(to) {
next = to
}
parts = append(parts, dayPart{day: from, down: next.Sub(from)})
from = next
}
return parts
}
// postfixAskFor is how long postfixStarted keeps asking a supervisord that
// does not answer - at the container's start it may not be listening yet.
var postfixAskFor, postfixAskEvery = 30 * time.Second, 2 * time.Second
// postfixStarted asks supervisord since when Postfix has been running, again
// and again for postfixAskFor while it does not answer. supervisor 4.2.5,
// supervisor/rpcinterface.py, _interpretProcessInfo(): the uptime runs from
// the spawn of program "postfix", which is build/postfix-wrapper.sh; it waits
// for the milter sockets (up to MILTER_WAIT_TIMEOUT) before it execs
// "postfix start-fg", so the time given is up to that much early
// (assumptions A-46). For startsecs - that wait plus ten seconds
// (build/supervisord.conf) - after its start the program is STARTING, not
// RUNNING, and is taken here as not running: a panel restart within that
// time after Postfix's own counts no downtime.
func postfixStarted() postfixState {
deadline := time.Now().Add(postfixAskFor)
for {
procs, err := health.Processes()
if err == nil {
for _, p := range procs {
if p.Name != "postfix" || p.State != "RUNNING" {
continue
}
if up, ok := health.Uptime(p.Detail); ok {
return postfixState{started: time.Now().Add(-up), running: true, known: true}
}
}
return postfixState{known: true}
}
if time.Now().After(deadline) {
// Said aloud: this start's downtime is then an estimate.
log.Printf("sender-check: statistics: when Postfix started is not known: %v", err)
return postfixState{}
}
time.Sleep(postfixAskEvery)
}
}
// keepAlive sets the mark until ctx ends, and once a day drops the days of
// the statistics that are older than a year.
func keepAlive(ctx context.Context, path string, st *store.Store) {
touch := func() {
now := time.Now()
if err := os.Chtimes(path, now, now); err != nil {
if err := os.WriteFile(path, nil, 0o644); err != nil {
log.Printf("sender-check: alive mark: %v", err)
}
}
}
touch()
tick := time.NewTicker(aliveEvery)
defer tick.Stop()
prune := time.NewTicker(24 * time.Hour)
defer prune.Stop()
for {
select {
case <-ctx.Done():
return
case <-tick.C:
touch()
case <-prune.C:
if err := st.PruneSenderCheckDays(time.Now()); err != nil {
log.Printf("sender-check: statistics: %v", err)
}
}
}
}
+64
View File
@@ -0,0 +1,64 @@
package main
import (
"testing"
"time"
)
// The check's downtime is the time since its last mark, under a Postfix that
// was running: a new container has no mark, and a Postfix that is not up, or
// came up later than the mark, was not kept waiting by the check for all of
// that time.
func TestSenderCheckDowntime(t *testing.T) {
now := time.Date(2026, 10, 12, 18, 0, 0, 0, time.UTC)
var mark time.Time // the alive mark's time; zero is no mark
set := func(at time.Time) { mark = at }
postfix := func(started time.Time, running bool) func() postfixState {
return func() postfixState { return postfixState{started: started, running: running, known: true} }
}
silent := func() postfixState { return postfixState{} }
down := func(ask func() postfixState) (time.Duration, bool) { return senderCheckDowntime(mark, now, ask) }
if d, _ := down(postfix(now.Add(-time.Hour), true)); d != 0 {
t.Errorf("no mark: %s", d)
}
set(now.Add(-40 * time.Second))
if d, est := down(postfix(now.Add(-time.Hour), true)); d != 40*time.Second || est {
t.Errorf("Postfix up throughout: %s %v, want 40s, not estimated", d, est)
}
if d, _ := down(postfix(now.Add(-10*time.Second), true)); d != 10*time.Second {
t.Errorf("Postfix started within the gap: %s, want 10s", d)
}
if d, _ := down(postfix(time.Time{}, false)); d != 0 {
t.Errorf("Postfix not running: %s", d)
}
// supervisord did not answer: the time since the mark, an estimate.
if d, est := down(silent); d != 40*time.Second || !est {
t.Errorf("supervisord silent: %s %v, want 40s, estimated", d, est)
}
set(now.Add(time.Minute)) // a clock set back
if d, _ := down(postfix(now.Add(-time.Hour), true)); d != 0 {
t.Errorf("a mark in the future: %s", d)
}
}
// Downtime is booked on the days it fell on.
func TestByDay(t *testing.T) {
at := func(d, h, m int) time.Time { return time.Date(2026, 10, d, h, m, 0, 0, time.UTC) }
got := byDay(at(12, 23, 50), at(14, 0, 10))
want := []dayPart{{at(12, 23, 50), 10 * time.Minute}, {at(13, 0, 0), 24 * time.Hour}, {at(14, 0, 0), 10 * time.Minute}}
if len(got) != len(want) {
t.Fatalf("parts = %+v", got)
}
for i := range want {
if !got[i].day.Equal(want[i].day) || got[i].down != want[i].down {
t.Errorf("part %d = %+v, want %+v", i, got[i], want[i])
}
}
if one := byDay(at(12, 10, 0), at(12, 10, 0).Add(40*time.Second)); len(one) != 1 || one[0].down != 40*time.Second {
t.Errorf("within a day: %+v", one)
}
if none := byDay(at(12, 10, 0), at(12, 10, 0)); len(none) != 0 {
t.Errorf("no time: %+v", none)
}
}
+4
View File
@@ -30,6 +30,10 @@ DMARC_REPORTS_ENABLE=false
# INBOUND_ANTISPAM_MILTER=inet:antispam:11332
# INBOUND_ANTISPAM_MILTER_ACTION=accept
# Days the inbound journal keeps a row (7-365) until the panel says otherwise;
# it never holds more than 10 000 rows.
# INBOUND_SPAM_LOG_RETENTION_DAYS=90
# Coarse per-client-IP limit and message size on inbound smtpd (port 25).
# INBOUND_RATE_LIMIT_MESSAGES_PER_IP=20
# INBOUND_MESSAGE_SIZE_LIMIT=26214400
+16 -3
View File
@@ -9,8 +9,10 @@
# Requires INBOUND_RELAY_ENABLE=true in .env. The milter sees the real client
# IP, HELO and PTR — unlike the upstream, which only sees SelfPost's address.
#
# Pin the rspamd tag the same way the SelfPost image is pinned; bump it
# deliberately. Default milter action is fail-open (accept) so a down sidecar
# The rspamd image is pinned to one build, 3.11.1-3 - "3.11" and "3.11.1"
# move with each rebuild (Docker Hub rspamd/rspamd tags 3.11.1-0 ... -3) -
# and it is the build the comments in antispam/rspamd/ were read against;
# bump it deliberately, and read them again. Default milter action is fail-open (accept) so a down sidecar
# does not block backup-MX. Set INBOUND_ANTISPAM_MILTER_ACTION=tempfail to
# defer instead.
@@ -24,6 +26,17 @@ services:
- antispam
antispam:
image: rspamd/rspamd:3.11
# The milter: rspamd 3.11.1, conf/worker-proxy.inc, "milter = yes" for the
# rspamd_proxy worker, which the image binds to *:11332 (rspamd-docker
# v3.11.1+3, patches/rspamd-docker-config.diff), and runs as uid 11333
# (its Dockerfile, USER 11333:11333).
image: rspamd/rspamd:3.11.1-3
restart: unless-stopped
# No host ports: only SelfPost talks to the milter on the compose network.
volumes:
# The verdict headers SelfPost's inbound journal reads.
- ./antispam/rspamd/milter_headers.conf:/etc/rspamd/local.d/milter_headers.conf:ro
# The allow and deny lists kept in the panel: the rules, and the files
# SelfPost writes them to.
- ./antispam/rspamd/multimap.conf:/etc/rspamd/local.d/multimap.conf:ro
- ./data/antispam:/etc/rspamd/selfpost:ro
@@ -0,0 +1,75 @@
# Mounted into the sidecar as /etc/rspamd/local.d/milter_headers.conf.
#
# rspamd writes its verdict into two headers of every message it lets through:
# the action it chose and the score with the symbols behind it. SelfPost's
# inbound journal reads them (Inbound > Log shows them next to the message),
# and the upstream mailbox server receives them with the mail.
#
# rspamd removes headers of these names that a sender wrote himself before it
# adds its own - for every client, because of the two lines below. Without
# them it would do neither for a client it counts as local or one that
# authenticated: rspamd 3.11.1, src/plugins/lua/milter_headers.lua,
# skip_wanted() - skip_local and skip_authenticated are on by default, and
# "local" is loopback, a unix socket, or an address in local_addrs, the
# private ranges (src/lua/lua_ip.c, lua_ip_is_local(); conf/options.inc). The
# sender's own verdict would then pass untouched while the signature below,
# a custom routine that skip_wanted() never consults, signed it (assumptions
# C-R7). A private client address is ordinary here: a LAN sender, or every
# sender when Docker's proxy relays port 25 from the bridge gateway.
# Tried on 2026-10-14 in a throwaway rspamd/rspamd:3.11.1-3 (rspamc --ucl, a
# message carrying its own X-Spamd-Result and X-Rspamd-Action): without the
# two lines, from 10.0.0.5 or as an authenticated user, the milter reply
# removed neither header, added neither, and added the signature; with
# them, both removed and rspamd's own added, as from a public address.
#
# Without this file the journal still lists accepted mail, with no verdict
# next to it.
#
# The third header is the proof that the other two are rspamd's. The filter is
# attached so that mail still flows when it is down, and then nothing removes
# what a sender wrote: he could send his own verdict. So rspamd signs each
# message - an HMAC-SHA256 of the message's queue id under a key SelfPost
# generates and leaves in the directory it shares with this container - and
# SelfPost reads a verdict only next to a signature it can recompute. The
# signature travels on with the mail and is of no use to whoever sees it: it
# fits that one queue id.
#
# rspamd 3.11.1, src/plugins/lua/milter_headers.lua: a custom routine is called
# as function(task, common_meta) and returns error, headers to add, headers to
# remove, metadata; a remove value of 0 removes every header of that name
# (docs.rspamd.com/modules/milter_headers, "Custom routines").
# src/lua/lua_cryptobox.c, rspamd_lua_hash_create(): "sha256" with a key is
# HMAC-SHA256 over EVP_sha256(), the key being the bytes given - here the hex
# text. task:get_queue_id() is the Queue-Id the proxy took from the milter
# macro "{i}" or "i" (src/libserver/milter.c, rspamd_milter_macro_http(); read
# back in src/libserver/protocol.c), which Postfix sends at
# end-of-message (Postfix 3.7.11, src/global/mail_params.h, DEF_MILT_EOD_MACROS
# "i").
use = ["x-spamd-result", "x-rspamd-action", "selfpost-signature"];
skip_local = false;
skip_authenticated = false;
custom {
selfpost-signature = <<EOD
return function(task, common_meta)
local remove = { ['X-SelfPost-Filter'] = 0 }
-- The key is read for every message and kept nowhere: SelfPost may make a
-- new one (the file was lost, a backup was restored), and a key remembered
-- here would go on signing what SelfPost no longer accepts until someone
-- restarted this container. The file is one short line.
local key
local f = io.open('/etc/rspamd/selfpost/filter.key', 'r')
if f then
key = f:read('*l')
f:close()
end
local qid = task:get_queue_id()
if not key or key == '' or not qid or qid == '' then
return nil, {}, remove, {}
end
local hash = require "rspamd_cryptobox_hash"
local sig = hash.create_specific_keyed(key, 'sha256', qid):hex()
return nil, { ['X-SelfPost-Filter'] = sig }, remove, {}
end
EOD;
}
+90
View File
@@ -0,0 +1,90 @@
# Mounted into the sidecar as /etc/rspamd/local.d/multimap.conf.
#
# The allow and deny lists kept in the panel (Inbound > Filter lists). SelfPost
# writes them as six files under data/antispam/, which the fragment mounts at
# /etc/rspamd/selfpost; rspamd rereads a file when it changes, no restart.
# (rspamd 3.11.1, src/libserver/maps/map.c: a file map is watched with libev's
# ev_stat - inotify where the kernel has it, else a stat poll every
# map_watch_interval x map_file_watch_multiplier, 5min x 0.1, conf/options.inc;
# docs.rspamd.com/configuration/maps asks for an atomic rename, which
# internal/inbound writes with.)
#
# Deny wins over allow, whichever entries match. A denied sender is refused
# whatever else is found (the rule's action is a pre-result, below). An
# allowed one is not accepted outright - rspamd would then let whichever of
# the two rules ran first decide - but given a weight no spam score
# outweighs, which a refusal by the deny list still overrides. (rspamd
# 3.11.1, src/plugins/lua/multimap.lua: every rule is registered as a
# "normal" symbol, one with an action with the "passthrough" augmentation;
# no "prefilter" setting is read.)
#
# A sender rule looks at the envelope sender only, extract_from = "smtp":
# by default multimap.lua takes the header From: when the envelope sender is
# empty (the "from" callback, has_from('smtp') then has_from('mime')), so a
# bounce would be judged by an address the panel never counts it under
# (assumptions C-R4). Tried on 2026-10-14 in rspamd/rspamd:3.11.1-3: a
# bounce whose From: domain is on the deny list was refused by default and
# is not with this setting; a sender on the list is refused either way.
#
# A domain entry is that domain, not its subdomains. (rspamd 3.11.1,
# src/libserver/maps/map_helpers.c: a hash map is looked up by the whole key,
# case-insensitively - rspamd_icase_hash, rspamd_lc_cmp; "email:domain" gives
# the address's domain part, src/plugins/lua/multimap.lua, apply_addr_filter().)
#
# A refusal is rspamd's "pre-result": multimap.lua calls task:set_pre_result()
# with the rule's action and message, and that action stands whatever the score
# (docs.rspamd.com/modules/multimap, "action"). The message reaches the client
# as "554 5.7.1 <message>": src/libserver/protocol.c, rspamd_scan_result_ucl(),
# puts it into messages.smtp_message, src/libserver/milter.c,
# rspamd_milter_send_task_results(), replies with it.
#
# SelfPost counts how often an entry decided a message by what these rules
# leave behind: the deny message below and the names of the allow symbols.
# Change either and Inbound > Filter lists shows "never" for every entry.
SELFPOST_DENY_IP {
type = "ip";
map = "/etc/rspamd/selfpost/deny_ip.map";
action = "reject";
message = "Refused by this server's deny list";
description = "Client address on the deny list";
}
SELFPOST_DENY_FROM {
type = "from";
extract_from = "smtp";
map = "/etc/rspamd/selfpost/deny_from.map";
action = "reject";
message = "Refused by this server's deny list";
description = "Sender address on the deny list";
}
SELFPOST_DENY_DOMAIN {
type = "from";
extract_from = "smtp";
filter = "email:domain";
map = "/etc/rspamd/selfpost/deny_domain.map";
action = "reject";
message = "Refused by this server's deny list";
description = "Sender domain on the deny list";
}
SELFPOST_ALLOW_IP {
type = "ip";
map = "/etc/rspamd/selfpost/allow_ip.map";
score = -100.0;
description = "Client address on the allow list";
}
SELFPOST_ALLOW_FROM {
type = "from";
extract_from = "smtp";
map = "/etc/rspamd/selfpost/allow_from.map";
score = -100.0;
description = "Sender address on the allow list";
}
SELFPOST_ALLOW_DOMAIN {
type = "from";
extract_from = "smtp";
filter = "email:domain";
map = "/etc/rspamd/selfpost/allow_domain.map";
score = -100.0;
description = "Sender domain on the allow list";
}
+9 -1
View File
@@ -11,10 +11,18 @@
# storage detail, not a documented public API — VERIFY the exact path against
# the Caddy version you deploy (`docker compose exec caddy find /data/caddy/certificates -name '*.crt'`)
# before wiring it into the volume mount below.
# Read in the source (caddyserver/certmagic, storage.go, KeyBuilder.SiteCert():
# certificates/<issuer key>/<domain>/<domain>.crt; acmeissuer.go, issuerKey():
# the CA URL's host and path, "acme-v02.api.letsencrypt.org-directory" for Let's
# Encrypt). caddy/modules/caddytls/automation.go, DefaultIssuers(): ZeroSSL is a
# second issuer only when an email is configured, and the Caddyfile below sets
# none. Read at the projects' master of 2026-10-14: `caddy:2` is a moving tag.
#
# Merge with the base file — run this from the deploy/ directory (Compose
# resolves every relative path in both files against the directory of the
# FIRST -f file, i.e. deploy/, which is why paths below are ./caddy/...):
# FIRST -f file, i.e. deploy/, which is why paths below are ./caddy/... -
# docs.docker.com/compose/how-tos/multiple-compose-files/merge/: "Paths are
# evaluated relative to the base file"):
#
# docker compose -f docker-compose.yml -f caddy/docker-compose.caddy.yml up -d
+11 -1
View File
@@ -18,10 +18,12 @@
# The image tag below is FIXED on purpose: backup restore compares the manifest
# version against the running binary's version, so ":latest" would make that
# check meaningless. Bump the tag deliberately when you want to upgrade.
# The 2.x line is not on ghcr.io yet: build it locally under this exact name
# (README, Quick start) and Compose will use it without pulling.
services:
selfpost:
image: ghcr.io/mixeme/selfpost:2.0.0
image: ghcr.io/mixeme/selfpost:2.5.1
restart: unless-stopped
environment:
SELFPOST_HOSTNAME: "${SELFPOST_HOSTNAME:?set the mail/panel hostname, e.g. mail.example.com}"
@@ -47,6 +49,7 @@ services:
# How long the send log keeps rows before the background sweep deletes
# them — the main driver of /data growth over time.
SEND_LOG_RETENTION_DAYS: "${SEND_LOG_RETENTION_DAYS:-90}"
INBOUND_SPAM_LOG_RETENTION_DAYS: "${INBOUND_SPAM_LOG_RETENTION_DAYS:-90}"
# Sliding idle timeout for the panel login session, in days.
PANEL_SESSION_IDLE_DAYS: "${PANEL_SESSION_IDLE_DAYS:-7}"
# Resolvers the panel's PTR/SPF/DKIM/DMARC checks query directly, so they
@@ -113,6 +116,13 @@ services:
# FOWNER/FSETID are required by build/entrypoint.sh's permission
# self-healing; without them chmod fails with EPERM and the container
# crash-loops on start.
# Each capability above as capabilities(7) defines it (man7.org,
# "Capabilities list"); SETUID/SETGID are also what pipe(8) uses to become
# user=panel for dmarc-ingest. Not listed, so dropped: SYS_CHROOT, which
# chroot(2) needs - build/postfix-config.sh turns Postfix's chroot off.
# no-new-privileges only changes execve() (Linux,
# Documentation/userspace-api/no_new_privs.rst): root may still call
# setuid()/setgid() with these capabilities.
security_opt:
- no-new-privileges:true
cap_drop:
+6 -1
View File
@@ -8,7 +8,9 @@
# takes over 80/443 and proxies to the panel over the compose network
# instead). Merge it with the base file — run this from the deploy/ directory
# (Compose resolves every relative path in both files against the directory of
# the FIRST -f file, i.e. deploy/, which is why paths below are ./nginx/...):
# the FIRST -f file, i.e. deploy/, which is why paths below are ./nginx/... -
# docs.docker.com/compose/how-tos/multiple-compose-files/merge/: "Paths are
# evaluated relative to the base file"):
#
# docker compose -f docker-compose.yml -f nginx/docker-compose.nginx.yml up -d
#
@@ -54,4 +56,7 @@ services:
- ./nginx/certbot-www:/var/www/certbot
# Renewal twice a day is certbot's own recommended cadence; it no-ops
# until a certificate is within its renewal window.
# (eff-certbot.readthedocs.io, User Guide, "Automated Renewals": the
# sample cron runs "certbot renew" at 0 and 12 h; renew acts only on a
# certificate near expiry.)
entrypoint: sh -c 'trap exit TERM; while :; do certbot renew; sleep 12h & wait $${!}; done'
+2
View File
@@ -17,6 +17,8 @@ server {
server {
listen 443 ssl;
# nginx.org/en/docs/http/ngx_http_v2_module.html, "http2": "This directive
# appeared in version 1.25.1" (the fragment runs nginx:1.27).
http2 on;
server_name mail.example.com;
+7 -2
View File
@@ -6,7 +6,9 @@
#
# Merge with the base file — run this from the deploy/ directory (Compose
# resolves every relative path in both files against the directory of the
# FIRST -f file, i.e. deploy/, which is why paths below are ./traefik/...):
# FIRST -f file, i.e. deploy/, which is why paths below are ./traefik/... -
# docs.docker.com/compose/how-tos/multiple-compose-files/merge/: "Paths are
# evaluated relative to the base file"):
#
# docker compose -f docker-compose.yml -f traefik/docker-compose.traefik.yml up -d
@@ -25,7 +27,10 @@ services:
# Traefik only proxies the panel; it never sees the mail ports.
# These live on the selfpost service (not traefik's) because the
# docker provider reads routing labels off the container being routed
# to, not off traefik itself.
# to, not off traefik itself. (doc.traefik.io/traefik/v3.1/providers/
# docker/, "Routing Configuration": "Traefik uses container labels";
# with exposedByDefault=false only containers labelled
# traefik.enable=true.)
- traefik.enable=true
- traefik.http.routers.selfpost.rule=Host(`mail.example.com`)
- traefik.http.routers.selfpost.entrypoints=websecure
+7
View File
@@ -6,6 +6,13 @@
# by SelfPost/Postfix.
#
# Requires jq. Usage: ./extract-cert.sh <acme.json path> <domain> <output dir>
#
# The layout is not documented (doc.traefik.io/traefik/v3.1/https/acme/ says
# only "JSON file"); read in Traefik v3.1.0: pkg/provider/acme/local_store.go
# keeps map[resolver name]*StoredData, store.go StoredData{Account,
# Certificates}, provider.go Certificate{domain, certificate, key} with the PEM
# as []byte - base64 in JSON (Go encoding/json). "le" is the resolver
# docker-compose.traefik.yml names.
set -eu
ACME_JSON="${1:?path to acme.json}"
+464 -52
View File
@@ -27,21 +27,33 @@ Managed programs ([build/supervisord.conf](../build/supervisord.conf)):
| Program | User | Priority | Role |
|---|---|---|---|
| `opendkim` | root → `opendkim` (`UserID` in opendkim.conf) | 100 | DKIM signing milter |
| `panel` | panel | 200 | HTTP UI + journal-milter + log-tailer + rate-limit recalc |
| `postfix` | root (wrapper) | 300 | MTA — started only after both milter sockets exist |
| `panel` | panel | 200 | HTTP UI + sender check + journal-milter + log-tailer + rate-limit recalc; with the inbound relay on also the inbound journal milter |
| `postfix` | root (wrapper) | 300 | MTA — started only after the milter sockets exist (OpenDKIM's, the sender check's, the journal milter's and, with the inbound relay on, the inbound journal milter's) |
| `postfix-reload` | root | — | On-demand `postfix reload` (autostart off) |
| `cert-reload` | root | 400 | Daily `postfix reload` for renewed TLS certs |
| `cert-reload` | root | 400 | Daily: re-copies a renewed TLS certificate for Postfix and reloads it |
| `logrotate` | root | 400 | Periodic `mail.log` rotation |
Start order: OpenDKIM → panel (opens journal-milter socket) → Postfix wrapper
polls unix sockets (timeout `MILTER_WAIT_TIMEOUT`, default 30s) then
`postfix start-fg`.
Start order: OpenDKIM → panel (opens the sender check's and the journal-milter's sockets) → Postfix wrapper
polls unix sockets (timeout `MILTER_WAIT_TIMEOUT`, default 30s) then runs
`postfix start-fg` and stays its parent: on TERM it runs `postfix stop`, so a
stop of the program or of the container stops Postfix as Postfix means it to
be stopped (master runs in a session of its own, which supervisord's signal
never reaches). Program `postfix` has `startsecs` = `MILTER_WAIT_TIMEOUT` + 10
(set by the entrypoint), so the wrapper giving up on the sockets, or a Postfix
that will not start, is an exit supervisord counts as too quick: once
`startretries` (3) are spent, the program is FATAL. For those seconds after
each start it is STARTING.
`crashexit` event listener exits the container on any managed process FATAL so
Docker `restart: unless-stopped` recreates a broken instance.
`crashexit` event listener exits the container when a program of the mail path
is FATAL, so Docker `restart: unless-stopped` starts it again. `logrotate`'s
FATAL (a config logrotate would ignore) stays on Server › Health instead.
The panel runs `postfix-reload` through supervisord's XML-RPC interface and
waits for its exit status (`internal/supervisor`, `RunOnce`); the program
rests EXITED after a run.
**Liveness:** `GET /healthz` (unauthenticated) returns 200 when opendkim,
panel, and postfix are RUNNING; Docker `HEALTHCHECK` uses the same probe.
panel, and postfix are RUNNING; Docker `HEALTHCHECK` uses the same probe (its
90-second start period covers postfix's STARTING).
---
@@ -50,6 +62,7 @@ panel, and postfix are RUNNING; Docker `HEALTHCHECK` uses the same probe.
```
Client ──TLS+SASL──► Postfix (465 smtps, optional 587 submission)
│
├─► sender check (From: against the login, tempfail on failure)
├─► OpenDKIM milter (sign, tempfail on failure)
├─► journal-milter (send log + L2 rate limits, fail-open)
└─► outbound MX delivery (port 25 client)
@@ -74,10 +87,65 @@ smtp/inet` removes Debian's stock smtpd). Outbound delivery still uses the
`smtpd_tls_security_level=encrypt`.
- **25/smtp inet** — when `INBOUND_RELAY_ENABLE=true` and/or
`DMARC_REPORTS_ENABLE=true`. No SASL, no OpenDKIM, no journal-milter —
`smtpd_milters` is always overridden on this service (to the antispam milter
or to nothing), because main.cf's submission chain would otherwise let a
signing outage defer relayed mail and would file inbound mail into the
outbound send log under the sender's domain.
`smtpd_milters` is always overridden on this service, because main.cf's
submission chain would otherwise let a signing outage defer relayed mail and
would file inbound mail into the outbound send log under the sender's
domain. With the inbound relay on the chain is the optional antispam milter,
then the panel's **inbound journal milter** (unix socket
`INBOUND_JOURNAL_MILTER_SOCKET`, default
`/run/selfpost/inbound-journal.sock`, `default_action=accept`); with DMARC
ingest alone it is empty. Postfix hands a message to its milters one after
another at end-of-message, each seeing what the one before added, so the
journal milter sees only mail the filter let through and reads the filter's
verdict headers (`X-Rspamd-Action`, `X-Spamd-Result`) off it. It writes one
`inbound_spam_log` row per recipient at an inbound domain - a DMARC report
address is not one - and never refuses anything
([internal/milter/inbound.go](../internal/milter/inbound.go)). The headers
are read only while a filter is attached, and only next to the filter's
signature: a third header, `X-SelfPost-Filter`, the HMAC-SHA256 of the
message's queue id under the key in `/data/antispam/filter.key`
(`inbound.FilterKey` makes it on first start; rspamd reads it through the
mounted directory and signs in a custom routine of `milter_headers.conf`).
With the filter down - it is attached fail-open - a sender's own verdict
headers carry no signature and the row is that of mail nobody judged.
Refused mail never reaches this milter.
Refusals come from `mail.log`: this listener logs as `postfix/inbound`
(`syslog_name`), and the log tailer turns a `reject:` line of
`postfix/inbound/smtpd` (Postfix's own policy) and a `milter-reject` /
`milter-hold` / `milter-discard` line of `cleanup` (the filter's verdict,
counted only for a queue id that listener opened) into a row, dated by the
line, when the recipient is at an inbound domain
([internal/logtail/inbound.go](../internal/logtail/inbound.go)). A line
names one recipient, so a message refused at end-of-message is one row
whatever its recipient count. The tailer saves its place every five seconds,
so after an unclean stop it reads some lines again; the lines it reads on
start are written only if the same row is not there already. The
journal is pruned every six hours: rows
older than the `inbound_spam_log_retention_days` setting, then down to
10 000.
**Filter lists** (`inbound_spam_list`): allow and deny entries for the
external filter - sender address, sender domain, client address or network -
validated and normalised by `internal/inbound` (`lists.go`) and written as
six map files under `/data/antispam/` (`ANTISPAM_DIR`), each atomically, on
every change and on every start with the relay on. SelfPost never consults
them. The sidecar mounts the directory and rspamd's multimap rules
([deploy/antispam/rspamd/multimap.conf](../deploy/antispam/rspamd/multimap.conf))
read them: a deny entry's rule carries the action `reject`, which rspamd
sets as a pre-result, an allow entry is a symbol weighing -100. Allow is
deliberately not an `accept` action: with two pre-results rspamd keeps the
one set first, and deny then won or lost by rule order (seen on rspamd
3.11, date not recorded). A refusal by the deny list overrides any score,
so deny wins whichever entries match. A sender entry is matched against
the envelope sender only (`extract_from = "smtp"`; by default rspamd takes
a bounce's `From:` header instead - rspamd 3.11.1,
`src/plugins/lua/multimap.lua`, the `from` callback).
How often an entry decided a message is not stored: `Lists.Usage` reads it
off the journal on each view of Filter lists - the rows a list decided (the
deny rule's reply, the allow rule's symbol, which the journal milter keeps
ahead of its six-symbol cap) matched against the entries as rspamd matches
them. When that stops being good enough is written in
[plans/inbound-antispam-panel.md](plans/inbound-antispam-panel.md) § Filter
list usage.
**Inbound relay** (`INBOUND_RELAY_ENABLE=true`): accepts only
`relay_domains` + `relay_recipient_maps` (`reject_unauth_destination`,
`reject_unlisted_recipient`). Maps under `/data/postfix/` (`relay_domains`,
@@ -93,32 +161,59 @@ smtp/inet` removes Debian's stock smtpd). Outbound delivery still uses the
`INBOUND_RATE_LIMIT_MESSAGES_PER_IP` and message size uses the greater of
`INBOUND_MESSAGE_SIZE_LIMIT` and `DMARC_MESSAGE_SIZE_LIMIT`.
- **No open relay** — `permit_sasl_authenticated`, `reject_unauth_destination`;
`smtpd_sender_login_maps` + `reject_sender_login_mismatch`.
`smtpd_sender_login_maps` + `reject_sender_login_mismatch`. That holds the
envelope sender; the `From:` header is held to the same table by the
panel's sender check (§ Panel roles).
- **Level-1 rate limit** — `smtpd_client_message_rate_limit` /
`anvil_rate_time_unit` from `RATE_LIMIT_*` env vars; independent of milter.
- **Chroot disabled** for all services (DNS/TLS inside container).
- **TLS certs** — read-only mount at `TLS_CERT_FILE` / `TLS_KEY_FILE`; daily
reload via `cert-reload`.
- **TLS certs** — read-only mount at `TLS_CERT_FILE` / `TLS_KEY_FILE`, copied
at start into `/etc/postfix/tls-internal` (Postfix wants the key root-owned
and `0600`, which the mount cannot give), and Postfix reads the copy.
`cert-reload` compares the mount with the copy daily; when they differ and
the certificate and key belong together it renews the copy (each file
renamed into place) and reloads Postfix.
### OpenDKIM
Per-domain keys under `/data/opendkim/keys`; `KeyTable` / `SigningTable` maintained
by the panel. Socket `/run/opendkim/opendkim.sock`.
The server's own name (`SELFPOST_HOSTNAME`) has a key and a line in both
tables too, without being a row of the registry
(`domain.Service.SetServerName`): the panel's notifications are sent as
`postmaster@` that name and signed with it. No application can use it as an
envelope sender - the sender map is built from the registry - but the
`From:` header is not held against the login, so an application can still
have a message signed as the server ([security.md](security.md) § Accepted
risks). A sending domain of the same name shares its key and selector with
the server; deleting that domain leaves the server signed under the default
selector, which is the same key only if that was the domain's. At its start
the notifier makes sure the tables have the line and name the selector the
panel shows (`EnsureServerSigned`): tables are otherwise rebuilt only when a
domain is added or deleted.
### Panel binary ([cmd/panel](../cmd/panel))
One process, four roles:
One process, five roles (six with the inbound relay on):
1. **HTTP server** — `:8080` (`PANEL_HTTP_ADDR`); HTTPS terminated by reverse
proxy only. On start it runs `postconf -h` once for the deferred-mail retry
proxy only. On start it runs `postconf -h -x` once for the deferred-mail retry
parameters (`queue_run_delay`, `minimal_backoff_time`,
`maximal_backoff_time`, `maximal_queue_lifetime`, `bounce_queue_lifetime`,
`delay_warning_time`) and caches the snapshot on the handlers config. The
`delay_warning_time`), reads each as Postfix does - one number and one
unit, the parameter's own default unit when there is none (`5d` for the
lifetimes, so a bare `3` is three days), `bounce_queue_lifetime` capped at
`maximal_queue_lifetime` as qmgr caps it (Postfix 3.7.11,
`src/global/conv_time.c`, `src/qmgr/qmgr.c`) - and caches the snapshot on
the handlers config. The
Outbound queue box and a delivery's `deferred` / `bounced` history print those
numbers; they never call `postconf` per request. If `postconf` is missing,
the panel logs a warning and uses Postfix 3.x compiled-in defaults
(`300s` / `4000s` / `5d` / `0`) with a muted note on the box.
2. **journal-milter** — unix socket `JOURNAL_MILTER_SOCKET`; records From/To/
2. **sender check** — unix socket `SENDER_CHECK_SOCKET`, first in the chain
and **strict** (`default_action=tempfail`): § Sender check below.
3. **journal-milter** — unix socket `JOURNAL_MILTER_SOCKET`; records From/To/
Subject/SASL user at DATA; enforces level-2 rate limits; **fail-open**
(`default_action=accept`) so milter failure does not stop mail. Domain
ceilings apply to every client IP; an application ceiling **overrides** the
@@ -130,13 +225,176 @@ One process, four roles:
this process has admitted but not yet written (`internal/milter/inflight.go`),
so concurrent sessions cannot each spend the same last slot; a reservation
is released at end-of-message, on ABORT, or after a 10-minute TTL.
3. **log-tailer** — follows `MAIL_LOG`, updates send-log delivery status by
4. **log-tailer** — follows `MAIL_LOG`, updates send-log delivery status by
queue-id. Send-log `queued → sent` transitions depend on this goroutine alone
(`UpdateStatus` is only called from [internal/logtail](../internal/logtail/logtail.go)).
4. **rate-limit recalc** — every six hours (and on demand from Domain settings and
It also keeps the lines Postfix writes under a queue id, for the message's
page (§ Log tailer, *Stored lines*), and counts the mail Postfix deferred
because the sender check did not answer.
5. **rate-limit recalc** — every six hours (and on demand from Domain settings and
the application form), recomputes level-2 **Auto** rate limits from send-log statistics.
6. **notifier** — writes to the panel's users when something needs them
(§ Notifications below).
Milter chain in Postfix: OpenDKIM (tempfail) then journal (accept on failure).
Milter chain in Postfix: the sender check (tempfail), OpenDKIM (tempfail),
then the journal (accept on failure). Mail submitted locally with
`sendmail(1)` — the notifier's — passes OpenDKIM alone (`non_smtpd_milters`):
it is signed, it is the panel's own and so needs no check of whose it is, and
it is not an application's mail to be journalled or limited. Bounces Postfix
generates pass no milter.
### Sender check
[internal/milter/sender.go](../internal/milter/sender.go) holds a message's
`From:` header against the application that sends it
([plans/from-header-binding.md](plans/from-header-binding.md)). Postfix holds
the envelope sender against the SASL login, and OpenDKIM chooses the signing
key by the header; until `2.4.0` nothing connected the two, and an
application could have mail signed as another domain of the server or as the
server itself.
- **The rule is the envelope's.** The check reads the file Postfix reads for
`smtpd_sender_login_maps` and makes the same lookup: the address itself is
bound to the login, or `@domain` is. So an application set to any address of
its domain may write any address of that domain, and one with a list only
the listed ones; everything else is refused - another registered domain,
the server's name, a domain the server knows nothing of. One table for
both senders, read as Postfix reads a texthash table and looked up as
Postfix looks it up: the address, the address without its `+extension`
(`recipient_delimiter`, which `build/postfix-config.sh` sets for that
reason), then `@domain` - and the **first of these keys in the table
decides**, its owners alone: an address listed for one application is not
open to another that holds the whole domain. Logins are compared without
regard to case, as Postfix compares them (Postfix 3.7.11,
`src/smtpd/smtpd_check.c`, `reject_auth_sender_login_mismatch()`, over
`src/global/mail_addr_find.c`, `mail_addr_find_opt()`). A key written
twice makes Postfix refuse the whole table, and the check defers then too.
- **How the header is read.** Exactly one `From:` field (none, or more than
one, is refused), parsed as an address list that must hold exactly one
address. Then a cruder second look at the raw value: OpenDKIM finds the
address with a parser of its own, and wherever the two could read a line
differently - a display name, a comment, a quoted string - the other
address would have to be spelled out in it. So no `@name` may appear there
that is another name the server signs for; those names are read from
OpenDKIM's own `SigningTable`; the display name is looked at once more
as decoded, since a mail program shows `=?utf-8?q?director=40example.org?=`
as an address though no `@` is written in it. And, needing no knowledge of
that parser:
outside quoted strings the line has exactly one `@`. A folded line is
unfolded first, and a display name in a charset Go does not convert is
passed through unread - only the address is judged.
- **The verdict** comes at the end of the headers: `550 5.7.1`, permanent,
naming the line, as a reply code of the milter's own. What the client wrote
is reduced to printable ASCII and cut before it is said back, logged or
stored. A message with no SASL login is refused as well; the submission
ports do not let one this far.
- **Strict, and therefore small.** Postfix attaches the socket with
`default_action=tempfail`: while the panel does not answer on it within the
milter timeouts, mail is deferred, not accepted unchecked. The role
therefore reads no database. Its two inputs are files the panel rewrites by
rename and the mail path itself goes by; they are read again when their
time or size has changed, and a file that cannot be read defers mail - a
table from before would let a deleted application go on sending. The
send-log row of a refusal (`status=rejected`, `reason=from`,
`header_from`) is handed to another goroutine through a buffer and is not
waited for. It is one row for the message, naming the first recipient and
how many more there were: a row per recipient would let one refused message
write a thousand.
- **What it costs is counted.** The role marks itself alive every two
seconds (the socket's path with `.alive`, under `/run`, so a new container
starts without one). At start the time since the mark is the check's
downtime - counted from Postfix's own start if that was later, and not at
all when supervisord says Postfix is not running, since mail was then
stopped for another reason. Supervisord is asked for half a minute; when it
does not answer, the time since the mark is taken as an upper bound and the
day is marked estimated (`sender_check_days.estimated`), which the card
shows as ≈. The log tailer counts the mail: Postfix
writes a warning naming the socket and then, from the same smtpd process,
`milter-reject: … 451 4.7.1 Service unavailable` from the connection on
(the client is told `454 4.3.0` at `AUTH` and leaves), so what is counted
is a connection turned away, once - a client brings a message when it
connects. The reject line does not name the milter and the journal milter's
level-2 refusal reads the same, hence the warning. A panel that is running
and stops answering in the middle of a message (its socket accepts, nothing
replies) is met by cleanup, not smtpd: past `DATA` it is cleanup that talks
to the milters, and after the content timeout it writes `warning: milter
unix:<socket>: can't read SMFIC_HEADER reply packet header` and refuses the
message with `4.7.1` (seen on the e2e stand with the panel frozen). One
cleanup process serves message after message, so each such warning is
counted as a message. The two are counted apart (the owner, 2026-10-14):
connections turned away in `deferred`, messages deferred mid-message in
`messages` of `sender_check_days`, a row per UTC day kept a year, and
Server › Health shows today, 30 days and 12 months in a column each. The
counts also see a panel that is running and not answering, which the mark
does not. Downtime is cut at the UTC midnights it crossed. A log read again
after an unclean stop does not count again: the count and the place in the
log it reached (the file's fingerprint and the offset past the line) are
saved in one transaction, and while the tailer catches up a warning at or
before that place in the same file is passed over - a second warning of
the same second is not lost; in another file the time decides.
- **Why it is not a process of its own.** Decided with the owner: measure
first. The role has a socket of its own so that the journal stays
fail-open, and moving it out later changes where it is started and nothing
else.
### Notifications
[internal/notify](../internal/notify/notify.go) reports four things to the
users of the panel: a sending domain's DNS record that stopped matching, a
level-2 rate limit that refused mail, the TLS certificate about to expire, a
message more than three days in the queue. The rules of when a message goes
out are the owner's and are written in
[plans/panel-notifications.md](plans/panel-notifications.md) § When a message
is sent; the operator's view is guide § Notifications.
- **Readings.** Once an hour, starting three minutes after the process does:
every sending domain's records through a DNS checker of the notifier's own,
forced past the cache (`cmd/panel/notifier.go` builds the same query a
domain's page does); the certificate file; `postqueue -j` for the arrival
time of each queued message. Once a minute: the send log's `rejected` rows
of the last day, through a partial index of their own.
- **A refused row is not believed whole.** Its domain is the one in
`MAIL FROM`, which the client typed and Postfix has not yet held against
the login when the milter refuses; the same status is written when an
application is refused for its client address or its `From:` line. Since
`2.4.0` a row says which (`send_log.reason`) and only a limit's rows are
taken; a row from before has no reason. So refusals are counted by
the domain the login is registered under, only for logins that exist, and
only while a level-2 limit is set on the login or its domain
(`Store.LimitRefusalsSince`). One subject per sending domain, whichever of
its applications are refused.
- **State** is a row per subject in `notify_state` (`dns:<domain>`, `cert`,
`queue`, `limit:<domain>`), which is what makes each thing be said
once and a restart say nothing again. A DNS subject goes *pending* on the
first bad check and *open*, with a message, on a later one at least 45
minutes after it - a count alone would let the last check before a restart
and the first after it stand for "an hour apart"; back the same way. The
certificate's row remembers which threshold was said for which expiry. A
deleted domain's row goes with it.
- **Said means sent.** A problem that arises while no user can be told, or
while Postfix does not take the message, stays unsaid and is told at a
later check. Its ending, with nobody left to tell, is simply over. If one
recipient's copy fails and another's goes out, the subject is said.
- **The notifier's own mail is not a subject.** A notification waiting for a
mailbox that does not answer, or its bounce coming back to
`postmaster@<hostname>` - which receives no mail - is left out of the
queue check.
- **A ceiling** of 60 messages an hour stands over all of it.
- **No information is not a finding.** A lookup that did not answer, a queue
that cannot be listed, a certificate that cannot be read move no state.
- **Recipients** come from the store at the moment of sending: global users
for everything, domain users for subjects that are a domain's; only users
with an e-mail and their own switch on. One message per recipient, so users
do not see each other's addresses.
- **The instance switch** (settings key `notifications`, on unless turned
off) stops the checks themselves, not only the mail: no state moves while
it is off.
- **Sending** is `sendmail -i -f postmaster@<hostname> -- <address>` as the
unprivileged panel user, which is a member of `postdrop` and so can write
the maildrop directory without the setgid bit `no-new-privileges` forbids.
A message that could not be handed over leaves the state where it was and
is tried at the next check. The text names the page to open and has no
link: the panel does not know its own address.
### Log tailer and `mail.log` rotation
@@ -157,41 +415,190 @@ the latter can drop `status=sent` lines and leave send-log rows stuck at
recreates the file lazily on first write as mode `0600`, which the unprivileged
panel user cannot read). `follow()` drains the old inode once more before
switching descriptors; the panel treats a missing log file as an empty tail, not
an error.
an error. That drain does not close the gap entirely: postlogd keeps writing to
the renamed file until the reload reaches it, seconds later, and a line written
there after the drain is not read. Keeping the old file open until it has
stopped growing is the fix; it waits for the measurement below to say how
much it matters.
**Delivery lines are read at the head of the line.** The pattern that turns a
line into a row's status (`deliveryRe`) is matched after the timestamp, the
host name and `program[pid]:`, at the queue id - like the stored lines' and the
inbound journal's. The tail of a line is the far end's words (a reply, a HELO
name), and a pattern that could start there let a stranger's HELO name file a
delivery (audit 2026-10-12, M12).
**Read offset is persisted** (`logtail_state` table, in the baseline schema): the
tailer stores its position plus a fingerprint of the log's first 512 bytes, and
on start resumes from it, parsing the tail written while the panel was down. If
the fingerprint no longer matches (rotated or recreated in the meantime) it reads
the current file from the start; re-parsing lines is harmless because
`UpdateStatus` writes the same status onto the same row. Only a first-ever start,
with nothing stored, begins at end-of-file, so installing the panel does not
replay a pre-existing log.
tailer stores its position plus a fingerprint of the log - the hash of its first
512 bytes and the file's device and inode, so two logs that begin alike are two
files and a renamed file keeps its fingerprint - and on start resumes from it,
parsing the tail written while the panel was down. If the fingerprint no longer
matches, the log was rotated in the meantime: the tailer first opens
`mail.log.1` (left plain by `delaycompress`), and if that is the file the saved
place belongs to it reads it from there to its end, then the new file from its
start - what Postfix wrote between the stop and the rotation is not lost. A
`mail.log.1` that is another file (rotated twice) is logged with the time from
which nothing was read. Re-parsing lines is harmless because `UpdateStatus`
writes the same status onto the same row. Only a first-ever start, with nothing
stored, begins at end-of-file, so installing the panel does not replay a
pre-existing log.
**A refusal's line, and a refusal the panel did not write.** A message refused
for its `From:` line has a queue id (cleanup gives one before the headers are
judged); the sender check writes it on the row, and the message's page shows
the lines stored under it - only those of the minutes around the refusal, since
Postfix frees the id at once. A message the journal milter refused at
`MAIL FROM` has none: the tailer reads smtpd's `NOQUEUE: milter-reject: MAIL
from <client>[<address>]: …; from=<…> … sasl_username=<login>` - the login is
Postfix's last field on an authenticated session (Postfix 3.7.11,
`src/smtpd/smtpd.c`, `smtpd_whatsup()`), and a line without one is no
application's - and puts it on the nearest refused row of that client
address, login and sender within two minutes
(`send_log.refusal_line`; the milter writes `send_log.client_ip`), retrying
for 30 s when the row is not written yet; a line already on a row goes on no
other. The sender check writes its
row through a buffer after answering 550; a panel stopped in between wrote
none. The tailer remembers each queue id's login from smtpd's `client=` line,
and 30 s after cleanup's `milter-reject: END-OF-MESSAGE … 5.7.1 From:` line it
writes the row from the line if no refused row of that id exists
(`send_log.recovered`, shown on the page). cleanup's line names one
recipient, the message's last (`src/cleanup/cleanup_milter.c`), so a row
written from it has that one, which its page says. It reads from its saved offset, so
a kill is covered as well as a clean stop.
**Expiry.** A message Postfix gives up on at the end of its queue lifetime has
no line per recipient: qmgr writes one line for the message, `<id>:
from=<...>, status=expired, returned to sender` (`force-expired` after
`postsuper -e`; Postfix 3.7.11, `src/qmgr/qmgr_active.c`,
`qmgr_active_done_25_generic()`), and the recipients' last lines say
`deferred`. The tailer reads that line and closes the message's open rows -
each recipient's newest, `queued` or `deferred` - as `bounced`.
**Queue reconcile** is the backstop for what the log cannot explain at all: a
row still `queued` more than two minutes after it was accepted, whose queue id
`postqueue -p` no longer lists, is marked `bounced` (swept every five minutes,
row still open - `queued`, or `deferred` and waiting - more than two minutes
after it was accepted, whose queue id `postqueue -p` no longer lists, is
marked `bounced` with the reason `reconcile` (swept every five minutes,
[internal/logtail](../internal/logtail/logtail.go),
[postfix.QueueIDs](../internal/postfix/queue.go)). Postfix having dropped the
message means nothing more will ever be reported about it, so the row can only
be closed on an assumption, and it is closed as a failure because a delivery the
panel cannot evidence must not be shown as one. Three things keep the sweep from
panel cannot evidence must not be shown as one. What keeps the sweep from
guessing where it need not: it starts only after the tailer has read to
end-of-file once (on a restart the log itself holds the answer), the two-minute
grace covers messages merely in flight, and a `postqueue` that cannot be read
leaves every row untouched rather than closing them all. Now that the log
survives the container, reaching this path means the lines are gone for good —
rotated past fourteen files while the panel was down, or deleted.
end-of-file once (on a restart the log itself holds the answer); the two-minute
grace covers messages merely in flight; a `postqueue` that cannot be read, or
whose listing is over the 4 MiB cap, leaves every row untouched rather than
closing them all or those past the cut; and the close, by the row's id, is
conditional on the row still being open, so a final status the log wrote in
the meantime stands. A deferred row is the sweep's too because Postfix can
drop a deferred message without a final line for its recipients: at expiry
before `2.5.2` read that line, and with `postsuper -d`, which logs only
`removed`. The listing's `#` - a message forced to expire - is read as part
of the id's flags (`src/postqueue/showq_compat.c`, `showq_message()`).
**Which row a delivery line is.** The journal milter writes a row's
recipient as Postfix will log it: the address the client gave, its quoted
local part unquoted and quoted again only where it needs to be (`"ab"@x`
becomes `ab@x`, `"a b"@x` stays; Postfix 3.7.11, `tok822_parse.c`,
`quote_822_local.c`), case kept, and a recipient given twice - in any case -
once, since cleanup keeps one (`cleanup_out_recipient.c`, `been_here()`
folding case). A delivery line names it in `orig_to=` when Postfix rewrote
the address, in `to=` otherwise (`log_adhoc.c` writes `orig_to=` only then),
and is matched without regard to case. It updates only the newest row of its
queue id and recipient: Postfix reuses a short queue id once the message has
left the queue, and an old row must not be rewritten. `UpdateStatus`'s transaction
starts with a write, so a commit by the `dmarc-ingest` process in between
cannot fail it with `database is locked`.
Now that the log survives the container, reaching this path should mean the
lines are gone for good — rotated past fourteen files while the panel was down,
or deleted — or that the tailer was not reading.
Whether the sweep earns its place is measured, not argued (audit 2026-10-12,
H3). `delivery_log_days` (migration `0007`) keeps, per UTC day for a year:
`reconcile_closed`, the rows the sweep closed; `reconcile_missed`, those of
them whose final line - sent, bounced, or the message's expiry - was in
`mail.log` or `mail.log.1` when the sweep closed them - the tailer did not
read a line that was there, which is what a lost line looks like (it is never
read late; a deferred line is not one, a deferred row has it by definition); `reconcile_contradicted`, those
a delivery line read later gave a status - the log's status replaces the
sweep's and the row's reason becomes `reconcile-contradicted`, so a row counts
once and can be found; `tailer_blind_seconds`, the time `follow()` could not
read `mail.log` (the file gone, reopening it failing; stretches under three
seconds - a rotation's tick between rename and create, which loses nothing -
are not counted); and `queued_over_lifetime`, the rows still open (`queued`
or `deferred`; `queued` only before `2.5.2`) past the queue lifetime plus a
maximal backoff, as the day's last sweep saw them - rows
stuck although the sweep runs (mail on hold, or a queue that could not be
listed). What removing the sweep would leave queued is closed − missed −
contradicted: the rows nothing in the log ever resolved. No page shows the
table: it is read off a copy of the database. The rule, written before the
data: `reconcile_closed = 0` for a month → remove the sweep; closed, but
neither missed nor contradicted → keep it; missed or contradicted → the tailer
loses or delays lines - fix the tailer (the two deferred steps: the rotation
window, pausing the sweep while blind) and keep the sweep; if what remains
after that is never resolved by the log, mark such rows `lost` or remove the
sweep. Decided after thirty days, or sooner if `reconcile_missed` or
`reconcile_contradicted` is ever non-zero.
**Stored lines** (`send_log_lines`, since 2.2.0;
[internal/logtail/lines.go](../internal/logtail/lines.go)). A send-log row
lives for the retention window, ninety days by default, and `mail.log` for a
day before it is rotated; a message's page (`/outbound/log/{id}`) shows what
Postfix wrote about it, so the tailer keeps those lines in the database as it
reads them:
- **What is kept:** a line headed by a queue id — after the timestamp, the
host name and `program[pid]:` — with that id, the line's own time and the
line as written, cut to 4096 bytes. The id is read from the head of the line
only: the tail is the far end's text. `NOQUEUE` and `warning` stand in the
same place and are not ids; the pattern accepts Postfix's short form
(capital hexadecimal) and its long form (no vowels).
- **Port-25 mail is left out.** A queue id opened by `postfix/inbound/smtpd`
is a stranger's message and never gets a send-log row; none of its lines
are stored, up to qmgr's `removed` or until another listener opens a
message under the id (4096 such ids are remembered). What a stranger can
make Postfix log is therefore not written to the database at his rate.
- **No check for a row at write time.** A message's first lines (`client=`,
`message-id=`) are logged before the journal milter writes its row at
end-of-message, so the tailer cannot ask whether the id is in the send log.
Lines that never get a row — bounces Postfix generates, port-25 mail met in
mid-flight after a restart — are kept for a while.
- **Written per pass, not per line:** the lines of one pass over the log go
in one transaction (`InsertSendLogLines`), 500 at most, before the read
offset is saved. A commit per line would be a disk sync per line on the
connection the journal milter shares.
- **Retention is one rule:** a line older than an hour whose queue id no
send-log row carries is deleted (`PruneSendLogLines`), at start and after
every six-hourly send-log sweep. That removes the lines of rows the sweep
deleted and the lines that never had a row. The send log is asked once per
queue id; the statement still walks the table, about a second per million
lines, with the connection taken meanwhile.
- **Which lines are a row's.** Postfix reuses a queue id once the message has
left the queue, and a row outlives that by months. `SendLogLines` gives a row
the lines of its queue id dated from one hour before the row was written to
the queue lifetime (`maximal_queue_lifetime`, which `bounce_queue_lifetime`
cannot exceed) plus a day after it. That limits a chance reuse of
the id to those days; it does not exclude it. A message with several
recipients is several rows under one id, and each row's page shows all of
the message's lines.
- **Read twice, stored once.** The read offset is saved every few seconds, so
after an unclean stop some lines are read again; while the tailer is
catching up at start it writes a line only if the same line is not there.
- **Not stored:** what the tailer never read — the tail of a file rotated away
while the panel was down.
- **Size:** six to ten lines of about 200 bytes per message — more than the row
itself. The table is in the database and therefore in a full backup, which
`log/` is not ([security.md](security.md) § Backup and domain export).
**Two one-shot reads** sit beside the follow loop and are unrelated to it, both
serving panel pages on request: `TailLines` (the last *n* lines, for
`/server/log`) and `QueueLines` (the lines carrying one queue-id, for
`/outbound/log/{id}`). `QueueLines` scans a bounded tail of the current file —
`/server/log`) and `QueueLines` (the lines carrying one queue-id). `QueueLines`
is what a message's page falls back to when no lines are stored for its row —
a message accepted before 2.2.0. It scans a bounded tail of the current file —
finding a message's lines means reading rather than seeking — and matches the id
anchored on the character before it, since queue ids are hexadecimal runs and a
shorter one is regularly the tail of a longer one. Send-log rows outlive the log
(retention 90 days, rotation 14 files), so an empty result is the expected end
state for an older message and the page reports it as such, not as a failure.
shorter one is regularly the tail of a longer one. For such a message an empty
result is the expected end state once the log has rotated, and the page reports
it as such, not as a failure.
---
@@ -531,16 +938,17 @@ full backups also archive the operator's `./certs` PEM files when present.
### Database
The schema is one embedded baseline, `internal/store/migrations/0001_init.sql`,
applied by `store.Open` into an empty file; `PRAGMA user_version` is `1` and
The schema is an embedded baseline, `internal/store/migrations/0001_init.sql`,
and the migrations after it (`0002_inbound_spam_log.sql`), applied by
`store.Open` in file order; `PRAGMA user_version` counts the files applied and
`PRAGMA application_id` is `0x53503230` ("SP20"). There is no 1.x migration
chain. **2.0 starts from an empty data directory:** a database with a schema
version but without the 2.x `application_id` — what every 1.x file looks like —
is refused at start with `ErrForeignSchema` and left untouched, as is one whose
version is newer than the build knows. A restored 1.x backup fails the same
way. Domains are carried across with a domain export, whose format is not the
database. Later schema changes are new files (`0002_*.sql`, …); once `2.0.0` is
released the baseline is never edited. History and the per-table list:
database. A schema change is a new file; the baseline has not been edited
since `2.0.0`. History and the per-table list:
[schema-migrations.md](schema-migrations.md).
The tables that carry the panel's model:
@@ -565,6 +973,7 @@ The tables that carry the panel's model:
(send-log retention). Nothing that belongs to a user.
- `applications` (with `auth_ip_restrict` / `auth_allowed_ips`),
`application_addresses`, `rate_limits` (no `allowed_ips` column), `send_log`,
`send_log_lines` (what Postfix logged under a queue id — § Log tailer),
`logtail_state`, `inbound_domains`, `inbound_transports`,
`inbound_recipients`, `dmarc_reports`, `dmarc_report_records`.
@@ -572,7 +981,8 @@ The panel is the only writer; SQLite runs in WAL mode on one connection (see
[Panel binary](#panel-binary-cmdpanel)), which is why a full backup snapshots
with `VACUUM INTO` instead of copying the file.
**Rotation:** send-log retention `SEND_LOG_RETENTION_DAYS` (default 90);
**Rotation:** send-log retention `SEND_LOG_RETENTION_DAYS` (default 90), which
the stored log lines follow;
`mail.log` via logrotate (14 rotated files, check every 6h, rename +
`postfix reload` in `postrotate` — see § Log tailer above).
@@ -625,7 +1035,8 @@ unsupported rather than as a missing doc:
- **Panel paths and tuning:** `SELFPOST_DATA_DIR` (`/data`), `SELFPOST_DB_PATH`
(`/data/selfpost.db`), `SELFPOST_SETUP_TOKEN_FILE`
(`/data/setup-token`), `PANEL_HTTP_ADDR` (`:8080`),
`JOURNAL_MILTER_SOCKET` (`/run/selfpost/journal.sock`), `MAIL_LOG`
`JOURNAL_MILTER_SOCKET` (`/run/selfpost/journal.sock`),
`SENDER_CHECK_SOCKET` (`/run/selfpost/sender.sock`), `MAIL_LOG`
(`/data/log/mail.log` — read by the panel and written by Postfix, so a change
here has to be matched in `build/postfix-config.sh`),
`PANEL_COOKIE_SECURE` (`true`), `OPENDKIM_SOCKET`
@@ -651,8 +1062,9 @@ unsupported rather than as a missing doc:
- **Milter and Postfix startup:** `MILTER_CONNECT_TIMEOUT` (`15s`),
`MILTER_COMMAND_TIMEOUT` (`15s`), `MILTER_CONTENT_TIMEOUT` (`30s`),
`MILTER_WAIT_TIMEOUT` (`30` seconds).
- **Background maintenance:** `TLS_RELOAD_INTERVAL_SECONDS` (`86400` — daily
`postfix reload` to pick up renewed certificates),
- **Background maintenance:** `TLS_RELOAD_INTERVAL_SECONDS` (`86400` — how
often a renewed certificate is looked for; Postfix is reloaded only when one
is found),
`LOGROTATE_INTERVAL_SECONDS` (`21600` — check `mail.log` rotation every six
hours; logrotate keeps 14 rotated files on a daily schedule, and each
rotation triggers `postfix reload`).
@@ -82,6 +82,8 @@
<div class="field"><label class="label" for="username">Username</label><div class="control has-icons-left"><input class="input" id="username" name="username" value="admin" autocomplete="username" spellcheck="false" required><span class="icon is-left"><i class="ti ti-user"></i></span></div></div>
<div class="field"><label class="label" for="email">E-mail</label><div class="control has-icons-left"><input class="input" id="email" name="email" type="email" value="mix@example.org" autocomplete="email"><span class="icon is-left"><i class="ti ti-mail"></i></span></div>
<p class="help">Your own address. A domain's report address can be filled with it from Domain settings.</p></div>
<div class="field"><label class="checkbox"><input type="checkbox" name="notify" checked> Send me notifications</label>
<p class="help">To this address, when something needs you: a DNS record of a domain stops matching, a rate limit refuses mail, the TLS certificate is about to expire, a message has waited in the queue for three days. <a href="help.html#notifications">What is sent and when</a></p></div>
<button type="submit" class="button is-primary">Save profile</button>
</form>
</div>
+1 -1
View File
@@ -78,7 +78,7 @@
<div class="box">
<div class="sp-box-head"><span class="sp-no">01</span><h2>Full backup</h2><a class="sp-help" href="help.html#backup" title="Backups, encryption and restore"><i class="ti ti-help-circle"></i></a></div>
<div class="sp-box-body">
<p class="mb-3"><code>data/</code> (database, DKIM keys, credentials, the queue), <code>docker-compose.yml</code>, <code>.env</code> and <code>certs/</code> in one archive. Restore into an empty project directory and start the <strong>same SelfPost version</strong>. The reverse-proxy vhost is not included.</p>
<p class="mb-3"><code>data/</code> (database, DKIM keys, credentials), <code>docker-compose.yml</code>, <code>.env</code> and <code>certs/</code> in one archive. The Postfix queue is left out: it cannot be copied reliably while the server is running. Restore into an empty project directory and start the <strong>same SelfPost version</strong>. The reverse-proxy vhost is not included.</p>
<form class="sp-form" method="post" action="#">
<div class="field"><label class="checkbox"><input type="checkbox" name="encrypt" checked> Encrypt with a password (<code>.spbk</code>)</label></div>
<div class="sp-pair">
+1 -1
View File
@@ -37,7 +37,7 @@ SIGNED_OUT = {"sp-signin", "sp-signin-brand", "sp-signin-form", "sp-foot"}
# Selectors allowed to set a width. Everything else gets its width from the
# layout it sits in — that is the whole point of the redesign.
WIDTH_OK = (".navbar-brand img", ".sp-head .sp-lead", ".sp-head .sp-grow", ".sp-postmark", ".table td.sp-clip",
".sp-h-icon", ".sp-signin", ".sp-timeline li::before", ".sp-perf")
".sp-h-icon", ".sp-signin", ".sp-timeline li::before", ".sp-perf", ".sp-sheet")
class Node:
@@ -84,13 +84,14 @@
<dl class="sp-facts">
<div><dt>list</dt><dd>page_head, then full-width boxes: the add form first, the table second. <a href="out-domains.html">Example</a></dd></div>
<div><dt>detail</dt><dd>page_head, then <code>columns</code>: <code>is-3</code> side menu + <code>is-9</code> boxes. <a href="out-domain.html">Example</a></dd></div>
<div><dt>cards</dt><dd>page_head, then <code>columns sp-cards</code>: two <code>is-6</code> stacks of numbered boxes read in rows, odd on the left and even on the right, each stack closed up. On a phone the boxes come in the order of their numbers. The two stacks under this box are laid out so. <a href="health.html">Example</a></dd></div>
<div><dt>form</dt><dd>page_head, then boxes in equal <code>columns</code>, one submit row under them. <a href="out-app.html">Example</a></dd></div>
<div><dt>signed out</dt><dd>The split brand / form screen, no navbar. <a href="login.html">Example</a></dd></div>
</dl>
</div>
</div>
<div class="columns">
<div class="columns sp-cards">
<div class="column is-6">
<div class="box">
<div class="sp-box-head"><span class="sp-no">01</span><h2>postmark</h2><div class="sp-end">Worst status of the page's checks</div></div>
@@ -114,6 +115,7 @@
<tr><td>error, fail, mismatch, bounced</td><td><span class="tag is-danger is-light">fail</span></td><td class="sp-mono sp-small">tag is-danger is-light</td></tr>
<tr><td>unknown, off, neutral counts</td><td><span class="tag is-light">off</span></td><td class="sp-mono sp-small">tag is-light</td></tr>
<tr><td>held: quarantine · role: global</td><td><span class="tag is-primary is-light">quarantine</span></td><td class="sp-mono sp-small">tag is-primary is-light</td></tr>
<tr><td>a tag that opens a <code>sheet</code></td><td><button type="button" class="tag is-danger is-light" popovertarget="kit-sheet">reject</button><div class="sp-sheet" id="kit-sheet" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-danger is-light">reject</span><h2>sheet</h2><button type="button" popovertarget="kit-sheet" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>What it holds</th><td>What does not fit the row, as name and value</td></tr><tr><th>How it closes</th><td class="sp-mono sp-small">Esc<br>a click outside<br>the cross</td></tr></tbody></table></div></div></div></div></td><td class="sp-mono sp-small">button.tag + sp-sheet (popover)</td></tr>
</tbody>
</table></div>
</div>
+1 -1
View File
@@ -80,7 +80,7 @@
<div><dt>Last report</dt><dd>2026-09-21 06:12 UTC</dd></div>
<div><dt>This week</dt><dd>14 kept · 0 parse failures</dd></div>
<div><dt>Hosted addresses</dt><dd class="sp-mono sp-small">dmarc-reports+&lt;domain&gt;@mail.example.org</dd></div>
<div><dt>Retention</dt><dd>500 reports or 90 days</dd></div>
<div><dt>Retention</dt><dd>500 reports a domain or 90 days</dd></div>
</dl>
</div>
</div>
+66 -48
View File
@@ -74,23 +74,10 @@
<form method="post" action="#"><button type="submit" class="button"><i class="ti ti-refresh"></i>Re-check DNS</button></form>
</div>
<div class="columns">
<div class="column is-6">
<div class="box" id="machine">
<div class="sp-box-head"><span class="sp-no">01</span><h2>Machine</h2><div class="sp-end"><span class="tag is-success is-light">ok</span></div><a class="sp-help" href="help.html#checks" title="What these readings mean"><i class="ti ti-help-circle"></i></a></div>
<div class="table-container"><table class="table is-fullwidth">
<thead><tr><th>Resource</th><th>Usage</th><th>Detail</th></tr></thead>
<tbody>
<tr><td>CPU</td><td><progress class="progress is-small is-success mb-1" value="12" max="100"></progress>12 %</td><td class="sp-muted">2 cores, load 0.21</td></tr>
<tr><td>Memory</td><td><progress class="progress is-small is-success mb-1" value="41" max="100"></progress>41 %</td><td class="sp-muted">812 MB of 1.9 GB</td></tr>
<tr><td>Network</td><td class="sp-nowrap">↓ 18 kB/s · ↑ 44 kB/s</td><td class="sp-muted">eth0: 2.1 GB in, 6.4 GB out</td></tr>
</tbody>
</table></div>
</div>
</div>
<div class="columns sp-cards">
<div class="column is-6">
<div class="box" id="processes">
<div class="sp-box-head"><span class="sp-no">02</span><h2>Processes</h2><div class="sp-end"><span class="tag is-success is-light">ok</span></div></div>
<div class="sp-box-head"><span class="sp-no">01</span><h2>Processes</h2><div class="sp-end"><span class="tag is-success is-light">ok</span></div></div>
<div class="table-container"><table class="table is-fullwidth">
<thead><tr><th>Program</th><th>State</th><th>Detail</th></tr></thead>
<tbody>
@@ -102,37 +89,20 @@
</tbody>
</table></div>
</div>
</div>
</div>
<div class="columns">
<div class="column is-4">
<div class="box" id="certificate">
<div class="sp-box-head"><span class="sp-no">03</span><h2>TLS certificate</h2><div class="sp-end"><span class="tag is-warning is-light">warn</span></div><a class="sp-help" href="help.html#certificate" title="When it warns and what to do"><i class="ti ti-help-circle"></i></a></div>
<div class="sp-box-body">
<dl class="sp-facts">
<div><dt>Expires</dt><dd class="sp-mono">2026-10-03 09:14 UTC</dd></div>
<div><dt>Names</dt><dd class="sp-mono">mail.example.org</dd></div>
</dl>
<p class="mt-4 has-text-danger">Expires in 12 days — renewal has not run.</p>
<p class="sp-muted">Renewal normally runs 30 days before expiry. Check that port 80 is reachable and look for <code>acme</code> lines in the <a href="system-log.html">system log</a>.</p>
</div>
</div>
</div>
<div class="column is-4">
<div class="box" id="sockets">
<div class="sp-box-head"><span class="sp-no">04</span><h2>Milter sockets</h2><div class="sp-end"><span class="tag is-success is-light">ok</span></div></div>
<div class="box" id="sender-check">
<div class="sp-box-head"><span class="sp-no">03</span><h2>Sender check</h2><div class="sp-end"><span class="tag is-success is-light">ok</span></div><a class="sp-help" href="help.html#sender-check" title="What is checked and what these numbers are for"><i class="ti ti-help-circle"></i></a></div>
<div class="table-container"><table class="table is-fullwidth">
<thead><tr><th>Milter</th><th>State</th></tr></thead>
<thead><tr><th>Period</th><th>Panel not answering</th><th>Connections turned away</th><th>Messages deferred</th></tr></thead>
<tbody>
<tr><td>opendkim<br><span class="sp-muted sp-mono sp-small">inet:127.0.0.1:8891</span></td><td><span class="tag is-success is-light">ok</span></td></tr>
<tr><td>journal-milter<br><span class="sp-muted sp-mono sp-small">unix:/run/selfpost/journal.sock</span></td><td><span class="tag is-success is-light">ok</span></td></tr>
<tr><td>rspamd <span class="sp-muted">· inbound spam filter</span><br><span class="sp-muted sp-mono sp-small">inet:antispam:11332</span></td><td><span class="tag is-success is-light">ok</span></td></tr>
<tr><td>Today</td><td>0 s</td><td>0</td><td>0</td></tr>
<tr><td>Last 30 days</td><td>4 s</td><td>0</td><td>0</td></tr>
<tr><td>Last 12 months</td><td>≈ 41 s</td><td>3</td><td>1</td></tr>
</tbody>
</table></div>
<div class="sp-box-body">
<p class="sp-muted">The panel holds each message's <code>From:</code> against the application that sends it. While the panel does not answer, Postfix defers mail instead of accepting it unchecked. Time is counted only while Postfix itself was running; ≈ marks a period with a start at which that could not be asked, and its time is at most what is shown. Counted since <span class="sp-mono">2026-10-13</span>.</p>
</div>
</div>
</div>
<div class="column is-4">
<div class="box" id="hostname">
<div class="sp-box-head"><span class="sp-no">05</span><h2>Hostname and reverse DNS</h2><div class="sp-end"><span class="tag is-success is-light">ok</span></div><a class="sp-help" href="help.html#rdns" title="Forward and reverse DNS"><i class="ti ti-help-circle"></i></a></div>
<div class="sp-box-body">
@@ -142,14 +112,62 @@
</dl>
</div>
</div>
<div class="box" id="machine">
<div class="sp-box-head"><span class="sp-no">07</span><h2>Machine</h2><div class="sp-end"><span class="tag is-success is-light">ok</span></div><a class="sp-help" href="help.html#checks" title="What these readings mean"><i class="ti ti-help-circle"></i></a></div>
<div class="table-container"><table class="table is-fullwidth">
<thead><tr><th>Resource</th><th>Usage</th><th>Detail</th></tr></thead>
<tbody>
<tr><td>CPU</td><td><progress class="progress is-small is-success mb-1" value="12" max="100"></progress>12 %</td><td class="sp-muted">2 cores, load 0.21</td></tr>
<tr><td>Memory</td><td><progress class="progress is-small is-success mb-1" value="41" max="100"></progress>41 %</td><td class="sp-muted">812 MB of 1.9 GB</td></tr>
<tr><td>Network</td><td class="sp-nowrap">↓ 18 kB/s · ↑ 44 kB/s</td><td class="sp-muted">eth0: 2.1 GB in, 6.4 GB out</td></tr>
</tbody>
</table></div>
</div>
</div>
</div>
<div class="box" id="configuration">
<div class="sp-box-head"><span class="sp-no">06</span><h2>Configuration</h2><div class="sp-end">Safe to run at any time</div></div>
<div class="sp-box-body">
<p class="mb-3">Regenerates the OpenDKIM and Postfix configuration from the database and reloads both daemons. Use it after editing the files by hand, after restoring a backup, or when the running configuration looks out of step with the domain and application lists. It does not touch the queue or the TLS certificate.</p>
<form method="post" action="#"><button type="submit" class="button"><i class="ti ti-reload"></i>Reload configuration</button></form>
<div class="column is-6">
<div class="box" id="sockets">
<div class="sp-box-head"><span class="sp-no">02</span><h2>Milter sockets</h2><div class="sp-end"><span class="tag is-success is-light">ok</span></div></div>
<div class="table-container"><table class="table is-fullwidth">
<thead><tr><th>Milter</th><th>State</th></tr></thead>
<tbody>
<tr><td>opendkim<br><span class="sp-muted sp-mono sp-small">inet:127.0.0.1:8891</span></td><td><span class="tag is-success is-light">ok</span></td></tr>
<tr><td>journal-milter<br><span class="sp-muted sp-mono sp-small">unix:/run/selfpost/journal.sock</span></td><td><span class="tag is-success is-light">ok</span></td></tr>
<tr><td>sender-check<br><span class="sp-muted sp-mono sp-small">unix:/run/selfpost/sender.sock</span></td><td><span class="tag is-success is-light">ok</span></td></tr>
<tr><td>rspamd <span class="sp-muted">· inbound spam filter</span><br><span class="sp-muted sp-mono sp-small">inet:antispam:11332</span></td><td><span class="tag is-success is-light">ok</span></td></tr>
</tbody>
</table></div>
</div>
<div class="box" id="certificate">
<div class="sp-box-head"><span class="sp-no">04</span><h2>TLS certificate</h2><div class="sp-end"><span class="tag is-warning is-light">warn</span></div><a class="sp-help" href="help.html#certificate" title="When it warns and what to do"><i class="ti ti-help-circle"></i></a></div>
<div class="sp-box-body">
<dl class="sp-facts">
<div><dt>Expires</dt><dd class="sp-mono">2026-10-03 09:14 UTC</dd></div>
<div><dt>Names</dt><dd class="sp-mono">mail.example.org</dd></div>
</dl>
<p class="mt-4 has-text-danger">Expires in 12 days — renewal has not run.</p>
<p class="sp-muted">Renewal normally runs 30 days before expiry. Check that port 80 is reachable and look for <code>acme</code> lines in the <a href="system-log.html">system log</a>.</p>
</div>
</div>
<div class="box" id="notification-mail">
<div class="sp-box-head"><span class="sp-no">06</span><h2>Notification mail</h2><div class="sp-end"><span class="tag is-warning is-light">warn</span></div><a class="sp-help" href="help.html#notifications" title="What the panel sends and what makes it arrive"><i class="ti ti-help-circle"></i></a></div>
<div class="table-container"><table class="table is-fullwidth">
<thead><tr><th>Record</th><th>State</th></tr></thead>
<tbody>
<tr><td>DKIM<br><span class="sp-muted sp-mono sp-small">sp2026._domainkey.mail.example.org</span></td><td><span class="tag is-success is-light">published</span></td></tr>
<tr><td>SPF<br><span class="sp-muted sp-mono sp-small">mail.example.org</span></td><td><span class="tag is-warning is-light">missing</span></td></tr>
</tbody>
</table></div>
<div class="sp-box-body">
<p class="sp-muted">Sent from <span class="sp-mono">postmaster@mail.example.org</span>. What to publish is under <a href="settings.html#notification-mail">Settings</a>.</p>
</div>
</div>
<div class="box" id="configuration">
<div class="sp-box-head"><span class="sp-no">08</span><h2>Configuration</h2><div class="sp-end">Safe to run at any time</div></div>
<div class="sp-box-body">
<p class="mb-3">Regenerates the OpenDKIM and Postfix configuration from the database and reloads both daemons. Use it after editing the files by hand, after restoring a backup, or when the running configuration looks out of step with the domain and application lists. It does not touch the queue or the TLS certificate.</p>
<form method="post" action="#"><button type="submit" class="button"><i class="ti ti-reload"></i>Reload configuration</button></form>
</div>
</div>
</div>
</div>
+12 -4
View File
@@ -86,6 +86,7 @@
<ul class="menu-list">
<li><a href="#dns"><i class="ti ti-world-check"></i>DNS records</a></li>
<li><a href="#apps"><i class="ti ti-apps"></i>Applications</a></li>
<li><a href="#sender-check"><i class="ti ti-shield-check"></i>Sender check</a></li>
<li><a href="#limits"><i class="ti ti-gauge"></i>Rate limits</a></li>
<li><a href="#log"><i class="ti ti-list-details"></i>Outbound log</a></li>
<li><a href="#queue"><i class="ti ti-stack-2"></i>Queue and retries</a></li>
@@ -97,12 +98,15 @@
<li><a href="#upstream"><i class="ti ti-arrow-forward-up"></i>Upstream</a></li>
<li><a href="#recipients"><i class="ti ti-address-book"></i>Valid recipients</a></li>
<li><a href="#filter"><i class="ti ti-shield-check"></i>Spam filter</a></li>
<li><a href="#inbound-log"><i class="ti ti-list-details"></i>Inbound log</a></li>
<li><a href="#filter-lists"><i class="ti ti-filter"></i>Filter lists</a></li>
</ul>
<p class="menu-label">Server</p>
<ul class="menu-list">
<li><a href="#backup"><i class="ti ti-archive"></i>Backup and moving a domain</a></li>
<li><a href="#users"><i class="ti ti-users"></i>Users and roles</a></li>
<li><a href="#settings"><i class="ti ti-settings"></i>Settings</a></li>
<li><a href="#notifications"><i class="ti ti-mail"></i>Notifications</a></li>
</ul>
<p class="menu-label">Account</p>
<ul class="menu-list">
@@ -121,8 +125,9 @@
<div class="sp-box-head"><span class="sp-no">02</span><h2>Outbound</h2></div>
<div class="sp-help-topic" id="dns"><h3>DNS records</h3><p>DKIM proves the message was signed by this relay, SPF lists the addresses allowed to send for the domain, DMARC tells receivers what to do when both fail and where to report. Publish all three exactly as shown; merge SPF into an existing record rather than adding a second one. A domain's status is the worst of its three records, and results are cached for a few minutes, so use Re-check DNS after publishing. The SPF check is shallow: it looks for this server's address in the record itself and does not follow <code>include:</code> or <code>redirect=</code>, so a record that relies on them is reported as a warning, not a failure. <code>p=none</code> does not affect delivery; tighten to <code>quarantine</code>, then <code>reject</code>, once the reports look clean. A domain's report address is set in its Domain settings.</p></div>
<div class="sp-help-topic" id="apps"><h3>Applications</h3><p>An application is one SASL login, unique across all domains. It may send as any address of its domain or only as listed addresses, and, with a list of client IPs set, only from those addresses. The password is shown once at creation or regeneration and is not stored in readable form.</p></div>
<div class="sp-help-topic" id="sender-check"><h3>Sender check</h3><p>An application sends only as itself, and that holds for both senders a message has. The envelope sender is held against the login by Postfix. The <code>From:</code> line, the one a mail program shows and the one the DKIM signature is chosen by, is held against it by the panel, under the same rule: an application set to any address of its domain may write any address of that domain, and one with a list of addresses only those. A message with any other <code>From:</code> - another of your domains, the server's own name, an address somewhere else, or none at all - is refused for good with <code>550 5.7.1</code>. It is a rejected row of the Outbound log, and its page says which address was refused. This is what keeps one application from getting mail signed with another domain's key.<br><br>The check is strict. While the panel does not answer, Postfix does not accept mail unchecked: it replies with a temporary error and the application sends again later. Server › Health counts what that costs: for how long the panel did not answer while Postfix itself was running; how many connections Postfix turned away because of it - it refuses from the connection on, before a message is offered, so a client that tries three times is three and a monitor that only connects is one; and how many messages were deferred in the middle, when the panel stopped answering after a message had begun. A time marked ≈ includes a start at which supervisord could not say since when Postfix was running: the time since the panel's last sign of life is counted, which may be more than Postfix was up. The card warns when a connection was turned away or a message deferred in the last 24 hours. If the numbers stay at nothing, the check is where it should be; if they do not, it belongs in a process of its own.</p></div>
<div class="sp-help-topic" id="limits"><h3>Rate limits</h3><p>Level 1 is a per-IP ceiling set in <code>.env</code>. Level 2 is optional, per domain and per application, entered by hand or computed from the last 30 days of sending, and can never exceed level 1. An application's own limit replaces its domain's.</p></div>
<div class="sp-help-topic" id="log"><h3>Outbound log</h3><p>One row per message and recipient, with the status Postfix last reported: queued (accepted, nothing reported yet), sent (the receiving server took it; what the mailbox does next is not reported back), deferred (not taken yet, Postfix tries again), bounced (failed for good, or the queue lifetime ran out) and rejected (refused before queueing under a level-2 rate limit, so it never reached Postfix). A message's page lists these steps and the lines Postfix wrote about it in <code>mail.log</code>, found by queue id; that file is rotated daily and about two weeks are kept, so an old message may show none. Rows are deleted after the retention set in Server › Settings, 90 days by default. A domain administrator sees only the rows of their own domains.</p></div>
<div class="sp-help-topic" id="log"><h3>Outbound log</h3><p>One row per message and recipient, with the status Postfix last reported: queued (accepted, nothing reported yet), sent (the receiving server took it; what the mailbox does next is not reported back), deferred (not taken yet, Postfix tries again), bounced (failed for good, or the queue lifetime ran out; or closed by the panel when Postfix no longer held the message and <code>mail.log</code> never said what became of it) and rejected (refused before queueing, so it never reached the queue: under a level-2 rate limit, from a client address the application does not allow, or for a <code>From:</code> line that is not the application's - the message's page says which). A message's page lists these steps and the lines Postfix wrote about it in <code>mail.log</code>, found by queue id - and for a message refused before it had one, Postfix's one line about the refusal. The lines are kept with the row: they stay on the page after that file has rotated, for as long as the row does. A message accepted before version 2.2.0 shows them only while they are still in the current file. Rows are deleted after the retention set in Server › Settings, 90 days by default, and their lines with them. A domain administrator sees only the rows of their own domains.</p></div>
<div class="sp-help-topic" id="queue"><h3>Queue and retries</h3><p>The page lists, as <code>postqueue -p</code> prints it, the messages Postfix has accepted and not yet delivered: mail being sent right now and mail deferred for a later attempt. Only the global role can open it, and it only shows; nothing here deletes or forces a message. A deferred message is tried again after the first retry delay, then at gaps that double up to a cap, until the receiving server takes it or the queue lifetime runs out and it bounces; there is no attempt counter, and the box on the page shows this server's values. A queue that keeps growing means a receiver is refusing or not answering: open the message from the Outbound log to read the reason Postfix logged.</p></div>
<div class="sp-help-topic" id="dmarc"><h3>DMARC reports</h3><p>An aggregate report is the summary a receiving provider sends, usually daily, to the <code>rua=</code> address of your DMARC record: which servers sent mail as your domain and whether SPF or DKIM passed for it. A domain's report address is one of three choices, set in its Domain settings: no reports (the default for a new domain), the SelfPost hosted address (an address on this server whose reports are parsed and shown under DMARC reports; offered only while hosted reports are enabled on the server), or an address typed for the domain (an ordinary mailbox that receives the raw XML; a button fills in your own profile e-mail). When the typed address is on another domain than the one sending, that domain must publish a report-authorization record, or providers will not send; the domain's page shows it with the other DNS records and checks it. In the tables pass means SPF or DKIM passed and aligned, fail means neither did, and your own server shows as <em>this relay</em>. A fail from any other source is someone else sending as your domain — a forwarder, an application you forgot, or abuse — so do not tighten <code>p=</code> until that sender is fixed or removed. A domain administrator reads the reports of their own domains.</p></div>
</div>
@@ -132,16 +137,19 @@
<div class="sp-help-topic" id="upstream"><h3>Upstream</h3><p>The upstream is the server that actually holds the mailboxes. Give its host and port — used exactly as written, with no MX lookup — and choose the TLS for the hop to it: Opportunistic uses TLS when the upstream offers it and otherwise sends in the clear, Required holds the mail until a TLS session is possible, Off never uses TLS. While the upstream does not answer, Postfix keeps the mail and retries as for any other mail.</p></div>
<div class="sp-help-topic" id="recipients"><h3>Valid recipients</h3><p>Choose whom this domain accepts. Listed addresses only: the relay accepts exactly the addresses you list, one per line or comma-separated, and refuses any other while the sender is still connected, so it never has to send a bounce for them. Any recipient: everything at the domain is accepted and passed on, and an address the upstream then rejects comes back as a bounce from this server to a sender who is often forged — backscatter. Prefer a list, and keep it in step with the mailboxes upstream.</p></div>
<div class="sp-help-topic" id="filter"><h3>Spam filter</h3><p>The spam filter belongs to the server, not to a domain. The server's administrator attaches it in the server's <code>.env</code>, and from then on every inbound domain is screened by it before mail is handed on; the box shows on or off and has nothing to switch. Off means mail goes to the upstream unfiltered. While the filter is down, mail is either accepted unfiltered (the default) or deferred, so the sending server tries again later, as the server's setting says; the box states which.</p></div>
<div class="sp-help-topic" id="inbound-log"><h3>Inbound log</h3><p>One row per recipient of a message offered on port 25 for an inbound domain, with what was decided: <span class="tag is-success is-light">accept</span> — handed on to the upstream; <span class="tag is-danger is-light">reject</span> — refused for good; <span class="tag is-warning is-light">tempfail</span> — the sender was asked to come back later; <span class="tag is-primary is-light">quarantine</span> — held by the filter. <em>Why</em> names who decided. <code>rspamd</code> is the spam filter, with its action, the score against the reject threshold and the heaviest symbols; <code>postfix</code> is the relay's own rule — an unknown recipient, a sender domain that does not exist — with its reply; <code>selfpost</code> means nobody judged the message: it was relayed with no filter attached. Accept says what this server did, not what the upstream did with the message afterwards.<br><br>Accepted mail is recorded as it passes; refused mail is read from <code>mail.log</code>, so its row has no subject, and a message refused after its text was sent is one row however many recipients it had. A refusal to relay for a domain that is not an inbound one is every scanner on the internet and is not recorded. Rows are kept for the number of days set in Settings, and never more than 10 000.</p></div>
<div class="sp-help-topic" id="filter-lists"><h3>Filter lists</h3><p>Two lists for the whole server that correct the spam filter by hand. An entry is a sender address, a sender domain — that domain, not its subdomains — or a client address or network in CIDR form. A sender on the Deny list is refused outright. A sender on the Allow list is given a weight no spam score outweighs, so the filter lets it through. The sender's address is whatever the sending server says it is - anyone can claim to be a sender you allow - so allow a client address or network where you can, and a sender address only for mail you would rather receive forged than lose. When entries of both lists match one message, <strong>Deny wins</strong>.<br><br>SelfPost does not apply the lists itself: it writes them as files under <code>/data/antispam/</code>, and rspamd rereads a file within seconds of a change. With the spam filter off the lists are kept and can be edited, but nothing reads them. The sender matched is the envelope sender — the one in the log — not the From line a mail program shows. Matched counts the messages in the Inbound log that the entry decided, so it reaches back as far as the log does; never means not once in that time, and such an entry is one to consider removing.</p></div>
</div>
<div class="box">
<div class="sp-box-head"><span class="sp-no">04</span><h2>Server</h2></div>
<div class="sp-help-topic" id="backup"><h3>Backup and moving a domain</h3><p>Full backup downloads one archive of the whole instance: <code>data/</code> (the database, DKIM keys, application credentials and the queue), <code>docker-compose.yml</code>, <code>.env</code> and <code>certs/</code>. It holds secrets, so leave Encrypt with a password ticked: the file is sealed as <code>.spbk</code>, the password is not stored, and without it nothing can be recovered. To restore, unpack it into an empty project directory (decrypt a <code>.spbk</code> first with <code>selfpost-backup -decrypt</code>) and start the same SelfPost version that made it; the panel refuses data from another version. The reverse-proxy configuration and <code>mail.log</code> are not included. One domain moves on its own: Export domain on its settings page writes a <code>.json</code> (or encrypted <code>.spde</code>) with its DKIM key and its applications' passwords, and Import a domain here reads it into another instance, so the published DNS needs no change.</p></div>
<div class="sp-help-topic" id="backup"><h3>Backup and moving a domain</h3><p>Full backup downloads one archive of the whole instance: <code>data/</code> (the database, DKIM keys and application credentials), <code>docker-compose.yml</code>, <code>.env</code> and <code>certs/</code>. The Postfix queue is not in it. The backup is taken while the server is running, and Postfix keeps moving messages through the queue, so a copy could hold a message twice or miss it; mail still waiting for delivery is therefore not carried over, and a restored server starts with an empty queue. To keep the queue, stop the container and archive the project directory instead. The private key in <code>certs/</code> has to be readable by the panel: give it to group 998 on the host (<code>chgrp 998</code>, <code>chmod 640</code>). Until then the panel refuses to make a backup and names the file, rather than hand over an archive without the key. It holds secrets, so leave Encrypt with a password ticked: the file is sealed as <code>.spbk</code>, the password is not stored, and without it nothing can be recovered. To restore, unpack it into an empty project directory (decrypt a <code>.spbk</code> first with <code>selfpost-backup -decrypt</code>) and start the same SelfPost version that made it; the panel refuses data from another version. The reverse-proxy configuration and <code>mail.log</code> are not included. One domain moves on its own: Export domain on its settings page writes a <code>.json</code> (or encrypted <code>.spde</code>) with its DKIM key and its applications' passwords, and Import a domain here reads it into another instance, so the published DNS needs no change.</p></div>
<div class="sp-help-topic" id="users"><h3>Users and roles</h3><p>The two roles are a reach, not a rank: every user is an administrator and signs in with a username and password. Global reaches the whole server — Overview, Server, the queue and every domain — and is the only role that adds or deletes domains and users. Domain reaches only the domains assigned to it, outbound and inbound separately (the same name on both lists is two assignments); <em>All</em> includes domains added later, and a domain user needs at least one domain or <em>All</em> on either list. The only global user cannot be demoted or deleted, and you cannot delete the account you are signed in with. A new password set here signs that user out of their other sessions.</p></div>
<div class="sp-help-topic" id="settings"><h3>Settings</h3><p>Settings holds what is true of the whole instance. Log retention is how many days the Outbound log keeps its rows, from 7 to 365 and 90 by default; a sweep every six hours deletes older ones, and it is the main driver of how fast <code>/data</code> grows. It does not change the rotation of <code>mail.log</code>, which is daily with about two weeks kept. The sending rate limit shown under it is level 1, the per-client-IP ceiling set in <code>.env</code>: changing it needs a restart, and no level-2 limit can exceed it.</p></div>
<div class="sp-help-topic" id="settings"><h3>Settings</h3><p>Settings holds what is true of the whole instance. Log retention is how many days the Outbound log keeps its rows, from 7 to 365 and 90 by default; a sweep every six hours deletes older ones, and it is the main driver of how fast <code>/data</code> grows. With the inbound relay on there is a second number, for the Inbound log, with the same range. It does not change the rotation of <code>mail.log</code>, which is daily with about two weeks kept. The sending rate limit shown under it is level 1, the per-client-IP ceiling set in <code>.env</code>: changing it needs a restart, and no level-2 limit can exceed it. Notifications are turned on or off here for everyone.</p></div>
<div class="sp-help-topic" id="notifications"><h3>Notifications</h3><p>The panel writes to its users when something needs them, from <code>postmaster@</code> the server's hostname. Four things are reported. A sending domain's SPF, DKIM or DMARC record stops matching: sent once two hourly checks in a row have found it so, and once more when the record matches again. A level-2 rate limit refuses mail: sent at the first refusal, then at most once a day for the same limit. The TLS certificate is 14 days from expiry, and again at 3 days if it is still the same certificate. A message has waited in the queue for more than three days, with one more message when none is left. A global user is told all of it; a domain administrator is told about the DNS records and limits of their own domains.<br><br>Each user turns notifications on or off under Account and needs an e-mail there; Server › Settings turns them off for everyone. They are the panel's own mail: not rows of the Outbound log, and not counted against any limit.<br><br>For that mail to arrive, the server's own name needs two DNS records of its own, as each of your domains does: a DKIM key, which the panel generates and signs notifications with, and an SPF record, <code>v=spf1 a -all</code>. Server › Settings gives both with the values to publish, and Server › Health shows whether they are in place. Without them notifications are still sent, and a receiver may file them as spam. Mail of your domains does not depend on either.</p></div>
</div>
<div class="box">
<div class="sp-box-head"><span class="sp-no">05</span><h2>Account</h2></div>
<div class="sp-help-topic" id="account"><h3>Your account</h3><p>Account is your own, whatever your role: your username and e-mail, and your password. Changing the password signs out every other session of this account and keeps the one you are using; applications have their own logins and are not affected.</p></div>
<div class="sp-help-topic" id="account"><h3>Your account</h3><p>Account is your own, whatever your role: your username and e-mail, whether the panel sends you notifications, and your password. Changing the password signs out every other session of this account and keeps the one you are using; applications have their own logins and are not affected.</p></div>
</div>
</div>
</div>
@@ -74,7 +74,7 @@
</div>
<div class="box">
<div class="sp-box-head"><span class="sp-no">01</span><h2>Add an entry</h2><a class="sp-help" href="help.html#inbound" title="How the lists are applied"><i class="ti ti-help-circle"></i></a></div>
<div class="sp-box-head"><span class="sp-no">01</span><h2>Add an entry</h2><a class="sp-help" href="help.html#filter-lists" title="How the lists are applied"><i class="ti ti-help-circle"></i></a></div>
<div class="sp-box-body">
<form class="sp-form" method="post" action="#">
<div class="columns">
@@ -84,7 +84,7 @@
</div>
<div class="column is-3">
<div class="field"><label class="label" for="etype">Entry is</label>
<div class="control"><div class="select is-fullwidth"><select id="etype" name="entry_type"><option>a sender address</option><option>a sender domain</option><option>a client IP or CIDR</option></select></div></div></div>
<div class="control"><div class="select is-fullwidth"><select id="etype" name="entry_type"><option>a sender address</option><option>a sender domain</option><option>a client address or network</option></select></div></div></div>
</div>
<div class="column is-6">
<div class="field"><label class="label" for="value">Value</label>
@@ -103,11 +103,11 @@
<div class="box">
<div class="sp-box-head"><span class="sp-no">02</span><h2>Deny</h2><div class="sp-end">3 entries · wins over Allow</div></div>
<div class="table-container"><table class="table is-fullwidth is-hoverable">
<thead><tr><th>Value</th><th>Note</th><th>Added</th><th></th></tr></thead>
<thead><tr><th>Value</th><th>Kind</th><th>Matched</th><th></th></tr></thead>
<tbody>
<tr><td class="sp-mono sp-small">@bulk.example <span class="tag is-light">domain</span></td><td class="sp-muted">newsletter nobody subscribed to</td><td class="sp-muted sp-nowrap sp-small">2026-09-02</td><td class="sp-actions"><a class="has-text-danger" href="#">Remove</a></td></tr>
<tr><td class="sp-mono sp-small">198.51.100.0/24 <span class="tag is-light">ip</span></td><td class="sp-muted">dictionary attack on lists.example.org</td><td class="sp-muted sp-nowrap sp-small">2026-08-19</td><td class="sp-actions"><a class="has-text-danger" href="#">Remove</a></td></tr>
<tr><td class="sp-mono sp-small">win@lottery.example <span class="tag is-light">address</span></td><td class="sp-muted">—</td><td class="sp-muted sp-nowrap sp-small">2026-07-30</td><td class="sp-actions"><a class="has-text-danger" href="#">Remove</a></td></tr>
<tr><td class="sp-mono sp-small">bulk.example</td><td><button type="button" class="tag is-light" popovertarget="deny-1">domain</button><div class="sp-sheet" id="deny-1" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-light">domain</span><h2>bulk.example</h2><button type="button" popovertarget="deny-1" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Note</th><td>newsletter nobody subscribed to</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-09-02</td></tr><tr><th>Matched</th><td>41 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-09-21 13:57</td></tr></tbody></table></div></div></div></div></td><td class="sp-nowrap">41</td><td class="sp-actions"><form method="post" action="#"><button type="submit" class="has-text-danger">Remove</button></form></td></tr>
<tr><td class="sp-mono sp-small">198.51.100.0/24</td><td><button type="button" class="tag is-light" popovertarget="deny-2">ip</button><div class="sp-sheet" id="deny-2" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-light">ip</span><h2>198.51.100.0/24</h2><button type="button" popovertarget="deny-2" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Note</th><td>dictionary attack on lists.example.org</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-08-19</td></tr><tr><th>Matched</th><td>1203 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-09-21 09:12</td></tr></tbody></table></div></div></div></div></td><td class="sp-nowrap">1203</td><td class="sp-actions"><form method="post" action="#"><button type="submit" class="has-text-danger">Remove</button></form></td></tr>
<tr><td class="sp-mono sp-small">win@lottery.example</td><td><button type="button" class="tag is-light" popovertarget="deny-3">address</button><div class="sp-sheet" id="deny-3" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-light">address</span><h2>win@lottery.example</h2><button type="button" popovertarget="deny-3" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Note</th><td>—</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-07-30</td></tr><tr><th>Matched</th><td>never in the last 90 days</td></tr></tbody></table></div></div></div></div></td><td class="sp-muted sp-nowrap">never</td><td class="sp-actions"><form method="post" action="#"><button type="submit" class="has-text-danger">Remove</button></form></td></tr>
</tbody>
</table></div>
</div>
@@ -116,10 +116,10 @@
<div class="box">
<div class="sp-box-head"><span class="sp-no">03</span><h2>Allow</h2><div class="sp-end">2 entries</div></div>
<div class="table-container"><table class="table is-fullwidth is-hoverable">
<thead><tr><th>Value</th><th>Note</th><th>Added</th><th></th></tr></thead>
<thead><tr><th>Value</th><th>Kind</th><th>Matched</th><th></th></tr></thead>
<tbody>
<tr><td class="sp-mono sp-small">@partner.example <span class="tag is-light">domain</span></td><td class="sp-muted">invoices were scored as spam</td><td class="sp-muted sp-nowrap sp-small">2026-09-11</td><td class="sp-actions"><a class="has-text-danger" href="#">Remove</a></td></tr>
<tr><td class="sp-mono sp-small">noreply@github.example <span class="tag is-light">address</span></td><td class="sp-muted">security advisories</td><td class="sp-muted sp-nowrap sp-small">2026-06-04</td><td class="sp-actions"><a class="has-text-danger" href="#">Remove</a></td></tr>
<tr><td class="sp-mono sp-small">partner.example</td><td><button type="button" class="tag is-light" popovertarget="allow-1">domain</button><div class="sp-sheet" id="allow-1" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-light">domain</span><h2>partner.example</h2><button type="button" popovertarget="allow-1" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Note</th><td>invoices were scored as spam</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-09-11</td></tr><tr><th>Matched</th><td>7 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-09-19 16:40</td></tr></tbody></table></div></div></div></div></td><td class="sp-nowrap">7</td><td class="sp-actions"><form method="post" action="#"><button type="submit" class="has-text-danger">Remove</button></form></td></tr>
<tr><td class="sp-mono sp-small">noreply@github.example</td><td><button type="button" class="tag is-light" popovertarget="allow-2">address</button><div class="sp-sheet" id="allow-2" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-light">address</span><h2>noreply@github.example</h2><button type="button" popovertarget="allow-2" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Note</th><td>security advisories</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-06-04</td></tr><tr><th>Matched</th><td>18 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-09-21 13:30</td></tr></tbody></table></div></div></div></div></td><td class="sp-nowrap">18</td><td class="sp-actions"><form method="post" action="#"><button type="submit" class="has-text-danger">Remove</button></form></td></tr>
</tbody>
</table></div>
</div>
@@ -130,10 +130,10 @@
<div class="sp-box-head"><span class="sp-no">04</span><h2>Where the lists go</h2><div class="sp-end"><span class="tag is-success is-light">applied</span></div></div>
<div class="sp-box-body">
<dl class="sp-facts">
<div><dt>Applied by</dt><dd>rspamd, from map files</dd></div>
<div><dt>Files</dt><dd class="sp-mono sp-small">/data/antispam/allow.map<br>/data/antispam/deny.map</dd></div>
<div><dt>Applied by</dt><dd>rspamd, from map files it rereads within seconds of a change</dd></div>
<div><dt>Files</dt><dd><span class="sp-mono">/data/antispam/</span> — six, one for each list and kind of entry</dd></div>
<div><dt>Last written</dt><dd>2026-09-11 08:40 UTC</dd></div>
<div><dt>When the filter is off</dt><dd class="sp-muted">The lists are kept and editable, but nothing reads them until <code>INBOUND_ANTISPAM_MILTER</code> is set.</dd></div>
<div><dt>When both lists match</dt><dd>Deny wins, whichever entries they are.</dd></div>
</dl>
</div>
</div>
+10 -9
View File
@@ -69,7 +69,7 @@
<div class="sp-grow">
<p class="sp-kicker">Inbound</p>
<h1 class="title is-3">Inbound log</h1>
<p class="sp-lead">What port 25 decided about each message, one row per recipient. Accepts come from the inbound journal, rejects from <code>mail.log</code>; the filter's reasons appear when the spam filter is on.</p>
<p class="sp-lead">What port 25 decided about each message, one row per recipient. Accepted mail is recorded as it passes, refused mail is read from <code>mail.log</code>; the filter's verdict appears when the spam filter is on. Rows older than 90 days are deleted — change that in <a href="settings.html">Settings</a>.</p>
</div>
</div>
@@ -83,19 +83,20 @@
<input class="input" type="date" name="to" value="2026-09-21" aria-label="To date">
<button type="submit" class="button"><i class="ti ti-filter"></i>Filter</button>
</form>
<a class="sp-help" href="help.html#inbound-log" title="What the decisions mean"><i class="ti ti-help-circle"></i></a>
</div>
<div class="table-container"><table class="table is-fullwidth is-hoverable">
<thead><tr><th>Time, UTC</th><th>From · client</th><th>To</th><th>Subject</th><th>Decision</th><th>Why</th><th></th></tr></thead>
<thead><tr><th>Time, UTC</th><th>From</th><th>To</th><th>Subject</th><th>Decision</th><th></th></tr></thead>
<tbody>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 14:00:18</td><td>t.berg@<wbr>uni.example<br><span class="sp-mono sp-muted sp-small">192.0.2.77</span></td><td>dev@<wbr>lists.example.org</td><td class="sp-clip">Re: [dev] patch series v3</td><td><span class="tag is-success is-light">accept</span></td><td class="sp-muted sp-small">rspamd · score 0.4</td><td class="sp-actions"><a href="in-filter-lists.html">Deny sender</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:59:02</td><td>invoice@<wbr>partner-billing.example<br><span class="sp-mono sp-muted sp-small">203.0.113.140</span></td><td>owner@<wbr>lists.example.org</td><td class="sp-clip">Outstanding invoice — action required</td><td><span class="tag is-primary is-light">quarantine</span></td><td class="sp-muted sp-small">rspamd · score 9.8 · PHISHING, FORGED_SENDER</td><td class="sp-actions"><a href="in-quarantine-message.html">Review</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:57:41</td><td>promo@<wbr>bulk.example<br><span class="sp-mono sp-muted sp-small">198.51.100.90</span></td><td>sales@<wbr>lists.example.org</td><td class="sp-clip sp-muted">—</td><td><span class="tag is-danger is-light">reject</span></td><td class="sp-muted sp-small">postfix · 550 5.1.1 unknown recipient</td><td class="sp-actions"><a href="in-filter-lists.html">Deny sender</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:51:30</td><td>billing@<wbr>partner.example<br><span class="sp-mono sp-muted sp-small">198.51.100.12</span></td><td>owner@<wbr>lists.example.org</td><td class="sp-clip">Invoice 2026-0917</td><td><span class="tag is-danger is-light">reject</span></td><td class="sp-muted sp-small">rspamd · score 16.2 · BAYES_SPAM</td><td class="sp-actions"><a href="in-filter-lists.html">Allow sender</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:44:02</td><td>hr@<wbr>partner.example<br><span class="sp-mono sp-muted sp-small">192.0.2.31</span></td><td>jobs@<wbr>acme.io</td><td class="sp-clip">Application: backend engineer</td><td><span class="tag is-warning is-light">tempfail</span></td><td class="sp-muted sp-small">selfpost · upstream 10.0.4.12 timed out</td><td class="sp-actions"></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:30:55</td><td>noreply@<wbr>github.example<br><span class="sp-mono sp-muted sp-small">192.0.2.200</span></td><td>security@<wbr>lists.example.org</td><td class="sp-clip">[advisory] new report</td><td><span class="tag is-success is-light">accept</span></td><td class="sp-muted sp-small">rspamd · allow list</td><td class="sp-actions"></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 14:00:18</td><td>t.berg@<wbr>uni.example<br><span class="sp-mono sp-muted sp-small">192.0.2.77</span></td><td>dev@<wbr>lists.example.org</td><td class="sp-clip">Re: [dev] patch series v3</td><td><button type="button" class="tag is-success is-light" popovertarget="why-1">accept</button><div class="sp-sheet" id="why-1" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-success is-light">accept</span><h2>Why</h2><button type="button" popovertarget="why-1" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Verdict</th><td>no action, score 0.40 / 15.00</td></tr><tr><th>Subject</th><td>Re: [dev] patch series v3</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"><a href="in-filter-lists.html">Deny sender</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:59:02</td><td>invoice@<wbr>partner-billing.example<br><span class="sp-mono sp-muted sp-small">203.0.113.140</span></td><td>owner@<wbr>lists.example.org</td><td class="sp-clip">Outstanding invoice — action required</td><td><button type="button" class="tag is-success is-light" popovertarget="why-2">accept</button><div class="sp-sheet" id="why-2" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-success is-light">accept</span><h2>Why</h2><button type="button" popovertarget="why-2" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Verdict</th><td>add header, score 9.80 / 15.00</td></tr><tr><th>Symbols</th><td class="sp-mono sp-small">PHISHING(5.00)<br>FORGED_SENDER(3.00)<br>R_SPF_FAIL(1.00)</td></tr><tr><th>Subject</th><td>Outstanding invoice — action required</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"><a href="in-filter-lists.html">Deny sender</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:57:41</td><td>promo@<wbr>bulk.example<br><span class="sp-mono sp-muted sp-small">198.51.100.90</span></td><td>sales@<wbr>lists.example.org</td><td class="sp-clip sp-muted">—</td><td><button type="button" class="tag is-danger is-light" popovertarget="why-3">reject</button><div class="sp-sheet" id="why-3" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-danger is-light">reject</span><h2>Why</h2><button type="button" popovertarget="why-3" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>postfix</td></tr><tr><th>Reply</th><td class="sp-mono sp-small">550 5.1.1 &lt;sales@lists.example.org&gt;: Recipient address rejected: User unknown in relay recipient table</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:51:30</td><td>billing@<wbr>partner.example<br><span class="sp-mono sp-muted sp-small">198.51.100.12</span></td><td>owner@<wbr>lists.example.org</td><td class="sp-clip sp-muted">—</td><td><button type="button" class="tag is-danger is-light" popovertarget="why-4">reject</button><div class="sp-sheet" id="why-4" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-danger is-light">reject</span><h2>Why</h2><button type="button" popovertarget="why-4" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Reply</th><td class="sp-mono sp-small">5.7.1 Spam message rejected</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"><a href="in-filter-lists.html">Allow sender</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:44:02</td><td>hr@<wbr>partner.example<br><span class="sp-mono sp-muted sp-small">192.0.2.31</span></td><td>jobs@<wbr>acme.io</td><td class="sp-clip sp-muted">—</td><td><button type="button" class="tag is-warning is-light" popovertarget="why-5">tempfail</button><div class="sp-sheet" id="why-5" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-warning is-light">tempfail</span><h2>Why</h2><button type="button" popovertarget="why-5" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Reply</th><td class="sp-mono sp-small">4.7.1 Try again later</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"><a href="in-filter-lists.html">Allow sender</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:30:55</td><td>noreply@<wbr>github.example<br><span class="sp-mono sp-muted sp-small">192.0.2.200</span></td><td>security@<wbr>lists.example.org</td><td class="sp-clip">[advisory] new report</td><td><button type="button" class="tag is-success is-light" popovertarget="why-6">accept</button><div class="sp-sheet" id="why-6" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-success is-light">accept</span><h2>Why</h2><button type="button" popovertarget="why-6" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Verdict</th><td>no action, score -98.10 / 15.00</td></tr><tr><th>Symbols</th><td class="sp-mono sp-small">SELFPOST_ALLOW_FROM(-100.00)<br>R_DKIM_ALLOW(-0.20)</td></tr><tr><th>Subject</th><td>[advisory] new report</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"><a href="in-filter-lists.html">Deny sender</a></td></tr>
</tbody>
</table></div>
<div class="sp-box-foot"><span>Page 1 of 38</span><a href="#">Older →</a><span class="sp-end">Allow / Deny sender opens <a href="in-filter-lists.html">Filter lists</a> with the entry filled in · global administrator only</span></div>
<div class="sp-box-foot"><span>Page 1 of 38</span><a href="#">Older →</a><span class="sp-end">A decision opens its reason · Allow / Deny sender opens <a href="in-filter-lists.html">Filter lists</a> with the entry filled in</span></div>
</div>
<footer class="sp-foot">SelfPost 1.8.0 · © 2026 Mix · <a href="#">License (AGPL-3.0)</a> · <a href="#">Source</a> · No warranty</footer>
@@ -91,6 +91,7 @@
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 14:01:50</td><td>shop@<wbr>shop.example.org</td><td>anna.lind@<wbr>outlook.com</td><td class="sp-clip">Your order #48212 has shipped</td><td><span class="tag is-success is-light">sent</span></td><td class="sp-actions"><a href="out-message.html">Details</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:58:07</td><td>alerts@<wbr>example.org</td><td>noc@<wbr>example.org</td><td class="sp-clip">[FIRING] disk usage above 90 % on db-2</td><td><span class="tag is-warning is-light">deferred</span></td><td class="sp-actions"><a href="out-message.html">Details</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:41:19</td><td>no-reply@<wbr>notify.acme.io</td><td>j.doe@<wbr>nonexistent.example</td><td class="sp-clip">Reset your password</td><td><span class="tag is-danger is-light">bounced</span></td><td class="sp-actions"><a href="out-message.html">Details</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:40:55</td><td>shop@<wbr>shop.example.org</td><td>k.berg@<wbr>web.de</td><td class="sp-clip">A message from the director</td><td><span class="tag is-danger is-light">wrong From</span></td><td class="sp-actions"><a href="out-message.html">Details</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:40:02</td><td>prod@<wbr>example.org</td><td>billing@<wbr>partner.example</td><td class="sp-clip">Invoice 2026-0917</td><td><span class="tag is-light">rate-limited</span></td><td class="sp-actions"><a href="out-message.html">Details</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:12:44</td><td>shop@<wbr>shop.example.org</td><td>p.novak@<wbr>seznam.cz</td><td class="sp-clip">We received your return request</td><td><span class="tag is-success is-light">sent</span></td><td class="sp-actions"><a href="out-message.html">Details</a></td></tr>
</tbody>
+15 -27
View File
@@ -79,36 +79,22 @@
"columns": [
{
"is-6": [
"box[Processes]:table",
"box[Sender check]:table+body",
"box[Hostname and reverse DNS]:body",
"box[Machine]:table"
]
},
{
"is-6": [
"box[Processes]:table"
"box[Milter sockets]:table",
"box[TLS certificate]:body",
"box[Notification mail]:table+body",
"box[Configuration]:body"
]
}
]
},
{
"columns": [
{
"is-4": [
"box[TLS certificate]:body"
]
},
{
"is-4": [
"box[Milter sockets]:table"
]
},
{
"is-4": [
"box[Hostname and reverse DNS]:body"
]
}
]
},
"box[Configuration]:body"
}
],
"help": [
"head",
@@ -122,9 +108,9 @@
{
"is-9": [
"box[Overview]:help×3",
"box[Outbound]:help×6",
"box[Inbound]:help×4",
"box[Server]:help×3",
"box[Outbound]:help×7",
"box[Inbound]:help×6",
"box[Server]:help×4",
"box[Account]:help"
]
}
@@ -406,12 +392,14 @@
"columns": [
{
"is-6": [
"box[Log retention]:body"
"box[Log retention]:body",
"box[Sending rate limits]:body"
]
},
{
"is-6": [
"box[Sending rate limits]:body"
"box[Notifications]:body",
"box[Notification mail]:record×2"
]
}
]
+40 -2
View File
@@ -82,11 +82,12 @@
<div class="field"><label class="label" for="ret">Keep outbound log rows, days</label>
<div class="control"><input class="input" id="ret" name="send_log_retention_days" type="number" min="7" max="365" value="30" required></div>
<p class="help">The main driver of <code>/data</code> growth. Does not affect <code>mail.log</code> rotation (fourteen daily files).</p></div>
<div class="field"><label class="label" for="inret">Keep inbound log rows, days</label>
<div class="control"><input class="input" id="inret" name="inbound_spam_log_retention_days" type="number" min="7" max="365" value="90" required></div>
<p class="help">The inbound log also never holds more than 10 000 rows: refused mail is logged at the rate strangers knock on port 25.</p></div>
</div>
</div>
</div>
<div class="column is-6">
<div class="box" id="rate-limits">
<div class="sp-box-head"><span class="sp-no">02</span><h2>Sending rate limits</h2><div class="sp-end">Read-only</div><a class="sp-help" href="help.html#limits" title="Levels 1 and 2"><i class="ti ti-help-circle"></i></a></div>
<div class="sp-box-body">
@@ -97,6 +98,43 @@
</div>
</div>
</div>
<div class="column is-6">
<div class="box" id="notifications">
<div class="sp-box-head"><span class="sp-no">03</span><h2>Notifications</h2><a class="sp-help" href="help.html#notifications" title="What is sent and when"><i class="ti ti-help-circle"></i></a></div>
<div class="sp-box-body">
<div class="field"><label class="checkbox"><input type="checkbox" name="notify" checked> Send notification e-mails</label>
<p class="help">From <span class="sp-mono">postmaster@mail.example.org</span>, to every user who has an e-mail and has not turned them off under <a href="account.html">Account</a>.</p></div>
</div>
</div>
<div class="box" id="notification-mail">
<div class="sp-box-head"><span class="sp-no">04</span><h2>Notification mail</h2><div class="sp-end">DNS records of the server's own name</div><a class="sp-help" href="help.html#notifications" title="What the panel sends and what makes it arrive"><i class="ti ti-help-circle"></i></a></div>
<div class="sp-record">
<div class="sp-record-head"><h3>DKIM</h3><span class="tag is-success is-light">published</span><small>Not a secret · selector <code>sp2026</code></small></div>
<div class="sp-row">
<div class="field"><label class="label">Host / name</label>
<div class="field has-addons"><div class="control is-expanded"><input class="input" readonly value="sp2026._domainkey.mail.example.org"></div><div class="control"><button type="button" class="button"><i class="ti ti-copy"></i>Copy</button></div></div></div>
<div class="field"><label class="label">Type</label><div class="control"><input class="input" readonly value="TXT"></div></div>
</div>
<div class="field"><label class="label">Value to publish</label>
<div class="field has-addons"><div class="control is-expanded"><textarea class="textarea" rows="5" readonly>v=DKIM1; k=rsa; p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAq7n2Lw0c5vXeR1dYt8uJb3mKs6pHf9aZoWg4iNy2TqCx0lVjE7rUh5sBk1dMa8fPz3oGc6yXw9tIe4nQv2uL…IDAQAB</textarea></div><div class="control"><button type="button" class="button"><i class="ti ti-copy"></i>Copy</button></div></div></div>
</div>
<div class="sp-record">
<div class="sp-record-head"><h3>SPF</h3><span class="tag is-warning is-light">missing</span><small>For the server's own name, not for your domains</small></div>
<div class="sp-row">
<div class="field"><label class="label">Host / name</label>
<div class="field has-addons"><div class="control is-expanded"><input class="input" readonly value="mail.example.org"></div><div class="control"><button type="button" class="button"><i class="ti ti-copy"></i>Copy</button></div></div></div>
<div class="field"><label class="label">Type</label><div class="control"><input class="input" readonly value="TXT"></div></div>
</div>
<div class="field"><label class="label">Value to publish</label>
<div class="field has-addons"><div class="control is-expanded"><input class="input" readonly value="v=spf1 a -all"></div><div class="control"><button type="button" class="button"><i class="ti ti-copy"></i>Copy</button></div></div></div>
<p class="help">Notifications are sent without it, and a receiver may file them as spam.</p>
</div>
</div>
</div>
</div>
<div class="buttons"><button type="submit" class="button is-primary">Save settings</button></div>
</form>
@@ -16,6 +16,8 @@
<div class="field"><label class="label" for="username">Username</label><div class="control has-icons-left"><input class="input" id="username" name="username" value="admin" autocomplete="username" spellcheck="false" required><span class="icon is-left"><i class="ti ti-user"></i></span></div></div>
<div class="field"><label class="label" for="email">E-mail</label><div class="control has-icons-left"><input class="input" id="email" name="email" type="email" value="mix@example.org" autocomplete="email"><span class="icon is-left"><i class="ti ti-mail"></i></span></div>
<p class="help">Your own address. A domain's report address can be filled with it from Domain settings.</p></div>
<div class="field"><label class="checkbox"><input type="checkbox" name="notify" checked> Send me notifications</label>
<p class="help">To this address, when something needs you: a DNS record of a domain stops matching, a rate limit refuses mail, the TLS certificate is about to expire, a message has waited in the queue for three days. <a href="help.html#notifications">What is sent and when</a></p></div>
<button type="submit" class="button is-primary">Save profile</button>
</form>
</div>
@@ -12,7 +12,7 @@
<div class="box">
<div class="sp-box-head"><span class="sp-no">01</span><h2>Full backup</h2><a class="sp-help" href="help.html#backup" title="Backups, encryption and restore"><i class="ti ti-help-circle"></i></a></div>
<div class="sp-box-body">
<p class="mb-3"><code>data/</code> (database, DKIM keys, credentials, the queue), <code>docker-compose.yml</code>, <code>.env</code> and <code>certs/</code> in one archive. Restore into an empty project directory and start the <strong>same SelfPost version</strong>. The reverse-proxy vhost is not included.</p>
<p class="mb-3"><code>data/</code> (database, DKIM keys, credentials), <code>docker-compose.yml</code>, <code>.env</code> and <code>certs/</code> in one archive. The Postfix queue is left out: it cannot be copied reliably while the server is running. Restore into an empty project directory and start the <strong>same SelfPost version</strong>. The reverse-proxy vhost is not included.</p>
<form class="sp-form" method="post" action="#">
<div class="field"><label class="checkbox"><input type="checkbox" name="encrypt" checked> Encrypt with a password (<code>.spbk</code>)</label></div>
<div class="sp-pair">
@@ -18,13 +18,14 @@
<dl class="sp-facts">
<div><dt>list</dt><dd>page_head, then full-width boxes: the add form first, the table second. <a href="out-domains.html">Example</a></dd></div>
<div><dt>detail</dt><dd>page_head, then <code>columns</code>: <code>is-3</code> side menu + <code>is-9</code> boxes. <a href="out-domain.html">Example</a></dd></div>
<div><dt>cards</dt><dd>page_head, then <code>columns sp-cards</code>: two <code>is-6</code> stacks of numbered boxes read in rows, odd on the left and even on the right, each stack closed up. On a phone the boxes come in the order of their numbers. The two stacks under this box are laid out so. <a href="health.html">Example</a></dd></div>
<div><dt>form</dt><dd>page_head, then boxes in equal <code>columns</code>, one submit row under them. <a href="out-app.html">Example</a></dd></div>
<div><dt>signed out</dt><dd>The split brand / form screen, no navbar. <a href="login.html">Example</a></dd></div>
</dl>
</div>
</div>
<div class="columns">
<div class="columns sp-cards">
<div class="column is-6">
<div class="box">
<div class="sp-box-head"><span class="sp-no">01</span><h2>postmark</h2><div class="sp-end">Worst status of the page's checks</div></div>
@@ -48,6 +49,7 @@
<tr><td>error, fail, mismatch, bounced</td><td><span class="tag is-danger is-light">fail</span></td><td class="sp-mono sp-small">tag is-danger is-light</td></tr>
<tr><td>unknown, off, neutral counts</td><td><span class="tag is-light">off</span></td><td class="sp-mono sp-small">tag is-light</td></tr>
<tr><td>held: quarantine · role: global</td><td><span class="tag is-primary is-light">quarantine</span></td><td class="sp-mono sp-small">tag is-primary is-light</td></tr>
<tr><td>a tag that opens a <code>sheet</code></td><td><button type="button" class="tag is-danger is-light" popovertarget="kit-sheet">reject</button><div class="sp-sheet" id="kit-sheet" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-danger is-light">reject</span><h2>sheet</h2><button type="button" popovertarget="kit-sheet" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>What it holds</th><td>What does not fit the row, as name and value</td></tr><tr><th>How it closes</th><td class="sp-mono sp-small">Esc<br>a click outside<br>the cross</td></tr></tbody></table></div></div></div></div></td><td class="sp-mono sp-small">button.tag + sp-sheet (popover)</td></tr>
</tbody>
</table></div>
</div>
@@ -14,7 +14,7 @@
<div><dt>Last report</dt><dd>2026-09-21 06:12 UTC</dd></div>
<div><dt>This week</dt><dd>14 kept · 0 parse failures</dd></div>
<div><dt>Hosted addresses</dt><dd class="sp-mono sp-small">dmarc-reports+&lt;domain&gt;@mail.example.org</dd></div>
<div><dt>Retention</dt><dd>500 reports or 90 days</dd></div>
<div><dt>Retention</dt><dd>500 reports a domain or 90 days</dd></div>
</dl>
</div>
</div>
@@ -8,23 +8,10 @@
<form method="post" action="#"><button type="submit" class="button"><i class="ti ti-refresh"></i>Re-check DNS</button></form>
</div>
<div class="columns">
<div class="column is-6">
<div class="box" id="machine">
<div class="sp-box-head"><span class="sp-no">01</span><h2>Machine</h2><div class="sp-end"><span class="tag is-success is-light">ok</span></div><a class="sp-help" href="help.html#checks" title="What these readings mean"><i class="ti ti-help-circle"></i></a></div>
<div class="table-container"><table class="table is-fullwidth">
<thead><tr><th>Resource</th><th>Usage</th><th>Detail</th></tr></thead>
<tbody>
<tr><td>CPU</td><td><progress class="progress is-small is-success mb-1" value="12" max="100"></progress>12 %</td><td class="sp-muted">2 cores, load 0.21</td></tr>
<tr><td>Memory</td><td><progress class="progress is-small is-success mb-1" value="41" max="100"></progress>41 %</td><td class="sp-muted">812 MB of 1.9 GB</td></tr>
<tr><td>Network</td><td class="sp-nowrap">↓ 18 kB/s · ↑ 44 kB/s</td><td class="sp-muted">eth0: 2.1 GB in, 6.4 GB out</td></tr>
</tbody>
</table></div>
</div>
</div>
<div class="columns sp-cards">
<div class="column is-6">
<div class="box" id="processes">
<div class="sp-box-head"><span class="sp-no">02</span><h2>Processes</h2><div class="sp-end"><span class="tag is-success is-light">ok</span></div></div>
<div class="sp-box-head"><span class="sp-no">01</span><h2>Processes</h2><div class="sp-end"><span class="tag is-success is-light">ok</span></div></div>
<div class="table-container"><table class="table is-fullwidth">
<thead><tr><th>Program</th><th>State</th><th>Detail</th></tr></thead>
<tbody>
@@ -36,37 +23,20 @@
</tbody>
</table></div>
</div>
</div>
</div>
<div class="columns">
<div class="column is-4">
<div class="box" id="certificate">
<div class="sp-box-head"><span class="sp-no">03</span><h2>TLS certificate</h2><div class="sp-end"><span class="tag is-warning is-light">warn</span></div><a class="sp-help" href="help.html#certificate" title="When it warns and what to do"><i class="ti ti-help-circle"></i></a></div>
<div class="sp-box-body">
<dl class="sp-facts">
<div><dt>Expires</dt><dd class="sp-mono">2026-10-03 09:14 UTC</dd></div>
<div><dt>Names</dt><dd class="sp-mono">mail.example.org</dd></div>
</dl>
<p class="mt-4 has-text-danger">Expires in 12 days — renewal has not run.</p>
<p class="sp-muted">Renewal normally runs 30 days before expiry. Check that port 80 is reachable and look for <code>acme</code> lines in the <a href="system-log.html">system log</a>.</p>
</div>
</div>
</div>
<div class="column is-4">
<div class="box" id="sockets">
<div class="sp-box-head"><span class="sp-no">04</span><h2>Milter sockets</h2><div class="sp-end"><span class="tag is-success is-light">ok</span></div></div>
<div class="box" id="sender-check">
<div class="sp-box-head"><span class="sp-no">03</span><h2>Sender check</h2><div class="sp-end"><span class="tag is-success is-light">ok</span></div><a class="sp-help" href="help.html#sender-check" title="What is checked and what these numbers are for"><i class="ti ti-help-circle"></i></a></div>
<div class="table-container"><table class="table is-fullwidth">
<thead><tr><th>Milter</th><th>State</th></tr></thead>
<thead><tr><th>Period</th><th>Panel not answering</th><th>Connections turned away</th><th>Messages deferred</th></tr></thead>
<tbody>
<tr><td>opendkim<br><span class="sp-muted sp-mono sp-small">inet:127.0.0.1:8891</span></td><td><span class="tag is-success is-light">ok</span></td></tr>
<tr><td>journal-milter<br><span class="sp-muted sp-mono sp-small">unix:/run/selfpost/journal.sock</span></td><td><span class="tag is-success is-light">ok</span></td></tr>
<tr><td>rspamd <span class="sp-muted">· inbound spam filter</span><br><span class="sp-muted sp-mono sp-small">inet:antispam:11332</span></td><td><span class="tag is-success is-light">ok</span></td></tr>
<tr><td>Today</td><td>0 s</td><td>0</td><td>0</td></tr>
<tr><td>Last 30 days</td><td>4 s</td><td>0</td><td>0</td></tr>
<tr><td>Last 12 months</td><td>≈ 41 s</td><td>3</td><td>1</td></tr>
</tbody>
</table></div>
<div class="sp-box-body">
<p class="sp-muted">The panel holds each message's <code>From:</code> against the application that sends it. While the panel does not answer, Postfix defers mail instead of accepting it unchecked. Time is counted only while Postfix itself was running; ≈ marks a period with a start at which that could not be asked, and its time is at most what is shown. Counted since <span class="sp-mono">2026-10-13</span>.</p>
</div>
</div>
</div>
<div class="column is-4">
<div class="box" id="hostname">
<div class="sp-box-head"><span class="sp-no">05</span><h2>Hostname and reverse DNS</h2><div class="sp-end"><span class="tag is-success is-light">ok</span></div><a class="sp-help" href="help.html#rdns" title="Forward and reverse DNS"><i class="ti ti-help-circle"></i></a></div>
<div class="sp-box-body">
@@ -76,13 +46,61 @@
</dl>
</div>
</div>
<div class="box" id="machine">
<div class="sp-box-head"><span class="sp-no">07</span><h2>Machine</h2><div class="sp-end"><span class="tag is-success is-light">ok</span></div><a class="sp-help" href="help.html#checks" title="What these readings mean"><i class="ti ti-help-circle"></i></a></div>
<div class="table-container"><table class="table is-fullwidth">
<thead><tr><th>Resource</th><th>Usage</th><th>Detail</th></tr></thead>
<tbody>
<tr><td>CPU</td><td><progress class="progress is-small is-success mb-1" value="12" max="100"></progress>12 %</td><td class="sp-muted">2 cores, load 0.21</td></tr>
<tr><td>Memory</td><td><progress class="progress is-small is-success mb-1" value="41" max="100"></progress>41 %</td><td class="sp-muted">812 MB of 1.9 GB</td></tr>
<tr><td>Network</td><td class="sp-nowrap">↓ 18 kB/s · ↑ 44 kB/s</td><td class="sp-muted">eth0: 2.1 GB in, 6.4 GB out</td></tr>
</tbody>
</table></div>
</div>
</div>
</div>
<div class="box" id="configuration">
<div class="sp-box-head"><span class="sp-no">06</span><h2>Configuration</h2><div class="sp-end">Safe to run at any time</div></div>
<div class="sp-box-body">
<p class="mb-3">Regenerates the OpenDKIM and Postfix configuration from the database and reloads both daemons. Use it after editing the files by hand, after restoring a backup, or when the running configuration looks out of step with the domain and application lists. It does not touch the queue or the TLS certificate.</p>
<form method="post" action="#"><button type="submit" class="button"><i class="ti ti-reload"></i>Reload configuration</button></form>
<div class="column is-6">
<div class="box" id="sockets">
<div class="sp-box-head"><span class="sp-no">02</span><h2>Milter sockets</h2><div class="sp-end"><span class="tag is-success is-light">ok</span></div></div>
<div class="table-container"><table class="table is-fullwidth">
<thead><tr><th>Milter</th><th>State</th></tr></thead>
<tbody>
<tr><td>opendkim<br><span class="sp-muted sp-mono sp-small">inet:127.0.0.1:8891</span></td><td><span class="tag is-success is-light">ok</span></td></tr>
<tr><td>journal-milter<br><span class="sp-muted sp-mono sp-small">unix:/run/selfpost/journal.sock</span></td><td><span class="tag is-success is-light">ok</span></td></tr>
<tr><td>sender-check<br><span class="sp-muted sp-mono sp-small">unix:/run/selfpost/sender.sock</span></td><td><span class="tag is-success is-light">ok</span></td></tr>
<tr><td>rspamd <span class="sp-muted">· inbound spam filter</span><br><span class="sp-muted sp-mono sp-small">inet:antispam:11332</span></td><td><span class="tag is-success is-light">ok</span></td></tr>
</tbody>
</table></div>
</div>
<div class="box" id="certificate">
<div class="sp-box-head"><span class="sp-no">04</span><h2>TLS certificate</h2><div class="sp-end"><span class="tag is-warning is-light">warn</span></div><a class="sp-help" href="help.html#certificate" title="When it warns and what to do"><i class="ti ti-help-circle"></i></a></div>
<div class="sp-box-body">
<dl class="sp-facts">
<div><dt>Expires</dt><dd class="sp-mono">2026-10-03 09:14 UTC</dd></div>
<div><dt>Names</dt><dd class="sp-mono">mail.example.org</dd></div>
</dl>
<p class="mt-4 has-text-danger">Expires in 12 days — renewal has not run.</p>
<p class="sp-muted">Renewal normally runs 30 days before expiry. Check that port 80 is reachable and look for <code>acme</code> lines in the <a href="system-log.html">system log</a>.</p>
</div>
</div>
<div class="box" id="notification-mail">
<div class="sp-box-head"><span class="sp-no">06</span><h2>Notification mail</h2><div class="sp-end"><span class="tag is-warning is-light">warn</span></div><a class="sp-help" href="help.html#notifications" title="What the panel sends and what makes it arrive"><i class="ti ti-help-circle"></i></a></div>
<div class="table-container"><table class="table is-fullwidth">
<thead><tr><th>Record</th><th>State</th></tr></thead>
<tbody>
<tr><td>DKIM<br><span class="sp-muted sp-mono sp-small">sp2026._domainkey.mail.example.org</span></td><td><span class="tag is-success is-light">published</span></td></tr>
<tr><td>SPF<br><span class="sp-muted sp-mono sp-small">mail.example.org</span></td><td><span class="tag is-warning is-light">missing</span></td></tr>
</tbody>
</table></div>
<div class="sp-box-body">
<p class="sp-muted">Sent from <span class="sp-mono">postmaster@mail.example.org</span>. What to publish is under <a href="settings.html#notification-mail">Settings</a>.</p>
</div>
</div>
<div class="box" id="configuration">
<div class="sp-box-head"><span class="sp-no">08</span><h2>Configuration</h2><div class="sp-end">Safe to run at any time</div></div>
<div class="sp-box-body">
<p class="mb-3">Regenerates the OpenDKIM and Postfix configuration from the database and reloads both daemons. Use it after editing the files by hand, after restoring a backup, or when the running configuration looks out of step with the domain and application lists. It does not touch the queue or the TLS certificate.</p>
<form method="post" action="#"><button type="submit" class="button"><i class="ti ti-reload"></i>Reload configuration</button></form>
</div>
</div>
</div>
</div>
+12 -4
View File
@@ -20,6 +20,7 @@
<ul class="menu-list">
<li><a href="#dns"><i class="ti ti-world-check"></i>DNS records</a></li>
<li><a href="#apps"><i class="ti ti-apps"></i>Applications</a></li>
<li><a href="#sender-check"><i class="ti ti-shield-check"></i>Sender check</a></li>
<li><a href="#limits"><i class="ti ti-gauge"></i>Rate limits</a></li>
<li><a href="#log"><i class="ti ti-list-details"></i>Outbound log</a></li>
<li><a href="#queue"><i class="ti ti-stack-2"></i>Queue and retries</a></li>
@@ -31,12 +32,15 @@
<li><a href="#upstream"><i class="ti ti-arrow-forward-up"></i>Upstream</a></li>
<li><a href="#recipients"><i class="ti ti-address-book"></i>Valid recipients</a></li>
<li><a href="#filter"><i class="ti ti-shield-check"></i>Spam filter</a></li>
<li><a href="#inbound-log"><i class="ti ti-list-details"></i>Inbound log</a></li>
<li><a href="#filter-lists"><i class="ti ti-filter"></i>Filter lists</a></li>
</ul>
<p class="menu-label">Server</p>
<ul class="menu-list">
<li><a href="#backup"><i class="ti ti-archive"></i>Backup and moving a domain</a></li>
<li><a href="#users"><i class="ti ti-users"></i>Users and roles</a></li>
<li><a href="#settings"><i class="ti ti-settings"></i>Settings</a></li>
<li><a href="#notifications"><i class="ti ti-mail"></i>Notifications</a></li>
</ul>
<p class="menu-label">Account</p>
<ul class="menu-list">
@@ -55,8 +59,9 @@
<div class="sp-box-head"><span class="sp-no">02</span><h2>Outbound</h2></div>
<div class="sp-help-topic" id="dns"><h3>DNS records</h3><p>DKIM proves the message was signed by this relay, SPF lists the addresses allowed to send for the domain, DMARC tells receivers what to do when both fail and where to report. Publish all three exactly as shown; merge SPF into an existing record rather than adding a second one. A domain's status is the worst of its three records, and results are cached for a few minutes, so use Re-check DNS after publishing. The SPF check is shallow: it looks for this server's address in the record itself and does not follow <code>include:</code> or <code>redirect=</code>, so a record that relies on them is reported as a warning, not a failure. <code>p=none</code> does not affect delivery; tighten to <code>quarantine</code>, then <code>reject</code>, once the reports look clean. A domain's report address is set in its Domain settings.</p></div>
<div class="sp-help-topic" id="apps"><h3>Applications</h3><p>An application is one SASL login, unique across all domains. It may send as any address of its domain or only as listed addresses, and, with a list of client IPs set, only from those addresses. The password is shown once at creation or regeneration and is not stored in readable form.</p></div>
<div class="sp-help-topic" id="sender-check"><h3>Sender check</h3><p>An application sends only as itself, and that holds for both senders a message has. The envelope sender is held against the login by Postfix. The <code>From:</code> line, the one a mail program shows and the one the DKIM signature is chosen by, is held against it by the panel, under the same rule: an application set to any address of its domain may write any address of that domain, and one with a list of addresses only those. A message with any other <code>From:</code> - another of your domains, the server's own name, an address somewhere else, or none at all - is refused for good with <code>550 5.7.1</code>. It is a rejected row of the Outbound log, and its page says which address was refused. This is what keeps one application from getting mail signed with another domain's key.<br><br>The check is strict. While the panel does not answer, Postfix does not accept mail unchecked: it replies with a temporary error and the application sends again later. Server › Health counts what that costs: for how long the panel did not answer while Postfix itself was running; how many connections Postfix turned away because of it - it refuses from the connection on, before a message is offered, so a client that tries three times is three and a monitor that only connects is one; and how many messages were deferred in the middle, when the panel stopped answering after a message had begun. A time marked ≈ includes a start at which supervisord could not say since when Postfix was running: the time since the panel's last sign of life is counted, which may be more than Postfix was up. The card warns when a connection was turned away or a message deferred in the last 24 hours. If the numbers stay at nothing, the check is where it should be; if they do not, it belongs in a process of its own.</p></div>
<div class="sp-help-topic" id="limits"><h3>Rate limits</h3><p>Level 1 is a per-IP ceiling set in <code>.env</code>. Level 2 is optional, per domain and per application, entered by hand or computed from the last 30 days of sending, and can never exceed level 1. An application's own limit replaces its domain's.</p></div>
<div class="sp-help-topic" id="log"><h3>Outbound log</h3><p>One row per message and recipient, with the status Postfix last reported: queued (accepted, nothing reported yet), sent (the receiving server took it; what the mailbox does next is not reported back), deferred (not taken yet, Postfix tries again), bounced (failed for good, or the queue lifetime ran out) and rejected (refused before queueing under a level-2 rate limit, so it never reached Postfix). A message's page lists these steps and the lines Postfix wrote about it in <code>mail.log</code>, found by queue id; that file is rotated daily and about two weeks are kept, so an old message may show none. Rows are deleted after the retention set in Server › Settings, 90 days by default. A domain administrator sees only the rows of their own domains.</p></div>
<div class="sp-help-topic" id="log"><h3>Outbound log</h3><p>One row per message and recipient, with the status Postfix last reported: queued (accepted, nothing reported yet), sent (the receiving server took it; what the mailbox does next is not reported back), deferred (not taken yet, Postfix tries again), bounced (failed for good, or the queue lifetime ran out; or closed by the panel when Postfix no longer held the message and <code>mail.log</code> never said what became of it) and rejected (refused before queueing, so it never reached the queue: under a level-2 rate limit, from a client address the application does not allow, or for a <code>From:</code> line that is not the application's - the message's page says which). A message's page lists these steps and the lines Postfix wrote about it in <code>mail.log</code>, found by queue id - and for a message refused before it had one, Postfix's one line about the refusal. The lines are kept with the row: they stay on the page after that file has rotated, for as long as the row does. A message accepted before version 2.2.0 shows them only while they are still in the current file. Rows are deleted after the retention set in Server › Settings, 90 days by default, and their lines with them. A domain administrator sees only the rows of their own domains.</p></div>
<div class="sp-help-topic" id="queue"><h3>Queue and retries</h3><p>The page lists, as <code>postqueue -p</code> prints it, the messages Postfix has accepted and not yet delivered: mail being sent right now and mail deferred for a later attempt. Only the global role can open it, and it only shows; nothing here deletes or forces a message. A deferred message is tried again after the first retry delay, then at gaps that double up to a cap, until the receiving server takes it or the queue lifetime runs out and it bounces; there is no attempt counter, and the box on the page shows this server's values. A queue that keeps growing means a receiver is refusing or not answering: open the message from the Outbound log to read the reason Postfix logged.</p></div>
<div class="sp-help-topic" id="dmarc"><h3>DMARC reports</h3><p>An aggregate report is the summary a receiving provider sends, usually daily, to the <code>rua=</code> address of your DMARC record: which servers sent mail as your domain and whether SPF or DKIM passed for it. A domain's report address is one of three choices, set in its Domain settings: no reports (the default for a new domain), the SelfPost hosted address (an address on this server whose reports are parsed and shown under DMARC reports; offered only while hosted reports are enabled on the server), or an address typed for the domain (an ordinary mailbox that receives the raw XML; a button fills in your own profile e-mail). When the typed address is on another domain than the one sending, that domain must publish a report-authorization record, or providers will not send; the domain's page shows it with the other DNS records and checks it. In the tables pass means SPF or DKIM passed and aligned, fail means neither did, and your own server shows as <em>this relay</em>. A fail from any other source is someone else sending as your domain — a forwarder, an application you forgot, or abuse — so do not tighten <code>p=</code> until that sender is fixed or removed. A domain administrator reads the reports of their own domains.</p></div>
</div>
@@ -66,16 +71,19 @@
<div class="sp-help-topic" id="upstream"><h3>Upstream</h3><p>The upstream is the server that actually holds the mailboxes. Give its host and port — used exactly as written, with no MX lookup — and choose the TLS for the hop to it: Opportunistic uses TLS when the upstream offers it and otherwise sends in the clear, Required holds the mail until a TLS session is possible, Off never uses TLS. While the upstream does not answer, Postfix keeps the mail and retries as for any other mail.</p></div>
<div class="sp-help-topic" id="recipients"><h3>Valid recipients</h3><p>Choose whom this domain accepts. Listed addresses only: the relay accepts exactly the addresses you list, one per line or comma-separated, and refuses any other while the sender is still connected, so it never has to send a bounce for them. Any recipient: everything at the domain is accepted and passed on, and an address the upstream then rejects comes back as a bounce from this server to a sender who is often forged — backscatter. Prefer a list, and keep it in step with the mailboxes upstream.</p></div>
<div class="sp-help-topic" id="filter"><h3>Spam filter</h3><p>The spam filter belongs to the server, not to a domain. The server's administrator attaches it in the server's <code>.env</code>, and from then on every inbound domain is screened by it before mail is handed on; the box shows on or off and has nothing to switch. Off means mail goes to the upstream unfiltered. While the filter is down, mail is either accepted unfiltered (the default) or deferred, so the sending server tries again later, as the server's setting says; the box states which.</p></div>
<div class="sp-help-topic" id="inbound-log"><h3>Inbound log</h3><p>One row per recipient of a message offered on port 25 for an inbound domain, with what was decided: <span class="tag is-success is-light">accept</span> — handed on to the upstream; <span class="tag is-danger is-light">reject</span> — refused for good; <span class="tag is-warning is-light">tempfail</span> — the sender was asked to come back later; <span class="tag is-primary is-light">quarantine</span> — held by the filter. <em>Why</em> names who decided. <code>rspamd</code> is the spam filter, with its action, the score against the reject threshold and the heaviest symbols; <code>postfix</code> is the relay's own rule — an unknown recipient, a sender domain that does not exist — with its reply; <code>selfpost</code> means nobody judged the message: it was relayed with no filter attached. Accept says what this server did, not what the upstream did with the message afterwards.<br><br>Accepted mail is recorded as it passes; refused mail is read from <code>mail.log</code>, so its row has no subject, and a message refused after its text was sent is one row however many recipients it had. A refusal to relay for a domain that is not an inbound one is every scanner on the internet and is not recorded. Rows are kept for the number of days set in Settings, and never more than 10 000.</p></div>
<div class="sp-help-topic" id="filter-lists"><h3>Filter lists</h3><p>Two lists for the whole server that correct the spam filter by hand. An entry is a sender address, a sender domain — that domain, not its subdomains — or a client address or network in CIDR form. A sender on the Deny list is refused outright. A sender on the Allow list is given a weight no spam score outweighs, so the filter lets it through. The sender's address is whatever the sending server says it is - anyone can claim to be a sender you allow - so allow a client address or network where you can, and a sender address only for mail you would rather receive forged than lose. When entries of both lists match one message, <strong>Deny wins</strong>.<br><br>SelfPost does not apply the lists itself: it writes them as files under <code>/data/antispam/</code>, and rspamd rereads a file within seconds of a change. With the spam filter off the lists are kept and can be edited, but nothing reads them. The sender matched is the envelope sender — the one in the log — not the From line a mail program shows. Matched counts the messages in the Inbound log that the entry decided, so it reaches back as far as the log does; never means not once in that time, and such an entry is one to consider removing.</p></div>
</div>
<div class="box">
<div class="sp-box-head"><span class="sp-no">04</span><h2>Server</h2></div>
<div class="sp-help-topic" id="backup"><h3>Backup and moving a domain</h3><p>Full backup downloads one archive of the whole instance: <code>data/</code> (the database, DKIM keys, application credentials and the queue), <code>docker-compose.yml</code>, <code>.env</code> and <code>certs/</code>. It holds secrets, so leave Encrypt with a password ticked: the file is sealed as <code>.spbk</code>, the password is not stored, and without it nothing can be recovered. To restore, unpack it into an empty project directory (decrypt a <code>.spbk</code> first with <code>selfpost-backup -decrypt</code>) and start the same SelfPost version that made it; the panel refuses data from another version. The reverse-proxy configuration and <code>mail.log</code> are not included. One domain moves on its own: Export domain on its settings page writes a <code>.json</code> (or encrypted <code>.spde</code>) with its DKIM key and its applications' passwords, and Import a domain here reads it into another instance, so the published DNS needs no change.</p></div>
<div class="sp-help-topic" id="backup"><h3>Backup and moving a domain</h3><p>Full backup downloads one archive of the whole instance: <code>data/</code> (the database, DKIM keys and application credentials), <code>docker-compose.yml</code>, <code>.env</code> and <code>certs/</code>. The Postfix queue is not in it. The backup is taken while the server is running, and Postfix keeps moving messages through the queue, so a copy could hold a message twice or miss it; mail still waiting for delivery is therefore not carried over, and a restored server starts with an empty queue. To keep the queue, stop the container and archive the project directory instead. The private key in <code>certs/</code> has to be readable by the panel: give it to group 998 on the host (<code>chgrp 998</code>, <code>chmod 640</code>). Until then the panel refuses to make a backup and names the file, rather than hand over an archive without the key. It holds secrets, so leave Encrypt with a password ticked: the file is sealed as <code>.spbk</code>, the password is not stored, and without it nothing can be recovered. To restore, unpack it into an empty project directory (decrypt a <code>.spbk</code> first with <code>selfpost-backup -decrypt</code>) and start the same SelfPost version that made it; the panel refuses data from another version. The reverse-proxy configuration and <code>mail.log</code> are not included. One domain moves on its own: Export domain on its settings page writes a <code>.json</code> (or encrypted <code>.spde</code>) with its DKIM key and its applications' passwords, and Import a domain here reads it into another instance, so the published DNS needs no change.</p></div>
<div class="sp-help-topic" id="users"><h3>Users and roles</h3><p>The two roles are a reach, not a rank: every user is an administrator and signs in with a username and password. Global reaches the whole server — Overview, Server, the queue and every domain — and is the only role that adds or deletes domains and users. Domain reaches only the domains assigned to it, outbound and inbound separately (the same name on both lists is two assignments); <em>All</em> includes domains added later, and a domain user needs at least one domain or <em>All</em> on either list. The only global user cannot be demoted or deleted, and you cannot delete the account you are signed in with. A new password set here signs that user out of their other sessions.</p></div>
<div class="sp-help-topic" id="settings"><h3>Settings</h3><p>Settings holds what is true of the whole instance. Log retention is how many days the Outbound log keeps its rows, from 7 to 365 and 90 by default; a sweep every six hours deletes older ones, and it is the main driver of how fast <code>/data</code> grows. It does not change the rotation of <code>mail.log</code>, which is daily with about two weeks kept. The sending rate limit shown under it is level 1, the per-client-IP ceiling set in <code>.env</code>: changing it needs a restart, and no level-2 limit can exceed it.</p></div>
<div class="sp-help-topic" id="settings"><h3>Settings</h3><p>Settings holds what is true of the whole instance. Log retention is how many days the Outbound log keeps its rows, from 7 to 365 and 90 by default; a sweep every six hours deletes older ones, and it is the main driver of how fast <code>/data</code> grows. With the inbound relay on there is a second number, for the Inbound log, with the same range. It does not change the rotation of <code>mail.log</code>, which is daily with about two weeks kept. The sending rate limit shown under it is level 1, the per-client-IP ceiling set in <code>.env</code>: changing it needs a restart, and no level-2 limit can exceed it. Notifications are turned on or off here for everyone.</p></div>
<div class="sp-help-topic" id="notifications"><h3>Notifications</h3><p>The panel writes to its users when something needs them, from <code>postmaster@</code> the server's hostname. Four things are reported. A sending domain's SPF, DKIM or DMARC record stops matching: sent once two hourly checks in a row have found it so, and once more when the record matches again. A level-2 rate limit refuses mail: sent at the first refusal, then at most once a day for the same limit. The TLS certificate is 14 days from expiry, and again at 3 days if it is still the same certificate. A message has waited in the queue for more than three days, with one more message when none is left. A global user is told all of it; a domain administrator is told about the DNS records and limits of their own domains.<br><br>Each user turns notifications on or off under Account and needs an e-mail there; Server › Settings turns them off for everyone. They are the panel's own mail: not rows of the Outbound log, and not counted against any limit.<br><br>For that mail to arrive, the server's own name needs two DNS records of its own, as each of your domains does: a DKIM key, which the panel generates and signs notifications with, and an SPF record, <code>v=spf1 a -all</code>. Server › Settings gives both with the values to publish, and Server › Health shows whether they are in place. Without them notifications are still sent, and a receiver may file them as spam. Mail of your domains does not depend on either.</p></div>
</div>
<div class="box">
<div class="sp-box-head"><span class="sp-no">05</span><h2>Account</h2></div>
<div class="sp-help-topic" id="account"><h3>Your account</h3><p>Account is your own, whatever your role: your username and e-mail, and your password. Changing the password signs out every other session of this account and keeps the one you are using; applications have their own logins and are not affected.</p></div>
<div class="sp-help-topic" id="account"><h3>Your account</h3><p>Account is your own, whatever your role: your username and e-mail, whether the panel sends you notifications, and your password. Changing the password signs out every other session of this account and keeps the one you are using; applications have their own logins and are not affected.</p></div>
</div>
</div>
</div>
@@ -8,7 +8,7 @@
</div>
<div class="box">
<div class="sp-box-head"><span class="sp-no">01</span><h2>Add an entry</h2><a class="sp-help" href="help.html#inbound" title="How the lists are applied"><i class="ti ti-help-circle"></i></a></div>
<div class="sp-box-head"><span class="sp-no">01</span><h2>Add an entry</h2><a class="sp-help" href="help.html#filter-lists" title="How the lists are applied"><i class="ti ti-help-circle"></i></a></div>
<div class="sp-box-body">
<form class="sp-form" method="post" action="#">
<div class="columns">
@@ -18,7 +18,7 @@
</div>
<div class="column is-3">
<div class="field"><label class="label" for="etype">Entry is</label>
<div class="control"><div class="select is-fullwidth"><select id="etype" name="entry_type"><option>a sender address</option><option>a sender domain</option><option>a client IP or CIDR</option></select></div></div></div>
<div class="control"><div class="select is-fullwidth"><select id="etype" name="entry_type"><option>a sender address</option><option>a sender domain</option><option>a client address or network</option></select></div></div></div>
</div>
<div class="column is-6">
<div class="field"><label class="label" for="value">Value</label>
@@ -37,11 +37,11 @@
<div class="box">
<div class="sp-box-head"><span class="sp-no">02</span><h2>Deny</h2><div class="sp-end">3 entries · wins over Allow</div></div>
<div class="table-container"><table class="table is-fullwidth is-hoverable">
<thead><tr><th>Value</th><th>Note</th><th>Added</th><th></th></tr></thead>
<thead><tr><th>Value</th><th>Kind</th><th>Matched</th><th></th></tr></thead>
<tbody>
<tr><td class="sp-mono sp-small">@bulk.example <span class="tag is-light">domain</span></td><td class="sp-muted">newsletter nobody subscribed to</td><td class="sp-muted sp-nowrap sp-small">2026-09-02</td><td class="sp-actions"><a class="has-text-danger" href="#">Remove</a></td></tr>
<tr><td class="sp-mono sp-small">198.51.100.0/24 <span class="tag is-light">ip</span></td><td class="sp-muted">dictionary attack on lists.example.org</td><td class="sp-muted sp-nowrap sp-small">2026-08-19</td><td class="sp-actions"><a class="has-text-danger" href="#">Remove</a></td></tr>
<tr><td class="sp-mono sp-small">win@lottery.example <span class="tag is-light">address</span></td><td class="sp-muted">—</td><td class="sp-muted sp-nowrap sp-small">2026-07-30</td><td class="sp-actions"><a class="has-text-danger" href="#">Remove</a></td></tr>
<tr><td class="sp-mono sp-small">bulk.example</td><td><button type="button" class="tag is-light" popovertarget="deny-1">domain</button><div class="sp-sheet" id="deny-1" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-light">domain</span><h2>bulk.example</h2><button type="button" popovertarget="deny-1" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Note</th><td>newsletter nobody subscribed to</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-09-02</td></tr><tr><th>Matched</th><td>41 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-09-21 13:57</td></tr></tbody></table></div></div></div></div></td><td class="sp-nowrap">41</td><td class="sp-actions"><form method="post" action="#"><button type="submit" class="has-text-danger">Remove</button></form></td></tr>
<tr><td class="sp-mono sp-small">198.51.100.0/24</td><td><button type="button" class="tag is-light" popovertarget="deny-2">ip</button><div class="sp-sheet" id="deny-2" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-light">ip</span><h2>198.51.100.0/24</h2><button type="button" popovertarget="deny-2" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Note</th><td>dictionary attack on lists.example.org</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-08-19</td></tr><tr><th>Matched</th><td>1203 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-09-21 09:12</td></tr></tbody></table></div></div></div></div></td><td class="sp-nowrap">1203</td><td class="sp-actions"><form method="post" action="#"><button type="submit" class="has-text-danger">Remove</button></form></td></tr>
<tr><td class="sp-mono sp-small">win@lottery.example</td><td><button type="button" class="tag is-light" popovertarget="deny-3">address</button><div class="sp-sheet" id="deny-3" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-light">address</span><h2>win@lottery.example</h2><button type="button" popovertarget="deny-3" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Note</th><td>—</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-07-30</td></tr><tr><th>Matched</th><td>never in the last 90 days</td></tr></tbody></table></div></div></div></div></td><td class="sp-muted sp-nowrap">never</td><td class="sp-actions"><form method="post" action="#"><button type="submit" class="has-text-danger">Remove</button></form></td></tr>
</tbody>
</table></div>
</div>
@@ -50,10 +50,10 @@
<div class="box">
<div class="sp-box-head"><span class="sp-no">03</span><h2>Allow</h2><div class="sp-end">2 entries</div></div>
<div class="table-container"><table class="table is-fullwidth is-hoverable">
<thead><tr><th>Value</th><th>Note</th><th>Added</th><th></th></tr></thead>
<thead><tr><th>Value</th><th>Kind</th><th>Matched</th><th></th></tr></thead>
<tbody>
<tr><td class="sp-mono sp-small">@partner.example <span class="tag is-light">domain</span></td><td class="sp-muted">invoices were scored as spam</td><td class="sp-muted sp-nowrap sp-small">2026-09-11</td><td class="sp-actions"><a class="has-text-danger" href="#">Remove</a></td></tr>
<tr><td class="sp-mono sp-small">noreply@github.example <span class="tag is-light">address</span></td><td class="sp-muted">security advisories</td><td class="sp-muted sp-nowrap sp-small">2026-06-04</td><td class="sp-actions"><a class="has-text-danger" href="#">Remove</a></td></tr>
<tr><td class="sp-mono sp-small">partner.example</td><td><button type="button" class="tag is-light" popovertarget="allow-1">domain</button><div class="sp-sheet" id="allow-1" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-light">domain</span><h2>partner.example</h2><button type="button" popovertarget="allow-1" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Note</th><td>invoices were scored as spam</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-09-11</td></tr><tr><th>Matched</th><td>7 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-09-19 16:40</td></tr></tbody></table></div></div></div></div></td><td class="sp-nowrap">7</td><td class="sp-actions"><form method="post" action="#"><button type="submit" class="has-text-danger">Remove</button></form></td></tr>
<tr><td class="sp-mono sp-small">noreply@github.example</td><td><button type="button" class="tag is-light" popovertarget="allow-2">address</button><div class="sp-sheet" id="allow-2" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-light">address</span><h2>noreply@github.example</h2><button type="button" popovertarget="allow-2" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Note</th><td>security advisories</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-06-04</td></tr><tr><th>Matched</th><td>18 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-09-21 13:30</td></tr></tbody></table></div></div></div></div></td><td class="sp-nowrap">18</td><td class="sp-actions"><form method="post" action="#"><button type="submit" class="has-text-danger">Remove</button></form></td></tr>
</tbody>
</table></div>
</div>
@@ -64,10 +64,10 @@
<div class="sp-box-head"><span class="sp-no">04</span><h2>Where the lists go</h2><div class="sp-end"><span class="tag is-success is-light">applied</span></div></div>
<div class="sp-box-body">
<dl class="sp-facts">
<div><dt>Applied by</dt><dd>rspamd, from map files</dd></div>
<div><dt>Files</dt><dd class="sp-mono sp-small">/data/antispam/allow.map<br>/data/antispam/deny.map</dd></div>
<div><dt>Applied by</dt><dd>rspamd, from map files it rereads within seconds of a change</dd></div>
<div><dt>Files</dt><dd><span class="sp-mono">/data/antispam/</span> — six, one for each list and kind of entry</dd></div>
<div><dt>Last written</dt><dd>2026-09-11 08:40 UTC</dd></div>
<div><dt>When the filter is off</dt><dd class="sp-muted">The lists are kept and editable, but nothing reads them until <code>INBOUND_ANTISPAM_MILTER</code> is set.</dd></div>
<div><dt>When both lists match</dt><dd>Deny wins, whichever entries they are.</dd></div>
</dl>
</div>
</div>
@@ -3,7 +3,7 @@
<div class="sp-grow">
<p class="sp-kicker">Inbound</p>
<h1 class="title is-3">Inbound log</h1>
<p class="sp-lead">What port 25 decided about each message, one row per recipient. Accepts come from the inbound journal, rejects from <code>mail.log</code>; the filter's reasons appear when the spam filter is on.</p>
<p class="sp-lead">What port 25 decided about each message, one row per recipient. Accepted mail is recorded as it passes, refused mail is read from <code>mail.log</code>; the filter's verdict appears when the spam filter is on. Rows older than 90 days are deleted — change that in <a href="settings.html">Settings</a>.</p>
</div>
</div>
@@ -17,17 +17,18 @@
<input class="input" type="date" name="to" value="2026-09-21" aria-label="To date">
<button type="submit" class="button"><i class="ti ti-filter"></i>Filter</button>
</form>
<a class="sp-help" href="help.html#inbound-log" title="What the decisions mean"><i class="ti ti-help-circle"></i></a>
</div>
<div class="table-container"><table class="table is-fullwidth is-hoverable">
<thead><tr><th>Time, UTC</th><th>From · client</th><th>To</th><th>Subject</th><th>Decision</th><th>Why</th><th></th></tr></thead>
<thead><tr><th>Time, UTC</th><th>From</th><th>To</th><th>Subject</th><th>Decision</th><th></th></tr></thead>
<tbody>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 14:00:18</td><td>t.berg@<wbr>uni.example<br><span class="sp-mono sp-muted sp-small">192.0.2.77</span></td><td>dev@<wbr>lists.example.org</td><td class="sp-clip">Re: [dev] patch series v3</td><td><span class="tag is-success is-light">accept</span></td><td class="sp-muted sp-small">rspamd · score 0.4</td><td class="sp-actions"><a href="in-filter-lists.html">Deny sender</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:59:02</td><td>invoice@<wbr>partner-billing.example<br><span class="sp-mono sp-muted sp-small">203.0.113.140</span></td><td>owner@<wbr>lists.example.org</td><td class="sp-clip">Outstanding invoice — action required</td><td><span class="tag is-primary is-light">quarantine</span></td><td class="sp-muted sp-small">rspamd · score 9.8 · PHISHING, FORGED_SENDER</td><td class="sp-actions"><a href="in-quarantine-message.html">Review</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:57:41</td><td>promo@<wbr>bulk.example<br><span class="sp-mono sp-muted sp-small">198.51.100.90</span></td><td>sales@<wbr>lists.example.org</td><td class="sp-clip sp-muted">—</td><td><span class="tag is-danger is-light">reject</span></td><td class="sp-muted sp-small">postfix · 550 5.1.1 unknown recipient</td><td class="sp-actions"><a href="in-filter-lists.html">Deny sender</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:51:30</td><td>billing@<wbr>partner.example<br><span class="sp-mono sp-muted sp-small">198.51.100.12</span></td><td>owner@<wbr>lists.example.org</td><td class="sp-clip">Invoice 2026-0917</td><td><span class="tag is-danger is-light">reject</span></td><td class="sp-muted sp-small">rspamd · score 16.2 · BAYES_SPAM</td><td class="sp-actions"><a href="in-filter-lists.html">Allow sender</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:44:02</td><td>hr@<wbr>partner.example<br><span class="sp-mono sp-muted sp-small">192.0.2.31</span></td><td>jobs@<wbr>acme.io</td><td class="sp-clip">Application: backend engineer</td><td><span class="tag is-warning is-light">tempfail</span></td><td class="sp-muted sp-small">selfpost · upstream 10.0.4.12 timed out</td><td class="sp-actions"></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:30:55</td><td>noreply@<wbr>github.example<br><span class="sp-mono sp-muted sp-small">192.0.2.200</span></td><td>security@<wbr>lists.example.org</td><td class="sp-clip">[advisory] new report</td><td><span class="tag is-success is-light">accept</span></td><td class="sp-muted sp-small">rspamd · allow list</td><td class="sp-actions"></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 14:00:18</td><td>t.berg@<wbr>uni.example<br><span class="sp-mono sp-muted sp-small">192.0.2.77</span></td><td>dev@<wbr>lists.example.org</td><td class="sp-clip">Re: [dev] patch series v3</td><td><button type="button" class="tag is-success is-light" popovertarget="why-1">accept</button><div class="sp-sheet" id="why-1" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-success is-light">accept</span><h2>Why</h2><button type="button" popovertarget="why-1" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Verdict</th><td>no action, score 0.40 / 15.00</td></tr><tr><th>Subject</th><td>Re: [dev] patch series v3</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"><a href="in-filter-lists.html">Deny sender</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:59:02</td><td>invoice@<wbr>partner-billing.example<br><span class="sp-mono sp-muted sp-small">203.0.113.140</span></td><td>owner@<wbr>lists.example.org</td><td class="sp-clip">Outstanding invoice — action required</td><td><button type="button" class="tag is-success is-light" popovertarget="why-2">accept</button><div class="sp-sheet" id="why-2" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-success is-light">accept</span><h2>Why</h2><button type="button" popovertarget="why-2" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Verdict</th><td>add header, score 9.80 / 15.00</td></tr><tr><th>Symbols</th><td class="sp-mono sp-small">PHISHING(5.00)<br>FORGED_SENDER(3.00)<br>R_SPF_FAIL(1.00)</td></tr><tr><th>Subject</th><td>Outstanding invoice — action required</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"><a href="in-filter-lists.html">Deny sender</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:57:41</td><td>promo@<wbr>bulk.example<br><span class="sp-mono sp-muted sp-small">198.51.100.90</span></td><td>sales@<wbr>lists.example.org</td><td class="sp-clip sp-muted">—</td><td><button type="button" class="tag is-danger is-light" popovertarget="why-3">reject</button><div class="sp-sheet" id="why-3" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-danger is-light">reject</span><h2>Why</h2><button type="button" popovertarget="why-3" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>postfix</td></tr><tr><th>Reply</th><td class="sp-mono sp-small">550 5.1.1 &lt;sales@lists.example.org&gt;: Recipient address rejected: User unknown in relay recipient table</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:51:30</td><td>billing@<wbr>partner.example<br><span class="sp-mono sp-muted sp-small">198.51.100.12</span></td><td>owner@<wbr>lists.example.org</td><td class="sp-clip sp-muted">—</td><td><button type="button" class="tag is-danger is-light" popovertarget="why-4">reject</button><div class="sp-sheet" id="why-4" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-danger is-light">reject</span><h2>Why</h2><button type="button" popovertarget="why-4" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Reply</th><td class="sp-mono sp-small">5.7.1 Spam message rejected</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"><a href="in-filter-lists.html">Allow sender</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:44:02</td><td>hr@<wbr>partner.example<br><span class="sp-mono sp-muted sp-small">192.0.2.31</span></td><td>jobs@<wbr>acme.io</td><td class="sp-clip sp-muted">—</td><td><button type="button" class="tag is-warning is-light" popovertarget="why-5">tempfail</button><div class="sp-sheet" id="why-5" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-warning is-light">tempfail</span><h2>Why</h2><button type="button" popovertarget="why-5" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Reply</th><td class="sp-mono sp-small">4.7.1 Try again later</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"><a href="in-filter-lists.html">Allow sender</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:30:55</td><td>noreply@<wbr>github.example<br><span class="sp-mono sp-muted sp-small">192.0.2.200</span></td><td>security@<wbr>lists.example.org</td><td class="sp-clip">[advisory] new report</td><td><button type="button" class="tag is-success is-light" popovertarget="why-6">accept</button><div class="sp-sheet" id="why-6" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-success is-light">accept</span><h2>Why</h2><button type="button" popovertarget="why-6" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Verdict</th><td>no action, score -98.10 / 15.00</td></tr><tr><th>Symbols</th><td class="sp-mono sp-small">SELFPOST_ALLOW_FROM(-100.00)<br>R_DKIM_ALLOW(-0.20)</td></tr><tr><th>Subject</th><td>[advisory] new report</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"><a href="in-filter-lists.html">Deny sender</a></td></tr>
</tbody>
</table></div>
<div class="sp-box-foot"><span>Page 1 of 38</span><a href="#">Older →</a><span class="sp-end">Allow / Deny sender opens <a href="in-filter-lists.html">Filter lists</a> with the entry filled in · global administrator only</span></div>
<div class="sp-box-foot"><span>Page 1 of 38</span><a href="#">Older →</a><span class="sp-end">A decision opens its reason · Allow / Deny sender opens <a href="in-filter-lists.html">Filter lists</a> with the entry filled in</span></div>
</div>
@@ -25,6 +25,7 @@
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 14:01:50</td><td>shop@<wbr>shop.example.org</td><td>anna.lind@<wbr>outlook.com</td><td class="sp-clip">Your order #48212 has shipped</td><td><span class="tag is-success is-light">sent</span></td><td class="sp-actions"><a href="out-message.html">Details</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:58:07</td><td>alerts@<wbr>example.org</td><td>noc@<wbr>example.org</td><td class="sp-clip">[FIRING] disk usage above 90 % on db-2</td><td><span class="tag is-warning is-light">deferred</span></td><td class="sp-actions"><a href="out-message.html">Details</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:41:19</td><td>no-reply@<wbr>notify.acme.io</td><td>j.doe@<wbr>nonexistent.example</td><td class="sp-clip">Reset your password</td><td><span class="tag is-danger is-light">bounced</span></td><td class="sp-actions"><a href="out-message.html">Details</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:40:55</td><td>shop@<wbr>shop.example.org</td><td>k.berg@<wbr>web.de</td><td class="sp-clip">A message from the director</td><td><span class="tag is-danger is-light">wrong From</span></td><td class="sp-actions"><a href="out-message.html">Details</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:40:02</td><td>prod@<wbr>example.org</td><td>billing@<wbr>partner.example</td><td class="sp-clip">Invoice 2026-0917</td><td><span class="tag is-light">rate-limited</span></td><td class="sp-actions"><a href="out-message.html">Details</a></td></tr>
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:12:44</td><td>shop@<wbr>shop.example.org</td><td>p.novak@<wbr>seznam.cz</td><td class="sp-clip">We received your return request</td><td><span class="tag is-success is-light">sent</span></td><td class="sp-actions"><a href="out-message.html">Details</a></td></tr>
</tbody>
@@ -16,11 +16,12 @@
<div class="field"><label class="label" for="ret">Keep outbound log rows, days</label>
<div class="control"><input class="input" id="ret" name="send_log_retention_days" type="number" min="7" max="365" value="30" required></div>
<p class="help">The main driver of <code>/data</code> growth. Does not affect <code>mail.log</code> rotation (fourteen daily files).</p></div>
<div class="field"><label class="label" for="inret">Keep inbound log rows, days</label>
<div class="control"><input class="input" id="inret" name="inbound_spam_log_retention_days" type="number" min="7" max="365" value="90" required></div>
<p class="help">The inbound log also never holds more than 10 000 rows: refused mail is logged at the rate strangers knock on port 25.</p></div>
</div>
</div>
</div>
<div class="column is-6">
<div class="box" id="rate-limits">
<div class="sp-box-head"><span class="sp-no">02</span><h2>Sending rate limits</h2><div class="sp-end">Read-only</div><a class="sp-help" href="help.html#limits" title="Levels 1 and 2"><i class="ti ti-help-circle"></i></a></div>
<div class="sp-box-body">
@@ -31,6 +32,43 @@
</div>
</div>
</div>
<div class="column is-6">
<div class="box" id="notifications">
<div class="sp-box-head"><span class="sp-no">03</span><h2>Notifications</h2><a class="sp-help" href="help.html#notifications" title="What is sent and when"><i class="ti ti-help-circle"></i></a></div>
<div class="sp-box-body">
<div class="field"><label class="checkbox"><input type="checkbox" name="notify" checked> Send notification e-mails</label>
<p class="help">From <span class="sp-mono">postmaster@mail.example.org</span>, to every user who has an e-mail and has not turned them off under <a href="account.html">Account</a>.</p></div>
</div>
</div>
<div class="box" id="notification-mail">
<div class="sp-box-head"><span class="sp-no">04</span><h2>Notification mail</h2><div class="sp-end">DNS records of the server's own name</div><a class="sp-help" href="help.html#notifications" title="What the panel sends and what makes it arrive"><i class="ti ti-help-circle"></i></a></div>
<div class="sp-record">
<div class="sp-record-head"><h3>DKIM</h3><span class="tag is-success is-light">published</span><small>Not a secret · selector <code>sp2026</code></small></div>
<div class="sp-row">
<div class="field"><label class="label">Host / name</label>
<div class="field has-addons"><div class="control is-expanded"><input class="input" readonly value="sp2026._domainkey.mail.example.org"></div><div class="control"><button type="button" class="button"><i class="ti ti-copy"></i>Copy</button></div></div></div>
<div class="field"><label class="label">Type</label><div class="control"><input class="input" readonly value="TXT"></div></div>
</div>
<div class="field"><label class="label">Value to publish</label>
<div class="field has-addons"><div class="control is-expanded"><textarea class="textarea" rows="5" readonly>v=DKIM1; k=rsa; p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAq7n2Lw0c5vXeR1dYt8uJb3mKs6pHf9aZoWg4iNy2TqCx0lVjE7rUh5sBk1dMa8fPz3oGc6yXw9tIe4nQv2uL…IDAQAB</textarea></div><div class="control"><button type="button" class="button"><i class="ti ti-copy"></i>Copy</button></div></div></div>
</div>
<div class="sp-record">
<div class="sp-record-head"><h3>SPF</h3><span class="tag is-warning is-light">missing</span><small>For the server's own name, not for your domains</small></div>
<div class="sp-row">
<div class="field"><label class="label">Host / name</label>
<div class="field has-addons"><div class="control is-expanded"><input class="input" readonly value="mail.example.org"></div><div class="control"><button type="button" class="button"><i class="ti ti-copy"></i>Copy</button></div></div></div>
<div class="field"><label class="label">Type</label><div class="control"><input class="input" readonly value="TXT"></div></div>
</div>
<div class="field"><label class="label">Value to publish</label>
<div class="field has-addons"><div class="control is-expanded"><input class="input" readonly value="v=spf1 a -all"></div><div class="control"><button type="button" class="button"><i class="ti ti-copy"></i>Copy</button></div></div></div>
<p class="help">Notifications are sent without it, and a receiver may file them as spam.</p>
</div>
</div>
</div>
</div>
<div class="buttons"><button type="submit" class="button is-primary">Save settings</button></div>
</form>
@@ -269,4 +269,44 @@ code { background: var(--sp-well); color: var(--sp-ink); border-radius: 3px; pad
.sp-record-head { flex-wrap: wrap; }
.sp-record-head small { margin-left: 0; text-align: left; flex-basis: 100%; }
.sp-postmark { display: none; }
.sp-cards { display: flex; flex-direction: column; padding-inline: .75rem; }
.sp-cards > .column { display: contents; }
.sp-cards .box { margin-bottom: 1.5rem; }
.sp-cards > .column:first-child > .box:nth-child(1) { order: 1; }
.sp-cards > .column:first-child > .box:nth-child(2) { order: 3; }
.sp-cards > .column:first-child > .box:nth-child(3) { order: 5; }
.sp-cards > .column:first-child > .box:nth-child(4) { order: 7; }
.sp-cards > .column:last-child > .box:nth-child(1) { order: 2; }
.sp-cards > .column:last-child > .box:nth-child(2) { order: 4; }
.sp-cards > .column:last-child > .box:nth-child(3) { order: 6; }
.sp-cards > .column:last-child > .box:nth-child(4) { order: 8; }
}
/* cards: two stacks of boxes that are read in rows - the odd ones on the left,
the even ones on the right, each stack closed up so a short box leaves no
gap beside a tall one. On a phone the stacks dissolve and the boxes come in
the order of their numbers (the rules above). */
/* sheet: what does not fit a table row, opened over the page from the row's
tag. It is the browser's own popover (the tag is a button with
popovertarget), so it needs no script: Esc, a click outside and the close
button all shut it, and focus returns to the tag. It rises from the bottom
edge across the whole window - the one component that is not laid out by the
page, hence the widths - with its content, a table of name and value, in a
column of reading width in the middle. */
.sp-sheet {
inset: auto 0 0 0; width: auto; margin: 0; padding: 0; border: 0; border-top: 3px solid var(--sp-brick);
max-height: 75vh; overflow: auto; background: #fff; color: inherit; box-shadow: 0 -12px 40px rgba(40, 20, 15, .22);
}
.sp-sheet-in { max-width: 44rem; margin: 0 auto; padding: 0 1.25rem; }
.sp-sheet::backdrop { background: rgba(40, 20, 15, .38); }
.sp-sheet-head { display: flex; align-items: center; gap: .7rem; padding: 1rem 0 .8rem; border-bottom: 1px solid var(--sp-rule); }
.sp-sheet-head h2 { font-size: 1.15rem; font-weight: 600; margin-right: auto; }
.sp-sheet-head button { border: 0; background: none; cursor: pointer; font-size: 1.3rem; color: var(--sp-muted); }
.sp-sheet-body { padding: .4rem 0 1.4rem; }
/* The sheet's content is a table of name and value: the name is a label, set like the dt of facts. */
.sp-sheet .table th { font-family: var(--sp-mono); font-size: var(--sp-fs-xs); font-weight: 500; letter-spacing: .08em; text-transform: uppercase; color: var(--sp-muted); white-space: nowrap; vertical-align: top; width: 1%; padding-right: 2rem; }
.sp-sheet .table td { overflow-wrap: anywhere; }
/* A tag that opens a sheet is a button: it keeps the tag's look and says it can
be pressed with the dotted underline links to more detail carry. */
button.tag { border: 0; cursor: pointer; text-decoration: underline dotted; text-underline-offset: 3px; }
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+64 -8
View File
@@ -107,7 +107,7 @@ than the default «meaningful step» rule in [§ Commits and release build](#com
2. Git tag `vX.Y.Z` and publish the GitHub Release only on explicit request (see
[§ Release image](#release-image)).
**Planned version cuts** (from pin `2.0.0`; adjust if semver changes mid-track):
**Planned version cuts** (from pin `2.5.1`; adjust if semver changes mid-track):
| Stage | ID | Cut |
|---|---|---|
@@ -124,7 +124,19 @@ than the default «meaningful step» rule in [§ Commits and release build](#com
| 11 | panel-redesign plan: component kit, stages, reorder (docs only, owner-requested cut) | `1.9.4` (PATCH) |
| 12 | GHCR release fix: keep per-arch images, sweep orphaned versions (CI only) | `1.9.5` (PATCH) |
| 13 | panel-redesign | `2.0.0` (MAJOR, no compatibility) |
| 14 | inbound-antispam-panel | `2.1.0` (planned; was `1.10.0`, reordered 2026-10-09) |
| 14 | DMARC ingest drops and bounds, full backup from the panel, no queue in a backup | `2.0.1` (PATCH) |
| 15 | inbound-antispam-panel | `2.1.0` (was `1.10.0`, reordered 2026-10-09) |
| 16 | delivery-log-storage | `2.2.0` |
| 17 | panel-notifications | `2.3.0` |
| 18 | from-header-binding | `2.4.0` |
| 19 | code audit 2026-10-12 and its plan (docs only, owner-requested cut) | `2.4.1` (PATCH) |
| 20 | audit-2026-10-12 stage 0: release hygiene | `2.4.2` (PATCH) |
| 21 | audit-2026-10-12 stages 1 and 2 | `2.5.0` |
| 22 | audit-2026-10-12 stage 2b: the log tailer | `2.5.1` |
| 23 | audit-2026-10-12 stage 2c: the sender check reads From: as OpenDKIM does | `2.5.2` |
| 24 | audit-2026-10-12 stage 3 | `2.5.3` |
| 25 | audit-2026-10-12 stage 4 | `2.5.4` |
| 26 | audit-2026-10-12 stage 5 | `2.6.0` |
Docs-only prep (checklists in plans, no product code) uses the same per-step
commit rule but **no** version cut until the next product stage ships.
@@ -139,7 +151,7 @@ commit rule but **no** version cut until the next product stage ships.
| **SQLite** | `modernc.org/sqlite` (pure Go, no cgo) |
| **Build** | [Makefile](../Makefile): primary targets `vet`, `test`, `build`, `e2e` (also `all`, `clean`) |
| **Container** | Docker + Compose v2 on the dev host and in CI |
| **Image (build stage)** | `golang:1.26-bookworm` — [build/Dockerfile](../build/Dockerfile) |
| **Image (build stage)** | `golang:1.26-bookworm` — [build/Dockerfile](../build/Dockerfile). The tag floats to the newest 1.26 patch release, and most standard-library security fixes (`net/http`, `html/template`, …) arrive only that way: rebuild the image after each Go patch release, no edit needed. Dependencies are raised by hand (`go get …@latest`, `go mod tidy`), at least at each audit |
| **Image (runtime)** | `debian:bookworm-slim` + Postfix, OpenDKIM, supervisord, SASL, logrotate |
| **CI** | GitHub Actions — [.github/workflows/](../.github/workflows/) |
| **Image registry** | `ghcr.io/mixeme/selfpost` |
@@ -203,6 +215,37 @@ Plex [OFL.txt](../internal/web/view/static/OFL.txt) under
`/usr/share/doc/selfpost/`. The panel serves the AGPL text at `/license` and
the OFL text at `/static/OFL.txt`.
### Code that relies on another component's behaviour
The owner's rule (2026-10-14): what another program does is read in its
documentation or its source - never guessed. SelfPost reads Postfix's log
lines and reply codes, feeds OpenDKIM's tables, parses `postqueue`,
`postconf`, `supervisorctl` and `db_dump` output, talks the milter protocol
through go-milter, relies on SQLite's locking, on logrotate's and Docker's
behaviour. Every block of code that does so carries a comment that says
what it relies on and where that is written, for the version the image runs
(the Debian bookworm package, not upstream's head):
```go
// Postfix 3.7.11, src/smtpd/smtpd_check.c,
// reject_auth_sender_login_mismatch(): the SASL login is compared with
// each owner in sender_login_maps by strcasecmp_utf8.
```
The Debian sources of the image's versions are fetched with `apt-get source`
inside a `debian:bookworm` container (`postfix`, `opendkim`, `cyrus-sasl2`,
`supervisor`, `logrotate`); `dev/upstream/` (not in git) is where they are
kept for reading. Go modules are read in the module cache at the version
`go.mod` pins.
A document instead of source: its URL and section. Where neither answers the
question, the comment says so and what was measured instead - "measured on
the e2e stand, 2026-10-14: ..." - and a measurement is never written as if
it were read. A test fixture copied from a real log says where and when it
was seen; one composed by hand says that it was composed. The heuristics
review of 2026-10-14 (audit-2026-10-12, stage 2c) brought the code that
existed then up to this rule.
---
## Building binaries and the image
@@ -264,13 +307,22 @@ The release image is published **only** for a SemVer version `X.Y.Z`: a
that supplies that version. Pushing a git tag alone does not publish. Ordinary
commits, and a dispatch from `main` without a version input, do not publish.
The version is the single source that drives the image tag and `-ldflags` in
the binaries so they cannot drift apart.
the binaries so they cannot drift apart. The e2e gate rebuilds the image it
then pushes, so `release.yml` hands it the version as `SELFPOST_E2E_VERSION`;
the suite's `image_stamped_with_version` and a check right before the push
both compare `panel -version` and `selfpost-backup -version` with it. (Until
`2.4.2` the rebuild stamped every published image `e2e`; those images stay as
they are.)
**Steps (on explicit request):**
1. Close `[Unreleased]` in [CHANGELOG.md](../CHANGELOG.md) and bump the pinned
tag in [deploy/docker-compose.yml](../deploy/docker-compose.yml) (and any
local-trial image references) in the **same** release commit.
local-trial image references) in the **same** release commit. While no
image of the current line is published, README, guide § Fixed image tag
and the compose file's header say so and show the local build under the
pinned name; bump the version there too. The first publication of the
line removes those notes.
2. Create and push git tag `vX.Y.Z` on that commit.
3. Publish the GitHub Release for `vX.Y.Z` (not a draft).
4. Workflow [release.yml](../.github/workflows/release.yml) builds, e2e-gates,
@@ -390,7 +442,8 @@ make e2e
[test/e2e/compose.override.yml](../test/e2e/compose.override.yml) — same
`cap_drop`/`cap_add`/`no-new-privileges` as production. Override: high ports
(`20465`/`20587`/`20080`), test hostname, self-signed TLS,
`PANEL_COOKIE_SECURE=false`, isolated compose project. Mail is hermetic:
`PANEL_COOKIE_SECURE=false`, isolated compose project, and the image's
`VERSION` from `SELFPOST_E2E_VERSION` (unset locally: `e2e`). Mail is hermetic:
CoreDNS (fake zone) + Postfix `smtp-sink` as sink-MX; DKIM TXT is scraped from
the panel and published into the zone — the test verifies the records an
operator would actually use.
@@ -417,7 +470,9 @@ Workflows in [.github/workflows/](../.github/workflows/). What each job runs —
`design-first` (needs the full history) → `gofmt -l` → `go vet ./...` →
`go test ./...` (main module, no e2e; includes the panel guard tests, see
[§ Panel pages](#panel-pages-the-component-kit-and-its-guards)).
[§ Panel pages](#panel-pages-the-component-kit-and-its-guards)) → the e2e
module compiled, not run (`go vet ./...` and `go test -c -o /dev/null .` in
`test/e2e`).
### `release.yml` — published GitHub Release, or `workflow_dispatch` with SemVer
@@ -435,7 +490,8 @@ release: published
prepare (version from release tag or workflow_dispatch input; checkout vX.Y.Z)
→ build [matrix: ubuntu-latest / ubuntu-24.04-arm]
→ docker build --load (VERSION from prepare)
→ e2e (test/e2e)
→ e2e (test/e2e; rebuilds selfpost:e2e with SELFPOST_E2E_VERSION=X.Y.Z)
→ check panel / selfpost-backup -version = X.Y.Z
→ push ghcr.io/...:X.Y.Z-amd64 | X.Y.Z-arm64
→ merge
→ docker buildx imagetools create → unified manifest X.Y.Z
+272 -28
View File
@@ -33,6 +33,7 @@ changed (see the CHANGELOG, *Removed*).
- [Queue and System log](#queue-and-system-log)
- [Account](#account)
- [Settings](#settings)
- [Notifications](#notifications)
- [Users](#users)
- [Sessions](#sessions)
- [Upgrading](#upgrading)
@@ -183,7 +184,18 @@ cat ./data/setup-token
#### Fixed image tag
`deploy/docker-compose.yml` pins an explicit version (`ghcr.io/mixeme/selfpost:X.Y.Z`),
deliberately never `:latest`. The current pin is `2.0.0`. Intermediate
deliberately never `:latest`. The current pin is `2.5.1`. **No 2.x image is
published on ghcr.io yet** (the newest there is `1.3.0`); until one is, build
it from the checkout under the pinned name, and Compose uses the local image
instead of pulling:
```sh
docker build -f build/Dockerfile --build-arg VERSION=2.5.1 \
-t ghcr.io/mixeme/selfpost:2.5.1 .
```
`VERSION` must equal the tag: it is the version the backup check compares
(see below). Intermediate
CHANGELOG sections (`0.2.0`…`0.6.0`) record development cuts from before that
image was published. Pinning matters because of the backup version check (see
[Full backup and restore](#full-backup-and-restore)): the panel binary's
@@ -209,11 +221,13 @@ expected to set; defaults match the code exactly.
| `DMARC_MESSAGE_SIZE_LIMIT` | Maximum report message size in bytes when DMARC ingest is on. | `5242880` (5 MiB) | `.env` |
| `INBOUND_ANTISPAM_MILTER` | Optional milter on the inbound relay listener only (requires `INBOUND_RELAY_ENABLE=true`; not 465/587 or DMARC-only port 25). Empty = off. Format `inet:host:port` or `unix:/path`. Example with [deploy/antispam/docker-compose.antispam.yml](../deploy/antispam/docker-compose.antispam.yml): `inet:antispam:11332`. The panel reads it to report on the *Spam filter* box of the inbound pages and to check that the filter answers (Server › Health). | *(empty)* | `.env` |
| `INBOUND_ANTISPAM_MILTER_ACTION` | What Postfix does if that milter is down: `accept` (fail-open) or `tempfail` (defer). The *Spam filter* box states which. | `accept` | `.env` |
| `INBOUND_SPAM_LOG_RETENTION_DAYS` | How long the inbound journal keeps a row, 7-365 days; it also never holds more than 10 000 rows. Seeds the setting on the first start with the inbound relay on. | `90` | `.env` |
| `INBOUND_JOURNAL_MILTER_SOCKET` | Unix socket of the panel's inbound journal milter, the last milter on port 25 while the inbound relay is on. It records accepted inbound mail and never refuses any; there is no reason to change it. | `/run/selfpost/inbound-journal.sock` | compose |
| `INBOUND_RATE_LIMIT_MESSAGES_PER_IP` | Coarse per-client-IP cap on inbound smtpd (`smtpd_client_message_rate_limit`). Uses the same window as `RATE_LIMIT_WINDOW_SECONDS`. | `20` | `.env` |
| `INBOUND_MESSAGE_SIZE_LIMIT` | Maximum message size in bytes on inbound smtpd (`message_size_limit`). When both inbound relay and DMARC ingest are on, the shared listener uses the **greater** of this and `DMARC_MESSAGE_SIZE_LIMIT`. | `26214400` (25 MiB) | `.env` |
| `RATE_LIMIT_MESSAGES_PER_IP` | Level-1 backstop: maximum messages one client IP may submit per window (Postfix `smtpd_client_message_rate_limit`). See [Rate limiting — level 1](#rate-limiting--level-1-ip-backstop). | `100` | `.env` |
| `RATE_LIMIT_WINDOW_SECONDS` | Level-1 window length in seconds (Postfix `anvil_rate_time_unit`). | `3600` | `.env` |
| `SEND_LOG_RETENTION_DAYS` | Initial default for how many days of send-log history are kept before the background sweep deletes rows — the main driver of `/data` growth over time. After the first panel start, change retention under **Server › Settings** (global role); the env value is only used to seed SQLite when the setting has never been saved. | `90` | `.env` |
| `SEND_LOG_RETENTION_DAYS` | Initial default for how many days of send-log history are kept before the background sweep deletes rows, and with them the `mail.log` lines kept for each message (1–2 KB a message) — the main driver of `/data` growth over time and of the size of a full backup. After the first panel start, change retention under **Server › Settings** (global role); the env value is only used to seed SQLite when the setting has never been saved. | `90` | `.env` |
| `PANEL_SESSION_IDLE_DAYS` | Sliding idle timeout for the panel login session, in days. There is no absolute cap: a user who keeps coming back stays signed in indefinitely. | `7` | `.env` |
| `SELFPOST_DNS_RESOLVERS` | Comma-separated recursive resolvers the panel's PTR/SPF/DKIM/DMARC checks query directly (so they report what the internet sees, not what this host's stub resolver synthesises). | `1.1.1.1:53`, `8.8.8.8:53`, `9.9.9.9:53` when unset | `.env` |
| `TRUSTED_PROXY_CIDR` | Comma-separated CIDRs (bare IPs allowed) of reverse proxies allowed to supply `X-Forwarded-For` for login, setup, and account-change rate-limiting. **Leave unset unless you know the exact address of your reverse proxy.** A wrong value lets a client spoof its rate-limit key by sending a forged `X-Forwarded-For` header — the panel trusts the last hop only when the TCP peer matches one of these CIDRs. Behind the default Apache host-network setup this is typically the Docker bridge gateway, e.g. `172.18.0.1`. | *(empty — XFF ignored)* | `.env` |
@@ -319,6 +333,10 @@ docker compose -f docker-compose.yml -f traefik/docker-compose.traefik.yml up -d
Schedule `extract-cert.sh` (cron or a timer) alongside Traefik's renewals.
Whatever the proxy, SelfPost notices a renewed certificate in the mount within
a day (`TLS_RELOAD_INTERVAL_SECONDS`) and starts serving it on 465 and 587
without a restart.
## Instance administration
After sign-in the global role opens on **Overview** — the place to answer "is
@@ -374,7 +392,7 @@ and reverse DNS. A check the panel could not run shows as a warning, never as
fine. Below the cards are the outbound domain list and, with the inbound relay
on, the inbound one. Each card leads to its table on **Server › Health**
(`/server/health`) — the queue card to the Outbound queue — and Health has the
readings:
readings, in the order of what stops mail first:
- **Machine** — processor (core and thread counts), memory and swap, and
per-interface network throughput and totals, read from the kernel's
@@ -383,7 +401,22 @@ readings:
memory is a warning, because both delay or kill the mail path; throughput is
only reported.
- **Processes**, **TLS certificate** (validity and expiry) and **Milter
sockets** (OpenDKIM and the journal-milter).
sockets** (OpenDKIM, the sender check and the journal-milter).
- **Sender check** — what the strict check of the `From:` line has cost
today, in the last 30 days and in the last 12 months: for how long the
panel did not answer while Postfix was running, how many connections
Postfix turned away because of it, and how many messages it deferred in
the middle (see [Who an application may send
as](#who-an-application-may-send-as)). A restart of the whole container is
not counted: mail was stopped for another reason then. A connection turned
away is refused before a message is offered, so a monitor that only
connects during an outage is counted too; a message deferred in the middle
is one the panel stopped answering for after it had begun. A time marked ≈
is at most what it shows: at that start supervisord could not say since
when Postfix ran. Days are UTC days. The card warns when
a connection was turned away or a message deferred in the last 24 hours. If these numbers do not stay
at nothing, the panel is in the way of your mail more than it should be -
report it.
- **Hostname and reverse DNS** — compares `SELFPOST_HOSTNAME` against the PTR
record the internet publishes for this server's IP (forward-confirmed
reverse DNS), see [Server-level DNS](#server-level-dns-ptrrdns); use
@@ -417,10 +450,11 @@ readings:
**Account** (`/account`, every role) is the signed-in user's own: **Profile**
(username and e-mail) and **Password**. Saving the profile does not ask for the
password; changing the password asks for the current one and is rate-limited
like a sign-in. The e-mail is the user's own address. Nothing is sent to it
yet; it is what a domain's *Use my e-mail* button fills in (see [Domain-level
DNS](#domain-level-dns-spf-dkim-dmarc)). Application SASL logins are separate
and are not changed here.
like a sign-in. The e-mail is the user's own address: it is where the panel
sends [notifications](#notifications) while *Send me notifications* is ticked
beside it, and what a domain's *Use my e-mail* button fills in (see
[Domain-level DNS](#domain-level-dns-spf-dkim-dmarc)). Application SASL logins
are separate and are not changed here.
### Settings
@@ -429,7 +463,58 @@ the whole instance: how many days the **Outbound log** keeps its rows before the
background sweep deletes them (7–365 days; takes effect on the next six-hour
prune cycle without a container restart), and, read-only, the level-1 rate
limit that comes from `.env` ([Rate limiting — level 1](#rate-limiting--level-1-ip-backstop)).
There is no panel-wide DMARC report address: each domain has its own.
There is no panel-wide DMARC report address: each domain has its own. The page
also has the switch for [notifications](#notifications) and the two DNS records
that make them arrive.
### Notifications
The panel writes to its users when something needs them. Four things are
reported, each once:
| What | When a message is sent | A second message |
|---|---|---|
| A sending domain's DKIM, SPF or DMARC record stops matching | after two hourly checks in a row found it wrong. A resolver that did not answer is not a mismatch; a missing DMARC record is advice on the domain's page, not a notification | when two checks in a row find the records right again |
| A level-2 rate limit refuses mail of a sending domain's applications | within a minute of the first refusal | at most once a day for the same domain, while it goes on |
| The TLS certificate is about to expire | 14 days before | 3 days before, if it is still the same certificate |
| A message has waited in the queue for more than three days | at the next hourly check | when no message that old is left |
A user with the **global** role is told all of it; a user with the **domain**
role is told about the DNS records and rate limits of the outbound domains
assigned to them. A message goes to the e-mail on the user's
[Account](#account) page and only while *Send me notifications* is ticked
there; a user without an e-mail gets none. **Server › Settings** has one switch
that turns notifications off for everyone. With it off nothing is checked, so
what is wrong when it is turned on again is reported then.
Notifications are the panel's own mail. They are sent from
`postmaster@<SELFPOST_HOSTNAME>`, one message per recipient, are not rows of
the Outbound log and count against no rate limit. They name the page of the
panel to open and carry no link: the panel does not know the address its
reverse proxy serves it under.
**Two DNS records make them arrive.** The server's own name is not one of your
sending domains, so it needs records of its own, shown with the values to
publish under **Server › Settings › Notification mail**:
- a **DKIM** key for the hostname, which the panel generates and signs
notifications with (`<selector>._domainkey.<hostname>`, TXT);
- an **SPF** record for the hostname: `v=spf1 a -all` (TXT).
**Server › Health** shows whether each is in place in the *Notification mail*
card. Without them notifications are still sent, and a receiver may file them
as spam or, if the parent domain publishes a strict DMARC policy, refuse them.
Mail of your sending domains depends on neither. If the hostname is itself one
of your sending domains, the DKIM record is that domain's and is already
published - and whoever administers that domain can then send mail that
looks like the panel's; a hostname that is nobody's sending domain is the
better choice.
The e-mail on Account may be at a public mail host (Gmail and the like); only
a domain's DMARC report address may not.
A relay that cannot send cannot say so: none of the four is "the relay is
down". Watch that from outside ([Container health](#container-health)).
### Users
@@ -542,10 +627,14 @@ journal-milter (level 2) is down. There is no per-IP bypass.
### Full backup and restore
**Full backup** is a self-contained project archive: `data/` (SQLite, all
domains' DKIM keys, all applications' SASL credentials, the Postfix queue,
`manifest.json` with the version that created it), plus `docker-compose.yml`,
`.env`, and `certs/` from the operator directory next to `./data`. Delivery
logs under `data/log/` are excluded. The base compose file mounts the project
domains' DKIM keys, all applications' SASL credentials, `manifest.json` with
the version that created it), plus `docker-compose.yml`, `.env`, and `certs/`
from the operator directory next to `./data`. Delivery logs under `data/log/`
are excluded, and so is the Postfix queue: the archive is taken while the
instance runs, and a copy of a queue Postfix is working on can hold a message
twice or not at all. Mail still waiting for delivery is therefore not carried
over, and a restored instance starts with an empty queue; to keep the queue,
stop the instance and use `tar` (methods 3 and 4 below). The base compose file mounts the project
directory read-only at `/selfpost-deploy` so the panel and CLI can read those
deploy files — without that mount, *Full backup* refuses with an error.
@@ -556,12 +645,25 @@ instance keeps running, what goes into the archive, and what happens on restore.
**1. Panel — while the container is running (no downtime).** *Server › Backup*
→ *Full backup*. *Encrypt with a password* is ticked; untick it for a plain
archive. Same archive as the CLI below.
archive. Same archive as the CLI below. The panel runs as its own user, not as
root, so **the private key in `certs/` has to be readable by it**: give the
file to the container's `selfpost` group, which has the fixed id 998 and holds
both the panel and Postfix —
```sh
chgrp 998 certs/privkey.pem && chmod 640 certs/privkey.pem
```
— and have whatever renews the certificate do the same (a certbot deploy hook,
for one). A key readable by root alone is enough for Postfix, but then the
panel refuses to make a backup and names the file: it does not hand over an
archive without the key.
**2. `selfpost-backup` CLI — while the container is running (no downtime).**
The panel button and this command call the same code path: a consistent SQLite
snapshot via `VACUUM INTO`, then a gzip tar of `data/` (minus diagnostics) plus
deploy files.
deploy files. `docker exec` runs the command as root, so it reads the private
key whatever its permissions.
```sh
docker exec <container> selfpost-backup > selfpost-backup.tar.gz
@@ -616,6 +718,8 @@ methods avoid it by stopping first.
| Self-contained archive | Yes | Yes | No — deploy files separate |
| SQLite consistency | `VACUUM INTO` snapshot | Safe when stopped | Safe when stopped |
| `data/log/` | Excluded | Included (optional exclude) | Included (optional exclude) |
| Postfix queue | No | Yes | Yes |
| `certs/` private key | Yes (the panel needs it readable by group 998) | Yes | No — deploy files separate |
| `data/manifest.json` | Written (version stamp) | Absent | Absent |
| Restore version check | Yes — must match image tag | No | No |
| Post-restore *Resync* | Yes, on first boot | No | No |
@@ -691,6 +795,15 @@ needed) for the new hostname or proxy, then `docker compose up -d`. The archive
carries `certs/` from the old host — re-issue certificates when the hostname or
IP changes. Set up the reverse-proxy vhost separately (not in the backup).
The inbound log and the filter lists are in the database and so in the
backup, and so are the list files under `data/antispam/`; the panel writes
those files again from the database on the first start after a restore. What
the backup does not hold is the spam filter's own side: the `antispam/`
directory with the compose fragment and the rspamd configuration. Like the
reverse-proxy fragments it is part of the installation, not of its state -
copy it from [deploy/antispam/](../deploy/antispam/) again, or the restored
instance runs with no filter.
**Restoring an encrypted (`.spbk`) backup** needs a running container to
decrypt it first — any container with the `selfpost-backup` CLI works; decryption
does not read `/data` and performs no version check. Start one normally
@@ -822,6 +935,58 @@ only sees SelfPost. Default action is fail-open (`accept`) so a down sidecar
does not block backup-MX; set `INBOUND_ANTISPAM_MILTER_ACTION=tempfail` to
defer instead.
The fragment mounts one file into the sidecar,
[deploy/antispam/rspamd/milter_headers.conf](../deploy/antispam/rspamd/milter_headers.conf):
it makes rspamd write its verdict - the action and the score with its symbols
- into two headers of every message it lets through. SelfPost records them
with the message, and the upstream receives them with the mail. Copy the
`antispam/rspamd/` directory next to the fragment; without the file accepted
mail is still recorded, with no verdict.
The same file makes rspamd **sign** each message, in a third header,
`X-SelfPost-Filter`. The filter is attached so that mail still flows when it
is down, and then a sender could write a verdict of his own into those
headers; SelfPost therefore records a verdict only next to a signature it can
recompute. The key is `data/antispam/filter.key`: SelfPost creates it on its
first start with the inbound relay on, and the sidecar reads it from the
directory it already mounts - there is nothing to configure. If Inbound › Log
shows *selfpost* for mail the filter did judge, the sidecar is running without
this file: copy `antispam/rspamd/` again and restart the sidecar. Nothing of
rspamd's needs clearing, then or after a change of the lists or of the key: it
rereads the list files by itself and reads the key anew for every message. A
restart is needed only for a change of its own configuration - the files of
`antispam/rspamd/`.
The same directory holds
[multimap.conf](../deploy/antispam/rspamd/multimap.conf), the rules for the
**allow and deny lists** kept in the panel. SelfPost does not apply the lists
itself: it writes them as six files under `data/antispam/` (`allow_ip.map`,
`allow_from.map`, `allow_domain.map` and the three `deny_` ones), the fragment
mounts that directory into the sidecar, and rspamd rereads a file within
seconds of a change - no restart. An entry is a sender address, a sender
domain (that domain, not its subdomains) or a client address or network. A
denied sender is refused outright. An allowed one is given a weight that no
spam score outweighs, so it is accepted - unless a deny entry matches the same
message: **deny wins**, whichever entries they are. **A sender address or
domain on the allow list is not proof of anything**: it is the envelope
sender, which the sending server states and nobody checks, so whoever knows an
allowed address can send as it. Allow a client address or network where you
can. The files are readable by
anyone on the host; they hold addresses, not secrets.
In the panel the lists are **Inbound › Filter lists** (global role), and what
port 25 decided about each message is **Inbound › Log**: accepted mail with the
filter's verdict, refused mail with the reply it was given. A row of the log
leads to the lists with its sender filled in. Beside each entry of a list is
how often it decided a message, counted from the log - so over the days the
log keeps - and *never* marks an entry that may have outlived its use. The
count depends on two things in
[multimap.conf](../deploy/antispam/rspamd/multimap.conf): the message of the
deny rules and the names of the allow symbols; change them and every entry
reads *never*. A user with the *domain* role
sees the log of the inbound domains assigned to them, and not the lists. How
long the log keeps a row is set under **Server › Settings**.
The panel only reports this setting; it does not change it. Every inbound
domain page has a **Spam filter** box — on or off, what happens to mail while
the filter is down, and that the filter belongs to the server, not to the
@@ -891,7 +1056,13 @@ sees the *Ingest* box — whether ingest works, the last report received, report
kept this week and parse failures. Open a domain's roll-up from the list or
from *DMARC reports* in the domain page's menu. A user with the *domain* role
sees the reports of the domains assigned to them, without the *Ingest* box.
Summaries are pruned (500 kept, 90 days max).
Summaries are pruned (500 kept for each domain, 90 days max).
A message that is not a report the relay keeps — unreadable as an aggregate
report, about a domain not sent from here, or delivered to another domain's
report address — is dropped without a retry and without a bounce. It adds one
to *parse failures*, and the reason is on the message's delivery line in the mail log
(*Server › System log*, `dropped: …`).
**Not an open relay.** The inbound smtpd offers no SASL. With DMARC ingest
alone, `check_recipient_access` permits only configured report addresses;
@@ -929,12 +1100,61 @@ to a domain:
who the application may send as, its client IP allow-list and its rate limit
are validated and saved together, or not at all, also when the application
is created. A password is shown once, on the page that answers the creation
or *New password*, and is never shown again.
or *New password*, and is never shown again. A new login is stored in lower
case - Postfix does not tell `Ops` from `ops`, so the panel does not let two
applications be told apart by case - and the page that shows the password
shows the login as it was stored.
- **Domain settings** (`…/settings`) — the *DMARC report address* (see [DMARC
reports](#dmarc-reports)), the domain's *Rate limit* (see [Rate limiting —
level 2](#rate-limiting--level-2-domain-and-application)), *Export domain*
and, for the global role, *Delete domain*, which asks once more.
#### Who an application may send as
An application sends as any address of its domain, or only as the addresses
listed for it. Since `2.4.0` that rule covers both senders a message has:
- the **envelope sender** (`MAIL FROM`), which Postfix checks;
- the **`From:` line** of the message - the one a mail program shows and the
one the DKIM signature is chosen by - which the panel checks.
An address with an extension counts as the address: an application listed as
`invoices@…` may also send as `invoices+march@…`, in the envelope and in
`From:` alike.
An address listed for an application is that application's alone: another
application of the same domain, set to any address, may not send as it -
Postfix asks who owns the address before it asks who owns the domain.
A message whose `From:` is anything else is refused with `550 5.7.1` and a
text naming the line: another of your domains, the server's own name, an
address at a domain that is not on this server, a line with two addresses,
or no `From:` at all. The line is read strictly, because two programs read
it - the check and the signer - and must not be able to read it differently:
outside a quoted display name it may hold one `@`, the address's own, and no
other name this server signs for may follow an `@` anywhere in it, a quoted
name included. A display name with a comma or brackets has to be quoted
(`"Shop, Inc." <orders@…>`), as the mail standard asks and mail libraries
do. The refusal
is permanent, so the application does not send the message again; it is a
row of the [Outbound log](#outbound-log) tagged `wrong From`, and the row's
page shows what the line said.
What this stops working when you upgrade from an earlier version:
- an application that sends with the `From:` of another of your domains -
give that domain an application of its own;
- an application with a list whose envelope address and `From:` address
differ - add the second address to the list;
- a contact form that puts its visitor's address into `From:` - put it into
`Reply-To:` and send from an address of your own, which is also what
DMARC requires.
The check is strict: while the panel does not answer, Postfix answers with a
temporary error (`451`, and `454` to the login) and applications send again
later; nothing is accepted unchecked. **Server › Health** counts how often
that happens.
*Export domain* writes a single-domain file; *Import a domain* on *Server ›
Backup* reads one back in (**global role only**) — see [Exporting and importing
a single domain](#exporting-and-importing-a-single-domain).
@@ -1008,8 +1228,8 @@ rate limits, and it takes exact addresses only — CIDR ranges are not parsed.
It is **not an authentication boundary.** The SASL login itself still succeeds;
the check runs in the journal-milter, on the same fail-open path as level 2
(below), so if the milter is unavailable the restriction is skipped rather than
enforced. Use it to narrow where an application may send from, not to contain a
(below), so if that milter does not answer the restriction is skipped rather
than enforced. Use it to narrow where an application may send from, not to contain a
leaked password — for that, regenerate the password.
**Level 2 — application** — optional override of the domain limit for one
@@ -1032,10 +1252,12 @@ hours and on demand via **Recalculate now**.
journal-milter and is deliberately fail-open: if the rate-limit lookup hits
a store error, or the connecting client's IP is not available to the
milter, level 2 is skipped and the message is accepted rather than held up.
The same applies if the milter itself is down — Postfix is configured to
accept rather than defer when it cannot reach it. Level 1 is the backstop
that keeps working even when level 2 cannot run, and it is the only one of
the two that does not depend on the panel process.
The same applies if the journal-milter itself does not answer — Postfix is
configured to accept rather than defer when it cannot reach it. (Since
`2.4.0` a panel that is down altogether defers mail anyway, through the
[sender check](#who-an-application-may-send-as) of the same process.) Level 1
is the backstop that keeps working even when level 2 cannot run, and it is
the only one of the two that does not depend on the panel process.
### Outbound log
@@ -1045,8 +1267,23 @@ rows of their domains. It refreshes its rows without resetting a filter being
chosen. A row identifies its message and nothing more — time,
sender, recipient, subject and status `queued` (accepted, not yet
delivered), `sent` (handed off successfully), `deferred` (Postfix is
retrying), `bounced` (final failure), or `rejected` (refused — for example
by a [level-2 rate limit](#rate-limiting--level-2-domain-and-application));
retrying), `bounced` (final failure), or refused before it was queued.
`bounced` is usually Postfix's word - a message it gave up on at the end of
its queue lifetime is `bounced` too, since Postfix returns it to the sender;
it is the panel's when Postfix no longer holds a message and `mail.log` never
said what became of it (the log rotated
away while the panel was down, or was not readable) - the panel then closes
the row as a failure rather than show a delivery it cannot evidence, and
counts that it did. A
refused row says why in its tag: `rate-limited` (a [level-2 rate
limit](#rate-limiting--level-2-domain-and-application)), `wrong client` (the
application's client IP list) or `wrong From` (see [Who an application may
send as](#who-an-application-may-send-as)); a row refused before `2.4.0`
says `rejected`. A refused message's page shows what Postfix wrote about the
refusal: for `wrong From` the lines of the queue id the message had, for the
other two the one `NOQUEUE` line; a row refused before `2.5.1` says the line
was not kept, and one the panel wrote from `mail.log` after a restart - it
had stopped before writing the refusal itself - says so;
*Details* opens that row's own page (`/outbound/log/{id}`). That page carries
the sending domain, the application it was submitted under, the Postfix
queue id and the journal id, beside the message's history — when it was
@@ -1056,8 +1293,10 @@ backoff cap, queue lifetime), the same numbers the Outbound queue shows; users
with the *domain* role see them here because they cannot open the queue. Under
both sit the `mail.log` lines for its queue id: the connection to the
receiving server, the server's reply, and the status that reply was filed
as. Rows outlive `mail.log`, so an older message's lines may have rotated
away; the page says so. Retention is set under **Server › Settings** (global
as. The lines are kept in the database with the row, so they stay on the
page after `mail.log` has rotated and are deleted together with the row; a
message accepted before `2.2.0` shows them only while they are still in the
current file, and the page says so when there are none. Retention is set under **Server › Settings** (global
role); `SEND_LOG_RETENTION_DAYS` in `.env` is only the initial default until it
is changed there.
@@ -1070,7 +1309,12 @@ applications' **working** SASL passwords, and configured **rate limits**
instance without regenerating anything, so DNS (the DKIM TXT record)
doesn't need to change. Unlike a full restore (see [Full backup and
restore](#full-backup-and-restore)), this works across different
hostnames/instances. *Import* is for the global role only; *export* is
hostnames/instances. The file is checked like the forms before anything is
created - the report address, every rate limit, every client address - and
refused with the form's own message, naming the field and the application.
An application login keeps the case it had (new logins are created in lower
case), and one that differs from a login of this server only in case is
refused. *Import* is for the global role only; *export* is
available to any user who can access the domain, **including a user with the
*domain* role** for a domain assigned to them — so that user can walk away with
the domain's working SASL passwords, in the clear if they untick the encryption
+184
View File
@@ -0,0 +1,184 @@
# Plan: delivery-log-storage
**Status:** shipped in `2.2.0` (2026-10-12); agreed the same day (owner: option 1 of the roadmap item, "store the lines")
**Date:** 2026-10-12
**Version:** `2.2.0` MINOR (one DDL migration; a 2.1.0 data directory boots unchanged).
---
## Goal
A message's page (`/outbound/log/{id}`, mockup `out-message`) shows what
Postfix wrote about the message for as long as its row is in the send log.
Today the *Delivery log* box is filled by a search of the current `mail.log`
on every page view, so it is empty for anything sent before the last daily
rotation, while the row itself lives for ninety days.
## Context (as-built)
- The log tailer ([internal/logtail](../../internal/logtail/logtail.go))
already reads every `mail.log` line once, from a persisted offset, to
advance delivery statuses.
- `logtail.QueueLines` searches the last 4 MB of the current file for one
queue id; `Handlers.deliveryLog` calls it per page view.
- A message with several recipients is several send-log rows under one queue
id; the page of each shows every line of the message, as the search does
today.
## Decisions
### What is stored
A `mail.log` line whose head is a queue id - after the timestamp, the host
name and `program[pid]:` - goes into a new table with that id, the line's own
time (`lineTime`, the same reading the inbound journal uses) and the line as
written, cut to 4096 bytes on a character boundary. The head is matched from
the start of the line: the tail is the far end's words. `NOQUEUE` and
`warning` are not queue ids - the pattern takes Postfix's short form
(hexadecimal) and its long form (no vowels) and nothing else.
The tailer does not ask whether the id belongs to a send-log row before it
writes. The first lines of a message (`client=`, `message-id=`) are logged
before the journal milter writes the row at end-of-message, so such a check
would drop exactly them.
**Mail that came in on port 25 is left out** (security review, 2026-10-12). A
queue id that `postfix/inbound/smtpd` opened is a stranger's message; it never
gets a send-log row, and what a stranger can make Postfix log - his own
`message-id`, up to the cut - is not stored to be pruned later. The tailer
remembers such ids (4096 at most) until qmgr's `removed` or until another
listener opens a message under the id. What is still stored without a row is
little: the bounces Postfix generates, and port-25 mail met in mid-flight
after a restart.
Lines are written in one transaction per pass over the log, 500 at most
(technical review): a commit per line is a disk sync per line on the one
connection the journal milter writes through.
### Which lines are a row's
A line is shown on a row's page when the queue id is the same **and** the
line's time lies between one hour before the row was written and the queue
lifetime (the larger of `maximal_queue_lifetime` and `bounce_queue_lifetime`,
as `postconf` reports them) plus a day after it. Postfix reuses a short queue
id once the message has left the queue - the id is the microsecond of arrival
and the inode of the queue file - so a row that lives ninety days can meet a
later message under its id. The window does not rule that out, it limits it
to the days the row's own message could still be in the queue; this is the
bound the roadmap item asked for. The search of the current file that the
page did before had the same exposure for a day. `enable_long_queue_ids`
would end it and is a change to the mail path, not part of this item.
### Retention
Lines live as long as a send-log row with their queue id does. One rule covers
both ends: a line older than an hour whose queue id has no send-log row is
deleted. That removes the lines of a row the retention sweep has just deleted,
and the lines of messages that never had a row. It runs at start and after
every send-log sweep, six-hourly. It asks the send log once per queue id; at a
million lines that is still a scan of the table, of the order of a second,
during which the one connection is taken.
There is no cap of its own: with port-25 mail left out, the table is bounded
by what authenticated applications send, which level 1 and level 2 limit.
### After an unclean stop
The tailer saves its place every few seconds, so after a crash it reads some
lines twice. While it is catching up at start a line is written only if the
same line is not there already - the rule the inbound journal follows.
### What is not kept
- Lines the tailer never reads are not stored: the tail of a file that was
rotated away while the panel was down, as with delivery statuses
(architecture.md § Log tailer).
- A message in the queue across the upgrade to 2.2.0 has only its later lines
stored, and the page then shows those alone: the search of the file runs
only for a row with nothing stored. The only installation is the owner's
test server; no merge of the two sources is built for one moment of it.
### The page
The page reads the table: the latest 200 lines of the row, oldest first. When
the table has none, it still searches the current `mail.log` as before - that
is what shows a message accepted before 2.2.0 while its lines are in today's
file. Lines in rotated files are not imported: the only installation is the
owner's test server.
Nothing in the page's layout changes; the `out-message` mockup already draws
the box as a `log_pane`. Two texts change, mockup first: the *Outbound log*
help topic (it says an old message may show no lines) and the note shown when
there are none.
### Size
Six to ten lines of about 200 bytes per message: 1-2 KB next to a row of a few
hundred bytes. The table becomes the larger part of what the send log costs in
`/data` and in a backup; the retention setting bounds both together.
## Schema (migration `0003_send_log_lines.sql`)
```sql
CREATE TABLE send_log_lines (
id INTEGER PRIMARY KEY AUTOINCREMENT,
queue_id TEXT NOT NULL,
logged_at TEXT NOT NULL,
line TEXT NOT NULL
);
CREATE INDEX idx_send_log_lines_queue_id ON send_log_lines (queue_id, logged_at);
```
No foreign key: a line is written before its row exists, and one queue id is
several rows.
## Scope
In: the table, the tailer writing and pruning it, the page reading it, the two
texts, docs.
Out: the inbound journal (it decides for itself, roadmap § delivery-log-storage);
importing rotated files; `enable_long_queue_ids`; any change to `mail.log`
rotation or to the Server › Mail log page.
## Security
- The lines carry the far end's text (replies, HELO names). They are shown
today from the file through the same escaping; storing them changes where
they are read from, not what is shown or to whom. A domain administrator
sees the lines of messages of their own domains, as now.
- They now enter the database and so a full backup, which `log/` does not:
submitting clients' addresses and SASL logins, message ids, receivers'
replies. [security.md](../security.md) says so.
- A line is matched from its head, so a reply cannot make itself a line of
another message; Postfix does not log raw line breaks, and `postlogd` is the
file's only writer.
- A stranger on port 25 does not write to the table (above).
## Done when
A message sent before the last rotation shows its lines on its page; the lines
disappear with the row; the page's wording no longer blames rotation for an
ordinary case.
## Model routing (this plan)
| Role | Model |
|------|-------|
| All implementation (code, tests, docs) | **Opus** |
| Technical review (reviewer ≠ author: a separate read-only agent) | **Opus** |
| Security review | **Fable** |
## Implementation checklist
Target version cut: **`2.2.0`** (MINOR). One commit per step;
[development.md](../development.md) § Plan checklists.
- [x] Migration `0003_send_log_lines.sql`, store methods (insert, insert once, lines of a row, prune) and their tests — **Opus**
- [x] Log tailer: store queue-id lines, write once while catching up, prune with the send-log sweep — **Opus** (hourly as first built; six-hourly after the reviews)
- [x] The two texts, mockup first: the *Outbound log* help topic and the empty note (`design:` commit) — **Opus** (the help topic is in the mockups; the empty note is not drawn there - the `out-message` mockup shows a message with lines - and changes with the page)
- [x] Message page reads the table, falls back to the current `mail.log`; help text in `help.go` — **Opus**
- [x] [architecture.md](../architecture.md), [guide.md](../guide.md), [schema-migrations.md](../schema-migrations.md) — **Opus** (schema-migrations.md went in with the migration)
- [x] `go vet`, `go test`, e2e: a message's page still lists its lines after a forced rotation — **Opus** (on the test server: vet ok, 22 packages ok, e2e ok with the new step `delivery_log_survives_rotation`)
- [x] Technical review — **Opus** (2026-10-12; no Critical or High. Fixed: one transaction per pass instead of a commit per line, the prune asks the send log once per queue id and runs six-hourly, the cut keeps characters whole, tests for what the comments claimed. Left as found and written down under *What is not kept*: lines the tailer never read, and a message in the queue across the upgrade)
- [x] Security review — **Fable** (2026-10-12; no Critical or High. Fixed: port-25 mail is not stored, so a stranger cannot grow the table; [security.md](../security.md) states what a backup now carries. The window against queue-id reuse is described as the limit it is)
+260
View File
@@ -0,0 +1,260 @@
# Plan: from-header-binding
**Status:** shipped in `2.4.0` (2026-10-12); agreed the same day - the design questions were closed with the owner one at a time and he took it into work ("Да")
**Date:** 2026-10-12
**Version:** `2.4.0` MINOR (one migration; it refuses mail that was accepted before).
---
## Goal
An application sends only as itself - in the `From:` line a mail program
shows and OpenDKIM chooses the key by, not only in the envelope.
Until now Postfix held the envelope sender against the SASL login and nothing
held the header. An application of one sending domain could write another
registered domain, or the server's own name, into `From:` and have the message
signed with that name's key (security review of panel-notifications,
2026-10-12).
## Decisions (owner, 2026-10-12)
### The rule
**`From:` is held to what the envelope is held to.** An application set to
any address of its domain may write any address of that domain; one with a
list of addresses, only those. Anything else is refused: another registered
domain, the server's name, an address at a domain this server knows nothing
of, no `From:` at all.
I proposed the narrower rule - refuse only a `From:` that names a domain this
server signs for, and let a stranger's address through unsigned as before -
and, for an application with a list, to check the domain alone. The owner
chose the wider one both times. What it stops working: a form that puts its
visitor's address into `From:`, and an application with a list whose envelope
address and header address differ, until the second is added to the list.
When the hostname is itself a sending domain its applications write
`…@<hostname>` by right, as the guide already says.
### Where it is checked, and what happens when it cannot be
**The panel checks, and the check is strict:** while the panel does not
answer, Postfix replies with a temporary error and accepts nothing unchecked.
I first proposed the opposite (fail-open, as the journal and the level-2 limit
are) and then a small process of its own. The owner's objection to the first:
a check that is absent while the panel is down does not close the hole, and
the panel only has to be slow - 15 seconds - for Postfix to go on without it.
His answer to the second: not yet; measure first.
So the check is a role of the panel process on a socket of its own, attached
with `default_action=tempfail`. The journal milter keeps its socket and stays
fail-open: a slow database still does not defer mail. The check reads no
database - it reads the file of sender bindings Postfix itself reads for the
envelope, so the two cannot disagree - and moving it into a process of its own
later is a change of where it is started.
### Statistics, to decide whether the check needs its own process
Per day, kept a year:
- **for how long the panel did not answer while Postfix was running.** The
panel marks itself alive every few seconds; at start the gap since the last
mark is its downtime, counted when Postfix was not restarted within it (a
rollout or a reboot stops mail for another reason);
- **how many messages were deferred because the check did not answer**, read
from `mail.log`. This one also sees a panel that is running and not
answering, which the mark does not.
Shown on **Server › Health** in a card of its own, *Sender check*: today, the
last 30 days, the last 12 months. The card warns when a message was deferred
in the last 24 hours; downtime that deferred nothing warns of nothing.
### What a refusal looks like
- **To the application:** `550 5.7.1`, permanent, naming the address. A
level-2 limit answers 4xx because the message will pass in an hour; a wrong
`From:` will not, and a 4xx would have the application send it again for
days.
- **In the Outbound log:** a row with the reason in the status tag -
`wrong From`, beside `rate-limited` and the tag of a refused client address.
The built panel wrote `rejected` for all of them; it follows the mockup now.
(The mockup's status filter is not built - the log filters by domain and
application - so there is nothing to keep in step with the tags.)
- **On the message's page:** a fact *From: line* with the address written in
the header, and a History step that says which address was refused, for
which application, what that application may write, and that it was told
`550 5.7.1`.
### Health, re-laid
The owner on the first picture of the new card: too many holes on the screen,
the numbers should follow importance, and the boxes should read left to right
and top to bottom - with stacked columns the third box was below the fold and
the fourth was not.
- Order: 01 Processes, 02 Milter sockets, 03 Sender check, 04 TLS
certificate, 05 Hostname and reverse DNS, 06 Notification mail, 07 Machine,
08 Configuration.
- Layout `cards` (new, on the components page): two stacks, odd numbers on the
left and even on the right, each closed up, so a short box leaves no gap
beside a tall one and the eye still goes 01, 02, 03. On a phone the boxes
come in the order of their numbers.
## Settled while building
- **The header is read twice.** One `From:` field holding one address, by
Go's parser; then every `@name` of the raw value is held against OpenDKIM's
own `SigningTable`. The first build took the names for the second look from
the sender bindings, and the e2e stand let a line through that named a
domain with a key and no application yet: a domain is a signing name from
the moment it is added, not from its first application.
- **What Postfix writes when the check does not answer** (seen on the stand):
a warning naming the socket, then `milter-reject: CONNECT … 451 4.7.1
Service unavailable` and the same for `EHLO`; the client is told `454 4.3.0
Try again later` at `AUTH` and leaves. No `MAIL` is ever offered, so what
is counted is the connection turned away, once. The journal milter's
level-2 refusal is written in the same words, which is why a reject line
counts only after the warning.
- **When Postfix started** is asked of supervisord (`supervisorctl status`,
the uptime of `postfix`).
- **The reply** is a milter reply code, `550 5.7.1 From: <line> - <why>`; what
the client wrote is reduced to printable ASCII and cut first.
- **The Delivery log box stays on a refused message's page**, with the one
sentence it already had for a message that never reached the queue; the
picture shown to the owner had left the box out.
## To settle while building (as first written)
- How the header is read so that the check and OpenDKIM cannot disagree: one
`From:` field, one address, and a cruder second look at the raw value.
- What Postfix writes in `mail.log` when a strict milter does not answer, and
how a line is told to be this milter's and not OpenDKIM's.
- How the panel learns when Postfix started.
- Schema: the reason of a refusal and the header address on a send-log row;
the table of days.
- That a refusal for `From:` is not taken for a refusal under a limit by the
notifier.
## Implementation checklist
- [x] Mockups: Health card and layout, the log row, help topic; accepted by the owner 2026-10-12 (`design:` commit)
- [x] Migration and store: refusal reason and header address, days of the sender check (`0006_sender_check.sql`)
- [x] The check: a role of the panel on its own socket, bindings read from Postfix's file; Postfix attaches it strictly; start order
- [x] Statistics: the alive mark and downtime, deferred mail from `mail.log`
- [x] Panel: Health (card, sockets row, layout), Outbound log tags, the message's page, help
- [x] guide.md, architecture.md, security.md (the accepted risk goes), schema-migrations.md
- [x] `go vet`, `go test`, e2e: refused for another domain's `From:`, for the hostname's and for an address off the list; accepted for its own (on the test server: vet ok, 23 packages ok, e2e ok - `sender_check_holds_the_from_line`, `panel_down_defers_mail`)
- [x] Technical review — **Opus** (2026-10-12; one Critical, fixed: a folded `From:` line - what a mail library makes of a long display name - was refused for good. Also fixed: a display name in a charset Go does not convert was refused; the statistics ran before the check served, and a hung `supervisorctl` would have kept it from serving; the count of deferred mail missed a check that stops answering in the middle of a message, and is now taken at Postfix's warning; a card whose figures could not be read said ok; a killed container's stale socket satisfied the wrapper's wait; outside quoted strings the line may now hold one `@` only. Three were first left as found and fixed the same evening at the owner's question: downtime is cut at the UTC midnights it crossed; the row of a message with the null envelope sender takes the domain the login is bound to; a log read again after an unclean stop does not count again what it counted before)
- [x] Security review — **Fable** (2026-10-12; no Critical or High. The same folded line, as its one Medium. Fixed: another sender's name in an encoded display name passed the second look; a client could arm the deferred counter by writing the socket's name where Postfix quotes it in a warning; a refused message wrote a row per recipient, a thousand at most - now one. Recorded: what a display name looks like to a person is not promised. The review also found the check stricter than Postfix for an address with an extension (`invoices+x@`, allowed in the envelope and refused in the header); the owner asked for it to work, and the check now makes Postfix's second try, with `recipient_delimiter` set explicitly. Confirmed from the code: no path to the signer around the check, and every failure of the check defers)
## Not done
Written at the owner's word when he had the version cut ("Запиши все
недоделки в план", 2026-10-12). Nothing here was promised and dropped in
silence; each line says what is missing and what it would take.
### Decided to wait
- **A process of its own for the check.** The owner: measure first. Server ›
Health now has the numbers; the decision is his when they have had time to
say something. Moving the role out is a change of where it is started -
a subcommand of the panel binary and a `[program:]` in supervisord - and
the downtime figure then needs a new meaning.
### Shown to the owner one way and built another
- **The Delivery log box is on a refused message's page.** The picture he
approved had left it out; the built page keeps it, with one sentence. And
the sentence is less than the truth for a `From:` refusal: cleanup did
write a line about it under a queue id (`milter-reject: END-OF-MESSAGE ...
5.7.1 From: ...`), but the check answers before the queue id reaches it, so
the row keeps none and the page cannot show that line. **Decided
2026-10-13 (owner):** the box is required - he had overlooked its absence
from the picture, it was never a decision to drop it. The tailer attaches
the line to the row, for all three kinds of refusal; a step of
[audit-2026-10-12](../audit-2026-10-12.md) stage 2b.
- **The Outbound log has no status filter.** The mockup draws one; the built
log filters by domain and application. With three kinds of refusal now
told apart by tag, a filter by status and reason would be worth having.
**Decided 2026-10-13 (owner):** a filter by status, without the split by
reason, and a filter by day as the Inbound log has; a step of [audit-2026-10-12](../audit-2026-10-12.md) stage 4.
- **"Messages deferred" counts connections Postfix turned away.** Postfix
refuses from the connection on, before a message is offered, so a monitor
that only connects during an outage is counted. The guide says so; the
card's column head and the help topic still say "messages". **Decided
2026-10-13 (owner):** the column is renamed *Connections turned away*; a
step of [audit-2026-10-12](../audit-2026-10-12.md) stage 2b.
### Known and small
- **"Today" on the card is the UTC day**, and the card does not say so.
**Decided 2026-10-13 (owner):** not a label - the local day; it became the
roadmap item `local-time`, for the whole panel.
- **A deferral in the very second of the last one counted before an unclean
stop is passed over** at the restart: `mail.log` stamps whole seconds.
Chosen over counting a stretch of the log twice. **Decided 2026-10-14
(owner):** counted exactly - the counter keeps a place in the log, saved
with the count; a step of [audit-2026-10-12](../audit-2026-10-12.md)
stage 2b.
- **Deferrals are lost when `mail.log` is rotated while the panel is down**:
the tailer starts on the new file. **Decided 2026-10-13 (owner):** the
tailer reads the rotated file to its end first - not the deferrals alone,
everything the panel takes from the log; a step of
[audit-2026-10-12](../audit-2026-10-12.md) stage 2b.
- **A start at which `supervisorctl` fails counts no downtime**; it is
logged. **Decided 2026-10-13 (owner):** the question is repeated, and
without an answer the downtime is counted from the alive mark and the day
marked as estimated; a step of
[audit-2026-10-12](../audit-2026-10-12.md) stage 2b.
- **Logins are compared as written**, Postfix compares them without regard
to case. The panel writes logins one way, so nothing differs today.
- **The phone order of the `cards` layout is written for four boxes to a
stack** and takes every child of a stack for a box (said at the partial).
The rule the technical review suggested instead - the order from the
box's own number - changes the component and would be drawn first.
**Decided 2026-10-13 (owner):** that rule is built; a step of
[audit-2026-10-12](../audit-2026-10-12.md) stage 4.
- **Health's socket check looks at the file, not at who listens**: a socket
nobody answers on would read "ok". The start now removes stale sockets,
which was the one way to get such a file. **Decided 2026-10-13 (owner):**
the check connects to the socket; a step of
[audit-2026-10-12](../audit-2026-10-12.md) stage 4.
- **The kit page's table of status tags does not list the new ones**
(`wrong From`, `wrong client`, `rate-limited`), and `rate-limited` is grey
beside two red tags - as the mockup has it, not as a decision.
- **Refusal rows still in the buffer when the panel stops are not written**;
the refusals themselves were answered. **Decided 2026-10-13 (owner):**
the tailer writes such a row from the refusal's line in `mail.log`; a step
of [audit-2026-10-12](../audit-2026-10-12.md) stage 2b.
### Accepted, in security.md
- **The panel's downtime is mail's.** The price of the strict check.
- **What a display name looks like to a person is not promised.** A name of
another sender written without an `@` the scan can find passes, signed
with the application's own key.
- **A malformed packet on the check's socket ends the panel** (the milter
library panics on an empty one). The socket is reachable only inside the
container, by the group Postfix is in; the journal milter always had this,
and what is new is that the price is deferred mail.
### Not verified
- **OpenDKIM's reading of a `From:` line** was reasoned about, not run: its
parser is not in this repository. The two rules that do not depend on it -
no other signing name after any `@`, one `@` outside quotes - are what the
claim rests on. A test that feeds the stand's OpenDKIM the lines the check
passes and reads which key it signs with would settle it. **Decided
2026-10-13 (owner):** that test is built; a step of
[audit-2026-10-12](../audit-2026-10-12.md) stage 2.
- **The pages were not looked at in a browser on the deployed panel**: I
cannot sign in there. They were rendered from the fixtures and
screenshotted, the polling and the phone order read from the code and
exercised by e2e only as far as the Health numbers.
- **No refusal has been seen on the test server itself**, only on the e2e
stand: there is no application there I can send as.
- **A check that stops answering in the middle of a message** is counted
from a warning composed from Postfix's source; the stand only produced the
case of nobody listening.
+90 -21
View File
@@ -1,6 +1,6 @@
# Plan: inbound-antispam-panel
**Status:** agreed
**Status:** shipped in `2.1.0` (2026-10-12)
**Date:** 2026-08-19
**Version:** `2.1.0` MINOR (opt-in; no change to outbound-only or inbound-without-filter paths). Was `1.10.0`; on 2026-10-09 the owner put the panel redesign first, so this ships on the 2.0 baseline and its screens are built from the redesign's component kit ([panel-redesign.md](panel-redesign.md) § Component kit).
@@ -34,8 +34,8 @@ Lists let the operator correct false positives/negatives (e.g. always allow
## Quarantine
**Not in this plan.** Held-mail review/release is tracked separately as roadmap
candidate [inbound-quarantine](inbound-quarantine.md). The journal may still
**Not in this plan.** Held-mail review/release is an idea kept outside the roadmap,
[inbound-quarantine](inbound-quarantine.md). The journal may still
record a `quarantine` **decision label** when the external filter reports it,
without SelfPost storing the message.
@@ -57,8 +57,14 @@ smtpd_milters = { antispam, inbound-journal }
```
Antispam runs first; inbound-journal runs at end-of-message on the **accept**
path and reads rspamd-added headers (`X-Rspamd-Action`, `X-Rspamd-Score`, symbol
summary when present).
path and reads rspamd-added headers (`X-Rspamd-Action`, and `X-Spamd-Result`
for the score and symbols - rspamd has no `X-Rspamd-Score`; the sidecar adds
both through `deploy/antispam/rspamd/milter_headers.conf`). As built they are
read only next to rspamd's signature of the message, `X-SelfPost-Filter`
(owner, 2026-10-12: "А rspamd не может дописывать какой-то заголовок, по
которому мы будем знать, что это заголовки нашего спам фильтра?") - an HMAC
of the queue id, since a fixed mark could be written by a sender as easily as
the verdict itself.
**Antispam rejects** do not reach end-of-message on the journal milter. v1 also
tails `mail.log` for inbound `postfix/smtpd` **`milter-reject`** / **`reject`**
@@ -73,7 +79,9 @@ antispam hook). Without an external filter:
- **accept** rows — message relayed (engine `selfpost`, detail empty);
- **reject** / **tempfail** rows — from the mail.log tailer (Postfix policy,
size limit, unknown recipient, etc.).
unknown recipient, a sender domain that does not exist. As built, a refusal
whose log line names no recipient - the size limit answered at MAIL FROM -
and the per-client rate limit, which Postfix logs as a warning, make no row).
Panel copy explains that rspamd symbols appear only when the antispam milter is
configured.
@@ -83,6 +91,25 @@ configured.
**Global administrator only** for journal and lists — same as inbound relay. No
domain-admin access in v1.
*As built (2026-10-11):* the lists are the global role's; the **log follows
the delegation of Inbound** that 2.0 introduced after this was written - a
user with the `domain` role sees the rows of the inbound domains assigned to
them and nothing else, without the *Allow / Deny sender* links. That is what
the accepted `in-log` mockup draws (no "global role" in its head). The owner
was told and can turn it back into a global-only page.
**Inbound › Log accepted by the owner on 2026-10-12** ("Этот экран
утверждаем"), from screenshots of the page built from its fixture, after his
changes to it: the sender column is headed *From*; *Why* is not a column but
a sheet over the page, opened by the decision tag, holding a centred table of
who decided, the verdict and symbols or the reply, and the subject whole
(kept on the sheet at his word: the table clips it); a message Postfix itself
refused offers no *Deny sender*. Inbound › Filter lists was redrawn at his word
the same day - kind in a column of its own, the note on the entry's sheet,
matches counted (§ Filter list usage) - and **accepted by him on 2026-10-12**
("Да"), after a deny entry he added in the panel refused a test message with
the list's reply and the journal recorded it.
### Retention
Mirror send-log retention:
@@ -101,13 +128,53 @@ Mirror send-log retention:
| Entry types | sender address, sender domain (`@domain` suffix), client IP / CIDR |
| Precedence | **deny overrides allow** |
| Scope | instance-wide (not per inbound domain in v1) |
| Where applied | **rspamd map files** under `/data/antispam/` (allow.map, deny.map); documented compose fragment mounts them into the sidecar |
| Where applied | **rspamd map files** under `/data/antispam/`, one per list and entry type (`allow_ip.map`, `allow_from.map`, `allow_domain.map`, `deny_…`) - an IP map and a sender map are different map types to rspamd; documented compose fragment mounts them into the sidecar. Deny is a prefilter reject, allow a weight of -100 (as built: two prefilters do not give "deny overrides allow") |
| SelfPost pre-check | **out of v1** — lists live in rspamd only |
| Edit access | global administrator only |
| When inactive | lists editable in the panel but UI warns that sync applies only when `INBOUND_ANTISPAM_MILTER` is set; `postfix reload` / rspamd reload documented in guide |
List change → atomic map rewrite (same pattern as Postfix maps) → rspamd reload
via documented operator step or sidecar `SIGHUP` in the compose fragment.
List change → atomic map rewrite (same pattern as Postfix maps). No reload:
rspamd rereads a changed map file by itself (about ten seconds on 3.11).
### Filter list usage (owner, 2026-10-12)
Each entry shows how often it decided a message (*Matched*, or *never*) and,
on its sheet, when it last did - so that entries nothing matches any more can
be removed. **The count is read off the journal on every view of the page;
nothing is stored for it.** A journal row counts for an entry when the list
decided the message - a rejected row whose reply is the deny rule's message,
an accepted row whose verdict carries the allow rule's symbol
(`SELFPOST_ALLOW_IP` / `_FROM` / `_DOMAIN`, which the journal milter always
keeps) - and the entry fits the row's sender or client address the way rspamd
matches it (`inbound.Lists.Usage`).
What follows from counting this way, and is accepted:
- the count reaches back as far as the journal does: its retention (90 days by
default), which is what the sheet says;
- a row counts for every fitting entry of the deciding list, since the journal
does not say which of two the filter met first;
- a message refused at end-of-message is one row whatever its recipient count,
so a count is of messages, roughly.
**When to move to a counter kept with each entry** (a migration adding hits
and last-hit to `inbound_spam_list`, bumped where the journal row is written;
the owner asked for the condition to be written down). Any one of:
1. **The journal is cut by its 10 000-row cap and not by age.** Then it holds
fewer days than its retention, the sheet's "in the last 90 days" is no
longer true, and *never* stops being evidence that an entry is stale. The
sign: the log tailer's prune line keeps reporting rows removed while the
oldest row of Inbound › Log is younger than the retention.
2. **The retention is set short** (towards its minimum of 7 days) for reasons
of its own, privacy or disk. A week without a match does not make an entry
stale.
3. **Counts are wanted that outlive the journal** - since the entry was added.
4. **The page is slow.** A view reads every journal row a list decided and
tries it against every entry. With the cap that is at most 10 000 rows
times the number of entries; past a few hundred entries, measure it.
Until then no migration is spent on it.
### Activation
@@ -200,7 +267,9 @@ flowchart LR
## Security
- Same validation whitelists as inbound relay ([security.md](../security.md)).
- Journal rows contain PII — retention and global-admin-only access.
- Journal rows contain PII — retention, and access by reach: the global role
reads all of it, a `domain` user the rows of assigned inbound domains (as
built; see § RBAC).
- Inbound journal milter fail-open must not block mail (backup-MX role).
- List map writes injection-safe; deny wins over allow.
@@ -243,14 +312,14 @@ Target version cut: **`2.1.0`** (MINOR), after `2.0.0`. One commit per step;
[development.md](../development.md) § Plan checklists.
- [x] Agree journal fields, retention, and RBAC (this plan § Decisions) — **Composer**
- [ ] Migration `0002_inbound_spam_log.sql` on the 2.0 baseline — **Composer**
- [ ] Inbound journal-milter + Postfix wiring — **Composer**
- [ ] mail.log tailer: inbound reject rows — **Composer**
- [ ] List CRUD + validation + atomic rspamd map sync — **Composer**
- [ ] Panel: Inbound › Log and Inbound › Filter lists from the kit partials, mockups `in-log` / `in-filter-lists` (add client IP and engine columns mockup-first), evidence pairs — **Composer**
- [ ] Backup includes `/data/antispam/` — **Composer**
- [ ] Unit + handler tests — **Composer**
- [ ] [guide.md](../guide.md), [architecture.md](../architecture.md) — **Composer**
- [ ] `go vet`, `go test`, e2e if applicable — **Composer**
- [ ] Technical review of inbound antispam changes — **Opus**
- [ ] Security review inbound antispam path — **Fable**
- [x] Migration `0002_inbound_spam_log.sql` on the 2.0 baseline — **Composer**
- [x] Inbound journal-milter + Postfix wiring — **Composer**
- [x] mail.log tailer: inbound reject rows — **Composer**
- [x] List CRUD + validation + atomic rspamd map sync — **Composer**
- [x] Panel: Inbound › Log and Inbound › Filter lists from the kit partials, mockups `in-log` / `in-filter-lists` (add client IP and engine columns mockup-first), evidence pairs — **Composer**
- [x] Backup includes `/data/antispam/` — **Composer** (it always did: a backup walks all of `/data`; now held by a test, and the guide says the `antispam/` fragment directory is not in it)
- [x] Unit + handler tests — **Composer** (written with each step: store, milter, tailer, lists and their usage, view fixtures held to the outlines, handlers for both roles)
- [x] [guide.md](../guide.md), [architecture.md](../architecture.md) — **Composer** (kept up with each step; [security.md](../security.md) follows the security review)
- [x] `go vet`, `go test`, e2e if applicable — **Composer** (on the test server for every step; last on `7192c38`: vet ok, 22 packages ok, e2e ok, which relays, refuses, reads the log page and edits a list through the panel)
- [x] Technical review of inbound antispam changes — **Opus** (2026-10-12; findings fixed: client address of a connection's later messages, the lists' writer under a lock, the tailer's queue-id memory, hold rows written twice, the cap by time; and, outside the feature, the level-2 limit and client-address list skipped after a connection's first message)
- [x] Security review inbound antispam path — **Fable** (2026-10-12; no Critical or High; HELO-forged journal rows and the six-hourly row cap fixed; a sender's own verdict while the filter is down closed by the filter's signature, the owner's idea; accepted risks recorded in [security.md](../security.md))
+1 -1
View File
@@ -1,6 +1,6 @@
# Plan: inbound-quarantine
**Status:** candidate — design TBD; needs explicit agreement before any code
**Status:** an idea to think about, not on the roadmap (owner, 2026-10-12) — design TBD; needs explicit agreement before any code
**Version:** TBD (`1.x` MINOR if opt-in and backward-compatible; may warrant `2.x`
if it changes inbound storage or backup contract).
+119
View File
@@ -0,0 +1,119 @@
# Plan: panel-notifications
**Status:** shipped in `2.3.0` (2026-10-12); agreed the same day - the design questions were closed with the owner and he took it into work ("Делаем")
**Date:** 2026-10-12
**Version:** `2.3.0` MINOR (one migration; opt-out by a switch).
---
## Goal
The panel writes to its administrators when something needs them, instead of
waiting to be opened.
## Decisions (owner, 2026-10-12)
### Events of the first version
Four, each of them something the panel can already tell and a person has to
act on:
| Event | Who is told |
|---|---|
| A sending domain's DNS record stopped matching (SPF, DKIM, DMARC) | global users, and `domain` users the domain is assigned to |
| A level-2 rate limit refused mail | the same, for the domain of the limit |
| The TLS certificate is about to expire | global users |
| A message has been in the queue for more than **three days** | global users |
Three days, not one (owner: "one day is not that much"): Postfix keeps a
message five, so two are left to act before it bounces. The threshold is a
constant, not a setting.
Left for later: a process or milter socket down (a relay in that state often
cannot send), an inbound upstream unreachable, DMARC failures appearing
(noisy). Mail held in quarantine is not an event: quarantine is an idea
outside the roadmap.
### Sender and path
From `postmaster@<SELFPOST_HOSTNAME>`, submitted locally. Not from one of the
sending domains (owner): with several domains the choice is arbitrary, and a
message about the certificate has no business coming from a domain.
These messages have to arrive (owner: "we need the best deliverability for
them"), so the server's own name gets what a sending domain has:
- a **DKIM key for the hostname**, generated by the panel, with which
notifications are signed - no domain has to be chosen for it. An SPF record
alone would pass DMARC, but not after the user's mailbox forwards the
message on, and large receivers treat unsigned mail more strictly;
- an **SPF record** for the hostname, `v=spf1 a -all`.
The records themselves are under **Server › Settings** (owner: "the DNS
records go to Settings, Health has only the status"), in a box of their own
drawn with the record component of a domain's DNS page: host, value to
publish, copy buttons, state. **Server › Health** has a card of its own,
*Notification mail*, with the state of each record and a link to Settings.
A DMARC record for the hostname is not asked for: the name inherits its
parent domain's policy, and with SPF and DKIM aligned it passes any.
Without the records notifications are still sent, and the card warns.
Notifications are the panel's own mail: they are not rows of the Outbound log
and count against no level-2 limit.
A relay that cannot send cannot say so; none of the four events is "the relay
is down", and that is why.
### Who chooses what
- One switch for the instance under **Server › Settings**.
- One checkbox per user under **Account**, "send me notifications" - on when
the profile has an e-mail. No choice per event: four events, none of them
noise. Per-event choice comes when there are more and some are.
- A user without an e-mail gets nothing.
### When a message is sent
A background check once an hour for DNS, certificate and queue.
| Event | Sent | "Cleared" message |
|---|---|---|
| DNS mismatch | after two checks in a row found it (it has held for an hour). A resolver that did not answer is not a mismatch | after two good checks in a row |
| Certificate | 14 days before expiry, and again 3 days before if it is still the same certificate | none |
| Level-2 limit refused mail | at the first refusal of a sending domain's applications, then at most once a day for that domain (one subject per domain, as built: what the refused row says of its domain is the client's own word, so the registry is asked) | none |
| Queue older than three days | when such a message appears | one, when none is left |
Event state is kept in the database, so a restart announces nothing a second
time.
## To settle while building
- How the panel submits: `sendmail` as the unprivileged panel user, and what
that needs of the image.
- Whether a notification carries a link to the panel - there is no setting
for the panel's own address.
- Where the refusal of a level-2 limit is noticed: in the journal milter as
it refuses, or from the `rejected` rows of the send log on the hourly check.
- Schema: the user's checkbox, the instance switch, the event state table.
## Screens
Not drawn. The Notifications box was taken out of the `account` and
`settings` mockups on 2026-10-10 (owner: 2.0 ships no box for a feature that
is not built). Drawing them again - one checkbox on Account, one switch on
Server › Settings, the SPF check on Server › Health - is the first step, shown
to the owner before any code ([panel-redesign.md](panel-redesign.md)
§ Enforcement).
## Implementation checklist
- [x] Mockups: Account checkbox, Settings switch and records, Health status card, help topic; accepted by the owner 2026-10-12 (`design:` commit)
- [x] Migration and store: user flag, instance switch, event state
- [x] Sending: local submission from `postmaster@<hostname>`, message text (`sendmail` as the panel user; signed through `non_smtpd_milters` with a key of the server's own name; no link in the text - the panel does not know its own address)
- [x] Background checker: DNS, certificate, queue; the rules of *When a message is sent*
- [x] Level-2 refusal event (from the `rejected` rows of the send log, looked at every minute)
- [x] Panel: Account, Settings, Health card, help
- [x] guide.md, architecture.md, security.md, schema-migrations.md
- [x] `go vet`, `go test`, e2e: a level-2 refusal reaches a mailbox as a signed notification (on the test server: vet ok, 23 packages ok, e2e ok; "once" and "clears" are held by the unit tests of `internal/notify`)
- [x] Technical review — **Opus** (2026-10-12; no Critical. Fixed: refusals counted by the registered domain and only under a limit, with an index of their own; account e-mails may be at public mail hosts; "said" means sent; DNS checks must be 45 minutes apart; the notifier's own mail is not a queue subject and a cut or unreadable listing is no listing; the tables must name the selector the panel shows; an unanswered `a`/`mx` lookup is not an SPF mismatch; a pass stops between domains at shutdown; days are rounded as a person counts. Left as found: a recipient whose copy failed while another's went out is not retried; a certificate that lives under 14 days is reported at each renewal)
- [x] Security review — **Fable** (2026-10-12; no Critical. Fixed: the limit notification was built on the domain a client writes in `MAIL FROM` - a mail cannon and a way to write to another tenant's users; a ceiling of 60 messages an hour; [security.md](../security.md) corrected on who can submit locally and on what the account e-mail now is. Recorded as accepted, with a roadmap candidate: an application can have a message signed as the server by writing its name into `From:` - the header is not held against the login, for sending domains either)
+4 -4
View File
@@ -235,9 +235,9 @@ the first step of that feature's plan — not invented during implementation.
| Roadmap item | Where it lands | Built from | Drawn |
|---|---|---|---|
| inbound-antispam-panel (agreed, `2.1.0`) | journal = **Inbound › Log** (add client IP and engine to the mockup's columns; `quarantine` joins the decisions); lists = new **Inbound › Filter lists**, global-only because the lists are instance-wide | list layout: add form + two tables, `status_tag`, filter form in `box_head`; the log links *Allow / Deny sender* to the lists | `in-log`, `in-filter-lists` |
| inbound-quarantine (candidate) | new **Inbound › Quarantine**, list + detail with *Release* / *Discard* | list layout; detail like `out-message` with `facts`, `log_pane` for headers only, Release beside a `sp-danger-zone` Discard | `in-quarantine`, `in-quarantine-message` — the open questions of its plan (where mail lives, what release means, retention, RBAC) stay open; the mockup assumes SelfPost-held mail, release to upstream, 14 days, delegated like the rest of Inbound |
| inbound-quarantine (an idea, off the roadmap since 2026-10-12) | new **Inbound › Quarantine**, list + detail with *Release* / *Discard* | list layout; detail like `out-message` with `facts`, `log_pane` for headers only, Release beside a `sp-danger-zone` Discard | `in-quarantine`, `in-quarantine-message` — the open questions of its plan (where mail lives, what release means, retention, RBAC) stay open; the mockup assumes SelfPost-held mail, release to upstream, 14 days, delegated like the rest of Inbound |
| preflight (candidate) | new **Server › Preflight**, next to Health: Health is what runs now, Preflight is the deeper on-demand installation check with a test e-mail form | `postmark` verdict, a table of checks with `status_tag` and a *What to do* column, one form box for the test e-mail | `preflight` |
| panel-notifications (candidate) | per-user event choice on **Account**; one switch under **Server › Settings** | checkboxes in a box, `help` | **no** — drawn once, taken out of `account` and `settings` on 2026-10-10 (owner): 2.0 ships no box for a feature that is not built; redrawn as the first step of its plan |
| panel-notifications (candidate) | one checkbox per user on **Account** (no choice per event, owner 2026-10-12); one switch under **Server › Settings** | a checkbox in a box, `help` | **no** — drawn once, taken out of `account` and `settings` on 2026-10-10 (owner): 2.0 ships no box for a feature that is not built; redrawn as the first step of its plan |
| password-reset (candidate) | a link on sign-in, two signed-out screens (request, set new password) | the signed-out layout | **no** |
| delivery-log-storage (candidate) | the *Delivery log* box of **Outbound › Log › message**, filled from a table instead of a grep of today's `mail.log` | `log_pane`, unchanged | `out-message` |
| csrf-tokens | no screen; every form gains a hidden field — forms are rendered through one helper so the token cannot be forgotten | — | n/a |
@@ -539,5 +539,5 @@ only when both are done. Every stage-2 step ends with the evidence of
order suits development, preflight, delivery-log-storage, panel-notifications,
password-reset (two screens to draw first), queue as a table with the inbound
queue, csrf-tokens
(one hidden field in the form partial once security.md is settled),
inbound-quarantine after its open questions.
(one hidden field in the form partial once security.md is settled).
inbound-quarantine is an idea outside the roadmap.
-77
View File
@@ -1,77 +0,0 @@
# Plan: preflight (installation check page)
**Status:** candidate
**Date:** 2026-08-19
**Version:** `1.x` MINOR; `candidate` until explicitly agreed.
---
## Goal
A dedicated `/server/preflight` page (global-admin only) that runs infrastructure-level
checks and helps the administrator confirm the instance is fully configured and
ready to deliver mail. Each failed check shows an actionable recommendation
(what to do or where to look).
## Mockup
[preflight](../assets/panel-redesign/panel/preflight.html), under the design
contract of [panel-redesign.md](panel-redesign.md): **Server › Preflight**,
path `/server/preflight`.
## Scope
**In:**
- rDNS / PTR verification (reuse `dnscheck.Checker.checkServer`)
- TLS certificate validity + hostname match (CN/SAN contains `SELFPOST_HOSTNAME`)
- Port connectivity: 25, 465, and 587 (if `SUBMISSION_ENABLE`) — dial own
public IP from inside the container, expect a 220 banner
- HELO/EHLO banner — connect to localhost:25, verify Postfix announces
`SELFPOST_HOSTNAME` (not `localhost` or a container ID)
- Trusted proxy headers:
- If `TRUSTED_PROXY_CIDR` set: verify peer IP is in range and
`X-Forwarded-For` is present
- If `PANEL_COOKIE_SECURE=true`: verify `X-Forwarded-Proto: https`
- If `TRUSTED_PROXY_CIDR` empty: informational warning (panel exposed
directly)
- DKIM signing — OpenDKIM milter socket reachable (reuse `health.CheckSocket`)
- Send test email — admin enters recipient address, system sends via
localhost:25, shows queued/error result
**Out:**
- Per-domain DNS checks (SPF, DKIM TXT, DMARC) — those live on domain detail
pages.
- Delivery tracking of test email (only confirms local MTA accepted).
## Done when
An administrator can open `/server/preflight`, see a traffic-light summary of all
infrastructure prerequisites, read specific fix instructions for any failures,
and send a test email to verify end-to-end flow.
## Risks
- Port-connectivity check from inside the container may not reflect external
reachability if NAT hairpin is unsupported (document limitation).
- Test email requires at least one configured domain (or falls back to
`postmaster@SELFPOST_HOSTNAME`).
## Implementation checklist
Target version cut: TBD (MINOR). One commit per step; code only after roadmap
status is **agreed**. See [development.md](../development.md) § Plan checklists.
- [ ] Port-connectivity helper (`internal/health/port.go`) — dial own IP, read
SMTP banner — **Sonnet**
- [ ] HELO banner check (connect localhost:25, parse 220 line) — **Sonnet**
- [ ] Hostname-in-certificate check (extend or wrap `health.CheckCertificate`)
— **Sonnet**
- [ ] Handler `handlers_preflight.go` (assemble all checks, actionable
recommendations) — **Sonnet**
- [ ] Test-email handler (`POST /server/preflight/test-email`, `net/smtp` to
localhost:25) — **Sonnet**
- [ ] Template `preflight.html` (traffic-light cards + test-email form) —
**Sonnet**
- [ ] Routes in `web.go`, nav link — **Sonnet**
+376 -162
View File
@@ -28,34 +28,46 @@ in `git log` and [CHANGELOG.md](../CHANGELOG.md).
| ID | Topic | Status | Progress | Plan |
|---|---|---|---|---|
| preflight | Installation check page (`/preflight`) | candidate | — | [plans/preflight.md](plans/preflight.md) |
| contributing | `CONTRIBUTING.md` | candidate | — | — |
| inbound-antispam-panel | Inbound antispam journal + allow/deny lists | agreed | 1/12 | [plans/inbound-antispam-panel.md](plans/inbound-antispam-panel.md) |
| inbound-quarantine | Inbound spam quarantine (hold / review / release) | candidate | — | [plans/inbound-quarantine.md](plans/inbound-quarantine.md) |
| csrf-tokens | Per-session CSRF tokens on state-changing forms | candidate | — | — |
| template-data-typing | Typed structs for template data instead of `map[string]any` | candidate | — | — |
| structured-logging | `log/slog` with levels and fields | candidate | — | — |
| review-2026-08-followups | Remaining findings of the 2026-08-19 code review | candidate | — | — |
| panel-notifications | E-mail notifications about important events | candidate | — | — |
| delivery-log-storage | Keep a message's `mail.log` lines with its send-log row | candidate | — | — |
| password-reset | Password reset by e-mail | candidate | — | — |
| health-setup-checks | Server › Health also checks the certificate's name, Postfix's greeting and the trusted proxy (was preflight) | agreed (2026-10-13) | — | — |
| test-email | Send a test message from the panel (was preflight) | agreed (2026-10-13) | — | — |
| inbound-antispam-panel | Inbound antispam journal + allow/deny lists | shipped `2.1.0` | 12/12 | [plans/inbound-antispam-panel.md](plans/inbound-antispam-panel.md) |
| origin-strict | Refuse a state-changing request that carries neither `Sec-Fetch-Site` nor `Origin` (was csrf-tokens) | agreed (2026-10-13) | — | — |
| backup-reauth | Ask for the account password before a backup or a domain export is downloaded | agreed (2026-10-13) | — | — |
| review-2026-08-followups | Remaining findings of the 2026-08-19 code review | closed 2026-10-14 (rows placed in [audit-2026-10-12](audit-2026-10-12.md)) | — | — |
| panel-notifications | E-mail notifications about important events | shipped `2.3.0` | 10/10 | [plans/panel-notifications.md](plans/panel-notifications.md) |
| delivery-log-storage | Keep a message's `mail.log` lines with its send-log row | shipped `2.2.0` | 8/8 | [plans/delivery-log-storage.md](plans/delivery-log-storage.md) |
| password-reset | Password reset by e-mail | agreed (2026-10-13) | — | — |
| local-time | The panel shows time, and counts its days, in a time zone other than UTC | agreed (2026-10-13; design questions open) | — | — |
| phone-desktop-mode | A phone browser in "desktop site" mode gets the phone navigation | agreed (2026-10-13: after stages 1 and 2 of audit-2026-10-12) | — | — |
| from-header-binding | Hold a message's `From:` header against the application that sends it | shipped `2.4.0` | 9/9 | [plans/from-header-binding.md](plans/from-header-binding.md) |
| panel-redesign | Panel redesign (accepted mockups, design contract) | shipped `2.0.0` | 27/27 | [plans/panel-redesign.md](plans/panel-redesign.md) |
| schema-squash | Squash SQLite migrations into a 2.x baseline | built in panel-redesign (stage 1) | — | [plans/panel-redesign.md](plans/panel-redesign.md) |
| audit-2026-10-12 | Follow-ups of the 2026-10-12 full-tree code audit, seven stages | in progress (stage 0 shipped `2.4.2`, stages 1+2 `2.5.0`, 2b `2.5.1`; next stage 2c → `2.5.2`) | 40/81 | [audit-2026-10-12.md](audit-2026-10-12.md) |
**Recommended order** (not binding; owner, 2026-10-09: feature order follows
development convenience): **panel-redesign** shipped as `2.0.0` (2026-10-11); the next stage is
**inbound-antispam-panel** (`2.1.0`), built in the new component kit —
styling its two screens once instead of twice. panel-docs shipped in
development convenience): **panel-redesign** shipped as `2.0.0` (2026-10-11) and
**inbound-antispam-panel** as `2.1.0` (2026-10-12), built in the new component
kit — styling its two screens once instead of twice. **delivery-log-storage**
shipped as `2.2.0` the same day, **panel-notifications** as `2.3.0` and
**from-header-binding** as `2.4.0`. Stage 0 of
[audit-2026-10-12](audit-2026-10-12.md) shipped as `2.4.2` (2026-10-14), its
stages 1 and 2 as `2.5.0` the same day, and stage 2b as `2.5.1`.
Nothing is agreed after it: every remaining
item is a candidate. panel-docs shipped in
[CHANGELOG.md](../CHANGELOG.md) `[1.8.0]`; dmarc-reports in `[1.7.0]`
(security review of the ingest path pending); domain-stats-auto-ratelimit in
(its ingest path first ran in `2.0.0` and was reviewed on 2026-10-11, see
[security.md](security.md)); domain-stats-auto-ratelimit in
`[1.6.0]`; send-log-retention in `[1.5.0]`; inbound-relay in `[1.4.0]`;
queue-retries in `[1.3.1]`; the 2026-08-13 full-tree review follow-ups are in
`[1.3.0]`. Candidates need explicit agreement before they join the queue.
The 2026-08-19 full-tree code review is closed except for four items, all
listed above and none of them urgent: [csrf-tokens](#csrf-tokens),
[template-data-typing](#template-data-typing),
[structured-logging](#structured-logging) and
listed above and none of them urgent: [origin-strict](#origin-strict) (its
"add CSRF tokens", decided otherwise on 2026-10-13),
template-data-typing (done by `2.0.0`; its last map is a step of
[audit-2026-10-12](audit-2026-10-12.md) stage 4),
structured-logging ([dropped](#dropped-with-the-condition-for-return) on
2026-10-13) and
[review-2026-08-followups](#review-2026-08-followups). What that review
produced is in [CHANGELOG.md](../CHANGELOG.md) `[1.9.2]`; its own document
was removed once these four items carried everything still open, and stays in
@@ -68,24 +80,6 @@ is `done/total` checklist steps in that plan ([development.md](development.md)
---
## contributing
**Goal:** `CONTRIBUTING.md` in the root — the dev loop, the checks to run
before a PR, the commit protocol; [development.md](development.md) links to it
rather than repeating it.
**Boundary:** process documentation; worth writing once there is an external
flow of PRs.
**Done when:** the file is in the root and development.md does not duplicate
it.
**Dependencies / risks:** with a single developer and no PRs, this is low
priority.
**Version:** no bearing on semver.
---
## panel-docs
**Status:** done — shipped in `[1.8.0]` (2026-08-18). The implementation plan
@@ -121,8 +115,8 @@ action) plus editable allow/deny lists that tune the external filter — without
shipping rspamd or another engine inside the image.
**Boundary:** metadata journal and list sync only; SelfPost does not run the
filter ([product.md](product.md)). Quarantine:
[inbound-quarantine](#inbound-quarantine). Design closed in
filter ([product.md](product.md)). Quarantine is not on the roadmap
([Ideas](#ideas-not-on-the-roadmap)). Design closed in
[plans/inbound-antispam-panel.md](plans/inbound-antispam-panel.md) § Decisions.
**Done when:** with inbound relay enabled, the operator sees journal rows
@@ -133,130 +127,126 @@ hook is set.
**Dependencies / risks:** inbound relay `[1.4.0]`; rspamd map/header coupling;
journal PII; global-admin-only RBAC.
**Version:** `2.1.0` MINOR (was `1.10.0`; reordered after the redesign on 2026-10-09); **agreed**.
**Version:** shipped `2.1.0` (2026-10-12), MINOR (was `1.10.0`; reordered after the redesign on 2026-10-09).
---
## inbound-quarantine
## health-setup-checks
**Goal:** optional hold for suspicious inbound mail — review in the panel, then
release to upstream, discard, or tune lists. Design not settled.
**Decided by the owner on 2026-10-13**, in place of the former candidate
*preflight* (a page of its own, Server › Preflight, planned in August before
Server › Health existed). Of that plan's eight checks Health already makes
three (reverse DNS, certificate validity, the DKIM socket); five were missing.
Three of the five go into Health, one becomes [test-email](#test-email), one
is not built ([Dropped](#dropped-with-the-condition-for-return)). No second
page of checks.
**Boundary:** expands inbound from pure relay toward short-lived message
storage; may conflict with "no mailboxes" in [product.md](product.md) until
product explicitly agrees. Alternatives (rspamd-only quarantine vs SelfPost
store) are listed in [plans/inbound-quarantine.md](plans/inbound-quarantine.md).
**Goal:** three more lines on Server › Health, in the cards that are there:
**Done when:** TBD after open questions in the plan are answered.
- **The certificate is for this server**: its names include
`SELFPOST_HOSTNAME` (TLS certificate card). Today a certificate for another
name is "valid".
- **Postfix greets with the hostname**: the 220 line on port 25 names
`SELFPOST_HOSTNAME`, not `localhost` or a container id (Hostname card).
- **The trusted proxy**: the address the panel sees this request from, and
whether it took it from `X-Forwarded-For`. With `TRUSTED_PROXY_CIDR` wrong
or empty behind a proxy the header is ignored
([internal/web/auth/auth.go](../internal/web/auth/auth.go)), every client
has the proxy's address and the sign-in limiter is shared by all of them -
silently.
**Dependencies / risks:** inbound relay `[1.4.0]`; storage/retention/backup
size; malware in held MIME; overlap with
[inbound-antispam-panel](#inbound-antispam-panel). No implementation checklist
yet.
**Boundary:** lines in existing cards, each with what to do when it fails;
no new card unless the mockup shows the lines do not fit. Mockup-first
([plans/panel-redesign.md](plans/panel-redesign.md) § Enforcement). The
mockup `preflight` stays drawn and unbuilt.
**Version:** TBD; `candidate` until explicitly agreed.
**Done when:** each line is shown wrong on the e2e stand or in a test with a
wrong certificate name, a wrong greeting and an untrusted proxy, and right on
the test server.
**Version:** `2.x` MINOR; `agreed`.
---
## preflight
## test-email
**Goal:** a `/preflight` page (global-admin only) that verifies instance-level
infrastructure: rDNS/PTR, TLS certificate + hostname match, port 25/465/587
reachability, HELO banner correctness, reverse-proxy headers, DKIM milter
socket, and a test-email sender. Each check shows an actionable fix
recommendation on failure.
**Decided by the owner on 2026-10-13**, taken out of the former *preflight*.
**Boundary:** instance infrastructure only; per-domain DNS checks remain on
domain detail pages. Does not track test-email delivery beyond local MTA
acceptance.
**Goal:** a global administrator sends a test message from the panel to an
address they type, and sees whether Postfix took it; the message then appears
in Outbound › Log like any other. Today nobody can send through the server
without setting up an application first.
**Done when:** the page loads, all checks run with traffic-light results and
recommendations, and the test-email form successfully submits a message through
Postfix.
**Boundary:** global role only; from `postmaster@SELFPOST_HOSTNAME`, the
sender notifications use, or from a sending domain the administrator picks -
settled at the mockup. It does not follow the message further than the log
does. Where the form lives (Health, or a sending domain's page) is settled at
the mockup; mockup-first.
**Dependencies / risks:** port-connectivity check may not reflect external
reachability behind NAT without hairpin; test email needs at least one domain or
falls back to `postmaster@SELFPOST_HOSTNAME`.
**Done when:** the form sends, the row appears in the log, the form is
rate-limited and refuses a recipient list, and it passes a security review -
a form that sends mail is a new surface.
**Version:** `1.x` MINOR; `candidate`.
**Version:** `2.x` MINOR; `agreed`.
---
## csrf-tokens
## origin-strict
**Goal:** a per-session CSRF token on every state-changing form, checked
server-side in addition to the `Sec-Fetch-Site` / `Origin` check the panel
relies on today (`originAllowed` in
[internal/web/security.go](../internal/web/security.go)).
**Decided by the owner on 2026-10-13**, in place of the former candidate
*csrf-tokens* (per-session tokens on every form, recorded by the 2026-08-19
review): no tokens; the origin check is made strict instead.
**Boundary:** defence in depth, not a replacement — the header check stays and
keeps covering requests that carry no form. The token is bound to the existing
session record; no new session model, no cookie changes.
**Goal:** `originAllowed`
([internal/web/security.go](../internal/web/security.go)) refuses a
state-changing request that carries neither `Sec-Fetch-Site` nor `Origin`.
Today such a request is let through, which leaves a client that sends neither
header open to a request forged by any site
([security.md](security.md) § Accepted risks).
**Done when:** every POST form and htmx fragment that changes state carries a
token, a missing or mismatched token is refused with a test that proves it, and
[security.md](security.md) describes the layered model in one place instead of
two.
**Boundary:** one branch of one function, its test, and the documents. No
template changes, no hidden fields. Before the change, find what posts to the
panel without a browser (the e2e stand, scripts) and make it send `Origin`.
**Dependencies / risks:** [security.md](security.md) currently records the
opposite decision — origin checking without tokens, with the accepted risk and
its revisit triggers written out in the ADR § *CSRF via origin checking*. The
2026-08-19 review recorded "add tokens" without turning it into a task, which is
why this item exists. Settle which decision stands before writing code; both
outcomes are honest, and "keep headers" means deleting this item rather than
leaving the contradiction standing. The work itself is a hidden field in roughly
two dozen forms plus the htmx swap paths.
**Done when:** a `POST` with neither header is refused with a test that proves
it, the e2e stand passes, the accepted risk about header-less requests is
struck from [security.md](security.md) and its ADR states the strict rule.
**Version:** `1.x` MINOR; `candidate`.
**Dependencies / risks:** a browser that sends neither header can no longer
save anything in the panel - accepted, the panel's users pick their own
browsers. Tokens stay a recorded alternative, not a task: the conditions under
which they are built are in [security.md](security.md), ADR § *CSRF via origin
checking* (**Revisit if**).
**Version:** `2.x` PATCH; `agreed`.
---
## template-data-typing
## backup-reauth
**Goal:** replace the `map[string]any` template payloads in
`internal/web/handlers` with typed structs (`DomainDetailData`, `SettingsData`,
...), so a renamed or mistyped key is a compile error instead of a silently
empty page section.
**Agreed with the owner on 2026-10-13**, out of
[review-2026-08-followups](#review-2026-08-followups).
**Boundary:** handler-to-template plumbing only; no template redesign, no route
or behaviour change. `renderDomainDetail`
([handlers_apps.go](../internal/web/handlers/handlers_apps.go)) sets around
forty keys on one map and is the case that motivates the item; roughly ten
render sites still pass maps.
**Problem:** Server › Backup hands the full backup and a domain export to any
live session of a global administrator
([handlers_backup.go](../internal/web/handlers/handlers_backup.go)). The
archive holds the whole database - panel password hashes, application (SASL)
passwords, DKIM keys - so a stolen session cookie, or a panel left open, is a
quiet copy of the server. *Encrypt with a password* does not help: the thief
chooses that password.
**Done when:** the page handlers pass structs, the shared footer/base fields
(`Version`, `Copyright`, `SourceURL`) still arrive on every page, and the
template guard tests pass unchanged.
**Goal:** both downloads ask for the current account password, checked the
way a password change checks it and under the same limiter.
**Dependencies / risks:** mechanical but wide — it touches every
`handlers_*.go` and every template that reads those keys, and a missed key is
invisible unless a test renders that page. Worth pairing with the handler tests
in [review-2026-08-followups](#review-2026-08-followups) so the pages are
covered before they are rewired. Reviewer's estimate: 2–3 hours.
**Boundary:** one field in each of the two forms (mockup-first); import is
not touched; the `selfpost-backup` CLI is not touched - it runs inside the
container.
**Version:** no bearing on semver; `candidate`.
**Done when:** a download without the password or with a wrong one is refused
with a test that proves it, attempts are limited, and
[security.md](security.md) § Backup says so.
---
## structured-logging
**Goal:** move from `log.Printf` to `log/slog` — levels, structured fields, and
one logger injected through constructors, instead of the package-level
`logf = log.Printf` alias repeated in five packages.
**Boundary:** internal logging only. The destination stays stdout/stderr under
supervisord; no log shipping, no new dependency. Lines the panel's own Status
page and log tailer depend on must keep parsing.
**Done when:** packages take a `*slog.Logger`, the per-package `logf` aliases
are gone, and levels are used deliberately (an error is not an info).
**Dependencies / risks:** the log tailer reads Postfix's log rather than the
panel's, so the blast radius is small — but anything that greps the panel's
output (e2e assertions, operator habits) sees a new format. Reviewer's
estimate: 3–4 hours.
**Version:** no bearing on semver; `candidate`.
**Version:** `2.x` MINOR; `agreed`.
---
@@ -269,14 +259,14 @@ is in the table below; the rest of it shipped in
| Area | Item |
|---|---|
| Security | Re-authenticate (ask for the current password) before a backup download — [handlers_backup.go](../internal/web/handlers/handlers_backup.go) hands the archive to any signed-in session |
| Tests | The `internal/dmarc` service layer (`service.go`, `addresses.go`) has no tests |
| Tests | No direct tests for the apps / domains / users / dmarc handlers; `authz_test` covers them only indirectly |
| Tests | Nothing covers error page templates, a corrupted backup archive, malformed or oversized DMARC XML, delivery-log pagination edges, concurrent store operations, or zero-traffic `store` stats |
| Edge case | `internal/secretfile` accepts an empty password at library level — guard it, or state that validation is the caller's job |
| Edge case | The log fingerprint in [logtail/offset.go](../internal/logtail/offset.go) can collide across a rotation — include the inode |
| Edge case | `parsePage` in [handlers_monitor.go](../internal/web/handlers/handlers_monitor.go) does not clamp to `lastPage` |
| Edge case | [dnscheck/resolver.go](../internal/dnscheck/resolver.go) fans out four parallel queries where a fallback chain would do — change it or write down why not |
| Security | ~~Re-authenticate (ask for the current password) before a backup download~~ — struck: agreed on 2026-10-13 as an item of its own, [backup-reauth](#backup-reauth) |
| Tests | ~~The `internal/dmarc` service layer (`service.go`, `addresses.go`) has no tests~~ — struck: a step of audit-2026-10-12 stage 4 (DMARC reports tested where they come in) |
| Tests | ~~No direct tests for the apps / domains / users / dmarc handlers; `authz_test` covers them only indirectly~~ — struck: users with stage 2 (`submitUserUpdate`), DMARC with stage 4; apps and domains get theirs when a step changes them - a test without a change finds nothing the e2e does not |
| Tests | ~~Nothing covers error page templates, a corrupted backup archive, malformed or oversized DMARC XML, delivery-log pagination edges, concurrent store operations, or zero-traffic `store` stats~~ — struck: the corrupted archive goes to stage 3, DMARC XML to stage 4, paging edges to stage 4 (`parsePage`), concurrency of the last global to stage 2; error templates, other concurrency and zero-traffic stats are dropped (owner, 2026-10-14) - code no step changes, and a test kept for its own sake finds nothing |
| Edge case | ~~`internal/secretfile` accepts an empty password at library level~~ — struck: audit-2026-10-12 stage 3 (M5) |
| Edge case | ~~The log fingerprint in `logtail/offset.go` can collide across a rotation — include the inode~~ — struck: agreed on 2026-10-13 as part of a step of [audit-2026-10-12](audit-2026-10-12.md) stage 2b (a log rotated while the panel was down is read to its end) |
| Edge case | ~~`parsePage` does not clamp to `lastPage`~~ — struck: audit-2026-10-12 stage 4 |
| Edge case | ~~`dnscheck/resolver.go` fans out four parallel queries where a fallback chain would do~~ — struck: finding M14 of [audit-2026-10-12](audit-2026-10-12.md) |
| UI | ~~`.pair` (DMARC templates) and `.split` (every other page) are two names for one layout~~ — struck: retired by [panel-redesign](#panel-redesign); both stylesheets are gone and every page is laid out by the component kit |
| UI | ~~Two `back_link` elements in a row in `dmarc_domain.html`~~ — struck: retired by [panel-redesign](#panel-redesign); the page is rebuilt on the kit with one breadcrumb in its head |
| UI | ~~`initShowWhen` is not re-run after an htmx swap~~ — struck: the show/hide rules listen on the document and are applied again on `htmx:load` |
@@ -286,10 +276,10 @@ apart from the backup re-auth prompt. Anything here that grows past an hour
becomes its own roadmap item instead.
**Done when:** each row is either implemented or struck with a written reason.
**Closed 2026-10-14** with the owner: every row is struck above, placed in a
stage of [audit-2026-10-12](audit-2026-10-12.md) or dropped with its reason.
**Dependencies / risks:** the test rows overlap
[template-data-typing](#template-data-typing) — write the handler tests first if
both are taken up. One finding of that review was **declined** and is
**Dependencies / risks:** One finding of that review was **declined** and is
deliberately absent: `//go:build linux` for the `/proc` reader in
[internal/health/machine.go](../internal/health/machine.go). That code is
parameterised by root directory, calls nothing platform-specific, already
@@ -348,38 +338,43 @@ time to final status plus the queue lifetime), not to the id alone; the table
grows with the send log (small — a few hundred bytes per message) and is
covered by the same retention.
**Version:** `2.x` MINOR; `candidate`. Fits after the redesign, built against
the kit's `log_pane`.
**Version:** shipped `2.2.0` (2026-10-12), MINOR; option 1. Decisions and
checklist: [plans/delivery-log-storage.md](plans/delivery-log-storage.md).
---
## panel-notifications
**Goal:** the panel tells its administrators when something needs them instead
of waiting to be opened: a DNS record stops matching, a domain or application
hits its rate limit and mail is refused, an inbound upstream is unreachable,
DMARC failures appear, mail is held in quarantine; for the global
administrator also the TLS certificate nearing expiry, a process or milter
socket down, a queue older than a day.
of waiting to be opened. First version, four events: a sending domain's DNS
record stops matching, a level-2 rate limit refuses mail, the TLS certificate
is about to expire, a message has been in the queue for more than three days.
**Boundary:** e-mail only, sent through the relay itself from
`postmaster@<hostname>`. Each user picks events under **Account**; a domain
administrator hears only about assigned domains. One message when a problem
starts and one when it clears — no streams, no digests in v1. One instance
switch under **Server › Settings**. No webhooks, no chat integrations.
**Boundary:** e-mail only, from `postmaster@<hostname>`, submitted locally -
not from one of the sending domains. One checkbox per user under **Account**
(no choice per event), one instance switch under **Server › Settings**; a
domain administrator hears only about assigned domains. No streams, no
digests, no webhooks, no chat integrations. Not in the first version: a
process or milter down, an inbound upstream unreachable, DMARC failures.
**Done when:** an event from the list reaches the account e-mail of every user
who chose it and may see its subject, once on start and once on clear; a user
without an e-mail gets nothing; the switch turns it all off.
**Done when:** each of the four events reaches the account e-mail of every
user who has notifications on and may see its subject, by the rules of the
plan (a "cleared" message for DNS and the queue only); a user without an
e-mail gets nothing; the switch turns it all off.
**Dependencies / risks:** the account e-mail (2.0 baseline, see
[plans/panel-redesign.md](plans/panel-redesign.md) § Account e-mail); a relay
that cannot send cannot report that it cannot send — server events need the
direct-to-MX path or stay panel-only; event state must survive restarts so a
reboot does not re-announce everything. Screens are drawn: `account`,
`settings`.
that cannot send cannot report that it cannot send - none of the four events
is "the relay is down"; mail from the bare hostname needs an SPF record for
it to be believed, which Server › Health will check; event state must survive
restarts so a reboot does not re-announce everything. The screens are not
drawn: the Notifications box was taken out of `account` and `settings` on
2026-10-10 and is redrawn as the first step.
**Version:** `2.x` MINOR; `candidate`.
**Version:** shipped `2.3.0` (2026-10-12), MINOR - the design questions were closed
with the owner on 2026-10-12
([plans/panel-notifications.md](plans/panel-notifications.md)) and he took it into
work the same day.
---
@@ -392,8 +387,22 @@ account e-mail instead of asking the global administrator to set a new one.
same answer whether or not the username exists. The only global administrator
without a working e-mail still recovers from the container (today's path).
**The e-mail must be confirmed (owner, 2026-10-13).** A reset link goes only
to an address its owner has confirmed; an account whose e-mail is not
confirmed has no reset, and recovers as today (the global administrator sets
a password, or the container). The panel does not confirm addresses today -
`users.email` is whatever was typed - so the item builds that first: saving
an e-mail under Account mails a one-time link to it, opening the link marks
the address confirmed (a migration: the mark on `users`), changing the
address clears the mark, and Account shows which it is. An address typed for
a user by the global administrator is unconfirmed until that user opens the
link. To settle at the mockup: whether notifications (2.3.0) keep going to an
unconfirmed address.
**Done when:** the flow works end to end, tokens expire and cannot be reused,
requests are rate-limited, and it passes a security review.
requests are rate-limited, an account with an unconfirmed e-mail gets no
link and the sign-in page answers the same for it, and it passes a security
review.
**Dependencies / risks:** the account e-mail; an account-takeover surface where
there was none — security review is part of the item, not an afterthought.
@@ -401,7 +410,145 @@ Not drawn yet. Changing the account e-mail must then require the current
password (2.0 security review), because the e-mail becomes where a reset link
goes and today Account saves it without the password.
**Version:** `2.x` MINOR; `candidate`.
**Version:** `2.x` MINOR; `agreed` (owner, 2026-10-13: domain administrators
to come need it - they are not the owner and should not have to ask him for
a new password). Mockup-first; changing the account e-mail asks for the
current password in the same cut.
---
## local-time
**Agreed with the owner on 2026-10-13.** It came from one cell: *Today* on
Health's Sender check card is the UTC day and does not say so. Offered a
label ("Today, UTC") or the local day, he chose the local day - which is a
decision about the whole panel, not one cell.
**Today:** every time the panel shows is UTC and says so (*Time, UTC* in the
logs, "UTC" after a date), and everything counted by day - the sender
check's days, the delivery-log measurement, the statistics behind the
charts, retention - is counted by the UTC day.
**Goal:** the panel shows time in a chosen time zone, and "today" is that
zone's day.
**Where the zone comes from (owner, 2026-10-13):** one setting of the
instance, *Time zone* under Server › Settings, UTC by default; it holds for
every user and for notification mail. Not the browser's zone: only a script
could read it, and the panel works without JavaScript. Not a setting of each
user yet - **that is added (the same list under Account, "as the server" by
default) when** a user of the panel lives in another zone than the one set
for the instance, which the first domain administrator who is not the owner
may bring.
**What is written beside a time (owner, 2026-10-13):** nothing. The "UTC"
labels go - the column is *Time*, a date is a date - and the zone is stated
in one place, the setting. His reason: there is nobody to send a line of the
panel to yet. **The offset is written where "UTC" is today ("Time, UTC+3",
"2026-09-21 16:40 UTC+3") when** times of the panel start to be read by
someone who does not know the instance's zone: a second user of the panel,
or lines copied out of it to another person.
**A trap this leaves, known in advance (owner asked for it to be written
down, 2026-10-13):** Server › System log, and the *Delivery log* box on a
message's page, show the lines of `mail.log` as Postfix wrote them, each with
its own stamp ("Oct 13 14:01:52"). Those stamps are the container's time -
UTC unless the container is given a `TZ` - and they are not rewritten. Once
the rest of the panel is in the instance's zone and no time carries a label,
a message that the Outbound log puts at 17:01 has lines stamped 14:01 on its
own page, with nothing on the screen to say why. The plan must close this
before the item ships, one way or another: rewrite the stamp when a line is
shown, say on those two screens which clock the lines use, or run the
container in the instance's zone (which moves `mail.log` itself and what the
tailer reads). The guide's section on time must say the same thing as the
screens.
**How a day is counted (owner, 2026-10-13):** the tables that keep a row
per day - `sender_check_days`, and `delivery_log_days` of
[audit-2026-10-12](audit-2026-10-12.md) stage 1 - key the row by the date in
the instance's zone at the moment of counting, from the release on. Rows
counted before it stay UTC days. Changing the zone later leaves the rows
already written as days of the old zone, a shift of a few hours in a
statistic; accepted - the zone is set once, and rows per hour (which would
survive a change) cost a migration for it. The charts and statistics group
the log by hour already and need nothing.
**Open, to settle at the mockups:** the trap above; which of the screens
that show a date show the year.
**Boundary:** what the panel shows and how it groups by day. `mail.log`,
the database's stored instants and the e2e stay as they are.
**Version:** `2.x` MINOR; `agreed`; the plan is written when it is taken up.
---
## phone-desktop-mode
**Problem (owner, 2026-10-11):** the panel opened on a phone with the
browser's "desktop site" switched on does not become the desktop panel.
Seen on Server › Health at `selfpost.mixfed.ru`: the page body is laid out in
columns (Machine and Processes side by side, three cards in the next row),
while the navigation is the phone one - not a bar but a stack with every
section open (Outbound, Inbound, Server, the user menu), filling the first
screen before the page starts. The content also sits flush against the left
edge of the screen.
**Goal:** find out why, then decide what the panel should be in that mode -
the desktop panel, or a phone panel that is at least consistent.
**Where to look first (a guess, not checked):** "desktop site" on Android
lays a page out about 980 px wide. Bulma turns the navigation into a bar at
1024 px and puts columns side by side from 769 px, so 980 px falls between
the two: desktop columns under a touch navigation. `panel.css` has its own
breakpoints at 768, 1023 and 1407 px. Check the width the phone really
reports, why every dropdown is open rather than folded behind the burger,
and where the left padding went.
**Boundary:** the navigation and page gutter between 769 and 1023 px, which
is also a tablet in portrait and a narrow desktop window - the same defect is
expected there. A design change is mockup-first
([plans/panel-redesign.md](plans/panel-redesign.md) § Enforcement).
**Done when:** the cause is written down here, and the panel at that width
has one coherent layout, checked on a phone in "desktop site" mode and in a
980 px window.
**Version:** `2.x` PATCH or MINOR; `agreed` (owner, 2026-10-13), taken up
after stages 1 and 2 of [audit-2026-10-12](audit-2026-10-12.md) - mail shown
wrong and the tenant boundary come before layout.
---
## from-header-binding
**Problem (security review of panel-notifications, 2026-10-12):** Postfix
holds the envelope sender against the SASL login
(`reject_sender_login_mismatch`), and OpenDKIM chooses the signing key by the
`From:` header. Nothing holds the header against the login. An application
of one sending domain can therefore write another registered domain - or,
since 2.3.0, the server's own name - into `From:` and have the message signed
with that name's key: mail that passes DMARC as a domain the application has
no right to, or that looks like a notification of the panel.
**Agreed with the owner on 2026-10-12**, built and reviewed the same day
([plans/from-header-binding.md](plans/from-header-binding.md); what is not
done is listed there under *Not done*). The rule he
chose is wider than the one first written here: `From:` is held to exactly
what the envelope is held to - any address of the application's domain, or
the addresses of its list - and anything else is refused, a stranger's domain
included. The check is a role of the panel on a strict socket of its own:
while the panel does not answer, mail is deferred, not accepted unchecked.
Server › Health counts what that costs, to decide later whether the check
needs a process of its own.
**Done when:** the e2e stand shows an application refused for another
tenant's `From:`, for the hostname's and for an address off its list, and
accepted for its own.
**Version:** shipped `2.4.0` (2026-10-12), MINOR. What it costs - the
panel's downtime is now mail's - is in [security.md](security.md) § Accepted
risks.
---
@@ -482,3 +629,70 @@ explicit major). Squashing the current short chain is not a reason to cut 2.x on
its own. A missed gate leaves a mid-chain 1.x database silently stuck.
**Version:** `2.x` major only; not a 1.x item.
---
## Dropped, with the condition for return
Items the owner took off the roadmap. Each names what would bring it back;
without that it is not offered again.
- **structured-logging** (dropped 2026-10-13) - move the panel from
`log.Printf` to `log/slog` with levels and fields, one logger passed through
constructors. Dropped because the panel's log has one reader, the owner, by
eye in `docker logs`; the change is wide (some 225 call sites), changes no
behaviour, and gives a new format to everything that reads the output.
**Returns if:** the log goes to an outside collector, or it has to be
filtered by level.
- **Certificate warnings measured by the certificate's lifetime** (left as
it is, owner, 2026-10-14). The panel warns 14 and 3 days before a
certificate expires, whatever its lifetime
([notify.go](../internal/notify/notify.go), `certFirstDays`,
`certLastDays`). A certificate issued for less than 14 days is inside the
first warning from its first hour, and each renewal of it is a new
certificate and a new message. Not reachable with the 90-day certificate
the instance has. **Returns if:** the instance takes a certificate that
lives less than 30 days. The rule then offered: the first warning at a
quarter of the lifetime left, the last at a tenth, and never earlier than
the 14 and 3 days of today - so a longer certificate, a yearly one
included, is warned about exactly as now.
- **contributing** (dropped 2026-10-13) - a `CONTRIBUTING.md` in the root:
the dev loop, the checks before a PR, the commit protocol. Dropped because
there is one developer and no PRs, and everything the file would say is in
[development.md](development.md); a second file is a second copy to keep
in step. **Returns if:** the first outside PR arrives, or the owner decides
to invite contributors.
- **A check that ports 25, 465 and 587 are reachable** (dropped 2026-10-13,
from the former *preflight*). The panel can only dial its own public
address from inside the container: behind NAT without hairpin that shows an
open port as closed, and with hairpin it shows as open a port the provider
blocks from outside. A card that is wrong both ways is worse than none, and
an honest check needs a probe outside the server, which is outside the
product ([product.md](product.md)). The administrator's browser cannot do
it either: 25, 465 and 587 are on the Fetch standard's list of blocked
ports. Two substitutes were offered and declined the same day: a Health
line "last connection from outside on each port", read from `mail.log`, and
a command in the help to run from one's own machine. **Returns if:** the
owner accepts an outside probe, or asks for one of the two substitutes.
- **preflight as a page of its own** (dropped 2026-10-13): see
[health-setup-checks](#health-setup-checks). Its plan,
`plans/preflight.md`, was removed and stays in git history.
- **csrf-tokens** (decided 2026-10-13) - session-bound tokens on every form.
Replaced by [origin-strict](#origin-strict); the conditions for building
tokens after all are in [security.md](security.md), ADR § *CSRF via origin
checking*.
---
## Ideas, not on the roadmap
Thoughts kept so they are not lost. They are not candidates: nothing here is
waiting for an OK, has a version or takes a place in the order above. An idea
joins the index only when the owner says so.
- **Inbound quarantine** (moved here by the owner on 2026-10-12: "an idea to
think about") - hold suspicious inbound mail, review it in the panel, then
release it to the upstream or discard it. It would take the relay from
passing mail on to storing it for a while, against "no mailboxes" in
[product.md](product.md). Notes and open questions:
[plans/inbound-quarantine.md](plans/inbound-quarantine.md); the screens
`in-quarantine` and `in-quarantine-message` are drawn and stay unbuilt.
+27 -10
View File
@@ -17,8 +17,8 @@ When you add or review a migration, update this file in the same change.
| Field | Value |
|---|---|
| Chain head | `user_version = 1` |
| Files | `0001_init.sql` (the 2.0 baseline) |
| Chain head | `user_version = 9` |
| Files | `0001_init.sql` (the 2.0 baseline), `0002_inbound_spam_log.sql`, `0003_send_log_lines.sql`, `0004_notifications.sql`, `0005_send_log_rejected_index.sql`, `0006_sender_check.sql`, `0007_delivery_log_days.sql`, `0008_tenant_boundary.sql`, `0009_refusal_line.sql` |
| File mark | `PRAGMA application_id = 0x53503230` ("SP20") |
| Database file | `/data/selfpost.db` (bind mount) |
| Compatibility | 2.x MINOR releases must boot a `2.0.0` data directory. **Nothing is carried over from 1.x.** |
@@ -60,10 +60,14 @@ What that means in practice:
**Implication:** once `2.0.0` is released, `0001_init.sql` is never edited
again and no file is deleted, renamed or reordered — every 2.x database has
already run it. Until that release the baseline is still being written:
later steps of the redesign and the features queued behind it (the next one
is `0002_inbound_spam_log.sql` of inbound-antispam-panel, `2.1.0`) add files
after it.
already run it. `2.0.0` and `2.0.1` shipped with the baseline alone; the
features after them add files, the first being `0002_inbound_spam_log.sql` of
inbound-antispam-panel (`2.1.0`), then `0003_send_log_lines.sql` of
delivery-log-storage (`2.2.0`), `0004_notifications.sql` and
`0005_send_log_rejected_index.sql` of panel-notifications (`2.3.0`), and
`0006_sender_check.sql` of from-header-binding (`2.4.0`), and
`0007_delivery_log_days.sql` and `0008_tenant_boundary.sql` of the 2026-10-12
audit (`2.5.0`).
---
@@ -72,18 +76,26 @@ after it.
| Ver | File | Shipped | Kind | Summary |
|-----|------|---------|------|---------|
| 1 | `0001_init.sql` | 2.0.0 | DDL | The whole 2.0 schema, below |
| 2 | `0002_inbound_spam_log.sql` | 2.1.0 | DDL | `inbound_spam_log`, `inbound_spam_list` and two indexes; nothing existing changes |
| 3 | `0003_send_log_lines.sql` | 2.2.0 | DDL | `send_log_lines` and one index; nothing existing changes |
| 4 | `0004_notifications.sql` | 2.3.0 | DDL | `users.notify` (default 1: every user starts with notifications on) and `notify_state` |
| 5 | `0005_send_log_rejected_index.sql` | 2.3.0 | DDL | `idx_send_log_rejected`, a partial index of the refused rows of `send_log` by time; a file of its own because the test installation had run 0004 before the review asked for it |
| 6 | `0006_sender_check.sql` | 2.4.0 | DDL | `send_log.reason` and `send_log.header_from` (both default `''`: a row refused earlier keeps no reason), and `sender_check_days` |
| 7 | `0007_delivery_log_days.sql` | 2.5.0 | DDL | `delivery_log_days`: what the reconcile sweep and the log tailer did, a row per UTC day, kept a year, shown nowhere - the measurement that decides the sweep's fate (audit 2026-10-12, stage 1). `send_log.reason` gains the values `reconcile` and `reconcile-contradicted` without a schema change |
| 8 | `0008_tenant_boundary.sql` | 2.5.0 | DDL + data | `idx_applications_login_nocase`: an application login is unique without regard to case (audit 2026-10-12, H4). `send_log.domain_id` and `inbound_spam_log.inbound_domain_id` (`ON DELETE SET NULL`), each with an index, filled once by name from the registry as it was then: a row belongs to a domain by id, and a domain deleted and added again starts with an empty log (L12) |
| 9 | `0009_refusal_line.sql` | 2.5.1 | DDL | `send_log.refusal_line`: Postfix's line about a refusal at MAIL FROM (a level-2 limit, a client address off the list), which has no queue id; the log tailer puts it on the row; `send_log.recovered`, a refusal row the tailer wrote from `mail.log` because the panel stopped before writing it; `sender_check_days.messages` (messages deferred mid-message, counted apart from the connections turned away) and `sender_check_days.estimated` (a day whose downtime is an upper bound) (audit 2026-10-12, stage 2b) |
**Kind:** *DDL* — schema only; *data* — `INSERT`/`UPDATE` that must stay correct
for operators upgrading from older 2.x images.
---
## Schema at head (v1)
## Schema at head (v2)
| Table | Role |
|-------|------|
| `settings` | Key/value settings of the instance — what is true whoever is signed in (send-log retention). Nothing that belongs to a user |
| `users` | Panel logins. `role` is `global` or `domain`; `email` is the user's own address; `all_domains` / `all_inbound_domains` widen a `domain` user to every domain of that direction |
| `users` | Panel logins. `role` is `global` or `domain`; `email` is the user's own address; `notify` is the user's own switch for notification e-mails (v4); `all_domains` / `all_inbound_domains` widen a `domain` user to every domain of that direction |
| `sessions` | Panel login sessions (token hash, owning user by id and deleted with them, sliding idle expiry) |
| `domains` | Sending (outbound) domains. `dmarc_rua` is where aggregate reports go (`''` = no reports) |
| `user_domains` | Outbound domains assigned to a `domain` user |
@@ -98,9 +110,14 @@ for operators upgrading from older 2.x images.
| `user_inbound_domains` | Inbound domains assigned to a `domain` user — granted separately from the outbound ones |
| `dmarc_reports` | Parsed aggregate report summaries |
| `dmarc_report_records` | Per-source rows inside a report |
| `inbound_spam_log` | Inbound antispam journal: one row per recipient of a message offered on port 25, with the decision (`accept`, `reject`, `tempfail`, `quarantine`) and who made it (`rspamd`, `postfix`, `selfpost`). Metadata only (v2) |
| `inbound_spam_list` | Allow / deny entries for the external filter (`address`, `domain`, `ip`), instance-wide; unique per list (v2) |
| `notify_state` | What the panel has already written to its users about, one row per subject (`dns:<domain>`, `cert`, `queue`, `limit:<domain>`), so that each thing is said once and a restart says nothing again (v4) |
| `send_log_lines` | The `mail.log` lines Postfix wrote under a queue id, with the line's own time; what a message's page shows once the log has rotated. No reference to `send_log`: a line belongs to a row by queue id and time, and goes when no row carries its id (v3) |
`TestBaselineSchema` (`internal/store/schema_test.go`) pins every table, column
and index of this list.
and index of this list; `TestOpenMigratesA20Database` opens a database that
holds the baseline alone.
### What changed against the last 1.x schema
@@ -129,7 +146,7 @@ for the domain. Nothing refers to a user: deleting one changes no domain.
1. Add `internal/store/migrations/00NN_short_name.sql` (next number).
2. Update the table in [Migration chain](#migration-chain), the snapshot at the
top, and the affected rows of [Schema at head](#schema-at-head-v1).
top, and the affected rows of [Schema at head](#schema-at-head-v2).
3. Update `TestBaselineSchema` if tables, columns or indexes change, and add a
test that opens a database at the previous version and migrates it.
4. A *data* migration states in a comment which 2.x versions it is written for.
+269 -15
View File
@@ -26,7 +26,31 @@ the user form did not end that user's sessions, internal errors reached forms a
(see [Authentication and sessions](#authentication-and-sessions) and [Errors
shown to users](#errors-shown-to-users)); two deliberate behaviours are
recorded under Accepted risks (upstream of a delegated inbound domain, profile
saved without the password).
saved without the password). The 2026-10-11 review (Fable) covered the DMARC
report path from port 25 to the report pages, which ran for the first time in
2.0.0 - the 2026-08-19 pass read code that Postfix could not yet start: no
Critical or High finding; a report's size was bounded only as bytes (now 5 000
records, 255 bytes a field), the retention cap was one for the whole server
(now per domain), the delivery command could run a hundred at once (now one),
and the reason logged for a dropped message carried the sender's bytes as they
came (now cut and reduced to printable ASCII). The 2026-10-12 reviews of
inbound-antispam-panel (`2.1.0`: the inbound journal milter, refusals read from
`mail.log`, the filter lists and their pages) were two, by different models
over the same diff. The technical one (Opus) found, outside the feature and
there since `1.9.0`, that **the level-2 rate limit and an application's
client-address list held only for the first message of a connection** - see
[Mail path](#mail-path-security-relevant) - which was reproduced on the test
stand and fixed. The security one (Fable) found no Critical or High: a client's
HELO name could supply the sender, recipient and reply of a journal row and so
make a refusal look like the deny list's (the reply now ends at Postfix's own
`; from=<`); the journal's row cap was applied only every six hours, between
which refusals were written at whatever rate a stranger sent recipients (the
tailer now cuts to the cap every 500 rows); and two behaviours are recorded
under Accepted risks (the allow list keyed on an address anyone can claim, the
list files readable by anyone). A third it named - with the filter down, a
sender's own verdict headers were recorded as the filter's - was closed the
same day at the owner's suggestion: the filter signs each message and a
verdict is read only next to its signature.
(2) **Accepted risks** —
deliberate departures beyond the mandatory, recorded so the decision is not
lost.
@@ -113,6 +137,8 @@ assigned outbound domains and inbound domains **separately**, each list with
| Outbound: domain page, applications, Domain settings (report address, rate limit, export), log | every domain | assigned outbound domains |
| Outbound › DMARC reports | every report, with the server's ingest statistics | reports of assigned domains, without the ingest box |
| Inbound: domain page, upstream, recipients | every inbound domain | assigned inbound domains |
| Inbound › Log | every row, including those of a domain deleted since | rows of assigned inbound domains (by id: not those of an earlier domain of the same name), without the links into the filter lists |
| Inbound › Filter lists; the inbound log's retention | yes | no — 404 |
| Add or delete a domain, either direction; import a domain | yes | no |
| Account, Help | yes | yes |
@@ -126,7 +152,17 @@ assigned outbound domains and inbound domains **separately**, each list with
(`lookupDomain`, `requireInboundDomain`) and lists are filtered by assignment
in the query. Each denied path has a test for a `domain` user, including
another tenant's id.
- The panel will not remove or demote the last `global` user.
- The panel will not remove or demote the last `global` user. The guard is
inside the transaction that deletes or demotes: the write comes first and
the count after it, so two administrators deleting or demoting each other
at the same moment cannot both succeed; first-user setup is one statement
that inserts only into an empty table. A user's name, password, e-mail,
role and assignments are saved in one transaction, so a failure part way
leaves no domain user with nothing assigned.
- A log row - outbound or inbound - belongs to a domain by its id, written
when the row is: the outbound row's from the application that sent it. A
domain deleted and added again starts with an empty log for whoever it is
assigned to; the rows of the deleted one are the global role's alone.
- A `domain` user can export an assigned domain, and an export carries the
working application passwords (encrypted unless the box is unticked): assign
domains with that in mind.
@@ -139,7 +175,7 @@ assigned outbound domains and inbound domains **separately**, each list with
(`originAllowed`, [internal/web/security.go](../internal/web/security.go)):
`Sec-Fetch-Site` must be `same-origin`, or `Origin` must name the panel's own
host; the session cookie is `SameSite` as well.
- There are **no CSRF tokens yet**. The decision, its price and the conditions
- There are **no CSRF tokens**. The decision, its price and the conditions
for revisiting it are in the accepted risks and the ADR below.
### Errors shown to users
@@ -162,6 +198,43 @@ operator can act on it.
- **Not an open relay** — SASL only on 465/587; `reject_unauth_destination`;
`smtpd_sender_login_maps` + `reject_sender_login_mismatch`.
- **An application signs only as itself (`2.4.0`).** OpenDKIM chooses the
key by the `From:` header and Postfix holds only the envelope sender
against the login, so until `2.4.0` an application of one sending domain
could write another registered domain - or, since `2.3.0`, the server's
own name - into `From:` and have the message signed with that name's key
(found by the security review of panel-notifications, 2026-10-12). The
panel's sender check now holds the header to the table the envelope is
held to and refuses anything else with `550 5.7.1`, a domain the server
does not sign for included (the owner's rule: the header may say what the
envelope may, nothing more). It is the first milter of the chain and
strict: a message it could not judge is deferred, never passed on to the
signer. It reads the header as one field holding one address and then
looks at the raw value once more for any other name of OpenDKIM's
`SigningTable`, so that a line its parser and OpenDKIM's could read
differently is refused rather than argued about; outside quoted strings
the line may hold one `@`. That the two agree is tested, not only argued:
the e2e sends every `From:` line the check passes - a list it shares with
the check's unit test, `internal/milter/testdata/from-lines-passed.txt` -
and wants the message signed for the application's domain and no other;
and every line of a second list, lines that name another sender of the
server in a quoted string, a comment, a quoted pair, capitals or an encoded
word (`from-lines-other.txt`), must be refused and never arrive signed for
that sender. The reply to a refusal says that the line names another
sender of the server, not which. That a line naming a domain is refused
when the domain is a sender of this server, and passed when it is not,
still tells an application which domains are hosted here; the probe needs
an application of this server, and refusing every foreign address in a
display name would change the 2.4.0 rule - accepted by the owner
(accepted risks). What this does not promise is what a display
name looks like to a person: a name of another sender written without an
`@` the scan can find (a full-width dot, a space inside the name) passes,
signed with the application's own key - a forged display name is as old
as mail, and no key of another's is involved. What the client wrote
reaches the reply, the log and the send log only as printable ASCII, cut,
and a refused message is one row of the log whatever its recipient count.
Mail the panel submits itself does not pass the check: only root,
`postfix` and the panel can submit locally. Architecture § Sender check.
- **Inbound relay (optional, `INBOUND_RELAY_ENABLE`)** — port 25 is not an
open relay either: SASL is off; `smtpd_relay_restrictions` /
`smtpd_recipient_restrictions` are `reject_unauth_destination` and
@@ -171,6 +244,47 @@ operator can act on it.
refuse unknown addresses at RCPT (no backscatter). OpenDKIM is not attached
on inbound. An optional antispam milter is inbound-only; default action is
fail-open (`accept`).
- **The panel's milters judge every message of a connection.** Postfix
introduces a client to a milter once per connection, and the milter library
replaces the milter's session after any reply that is not "continue". Until
`2.1.0` the journal milter knew the client's address only from that
introduction, and both of its checks passed a message that came without one:
a second `MAIL FROM` after a refused one, or any message after the first on
one connection, got past the level-2 limit and past an application's
client-address list. Now the address comes with every `MAIL FROM`
(`milter_mail_macros` carries `{client_addr}`), the level-2 limit does not
ask for it at all, an application with a client-address list is refused when
no address is known, and neither milter replies anything but "continue" to a
message it lets through. `test/e2e` sends two `MAIL FROM` on one connection.
- **The inbound journal is written from what strangers send.** The inbound
journal milter is the last on port 25, read-only and attached fail-open; it
keeps of the filter's verdict only what its patterns recognise. **A verdict
is read only next to the filter's signature.** The filter is attached so
that mail flows when it is down, and then nothing removes verdict headers a
sender wrote himself. Nor would rspamd itself by default for a client it
counts as local - a private address, which every client has when Docker's
proxy relays port 25 - or an authenticated one: `skip_local` and
`skip_authenticated` (rspamd 3.11.1, `milter_headers.lua`, `skip_wanted()`)
leave the sender's headers in place, while the signing routine below runs
regardless; both are turned off in `milter_headers.conf` (assumptions
C-R7, tried on 2026-10-14). So rspamd adds a third header, `X-SelfPost-Filter`:
the HMAC-SHA256 of the message's queue id under a key SelfPost generates
(`data/antispam/filter.key`) and the sidecar reads from the directory it
mounts. The journal milter recomputes it at end-of-message; with no
signature, or one that does not fit, the row is that of mail nobody judged.
The signature goes on to the upstream with the mail and is of no use there:
it fits one queue id. Refusals are
parsed from `mail.log` from the head of the line, with the reply ending at
Postfix's own `; from=<` so that the HELO name after it - the client's text,
logged as written - cannot stand in for the sender, the recipient or the
reply; an address with a quote, a bracket or a space in it is not kept.
Only a recipient at an inbound domain makes a row. The journal is cut to
10 000 rows every 500 rows the tailer writes, and by age every six hours.
- **Filter lists** reach another container as files: values are whitelisted on
the way in (address, domain, canonical IP or network; no `/0`, no
IPv4-mapped network), the writer refuses a stored value with whitespace, a
newline or `#`, and a change of the table and the rewrite of the six files
are one step under a lock.
- TLS is mandatory before credentials are transmitted (465 wrapper / 587
`encrypt`).
- `TRUSTED_PROXY_CIDR` — only explicitly trusted proxies may supply
@@ -183,6 +297,11 @@ operator can act on it.
private key from `certs/` when present; a domain export carries the DKIM key
and **working** application passwords in the clear (otherwise a transfer
without recreating credentials would be impossible).
- Since 2.2.0 a full backup also carries, in the database, what Postfix logged
about each message still in the send log (`send_log_lines`): the submitting
client's address and SASL login, the message id, the receiving server and
its reply. `mail.log` itself stays out of a backup; these lines are state
the panel shows, kept for the send log's retention and no longer.
- Both downloads are encrypted with a password unless the box on the form is
unticked (it is ticked by default):
scrypt (N=2¹⁵, r=8, p=1) → AES-256-GCM, streamed in 64 KiB chunks, each
@@ -216,7 +335,9 @@ deferred item from the roadmap.
against, and a strict mode would not "protect" such a client, it would
simply break the panel in it. Tightening is one line in `originAllowed`
([internal/web/security.go](../internal/web/security.go)): return `false`
instead of `true` in the "neither header present" branch.
instead of `true` in the "neither header present" branch. That tightening
is agreed (owner, 2026-10-13) as
[origin-strict](roadmap.md#origin-strict); this bullet goes when it ships.
- **Session-bound CSRF tokens are not implemented.** The origin check closes the
neighbouring-subdomain case but depends on browser behaviour; a token does
not. The price is a hidden field in roughly two dozen forms. The trigger to
@@ -225,10 +346,12 @@ deferred item from the roadmap.
(see the ADR below). A token would not save the panel from XSS inside it
either: code executing in the panel's origin sends the request itself —
against that, `html/template` auto-escaping and CSP do the work, which is why
templates must contain no inline scripts and no inline styles. Tracked as
[csrf-tokens](roadmap.md#csrf-tokens): implementing that item replaces this
bullet and the ADR below, and dropping it leaves them as the single record of
the decision. The two must not disagree.
templates must contain no inline scripts and no inline styles. Decided by
the owner on 2026-10-13: no tokens; the roadmap item that asked for them is
dropped and the origin check is made strict instead
([origin-strict](roadmap.md#origin-strict)). This bullet and the ADR below
are the single record of the decision, and the ADR's **Revisit if** says
when tokens are built after all.
- **Destructive-action confirmation (`data-confirm`) is JavaScript-only.**
Delete, regenerate-password, and clear-rate-limit forms carry a
`data-confirm` prompt handled entirely in
@@ -274,20 +397,73 @@ deferred item from the roadmap.
reporters are arbitrary third parties and the mailbox address is published in
DNS. The ingest already constrains the meaningful part — a report cannot be
filed under a domain this instance does not host, and the claimed domain is
cross-checked against the `+tag` of the delivery address — and the data is
advisory statistics only; nothing enforces off it. The trigger to revisit is
cross-checked against the `+tag` of the delivery address (effective since
2.0.0: before it the recipient never reached the command) — and the data is
advisory statistics only; nothing enforces off it. What a stranger can do is
limited to a domain that chose a SelfPost-hosted address, through that
address: file reports under it, replace one whose reporter and id they know,
and fill that domain's 500 — not another domain's. A message that is not a
report the relay keeps is dropped, not bounced (the envelope sender is the
author's to write); a genuine report the parser cannot read is lost the same
way, with a count and a line in the mail log. The trigger to revisit is
any feature that lets report contents drive an automatic action (policy
changes, rate limiting, alerting an operator into a config change).
- **The application auth-IP allow-list is enforced fail-open.** The check runs
in the journal milter (`internal/milter/authip.go`) with
`default_action=accept` and skips on store errors, so if the milter is down
the restriction is not applied; the SASL login itself always succeeds. This
`default_action=accept` and skips on store errors, so if the journal milter
does not answer the restriction is not applied; the SASL login itself
always succeeds. (While the milter is up the list holds for every message,
and an unknown client address is refused - see Mail path; what is accepted
here is the milter not answering. Since `2.4.0` a panel that is down
altogether defers mail through the sender check, which shares its process;
what is left of this window is a journal milter that hangs or a database
that errs while the check beside it still answers.) This
is documented in [guide.md](guide.md) as **not an authentication boundary** —
it is a blast-radius limiter for a leaked application password, on the same
fail-open path as level-2 rate limiting, and a milter outage already stops
the journal from recording sends at all. The trigger to revisit is a
requirement for the allow-list to hold as a security boundary — at which
point it must move to the SASL/smtpd layer and fail closed.
- **The key the spam filter signs with is readable by anyone on the host.**
The inbound journal reads the filter's verdict only next to the filter's
signature (see Mail path), and the key for it, `data/antispam/filter.key`,
is 0644 like the list files beside it: the filter is another user in another
container. Someone with a shell on the host could therefore sign a verdict
of his own - into a monitoring page, on a host where he could do far more.
Senders on the internet, whom the signature is for, cannot read it. Trigger
to revisit: the same as for the list files - a shared host.
- **An allow entry for a sender address or domain trusts what the sending
server says.** The filter matches the envelope sender (only that: a
bounce's `From:` header is not looked at, `extract_from = "smtp"`), which
is not authenticated: anyone who knows or guesses an allowed address can send as it
and is given the entry's weight, which no spam score outweighs. The page's
Help and the guide say so and recommend an entry for the client's address or
network instead. Deny entries have no such problem - claiming to be a denied
sender only gets mail refused - and deny wins over allow. Trigger to
revisit: allow entries being used as the main way to receive mail from a
sender, at which point the rule should also require the sender's SPF or
DKIM to pass.
- **The filter lists are files readable by anyone on the host.**
`/data/antispam` is 0755 and its six files 0644, unlike the rest of `/data`:
the filter runs under another user in another container. They hold sender
addresses, domains and networks - who the operator blocks and whom he lets
through - but no message data. The inbound journal itself is in the
database, 0640. Trigger to revisit: a shared host, where the sidecar should
instead be given the files through a group it shares with the panel.
- **An application can learn which domains are senders of this server.**
A `From:` whose display name or comment names an address at a domain this
server signs for is refused; one naming any other domain passes. The reply
no longer names the domain (audit 2026-10-12, L8), but refused or passed
answers the question for whoever sends the line. Accepted by the owner,
2026-10-14: it needs an application of this server - a trusted tenant
guessing names one at a time, as it already had to before L8, when the
reply only said in words what the line had named - and closing it means
refusing every foreign address in a display name, which would change his
2.4.0 rule (a stranger's address in the name passes). Trigger to revisit:
tenants who are not trusted, or a display-name rule decided anew.
- ~~**A row of the inbound log belongs to a domain by name.**~~ Struck in
`2.5.0` (audit 2026-10-12, L12): both logs now hold a domain by id, see
§ Roles.
- **Level-2 in-flight reservations can outlive a dropped SMTP session until TTL.**
The journal milter cannot observe connection-close directly through
go-milter, so if a client disconnects without an explicit `Abort`, its
@@ -303,7 +479,70 @@ deferred item from the roadmap.
the file is inaccessible to others. The log holds envelope addresses and
client IPs, but neither message bodies nor headers; it is excluded from
backups (`log/` is skipped) so that a dump stays state rather than
diagnostics.
diagnostics. Since 2.2.0 the tailer copies the lines of queued outbound
messages into the database for the message's page, and those are in a
backup (§ Backup and domain export); mail offered on port 25 is not copied.
- **The panel submits mail of its own.** Notifications leave through
`sendmail(1)` run by the unprivileged panel user: a fixed argument list, the
two addresses after `--`, no shell. The recipient is a panel user's account
e-mail, validated when it was saved and checked again before it reaches a
header or the command line; the subject and body are composed by the panel
from its own readings. Two of those readings can carry text from outside - a
DNS check's finding, and a queued message's sender address - and both are
reduced to printable ASCII and cut before they are printed, so a
notification's body can contain an address a stranger chose, as a few plain
characters and nothing else. A refused-mail notification names the domain
and the applications from the registry, never the domain a client wrote in
`MAIL FROM`: that string is not yet held against the login when the milter
refuses, and the first build took it for the subject - an application over
its limit could have made the panel write a message per made-up domain, to
another tenant's users among others (security review, 2026-10-12; fixed
before release). The panel's membership in `postdrop` is what lets it
submit: under `no-new-privileges` the setgid bit of `postdrop` does not
take, so only root, `postfix` and the panel can. Notifications are signed
with a key of the server's own name and bypass the level-1 and level-2
limits. What bounds them: the subjects are the registered domains, the
certificate and the queue; a domain's DNS can be reported and cleared about
a dozen times a day at most by someone who flaps the zone; a domain's
limits once a day; and no more than 60 messages leave in any hour whatever
the rules arrive at.
- **The sender check stops mail while the panel does not answer.** It is a
role of the panel process, attached with `default_action=tempfail`, so a
panel that is down or slower than the milter timeouts (15 s to connect
and for a command, 30 s for the verdict at the end of the headers) defers
every submission: availability of sending now depends on the panel,
which it did not before `2.4.0`. Chosen by the owner over the alternative,
a check that is absent whenever the panel is - that one could be switched
off by whoever can make the panel slow. Supervisord restarts the panel at
once and takes the container down after three failed starts, so the
outage is seconds or the whole instance; Server › Health counts the
downtime and the mail deferred for it. Trigger to revisit: those numbers
not staying at nothing - then the check moves into a process of its own
([plans/from-header-binding.md](plans/from-header-binding.md)).
- **A malformed packet on a milter socket of the panel ends the panel.** The
milter library indexes into an empty packet and nobody recovers the panic,
so the process exits and supervisord starts it again. The sockets are
reachable only inside the container, by the group Postfix is in (mode
0660); the journal milter always had this, and since `2.4.0` the seconds
of the restart defer mail instead of passing it unjournalled (technical
review, 2026-10-12). Trigger to revisit: anything untrusted able to open
a Unix socket in the container.
- **When the hostname is itself a sending domain,** the server's mail is
signed with that domain's key. Whoever may export the domain has the key,
and its applications may send as `postmaster@<hostname>` by right. The
guide says so; a hostname that is nobody's sending domain avoids it.
- **A message's page can show another message's log lines after a queue-id
collision.** Postfix's short queue ids repeat: the id is the microsecond of
arrival and the queue file's inode, and inodes come round within seconds.
The page takes the stored lines of the row's queue id dated within the
row's own queue lifetime (about six days), so a second message has to get
the same microsecond on the same inode in that span - about one in a
million per reuse of the inode, and not something a sender can aim. When it
happens, a `domain` user could read the log lines of a message that is not
theirs. The search of the current `mail.log` that the page did before
2.2.0 had the same exposure over a day. Trigger to revisit: a report of
foreign lines on a page, or a reason to turn on `enable_long_queue_ids`,
which ends it.
- **A `domain` user may point an inbound domain's upstream at any host and
port.** The upstream form checks that the host is a well-formed name or IP
and the port is 1–65535; loopback, private and link-local addresses are
@@ -319,8 +558,13 @@ deferred item from the roadmap.
field to the global role, is the fix.
- **Account's profile (username, e-mail) is saved without the password.**
Saving the profile does not ask for the current password, as changing the
password does. Today the e-mail is only the address the *Use my e-mail*
button fills in, so a stolen session gains nothing it did not have. It
password does. Since 2.3.0 the e-mail is where notifications go, so a stolen
session can turn them to another address: the thief then learns what they
say - which domain's DNS is wrong, which application is refused, when the
certificate expires, sender addresses of old queued mail - and the user
stops hearing it. Nothing in a notification gives access. A `domain` user
can likewise point signed mail of this server at a third party's address,
a dozen messages a day per assigned domain at most. It
becomes a requirement to ask for the current password when changing
`users.email` as soon as [password-reset](roadmap.md#password-reset) mails a
link to it — from then on whoever can change the address could take the
@@ -372,6 +616,16 @@ whose browsers/devices are not vetted) — at that point cross-role request
forgery inside the panel would need its own analysis, separate from the
external-site case this ADR covers.
**Decision of 2026-10-13 (owner).** Asked to choose between this ADR, tokens,
and a strict origin check, the owner chose the ADR with the strict check
([origin-strict](roadmap.md#origin-strict)) and no tokens. Session-bound tokens
are the answer to return to - a hidden field in every state-changing form and
htmx path, checked against the session, on top of the origin check - when
either condition above comes true, and also when: a browser the panel must
support is found to send neither header or to send them wrongly; the panel
gains a state-changing request that a browser sends without these headers; or
an outside review requires a defence that the server alone holds.
## How this list grows
The pre-release vulnerability review (history — CHANGELOG `[0.5.0]` Security)
+7 -7
View File
@@ -1,22 +1,22 @@
module github.com/mixeme/selfpost
go 1.26
go 1.26.0
require (
github.com/emersion/go-milter v0.4.1
golang.org/x/crypto v0.54.0
modernc.org/sqlite v1.53.0
golang.org/x/crypto v0.58.0
modernc.org/sqlite v1.60.1
)
require (
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/emersion/go-message v0.18.1 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/mattn/go-isatty v0.0.24 // indirect
github.com/ncruces/go-strftime v1.0.0 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
golang.org/x/sys v0.47.0 // indirect
modernc.org/libc v1.73.4 // indirect
golang.org/x/sys v0.49.0 // indirect
modernc.org/libc v1.77.1 // indirect
modernc.org/mathutil v1.7.1 // indirect
modernc.org/memory v1.11.0 // indirect
modernc.org/memory v1.12.1 // indirect
)
+26 -27
View File
@@ -4,14 +4,14 @@ github.com/emersion/go-message v0.18.1 h1:tfTxIoXFSFRwWaZsgnqS1DSZuGpYGzSmCZD8SK
github.com/emersion/go-message v0.18.1/go.mod h1:XpJyL70LwRvq2a8rVbHXikPgKj8+aI0kGdHlg16ibYA=
github.com/emersion/go-milter v0.4.1 h1:gLs9QD0zEHF8omgEw8M+aGz6iwBNpWLAcwgSur0ra4M=
github.com/emersion/go-milter v0.4.1/go.mod h1:erCQVl0mH4SX9jEvwe+wyndit0rQtmvMLH86V6NGtkI=
github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e h1:ijClszYn+mADRFY17kjQEVQ1XRhq2/JR1M3sGqeJoxs=
github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e/go.mod h1:boTsfXsheKC2y+lKOCMpSfarhxDeIzfZG1jqGcPl3cA=
github.com/google/pprof v0.0.0-20260802141513-ef3492d7dac3 h1:LMLX+LgTNWpfvCBdFebv6EsYotImrt/Ppc5cXIriCSo=
github.com/google/pprof v0.0.0-20260802141513-ef3492d7dac3/go.mod h1:jl5iWTm0/hd5PjEYEOuwAJ57L/CibdZfrqZ5XA5GrCk=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/mattn/go-isatty v0.0.24 h1:tGZZoVgT/KiqK1c8ocVLeDS8BSWMRd47J3Lbz7vsReI=
github.com/mattn/go-isatty v0.0.24/go.mod h1:nMCL3Zebbrt45jsMDgnfIwz6ydEQApk5oEI3HqDio6A=
github.com/ncruces/go-strftime v1.0.0 h1:HMFp8mLCTPp341M/ZnA4qaf7ZlsbTc+miZjCLOFAw7w=
github.com/ncruces/go-strftime v1.0.0/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
@@ -19,12 +19,12 @@ github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qq
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.54.0 h1:YLIA59K4fiNzHzjnZt2tUJQjQtUWfWbeHBqKtk3eScw=
golang.org/x/crypto v0.54.0/go.mod h1:KWL8ny2AZdGR2cWmzeHrp2azQPGogOv+HeQaVEXC2dk=
golang.org/x/crypto v0.58.0 h1:COkYLr4k7nDI4r6QtMML2AfzeIswWHAJthLS6K/M/54=
golang.org/x/crypto v0.58.0/go.mod h1:Xh+kl5A+M0gMiWDytH2Hhr5Z7WbNCpv/eB/EFgfSk+w=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/mod v0.36.0 h1:JJjpVx6myfUsUdAzZuOSTTmRE0PfZeNWzzvKrP7amb4=
golang.org/x/mod v0.36.0/go.mod h1:moc6ELqsWcOw5Ef3xVprK5ul/MvtVvkIXLziUOICjUQ=
golang.org/x/mod v0.41.0 h1:qJmnOUb4YB+FsEuM3HcWucdZASCPGhsX6uljO6pog0c=
golang.org/x/mod v0.41.0/go.mod h1:Ek9pY8RKWXwsWvd3rQiHYtMqkjSUV+s1Rj7j4H5Ur6o=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
@@ -32,17 +32,16 @@ golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.20.0 h1:e0PTpb7pjO8GAtTs2dQ6jYa5BWYlMuX047Dco/pItO4=
golang.org/x/sync v0.20.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
golang.org/x/sync v0.23.0 h1:KameEIfc1IkluZyXWLn39Wd4tURc6GbCiISGiZm2bQk=
golang.org/x/sync v0.23.0/go.mod h1:sUUOizhqBxiL6pEWpqNLUiaJn1ShEbZ6BBqskPbjZm0=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs=
golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/sys v0.49.0 h1:XbzkgYJHdqh/8m2Uu0W/dQv8nktxx4BFHp1M0gROTXA=
golang.org/x/sys v0.49.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
@@ -55,33 +54,33 @@ golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGm
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
golang.org/x/tools v0.45.0 h1:18qN3FAooORvApf5XjCXgsuayZOEtXf6JK18I3+ONa8=
golang.org/x/tools v0.45.0/go.mod h1:LuUGqqaXcXMEFEruIVJVm5mgDD8vww/z/SR1gQ4uE/0=
golang.org/x/tools v0.50.0 h1:c2ifzfcuY7L90lZ2aKd8S4K2NpASF08SZx9ZuJkHmSU=
golang.org/x/tools v0.50.0/go.mod h1:7ulVMw3831Mwi5EZD6RomGyffr4VFjuNYXf2BbCEAV0=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
modernc.org/cc/v4 v4.28.4 h1:Hd/4Es+MBj+/7hSdZaisNyu6bv3V0Dp2MdllyfqaH+c=
modernc.org/cc/v4 v4.28.4/go.mod h1:OnovgIhbbMXMu1aISnJ0wvVD1KnW+cAUJkIrAWh+kVI=
modernc.org/ccgo/v4 v4.34.4 h1:OVnSOWQjVKOYkFxoHYB+qQmSHK5gqMqARM+K9DpR/Ws=
modernc.org/ccgo/v4 v4.34.4/go.mod h1:qdKqE8FNIYyysougB1RX9MxCzp5oJOcQXSobANJ4TuE=
modernc.org/cc/v4 v4.29.7 h1:q+NXGJ0bK3b4TXFYQQVr9pYETGnmwFWkrUzJnMya/Tg=
modernc.org/cc/v4 v4.29.7/go.mod h1:OnovgIhbbMXMu1aISnJ0wvVD1KnW+cAUJkIrAWh+kVI=
modernc.org/ccgo/v4 v4.36.1 h1:ZNIUZAryN0UgnJwtyxrdEzcFc3yD4Cu4AzjfPXsLsIE=
modernc.org/ccgo/v4 v4.36.1/go.mod h1:rrtGc2QkS239nYb/mQNuBMyjq3/y3ZXWbBjPoV3wqzA=
modernc.org/fileutil v1.4.0 h1:j6ZzNTftVS054gi281TyLjHPp6CPHr2KCxEXjEbD6SM=
modernc.org/fileutil v1.4.0/go.mod h1:EqdKFDxiByqxLk8ozOxObDSfcVOv/54xDs/DUHdvCUU=
modernc.org/gc/v2 v2.6.5 h1:nyqdV8q46KvTpZlsw66kWqwXRHdjIlJOhG6kxiV/9xI=
modernc.org/gc/v2 v2.6.5/go.mod h1:YgIahr1ypgfe7chRuJi2gD7DBQiKSLMPgBQe9oIiito=
modernc.org/gc/v3 v3.1.3 h1:6QAplYyVO+KdPW3pGnqmJDUxtkec8ooEWvks/hhU3lc=
modernc.org/gc/v3 v3.1.3/go.mod h1:HFK/6AGESC7Ex+EZJhJ2Gni6cTaYpSMmU/cT9RmlfYY=
modernc.org/gc/v3 v3.1.5 h1:21ldfPfRYE31Tb7B3mwAK8gy1AxP4+dKjrOQPfqakoc=
modernc.org/gc/v3 v3.1.5/go.mod h1:HFK/6AGESC7Ex+EZJhJ2Gni6cTaYpSMmU/cT9RmlfYY=
modernc.org/goabi0 v0.2.0 h1:HvEowk7LxcPd0eq6mVOAEMai46V+i7Jrj13t4AzuNks=
modernc.org/goabi0 v0.2.0/go.mod h1:CEFRnnJhKvWT1c1JTI3Avm+tgOWbkOu5oPA8eH8LnMI=
modernc.org/libc v1.73.4 h1:+ra4Ui8ngyt8HDcO1FTDPWlkAh6yOdaO2yAoh8MddQA=
modernc.org/libc v1.73.4/go.mod h1:DXZ3eO8qMCNn2SnmTNCiC71nJ9Rcq3PsnpU6Vc4rWK8=
modernc.org/libc v1.77.1 h1:Ct8j47QtiZ1Enj2DtFXQtUqrPCAjdCmPjtCuvrYQ0Hs=
modernc.org/libc v1.77.1/go.mod h1:87/pZ4L6nD1zqW4nItuS12YO7hN1igAah34xjnQo/W0=
modernc.org/mathutil v1.7.1 h1:GCZVGXdaN8gTqB1Mf/usp1Y/hSqgI2vAGGP4jZMCxOU=
modernc.org/mathutil v1.7.1/go.mod h1:4p5IwJITfppl0G4sUEDtCr4DthTaT47/N3aT6MhfgJg=
modernc.org/memory v1.11.0 h1:o4QC8aMQzmcwCK3t3Ux/ZHmwFPzE6hf2Y5LbkRs+hbI=
modernc.org/memory v1.11.0/go.mod h1:/JP4VbVC+K5sU2wZi9bHoq2MAkCnrt2r98UGeSK7Mjw=
modernc.org/memory v1.12.1 h1:nFMiWrpStgZczNl6XI9GnIk/rWhYIyHGUaR04pGbp9g=
modernc.org/memory v1.12.1/go.mod h1:/JP4VbVC+K5sU2wZi9bHoq2MAkCnrt2r98UGeSK7Mjw=
modernc.org/opt v0.2.0 h1:tGyef5ApycA7FSEOMraay9SaTk5zmbx7Tu+cJs4QKZg=
modernc.org/opt v0.2.0/go.mod h1:03fq9lsNfvkYSfxrfUhZCWPk1lm4cq4N+Bh//bEtgns=
modernc.org/sortutil v1.2.1 h1:+xyoGf15mM3NMlPDnFqrteY07klSFxLElE2PVuWIJ7w=
modernc.org/sortutil v1.2.1/go.mod h1:7ZI3a3REbai7gzCLcotuw9AC4VZVpYMjDzETGsSMqJE=
modernc.org/sqlite v1.53.0 h1:20WG8N9q4ji/dEqGk4uiI0c6OPjSeLTNYGFCc3+7c1M=
modernc.org/sqlite v1.53.0/go.mod h1:xoEpOIpGrgT48H5iiyt/YXPCZPEzlfmfFwtk8Lklw8s=
modernc.org/sqlite v1.60.1 h1:/blz53O951KWFOso4QQvEs/Fq6cDBKLtMVrYNSeJVKw=
modernc.org/sqlite v1.60.1/go.mod h1:1dIoEagfDE72QytD5scH1lxARtaUgKgHC/NuApA27r0=
modernc.org/strutil v1.2.1 h1:UneZBkQA+DX2Rp35KcM69cSsNES9ly8mQWD71HKlOA0=
modernc.org/strutil v1.2.1/go.mod h1:EHkiggD70koQxjVdSBM3JKM7k6L0FbGE5eymy9i3B9A=
modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y=
+136 -5
View File
@@ -4,23 +4,45 @@ import (
"bufio"
"bytes"
"encoding/hex"
"errors"
"fmt"
"io"
"os"
"os/exec"
"strings"
"sync"
)
// SASLDB manages the Cyrus SASL account database (sasldb2) the panel maintains
// for application credentials (architecture.md § Mail path). The panel is the
// only writer; Postfix reads it to authenticate SMTP clients. Accounts are
// created and removed with the standard saslpasswd2 tool.
//
// saslpasswd2 is never pointed at the live file. Cyrus SASL 2.1.28 opens
// sasldb2 with no Berkeley DB environment, so without locks
// (sasldb/db_berkeley.c, berkeleydb_open(): db_create(mbdb, NULL, 0)), and a
// write puts and deletes in the file itself (_sasldb_putdata()); smtpd opens
// and closes it for every lookup (_sasldb_getdata()), and the backup reads it
// whenever it runs. Either could read a page half written: a spurious 535 for
// an application, or a torn sasldb2 in a backup whose restore breaks every
// application's AUTH (assumptions C-S3, C-Q3). So each change is made to a
// copy, which is then renamed over the file (replaceSASLDB): a reader opens
// the old file or the new one, whole, and needs no lock. mu keeps the
// panel's own changes from copying the same old file at once.
type SASLDB struct {
path string // sasldb2 file, under /data so it survives restarts
realm string // SASL realm, so lookups match what Postfix's SASL uses
mu sync.Mutex
// run executes saslpasswd2. It is a field so tests can substitute a fake;
// the default shells out to the real binary via runSaslpasswd2.
run func(args []string, stdin []byte) error
// replace makes a change to the database through a copy: edit gets the
// path to give saslpasswd2. The default is replaceSASLDB.
replace func(path string, edit func(path string) error) error
// dump reads the raw sasldb2 as db_dump key/value pairs (Berkeley DB). It is
// a field so tests can substitute a fake; the default runs db_dump.
dump func(path string) ([]byte, error)
@@ -29,13 +51,96 @@ type SASLDB struct {
// NewSASLDB builds a manager for the sasldb2 at path with the given realm. The
// realm should match SELFPOST_HOSTNAME so the account identity lines up with
// Postfix's SASL configuration.
//
// Postfix 3.7.11, src/xsasl/xsasl_cyrus_server.c, xsasl_cyrus_server_create():
// sasl_server_new() gets $myhostname and, with smtpd_sasl_local_domain empty,
// no user realm; Cyrus SASL 2.1.28, plugins/sasldb.c,
// sasldb_auxprop_lookup(), then looks a login without "@" up under the
// server's name (common/plugin_common.c, _plug_parseuser()).
func NewSASLDB(path, realm string) *SASLDB {
return &SASLDB{path: path, realm: realm, run: runSaslpasswd2, dump: dumpSASLDB}
return &SASLDB{path: path, realm: realm, run: runSaslpasswd2, dump: dumpSASLDB, replace: replaceSASLDB}
}
// change runs saslpasswd2 with args on a copy of the database and puts the
// copy in its place; at in args stands for the path of the copy.
func (s *SASLDB) change(args []string, at int, stdin []byte) error {
s.mu.Lock()
defer s.mu.Unlock()
return s.replace(s.path, func(path string) error {
a := append([]string(nil), args...)
a[at] = path
return s.run(a, stdin)
})
}
// replaceSASLDB copies the database at path to path+".new", lets edit change
// the copy, and renames it over path. With no database yet, edit gets a path
// where none exists, and saslpasswd2 creates one there (db_berkeley.c,
// berkeleydb_open(): DB_CREATE for a writer). The copy is a byte copy of a
// file no one else writes, which is a whole Berkeley DB file: smtpd and
// db_dump only read it. The new file is 0640 in the setgid directory, group
// selfpost, as build/entrypoint.sh sets the old one, so Postfix reads it.
// A copy left by a panel killed mid-change is removed first.
func replaceSASLDB(path string, edit func(path string) error) error {
tmp := path + ".new"
if err := os.Remove(tmp); err != nil && !errors.Is(err, os.ErrNotExist) {
return fmt.Errorf("remove %s: %w", tmp, err)
}
if err := copySASLDB(path, tmp); err != nil {
_ = os.Remove(tmp)
return err
}
if err := edit(tmp); err != nil {
_ = os.Remove(tmp)
return err
}
// Read-write: fsync of a file opened read-only is refused on Windows.
f, err := os.OpenFile(tmp, os.O_RDWR, 0)
if err == nil {
err = f.Chmod(0o640)
if err == nil {
err = f.Sync()
}
if cerr := f.Close(); err == nil {
err = cerr
}
}
if err != nil {
_ = os.Remove(tmp)
return fmt.Errorf("finish %s: %w", tmp, err)
}
if err := os.Rename(tmp, path); err != nil {
_ = os.Remove(tmp)
return fmt.Errorf("rename %s -> %s: %w", tmp, path, err)
}
return nil
}
// copySASLDB copies path to tmp; with no file at path it makes no copy.
func copySASLDB(path, tmp string) error {
src, err := os.Open(path)
if errors.Is(err, os.ErrNotExist) {
return nil
}
if err != nil {
return fmt.Errorf("open %s: %w", path, err)
}
defer src.Close()
dst, err := os.OpenFile(tmp, os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0o640)
if err != nil {
return fmt.Errorf("create %s: %w", tmp, err)
}
if _, err := io.Copy(dst, src); err != nil {
dst.Close()
return fmt.Errorf("copy %s: %w", path, err)
}
return dst.Close()
}
// Set creates or updates an application's SASL account with the given password
// (architecture.md § Mail path). Used both at creation and when a password is
// regenerated; saslpasswd2 overwrites an existing entry in place.
// regenerated; saslpasswd2 overwrites an existing entry (in the copy, see
// SASLDB).
//
// The password is passed to saslpasswd2 on stdin (never as an argument, so it
// cannot leak through the process table or logs). The login is passed as a
@@ -46,6 +151,11 @@ func (s *SASLDB) Set(login, password string) error {
if err := validateLogin(login); err != nil {
return err
}
// Cyrus SASL 2.1.28, utils/saslpasswd.c, main(): getopt(), which ends the
// options at "--"; -p reads the password with one read() and no prompt
// (read_password()); -c is SASL_SET_CREATE, which lib/server.c,
// sasl_setpass(), checks against nothing - an existing entry is written
// over (sasldb/db_berkeley.c, _sasldb_putdata(), put with no flags).
// -p: read the passphrase from stdin (pipe mode, no tty prompt).
// -c: create the account / set the password.
// -f: operate on our sasldb2 rather than the system default path.
@@ -54,7 +164,7 @@ func (s *SASLDB) Set(login, password string) error {
// whitelist already forbids nothing that getopt would eat, but a login
// starting with '-' is legal there — this keeps it an operand).
args := []string{"-p", "-c", "-f", s.path, "-u", s.realm, "--", login}
if err := s.run(args, []byte(password)); err != nil {
if err := s.change(args, 3, []byte(password)); err != nil {
return fmt.Errorf("saslpasswd2 set %q: %w", login, err)
}
return nil
@@ -62,13 +172,18 @@ func (s *SASLDB) Set(login, password string) error {
// Delete removes an application's SASL account (product.md). A missing account
// is not treated as an error, so deletion is idempotent and safe to retry.
//
// Cyrus SASL 2.1.28, plugins/sasldb.c, sasldb_auxprop_store(): "Deleting
// something which is not there is not an error" - the SASL_NOUSER of
// sasldb/db_berkeley.c, _sasldb_putdata(), becomes SASL_OK, so saslpasswd2 -d
// on a missing login exits 0 (utils/saslpasswd.c, main()).
func (s *SASLDB) Delete(login string) error {
if err := validateLogin(login); err != nil {
return err
}
// -d: delete the account. "--" as in Set: the login is always an operand.
args := []string{"-d", "-f", s.path, "-u", s.realm, "--", login}
if err := s.run(args, nil); err != nil {
if err := s.change(args, 2, nil); err != nil {
return fmt.Errorf("saslpasswd2 delete %q: %w", login, err)
}
return nil
@@ -110,6 +225,17 @@ func (s *SASLDB) Secret(login string) (string, error) {
// parseSASLSecret scans db_dump's byte-value output for the userPassword entry
// keyed by (login, realm). sasldb2 keys are NUL-separated tuples
// "<login>\0<realm>\0<property>"; the matching value is the stored password.
//
// Cyrus SASL 2.1.28: sasldb/allockey.c, _sasldb_alloc_key(), builds the key
// with no NUL at its end; plugins/sasldb.c, sasldb_auxprop_store(), stores the
// value with strlen(); lib/server.c, sasl_setpass(), stores the plaintext
// userPassword unless SASL_SET_NOPLAIN (saslpasswd2 -n, which Set does not
// pass). Built against Berkeley DB (m4/sasldb.m4 auto_detect; debian/rules
// records BDB_VERSION). Berkeley DB 5.3 Programmer's Reference, "Dump output
// formats" (docs.oracle.com/cd/E17076_02/html/programmer_reference/
// dumpload_format.html): header lines up to HEADER=END, then for a hash or
// btree database key and data lines in turn, each a single space and hex,
// and DATA=END.
func parseSASLSecret(dump []byte, login, realm string) (string, bool, error) {
sc := bufio.NewScanner(bytes.NewReader(dump))
// sasldb2 records are tiny, but raise the line cap so a long hex line is
@@ -186,8 +312,13 @@ func dumpSASLDB(path string) ([]byte, error) {
// WithRunner replaces the command that runs saslpasswd2 and returns the same
// manager. It exists for tests outside this package that need an application
// service without the real binary; the panel never calls it.
// service without the real binary; the panel never calls it. With no real
// database either, the runner is given the database's own path, not a copy.
func (s *SASLDB) WithRunner(run func(args []string, stdin []byte) error) *SASLDB {
s.run = run
s.replace = inPlace
return s
}
// inPlace hands edit the path itself: for tests whose runner touches no file.
func inPlace(path string, edit func(path string) error) error { return edit(path) }
+114
View File
@@ -0,0 +1,114 @@
package app
import (
"errors"
"os"
"path/filepath"
"runtime"
"testing"
)
// A change is made to a copy and renamed over the database: saslpasswd2
// never gets the live file, and a reader that opened the old file keeps
// reading it whole (assumptions C-S3).
func TestSASLChangeGoesThroughACopy(t *testing.T) {
dir := t.TempDir()
db := filepath.Join(dir, "sasldb2")
if err := os.WriteFile(db, []byte("old database"), 0o640); err != nil {
t.Fatal(err)
}
// smtpd in the middle of a lookup. Windows refuses a rename over an open
// file; the container is Linux, where the reader keeps the old inode.
var reader *os.File
if runtime.GOOS != "windows" {
var err error
if reader, err = os.Open(db); err != nil {
t.Fatal(err)
}
defer reader.Close()
}
s := NewSASLDB(db, "mail.example.com")
var gotPath string
s.run = func(args []string, stdin []byte) error {
gotPath = args[3]
before, err := os.ReadFile(gotPath)
if err != nil {
return err
}
if string(before) != "old database" {
t.Errorf("the copy holds %q, want the database's bytes", before)
}
return os.WriteFile(gotPath, []byte("new database"), 0o600)
}
if err := s.Set("alerts", "pw"); err != nil {
t.Fatalf("Set: %v", err)
}
if gotPath == db {
t.Fatal("saslpasswd2 was given the live database")
}
if got, _ := os.ReadFile(db); string(got) != "new database" {
t.Fatalf("database after the change: %q", got)
}
if runtime.GOOS != "windows" {
if fi, _ := os.Stat(db); fi.Mode().Perm() != 0o640 {
t.Fatalf("database mode %v, want 0640 for Postfix's group", fi.Mode().Perm())
}
old := make([]byte, 64)
n, _ := reader.ReadAt(old, 0)
if string(old[:n]) != "old database" {
t.Fatalf("a reader of the old file read %q", old[:n])
}
}
if _, err := os.Stat(db + ".new"); !errors.Is(err, os.ErrNotExist) {
t.Fatalf("the copy is left behind: %v", err)
}
}
func TestSASLFailedChangeLeavesTheDatabase(t *testing.T) {
dir := t.TempDir()
db := filepath.Join(dir, "sasldb2")
if err := os.WriteFile(db, []byte("old database"), 0o640); err != nil {
t.Fatal(err)
}
// A copy a killed panel left behind is not taken for the database.
if err := os.WriteFile(db+".new", []byte("stale"), 0o640); err != nil {
t.Fatal(err)
}
s := NewSASLDB(db, "mail.example.com")
s.run = func(args []string, stdin []byte) error {
if b, _ := os.ReadFile(args[2]); string(b) != "old database" {
t.Errorf("the copy holds %q", b)
}
_ = os.WriteFile(args[2], []byte("half"), 0o640)
return errors.New("saslpasswd2: exit status 1")
}
if err := s.Delete("alerts"); err == nil {
t.Fatal("Delete succeeded with saslpasswd2 failing")
}
if got, _ := os.ReadFile(db); string(got) != "old database" {
t.Fatalf("database after a failed change: %q", got)
}
if _, err := os.Stat(db + ".new"); !errors.Is(err, os.ErrNotExist) {
t.Fatalf("the copy is left behind: %v", err)
}
}
// The first account: there is no database to copy, and saslpasswd2 creates
// one at the path it is given.
func TestSASLFirstAccountCreatesTheDatabase(t *testing.T) {
db := filepath.Join(t.TempDir(), "sasldb2")
s := NewSASLDB(db, "mail.example.com")
s.run = func(args []string, stdin []byte) error {
if _, err := os.Stat(args[3]); !errors.Is(err, os.ErrNotExist) {
t.Errorf("saslpasswd2 given an existing file %s: %v", args[3], err)
}
return os.WriteFile(args[3], []byte("created"), 0o660)
}
if err := s.Set("alerts", "pw"); err != nil {
t.Fatalf("Set: %v", err)
}
if got, _ := os.ReadFile(db); string(got) != "created" {
t.Fatalf("database: %q", got)
}
}
+5 -2
View File
@@ -17,6 +17,7 @@ type fakeRun struct {
func newFakeSASL() (*SASLDB, *fakeRun) {
fr := &fakeRun{}
s := NewSASLDB("/data/sasl/sasldb2", "mail.example.com")
s.replace = inPlace
s.run = func(args []string, stdin []byte) error {
fr.calls++
fr.args = args
@@ -66,8 +67,10 @@ func TestSASLDeleteArgs(t *testing.T) {
}
}
// makeDump builds a db_dump byte-value document from key/value byte pairs, the
// same shape `db_dump <sasldb2>` emits.
// makeDump builds a db_dump byte-value document from key/value byte pairs. It
// is composed by hand in the shape the Berkeley DB 5.3 "Dump output formats"
// page gives; a real dump may have more name=value header lines (the
// db_load keywords) before HEADER=END, which the parser skips.
func makeDump(pairs [][2][]byte) []byte {
var b strings.Builder
b.WriteString("VERSION=3\nformat=bytevalue\ntype=hash\nHEADER=END\n")
+2 -1
View File
@@ -55,6 +55,7 @@ func (s *Service) Get(id int64) (store.Application, error) {
// the map rebuild fails, everything is rolled back so we never leave an
// application the panel cannot fully account for.
func (s *Service) Create(domainID int64, login, mode string, rawAddresses []string) (store.Application, string, error) {
login = foldLogin(login)
addresses, err := s.validateForDomain(domainID, login, mode, rawAddresses)
if err != nil {
return store.Application{}, "", err
@@ -313,7 +314,7 @@ func (s *Service) CreateWithSettings(domainID int64, login string, set Settings,
}
if err := s.applyAccess(a.ID, set, recalc); err != nil {
_ = s.ClearRateLimit(a.ID)
s.rollbackCreate(a.ID, login)
s.rollbackCreate(a.ID, a.Login) // the login as stored, folded
_ = s.Resync()
return store.Application{}, "", err
}
+35
View File
@@ -44,6 +44,7 @@ func newServiceHarness(t *testing.T) (*Service, *store.Store, *saslRecorder, *fa
rec := &saslRecorder{set: map[string]string{}}
sasl := NewSASLDB("/data/sasl/sasldb2", "mail.example.com")
sasl.replace = inPlace
sasl.run = func(args []string, stdin []byte) error {
if rec.failNext {
rec.failNext = false
@@ -113,6 +114,40 @@ func TestServiceCreateListValidatesDomain(t *testing.T) {
}
}
// A new login is stored in lower case, and the sender map names it so: to
// Postfix "Ops" and "ops" are one login (audit 2026-10-12, H4). A second
// application that differs only in case is refused, created or imported,
// whichever domain it is for.
func TestServiceCreateFoldsTheLogin(t *testing.T) {
svc, st, rec, maps := newServiceHarness(t)
a1 := addDomain(t, st, "a.example")
b := addDomain(t, st, "b.example")
a, _, err := svc.Create(a1.ID, "Ops", store.AddressModeWildcard, nil)
if err != nil {
t.Fatal(err)
}
if a.Login != "ops" {
t.Fatalf("login stored as %q, want ops", a.Login)
}
if _, ok := rec.set["ops"]; !ok {
t.Fatalf("SASL account written as %v, want ops", rec.set)
}
if len(maps.last) != 1 || maps.last[0].Login != "ops" {
t.Fatalf("map bindings = %+v, want the login in lower case", maps.last)
}
if _, _, err := svc.Create(b.ID, "OPS", store.AddressModeWildcard, nil); !errors.Is(err, store.ErrLoginExists) {
t.Fatalf("OPS in another domain = %v, want ErrLoginExists", err)
}
if err := svc.ImportApplication(b.ID, "oPs", store.AddressModeWildcard, nil, "imported-password"); !errors.Is(err, store.ErrLoginExists) {
t.Fatalf("an imported oPs = %v, want ErrLoginExists", err)
}
if _, ok := rec.set["OPS"]; ok {
t.Fatal("the refused OPS got a SASL account")
}
}
func TestServiceCreateDuplicateLogin(t *testing.T) {
svc, st, _, _ := newServiceHarness(t)
d := addDomain(t, st, "example.com")
+17 -1
View File
@@ -24,10 +24,26 @@ func invalid(format string, args ...any) error {
return &ValidationError{msg: fmt.Sprintf(format, args...)}
}
// foldLogin is a new application's login as it is stored: in lower case
// (audit 2026-10-12, H4; owner: fold, do not refuse). Postfix compares the
// SASL login with the sender map's without regard to case, so two logins
// that differ only in case would be one sender to Postfix and two
// applications to the panel. The store refuses such a pair as well
// (idx_applications_login_nocase); a login imported from another instance is
// not folded, since its SASL account and its client use it as it is.
//
// Postfix 3.7.11, src/smtpd/smtpd_check.c, reject_auth_sender_login_mismatch():
// the SASL login is compared with each owner by strcasecmp_utf8().
func foldLogin(login string) string {
return strings.ToLower(login)
}
// validateLogin enforces a strict server-side whitelist for the SASL login
// (security.md). It intentionally excludes '@': the login is stored in sasldb2,
// where '@' separates the user from the realm, so allowing it would change the
// account's identity. Client validation is never trusted.
// account's identity. Client validation is never trusted. (Cyrus SASL 2.1.28,
// common/plugin_common.c, _plug_parseuser(): a login is split into user and
// realm at its first "@", for saslpasswd2's store and for the lookup alike.)
//
// The login is the one piece of user input that is passed to saslpasswd2 as a
// command argument (never through a shell, security.md); this whitelist is what
+106 -12
View File
@@ -1,16 +1,16 @@
// Package backup implements SelfPost's full-server backup and the restore
// version guard (architecture.md § Persistence). A full backup is a
// gzip-compressed tar of the consolidated persistent state under /data — the
// SQLite database (as a consistent snapshot), the per-domain DKIM keys, the
// SASL database, and the Postfix queue — plus docker-compose.yml, .env, and
// certs/ from the operator's deploy directory, and a manifest recording the
// SelfPost version that produced it. Postfix delivery logs under log/ are
// excluded (diagnostics, not state).
// SQLite database (as a consistent snapshot), the per-domain DKIM keys and the
// SASL database — plus docker-compose.yml, .env, and certs/ from the
// operator's deploy directory, and a manifest recording the SelfPost version
// that produced it. Postfix delivery logs under log/ are excluded
// (diagnostics, not state), and so is the Postfix queue (see QueueDir).
//
// Restore is not a separate code path in the panel: a backup is extracted into
// the operator's project directory (data/, docker-compose.yml, .env, certs/)
// before first start. The archive carries everything needed to bring the
// instance back — DKIM keys, sasldb2, sender map, queue, and deploy files —
// instance back — DKIM keys, sasldb2, sender map, and deploy files —
// so the operator only adjusts hostname or proxy settings on a new host. The
// restore-specific steps the panel runs are CheckRestore, which refuses to
// boot if the manifest's version does not match the running binary so
@@ -28,6 +28,7 @@ import (
"compress/gzip"
"database/sql"
"encoding/json"
"errors"
"fmt"
"io"
"io/fs"
@@ -81,6 +82,20 @@ type Params struct {
OnWarn func(string)
}
// QueueDir is the Postfix queue, relative to the data directory. No backup
// holds it (owner, 2026-10-11). An archive is made while the instance runs,
// and Postfix moves messages between the queue's directories all the time: a
// copy taken then can hold a message twice or not at all, so what it would
// bring back is not the queue. Nor could the panel read it - Postfix keeps
// every queue file readable by its own user alone, which is why the panel's
// backup failed outright from 1.3.0 to 2.0.0 (date not recorded; Postfix
// 3.7.11, conf/postfix-files: active, deferred, incoming and the other queue
// directories are $mail_owner 700). What is not carried over is mail
// still waiting for delivery; a start without a queue directory builds an
// empty one. Whoever needs the queue stops the instance and archives the
// project directory (guide § Taking a full backup).
const QueueDir = "postfix/queue"
// excludedFromArchive lists the data-directory entries a backup never carries.
// The live database files are replaced by a consistent VACUUM INTO snapshot
// written under the canonical name; the setup token is transient bootstrap
@@ -145,7 +160,7 @@ func Create(w io.Writer, p Params) error {
return err
}
if err := addTree(tw, p.DataDir, DataArchivePrefix, excludedFromArchive); err != nil {
if err := addTree(tw, p.DataDir, DataArchivePrefix, excludedFromArchive, skipQueue); err != nil {
return err
}
@@ -181,6 +196,52 @@ func validateDeployRoot(deployRoot string) error {
return nil
}
// skipFunc says whether the entry at path (rel to the root being walked) is
// left out of the archive. A directory left out is not descended into.
type skipFunc func(path, rel string, d fs.DirEntry) bool
// skipQueue leaves the Postfix queue out of every archive.
func skipQueue(_, rel string, _ fs.DirEntry) bool {
return filepath.ToSlash(rel) == QueueDir
}
// Check reads through everything Create would put into the archive, without
// writing one, and returns the first thing that cannot be read. The panel
// streams its archive to the browser, so once Create has begun a failure can
// only cut the download short; this is run first, while a refusal can still be
// shown as a page.
//
// The panel is not root, and nothing is left out on that account: its archive
// is the one selfpost-backup makes, or it is not made. What it most likely
// cannot read is the private key under certs/ - see UnreadableCert.
func Check(p Params) error {
if p.DataDir == "" || p.DBPath == "" {
return fmt.Errorf("backup: DataDir and DBPath are required")
}
if err := ValidateDeployRoot(p.DeployRoot); err != nil {
return err
}
if err := addTree(nil, p.DataDir, DataArchivePrefix, excludedFromArchive, skipQueue); err != nil {
return err
}
return addDeployFiles(nil, p.DeployRoot, nil)
}
// UnreadableCert reports whether err, from Check or Create, is a file under
// the deploy root's certs/ that the caller is not allowed to read, and names
// it relative to the deploy root ("certs/privkey.pem").
func UnreadableCert(err error, deployRoot string) (name string, ok bool) {
var pathErr *fs.PathError
if !errors.As(err, &pathErr) || !errors.Is(err, fs.ErrPermission) {
return "", false
}
rel, relErr := filepath.Rel(filepath.Join(deployRoot, CertsDirName), pathErr.Path)
if relErr != nil || rel == ".." || strings.HasPrefix(filepath.ToSlash(rel), "../") {
return "", false
}
return CertsDirName + "/" + filepath.ToSlash(rel), true
}
func addDeployFiles(tw *tar.Writer, deployRoot string, onWarn func(string)) error {
for _, name := range []string{ComposeFileName, EnvFileName} {
src := filepath.Join(deployRoot, name)
@@ -206,14 +267,16 @@ func addDeployFiles(tw *tar.Writer, deployRoot string, onWarn func(string)) erro
}
return fmt.Errorf("backup: stat %s: %w", CertsDirName, err)
}
return addTree(tw, certsDir, CertsDirName+"/", nil)
return addTree(tw, certsDir, CertsDirName+"/", nil, nil)
}
// addTree walks root and adds every regular file (and directory, to preserve
// empty ones and modes) to tw under archivePrefix + path relative to root.
// When exclude is non-nil, top-level names relative to root are skipped.
// When exclude is non-nil, top-level names relative to root are skipped; skip,
// when non-nil, leaves out whatever it says. With a nil tw nothing is written:
// every file is opened and closed, which is how Check finds what cannot be read.
// Non-regular, non-directory entries (symlinks, sockets) are skipped.
func addTree(tw *tar.Writer, root, archivePrefix string, exclude map[string]bool) error {
func addTree(tw *tar.Writer, root, archivePrefix string, exclude map[string]bool, skip skipFunc) error {
return filepath.WalkDir(root, func(path string, d fs.DirEntry, err error) error {
if err != nil {
return err
@@ -236,12 +299,22 @@ func addTree(tw *tar.Writer, root, archivePrefix string, exclude map[string]bool
}
}
if skip != nil && skip(path, rel, d) {
if d.IsDir() {
return fs.SkipDir
}
return nil
}
info, err := d.Info()
if err != nil {
return err
}
switch {
case d.IsDir():
if tw == nil {
return nil
}
hdr := &tar.Header{
Typeflag: tar.TypeDir,
Name: name + "/",
@@ -250,7 +323,13 @@ func addTree(tw *tar.Writer, root, archivePrefix string, exclude map[string]bool
}
return tw.WriteHeader(hdr)
case info.Mode().IsRegular():
return writeTarFile(tw, name, info.Mode().Perm(), path)
err := writeTarFile(tw, name, info.Mode().Perm(), path)
if errors.Is(err, fs.ErrNotExist) {
// A writer's temporary file, renamed into place between
// the listing and the open: not part of /data's state.
return nil
}
return err
default:
return nil
}
@@ -276,12 +355,23 @@ func writeTarBytes(tw *tar.Writer, name string, mode int64, data []byte) error {
}
// writeTarFile streams a file from disk into the archive under name.
//
// The file is read through one descriptor, sized by that descriptor: the
// panel's writers of what a backup carries put a new file in place by rename
// (atomicfile.Write; sasldb2 through a copy, app.replaceSASLDB - the setup
// token alone is written in place, and it is not carried; the database and
// the queue are not walked), and a rename leaves the file open here as it
// was: the archive holds the old file or the new one, whole (assumptions
// C-Q3).
func writeTarFile(tw *tar.Writer, name string, mode fs.FileMode, srcPath string) error {
f, err := os.Open(srcPath)
if err != nil {
return fmt.Errorf("backup: open %s: %w", srcPath, err)
}
defer f.Close()
if tw == nil {
return nil
}
info, err := f.Stat()
if err != nil {
@@ -306,7 +396,11 @@ func writeTarFile(tw *tar.Writer, name string, mode fs.FileMode, srcPath string)
// snapshotDB produces a consistent copy of the SQLite database at dbPath using
// VACUUM INTO, so the backup captures a coherent point-in-time image even while
// the panel is writing to the live file under WAL. It returns the snapshot path
// and a cleanup function the caller must defer.
// and a cleanup function the caller must defer. (sqlite.org/lang_vacuum.html,
// "VACUUM INTO": "transactional in the sense that the generated output
// database is a consistent snapshot"; modernc.org/sqlite v1.60.1, sqlite.go,
// applyQueryParams(): each _pragma is run as "pragma <value>" on connect,
// busy_timeout first.)
func snapshotDB(dbPath string) (path string, cleanup func(), err error) {
dir, err := os.MkdirTemp("", "selfpost-backup-")
if err != nil {
+77 -2
View File
@@ -5,7 +5,9 @@ import (
"bytes"
"compress/gzip"
"encoding/json"
"fmt"
"io"
"io/fs"
"os"
"path/filepath"
"strings"
@@ -40,13 +42,15 @@ func seedDataDir(t *testing.T) (dataDir, dbPath, deployRoot string) {
writeFile(t, filepath.Join(dataDir, "opendkim", "keys", "example.com", "selfpost.private"), "PRIVATE KEY")
writeFile(t, filepath.Join(dataDir, "sasl", "sasldb2"), "SASLDB")
writeFile(t, filepath.Join(dataDir, "postfix", "sender_login_maps"), "@example.com login")
writeFile(t, filepath.Join(dataDir, "antispam", "deny_domain.map"), "evil.example\n")
writeFile(t, filepath.Join(dataDir, "postfix", "queue", "deferred", "sample"), "queue-file")
// Transient files that must NOT be archived.
writeFile(t, filepath.Join(dataDir, "setup-token"), "secret-token")
writeFile(t, filepath.Join(dataDir, "selfpost.db-wal"), "wal")
writeFile(t, filepath.Join(dataDir, "selfpost.db-shm"), "shm")
// Postfix's delivery log and its rotated files: diagnostic output, not
// state, and the bulkiest thing under /data.
// state, and the bulkiest thing under /data. The line is composed by hand;
// only its absence from the archive matters.
writeFile(t, filepath.Join(dataDir, "log", "mail.log"), "Aug 8 07:26:41 mail postfix/smtp[1]: ABC: to=<a@example.net>, status=sent (ok)")
writeFile(t, filepath.Join(dataDir, "log", "mail.log.1"), "older")
writeFile(t, filepath.Join(deployRoot, ComposeFileName), "services:\n selfpost:\n image: test\n")
@@ -112,7 +116,10 @@ func TestCreateIncludesStateExcludesTransient(t *testing.T) {
DataArchivePrefix + "opendkim/keys/example.com/selfpost.private",
DataArchivePrefix + "sasl/sasldb2",
DataArchivePrefix + "postfix/sender_login_maps",
DataArchivePrefix + "postfix/queue/deferred/sample",
// The filter lists as the spam filter reads them. They are rewritten
// from the database on the first start after a restore; carrying them
// means the filter has its lists before the panel is up.
DataArchivePrefix + "antispam/deny_domain.map",
ComposeFileName,
EnvFileName,
CertsDirName + "/fullchain.pem",
@@ -129,6 +136,7 @@ func TestCreateIncludesStateExcludesTransient(t *testing.T) {
DataArchivePrefix + "selfpost.db-shm",
DataArchivePrefix + "log/mail.log",
DataArchivePrefix + "log/mail.log.1",
DataArchivePrefix + "postfix/queue/deferred/sample",
} {
if _, ok := files[name]; ok {
t.Errorf("archive should not contain %s", name)
@@ -235,3 +243,70 @@ func TestCheckRestoreWrongFormatRejected(t *testing.T) {
t.Error("CheckRestore wrong format reported a restore")
}
}
// No archive holds the Postfix queue; the rest is there, the private key
// included, and Check agrees that it can be made.
func TestCreateLeavesTheQueueOut(t *testing.T) {
dataDir, dbPath, deployRoot := seedDataDir(t)
p := Params{DataDir: dataDir, DBPath: dbPath, Version: "1.2.3", DeployRoot: deployRoot}
if err := Check(p); err != nil {
t.Fatalf("Check: %v", err)
}
var buf bytes.Buffer
if err := Create(&buf, p); err != nil {
t.Fatalf("Create: %v", err)
}
files := readArchive(t, buf.Bytes())
for name := range files {
if strings.HasPrefix(name, DataArchivePrefix+QueueDir) {
t.Errorf("the panel's archive holds %s", name)
}
}
for _, name := range []string{
DataArchivePrefix + "selfpost.db",
DataArchivePrefix + "opendkim/keys/example.com/selfpost.private",
DataArchivePrefix + "sasl/sasldb2",
DataArchivePrefix + "postfix/sender_login_maps",
ComposeFileName, EnvFileName,
CertsDirName + "/fullchain.pem",
CertsDirName + "/privkey.pem",
} {
if _, ok := files[name]; !ok {
t.Errorf("archive missing %s", name)
}
}
}
// A file under certs/ that may not be read is told apart from any other
// failure, so the panel can say what to do about it.
func TestUnreadableCert(t *testing.T) {
root := filepath.Join("srv", "deploy")
denied := func(path string) error {
return fmt.Errorf("backup: open %s: %w", path, &fs.PathError{Op: "open", Path: path, Err: fs.ErrPermission})
}
if name, ok := UnreadableCert(denied(filepath.Join(root, "certs", "privkey.pem")), root); !ok || name != "certs/privkey.pem" {
t.Errorf("certs/privkey.pem: got %q, %v", name, ok)
}
if name, ok := UnreadableCert(denied(filepath.Join(root, ".env")), root); ok {
t.Errorf(".env was taken for a certificate file: %q", name)
}
if name, ok := UnreadableCert(denied(filepath.Join("data", "postfix", "x")), root); ok {
t.Errorf("a data file was taken for a certificate file: %q", name)
}
missing := &fs.PathError{Op: "open", Path: filepath.Join(root, "certs", "privkey.pem"), Err: fs.ErrNotExist}
if _, ok := UnreadableCert(missing, root); ok {
t.Error("a missing file was taken for one that may not be read")
}
}
// Check finds what Create would stumble over, before anything is written.
func TestCheckReportsWhatCannotBeRead(t *testing.T) {
dataDir, dbPath, deployRoot := seedDataDir(t)
if err := os.Remove(filepath.Join(deployRoot, EnvFileName)); err != nil {
t.Fatal(err)
}
if err := Check(Params{DataDir: dataDir, DBPath: dbPath, Version: "1.2.3", DeployRoot: deployRoot}); err == nil {
t.Fatal("Check passed with .env missing from the deploy root")
}
}
+109
View File
@@ -0,0 +1,109 @@
package dmarc
import (
"errors"
"strings"
"testing"
"time"
"github.com/mixeme/selfpost/internal/store"
)
func reportXML(orgName, domain, records string) string {
return `<?xml version="1.0"?><feedback><report_metadata><org_name>` + orgName +
`</org_name><email>a@example.net</email><report_id>1</report_id>` +
`<date_range><begin>1723593600</begin><end>1723680000</end></date_range></report_metadata>` +
`<policy_published><domain>` + domain + `</domain><p>none</p></policy_published>` + records + `</feedback>`
}
const oneRecord = `<record><row><source_ip>203.0.113.10</source_ip><count>3</count>` +
`<policy_evaluated><disposition>none</disposition><dkim>pass</dkim><spf>pass</spf></policy_evaluated></row>` +
`<identifiers><header_from>example.com</header_from></identifiers></record>`
// The XML is a stranger's: what it may make the relay store is bounded.
func TestParseAggregateBoundsWhatAReportHolds(t *testing.T) {
long := strings.Repeat("é", 4000)
got, err := ParseAggregate([]byte(reportXML(long, "example.com", oneRecord)))
if err != nil {
t.Fatal(err)
}
if len(got.Reporter) > maxFieldLen || !strings.HasPrefix(long, got.Reporter) || got.Reporter == "" {
t.Fatalf("reporter name is %d bytes, want the start of the name in at most %d", len(got.Reporter), maxFieldLen)
}
if strings.ContainsRune(got.Reporter, '�') {
t.Fatal("the name was cut in the middle of a character")
}
huge := `<record><row><source_ip>203.0.113.10</source_ip><count>9223372036854775807</count>` +
`<policy_evaluated><dkim>pass</dkim></policy_evaluated></row></record>`
got, err = ParseAggregate([]byte(reportXML("r", "example.com", huge+huge)))
if err != nil {
t.Fatal(err)
}
if got.PassCount != 2*maxRecordCount {
t.Fatalf("pass count = %d, want two records clamped to %d each", got.PassCount, maxRecordCount)
}
if _, err := ParseAggregate([]byte(reportXML("r", "example.com", strings.Repeat(oneRecord, maxReportRecords)))); err != nil {
t.Fatalf("a report of %d records was refused: %v", maxReportRecords, err)
}
if _, err := ParseAggregate([]byte(reportXML("r", "example.com", strings.Repeat(oneRecord, maxReportRecords+1)))); err == nil {
t.Fatalf("a report of %d records was accepted", maxReportRecords+1)
}
}
// A report over the bound is one the relay does not keep: dropped, not retried.
func TestIngestMessageRejectsAReportWithTooManyRecords(t *testing.T) {
st, err := store.Open(t.TempDir() + "/test.db")
if err != nil {
t.Fatal(err)
}
defer st.Close()
if _, err := st.AddDomain("example.com", "selfpost"); err != nil {
t.Fatal(err)
}
xml := reportXML("r", "example.com", strings.Repeat(oneRecord, maxReportRecords+1))
err = IngestMessage(st, strings.NewReader(gzipMessage(t, xml)), "dmarc-reports+example.com@mail.example.com", time.Now().UTC())
if !errors.Is(err, ErrRejected) {
t.Fatalf("err = %v, want ErrRejected", err)
}
}
// Reports filed under one domain do not push another domain's reports out.
func TestIngestKeepsEachDomainsReportsApart(t *testing.T) {
st, err := store.Open(t.TempDir() + "/test.db")
if err != nil {
t.Fatal(err)
}
defer st.Close()
now := time.Now().UTC()
if _, err := st.InsertDMARCReport(store.DMARCReport{
Domain: "quiet.example", Reporter: "genuine", ReportID: "1", PeriodBegin: now, PeriodEnd: now, ReceivedAt: now.Add(-time.Hour),
}); err != nil {
t.Fatal(err)
}
for i := 0; i < store.DMARCReportsMaxKeep+20; i++ {
if _, err := st.InsertDMARCReport(store.DMARCReport{
Domain: "flooded.example", Reporter: "stranger", ReportID: strings.Repeat("x", 1+i), PeriodBegin: now, PeriodEnd: now, ReceivedAt: now,
}); err != nil {
t.Fatal(err)
}
}
if err := st.PruneDMARCReports("flooded.example"); err != nil {
t.Fatal(err)
}
quiet, err := st.ListDMARCReports(store.DMARCReportScope{Domains: []string{"quiet.example"}}, 10)
if err != nil {
t.Fatal(err)
}
if len(quiet) != 1 {
t.Fatalf("the other domain has %d reports after the flood, want its 1", len(quiet))
}
flooded, err := st.ListDMARCReports(store.DMARCReportScope{Domains: []string{"flooded.example"}}, 10000)
if err != nil {
t.Fatal(err)
}
if len(flooded) > store.DMARCReportsMaxKeep {
t.Fatalf("the flooded domain keeps %d reports, want at most %d", len(flooded), store.DMARCReportsMaxKeep)
}
}
+20 -6
View File
@@ -3,6 +3,7 @@ package dmarc
import (
"bytes"
"encoding/base64"
"errors"
"fmt"
"io"
"mime"
@@ -16,6 +17,15 @@ import (
"github.com/mixeme/selfpost/internal/store"
)
// ErrRejected marks a message that is not a report this relay keeps and never
// will be: it cannot be read as an aggregate report, or it reports on a domain
// that is not sent from here, or it came to another domain's address. Sending
// it again changes nothing, so the caller drops it. Any other error is the
// store's and worth another try.
var ErrRejected = errors.New("dmarc report rejected")
func rejected(err error) error { return fmt.Errorf("%w: %w", ErrRejected, err) }
// IngestMessage parses a raw RFC 5322 message from r and stores any DMARC
// aggregate attachment it finds.
//
@@ -34,11 +44,11 @@ func IngestMessage(st *store.Store, r io.Reader, recipient string, receivedAt ti
}
payload, err := extractAggregatePayload(raw)
if err != nil {
return err
return rejected(err)
}
parsed, err := ParseAggregate(payload)
if err != nil {
return err
return rejected(err)
}
recipient = strings.ToLower(strings.TrimSpace(recipient))
if err := checkReportDomain(st, parsed.Domain, recipient); err != nil {
@@ -74,7 +84,7 @@ func IngestMessage(st *store.Store, r io.Reader, recipient string, receivedAt ti
if _, err := st.InsertDMARCReport(rep); err != nil {
return err
}
return st.PruneDMARCReports()
return st.PruneDMARCReports(rep.Domain)
}
// checkReportDomain refuses a report whose policy domain is not one of this
@@ -82,17 +92,17 @@ func IngestMessage(st *store.Store, r io.Reader, recipient string, receivedAt ti
// delivered to.
func checkReportDomain(st *store.Store, domain, recipient string) error {
if domain == "" {
return fmt.Errorf("dmarc report: no policy domain")
return rejected(fmt.Errorf("no policy domain"))
}
if tagged := TaggedDomain(recipient); tagged != "" && tagged != domain {
return fmt.Errorf("dmarc report: domain %q does not match the address it was sent to (%s)", domain, recipient)
return rejected(fmt.Errorf("domain %q does not match the address it was sent to (%s)", domain, recipient))
}
ok, err := st.DomainExists(domain)
if err != nil {
return err
}
if !ok {
return fmt.Errorf("dmarc report: %q is not a domain configured on this relay", domain)
return rejected(fmt.Errorf("%q is not a domain configured on this relay", domain))
}
return nil
}
@@ -144,6 +154,10 @@ func extractAggregatePayload(raw []byte) ([]byte, error) {
return readEncodedBody(textproto.MIMEHeader(msg.Header), msg.Body)
}
// looksLikeAggregate takes a part by its file name or Content-Type. RFC 7489
// § 7.2.1.1: the report is "application/gzip" if compressed and "text/xml"
// otherwise, its name ending ".xml" or ".xml.gz"; zip is outside the RFC and
// is what some reporters send (see decompress).
func looksLikeAggregate(name string) bool {
name = strings.ToLower(name)
return strings.Contains(name, "xml") || strings.Contains(name, "gzip") || strings.Contains(name, "zip")
+60 -16
View File
@@ -9,6 +9,7 @@ import (
"io"
"strings"
"time"
"unicode/utf8"
)
// ParsedReport is the structured form of a DMARC aggregate XML payload.
@@ -39,6 +40,11 @@ type ParsedRecord struct {
HeaderFrom string
}
// feedbackXML follows RFC 7489, Appendix C. The schema's targetNamespace is
// "http://dmarc.org/dmarc-xml/0.1"; a tag without a namespace matches the
// local name in any namespace (Go, src/encoding/xml/read.go: the namespace is
// compared only when the tag names one), so a report that declares one is
// read the same.
type feedbackXML struct {
XMLName xml.Name `xml:"feedback"`
Metadata struct {
@@ -74,6 +80,31 @@ type feedbackXML struct {
} `xml:"record"`
}
// What a report may hold. The XML comes from anyone who knows a report address,
// and eight megabytes of it (maxAggregateXML) fit some 160 000 empty records or
// one name of eight megabytes - every one of them a row or a table cell later.
// A real report has tens of records, a few thousand from the largest
// receivers, and no field longer than a line.
const (
maxReportRecords = 5000
maxFieldLen = 255
maxRecordCount = 1_000_000_000
)
// field trims a text value of the report and cuts it to maxFieldLen bytes, on
// a character boundary.
func field(s string) string {
s = strings.TrimSpace(s)
if len(s) <= maxFieldLen {
return s
}
s = s[:maxFieldLen]
for len(s) > 0 && !utf8.ValidString(s) {
s = s[:len(s)-1]
}
return s
}
// ParseAggregate decodes gzip- or zip-compressed, or raw, DMARC aggregate XML.
func ParseAggregate(raw []byte) (ParsedReport, error) {
data, err := decompress(raw)
@@ -87,34 +118,43 @@ func ParseAggregate(raw []byte) (ParsedReport, error) {
if doc.Metadata.ReportID == "" || doc.Policy.Domain == "" {
return ParsedReport{}, fmt.Errorf("dmarc xml: missing report_id or domain")
}
if len(doc.Records) > maxReportRecords {
return ParsedReport{}, fmt.Errorf("dmarc xml: %d records, more than the %d a report may hold", len(doc.Records), maxReportRecords)
}
// Cut, not refused: a domain name is at most 253 bytes, so a longer one
// names nothing this relay sends for and is turned away as such.
out := ParsedReport{
Reporter: strings.TrimSpace(doc.Metadata.OrgName),
ReportID: strings.TrimSpace(doc.Metadata.ReportID),
ContactEmail: strings.TrimSpace(doc.Metadata.Email),
Reporter: field(doc.Metadata.OrgName),
ReportID: field(doc.Metadata.ReportID),
ContactEmail: field(doc.Metadata.Email),
PeriodBegin: time.Unix(doc.Metadata.DateRange.Begin, 0).UTC(),
PeriodEnd: time.Unix(doc.Metadata.DateRange.End, 0).UTC(),
Domain: strings.ToLower(strings.TrimSpace(doc.Policy.Domain)),
PolicyP: strings.TrimSpace(doc.Policy.P),
PolicySP: strings.TrimSpace(doc.Policy.SP),
Domain: strings.ToLower(field(doc.Policy.Domain)),
PolicyP: field(doc.Policy.P),
PolicySP: field(doc.Policy.SP),
PolicyPct: doc.Policy.Pct,
PolicyADKIM: strings.TrimSpace(doc.Policy.ADKIM),
PolicyASPF: strings.TrimSpace(doc.Policy.ASPF),
PolicyADKIM: field(doc.Policy.ADKIM),
PolicyASPF: field(doc.Policy.ASPF),
}
if out.PolicyPct == 0 {
out.PolicyPct = 100
}
for _, rec := range doc.Records {
row := ParsedRecord{
SourceIP: strings.TrimSpace(rec.Row.SourceIP),
SourceIP: field(rec.Row.SourceIP),
Count: rec.Row.Count,
Disposition: strings.TrimSpace(rec.Row.Policy.Disposition),
SPFResult: strings.TrimSpace(rec.Row.Policy.SPF),
DKIMResult: strings.TrimSpace(rec.Row.Policy.DKIM),
HeaderFrom: strings.TrimSpace(rec.Identifiers.HeaderFrom),
Disposition: field(rec.Row.Policy.Disposition),
SPFResult: field(rec.Row.Policy.SPF),
DKIMResult: field(rec.Row.Policy.DKIM),
HeaderFrom: field(rec.Identifiers.HeaderFrom),
}
if row.Count <= 0 {
row.Count = 1
}
// Bounded so that the sums of a report cannot wrap around.
if row.Count > maxRecordCount {
row.Count = maxRecordCount
}
if dmarcRecordPasses(row) {
out.PassCount += row.Count
} else {
@@ -125,6 +165,9 @@ func ParseAggregate(raw []byte) (ParsedReport, error) {
return out, nil
}
// dmarcRecordPasses: RFC 7489, Appendix C, PolicyEvaluatedType - dkim and spf
// are "The DMARC-aligned authentication result"; § 4.2: DMARC passes when one
// supported mechanism passes aligned.
func dmarcRecordPasses(r ParsedRecord) bool {
return strings.EqualFold(r.SPFResult, "pass") || strings.EqualFold(r.DKIMResult, "pass")
}
@@ -135,7 +178,8 @@ const maxAggregateXML = 8 << 20
// decompress unwraps the container the reporter used. Both are seen in the
// wild: most senders gzip the XML, some (notably several Microsoft and Yahoo
// endpoints) ship it as a single-entry zip.
// endpoints) ship it as a single-entry zip. (A sighting: where and when not
// recorded. RFC 7489 § 7.2.1.1 names gzip only.)
func decompress(raw []byte) ([]byte, error) {
switch {
case len(raw) >= 2 && raw[0] == 0x1f && raw[1] == 0x8b:
@@ -172,12 +216,12 @@ func unzip(raw []byte) ([]byte, error) {
}
rc, err := f.Open()
if err != nil {
return nil, fmt.Errorf("dmarc zip open %s: %w", f.Name, err)
return nil, fmt.Errorf("dmarc zip open %q: %w", field(f.Name), err)
}
data, err := io.ReadAll(io.LimitReader(rc, maxAggregateXML))
rc.Close()
if err != nil {
return nil, fmt.Errorf("dmarc zip read %s: %w", f.Name, err)
return nil, fmt.Errorf("dmarc zip read %q: %w", field(f.Name), err)
}
return data, nil
}
+48 -4
View File
@@ -4,12 +4,16 @@ import (
"archive/zip"
"bytes"
"compress/gzip"
"errors"
"testing"
"time"
"github.com/mixeme/selfpost/internal/store"
)
// sampleXML and the messages in this file are composed by hand, in the shape
// RFC 7489 gives (Appendix C, the schema; § 7.2.1.1, application/gzip and the
// .xml.gz name). The org name is any reporter's; no report was copied.
const sampleXML = `<?xml version="1.0"?>
<feedback>
<report_metadata>
@@ -126,8 +130,9 @@ func TestIngestMessageRejectsUnknownDomain(t *testing.T) {
}
defer st.Close()
if err := IngestMessage(st, bytes.NewBufferString(gzipMessage(t, sampleXML)), "dmarc-reports@mail.example.com", time.Now().UTC()); err == nil {
t.Fatal("report for an unconfigured domain was accepted")
err = IngestMessage(st, bytes.NewBufferString(gzipMessage(t, sampleXML)), "dmarc-reports@mail.example.com", time.Now().UTC())
if !errors.Is(err, ErrRejected) {
t.Fatalf("report for an unconfigured domain: err = %v, want ErrRejected", err)
}
list, err := st.ListDMARCReports(store.DMARCReportScope{AllDomains: true}, 10)
if err != nil {
@@ -155,8 +160,47 @@ func TestIngestMessageRejectsTagMismatch(t *testing.T) {
err = IngestMessage(st, bytes.NewBufferString(gzipMessage(t, sampleXML)),
"dmarc-reports+other.example@mail.example.com", time.Now().UTC())
if err == nil {
t.Fatal("report was accepted at an address tagged for another domain")
if !errors.Is(err, ErrRejected) {
t.Fatalf("report at an address tagged for another domain: err = %v, want ErrRejected", err)
}
}
// What cannot be read as a report is rejected, not put off: Postfix would
// offer the same bytes again for days.
func TestIngestMessageRejectsWhatIsNotAReport(t *testing.T) {
st, err := store.Open(t.TempDir() + "/test.db")
if err != nil {
t.Fatal(err)
}
defer st.Close()
if _, err := st.AddDomain("example.com", "selfpost"); err != nil {
t.Fatal(err)
}
for name, msg := range map[string]string{
"plain text": "From: someone@example.net\r\nSubject: hello\r\n\r\nnot a report\r\n",
"no headers": "garbage",
"broken gzip": "From: a@example.net\r\nContent-Type: application/gzip\r\n\r\nnot gzip at all",
} {
err := IngestMessage(st, bytes.NewBufferString(msg), "dmarc-reports+example.com@mail.example.com", time.Now().UTC())
if !errors.Is(err, ErrRejected) {
t.Errorf("%s: err = %v, want ErrRejected", name, err)
}
}
}
// A failing store is not the report's fault: the message is to be tried again.
func TestIngestMessageStoreFailureIsNotARejection(t *testing.T) {
st, err := store.Open(t.TempDir() + "/test.db")
if err != nil {
t.Fatal(err)
}
if _, err := st.AddDomain("example.com", "selfpost"); err != nil {
t.Fatal(err)
}
st.Close()
err = IngestMessage(st, bytes.NewBufferString(gzipMessage(t, sampleXML)), "dmarc-reports+example.com@mail.example.com", time.Now().UTC())
if err == nil || errors.Is(err, ErrRejected) {
t.Fatalf("closed store: err = %v, want an error that is not ErrRejected", err)
}
}
+52
View File
@@ -100,6 +100,7 @@ type Checker struct {
servers map[string]cached[Server]
domains map[string]cached[Domain]
inbound map[string]cached[Result]
mail map[string]cached[Mail]
}
type cached[T any] struct {
@@ -123,6 +124,7 @@ func newChecker(r resolver, timeout, srvTTL, domTTL time.Duration) *Checker {
servers: make(map[string]cached[Server]),
domains: make(map[string]cached[Domain]),
inbound: make(map[string]cached[Result]),
mail: make(map[string]cached[Mail]),
}
}
@@ -170,6 +172,56 @@ func (c *Checker) Domain(q Query, force bool) Domain {
return d
}
// Mail is the published-DNS state of the server's own name as a sender: the
// panel's notifications go out from postmaster@ the hostname, signed with a
// key of that name.
type Mail struct {
Name string
DKIM Result
SPF Result
Overall health.Status
CheckedAt time.Time
}
// ServerMailSPF is the SPF record to publish for the server's own name: its
// address record is the one sender.
const ServerMailSPF = "v=spf1 a -all"
// ServerMail checks the DKIM and SPF records of the server's own name. q is as
// for a domain, with Name the hostname. Cached like a domain; force skips the
// cache.
func (c *Checker) ServerMail(q Query, force bool) Mail {
key := "\x00mail\x00" + q.Name
if !force {
c.mu.Lock()
entry, ok := c.mail[key]
c.mu.Unlock()
if ok && time.Now().Before(entry.expires) {
return entry.value
}
}
ctx, cancel := context.WithTimeout(context.Background(), c.timeout)
defer cancel()
m := Mail{Name: q.Name, CheckedAt: time.Now()}
var wg sync.WaitGroup
wg.Add(2)
go func() { defer wg.Done(); m.DKIM = c.checkDKIM(ctx, q) }()
go func() { defer wg.Done(); m.SPF = c.checkSPF(ctx, q) }()
wg.Wait()
// checkSPF words a missing record for a sending domain; for the server's
// own name the record to publish is a different one.
if m.SPF.Status == health.StatusError && len(m.SPF.Records) == 0 {
m.SPF.Detail = fmt.Sprintf("No SPF record is published for %s. Publish the TXT record %q - without it a receiver has nothing authorising this server to send as its own name.", q.Name, ServerMailSPF)
}
m.Overall = health.Worst(m.DKIM.Status, m.SPF.Status)
c.mu.Lock()
c.mail[key] = cached[Mail]{value: m, expires: m.CheckedAt.Add(c.domainTTL)}
c.mu.Unlock()
return m
}
// Forget drops a domain's cached result, so the next page view re-checks it.
// Used when a domain is removed or re-imported.
func (c *Checker) Forget(domainName string) {
+3
View File
@@ -42,6 +42,9 @@ func (f *fakeResolver) LookupTXT(_ context.Context, name string) ([]string, erro
func (f *fakeResolver) LookupIPAddr(_ context.Context, host string) ([]net.IPAddr, error) {
f.lookups++
if f.fail[host] {
return nil, &net.DNSError{Err: "server misbehaving", Name: host, IsTemporary: true}
}
if v, ok := f.addr[host]; ok {
return v, nil
}
+104
View File
@@ -0,0 +1,104 @@
package dnscheck
import (
"net"
"strings"
"testing"
"github.com/mixeme/selfpost/internal/health"
)
const serverDKIM = "v=DKIM1; k=rsa; p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAserverkey"
func serverMailQuery() Query {
return Query{
Name: "mail.example.org",
Selector: "sp",
ExpectedDKIM: serverDKIM,
Hostname: "mail.example.org",
ServerIPs: []string{"203.0.113.25"},
}
}
// The server's own name passes with its key published and "v=spf1 a -all":
// the name's address record is this server.
func TestServerMailPublished(t *testing.T) {
f := &fakeResolver{
txt: map[string][]string{
"sp._domainkey.mail.example.org": {serverDKIM},
"mail.example.org": {ServerMailSPF},
},
addr: map[string][]net.IPAddr{"mail.example.org": ipAddrs("203.0.113.25")},
}
c := newTestChecker(f)
m := c.ServerMail(serverMailQuery(), false)
if m.DKIM.Status != health.StatusOK || m.SPF.Status != health.StatusOK || m.Overall != health.StatusOK {
t.Fatalf("DKIM %s (%s), SPF %s (%s), overall %s", m.DKIM.Status, m.DKIM.Detail, m.SPF.Status, m.SPF.Detail, m.Overall)
}
// Cached like a domain, and re-read when forced.
before := f.lookups
c.ServerMail(serverMailQuery(), false)
if f.lookups != before {
t.Errorf("a second reading looked DNS up again (%d lookups)", f.lookups-before)
}
c.ServerMail(serverMailQuery(), true)
if f.lookups == before {
t.Error("a forced reading used the cache")
}
}
// With nothing published both records are errors, and the SPF advice names the
// record for a server's own name, not the one for a sending domain.
func TestServerMailMissing(t *testing.T) {
c := newTestChecker(&fakeResolver{})
m := c.ServerMail(serverMailQuery(), false)
if m.DKIM.Status != health.StatusError || m.SPF.Status != health.StatusError || m.Overall != health.StatusError {
t.Fatalf("DKIM %s, SPF %s, overall %s; want errors", m.DKIM.Status, m.SPF.Status, m.Overall)
}
if !strings.Contains(m.SPF.Detail, ServerMailSPF) || strings.Contains(m.SPF.Detail, "a:mail.example.org") {
t.Errorf("SPF advice: %s", m.SPF.Detail)
}
}
// A sending domain of the same name does not share the cache entry.
func TestServerMailCacheIsItsOwn(t *testing.T) {
f := &fakeResolver{
txt: map[string][]string{
"sp._domainkey.mail.example.org": {serverDKIM},
"mail.example.org": {ServerMailSPF},
},
addr: map[string][]net.IPAddr{"mail.example.org": ipAddrs("203.0.113.25")},
}
c := newTestChecker(f)
c.Domain(serverMailQuery(), false)
before := f.lookups
c.ServerMail(serverMailQuery(), false)
if f.lookups == before {
t.Error("the server's mail check was answered from the domain's cache entry")
}
}
// The address lookup behind an "a" mechanism did not answer: nothing was
// learned, and the record is not called wrong for it. A name that answers
// "no such host" is a finding, and the record is wrong.
func TestSPFLookupThatDidNotAnswerIsNotAMismatch(t *testing.T) {
// The fake fails by name, so the mechanism names a host of its own: the
// TXT lookup of the server's name answers, the address lookup does not.
f := &fakeResolver{
txt: map[string][]string{"mail.example.org": {"v=spf1 a:addr.example.org -all"}},
fail: map[string]bool{"addr.example.org": true},
}
c := newTestChecker(f)
q := serverMailQuery()
if got := c.ServerMail(q, true).SPF; got.Status != health.StatusWarn {
t.Errorf("the lookup did not answer: SPF is %s (%s), want a warning", got.Status, got.Detail)
}
f.fail = nil
if got := c.ServerMail(q, true).SPF; got.Status != health.StatusError {
t.Errorf("the name does not exist: SPF is %s (%s), want an error", got.Status, got.Detail)
}
}
+35 -12
View File
@@ -2,6 +2,7 @@ package dnscheck
import (
"context"
"errors"
"fmt"
"net"
"strings"
@@ -133,7 +134,14 @@ func (c *Checker) evaluateSPF(ctx context.Context, record, domainName string, ip
continue
}
budget--
if qualifier == '+' && c.resolvesTo(ctx, target, ips) {
match, failed := c.resolvesTo(ctx, target, ips)
if failed {
// Nothing was learned about this mechanism: "cannot tell",
// not "does not authorise".
note("a (the lookup did not answer)")
continue
}
if qualifier == '+' && match {
return term, unfollowed
}
@@ -151,7 +159,12 @@ func (c *Checker) evaluateSPF(ctx context.Context, record, domainName string, ip
continue
}
budget--
if qualifier == '+' && c.mxResolvesTo(ctx, target, ips, &budget) {
match, failed := c.mxResolvesTo(ctx, target, ips, &budget)
if failed {
note("mx (the lookup did not answer)")
continue
}
if qualifier == '+' && match {
return term, unfollowed
}
@@ -176,39 +189,49 @@ func (c *Checker) evaluateSPF(ctx context.Context, record, domainName string, ip
return "", unfollowed
}
// lookupUnanswered tells a lookup that failed from one that answered "no such
// name": only the second is a finding.
func lookupUnanswered(err error) bool {
var dnsErr *net.DNSError
return err != nil && !(errors.As(err, &dnsErr) && dnsErr.IsNotFound)
}
// resolvesTo reports whether name resolves to one of the server's addresses.
func (c *Checker) resolvesTo(ctx context.Context, name string, ips []net.IP) bool {
// failed is set when the lookup did not answer, which says nothing either way.
func (c *Checker) resolvesTo(ctx context.Context, name string, ips []net.IP) (match, failed bool) {
addrs, err := c.resolver.LookupIPAddr(ctx, name)
if err != nil {
return false
return false, lookupUnanswered(err)
}
for _, a := range addrs {
for _, ip := range ips {
if a.IP.Equal(ip) {
return true
return true, false
}
}
}
return false
return false, false
}
// mxResolvesTo reports whether any of name's MX hosts resolves to one of the
// server's addresses, spending at most the remaining lookup budget.
func (c *Checker) mxResolvesTo(ctx context.Context, name string, ips []net.IP, budget *int) bool {
func (c *Checker) mxResolvesTo(ctx context.Context, name string, ips []net.IP, budget *int) (match, failed bool) {
mxs, err := c.resolver.LookupMX(ctx, name)
if err != nil {
return false
return false, lookupUnanswered(err)
}
for _, mx := range mxs {
if *budget <= 0 {
return false
return false, failed
}
*budget--
if c.resolvesTo(ctx, strings.TrimSuffix(mx.Host, "."), ips) {
return true
m, f := c.resolvesTo(ctx, strings.TrimSuffix(mx.Host, "."), ips)
if m {
return true, false
}
failed = failed || f
}
return false
return false, failed
}
// coversAny reports whether an ip4:/ip6: value — a bare address or a CIDR —
+6
View File
@@ -37,6 +37,12 @@ func generateDKIMKey() (*rsa.PrivateKey, error) {
// and entrypoint.sh); the parent directory carries setgid so the group is
// inherited. The write is atomic (temp file + rename) so OpenDKIM never observes
// a half-written key.
//
// OpenDKIM 2.11.0~beta2, libopendkim/dkim.c, dkim_privkey_load(): a key that
// starts as PEM goes to OpenSSL's PEM_read_bio_PrivateKey(), which takes this
// PKCS#1 form; opendkim/opendkim.c, dkimf_add_signrequest(): a key file
// dkimf_securefile() finds insecure is only logged while RequireSafeKeys is no
// (build/opendkim.conf).
func writePrivateKeyPEM(path string, key *rsa.PrivateKey) error {
block := &pem.Block{
Type: "RSA PRIVATE KEY",
+57 -1
View File
@@ -144,6 +144,12 @@ func (o *OpenDKIM) Record(domainName, selector string) (DKIMRecord, error) {
return dkimRecord(selector, domainName, &key.PublicKey)
}
// SigningTablePath is the SigningTable's location: the names this server
// signs for, which the sender check reads (internal/milter/sender.go).
func (o *OpenDKIM) SigningTablePath() string {
return o.signingTablePath
}
// Rebuild regenerates KeyTable and SigningTable from the full domain set and
// reloads OpenDKIM (architecture.md § OpenDKIM). Full regeneration (rather
// than incremental edits) keeps the files a pure function of the registry, so
@@ -163,6 +169,39 @@ func (o *OpenDKIM) Rebuild(domains []SigningDomain) error {
return o.reload()
}
// signs reports whether the tables on disk sign name with the key of
// selector: a line for the name in the SigningTable, and in the KeyTable the
// line that gives it that selector. Tables that do not exist yet sign nothing.
// The selector is part of the question because the panel shows and checks the
// record of one key; tables that name another would have OpenDKIM sign with a
// key whose record nobody is told to publish.
//
// OpenDKIM 2.11.0~beta2, opendkim/util.c, dkimf_mkregexp(): "*@name" is
// matched as ^.*@name$ (case aside), so it signs name and none of its
// subdomains (see Signers.Signs in internal/milter/sender.go).
func (o *OpenDKIM) signs(name, selector string) (bool, error) {
has := func(path, prefix string) (bool, error) {
data, err := os.ReadFile(path)
if os.IsNotExist(err) {
return false, nil
}
if err != nil {
return false, fmt.Errorf("read %s: %w", filepath.Base(path), err)
}
for _, line := range strings.Split(string(data), "\n") {
if strings.HasPrefix(line, prefix) {
return true, nil
}
}
return false, nil
}
signed, err := has(o.signingTablePath, "*@"+name+" ")
if err != nil || !signed {
return false, err
}
return has(o.keyTablePath, name+" "+name+":"+selector+":")
}
// Reload asks OpenDKIM to re-read its tables without regenerating them. It
// backs the panel's manual reload button (architecture.md § Panel HTTP
// surface).
@@ -187,6 +226,13 @@ func renderTables(keysDir string, domains []SigningDomain) (keyTable, signingTab
sorted := append([]SigningDomain(nil), domains...)
sort.Slice(sorted, func(i, j int) bool { return sorted[i].Name < sorted[j].Name })
// OpenDKIM 2.11.0~beta2: KeyTable is a file: table, the key up to the
// first blank (opendkim/opendkim-db.c, dkimf_db_open(), DKIMF_DB_TYPE_FILE);
// opendkim/opendkim.c, dkimf_add_signrequest(), reads domain, selector and
// key from its value, split at ":" (opendkim-db.c, dkimf_db_datasplit()),
// a key starting with "/" being a file. SigningTable is refile:, a pattern
// up to the first blank and the key name after it (dkimf_db_open(), case
// DKIMF_DB_TYPE_REFILE).
var kt, st strings.Builder
for _, d := range sorted {
if err := assertConfigSafe(d.Name, d.Selector); err != nil {
@@ -205,7 +251,9 @@ func renderTables(keysDir string, domains []SigningDomain) (keyTable, signingTab
// opendkimForbidden is what may never appear in a KeyTable or SigningTable
// value: whitespace and the newline that would end the line, and the colon and
// path separators that carry meaning inside a table entry.
// path separators that carry meaning inside a table entry. (OpenDKIM
// 2.11.0~beta2, opendkim/opendkim-db.c, dkimf_db_datasplit(), splits a
// KeyTable value at ":"; the domain and selector are also the key's path.)
const opendkimForbidden = " \t\r\n:/\\"
// assertConfigSafe rejects any domain/selector value that could break out of a
@@ -233,6 +281,14 @@ func assertConfigSafe(domainName, selector string) error {
//
// Arguments are fixed literals — no user input is interpolated into the command,
// and it never goes through a shell (security.md).
//
// OpenDKIM 2.11.0~beta2, opendkim/opendkim.c, dkimf_reloader(): SIGUSR1 sets
// reload when a configuration file was given (-x in build/supervisord.conf);
// dkimf_config_reload(), from mlfi_negotiate() at the next milter connection,
// loads the configuration and opens the tables again (opendkim.8, OPERATION).
// supervisor 4.2.5, supervisor/rpcinterface.py, signalProcess(), and
// process.py, signal(): kill() of the program's own pid - opendkim itself, run
// with -f (not forked into the background) and exec'd with no shell.
func reloadViaSupervisor() error {
return supervisor.Signal("USR1", "opendkim")
}
+170
View File
@@ -0,0 +1,170 @@
package domain
import (
"os"
"strings"
"testing"
)
func readTables(t *testing.T, odk *OpenDKIM) (keyTable, signingTable string) {
t.Helper()
kt, err := os.ReadFile(odk.keyTablePath)
if err != nil {
t.Fatalf("read KeyTable: %v", err)
}
st, err := os.ReadFile(odk.signingTablePath)
if err != nil {
t.Fatalf("read SigningTable: %v", err)
}
return string(kt), string(st)
}
// The server's own name is signed like a sending domain - a key, a line in
// each table - without being one: no row in the registry, so no application
// can send as it.
func TestServerNameIsSignedWithoutBeingADomain(t *testing.T) {
svc, odk := newTestService(t, &fakeApps{})
svc.SetServerName("mail.example.org")
if _, err := svc.Add("example.com"); err != nil {
t.Fatalf("add domain: %v", err)
}
kt, st := readTables(t, odk)
for _, want := range []string{"mail.example.org mail.example.org:selfpost:", "example.com example.com:selfpost:"} {
if !strings.Contains(kt, want) {
t.Errorf("KeyTable lacks %q:\n%s", want, kt)
}
}
if !strings.Contains(st, "*@mail.example.org mail.example.org\n") {
t.Errorf("SigningTable does not sign the server's own name:\n%s", st)
}
domains, err := svc.List()
if err != nil || len(domains) != 1 {
t.Fatalf("registry holds %d domains, %v; want the one that was added", len(domains), err)
}
mail, err := svc.ServerMail()
if err != nil {
t.Fatalf("ServerMail: %v", err)
}
if mail.Name != "mail.example.org" || mail.Selector != "selfpost" ||
mail.Record.Name != "selfpost._domainkey.mail.example.org" || !strings.HasPrefix(mail.Record.Value, "v=DKIM1;") {
t.Errorf("server mail = %+v", mail)
}
// Asked again, it is the same key: the record published in DNS stays valid.
again, err := svc.ServerMail()
if err != nil || again.Record.Value != mail.Record.Value {
t.Errorf("the server's key changed between two readings (%v)", err)
}
}
// When the hostname is itself a sending domain, there is one key and one line,
// the domain's; deleting the domain leaves the server's mail signed with it.
func TestServerNameThatIsAlsoADomain(t *testing.T) {
svc, odk := newTestService(t, &fakeApps{})
svc.SetServerName("example.com")
d, err := svc.Add("example.com")
if err != nil {
t.Fatalf("add domain: %v", err)
}
_, st := readTables(t, odk)
if strings.Count(st, "*@example.com ") != 1 {
t.Errorf("the name is in the SigningTable more than once:\n%s", st)
}
asDomain, err := svc.DKIMRecord(d)
if err != nil {
t.Fatal(err)
}
mail, err := svc.ServerMail()
if err != nil || mail.Record.Value != asDomain.Value {
t.Fatalf("the server's record is not the domain's (%v)", err)
}
if err := svc.Delete(d.ID); err != nil {
t.Fatalf("delete domain: %v", err)
}
_, st = readTables(t, odk)
if !strings.Contains(st, "*@example.com example.com\n") {
t.Errorf("deleting the domain stopped the server's own mail being signed:\n%s", st)
}
after, err := svc.ServerMail()
if err != nil || after.Record.Value != asDomain.Value {
t.Errorf("deleting the domain changed the server's key (%v)", err)
}
}
// An installation upgraded to a version that signs the server's own name has
// tables without its line until something rebuilds them. The start does, once.
func TestEnsureServerSignedRebuildsOnce(t *testing.T) {
svc, odk := newTestService(t, &fakeApps{})
if _, err := svc.Add("example.com"); err != nil {
t.Fatal(err)
}
reloads := 0
odk.reload = func() error { reloads++; return nil }
// The tables as the earlier version left them; now the name is known.
svc.SetServerName("mail.example.org")
if rebuilt, err := svc.EnsureServerSigned(); err != nil || !rebuilt {
t.Fatalf("first start: rebuilt = %v, %v; want a rebuild", rebuilt, err)
}
if _, st := readTables(t, odk); !strings.Contains(st, "*@mail.example.org mail.example.org\n") || !strings.Contains(st, "*@example.com ") {
t.Errorf("SigningTable after the rebuild:\n%s", st)
}
if rebuilt, err := svc.EnsureServerSigned(); err != nil || rebuilt {
t.Errorf("second start: rebuilt = %v, %v; want nothing to do", rebuilt, err)
}
if reloads != 1 {
t.Errorf("OpenDKIM was reloaded %d times, want once", reloads)
}
// A name that only begins like the server's is not it.
other, otherOdk := newTestService(t, &fakeApps{})
other.SetServerName("mail.example.org.evil.example")
if _, err := other.EnsureServerSigned(); err != nil {
t.Fatal(err)
}
other.SetServerName("mail.example.org")
if signed, err := otherOdk.signs("mail.example.org", "selfpost"); err != nil || signed {
t.Errorf("a longer name was taken for the server's: %v, %v", signed, err)
}
}
// Tables that sign the server's name with the key of another selector - the
// default selector was changed - are rebuilt: OpenDKIM must sign with the key
// whose record the panel shows.
func TestEnsureServerSignedFollowsTheSelector(t *testing.T) {
svc, odk := newTestService(t, &fakeApps{})
svc.SetServerName("mail.example.org")
if _, err := svc.EnsureServerSigned(); err != nil {
t.Fatal(err)
}
svc.selector = "sp2027"
if rebuilt, err := svc.EnsureServerSigned(); err != nil || !rebuilt {
t.Fatalf("after the selector changed: rebuilt = %v, %v; want a rebuild", rebuilt, err)
}
kt, _ := readTables(t, odk)
if !strings.Contains(kt, "mail.example.org mail.example.org:sp2027:") || strings.Contains(kt, ":selfpost:") {
t.Errorf("KeyTable after the rebuild:\n%s", kt)
}
mail, err := svc.ServerMail()
if err != nil || mail.Selector != "sp2027" {
t.Errorf("the panel shows selector %q (%v), the tables sign with sp2027", mail.Selector, err)
}
}
// Without a hostname nothing is added and nothing is signed for it.
func TestNoServerName(t *testing.T) {
svc, odk := newTestService(t, &fakeApps{})
if _, err := svc.Add("example.com"); err != nil {
t.Fatal(err)
}
if _, st := readTables(t, odk); strings.Count(st, "\n") != 1 {
t.Errorf("SigningTable has more than the one domain:\n%s", st)
}
if _, err := svc.ServerMail(); err == nil {
t.Error("ServerMail without a hostname returned no error")
}
}
+104 -1
View File
@@ -38,6 +38,89 @@ type Service struct {
odk *OpenDKIM
apps Applications
selector string
// serverName is the server's own hostname, which the panel's notifications
// are sent from and signed as (SetServerName). Empty: nothing is signed
// for it.
serverName string
}
// SetServerName gives the service the server's own hostname. The panel sends
// its notifications from postmaster@ that name, and for a receiver to believe
// them the name is signed like a sending domain: it gets a DKIM key and a line
// in OpenDKIM's tables, without being a row of the registry - no application
// can send as it. The name comes from SELFPOST_HOSTNAME, which the entrypoint
// has validated; it is not user input.
func (s *Service) SetServerName(hostname string) {
s.serverName = hostname
}
// ServerMail is how the server's own name is signed.
type ServerMail struct {
Name string
Selector string
Record DKIMRecord
}
// ServerMail returns the name notifications are sent as, its DKIM selector and
// the record to publish for it, creating the key the first time. When the
// hostname is itself a sending domain, it is that domain's key and selector:
// one name has one key.
func (s *Service) ServerMail() (ServerMail, error) {
if s.serverName == "" {
return ServerMail{}, fmt.Errorf("server mail: no hostname")
}
selector, err := s.serverSelector()
if err != nil {
return ServerMail{}, err
}
if _, err := s.odk.EnsureKey(s.serverName, selector); err != nil {
return ServerMail{}, err
}
rec, err := s.odk.Record(s.serverName, selector)
if err != nil {
return ServerMail{}, err
}
return ServerMail{Name: s.serverName, Selector: selector, Record: rec}, nil
}
// EnsureServerSigned makes sure OpenDKIM's tables have the line for the
// server's own name, rebuilding them if they do not. The tables are otherwise
// rebuilt only when a domain is added or deleted, so an installation that was
// upgraded to a version with notifications would go on sending them unsigned
// until then, and one whose DKIM_SELECTOR_DEFAULT was changed would go on
// signing with the key of the old selector while the panel shows the record
// of the new. It reports whether it had to rebuild.
func (s *Service) EnsureServerSigned() (bool, error) {
if s.serverName == "" {
return false, nil
}
selector, err := s.serverSelector()
if err != nil {
return false, err
}
signed, err := s.odk.signs(s.serverName, selector)
if err != nil {
return false, err
}
if signed {
return false, nil
}
return true, s.resync()
}
// serverSelector is the selector the server's own name is signed with: that of
// the sending domain of the same name if there is one, the default otherwise.
func (s *Service) serverSelector() (string, error) {
domains, err := s.store.ListDomains()
if err != nil {
return "", err
}
for _, d := range domains {
if d.Name == s.serverName {
return d.DKIMSelector, nil
}
}
return s.selector, nil
}
// NewService builds the domain service. selectorDefault is the DKIM selector
@@ -125,6 +208,14 @@ func (s *Service) Delete(id int64) error {
if err := s.apps.Resync(); err != nil {
return fmt.Errorf("rebuild sender map after deleting %s: %w", d.Name, err)
}
// A sending domain named like the server shared its key with the server's
// own mail. The resync above kept the name in the tables under the default
// selector: the same key if that was the domain's selector, a new one -
// with a new record to publish, which Settings shows - if it was not.
// Either way the directory holds the key now in use.
if d.Name == s.serverName {
return nil
}
if err := s.odk.RemoveKey(d.Name); err != nil {
// The domain is gone from the registry and tables; a leftover key
// directory is harmless. Surface it so it is not silently ignored.
@@ -172,9 +263,21 @@ func (s *Service) resync() error {
if err != nil {
return err
}
signing := make([]SigningDomain, 0, len(domains))
signing := make([]SigningDomain, 0, len(domains)+1)
registered := false
for _, d := range domains {
signing = append(signing, SigningDomain{Name: d.Name, Selector: d.DKIMSelector})
if d.Name == s.serverName {
registered = true
}
}
// The server's own name, for the panel's notifications - unless it is a
// sending domain already and so in the tables above.
if s.serverName != "" && !registered {
if _, err := s.odk.EnsureKey(s.serverName, s.selector); err != nil {
return fmt.Errorf("dkim key for %s: %w", s.serverName, err)
}
signing = append(signing, SigningDomain{Name: s.serverName, Selector: s.selector})
}
return s.odk.Rebuild(signing)
}
+10 -1
View File
@@ -9,13 +9,22 @@ import (
)
// certWarnDays is how close to expiry the certificate has to be before the
// status page complains. Let's Encrypt renews at 30 days left, so two weeks is
// status page complains. certbot renews a 90-day certificate at 30 days left
// - a fixed 30 days before 4.0, a third of the lifetime from 4.0 (certbot
// docs, certbot/docs/using.rst, "Renewing certificates") - so two weeks is
// comfortably past the point where automatic renewal should have happened.
// For a 45-day certificate a third is 15 days, one day ahead of this warning
// (assumptions A-64).
const certWarnDays = 14
// Certificate is the state of the TLS certificate Postfix serves on 465/587
// (guide § Environment variables: TLS_CERT_FILE). The panel only reads it —
// the file is supplied by the reverse proxy through a read-only mount.
// Postfix itself serves the copy build/postfix-config.sh makes in
// /etc/postfix/tls-internal, refreshed by build/postfix-cert-reload.sh up to
// TLS_RELOAD_INTERVAL_SECONDS later, and each smtpd loads it when it starts
// (Postfix 3.7.11, src/smtpd/smtpd.c, pre_jail_init(), TLS_SERVER_INIT) - so
// the mounted file can be newer than what is served (assumptions A-65).
type Certificate struct {
Path string
Subject string
+26 -1
View File
@@ -33,7 +33,10 @@ func TestWorstPicksMostSevere(t *testing.T) {
}
func TestParseProcesses(t *testing.T) {
// Real supervisorctl output: column-aligned, one line per program.
// Real supervisorctl output: column-aligned, one line per program. Where
// and when it was seen is not recorded; its columns are those of
// supervisor 4.2.5 supervisorctl.py _show_statuses() and its FATAL text
// supervisor/process.py's spawnerr.
out := `opendkim RUNNING pid 21, uptime 0:04:10
panel RUNNING pid 22, uptime 0:04:09
postfix FATAL Exited too quickly (process log may have details)
@@ -165,6 +168,7 @@ func writeCert(t *testing.T, path, cn string, validFor time.Duration) {
}
func TestLivenessFromParsedProcesses(t *testing.T) {
// Composed, in the columns of TestParseProcesses.
allRunning := `opendkim RUNNING pid 21, uptime 0:04:10
panel RUNNING pid 22, uptime 0:04:09
postfix RUNNING pid 23, uptime 0:04:08
@@ -188,3 +192,24 @@ postfix FATAL Exited too quickly
}
}
}
// Composed, after supervisor 4.2.5 rpcinterface.py _interpretProcessInfo()
// (str() of a timedelta). "127:00:01" is not a form supervisor prints - it
// only shows that the parser takes hours past 24.
func TestUptime(t *testing.T) {
for detail, want := range map[string]time.Duration{
"pid 41, uptime 0:03:12": 3*time.Minute + 12*time.Second,
"pid 41, uptime 18 days, 3:02:11": 18*24*time.Hour + 3*time.Hour + 2*time.Minute + 11*time.Second,
"pid 41, uptime 1 day, 0:00:00": 24 * time.Hour,
"pid 7, uptime 127:00:01": 127*time.Hour + time.Second,
} {
if got, ok := Uptime(detail); !ok || got != want {
t.Errorf("Uptime(%q) = %s, %v; want %s", detail, got, ok, want)
}
}
for _, detail := range []string{"", "Not started", "Oct 12 04:00 PM", "pid 41, uptime soon", "pid 41, uptime x days, 0:00:01", "pid 41, uptime 0:00"} {
if got, ok := Uptime(detail); ok {
t.Errorf("Uptime(%q) = %s, want none", detail, got)
}
}
}
+31 -11
View File
@@ -33,11 +33,14 @@ const (
// status page's answer to "is the server itself under strain", next to the
// component checks that answer "is mail flowing".
//
// The numbers come from the kernel's /proc filesystem, which a container shares
// with its host unless it was started with its own namespaces — so CPU and
// memory describe the host, while /proc/net/dev describes whatever network
// namespace the container is in (its own veth under the default bridge, the
// host's interfaces under network_mode: host).
// The numbers come from the kernel's /proc filesystem. /proc/stat and
// /proc/meminfo are system-wide and no namespace changes them (proc(5):
// "kernel/system statistics", "statistics about memory usage on the
// system"), so CPU and memory describe the host, while /proc/net/dev
// describes whatever network namespace the container is in
// (network_namespaces(7) names /proc/net among what a network namespace
// isolates) - its own veth under the default bridge, the host's interfaces
// under network_mode: host.
type Machine struct {
CPU CPU
Memory Memory
@@ -276,7 +279,9 @@ func (c CPU) contextText() string {
// MemAvailable rather than MemFree: Linux spends every spare page on cache, so
// MemFree on a healthy machine is near zero and would report a permanent
// emergency. MemAvailable is the kernel's own estimate of what a new workload
// could actually get.
// could actually get (proc(5), /proc/meminfo, MemAvailable "since Linux
// 3.14": "An estimate of how much memory is available for starting new
// applications, without swapping").
func readMemory(root string) Memory {
var m Memory
fields, err := readMeminfo(root)
@@ -398,11 +403,15 @@ func readCPUTimes(root string) (cpuTimes, int, error) {
continue
}
if fields[0] != "cpu" {
cores++ // cpu0, cpu1, … — one line per core
cores++ // cpu0, cpu1, … — one line per online logical CPU
continue
}
// user nice system idle iowait irq softirq steal (guest fields are
// already counted inside user/nice, so they are left out).
// already counted inside user/nice, so they are left out). Field
// order: proc(5), /proc/stat. That guest time is in user/nice too:
// Linux kernel/sched/cputime.c, account_guest_time() adds it to
// CPUTIME_USER (or CPUTIME_NICE) as well as CPUTIME_GUEST (torvalds/
// linux master read 2026-10-14; the kernel is the host's).
for i, f := range fields[1:] {
if i >= 8 {
break
@@ -428,7 +437,11 @@ func readCPUTimes(root string) (cpuTimes, int, error) {
// readCPUTopology returns physical core and logical thread counts from
// /proc/cpuinfo. Zeroes mean the file was missing or empty; the caller falls
// back to the cpuN count from /proc/stat.
// back to the cpuN count from /proc/stat. The keys are x86's: Linux
// arch/x86/kernel/cpu/proc.c, show_cpuinfo() prints "processor" and, on SMP,
// "physical id" and "core id" (show_cpuinfo_core(); torvalds/linux master
// read 2026-10-14). proc(5) says only that the items are architecture
// dependent; where they are missing each processor counts as a core.
func readCPUTopology(root string) (cores, threads int) {
data, err := os.ReadFile(filepath.Join(root, "cpuinfo"))
if err != nil {
@@ -522,7 +535,9 @@ func isZeroLoad(load [3]float64) bool {
}
// readMeminfo returns the /proc/meminfo entries this package uses, in bytes.
// The file reports kB (kibibytes, despite the label) for these fields.
// The file reports kB (kibibytes, despite the label) for these fields: Linux
// fs/proc/meminfo.c, show_val_kb() prints pages << (PAGE_SHIFT - 10)
// (torvalds/linux master read 2026-10-14).
func readMeminfo(root string) (map[string]uint64, error) {
data, err := os.ReadFile(filepath.Join(root, "meminfo"))
if err != nil {
@@ -557,7 +572,12 @@ func readMeminfo(root string) (map[string]uint64, error) {
// readNetDev returns each interface's byte counters from /proc/net/dev, loopback
// excluded. Each line is " eth0: <rx bytes> <rx packets> … <tx bytes> …" — the
// name is separated by a colon, which may or may not have a space after it, so
// the split is on the colon and not on whitespace.
// the split is on the colon and not on whitespace. proc(5), /proc/net/dev:
// eight receive columns (bytes first), then the transmit ones (bytes first).
// Linux net/core/net-procfs.c, dev_seq_printf_stats(): "%6s: %7llu ..." in
// master read 2026-10-14; the host's kernel may be older, and the colon is
// all this relies on (net/core/dev.c, dev_valid_name() refuses ':' in a
// device name).
func readNetDev(root string) (map[string]netCounters, error) {
data, err := os.ReadFile(filepath.Join(root, "net", "dev"))
if err != nil {
+3 -1
View File
@@ -9,7 +9,9 @@ import (
// fakeProc writes a /proc-shaped directory the sampler can be pointed at, so
// the parsing and the arithmetic are tested against known counters instead of
// whatever the machine running the tests happens to be doing.
// whatever the machine running the tests happens to be doing. The file
// contents in this test are composed, in the layouts proc(5) gives for
// /proc/stat, /proc/meminfo and /proc/net/dev.
func fakeProc(t *testing.T, stat, meminfo, loadavg, netdev string) string {
t.Helper()
dir := t.TempDir()
+3 -1
View File
@@ -16,7 +16,9 @@ import (
// add to a poll.
const milterDialTimeout = time.Second
// CheckMilter probes a milter address in the two forms Postfix takes:
// CheckMilter probes a milter address in the two forms build/postfix-config.sh
// lets INBOUND_ANTISPAM_MILTER take (Postfix 3.7.11, src/milter/milter8.c,
// milter8_connect(), also accepts "local:", which that script refuses):
// unix:/path is stat-ed like any milter socket, and inet:host:port is dialled
// and the connection closed at once, with nothing written. required says what
// a failure costs, as for CheckSocket: an error when mail stops without the
+46 -2
View File
@@ -3,7 +3,9 @@ package health
import (
"fmt"
"os/exec"
"strconv"
"strings"
"time"
)
// supervisorConf is the supervisord configuration the panel's other control
@@ -36,7 +38,9 @@ type Process struct {
// The command takes fixed arguments and no user input, so it never goes through
// a shell (security.md). `supervisorctl status` deliberately exits non-zero when
// some program is not running, so the output is parsed first and the exit status
// only matters when nothing could be parsed from it.
// only matters when nothing could be parsed from it. supervisor 4.2.5,
// supervisor/supervisorctl.py, do_status(): exit status 3 (NOT_RUNNING) when
// any program is in STOPPED_STATES, 4 when supervisord does not answer.
func Processes() ([]Process, error) {
cmd := exec.Command("supervisorctl", "-c", supervisorConf, "status")
out, err := cmd.CombinedOutput()
@@ -52,7 +56,8 @@ func Processes() ([]Process, error) {
// supervisorStates are the state words supervisord prints. Lines whose second
// field is not one of them are not status lines (banners, error text) and are
// skipped, so unexpected output cannot masquerade as a process.
// skipped, so unexpected output cannot masquerade as a process. supervisor
// 4.2.5, supervisor/states.py, ProcessStates.
var supervisorStates = map[string]bool{
"STOPPED": true,
"STARTING": true,
@@ -66,6 +71,10 @@ var supervisorStates = map[string]bool{
// parseProcesses turns supervisorctl's tabular output into Process values. Each
// status line is "<name> <STATE> <detail...>", column-aligned with spaces.
// supervisor 4.2.5, supervisor/supervisorctl.py, _show_statuses(): the
// template is "%(namespec)-<maxlen+3>s%(state)-10s%(desc)s" (maxlen at least
// 30, no state word longer than 8), and an unknown name prints
// "<name>: ERROR (no such process)".
func parseProcesses(out string) []Process {
var procs []Process
for _, line := range strings.Split(out, "\n") {
@@ -85,6 +94,41 @@ func parseProcesses(out string) []Process {
return procs
}
// Uptime reads for how long a running program has been up from the detail
// supervisord prints for it: "pid 41, uptime 0:03:12", or with days in front,
// "pid 41, uptime 18 days, 3:02:11". supervisor 4.2.5,
// supervisor/rpcinterface.py, _interpretProcessInfo(): 'pid %s, uptime %s'
// with Python's str() of a timedelta from the spawn time ("1 day, ...").
func Uptime(detail string) (time.Duration, bool) {
_, rest, found := strings.Cut(detail, "uptime ")
if !found {
return 0, false
}
var days int
if d, clock, hasDays := strings.Cut(rest, ", "); hasDays {
n, _, _ := strings.Cut(d, " ")
v, err := strconv.Atoi(n)
if err != nil || v < 0 {
return 0, false
}
days, rest = v, clock
}
parts := strings.Split(strings.TrimSpace(rest), ":")
if len(parts) != 3 {
return 0, false
}
var hms [3]int
for i, p := range parts {
v, err := strconv.Atoi(p)
if err != nil || v < 0 {
return 0, false
}
hms[i] = v
}
return time.Duration(days)*24*time.Hour + time.Duration(hms[0])*time.Hour +
time.Duration(hms[1])*time.Minute + time.Duration(hms[2])*time.Second, true
}
// processStatus grades a supervisord state. A one-shot program that is not
// running is healthy; anything else that is not RUNNING means a component of
// the mail path is down or flapping.

Some files were not shown because too many files have changed in this diff Show More