Compare commits
38
Commits
v2.0.0
..
ae06d8b47e
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
ae06d8b47e | ||
|
|
1f44e40382 | ||
|
|
d398f488e2 | ||
|
|
89774ab847 | ||
|
|
c731a98a0f | ||
|
|
5f86457848 | ||
|
|
00feb7c1c7 | ||
|
|
d2a778edc0 | ||
|
|
7192c3861e | ||
|
|
e0ccff257d | ||
|
|
c1da784193 | ||
|
|
9d330bed77 | ||
|
|
970c8ca8da | ||
|
|
19156814b8 | ||
|
|
d2b97b3963 | ||
|
|
d4efdb483d | ||
|
|
56c5dfee40 | ||
|
|
82d4a209be | ||
|
|
737611a07e | ||
|
|
eea40bcd49 | ||
|
|
27fd355e53 | ||
|
|
94e94490af | ||
|
|
28ec784ae2 | ||
|
|
a3cc52d6d4 | ||
|
|
d65d632505 | ||
|
|
dd0fefdcb7 | ||
|
|
75ebbf6a57 | ||
|
|
ede3783734 | ||
|
|
d70371a50c | ||
|
|
21e961cb29 | ||
|
|
82f212ab39 | ||
|
|
b55b33ae92 | ||
|
|
8c9a71e6a0 | ||
|
|
646ec1e149 | ||
|
|
8496b289ec | ||
|
|
4dd46ea355 | ||
|
|
505260e227 | ||
|
|
09d9625f32 |
+176
@@ -5,6 +5,182 @@ Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); version
|
|||||||
|
|
||||||
## [Unreleased]
|
## [Unreleased]
|
||||||
|
|
||||||
|
## [2.1.0] - 2026-10-12
|
||||||
|
|
||||||
|
### Security
|
||||||
|
|
||||||
|
- **The level-2 rate limit and an application's client IP list held only for
|
||||||
|
the first message of a connection.** After a refused `MAIL FROM`, a second
|
||||||
|
one on the same connection was accepted, and so was every message after the
|
||||||
|
first that a connection carried: an application over its limit could go on
|
||||||
|
sending, and one restricted to certain client addresses could send from any
|
||||||
|
other with a retry. In the code since `1.9.0` (the client IP list) and
|
||||||
|
earlier (the limit); found by the technical review of 2026-10-12 and
|
||||||
|
reproduced on the test stand. Postfix now gives the panel's milters the
|
||||||
|
client address with every message, the limit no longer depends on it, and an
|
||||||
|
application with a client IP list is refused when no address is known.
|
||||||
|
- Reviews of the inbound antispam feature before its release (technical and
|
||||||
|
security, 2026-10-12; nothing Critical or High in the feature itself). Fixed:
|
||||||
|
a client's HELO name could supply the sender, recipient and reply of an
|
||||||
|
inbound log row; the log's row cap was applied only every six hours; rows of
|
||||||
|
a connection's later messages had no client address; two changes of the
|
||||||
|
filter lists at once could leave the filter's files behind the page; a held
|
||||||
|
message was logged twice; after an unclean stop of the panel the refusals of
|
||||||
|
its last seconds were logged twice; with the spam filter down, a sender's
|
||||||
|
own verdict headers were recorded as the filter's. Recorded as accepted in
|
||||||
|
[docs/security.md](docs/security.md): an allow entry for a sender address
|
||||||
|
trusts an address anyone can claim; the list files are readable by anyone on
|
||||||
|
the host.
|
||||||
|
- `INBOUND_ANTISPAM_MILTER` and `INBOUND_JOURNAL_MILTER_SOCKET` are checked
|
||||||
|
character by character at start; the check looked at their first characters
|
||||||
|
only.
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
|
||||||
|
- **Port 25 logs as `postfix/inbound`.** Its lines in the mail log (Server ›
|
||||||
|
System log) now read `postfix/inbound/smtpd[…]` where they read
|
||||||
|
`postfix/smtpd[…]`; the submission ports keep `postfix/smtpd`. This is how a
|
||||||
|
refusal on port 25 is told from one on 465 / 587.
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- Schema: migration `0002_inbound_spam_log.sql`, the first after the 2.0
|
||||||
|
baseline - two empty tables for the inbound antispam journal and the filter
|
||||||
|
lists (inbound-antispam-panel). A 2.0.x database is migrated when the panel
|
||||||
|
starts.
|
||||||
|
- **Inbound mail is recorded.** With the inbound relay on, the panel runs a
|
||||||
|
second milter, the last one on port 25, and writes a row for every recipient
|
||||||
|
of a message that was accepted for an inbound domain: client address,
|
||||||
|
envelope, subject and - with a spam filter attached - the filter's action,
|
||||||
|
score and heaviest symbols. It only watches: it refuses nothing and Postfix
|
||||||
|
accepts mail when it is down.
|
||||||
|
- **Refused inbound mail is recorded too**, from the mail log: a recipient
|
||||||
|
Postfix refused at an inbound domain (unknown user, sender domain not
|
||||||
|
found), and a message the spam filter rejected, deferred, held or
|
||||||
|
discarded. The row is dated by its log line and carries the reply text. A
|
||||||
|
refusal to relay for a domain that is not an inbound one is not recorded.
|
||||||
|
- The inbound journal is pruned every six hours: rows older than 90 days
|
||||||
|
(`INBOUND_SPAM_LOG_RETENTION_DAYS`, 7-365) and anything beyond 10 000 rows.
|
||||||
|
- The antispam compose fragment mounts `antispam/rspamd/milter_headers.conf`
|
||||||
|
into the rspamd sidecar, which makes it write its verdict into
|
||||||
|
`X-Rspamd-Action` and `X-Spamd-Result`. **Copy `deploy/antispam/rspamd/`
|
||||||
|
next to the fragment before starting it** - Docker creates a directory in
|
||||||
|
place of a missing file and the sidecar then fails to read its
|
||||||
|
configuration.
|
||||||
|
- **Allow and deny lists for the spam filter**, kept by SelfPost and applied
|
||||||
|
by rspamd: sender addresses, sender domains, client addresses and networks.
|
||||||
|
They are written as six files under `data/antispam/`, which the fragment
|
||||||
|
mounts into the sidecar together with `antispam/rspamd/multimap.conf`;
|
||||||
|
rspamd rereads a changed file by itself. Deny wins over allow. The files
|
||||||
|
are there from the first start with the inbound relay on.
|
||||||
|
- **Inbound › Log**: the journal as a page - time, sender and client address,
|
||||||
|
recipient, subject and decision, filtered by domain, decision and day. The
|
||||||
|
decision opens a sheet over the page with who decided and why: the filter's
|
||||||
|
verdict and symbols, or the reply a refusal was given. A user with the
|
||||||
|
*domain* role sees the rows of the inbound domains assigned to them. For the
|
||||||
|
global role a row links to the filter lists with the sender filled in:
|
||||||
|
*Deny sender* on accepted mail, *Allow sender* on mail the filter refused -
|
||||||
|
unless the sender is spelt in a way the lists do not take.
|
||||||
|
- **Inbound › Filter lists** (global role): add and remove allow and deny
|
||||||
|
entries; the page says where the lists go and whether the filter is on to
|
||||||
|
read them. Each entry shows how often it decided a message - *Matched*, or
|
||||||
|
*never* - and its kind opens a sheet with the note, the day it was added and
|
||||||
|
the last match, so that entries nothing matches any more can be found and
|
||||||
|
removed. The count is read from the inbound log and reaches back as far as
|
||||||
|
the log does. A network of one address (`192.0.2.7/32`) is stored as the
|
||||||
|
address, so it cannot be on a list twice.
|
||||||
|
- **Server › Settings**: *Keep inbound log rows, days*, shown with the inbound
|
||||||
|
relay on.
|
||||||
|
- Help: *Inbound log* and *Filter lists*.
|
||||||
|
- New setting `ANTISPAM_DIR` (default `/data/antispam`).
|
||||||
|
- **The spam filter signs what it judged.** rspamd adds `X-SelfPost-Filter` to
|
||||||
|
every message it lets through - an HMAC of the message's queue id under a
|
||||||
|
key SelfPost creates in `data/antispam/filter.key` - and the inbound log
|
||||||
|
shows a verdict only next to a signature that fits. Nothing to configure,
|
||||||
|
but the sidecar needs the new `antispam/rspamd/milter_headers.conf` and a
|
||||||
|
restart to read it; until then accepted mail is logged as *selfpost*, with
|
||||||
|
no verdict.
|
||||||
|
- A full backup carries the inbound log, the filter lists and their files
|
||||||
|
under `data/antispam/`. It does not carry the `antispam/` directory with the
|
||||||
|
compose fragment and the rspamd configuration - copy that from
|
||||||
|
`deploy/antispam/` when restoring.
|
||||||
|
- New setting `INBOUND_JOURNAL_MILTER_SOCKET` (default
|
||||||
|
`/run/selfpost/inbound-journal.sock`).
|
||||||
|
|
||||||
|
## [2.0.1] - 2026-10-11
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
|
||||||
|
- **A message that is not a report this relay keeps is dropped at once.**
|
||||||
|
One that cannot be read as an aggregate report, reports on a domain not
|
||||||
|
sent from here, or came to another domain's report address used to be put
|
||||||
|
off and offered again by Postfix for days, counted as a parse failure each
|
||||||
|
time. It is now accepted and discarded - no retry, and no bounce, since the
|
||||||
|
envelope sender of such a message is whatever its author wrote. It counts
|
||||||
|
as one parse failure and the reason is in the mail log, on the message's
|
||||||
|
own delivery line (Server › System log, `dropped: …`). Only a failure of
|
||||||
|
the database is still retried.
|
||||||
|
- **A full backup no longer holds the Postfix queue**, from the panel or from
|
||||||
|
`selfpost-backup`. It is taken while the instance runs, and a copy of a
|
||||||
|
queue Postfix is working on can hold a message twice or not at all. Mail
|
||||||
|
still waiting for delivery is not carried over; a restored instance starts
|
||||||
|
with an empty queue. To keep the queue, stop the instance and `tar` the
|
||||||
|
project directory.
|
||||||
|
- DMARC reports are kept 500 **for each domain** (and 90 days), not 500 for
|
||||||
|
the whole server - see Security.
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
|
||||||
|
- **Full backup from the panel works.** Since the Postfix queue moved under
|
||||||
|
`/data` in 1.3.0 the button answered with a 200 and ten bytes: the archive
|
||||||
|
includes the queue, Postfix keeps the queue readable by its own user alone,
|
||||||
|
and the panel is not that user ("open /data/postfix/queue/active:
|
||||||
|
permission denied" in the log). `selfpost-backup` run with `docker exec`
|
||||||
|
was not affected - it runs as root. The queue is no longer part of a backup
|
||||||
|
(see Changed), so the two archives are the same again. Everything is read
|
||||||
|
through before the download starts, so a failure is a message on the page
|
||||||
|
and not a file cut short. One thing the panel may still be unable to read
|
||||||
|
is the private key in `certs/`: it then refuses and says what to do -
|
||||||
|
`chgrp 998 certs/privkey.pem && chmod 640 certs/privkey.pem` on the host,
|
||||||
|
998 being the container's `selfpost` group, whose id is now fixed in the
|
||||||
|
image.
|
||||||
|
|
||||||
|
### Security
|
||||||
|
|
||||||
|
Review of the DMARC report path (Fable, 2026-10-11), which started to run for
|
||||||
|
the first time in 2.0.0. Nothing Critical or High.
|
||||||
|
|
||||||
|
- **A report is bounded before it is stored.** The XML comes from anyone who
|
||||||
|
knows a report address, and its 8 MiB limit left room for some 160 000
|
||||||
|
records or a single name of megabytes - rows in the database and cells of
|
||||||
|
the report page. A report holds at most 5 000 records (a larger one is
|
||||||
|
dropped), every text field is cut to 255 bytes, and a record's count is
|
||||||
|
capped so the sums cannot wrap.
|
||||||
|
- **One domain's reports cannot push out another's.** The cap of 500 was for
|
||||||
|
the whole server, so a stranger filing reports under one domain's address
|
||||||
|
could evict every other domain's history. It is counted per domain now.
|
||||||
|
- The delivery command runs one at a time, not up to a hundred in parallel.
|
||||||
|
- The reason written to the mail log for a dropped message is cut to 200
|
||||||
|
bytes and anything outside printable ASCII is replaced: parts of it are the
|
||||||
|
sender's own bytes (a header line, a file name inside an archive).
|
||||||
|
- `SELFPOST_DATA_DIR` / `SELFPOST_DB_PATH` with whitespace or `$` are refused
|
||||||
|
at start when DMARC ingest is on, instead of handing the delivery command
|
||||||
|
another path and so an empty database of its own.
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- e2e: the run now ends on port 25 with both optional features on. A DMARC
|
||||||
|
report is mailed to a domain's hosted address and has to appear under
|
||||||
|
Outbound › DMARC reports; a report on another domain and a message with no
|
||||||
|
report are accepted and dropped; mail for an inbound domain reaches its
|
||||||
|
upstream and other recipients are refused. Until now the suite only read
|
||||||
|
back the generated configuration of these features.
|
||||||
|
- e2e: full backup and restore. The run takes a backup from the panel, throws
|
||||||
|
the installation away, unpacks the archive into the empty project directory
|
||||||
|
and starts it: the administrator signs in, the domain has the same DKIM key,
|
||||||
|
and mail sent by the application verifies against the record published
|
||||||
|
before the backup.
|
||||||
|
|
||||||
## [2.0.0] - 2026-10-11
|
## [2.0.0] - 2026-10-11
|
||||||
|
|
||||||
The panel redesign. A breaking release: every panel URL changed and the
|
The panel redesign. A breaking release: every panel URL changed and the
|
||||||
|
|||||||
@@ -112,7 +112,7 @@ docker run --rm -d --name selfpost-try \
|
|||||||
-e SELFPOST_HOSTNAME=mail.local.test \
|
-e SELFPOST_HOSTNAME=mail.local.test \
|
||||||
-e PANEL_COOKIE_SECURE=false \
|
-e PANEL_COOKIE_SECURE=false \
|
||||||
-v selfpost-try-data:/data \
|
-v selfpost-try-data:/data \
|
||||||
ghcr.io/mixeme/selfpost:2.0.0
|
ghcr.io/mixeme/selfpost:2.1.0
|
||||||
```
|
```
|
||||||
|
|
||||||
**Get the setup URL** (pick one):
|
**Get the setup URL** (pick one):
|
||||||
|
|||||||
+4
-1
@@ -62,7 +62,10 @@ RUN useradd --system --no-create-home --shell /usr/sbin/nologin panel
|
|||||||
# entrypoint.sh) — lets OpenDKIM read the panel-owned keys and lets Postfix read
|
# entrypoint.sh) — lets OpenDKIM read the panel-owned keys and lets Postfix read
|
||||||
# the sasldb2/sender map, and lets the panel reach the supervisor control socket
|
# the sasldb2/sender map, and lets the panel reach the supervisor control socket
|
||||||
# to signal OpenDKIM/Postfix reloads without any process running as root.
|
# to signal OpenDKIM/Postfix reloads without any process running as root.
|
||||||
RUN groupadd --system selfpost \
|
# The id is fixed, and it is the one the group has always had: an operator
|
||||||
|
# gives the TLS private key on the host to this group by number, so that the
|
||||||
|
# panel can put it into a full backup (guide § Taking a full backup).
|
||||||
|
RUN groupadd --system --gid 998 selfpost \
|
||||||
&& usermod -aG selfpost panel \
|
&& usermod -aG selfpost panel \
|
||||||
&& usermod -aG selfpost opendkim \
|
&& usermod -aG selfpost opendkim \
|
||||||
&& usermod -aG selfpost postfix
|
&& usermod -aG selfpost postfix
|
||||||
|
|||||||
@@ -112,6 +112,15 @@ for f in relay_domains transport relay_recipients tls_policy; do
|
|||||||
done
|
done
|
||||||
chown panel:selfpost /data/postfix
|
chown panel:selfpost /data/postfix
|
||||||
chmod 2750 /data/postfix
|
chmod 2750 /data/postfix
|
||||||
|
|
||||||
|
# Filter lists for the external spam filter (written by the panel when
|
||||||
|
# INBOUND_RELAY_ENABLE=true, read by the antispam sidecar, which mounts this
|
||||||
|
# directory). The sidecar runs under a user of its own in another container,
|
||||||
|
# so the directory and its files are readable by anyone: they hold sender
|
||||||
|
# addresses and networks, no secrets.
|
||||||
|
mkdir -p /data/antispam
|
||||||
|
chown panel:selfpost /data/antispam
|
||||||
|
chmod 0755 /data/antispam
|
||||||
chown panel:selfpost /data/postfix/sender_login_maps
|
chown panel:selfpost /data/postfix/sender_login_maps
|
||||||
chmod 0640 /data/postfix/sender_login_maps
|
chmod 0640 /data/postfix/sender_login_maps
|
||||||
|
|
||||||
|
|||||||
+57
-2
@@ -117,6 +117,9 @@ SUBMISSION_ENABLE="${SUBMISSION_ENABLE:-false}"
|
|||||||
INBOUND_RELAY_ENABLE="${INBOUND_RELAY_ENABLE:-false}"
|
INBOUND_RELAY_ENABLE="${INBOUND_RELAY_ENABLE:-false}"
|
||||||
INBOUND_ANTISPAM_MILTER="${INBOUND_ANTISPAM_MILTER:-}"
|
INBOUND_ANTISPAM_MILTER="${INBOUND_ANTISPAM_MILTER:-}"
|
||||||
INBOUND_ANTISPAM_MILTER_ACTION="${INBOUND_ANTISPAM_MILTER_ACTION:-accept}"
|
INBOUND_ANTISPAM_MILTER_ACTION="${INBOUND_ANTISPAM_MILTER_ACTION:-accept}"
|
||||||
|
# The panel's inbound journal milter (monitoring, fail-open), on port 25 only
|
||||||
|
# and only while the inbound relay is on. Matched by postfix-wrapper.sh.
|
||||||
|
INBOUND_JOURNAL_SOCK="${INBOUND_JOURNAL_MILTER_SOCKET:-/run/selfpost/inbound-journal.sock}"
|
||||||
INBOUND_RATE_LIMIT_MESSAGES_PER_IP="${INBOUND_RATE_LIMIT_MESSAGES_PER_IP:-20}"
|
INBOUND_RATE_LIMIT_MESSAGES_PER_IP="${INBOUND_RATE_LIMIT_MESSAGES_PER_IP:-20}"
|
||||||
INBOUND_MESSAGE_SIZE_LIMIT="${INBOUND_MESSAGE_SIZE_LIMIT:-26214400}"
|
INBOUND_MESSAGE_SIZE_LIMIT="${INBOUND_MESSAGE_SIZE_LIMIT:-26214400}"
|
||||||
RELAY_DOMAINS_MAP="${POSTFIX_RELAY_DOMAINS:-/data/postfix/relay_domains}"
|
RELAY_DOMAINS_MAP="${POSTFIX_RELAY_DOMAINS:-/data/postfix/relay_domains}"
|
||||||
@@ -227,6 +230,14 @@ postconf -e \
|
|||||||
"smtpd_milters={ unix:${OPENDKIM_SOCK}, default_action=tempfail }, { unix:${JOURNAL_SOCK}, default_action=accept }" \
|
"smtpd_milters={ unix:${OPENDKIM_SOCK}, default_action=tempfail }, { unix:${JOURNAL_SOCK}, default_action=accept }" \
|
||||||
"non_smtpd_milters="
|
"non_smtpd_milters="
|
||||||
|
|
||||||
|
# What Postfix tells a milter with MAIL FROM: its defaults plus the client's
|
||||||
|
# address. The panel's milters judge every message by that address (an
|
||||||
|
# application's client-address list) and record it (the inbound journal), and
|
||||||
|
# the address given once at connect is not there for a connection's later
|
||||||
|
# messages - see MailFrom in internal/milter/milter.go.
|
||||||
|
postconf -e \
|
||||||
|
"milter_mail_macros=i {auth_type} {auth_authen} {auth_author} {mail_addr} {mail_host} {mail_mailer} {client_addr}"
|
||||||
|
|
||||||
# Bounded milter timeouts: a *hung* milter (socket accepts but never replies)
|
# Bounded milter timeouts: a *hung* milter (socket accepts but never replies)
|
||||||
# must fail open just like a crash, not stall mail acceptance until the Postfix
|
# must fail open just like a crash, not stall mail acceptance until the Postfix
|
||||||
# defaults (300s content) elapse. With default_action per milter, a
|
# defaults (300s content) elapse. With default_action per milter, a
|
||||||
@@ -291,11 +302,23 @@ if [ "${INBOUND_RELAY_ENABLE}" = "true" ] || [ "${DMARC_REPORTS_ENABLE}" = "true
|
|||||||
# attribute" on every report. They go into the service line itself.
|
# attribute" on every report. They go into the service line itself.
|
||||||
# ${recipient} is expanded by pipe(8), not by the shell: the command
|
# ${recipient} is expanded by pipe(8), not by the shell: the command
|
||||||
# learns which report address the message came to from its argument.
|
# learns which report address the message came to from its argument.
|
||||||
postconf -M 'dmarc-ingest/unix=dmarc-ingest unix - n n - - pipe flags=Rq user=panel argv=/usr/local/bin/dmarc-ingest ${recipient}'
|
# One at a time (the 1 before "pipe"): reports are in no hurry, every run
|
||||||
|
# writes to the database the panel writes to, and a hundred of them in
|
||||||
|
# parallel - Postfix's default - is what a flood of reports would start.
|
||||||
|
postconf -M 'dmarc-ingest/unix=dmarc-ingest unix - n n - 1 pipe flags=Rq user=panel argv=/usr/local/bin/dmarc-ingest ${recipient}'
|
||||||
# One recipient per run, so ${recipient} is one argument.
|
# One recipient per run, so ${recipient} is one argument.
|
||||||
postconf -e "dmarc-ingest_destination_recipient_limit=1"
|
postconf -e "dmarc-ingest_destination_recipient_limit=1"
|
||||||
# pipe(8) starts the command with export_environment only, so what the
|
# pipe(8) starts the command with export_environment only, so what the
|
||||||
# command reads from the container's environment is named there.
|
# command reads from the container's environment is named there.
|
||||||
|
# export_environment is a list split at whitespace in which Postfix
|
||||||
|
# expands "$": a path with either would reach the command as another
|
||||||
|
# path, and it would quietly open a database of its own.
|
||||||
|
case "${SELFPOST_DATA_DIR:-}${SELFPOST_DB_PATH:-}" in
|
||||||
|
*[[:space:]]* | *'$'*)
|
||||||
|
echo "postfix-config: SELFPOST_DATA_DIR and SELFPOST_DB_PATH must not contain whitespace or \$ when DMARC_REPORTS_ENABLE=true" >&2
|
||||||
|
exit 1
|
||||||
|
;;
|
||||||
|
esac
|
||||||
DMARC_INGEST_ENV="DMARC_REPORTS_ENABLE=true"
|
DMARC_INGEST_ENV="DMARC_REPORTS_ENABLE=true"
|
||||||
[ -z "${SELFPOST_DATA_DIR:-}" ] || DMARC_INGEST_ENV="${DMARC_INGEST_ENV} SELFPOST_DATA_DIR=${SELFPOST_DATA_DIR}"
|
[ -z "${SELFPOST_DATA_DIR:-}" ] || DMARC_INGEST_ENV="${DMARC_INGEST_ENV} SELFPOST_DATA_DIR=${SELFPOST_DATA_DIR}"
|
||||||
[ -z "${SELFPOST_DB_PATH:-}" ] || DMARC_INGEST_ENV="${DMARC_INGEST_ENV} SELFPOST_DB_PATH=${SELFPOST_DB_PATH}"
|
[ -z "${SELFPOST_DB_PATH:-}" ] || DMARC_INGEST_ENV="${DMARC_INGEST_ENV} SELFPOST_DB_PATH=${SELFPOST_DB_PATH}"
|
||||||
@@ -345,10 +368,18 @@ if [ "${INBOUND_RELAY_ENABLE}" = "true" ] || [ "${DMARC_REPORTS_ENABLE}" = "true
|
|||||||
# would file inbound mail into the outbound send log under the *sender's*
|
# would file inbound mail into the outbound send log under the *sender's*
|
||||||
# domain — polluting Deliveries and letting a forged From: consume that
|
# domain — polluting Deliveries and letting a forged From: consume that
|
||||||
# domain's level-2 rate-limit budget. So smtpd_milters is always overridden
|
# domain's level-2 rate-limit budget. So smtpd_milters is always overridden
|
||||||
# here: either the optional antispam milter, or nothing at all.
|
# here: the optional antispam milter, then - with the inbound relay on - the
|
||||||
|
# panel's inbound journal milter, or nothing at all.
|
||||||
INBOUND_MILTERS=""
|
INBOUND_MILTERS=""
|
||||||
if [ "${INBOUND_RELAY_ENABLE}" = "true" ] && [ -n "${INBOUND_ANTISPAM_MILTER}" ]; then
|
if [ "${INBOUND_RELAY_ENABLE}" = "true" ] && [ -n "${INBOUND_ANTISPAM_MILTER}" ]; then
|
||||||
|
# A glob's "*" matches anything, so the shape alone would pass
|
||||||
|
# "inet:a:1},{unix:/elsewhere": characters outside the set are refused
|
||||||
|
# first, then the shape is checked.
|
||||||
case "${INBOUND_ANTISPAM_MILTER}" in
|
case "${INBOUND_ANTISPAM_MILTER}" in
|
||||||
|
*[!A-Za-z0-9._:/-]*)
|
||||||
|
echo "FATAL: INBOUND_ANTISPAM_MILTER must be inet:host:port or unix:/path, got: ${INBOUND_ANTISPAM_MILTER}" >&2
|
||||||
|
exit 1
|
||||||
|
;;
|
||||||
inet:[A-Za-z0-9._-]*:[0-9]* | unix:/[A-Za-z0-9._/-]* ) ;;
|
inet:[A-Za-z0-9._-]*:[0-9]* | unix:/[A-Za-z0-9._/-]* ) ;;
|
||||||
*)
|
*)
|
||||||
echo "FATAL: INBOUND_ANTISPAM_MILTER must be inet:host:port or unix:/path, got: ${INBOUND_ANTISPAM_MILTER}" >&2
|
echo "FATAL: INBOUND_ANTISPAM_MILTER must be inet:host:port or unix:/path, got: ${INBOUND_ANTISPAM_MILTER}" >&2
|
||||||
@@ -364,6 +395,26 @@ if [ "${INBOUND_RELAY_ENABLE}" = "true" ] || [ "${DMARC_REPORTS_ENABLE}" = "true
|
|||||||
esac
|
esac
|
||||||
INBOUND_MILTERS="{${INBOUND_ANTISPAM_MILTER},default_action=${INBOUND_ANTISPAM_MILTER_ACTION}}"
|
INBOUND_MILTERS="{${INBOUND_ANTISPAM_MILTER},default_action=${INBOUND_ANTISPAM_MILTER_ACTION}}"
|
||||||
fi
|
fi
|
||||||
|
if [ "${INBOUND_RELAY_ENABLE}" = "true" ]; then
|
||||||
|
# Last in the chain: Postfix hands a message to its milters one after
|
||||||
|
# another at end-of-message, so this one reads the verdict headers the
|
||||||
|
# filter added and sees only what the filter let through. It watches and
|
||||||
|
# never refuses, and must never hold up someone else's mail: accept when
|
||||||
|
# it is down. A comma, not a space, between the two - see
|
||||||
|
# master_cf_set_smtp_inet_o.
|
||||||
|
case "${INBOUND_JOURNAL_SOCK}" in
|
||||||
|
*[!A-Za-z0-9._/-]*)
|
||||||
|
echo "FATAL: INBOUND_JOURNAL_MILTER_SOCKET must be an absolute path, got: ${INBOUND_JOURNAL_SOCK}" >&2
|
||||||
|
exit 1
|
||||||
|
;;
|
||||||
|
/[A-Za-z0-9._/-]*) ;;
|
||||||
|
*)
|
||||||
|
echo "FATAL: INBOUND_JOURNAL_MILTER_SOCKET must be an absolute path, got: ${INBOUND_JOURNAL_SOCK}" >&2
|
||||||
|
exit 1
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
INBOUND_MILTERS="${INBOUND_MILTERS:+${INBOUND_MILTERS},}{unix:${INBOUND_JOURNAL_SOCK},default_action=accept}"
|
||||||
|
fi
|
||||||
|
|
||||||
PORT25_RATE="${INBOUND_RATE_LIMIT_MESSAGES_PER_IP}"
|
PORT25_RATE="${INBOUND_RATE_LIMIT_MESSAGES_PER_IP}"
|
||||||
PORT25_SIZE="${INBOUND_MESSAGE_SIZE_LIMIT}"
|
PORT25_SIZE="${INBOUND_MESSAGE_SIZE_LIMIT}"
|
||||||
@@ -380,8 +431,12 @@ if [ "${INBOUND_RELAY_ENABLE}" = "true" ] || [ "${DMARC_REPORTS_ENABLE}" = "true
|
|||||||
fi
|
fi
|
||||||
fi
|
fi
|
||||||
|
|
||||||
|
# syslog_name: this listener's lines read postfix/inbound/smtpd, the
|
||||||
|
# submission ports' postfix/smtpd. The log tailer files a refusal on port 25
|
||||||
|
# into the inbound journal and has to tell the two apart.
|
||||||
postconf -M "smtp/inet=smtp inet n - n - - smtpd"
|
postconf -M "smtp/inet=smtp inet n - n - - smtpd"
|
||||||
postconf -P \
|
postconf -P \
|
||||||
|
"smtp/inet/syslog_name=postfix/inbound" \
|
||||||
"smtp/inet/smtpd_sasl_auth_enable=no" \
|
"smtp/inet/smtpd_sasl_auth_enable=no" \
|
||||||
"smtp/inet/smtpd_tls_auth_only=no" \
|
"smtp/inet/smtpd_tls_auth_only=no" \
|
||||||
"smtp/inet/smtpd_sender_login_maps=" \
|
"smtp/inet/smtpd_sender_login_maps=" \
|
||||||
|
|||||||
@@ -2,8 +2,9 @@
|
|||||||
# Postfix start wrapper: solves the cold-start race where Postfix would try to
|
# Postfix start wrapper: solves the cold-start race where Postfix would try to
|
||||||
# reach the milter sockets before they are listening.
|
# reach the milter sockets before they are listening.
|
||||||
#
|
#
|
||||||
# It blocks until BOTH milter sockets — OpenDKIM and the panel's journal-milter
|
# It blocks until the milter sockets — OpenDKIM, the panel's journal-milter
|
||||||
# — are present, then execs `postfix start-fg`. If they are not ready within the
|
# and, with the inbound relay on, its inbound journal milter — are present,
|
||||||
|
# then execs `postfix start-fg`. If they are not ready within the
|
||||||
# timeout it exits non-zero WITHOUT starting Postfix, so supervisord/Docker see
|
# timeout it exits non-zero WITHOUT starting Postfix, so supervisord/Docker see
|
||||||
# an explicit startup failure instead of a relay running blind.
|
# an explicit startup failure instead of a relay running blind.
|
||||||
#
|
#
|
||||||
@@ -16,8 +17,13 @@ JOURNAL_SOCK="${JOURNAL_MILTER_SOCKET:-/run/selfpost/journal.sock}"
|
|||||||
TIMEOUT="${MILTER_WAIT_TIMEOUT:-30}"
|
TIMEOUT="${MILTER_WAIT_TIMEOUT:-30}"
|
||||||
INTERVAL=1
|
INTERVAL=1
|
||||||
|
|
||||||
|
SOCKS="$OPENDKIM_SOCK $JOURNAL_SOCK"
|
||||||
|
if [ "${INBOUND_RELAY_ENABLE:-false}" = "true" ]; then
|
||||||
|
SOCKS="$SOCKS ${INBOUND_JOURNAL_MILTER_SOCKET:-/run/selfpost/inbound-journal.sock}"
|
||||||
|
fi
|
||||||
|
|
||||||
elapsed=0
|
elapsed=0
|
||||||
for sock in "$OPENDKIM_SOCK" "$JOURNAL_SOCK"; do
|
for sock in $SOCKS; do
|
||||||
while [ ! -S "$sock" ]; do
|
while [ ! -S "$sock" ]; do
|
||||||
if [ "$elapsed" -ge "$TIMEOUT" ]; then
|
if [ "$elapsed" -ge "$TIMEOUT" ]; then
|
||||||
echo "postfix-wrapper: timed out after ${TIMEOUT}s waiting for milter socket $sock" >&2
|
echo "postfix-wrapper: timed out after ${TIMEOUT}s waiting for milter socket $sock" >&2
|
||||||
@@ -29,5 +35,5 @@ for sock in "$OPENDKIM_SOCK" "$JOURNAL_SOCK"; do
|
|||||||
echo "postfix-wrapper: milter socket ready: $sock"
|
echo "postfix-wrapper: milter socket ready: $sock"
|
||||||
done
|
done
|
||||||
|
|
||||||
echo "postfix-wrapper: both milter sockets ready, starting postfix"
|
echo "postfix-wrapper: milter sockets ready, starting postfix"
|
||||||
exec postfix start-fg
|
exec postfix start-fg
|
||||||
|
|||||||
@@ -1,6 +1,7 @@
|
|||||||
package main
|
package main
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
"io"
|
"io"
|
||||||
"log"
|
"log"
|
||||||
@@ -28,14 +29,47 @@ func runDMARCIngestMode() error {
|
|||||||
|
|
||||||
recipient := ingestRecipient(os.Args[1:])
|
recipient := ingestRecipient(os.Args[1:])
|
||||||
if err := dmarc.IngestMessage(st, io.LimitReader(os.Stdin, maxIngestStdin), recipient, time.Now().UTC()); err != nil {
|
if err := dmarc.IngestMessage(st, io.LimitReader(os.Stdin, maxIngestStdin), recipient, time.Now().UTC()); err != nil {
|
||||||
|
if !errors.Is(err, dmarc.ErrRejected) {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
// Not a report this relay keeps. It is dropped: deferring would have
|
||||||
|
// Postfix try it again for days, and a bounce would go to an envelope
|
||||||
|
// sender anyone can write - the report addresses are public.
|
||||||
if incrErr := st.IncrDMARCParseFailures(); incrErr != nil {
|
if incrErr := st.IncrDMARCParseFailures(); incrErr != nil {
|
||||||
log.Printf("dmarc ingest: record failure: %v", incrErr)
|
log.Printf("dmarc ingest: record failure: %v", incrErr)
|
||||||
}
|
}
|
||||||
return err
|
logRejected(err)
|
||||||
}
|
}
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// logRejected leaves the reason in the mail log. Postfix puts the start of
|
||||||
|
// what a delivery command wrote - about a hundred bytes - into its own line
|
||||||
|
// for the message ("status=sent (delivered via dmarc-ingest service (...))"),
|
||||||
|
// so the reason goes out first and bare: no prefix, no date, no recipient,
|
||||||
|
// all of which that line already has. postlog is no use here: pipe(8) starts
|
||||||
|
// the command without the group that may write to Postfix's log socket.
|
||||||
|
func logRejected(reason error) {
|
||||||
|
fmt.Fprintln(os.Stderr, "dropped: "+logSafe(strings.TrimPrefix(reason.Error(), dmarc.ErrRejected.Error()+": ")))
|
||||||
|
}
|
||||||
|
|
||||||
|
// logSafe makes a reason fit for a log line. Parts of it are the sender's own
|
||||||
|
// bytes - a header line, a file name inside an archive, a piece of XML - so it
|
||||||
|
// is cut short and anything that is not printable ASCII becomes "?", whatever
|
||||||
|
// Postfix would have done with it.
|
||||||
|
func logSafe(s string) string {
|
||||||
|
const max = 200
|
||||||
|
if len(s) > max {
|
||||||
|
s = s[:max]
|
||||||
|
}
|
||||||
|
return strings.Map(func(r rune) rune {
|
||||||
|
if r < 0x20 || r > 0x7e {
|
||||||
|
return '?'
|
||||||
|
}
|
||||||
|
return r
|
||||||
|
}, s)
|
||||||
|
}
|
||||||
|
|
||||||
// ingestRecipient is the report address the message was delivered to. Postfix's
|
// ingestRecipient is the report address the message was delivered to. Postfix's
|
||||||
// pipe(8) hands it over as an argument (${recipient} in the service's argv,
|
// pipe(8) hands it over as an argument (${recipient} in the service's argv,
|
||||||
// build/postfix-config.sh) - it sets no RECIPIENT variable, that is local(8).
|
// build/postfix-config.sh) - it sets no RECIPIENT variable, that is local(8).
|
||||||
|
|||||||
@@ -18,3 +18,17 @@ func TestIngestRecipient(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestLogSafe(t *testing.T) {
|
||||||
|
got := logSafe("zip entry \"a\nstatus=bounced\tx\" \x1b[31m é")
|
||||||
|
if want := "zip entry \"a?status=bounced?x\" ?[31m ?"; got != want {
|
||||||
|
t.Errorf("logSafe = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
long := make([]byte, 1000)
|
||||||
|
for i := range long {
|
||||||
|
long[i] = 'a'
|
||||||
|
}
|
||||||
|
if got := logSafe(string(long)); len(got) != 200 {
|
||||||
|
t.Errorf("logSafe kept %d bytes of 1000, want 200", len(got))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
+10
-1
@@ -24,6 +24,7 @@ type mailStack struct {
|
|||||||
Domains *domain.Service
|
Domains *domain.Service
|
||||||
Apps *app.Service
|
Apps *app.Service
|
||||||
Inbound *inbound.Service
|
Inbound *inbound.Service
|
||||||
|
Lists *inbound.Lists
|
||||||
DMARC *dmarc.Service
|
DMARC *dmarc.Service
|
||||||
pf *postfix.Postfix
|
pf *postfix.Postfix
|
||||||
odk *domain.OpenDKIM
|
odk *domain.OpenDKIM
|
||||||
@@ -40,6 +41,7 @@ func newMailStack(cfg config, st *store.Store) *mailStack {
|
|||||||
dmarcSvc := dmarc.NewService(st, pf, cfg.hostname, cfg.dmarcEnabled)
|
dmarcSvc := dmarc.NewService(st, pf, cfg.hostname, cfg.dmarcEnabled)
|
||||||
return &mailStack{
|
return &mailStack{
|
||||||
Domains: domains, Apps: apps, Inbound: inb, DMARC: dmarcSvc,
|
Domains: domains, Apps: apps, Inbound: inb, DMARC: dmarcSvc,
|
||||||
|
Lists: inbound.NewLists(st, cfg.antispamDir),
|
||||||
pf: pf, odk: odk, inboundEnabled: cfg.inboundEnabled, dmarcEnabled: cfg.dmarcEnabled,
|
pf: pf, odk: odk, inboundEnabled: cfg.inboundEnabled, dmarcEnabled: cfg.dmarcEnabled,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -57,6 +59,11 @@ func (m *mailStack) Resync() error {
|
|||||||
if err := m.Inbound.Resync(); err != nil {
|
if err := m.Inbound.Resync(); err != nil {
|
||||||
return fmt.Errorf("inbound maps resync: %w", err)
|
return fmt.Errorf("inbound maps resync: %w", err)
|
||||||
}
|
}
|
||||||
|
// The filter's configuration names all six list files; they exist from
|
||||||
|
// the first start, empty until an entry is added.
|
||||||
|
if err := m.Lists.Sync(); err != nil {
|
||||||
|
return fmt.Errorf("filter lists resync: %w", err)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
if m.dmarcEnabled {
|
if m.dmarcEnabled {
|
||||||
if err := m.DMARC.Resync(); err != nil {
|
if err := m.DMARC.Resync(); err != nil {
|
||||||
@@ -91,7 +98,7 @@ func newPanel(cfg config, st *store.Store) (*web.Server, error) {
|
|||||||
// effective config, including a manual override; the panel keeps this
|
// effective config, including a manual override; the panel keeps this
|
||||||
// snapshot for the process lifetime (architecture.md).
|
// snapshot for the process lifetime (architecture.md).
|
||||||
retryPolicy := postfix.LoadRetryPolicy()
|
retryPolicy := postfix.LoadRetryPolicy()
|
||||||
return web.New(st, ms.Domains, ms.Apps, ms.Inbound, ms.DMARC, web.Config{
|
return web.New(st, ms.Domains, ms.Apps, ms.Inbound, ms.Lists, ms.DMARC, web.Config{
|
||||||
Hostname: cfg.hostname,
|
Hostname: cfg.hostname,
|
||||||
CookieSecure: cfg.cookieSecure,
|
CookieSecure: cfg.cookieSecure,
|
||||||
SubmissionEnabled: cfg.submissionEnabled,
|
SubmissionEnabled: cfg.submissionEnabled,
|
||||||
@@ -114,6 +121,8 @@ func newPanel(cfg config, st *store.Store) (*web.Server, error) {
|
|||||||
InboundAntispamAction: cfg.inboundAntispamAction,
|
InboundAntispamAction: cfg.inboundAntispamAction,
|
||||||
DMARCEnabled: cfg.dmarcEnabled,
|
DMARCEnabled: cfg.dmarcEnabled,
|
||||||
SendLogRetentionEnvDefault: cfg.retentionDays,
|
SendLogRetentionEnvDefault: cfg.retentionDays,
|
||||||
|
|
||||||
|
InboundSpamLogRetentionEnvDefault: cfg.inboundSpamLogRetentionDays,
|
||||||
}, cfg.setupTokenPath)
|
}, cfg.setupTokenPath)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
+39
-9
@@ -8,6 +8,7 @@ import (
|
|||||||
"os"
|
"os"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
|
|
||||||
|
"github.com/mixeme/selfpost/internal/inbound"
|
||||||
"github.com/mixeme/selfpost/internal/milter"
|
"github.com/mixeme/selfpost/internal/milter"
|
||||||
"github.com/mixeme/selfpost/internal/store"
|
"github.com/mixeme/selfpost/internal/store"
|
||||||
)
|
)
|
||||||
@@ -15,21 +16,52 @@ import (
|
|||||||
// serveJournal opens the journal-milter Unix socket and runs the real milter
|
// serveJournal opens the journal-milter Unix socket and runs the real milter
|
||||||
// (architecture.md § Mail path), recording accepted messages into the send
|
// (architecture.md § Mail path), recording accepted messages into the send
|
||||||
// log. Socket lifecycle (creation, stale cleanup, group permissions) lives
|
// log. Socket lifecycle (creation, stale cleanup, group permissions) lives
|
||||||
// here; the protocol handler lives in internal/milter.
|
// in listenMilter; the protocol handler lives in internal/milter.
|
||||||
func serveJournal(ctx context.Context, cfg config, st *store.Store) error {
|
func serveJournal(ctx context.Context, cfg config, st *store.Store) error {
|
||||||
socketPath := cfg.journalSocket
|
ln, err := listenMilter(cfg.journalSocket)
|
||||||
if err := os.MkdirAll(filepath.Dir(socketPath), 0o755); err != nil {
|
if err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
log.Printf("journal-milter listening on %s", cfg.journalSocket)
|
||||||
|
return milter.Serve(ctx, ln, st)
|
||||||
|
}
|
||||||
|
|
||||||
|
// serveInboundJournal runs the inbound journal milter, the last milter on
|
||||||
|
// port 25 (internal/milter/inbound.go). It exists only while the inbound relay
|
||||||
|
// is on: build/postfix-config.sh attaches the socket under the same condition.
|
||||||
|
func serveInboundJournal(ctx context.Context, cfg config, st *store.Store) error {
|
||||||
|
ln, err := listenMilter(cfg.inboundJournalSocket)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
// With a filter attached, the key it signs with. Without the key no
|
||||||
|
// verdict can be told from a sender's own, so none is read: the journal
|
||||||
|
// still records the mail, and the log says why it shows no verdicts.
|
||||||
|
key := ""
|
||||||
|
if cfg.inboundAntispamMilter != "" {
|
||||||
|
if key, err = inbound.FilterKey(cfg.antispamDir); err != nil {
|
||||||
|
log.Printf("inbound-journal-milter: no key for the filter's signature, verdicts will not be recorded: %v", err)
|
||||||
|
key = ""
|
||||||
|
}
|
||||||
|
}
|
||||||
|
log.Printf("inbound-journal-milter listening on %s", cfg.inboundJournalSocket)
|
||||||
|
return milter.ServeInbound(ctx, ln, st, key)
|
||||||
|
}
|
||||||
|
|
||||||
|
// listenMilter opens a milter's Unix socket where Postfix can reach it.
|
||||||
|
func listenMilter(socketPath string) (net.Listener, error) {
|
||||||
|
if err := os.MkdirAll(filepath.Dir(socketPath), 0o755); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
// Clear a stale socket left behind by an unclean shutdown, otherwise the
|
// Clear a stale socket left behind by an unclean shutdown, otherwise the
|
||||||
// listen below fails with "address already in use".
|
// listen below fails with "address already in use".
|
||||||
if err := os.Remove(socketPath); err != nil && !errors.Is(err, os.ErrNotExist) {
|
if err := os.Remove(socketPath); err != nil && !errors.Is(err, os.ErrNotExist) {
|
||||||
return err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
|
||||||
ln, err := net.Listen("unix", socketPath)
|
ln, err := net.Listen("unix", socketPath)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return nil, err
|
||||||
}
|
}
|
||||||
// Postfix (user `postfix`) connects to this socket as a milter and needs
|
// Postfix (user `postfix`) connects to this socket as a milter and needs
|
||||||
// group access. The socket inherits group `selfpost` from the setgid parent
|
// group access. The socket inherits group `selfpost` from the setgid parent
|
||||||
@@ -37,9 +69,7 @@ func serveJournal(ctx context.Context, cfg config, st *store.Store) error {
|
|||||||
// it (connecting to a Unix socket needs write permission on the node).
|
// it (connecting to a Unix socket needs write permission on the node).
|
||||||
if err := os.Chmod(socketPath, 0o660); err != nil {
|
if err := os.Chmod(socketPath, 0o660); err != nil {
|
||||||
ln.Close()
|
ln.Close()
|
||||||
return err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
return ln, nil
|
||||||
log.Printf("journal-milter listening on %s", socketPath)
|
|
||||||
return milter.Serve(ctx, ln, st)
|
|
||||||
}
|
}
|
||||||
|
|||||||
+56
-2
@@ -23,6 +23,7 @@ import (
|
|||||||
"github.com/mixeme/selfpost/internal/backup"
|
"github.com/mixeme/selfpost/internal/backup"
|
||||||
"github.com/mixeme/selfpost/internal/buildinfo"
|
"github.com/mixeme/selfpost/internal/buildinfo"
|
||||||
"github.com/mixeme/selfpost/internal/dnscheck"
|
"github.com/mixeme/selfpost/internal/dnscheck"
|
||||||
|
"github.com/mixeme/selfpost/internal/inbound"
|
||||||
"github.com/mixeme/selfpost/internal/logtail"
|
"github.com/mixeme/selfpost/internal/logtail"
|
||||||
"github.com/mixeme/selfpost/internal/store"
|
"github.com/mixeme/selfpost/internal/store"
|
||||||
)
|
)
|
||||||
@@ -89,6 +90,9 @@ type config struct {
|
|||||||
saslRealm string
|
saslRealm string
|
||||||
postfixDir string
|
postfixDir string
|
||||||
deployRoot string
|
deployRoot string
|
||||||
|
// Where the filter lists are written as map files for the external spam
|
||||||
|
// filter; the antispam compose fragment mounts it into the sidecar.
|
||||||
|
antispamDir string
|
||||||
|
|
||||||
inboundEnabled bool
|
inboundEnabled bool
|
||||||
// The optional milter on the inbound listener (INBOUND_ANTISPAM_MILTER) and
|
// The optional milter on the inbound listener (INBOUND_ANTISPAM_MILTER) and
|
||||||
@@ -96,6 +100,11 @@ type config struct {
|
|||||||
// what attaches it; the panel only says so. See antispamAction.
|
// what attaches it; the panel only says so. See antispamAction.
|
||||||
inboundAntispamMilter string
|
inboundAntispamMilter string
|
||||||
inboundAntispamAction string
|
inboundAntispamAction string
|
||||||
|
// The inbound journal milter's socket; the default mirrors
|
||||||
|
// build/postfix-config.sh. Served only while the inbound relay is on.
|
||||||
|
inboundJournalSocket string
|
||||||
|
// Retention of the inbound journal until Server > Settings says otherwise.
|
||||||
|
inboundSpamLogRetentionDays int
|
||||||
dmarcEnabled bool
|
dmarcEnabled bool
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -157,11 +166,16 @@ func loadConfig() config {
|
|||||||
saslDBPath: envDefault("SASL_DB_PATH", filepath.Join(dataDir, "sasl", "sasldb2")),
|
saslDBPath: envDefault("SASL_DB_PATH", filepath.Join(dataDir, "sasl", "sasldb2")),
|
||||||
saslRealm: saslRealm(),
|
saslRealm: saslRealm(),
|
||||||
postfixDir: envDefault("POSTFIX_DIR", filepath.Join(dataDir, "postfix")),
|
postfixDir: envDefault("POSTFIX_DIR", filepath.Join(dataDir, "postfix")),
|
||||||
|
|
||||||
|
antispamDir: envDefault("ANTISPAM_DIR", filepath.Join(dataDir, "antispam")),
|
||||||
deployRoot: envDefault("SELFPOST_DEPLOY_ROOT", "/selfpost-deploy"),
|
deployRoot: envDefault("SELFPOST_DEPLOY_ROOT", "/selfpost-deploy"),
|
||||||
// Optional inbound relay (backup-MX / forwarder). Off unless exactly "true".
|
// Optional inbound relay (backup-MX / forwarder). Off unless exactly "true".
|
||||||
inboundEnabled: os.Getenv("INBOUND_RELAY_ENABLE") == "true",
|
inboundEnabled: os.Getenv("INBOUND_RELAY_ENABLE") == "true",
|
||||||
inboundAntispamMilter: os.Getenv("INBOUND_ANTISPAM_MILTER"),
|
inboundAntispamMilter: os.Getenv("INBOUND_ANTISPAM_MILTER"),
|
||||||
inboundAntispamAction: antispamAction(os.Getenv("INBOUND_ANTISPAM_MILTER_ACTION")),
|
inboundAntispamAction: antispamAction(os.Getenv("INBOUND_ANTISPAM_MILTER_ACTION")),
|
||||||
|
inboundJournalSocket: envDefault("INBOUND_JOURNAL_MILTER_SOCKET", "/run/selfpost/inbound-journal.sock"),
|
||||||
|
|
||||||
|
inboundSpamLogRetentionDays: envInt("INBOUND_SPAM_LOG_RETENTION_DAYS", store.InboundSpamLogRetentionDaysDefault),
|
||||||
// Optional DMARC aggregate ingest on port 25. Off unless exactly "true".
|
// Optional DMARC aggregate ingest on port 25. Off unless exactly "true".
|
||||||
dmarcEnabled: os.Getenv("DMARC_REPORTS_ENABLE") == "true",
|
dmarcEnabled: os.Getenv("DMARC_REPORTS_ENABLE") == "true",
|
||||||
}
|
}
|
||||||
@@ -289,12 +303,45 @@ func run() error {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// The filter lists as files, on every start: the spam filter's
|
||||||
|
// configuration names all six, and a data directory from before the lists
|
||||||
|
// existed has none. Not fatal - the lists are the filter's business, and
|
||||||
|
// the panel has mail to relay either way.
|
||||||
|
if cfg.inboundEnabled && !restored {
|
||||||
|
if err := inbound.NewLists(st, cfg.antispamDir).Sync(); err != nil {
|
||||||
|
log.Printf("filter lists bootstrap: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// The key the filter signs with has to be in the shared directory before
|
||||||
|
// the filter reads it, whether or not a filter is attached yet.
|
||||||
|
if cfg.inboundEnabled {
|
||||||
|
if _, err := inbound.FilterKey(cfg.antispamDir); err != nil {
|
||||||
|
log.Printf("filter key bootstrap: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
if err := st.EnsureSendLogRetentionDays(cfg.retentionDays); err != nil {
|
if err := st.EnsureSendLogRetentionDays(cfg.retentionDays); err != nil {
|
||||||
return fmt.Errorf("bootstrap send-log retention: %w", err)
|
return fmt.Errorf("bootstrap send-log retention: %w", err)
|
||||||
}
|
}
|
||||||
|
// With the inbound relay on, the tailer also writes the inbound journal's
|
||||||
|
// rows for refused mail and prunes that journal.
|
||||||
|
var inboundTail *logtail.Inbound
|
||||||
|
if cfg.inboundEnabled {
|
||||||
|
if err := st.EnsureInboundSpamLogRetentionDays(cfg.inboundSpamLogRetentionDays); err != nil {
|
||||||
|
return fmt.Errorf("bootstrap inbound journal retention: %w", err)
|
||||||
|
}
|
||||||
|
inboundTail = &logtail.Inbound{Store: st, Retention: func() int {
|
||||||
|
days, err := st.GetInboundSpamLogRetentionDays(cfg.inboundSpamLogRetentionDays)
|
||||||
|
if err != nil {
|
||||||
|
log.Printf("inbound journal retention: %v", err)
|
||||||
|
return cfg.inboundSpamLogRetentionDays
|
||||||
|
}
|
||||||
|
return days
|
||||||
|
}}
|
||||||
|
}
|
||||||
|
|
||||||
var wg sync.WaitGroup
|
var wg sync.WaitGroup
|
||||||
errc := make(chan error, 4)
|
errc := make(chan error, 5)
|
||||||
|
|
||||||
roles := []struct {
|
roles := []struct {
|
||||||
name string
|
name string
|
||||||
@@ -310,7 +357,7 @@ func run() error {
|
|||||||
return cfg.retentionDays
|
return cfg.retentionDays
|
||||||
}
|
}
|
||||||
return days
|
return days
|
||||||
})
|
}, inboundTail)
|
||||||
}},
|
}},
|
||||||
{"rate-limit-recalc", func(ctx context.Context) error {
|
{"rate-limit-recalc", func(ctx context.Context) error {
|
||||||
runAutoRateLimitRecalc(ctx, cfg, st)
|
runAutoRateLimitRecalc(ctx, cfg, st)
|
||||||
@@ -318,6 +365,13 @@ func run() error {
|
|||||||
}},
|
}},
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if cfg.inboundEnabled {
|
||||||
|
roles = append(roles, struct {
|
||||||
|
name string
|
||||||
|
fn func(context.Context) error
|
||||||
|
}{"inbound-journal-milter", func(ctx context.Context) error { return serveInboundJournal(ctx, cfg, st) }})
|
||||||
|
}
|
||||||
|
|
||||||
for _, r := range roles {
|
for _, r := range roles {
|
||||||
wg.Add(1)
|
wg.Add(1)
|
||||||
go func(name string, fn func(context.Context) error) {
|
go func(name string, fn func(context.Context) error) {
|
||||||
|
|||||||
@@ -30,6 +30,10 @@ DMARC_REPORTS_ENABLE=false
|
|||||||
# INBOUND_ANTISPAM_MILTER=inet:antispam:11332
|
# INBOUND_ANTISPAM_MILTER=inet:antispam:11332
|
||||||
# INBOUND_ANTISPAM_MILTER_ACTION=accept
|
# INBOUND_ANTISPAM_MILTER_ACTION=accept
|
||||||
|
|
||||||
|
# Days the inbound journal keeps a row (7-365) until the panel says otherwise;
|
||||||
|
# it never holds more than 10 000 rows.
|
||||||
|
# INBOUND_SPAM_LOG_RETENTION_DAYS=90
|
||||||
|
|
||||||
# Coarse per-client-IP limit and message size on inbound smtpd (port 25).
|
# Coarse per-client-IP limit and message size on inbound smtpd (port 25).
|
||||||
# INBOUND_RATE_LIMIT_MESSAGES_PER_IP=20
|
# INBOUND_RATE_LIMIT_MESSAGES_PER_IP=20
|
||||||
# INBOUND_MESSAGE_SIZE_LIMIT=26214400
|
# INBOUND_MESSAGE_SIZE_LIMIT=26214400
|
||||||
|
|||||||
@@ -27,3 +27,10 @@ services:
|
|||||||
image: rspamd/rspamd:3.11
|
image: rspamd/rspamd:3.11
|
||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
# No host ports: only SelfPost talks to the milter on the compose network.
|
# No host ports: only SelfPost talks to the milter on the compose network.
|
||||||
|
volumes:
|
||||||
|
# The verdict headers SelfPost's inbound journal reads.
|
||||||
|
- ./antispam/rspamd/milter_headers.conf:/etc/rspamd/local.d/milter_headers.conf:ro
|
||||||
|
# The allow and deny lists kept in the panel: the rules, and the files
|
||||||
|
# SelfPost writes them to.
|
||||||
|
- ./antispam/rspamd/multimap.conf:/etc/rspamd/local.d/multimap.conf:ro
|
||||||
|
- ./data/antispam:/etc/rspamd/selfpost:ro
|
||||||
|
|||||||
@@ -0,0 +1,45 @@
|
|||||||
|
# Mounted into the sidecar as /etc/rspamd/local.d/milter_headers.conf.
|
||||||
|
#
|
||||||
|
# rspamd writes its verdict into two headers of every message it lets through:
|
||||||
|
# the action it chose and the score with the symbols behind it. SelfPost's
|
||||||
|
# inbound journal reads them (Inbound > Log shows them next to the message),
|
||||||
|
# and the upstream mailbox server receives them with the mail.
|
||||||
|
#
|
||||||
|
# rspamd removes headers of these names that a sender wrote himself before it
|
||||||
|
# adds its own. Without this file the journal still lists accepted mail, with
|
||||||
|
# no verdict next to it.
|
||||||
|
#
|
||||||
|
# The third header is the proof that the other two are rspamd's. The filter is
|
||||||
|
# attached so that mail still flows when it is down, and then nothing removes
|
||||||
|
# what a sender wrote: he could send his own verdict. So rspamd signs each
|
||||||
|
# message - an HMAC-SHA256 of the message's queue id under a key SelfPost
|
||||||
|
# generates and leaves in the directory it shares with this container - and
|
||||||
|
# SelfPost reads a verdict only next to a signature it can recompute. The
|
||||||
|
# signature travels on with the mail and is of no use to whoever sees it: it
|
||||||
|
# fits that one queue id.
|
||||||
|
use = ["x-spamd-result", "x-rspamd-action", "selfpost-signature"];
|
||||||
|
|
||||||
|
custom {
|
||||||
|
selfpost-signature = <<EOD
|
||||||
|
return function(task, common_meta)
|
||||||
|
local remove = { ['X-SelfPost-Filter'] = 0 }
|
||||||
|
-- The key is read for every message and kept nowhere: SelfPost may make a
|
||||||
|
-- new one (the file was lost, a backup was restored), and a key remembered
|
||||||
|
-- here would go on signing what SelfPost no longer accepts until someone
|
||||||
|
-- restarted this container. The file is one short line.
|
||||||
|
local key
|
||||||
|
local f = io.open('/etc/rspamd/selfpost/filter.key', 'r')
|
||||||
|
if f then
|
||||||
|
key = f:read('*l')
|
||||||
|
f:close()
|
||||||
|
end
|
||||||
|
local qid = task:get_queue_id()
|
||||||
|
if not key or key == '' or not qid or qid == '' then
|
||||||
|
return nil, {}, remove, {}
|
||||||
|
end
|
||||||
|
local hash = require "rspamd_cryptobox_hash"
|
||||||
|
local sig = hash.create_specific_keyed(key, 'sha256', qid):hex()
|
||||||
|
return nil, { ['X-SelfPost-Filter'] = sig }, remove, {}
|
||||||
|
end
|
||||||
|
EOD;
|
||||||
|
}
|
||||||
@@ -0,0 +1,63 @@
|
|||||||
|
# Mounted into the sidecar as /etc/rspamd/local.d/multimap.conf.
|
||||||
|
#
|
||||||
|
# The allow and deny lists kept in the panel (Inbound > Filter lists). SelfPost
|
||||||
|
# writes them as six files under data/antispam/, which the fragment mounts at
|
||||||
|
# /etc/rspamd/selfpost; rspamd rereads a file when it changes, no restart.
|
||||||
|
#
|
||||||
|
# Deny wins over allow, whichever entries match. A denied sender is refused
|
||||||
|
# before anything else is looked at (a prefilter). An allowed one is not
|
||||||
|
# accepted outright - rspamd would then let whichever of the two rules ran
|
||||||
|
# first decide - but given a weight no spam score outweighs, which a refusal
|
||||||
|
# by the deny list still overrides.
|
||||||
|
#
|
||||||
|
# A domain entry is that domain, not its subdomains.
|
||||||
|
#
|
||||||
|
# SelfPost counts how often an entry decided a message by what these rules
|
||||||
|
# leave behind: the deny message below and the names of the allow symbols.
|
||||||
|
# Change either and Inbound > Filter lists shows "never" for every entry.
|
||||||
|
|
||||||
|
SELFPOST_DENY_IP {
|
||||||
|
type = "ip";
|
||||||
|
map = "/etc/rspamd/selfpost/deny_ip.map";
|
||||||
|
prefilter = true;
|
||||||
|
action = "reject";
|
||||||
|
message = "Refused by this server's deny list";
|
||||||
|
description = "Client address on the deny list";
|
||||||
|
}
|
||||||
|
SELFPOST_DENY_FROM {
|
||||||
|
type = "from";
|
||||||
|
map = "/etc/rspamd/selfpost/deny_from.map";
|
||||||
|
prefilter = true;
|
||||||
|
action = "reject";
|
||||||
|
message = "Refused by this server's deny list";
|
||||||
|
description = "Sender address on the deny list";
|
||||||
|
}
|
||||||
|
SELFPOST_DENY_DOMAIN {
|
||||||
|
type = "from";
|
||||||
|
filter = "email:domain";
|
||||||
|
map = "/etc/rspamd/selfpost/deny_domain.map";
|
||||||
|
prefilter = true;
|
||||||
|
action = "reject";
|
||||||
|
message = "Refused by this server's deny list";
|
||||||
|
description = "Sender domain on the deny list";
|
||||||
|
}
|
||||||
|
|
||||||
|
SELFPOST_ALLOW_IP {
|
||||||
|
type = "ip";
|
||||||
|
map = "/etc/rspamd/selfpost/allow_ip.map";
|
||||||
|
score = -100.0;
|
||||||
|
description = "Client address on the allow list";
|
||||||
|
}
|
||||||
|
SELFPOST_ALLOW_FROM {
|
||||||
|
type = "from";
|
||||||
|
map = "/etc/rspamd/selfpost/allow_from.map";
|
||||||
|
score = -100.0;
|
||||||
|
description = "Sender address on the allow list";
|
||||||
|
}
|
||||||
|
SELFPOST_ALLOW_DOMAIN {
|
||||||
|
type = "from";
|
||||||
|
filter = "email:domain";
|
||||||
|
map = "/etc/rspamd/selfpost/allow_domain.map";
|
||||||
|
score = -100.0;
|
||||||
|
description = "Sender domain on the allow list";
|
||||||
|
}
|
||||||
@@ -21,7 +21,7 @@
|
|||||||
|
|
||||||
services:
|
services:
|
||||||
selfpost:
|
selfpost:
|
||||||
image: ghcr.io/mixeme/selfpost:2.0.0
|
image: ghcr.io/mixeme/selfpost:2.1.0
|
||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
environment:
|
environment:
|
||||||
SELFPOST_HOSTNAME: "${SELFPOST_HOSTNAME:?set the mail/panel hostname, e.g. mail.example.com}"
|
SELFPOST_HOSTNAME: "${SELFPOST_HOSTNAME:?set the mail/panel hostname, e.g. mail.example.com}"
|
||||||
@@ -47,6 +47,7 @@ services:
|
|||||||
# How long the send log keeps rows before the background sweep deletes
|
# How long the send log keeps rows before the background sweep deletes
|
||||||
# them — the main driver of /data growth over time.
|
# them — the main driver of /data growth over time.
|
||||||
SEND_LOG_RETENTION_DAYS: "${SEND_LOG_RETENTION_DAYS:-90}"
|
SEND_LOG_RETENTION_DAYS: "${SEND_LOG_RETENTION_DAYS:-90}"
|
||||||
|
INBOUND_SPAM_LOG_RETENTION_DAYS: "${INBOUND_SPAM_LOG_RETENTION_DAYS:-90}"
|
||||||
# Sliding idle timeout for the panel login session, in days.
|
# Sliding idle timeout for the panel login session, in days.
|
||||||
PANEL_SESSION_IDLE_DAYS: "${PANEL_SESSION_IDLE_DAYS:-7}"
|
PANEL_SESSION_IDLE_DAYS: "${PANEL_SESSION_IDLE_DAYS:-7}"
|
||||||
# Resolvers the panel's PTR/SPF/DKIM/DMARC checks query directly, so they
|
# Resolvers the panel's PTR/SPF/DKIM/DMARC checks query directly, so they
|
||||||
|
|||||||
+62
-10
@@ -27,8 +27,8 @@ Managed programs ([build/supervisord.conf](../build/supervisord.conf)):
|
|||||||
| Program | User | Priority | Role |
|
| Program | User | Priority | Role |
|
||||||
|---|---|---|---|
|
|---|---|---|---|
|
||||||
| `opendkim` | root → `opendkim` (`UserID` in opendkim.conf) | 100 | DKIM signing milter |
|
| `opendkim` | root → `opendkim` (`UserID` in opendkim.conf) | 100 | DKIM signing milter |
|
||||||
| `panel` | panel | 200 | HTTP UI + journal-milter + log-tailer + rate-limit recalc |
|
| `panel` | panel | 200 | HTTP UI + journal-milter + log-tailer + rate-limit recalc; with the inbound relay on also the inbound journal milter |
|
||||||
| `postfix` | root (wrapper) | 300 | MTA — started only after both milter sockets exist |
|
| `postfix` | root (wrapper) | 300 | MTA — started only after the milter sockets exist (OpenDKIM's, the journal milter's and, with the inbound relay on, the inbound journal milter's) |
|
||||||
| `postfix-reload` | root | — | On-demand `postfix reload` (autostart off) |
|
| `postfix-reload` | root | — | On-demand `postfix reload` (autostart off) |
|
||||||
| `cert-reload` | root | 400 | Daily `postfix reload` for renewed TLS certs |
|
| `cert-reload` | root | 400 | Daily `postfix reload` for renewed TLS certs |
|
||||||
| `logrotate` | root | 400 | Periodic `mail.log` rotation |
|
| `logrotate` | root | 400 | Periodic `mail.log` rotation |
|
||||||
@@ -74,10 +74,61 @@ smtp/inet` removes Debian's stock smtpd). Outbound delivery still uses the
|
|||||||
`smtpd_tls_security_level=encrypt`.
|
`smtpd_tls_security_level=encrypt`.
|
||||||
- **25/smtp inet** — when `INBOUND_RELAY_ENABLE=true` and/or
|
- **25/smtp inet** — when `INBOUND_RELAY_ENABLE=true` and/or
|
||||||
`DMARC_REPORTS_ENABLE=true`. No SASL, no OpenDKIM, no journal-milter —
|
`DMARC_REPORTS_ENABLE=true`. No SASL, no OpenDKIM, no journal-milter —
|
||||||
`smtpd_milters` is always overridden on this service (to the antispam milter
|
`smtpd_milters` is always overridden on this service, because main.cf's
|
||||||
or to nothing), because main.cf's submission chain would otherwise let a
|
submission chain would otherwise let a signing outage defer relayed mail and
|
||||||
signing outage defer relayed mail and would file inbound mail into the
|
would file inbound mail into the outbound send log under the sender's
|
||||||
outbound send log under the sender's domain.
|
domain. With the inbound relay on the chain is the optional antispam milter,
|
||||||
|
then the panel's **inbound journal milter** (unix socket
|
||||||
|
`INBOUND_JOURNAL_MILTER_SOCKET`, default
|
||||||
|
`/run/selfpost/inbound-journal.sock`, `default_action=accept`); with DMARC
|
||||||
|
ingest alone it is empty. Postfix hands a message to its milters one after
|
||||||
|
another at end-of-message, each seeing what the one before added, so the
|
||||||
|
journal milter sees only mail the filter let through and reads the filter's
|
||||||
|
verdict headers (`X-Rspamd-Action`, `X-Spamd-Result`) off it. It writes one
|
||||||
|
`inbound_spam_log` row per recipient at an inbound domain - a DMARC report
|
||||||
|
address is not one - and never refuses anything
|
||||||
|
([internal/milter/inbound.go](../internal/milter/inbound.go)). The headers
|
||||||
|
are read only while a filter is attached, and only next to the filter's
|
||||||
|
signature: a third header, `X-SelfPost-Filter`, the HMAC-SHA256 of the
|
||||||
|
message's queue id under the key in `/data/antispam/filter.key`
|
||||||
|
(`inbound.FilterKey` makes it on first start; rspamd reads it through the
|
||||||
|
mounted directory and signs in a custom routine of `milter_headers.conf`).
|
||||||
|
With the filter down - it is attached fail-open - a sender's own verdict
|
||||||
|
headers carry no signature and the row is that of mail nobody judged.
|
||||||
|
Refused mail never reaches this milter.
|
||||||
|
Refusals come from `mail.log`: this listener logs as `postfix/inbound`
|
||||||
|
(`syslog_name`), and the log tailer turns a `reject:` line of
|
||||||
|
`postfix/inbound/smtpd` (Postfix's own policy) and a `milter-reject` /
|
||||||
|
`milter-hold` / `milter-discard` line of `cleanup` (the filter's verdict,
|
||||||
|
counted only for a queue id that listener opened) into a row, dated by the
|
||||||
|
line, when the recipient is at an inbound domain
|
||||||
|
([internal/logtail/inbound.go](../internal/logtail/inbound.go)). A line
|
||||||
|
names one recipient, so a message refused at end-of-message is one row
|
||||||
|
whatever its recipient count. The tailer saves its place every five seconds,
|
||||||
|
so after an unclean stop it reads some lines again; the lines it reads on
|
||||||
|
start are written only if the same row is not there already. The
|
||||||
|
journal is pruned every six hours: rows
|
||||||
|
older than the `inbound_spam_log_retention_days` setting, then down to
|
||||||
|
10 000.
|
||||||
|
**Filter lists** (`inbound_spam_list`): allow and deny entries for the
|
||||||
|
external filter - sender address, sender domain, client address or network -
|
||||||
|
validated and normalised by `internal/inbound` (`lists.go`) and written as
|
||||||
|
six map files under `/data/antispam/` (`ANTISPAM_DIR`), each atomically, on
|
||||||
|
every change and on every start with the relay on. SelfPost never consults
|
||||||
|
them. The sidecar mounts the directory and rspamd's multimap rules
|
||||||
|
([deploy/antispam/rspamd/multimap.conf](../deploy/antispam/rspamd/multimap.conf))
|
||||||
|
read them: a deny entry is a prefilter `reject`, an allow entry a symbol
|
||||||
|
weighing -100. Allow is deliberately not a prefilter `accept`: with two
|
||||||
|
prefilters rspamd lets the one that ran first decide, and deny then won or
|
||||||
|
lost by rule order (seen on rspamd 3.11). A refusal by the deny list
|
||||||
|
overrides any score, so deny wins whichever entries match.
|
||||||
|
How often an entry decided a message is not stored: `Lists.Usage` reads it
|
||||||
|
off the journal on each view of Filter lists - the rows a list decided (the
|
||||||
|
deny rule's reply, the allow rule's symbol, which the journal milter keeps
|
||||||
|
ahead of its six-symbol cap) matched against the entries as rspamd matches
|
||||||
|
them. When that stops being good enough is written in
|
||||||
|
[plans/inbound-antispam-panel.md](plans/inbound-antispam-panel.md) § Filter
|
||||||
|
list usage.
|
||||||
**Inbound relay** (`INBOUND_RELAY_ENABLE=true`): accepts only
|
**Inbound relay** (`INBOUND_RELAY_ENABLE=true`): accepts only
|
||||||
`relay_domains` + `relay_recipient_maps` (`reject_unauth_destination`,
|
`relay_domains` + `relay_recipient_maps` (`reject_unauth_destination`,
|
||||||
`reject_unlisted_recipient`). Maps under `/data/postfix/` (`relay_domains`,
|
`reject_unlisted_recipient`). Maps under `/data/postfix/` (`relay_domains`,
|
||||||
@@ -531,16 +582,17 @@ full backups also archive the operator's `./certs` PEM files when present.
|
|||||||
|
|
||||||
### Database
|
### Database
|
||||||
|
|
||||||
The schema is one embedded baseline, `internal/store/migrations/0001_init.sql`,
|
The schema is an embedded baseline, `internal/store/migrations/0001_init.sql`,
|
||||||
applied by `store.Open` into an empty file; `PRAGMA user_version` is `1` and
|
and the migrations after it (`0002_inbound_spam_log.sql`), applied by
|
||||||
|
`store.Open` in file order; `PRAGMA user_version` counts the files applied and
|
||||||
`PRAGMA application_id` is `0x53503230` ("SP20"). There is no 1.x migration
|
`PRAGMA application_id` is `0x53503230` ("SP20"). There is no 1.x migration
|
||||||
chain. **2.0 starts from an empty data directory:** a database with a schema
|
chain. **2.0 starts from an empty data directory:** a database with a schema
|
||||||
version but without the 2.x `application_id` — what every 1.x file looks like —
|
version but without the 2.x `application_id` — what every 1.x file looks like —
|
||||||
is refused at start with `ErrForeignSchema` and left untouched, as is one whose
|
is refused at start with `ErrForeignSchema` and left untouched, as is one whose
|
||||||
version is newer than the build knows. A restored 1.x backup fails the same
|
version is newer than the build knows. A restored 1.x backup fails the same
|
||||||
way. Domains are carried across with a domain export, whose format is not the
|
way. Domains are carried across with a domain export, whose format is not the
|
||||||
database. Later schema changes are new files (`0002_*.sql`, …); once `2.0.0` is
|
database. A schema change is a new file; the baseline has not been edited
|
||||||
released the baseline is never edited. History and the per-table list:
|
since `2.0.0`. History and the per-table list:
|
||||||
[schema-migrations.md](schema-migrations.md).
|
[schema-migrations.md](schema-migrations.md).
|
||||||
|
|
||||||
The tables that carry the panel's model:
|
The tables that carry the panel's model:
|
||||||
|
|||||||
@@ -78,7 +78,7 @@
|
|||||||
<div class="box">
|
<div class="box">
|
||||||
<div class="sp-box-head"><span class="sp-no">01</span><h2>Full backup</h2><a class="sp-help" href="help.html#backup" title="Backups, encryption and restore"><i class="ti ti-help-circle"></i></a></div>
|
<div class="sp-box-head"><span class="sp-no">01</span><h2>Full backup</h2><a class="sp-help" href="help.html#backup" title="Backups, encryption and restore"><i class="ti ti-help-circle"></i></a></div>
|
||||||
<div class="sp-box-body">
|
<div class="sp-box-body">
|
||||||
<p class="mb-3"><code>data/</code> (database, DKIM keys, credentials, the queue), <code>docker-compose.yml</code>, <code>.env</code> and <code>certs/</code> in one archive. Restore into an empty project directory and start the <strong>same SelfPost version</strong>. The reverse-proxy vhost is not included.</p>
|
<p class="mb-3"><code>data/</code> (database, DKIM keys, credentials), <code>docker-compose.yml</code>, <code>.env</code> and <code>certs/</code> in one archive. The Postfix queue is left out: it cannot be copied reliably while the server is running. Restore into an empty project directory and start the <strong>same SelfPost version</strong>. The reverse-proxy vhost is not included.</p>
|
||||||
<form class="sp-form" method="post" action="#">
|
<form class="sp-form" method="post" action="#">
|
||||||
<div class="field"><label class="checkbox"><input type="checkbox" name="encrypt" checked> Encrypt with a password (<code>.spbk</code>)</label></div>
|
<div class="field"><label class="checkbox"><input type="checkbox" name="encrypt" checked> Encrypt with a password (<code>.spbk</code>)</label></div>
|
||||||
<div class="sp-pair">
|
<div class="sp-pair">
|
||||||
|
|||||||
@@ -37,7 +37,7 @@ SIGNED_OUT = {"sp-signin", "sp-signin-brand", "sp-signin-form", "sp-foot"}
|
|||||||
# Selectors allowed to set a width. Everything else gets its width from the
|
# Selectors allowed to set a width. Everything else gets its width from the
|
||||||
# layout it sits in — that is the whole point of the redesign.
|
# layout it sits in — that is the whole point of the redesign.
|
||||||
WIDTH_OK = (".navbar-brand img", ".sp-head .sp-lead", ".sp-head .sp-grow", ".sp-postmark", ".table td.sp-clip",
|
WIDTH_OK = (".navbar-brand img", ".sp-head .sp-lead", ".sp-head .sp-grow", ".sp-postmark", ".table td.sp-clip",
|
||||||
".sp-h-icon", ".sp-signin", ".sp-timeline li::before", ".sp-perf")
|
".sp-h-icon", ".sp-signin", ".sp-timeline li::before", ".sp-perf", ".sp-sheet")
|
||||||
|
|
||||||
|
|
||||||
class Node:
|
class Node:
|
||||||
|
|||||||
@@ -114,6 +114,7 @@
|
|||||||
<tr><td>error, fail, mismatch, bounced</td><td><span class="tag is-danger is-light">fail</span></td><td class="sp-mono sp-small">tag is-danger is-light</td></tr>
|
<tr><td>error, fail, mismatch, bounced</td><td><span class="tag is-danger is-light">fail</span></td><td class="sp-mono sp-small">tag is-danger is-light</td></tr>
|
||||||
<tr><td>unknown, off, neutral counts</td><td><span class="tag is-light">off</span></td><td class="sp-mono sp-small">tag is-light</td></tr>
|
<tr><td>unknown, off, neutral counts</td><td><span class="tag is-light">off</span></td><td class="sp-mono sp-small">tag is-light</td></tr>
|
||||||
<tr><td>held: quarantine · role: global</td><td><span class="tag is-primary is-light">quarantine</span></td><td class="sp-mono sp-small">tag is-primary is-light</td></tr>
|
<tr><td>held: quarantine · role: global</td><td><span class="tag is-primary is-light">quarantine</span></td><td class="sp-mono sp-small">tag is-primary is-light</td></tr>
|
||||||
|
<tr><td>a tag that opens a <code>sheet</code></td><td><button type="button" class="tag is-danger is-light" popovertarget="kit-sheet">reject</button><div class="sp-sheet" id="kit-sheet" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-danger is-light">reject</span><h2>sheet</h2><button type="button" popovertarget="kit-sheet" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>What it holds</th><td>What does not fit the row, as name and value</td></tr><tr><th>How it closes</th><td class="sp-mono sp-small">Esc<br>a click outside<br>the cross</td></tr></tbody></table></div></div></div></div></td><td class="sp-mono sp-small">button.tag + sp-sheet (popover)</td></tr>
|
||||||
</tbody>
|
</tbody>
|
||||||
</table></div>
|
</table></div>
|
||||||
</div>
|
</div>
|
||||||
|
|||||||
@@ -80,7 +80,7 @@
|
|||||||
<div><dt>Last report</dt><dd>2026-09-21 06:12 UTC</dd></div>
|
<div><dt>Last report</dt><dd>2026-09-21 06:12 UTC</dd></div>
|
||||||
<div><dt>This week</dt><dd>14 kept · 0 parse failures</dd></div>
|
<div><dt>This week</dt><dd>14 kept · 0 parse failures</dd></div>
|
||||||
<div><dt>Hosted addresses</dt><dd class="sp-mono sp-small">dmarc-reports+<domain>@mail.example.org</dd></div>
|
<div><dt>Hosted addresses</dt><dd class="sp-mono sp-small">dmarc-reports+<domain>@mail.example.org</dd></div>
|
||||||
<div><dt>Retention</dt><dd>500 reports or 90 days</dd></div>
|
<div><dt>Retention</dt><dd>500 reports a domain or 90 days</dd></div>
|
||||||
</dl>
|
</dl>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|||||||
@@ -97,6 +97,8 @@
|
|||||||
<li><a href="#upstream"><i class="ti ti-arrow-forward-up"></i>Upstream</a></li>
|
<li><a href="#upstream"><i class="ti ti-arrow-forward-up"></i>Upstream</a></li>
|
||||||
<li><a href="#recipients"><i class="ti ti-address-book"></i>Valid recipients</a></li>
|
<li><a href="#recipients"><i class="ti ti-address-book"></i>Valid recipients</a></li>
|
||||||
<li><a href="#filter"><i class="ti ti-shield-check"></i>Spam filter</a></li>
|
<li><a href="#filter"><i class="ti ti-shield-check"></i>Spam filter</a></li>
|
||||||
|
<li><a href="#inbound-log"><i class="ti ti-list-details"></i>Inbound log</a></li>
|
||||||
|
<li><a href="#filter-lists"><i class="ti ti-filter"></i>Filter lists</a></li>
|
||||||
</ul>
|
</ul>
|
||||||
<p class="menu-label">Server</p>
|
<p class="menu-label">Server</p>
|
||||||
<ul class="menu-list">
|
<ul class="menu-list">
|
||||||
@@ -132,12 +134,14 @@
|
|||||||
<div class="sp-help-topic" id="upstream"><h3>Upstream</h3><p>The upstream is the server that actually holds the mailboxes. Give its host and port — used exactly as written, with no MX lookup — and choose the TLS for the hop to it: Opportunistic uses TLS when the upstream offers it and otherwise sends in the clear, Required holds the mail until a TLS session is possible, Off never uses TLS. While the upstream does not answer, Postfix keeps the mail and retries as for any other mail.</p></div>
|
<div class="sp-help-topic" id="upstream"><h3>Upstream</h3><p>The upstream is the server that actually holds the mailboxes. Give its host and port — used exactly as written, with no MX lookup — and choose the TLS for the hop to it: Opportunistic uses TLS when the upstream offers it and otherwise sends in the clear, Required holds the mail until a TLS session is possible, Off never uses TLS. While the upstream does not answer, Postfix keeps the mail and retries as for any other mail.</p></div>
|
||||||
<div class="sp-help-topic" id="recipients"><h3>Valid recipients</h3><p>Choose whom this domain accepts. Listed addresses only: the relay accepts exactly the addresses you list, one per line or comma-separated, and refuses any other while the sender is still connected, so it never has to send a bounce for them. Any recipient: everything at the domain is accepted and passed on, and an address the upstream then rejects comes back as a bounce from this server to a sender who is often forged — backscatter. Prefer a list, and keep it in step with the mailboxes upstream.</p></div>
|
<div class="sp-help-topic" id="recipients"><h3>Valid recipients</h3><p>Choose whom this domain accepts. Listed addresses only: the relay accepts exactly the addresses you list, one per line or comma-separated, and refuses any other while the sender is still connected, so it never has to send a bounce for them. Any recipient: everything at the domain is accepted and passed on, and an address the upstream then rejects comes back as a bounce from this server to a sender who is often forged — backscatter. Prefer a list, and keep it in step with the mailboxes upstream.</p></div>
|
||||||
<div class="sp-help-topic" id="filter"><h3>Spam filter</h3><p>The spam filter belongs to the server, not to a domain. The server's administrator attaches it in the server's <code>.env</code>, and from then on every inbound domain is screened by it before mail is handed on; the box shows on or off and has nothing to switch. Off means mail goes to the upstream unfiltered. While the filter is down, mail is either accepted unfiltered (the default) or deferred, so the sending server tries again later, as the server's setting says; the box states which.</p></div>
|
<div class="sp-help-topic" id="filter"><h3>Spam filter</h3><p>The spam filter belongs to the server, not to a domain. The server's administrator attaches it in the server's <code>.env</code>, and from then on every inbound domain is screened by it before mail is handed on; the box shows on or off and has nothing to switch. Off means mail goes to the upstream unfiltered. While the filter is down, mail is either accepted unfiltered (the default) or deferred, so the sending server tries again later, as the server's setting says; the box states which.</p></div>
|
||||||
|
<div class="sp-help-topic" id="inbound-log"><h3>Inbound log</h3><p>One row per recipient of a message offered on port 25 for an inbound domain, with what was decided: <span class="tag is-success is-light">accept</span> — handed on to the upstream; <span class="tag is-danger is-light">reject</span> — refused for good; <span class="tag is-warning is-light">tempfail</span> — the sender was asked to come back later; <span class="tag is-primary is-light">quarantine</span> — held by the filter. <em>Why</em> names who decided. <code>rspamd</code> is the spam filter, with its action, the score against the reject threshold and the heaviest symbols; <code>postfix</code> is the relay's own rule — an unknown recipient, a sender domain that does not exist — with its reply; <code>selfpost</code> means nobody judged the message: it was relayed with no filter attached. Accept says what this server did, not what the upstream did with the message afterwards.<br><br>Accepted mail is recorded as it passes; refused mail is read from <code>mail.log</code>, so its row has no subject, and a message refused after its text was sent is one row however many recipients it had. A refusal to relay for a domain that is not an inbound one is every scanner on the internet and is not recorded. Rows are kept for the number of days set in Settings, and never more than 10 000.</p></div>
|
||||||
|
<div class="sp-help-topic" id="filter-lists"><h3>Filter lists</h3><p>Two lists for the whole server that correct the spam filter by hand. An entry is a sender address, a sender domain — that domain, not its subdomains — or a client address or network in CIDR form. A sender on the Deny list is refused outright. A sender on the Allow list is given a weight no spam score outweighs, so the filter lets it through. The sender's address is whatever the sending server says it is - anyone can claim to be a sender you allow - so allow a client address or network where you can, and a sender address only for mail you would rather receive forged than lose. When entries of both lists match one message, <strong>Deny wins</strong>.<br><br>SelfPost does not apply the lists itself: it writes them as files under <code>/data/antispam/</code>, and rspamd rereads a file within seconds of a change. With the spam filter off the lists are kept and can be edited, but nothing reads them. The sender matched is the envelope sender — the one in the log — not the From line a mail program shows. Matched counts the messages in the Inbound log that the entry decided, so it reaches back as far as the log does; never means not once in that time, and such an entry is one to consider removing.</p></div>
|
||||||
</div>
|
</div>
|
||||||
<div class="box">
|
<div class="box">
|
||||||
<div class="sp-box-head"><span class="sp-no">04</span><h2>Server</h2></div>
|
<div class="sp-box-head"><span class="sp-no">04</span><h2>Server</h2></div>
|
||||||
<div class="sp-help-topic" id="backup"><h3>Backup and moving a domain</h3><p>Full backup downloads one archive of the whole instance: <code>data/</code> (the database, DKIM keys, application credentials and the queue), <code>docker-compose.yml</code>, <code>.env</code> and <code>certs/</code>. It holds secrets, so leave Encrypt with a password ticked: the file is sealed as <code>.spbk</code>, the password is not stored, and without it nothing can be recovered. To restore, unpack it into an empty project directory (decrypt a <code>.spbk</code> first with <code>selfpost-backup -decrypt</code>) and start the same SelfPost version that made it; the panel refuses data from another version. The reverse-proxy configuration and <code>mail.log</code> are not included. One domain moves on its own: Export domain on its settings page writes a <code>.json</code> (or encrypted <code>.spde</code>) with its DKIM key and its applications' passwords, and Import a domain here reads it into another instance, so the published DNS needs no change.</p></div>
|
<div class="sp-help-topic" id="backup"><h3>Backup and moving a domain</h3><p>Full backup downloads one archive of the whole instance: <code>data/</code> (the database, DKIM keys and application credentials), <code>docker-compose.yml</code>, <code>.env</code> and <code>certs/</code>. The Postfix queue is not in it. The backup is taken while the server is running, and Postfix keeps moving messages through the queue, so a copy could hold a message twice or miss it; mail still waiting for delivery is therefore not carried over, and a restored server starts with an empty queue. To keep the queue, stop the container and archive the project directory instead. The private key in <code>certs/</code> has to be readable by the panel: give it to group 998 on the host (<code>chgrp 998</code>, <code>chmod 640</code>). Until then the panel refuses to make a backup and names the file, rather than hand over an archive without the key. It holds secrets, so leave Encrypt with a password ticked: the file is sealed as <code>.spbk</code>, the password is not stored, and without it nothing can be recovered. To restore, unpack it into an empty project directory (decrypt a <code>.spbk</code> first with <code>selfpost-backup -decrypt</code>) and start the same SelfPost version that made it; the panel refuses data from another version. The reverse-proxy configuration and <code>mail.log</code> are not included. One domain moves on its own: Export domain on its settings page writes a <code>.json</code> (or encrypted <code>.spde</code>) with its DKIM key and its applications' passwords, and Import a domain here reads it into another instance, so the published DNS needs no change.</p></div>
|
||||||
<div class="sp-help-topic" id="users"><h3>Users and roles</h3><p>The two roles are a reach, not a rank: every user is an administrator and signs in with a username and password. Global reaches the whole server — Overview, Server, the queue and every domain — and is the only role that adds or deletes domains and users. Domain reaches only the domains assigned to it, outbound and inbound separately (the same name on both lists is two assignments); <em>All</em> includes domains added later, and a domain user needs at least one domain or <em>All</em> on either list. The only global user cannot be demoted or deleted, and you cannot delete the account you are signed in with. A new password set here signs that user out of their other sessions.</p></div>
|
<div class="sp-help-topic" id="users"><h3>Users and roles</h3><p>The two roles are a reach, not a rank: every user is an administrator and signs in with a username and password. Global reaches the whole server — Overview, Server, the queue and every domain — and is the only role that adds or deletes domains and users. Domain reaches only the domains assigned to it, outbound and inbound separately (the same name on both lists is two assignments); <em>All</em> includes domains added later, and a domain user needs at least one domain or <em>All</em> on either list. The only global user cannot be demoted or deleted, and you cannot delete the account you are signed in with. A new password set here signs that user out of their other sessions.</p></div>
|
||||||
<div class="sp-help-topic" id="settings"><h3>Settings</h3><p>Settings holds what is true of the whole instance. Log retention is how many days the Outbound log keeps its rows, from 7 to 365 and 90 by default; a sweep every six hours deletes older ones, and it is the main driver of how fast <code>/data</code> grows. It does not change the rotation of <code>mail.log</code>, which is daily with about two weeks kept. The sending rate limit shown under it is level 1, the per-client-IP ceiling set in <code>.env</code>: changing it needs a restart, and no level-2 limit can exceed it.</p></div>
|
<div class="sp-help-topic" id="settings"><h3>Settings</h3><p>Settings holds what is true of the whole instance. Log retention is how many days the Outbound log keeps its rows, from 7 to 365 and 90 by default; a sweep every six hours deletes older ones, and it is the main driver of how fast <code>/data</code> grows. With the inbound relay on there is a second number, for the Inbound log, with the same range. It does not change the rotation of <code>mail.log</code>, which is daily with about two weeks kept. The sending rate limit shown under it is level 1, the per-client-IP ceiling set in <code>.env</code>: changing it needs a restart, and no level-2 limit can exceed it.</p></div>
|
||||||
</div>
|
</div>
|
||||||
<div class="box">
|
<div class="box">
|
||||||
<div class="sp-box-head"><span class="sp-no">05</span><h2>Account</h2></div>
|
<div class="sp-box-head"><span class="sp-no">05</span><h2>Account</h2></div>
|
||||||
|
|||||||
@@ -74,7 +74,7 @@
|
|||||||
</div>
|
</div>
|
||||||
|
|
||||||
<div class="box">
|
<div class="box">
|
||||||
<div class="sp-box-head"><span class="sp-no">01</span><h2>Add an entry</h2><a class="sp-help" href="help.html#inbound" title="How the lists are applied"><i class="ti ti-help-circle"></i></a></div>
|
<div class="sp-box-head"><span class="sp-no">01</span><h2>Add an entry</h2><a class="sp-help" href="help.html#filter-lists" title="How the lists are applied"><i class="ti ti-help-circle"></i></a></div>
|
||||||
<div class="sp-box-body">
|
<div class="sp-box-body">
|
||||||
<form class="sp-form" method="post" action="#">
|
<form class="sp-form" method="post" action="#">
|
||||||
<div class="columns">
|
<div class="columns">
|
||||||
@@ -84,7 +84,7 @@
|
|||||||
</div>
|
</div>
|
||||||
<div class="column is-3">
|
<div class="column is-3">
|
||||||
<div class="field"><label class="label" for="etype">Entry is</label>
|
<div class="field"><label class="label" for="etype">Entry is</label>
|
||||||
<div class="control"><div class="select is-fullwidth"><select id="etype" name="entry_type"><option>a sender address</option><option>a sender domain</option><option>a client IP or CIDR</option></select></div></div></div>
|
<div class="control"><div class="select is-fullwidth"><select id="etype" name="entry_type"><option>a sender address</option><option>a sender domain</option><option>a client address or network</option></select></div></div></div>
|
||||||
</div>
|
</div>
|
||||||
<div class="column is-6">
|
<div class="column is-6">
|
||||||
<div class="field"><label class="label" for="value">Value</label>
|
<div class="field"><label class="label" for="value">Value</label>
|
||||||
@@ -103,11 +103,11 @@
|
|||||||
<div class="box">
|
<div class="box">
|
||||||
<div class="sp-box-head"><span class="sp-no">02</span><h2>Deny</h2><div class="sp-end">3 entries · wins over Allow</div></div>
|
<div class="sp-box-head"><span class="sp-no">02</span><h2>Deny</h2><div class="sp-end">3 entries · wins over Allow</div></div>
|
||||||
<div class="table-container"><table class="table is-fullwidth is-hoverable">
|
<div class="table-container"><table class="table is-fullwidth is-hoverable">
|
||||||
<thead><tr><th>Value</th><th>Note</th><th>Added</th><th></th></tr></thead>
|
<thead><tr><th>Value</th><th>Kind</th><th>Matched</th><th></th></tr></thead>
|
||||||
<tbody>
|
<tbody>
|
||||||
<tr><td class="sp-mono sp-small">@bulk.example <span class="tag is-light">domain</span></td><td class="sp-muted">newsletter nobody subscribed to</td><td class="sp-muted sp-nowrap sp-small">2026-09-02</td><td class="sp-actions"><a class="has-text-danger" href="#">Remove</a></td></tr>
|
<tr><td class="sp-mono sp-small">bulk.example</td><td><button type="button" class="tag is-light" popovertarget="deny-1">domain</button><div class="sp-sheet" id="deny-1" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-light">domain</span><h2>bulk.example</h2><button type="button" popovertarget="deny-1" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Note</th><td>newsletter nobody subscribed to</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-09-02</td></tr><tr><th>Matched</th><td>41 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-09-21 13:57</td></tr></tbody></table></div></div></div></div></td><td class="sp-nowrap">41</td><td class="sp-actions"><form method="post" action="#"><button type="submit" class="has-text-danger">Remove</button></form></td></tr>
|
||||||
<tr><td class="sp-mono sp-small">198.51.100.0/24 <span class="tag is-light">ip</span></td><td class="sp-muted">dictionary attack on lists.example.org</td><td class="sp-muted sp-nowrap sp-small">2026-08-19</td><td class="sp-actions"><a class="has-text-danger" href="#">Remove</a></td></tr>
|
<tr><td class="sp-mono sp-small">198.51.100.0/24</td><td><button type="button" class="tag is-light" popovertarget="deny-2">ip</button><div class="sp-sheet" id="deny-2" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-light">ip</span><h2>198.51.100.0/24</h2><button type="button" popovertarget="deny-2" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Note</th><td>dictionary attack on lists.example.org</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-08-19</td></tr><tr><th>Matched</th><td>1203 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-09-21 09:12</td></tr></tbody></table></div></div></div></div></td><td class="sp-nowrap">1203</td><td class="sp-actions"><form method="post" action="#"><button type="submit" class="has-text-danger">Remove</button></form></td></tr>
|
||||||
<tr><td class="sp-mono sp-small">win@lottery.example <span class="tag is-light">address</span></td><td class="sp-muted">—</td><td class="sp-muted sp-nowrap sp-small">2026-07-30</td><td class="sp-actions"><a class="has-text-danger" href="#">Remove</a></td></tr>
|
<tr><td class="sp-mono sp-small">win@lottery.example</td><td><button type="button" class="tag is-light" popovertarget="deny-3">address</button><div class="sp-sheet" id="deny-3" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-light">address</span><h2>win@lottery.example</h2><button type="button" popovertarget="deny-3" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Note</th><td>—</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-07-30</td></tr><tr><th>Matched</th><td>never in the last 90 days</td></tr></tbody></table></div></div></div></div></td><td class="sp-muted sp-nowrap">never</td><td class="sp-actions"><form method="post" action="#"><button type="submit" class="has-text-danger">Remove</button></form></td></tr>
|
||||||
</tbody>
|
</tbody>
|
||||||
</table></div>
|
</table></div>
|
||||||
</div>
|
</div>
|
||||||
@@ -116,10 +116,10 @@
|
|||||||
<div class="box">
|
<div class="box">
|
||||||
<div class="sp-box-head"><span class="sp-no">03</span><h2>Allow</h2><div class="sp-end">2 entries</div></div>
|
<div class="sp-box-head"><span class="sp-no">03</span><h2>Allow</h2><div class="sp-end">2 entries</div></div>
|
||||||
<div class="table-container"><table class="table is-fullwidth is-hoverable">
|
<div class="table-container"><table class="table is-fullwidth is-hoverable">
|
||||||
<thead><tr><th>Value</th><th>Note</th><th>Added</th><th></th></tr></thead>
|
<thead><tr><th>Value</th><th>Kind</th><th>Matched</th><th></th></tr></thead>
|
||||||
<tbody>
|
<tbody>
|
||||||
<tr><td class="sp-mono sp-small">@partner.example <span class="tag is-light">domain</span></td><td class="sp-muted">invoices were scored as spam</td><td class="sp-muted sp-nowrap sp-small">2026-09-11</td><td class="sp-actions"><a class="has-text-danger" href="#">Remove</a></td></tr>
|
<tr><td class="sp-mono sp-small">partner.example</td><td><button type="button" class="tag is-light" popovertarget="allow-1">domain</button><div class="sp-sheet" id="allow-1" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-light">domain</span><h2>partner.example</h2><button type="button" popovertarget="allow-1" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Note</th><td>invoices were scored as spam</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-09-11</td></tr><tr><th>Matched</th><td>7 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-09-19 16:40</td></tr></tbody></table></div></div></div></div></td><td class="sp-nowrap">7</td><td class="sp-actions"><form method="post" action="#"><button type="submit" class="has-text-danger">Remove</button></form></td></tr>
|
||||||
<tr><td class="sp-mono sp-small">noreply@github.example <span class="tag is-light">address</span></td><td class="sp-muted">security advisories</td><td class="sp-muted sp-nowrap sp-small">2026-06-04</td><td class="sp-actions"><a class="has-text-danger" href="#">Remove</a></td></tr>
|
<tr><td class="sp-mono sp-small">noreply@github.example</td><td><button type="button" class="tag is-light" popovertarget="allow-2">address</button><div class="sp-sheet" id="allow-2" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-light">address</span><h2>noreply@github.example</h2><button type="button" popovertarget="allow-2" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Note</th><td>security advisories</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-06-04</td></tr><tr><th>Matched</th><td>18 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-09-21 13:30</td></tr></tbody></table></div></div></div></div></td><td class="sp-nowrap">18</td><td class="sp-actions"><form method="post" action="#"><button type="submit" class="has-text-danger">Remove</button></form></td></tr>
|
||||||
</tbody>
|
</tbody>
|
||||||
</table></div>
|
</table></div>
|
||||||
</div>
|
</div>
|
||||||
@@ -130,10 +130,10 @@
|
|||||||
<div class="sp-box-head"><span class="sp-no">04</span><h2>Where the lists go</h2><div class="sp-end"><span class="tag is-success is-light">applied</span></div></div>
|
<div class="sp-box-head"><span class="sp-no">04</span><h2>Where the lists go</h2><div class="sp-end"><span class="tag is-success is-light">applied</span></div></div>
|
||||||
<div class="sp-box-body">
|
<div class="sp-box-body">
|
||||||
<dl class="sp-facts">
|
<dl class="sp-facts">
|
||||||
<div><dt>Applied by</dt><dd>rspamd, from map files</dd></div>
|
<div><dt>Applied by</dt><dd>rspamd, from map files it rereads within seconds of a change</dd></div>
|
||||||
<div><dt>Files</dt><dd class="sp-mono sp-small">/data/antispam/allow.map<br>/data/antispam/deny.map</dd></div>
|
<div><dt>Files</dt><dd><span class="sp-mono">/data/antispam/</span> — six, one for each list and kind of entry</dd></div>
|
||||||
<div><dt>Last written</dt><dd>2026-09-11 08:40 UTC</dd></div>
|
<div><dt>Last written</dt><dd>2026-09-11 08:40 UTC</dd></div>
|
||||||
<div><dt>When the filter is off</dt><dd class="sp-muted">The lists are kept and editable, but nothing reads them until <code>INBOUND_ANTISPAM_MILTER</code> is set.</dd></div>
|
<div><dt>When both lists match</dt><dd>Deny wins, whichever entries they are.</dd></div>
|
||||||
</dl>
|
</dl>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|||||||
@@ -69,7 +69,7 @@
|
|||||||
<div class="sp-grow">
|
<div class="sp-grow">
|
||||||
<p class="sp-kicker">Inbound</p>
|
<p class="sp-kicker">Inbound</p>
|
||||||
<h1 class="title is-3">Inbound log</h1>
|
<h1 class="title is-3">Inbound log</h1>
|
||||||
<p class="sp-lead">What port 25 decided about each message, one row per recipient. Accepts come from the inbound journal, rejects from <code>mail.log</code>; the filter's reasons appear when the spam filter is on.</p>
|
<p class="sp-lead">What port 25 decided about each message, one row per recipient. Accepted mail is recorded as it passes, refused mail is read from <code>mail.log</code>; the filter's verdict appears when the spam filter is on. Rows older than 90 days are deleted — change that in <a href="settings.html">Settings</a>.</p>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
@@ -83,19 +83,20 @@
|
|||||||
<input class="input" type="date" name="to" value="2026-09-21" aria-label="To date">
|
<input class="input" type="date" name="to" value="2026-09-21" aria-label="To date">
|
||||||
<button type="submit" class="button"><i class="ti ti-filter"></i>Filter</button>
|
<button type="submit" class="button"><i class="ti ti-filter"></i>Filter</button>
|
||||||
</form>
|
</form>
|
||||||
|
<a class="sp-help" href="help.html#inbound-log" title="What the decisions mean"><i class="ti ti-help-circle"></i></a>
|
||||||
</div>
|
</div>
|
||||||
<div class="table-container"><table class="table is-fullwidth is-hoverable">
|
<div class="table-container"><table class="table is-fullwidth is-hoverable">
|
||||||
<thead><tr><th>Time, UTC</th><th>From · client</th><th>To</th><th>Subject</th><th>Decision</th><th>Why</th><th></th></tr></thead>
|
<thead><tr><th>Time, UTC</th><th>From</th><th>To</th><th>Subject</th><th>Decision</th><th></th></tr></thead>
|
||||||
<tbody>
|
<tbody>
|
||||||
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 14:00:18</td><td>t.berg@<wbr>uni.example<br><span class="sp-mono sp-muted sp-small">192.0.2.77</span></td><td>dev@<wbr>lists.example.org</td><td class="sp-clip">Re: [dev] patch series v3</td><td><span class="tag is-success is-light">accept</span></td><td class="sp-muted sp-small">rspamd · score 0.4</td><td class="sp-actions"><a href="in-filter-lists.html">Deny sender</a></td></tr>
|
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 14:00:18</td><td>t.berg@<wbr>uni.example<br><span class="sp-mono sp-muted sp-small">192.0.2.77</span></td><td>dev@<wbr>lists.example.org</td><td class="sp-clip">Re: [dev] patch series v3</td><td><button type="button" class="tag is-success is-light" popovertarget="why-1">accept</button><div class="sp-sheet" id="why-1" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-success is-light">accept</span><h2>Why</h2><button type="button" popovertarget="why-1" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Verdict</th><td>no action, score 0.40 / 15.00</td></tr><tr><th>Subject</th><td>Re: [dev] patch series v3</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"><a href="in-filter-lists.html">Deny sender</a></td></tr>
|
||||||
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:59:02</td><td>invoice@<wbr>partner-billing.example<br><span class="sp-mono sp-muted sp-small">203.0.113.140</span></td><td>owner@<wbr>lists.example.org</td><td class="sp-clip">Outstanding invoice — action required</td><td><span class="tag is-primary is-light">quarantine</span></td><td class="sp-muted sp-small">rspamd · score 9.8 · PHISHING, FORGED_SENDER</td><td class="sp-actions"><a href="in-quarantine-message.html">Review</a></td></tr>
|
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:59:02</td><td>invoice@<wbr>partner-billing.example<br><span class="sp-mono sp-muted sp-small">203.0.113.140</span></td><td>owner@<wbr>lists.example.org</td><td class="sp-clip">Outstanding invoice — action required</td><td><button type="button" class="tag is-success is-light" popovertarget="why-2">accept</button><div class="sp-sheet" id="why-2" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-success is-light">accept</span><h2>Why</h2><button type="button" popovertarget="why-2" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Verdict</th><td>add header, score 9.80 / 15.00</td></tr><tr><th>Symbols</th><td class="sp-mono sp-small">PHISHING(5.00)<br>FORGED_SENDER(3.00)<br>R_SPF_FAIL(1.00)</td></tr><tr><th>Subject</th><td>Outstanding invoice — action required</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"><a href="in-filter-lists.html">Deny sender</a></td></tr>
|
||||||
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:57:41</td><td>promo@<wbr>bulk.example<br><span class="sp-mono sp-muted sp-small">198.51.100.90</span></td><td>sales@<wbr>lists.example.org</td><td class="sp-clip sp-muted">—</td><td><span class="tag is-danger is-light">reject</span></td><td class="sp-muted sp-small">postfix · 550 5.1.1 unknown recipient</td><td class="sp-actions"><a href="in-filter-lists.html">Deny sender</a></td></tr>
|
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:57:41</td><td>promo@<wbr>bulk.example<br><span class="sp-mono sp-muted sp-small">198.51.100.90</span></td><td>sales@<wbr>lists.example.org</td><td class="sp-clip sp-muted">—</td><td><button type="button" class="tag is-danger is-light" popovertarget="why-3">reject</button><div class="sp-sheet" id="why-3" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-danger is-light">reject</span><h2>Why</h2><button type="button" popovertarget="why-3" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>postfix</td></tr><tr><th>Reply</th><td class="sp-mono sp-small">550 5.1.1 <sales@lists.example.org>: Recipient address rejected: User unknown in relay recipient table</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"></td></tr>
|
||||||
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:51:30</td><td>billing@<wbr>partner.example<br><span class="sp-mono sp-muted sp-small">198.51.100.12</span></td><td>owner@<wbr>lists.example.org</td><td class="sp-clip">Invoice 2026-0917</td><td><span class="tag is-danger is-light">reject</span></td><td class="sp-muted sp-small">rspamd · score 16.2 · BAYES_SPAM</td><td class="sp-actions"><a href="in-filter-lists.html">Allow sender</a></td></tr>
|
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:51:30</td><td>billing@<wbr>partner.example<br><span class="sp-mono sp-muted sp-small">198.51.100.12</span></td><td>owner@<wbr>lists.example.org</td><td class="sp-clip sp-muted">—</td><td><button type="button" class="tag is-danger is-light" popovertarget="why-4">reject</button><div class="sp-sheet" id="why-4" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-danger is-light">reject</span><h2>Why</h2><button type="button" popovertarget="why-4" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Reply</th><td class="sp-mono sp-small">5.7.1 Spam message rejected</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"><a href="in-filter-lists.html">Allow sender</a></td></tr>
|
||||||
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:44:02</td><td>hr@<wbr>partner.example<br><span class="sp-mono sp-muted sp-small">192.0.2.31</span></td><td>jobs@<wbr>acme.io</td><td class="sp-clip">Application: backend engineer</td><td><span class="tag is-warning is-light">tempfail</span></td><td class="sp-muted sp-small">selfpost · upstream 10.0.4.12 timed out</td><td class="sp-actions"></td></tr>
|
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:44:02</td><td>hr@<wbr>partner.example<br><span class="sp-mono sp-muted sp-small">192.0.2.31</span></td><td>jobs@<wbr>acme.io</td><td class="sp-clip sp-muted">—</td><td><button type="button" class="tag is-warning is-light" popovertarget="why-5">tempfail</button><div class="sp-sheet" id="why-5" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-warning is-light">tempfail</span><h2>Why</h2><button type="button" popovertarget="why-5" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Reply</th><td class="sp-mono sp-small">4.7.1 Try again later</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"><a href="in-filter-lists.html">Allow sender</a></td></tr>
|
||||||
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:30:55</td><td>noreply@<wbr>github.example<br><span class="sp-mono sp-muted sp-small">192.0.2.200</span></td><td>security@<wbr>lists.example.org</td><td class="sp-clip">[advisory] new report</td><td><span class="tag is-success is-light">accept</span></td><td class="sp-muted sp-small">rspamd · allow list</td><td class="sp-actions"></td></tr>
|
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:30:55</td><td>noreply@<wbr>github.example<br><span class="sp-mono sp-muted sp-small">192.0.2.200</span></td><td>security@<wbr>lists.example.org</td><td class="sp-clip">[advisory] new report</td><td><button type="button" class="tag is-success is-light" popovertarget="why-6">accept</button><div class="sp-sheet" id="why-6" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-success is-light">accept</span><h2>Why</h2><button type="button" popovertarget="why-6" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Verdict</th><td>no action, score -98.10 / 15.00</td></tr><tr><th>Symbols</th><td class="sp-mono sp-small">SELFPOST_ALLOW_FROM(-100.00)<br>R_DKIM_ALLOW(-0.20)</td></tr><tr><th>Subject</th><td>[advisory] new report</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"><a href="in-filter-lists.html">Deny sender</a></td></tr>
|
||||||
</tbody>
|
</tbody>
|
||||||
</table></div>
|
</table></div>
|
||||||
<div class="sp-box-foot"><span>Page 1 of 38</span><a href="#">Older →</a><span class="sp-end">Allow / Deny sender opens <a href="in-filter-lists.html">Filter lists</a> with the entry filled in · global administrator only</span></div>
|
<div class="sp-box-foot"><span>Page 1 of 38</span><a href="#">Older →</a><span class="sp-end">A decision opens its reason · Allow / Deny sender opens <a href="in-filter-lists.html">Filter lists</a> with the entry filled in</span></div>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<footer class="sp-foot">SelfPost 1.8.0 · © 2026 Mix · <a href="#">License (AGPL-3.0)</a> · <a href="#">Source</a> · No warranty</footer>
|
<footer class="sp-foot">SelfPost 1.8.0 · © 2026 Mix · <a href="#">License (AGPL-3.0)</a> · <a href="#">Source</a> · No warranty</footer>
|
||||||
|
|||||||
@@ -123,7 +123,7 @@
|
|||||||
"is-9": [
|
"is-9": [
|
||||||
"box[Overview]:help×3",
|
"box[Overview]:help×3",
|
||||||
"box[Outbound]:help×6",
|
"box[Outbound]:help×6",
|
||||||
"box[Inbound]:help×4",
|
"box[Inbound]:help×6",
|
||||||
"box[Server]:help×3",
|
"box[Server]:help×3",
|
||||||
"box[Account]:help"
|
"box[Account]:help"
|
||||||
]
|
]
|
||||||
|
|||||||
@@ -82,6 +82,9 @@
|
|||||||
<div class="field"><label class="label" for="ret">Keep outbound log rows, days</label>
|
<div class="field"><label class="label" for="ret">Keep outbound log rows, days</label>
|
||||||
<div class="control"><input class="input" id="ret" name="send_log_retention_days" type="number" min="7" max="365" value="30" required></div>
|
<div class="control"><input class="input" id="ret" name="send_log_retention_days" type="number" min="7" max="365" value="30" required></div>
|
||||||
<p class="help">The main driver of <code>/data</code> growth. Does not affect <code>mail.log</code> rotation (fourteen daily files).</p></div>
|
<p class="help">The main driver of <code>/data</code> growth. Does not affect <code>mail.log</code> rotation (fourteen daily files).</p></div>
|
||||||
|
<div class="field"><label class="label" for="inret">Keep inbound log rows, days</label>
|
||||||
|
<div class="control"><input class="input" id="inret" name="inbound_spam_log_retention_days" type="number" min="7" max="365" value="90" required></div>
|
||||||
|
<p class="help">The inbound log also never holds more than 10 000 rows: refused mail is logged at the rate strangers knock on port 25.</p></div>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|||||||
@@ -12,7 +12,7 @@
|
|||||||
<div class="box">
|
<div class="box">
|
||||||
<div class="sp-box-head"><span class="sp-no">01</span><h2>Full backup</h2><a class="sp-help" href="help.html#backup" title="Backups, encryption and restore"><i class="ti ti-help-circle"></i></a></div>
|
<div class="sp-box-head"><span class="sp-no">01</span><h2>Full backup</h2><a class="sp-help" href="help.html#backup" title="Backups, encryption and restore"><i class="ti ti-help-circle"></i></a></div>
|
||||||
<div class="sp-box-body">
|
<div class="sp-box-body">
|
||||||
<p class="mb-3"><code>data/</code> (database, DKIM keys, credentials, the queue), <code>docker-compose.yml</code>, <code>.env</code> and <code>certs/</code> in one archive. Restore into an empty project directory and start the <strong>same SelfPost version</strong>. The reverse-proxy vhost is not included.</p>
|
<p class="mb-3"><code>data/</code> (database, DKIM keys, credentials), <code>docker-compose.yml</code>, <code>.env</code> and <code>certs/</code> in one archive. The Postfix queue is left out: it cannot be copied reliably while the server is running. Restore into an empty project directory and start the <strong>same SelfPost version</strong>. The reverse-proxy vhost is not included.</p>
|
||||||
<form class="sp-form" method="post" action="#">
|
<form class="sp-form" method="post" action="#">
|
||||||
<div class="field"><label class="checkbox"><input type="checkbox" name="encrypt" checked> Encrypt with a password (<code>.spbk</code>)</label></div>
|
<div class="field"><label class="checkbox"><input type="checkbox" name="encrypt" checked> Encrypt with a password (<code>.spbk</code>)</label></div>
|
||||||
<div class="sp-pair">
|
<div class="sp-pair">
|
||||||
|
|||||||
@@ -48,6 +48,7 @@
|
|||||||
<tr><td>error, fail, mismatch, bounced</td><td><span class="tag is-danger is-light">fail</span></td><td class="sp-mono sp-small">tag is-danger is-light</td></tr>
|
<tr><td>error, fail, mismatch, bounced</td><td><span class="tag is-danger is-light">fail</span></td><td class="sp-mono sp-small">tag is-danger is-light</td></tr>
|
||||||
<tr><td>unknown, off, neutral counts</td><td><span class="tag is-light">off</span></td><td class="sp-mono sp-small">tag is-light</td></tr>
|
<tr><td>unknown, off, neutral counts</td><td><span class="tag is-light">off</span></td><td class="sp-mono sp-small">tag is-light</td></tr>
|
||||||
<tr><td>held: quarantine · role: global</td><td><span class="tag is-primary is-light">quarantine</span></td><td class="sp-mono sp-small">tag is-primary is-light</td></tr>
|
<tr><td>held: quarantine · role: global</td><td><span class="tag is-primary is-light">quarantine</span></td><td class="sp-mono sp-small">tag is-primary is-light</td></tr>
|
||||||
|
<tr><td>a tag that opens a <code>sheet</code></td><td><button type="button" class="tag is-danger is-light" popovertarget="kit-sheet">reject</button><div class="sp-sheet" id="kit-sheet" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-danger is-light">reject</span><h2>sheet</h2><button type="button" popovertarget="kit-sheet" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>What it holds</th><td>What does not fit the row, as name and value</td></tr><tr><th>How it closes</th><td class="sp-mono sp-small">Esc<br>a click outside<br>the cross</td></tr></tbody></table></div></div></div></div></td><td class="sp-mono sp-small">button.tag + sp-sheet (popover)</td></tr>
|
||||||
</tbody>
|
</tbody>
|
||||||
</table></div>
|
</table></div>
|
||||||
</div>
|
</div>
|
||||||
|
|||||||
@@ -14,7 +14,7 @@
|
|||||||
<div><dt>Last report</dt><dd>2026-09-21 06:12 UTC</dd></div>
|
<div><dt>Last report</dt><dd>2026-09-21 06:12 UTC</dd></div>
|
||||||
<div><dt>This week</dt><dd>14 kept · 0 parse failures</dd></div>
|
<div><dt>This week</dt><dd>14 kept · 0 parse failures</dd></div>
|
||||||
<div><dt>Hosted addresses</dt><dd class="sp-mono sp-small">dmarc-reports+<domain>@mail.example.org</dd></div>
|
<div><dt>Hosted addresses</dt><dd class="sp-mono sp-small">dmarc-reports+<domain>@mail.example.org</dd></div>
|
||||||
<div><dt>Retention</dt><dd>500 reports or 90 days</dd></div>
|
<div><dt>Retention</dt><dd>500 reports a domain or 90 days</dd></div>
|
||||||
</dl>
|
</dl>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|||||||
@@ -31,6 +31,8 @@
|
|||||||
<li><a href="#upstream"><i class="ti ti-arrow-forward-up"></i>Upstream</a></li>
|
<li><a href="#upstream"><i class="ti ti-arrow-forward-up"></i>Upstream</a></li>
|
||||||
<li><a href="#recipients"><i class="ti ti-address-book"></i>Valid recipients</a></li>
|
<li><a href="#recipients"><i class="ti ti-address-book"></i>Valid recipients</a></li>
|
||||||
<li><a href="#filter"><i class="ti ti-shield-check"></i>Spam filter</a></li>
|
<li><a href="#filter"><i class="ti ti-shield-check"></i>Spam filter</a></li>
|
||||||
|
<li><a href="#inbound-log"><i class="ti ti-list-details"></i>Inbound log</a></li>
|
||||||
|
<li><a href="#filter-lists"><i class="ti ti-filter"></i>Filter lists</a></li>
|
||||||
</ul>
|
</ul>
|
||||||
<p class="menu-label">Server</p>
|
<p class="menu-label">Server</p>
|
||||||
<ul class="menu-list">
|
<ul class="menu-list">
|
||||||
@@ -66,12 +68,14 @@
|
|||||||
<div class="sp-help-topic" id="upstream"><h3>Upstream</h3><p>The upstream is the server that actually holds the mailboxes. Give its host and port — used exactly as written, with no MX lookup — and choose the TLS for the hop to it: Opportunistic uses TLS when the upstream offers it and otherwise sends in the clear, Required holds the mail until a TLS session is possible, Off never uses TLS. While the upstream does not answer, Postfix keeps the mail and retries as for any other mail.</p></div>
|
<div class="sp-help-topic" id="upstream"><h3>Upstream</h3><p>The upstream is the server that actually holds the mailboxes. Give its host and port — used exactly as written, with no MX lookup — and choose the TLS for the hop to it: Opportunistic uses TLS when the upstream offers it and otherwise sends in the clear, Required holds the mail until a TLS session is possible, Off never uses TLS. While the upstream does not answer, Postfix keeps the mail and retries as for any other mail.</p></div>
|
||||||
<div class="sp-help-topic" id="recipients"><h3>Valid recipients</h3><p>Choose whom this domain accepts. Listed addresses only: the relay accepts exactly the addresses you list, one per line or comma-separated, and refuses any other while the sender is still connected, so it never has to send a bounce for them. Any recipient: everything at the domain is accepted and passed on, and an address the upstream then rejects comes back as a bounce from this server to a sender who is often forged — backscatter. Prefer a list, and keep it in step with the mailboxes upstream.</p></div>
|
<div class="sp-help-topic" id="recipients"><h3>Valid recipients</h3><p>Choose whom this domain accepts. Listed addresses only: the relay accepts exactly the addresses you list, one per line or comma-separated, and refuses any other while the sender is still connected, so it never has to send a bounce for them. Any recipient: everything at the domain is accepted and passed on, and an address the upstream then rejects comes back as a bounce from this server to a sender who is often forged — backscatter. Prefer a list, and keep it in step with the mailboxes upstream.</p></div>
|
||||||
<div class="sp-help-topic" id="filter"><h3>Spam filter</h3><p>The spam filter belongs to the server, not to a domain. The server's administrator attaches it in the server's <code>.env</code>, and from then on every inbound domain is screened by it before mail is handed on; the box shows on or off and has nothing to switch. Off means mail goes to the upstream unfiltered. While the filter is down, mail is either accepted unfiltered (the default) or deferred, so the sending server tries again later, as the server's setting says; the box states which.</p></div>
|
<div class="sp-help-topic" id="filter"><h3>Spam filter</h3><p>The spam filter belongs to the server, not to a domain. The server's administrator attaches it in the server's <code>.env</code>, and from then on every inbound domain is screened by it before mail is handed on; the box shows on or off and has nothing to switch. Off means mail goes to the upstream unfiltered. While the filter is down, mail is either accepted unfiltered (the default) or deferred, so the sending server tries again later, as the server's setting says; the box states which.</p></div>
|
||||||
|
<div class="sp-help-topic" id="inbound-log"><h3>Inbound log</h3><p>One row per recipient of a message offered on port 25 for an inbound domain, with what was decided: <span class="tag is-success is-light">accept</span> — handed on to the upstream; <span class="tag is-danger is-light">reject</span> — refused for good; <span class="tag is-warning is-light">tempfail</span> — the sender was asked to come back later; <span class="tag is-primary is-light">quarantine</span> — held by the filter. <em>Why</em> names who decided. <code>rspamd</code> is the spam filter, with its action, the score against the reject threshold and the heaviest symbols; <code>postfix</code> is the relay's own rule — an unknown recipient, a sender domain that does not exist — with its reply; <code>selfpost</code> means nobody judged the message: it was relayed with no filter attached. Accept says what this server did, not what the upstream did with the message afterwards.<br><br>Accepted mail is recorded as it passes; refused mail is read from <code>mail.log</code>, so its row has no subject, and a message refused after its text was sent is one row however many recipients it had. A refusal to relay for a domain that is not an inbound one is every scanner on the internet and is not recorded. Rows are kept for the number of days set in Settings, and never more than 10 000.</p></div>
|
||||||
|
<div class="sp-help-topic" id="filter-lists"><h3>Filter lists</h3><p>Two lists for the whole server that correct the spam filter by hand. An entry is a sender address, a sender domain — that domain, not its subdomains — or a client address or network in CIDR form. A sender on the Deny list is refused outright. A sender on the Allow list is given a weight no spam score outweighs, so the filter lets it through. The sender's address is whatever the sending server says it is - anyone can claim to be a sender you allow - so allow a client address or network where you can, and a sender address only for mail you would rather receive forged than lose. When entries of both lists match one message, <strong>Deny wins</strong>.<br><br>SelfPost does not apply the lists itself: it writes them as files under <code>/data/antispam/</code>, and rspamd rereads a file within seconds of a change. With the spam filter off the lists are kept and can be edited, but nothing reads them. The sender matched is the envelope sender — the one in the log — not the From line a mail program shows. Matched counts the messages in the Inbound log that the entry decided, so it reaches back as far as the log does; never means not once in that time, and such an entry is one to consider removing.</p></div>
|
||||||
</div>
|
</div>
|
||||||
<div class="box">
|
<div class="box">
|
||||||
<div class="sp-box-head"><span class="sp-no">04</span><h2>Server</h2></div>
|
<div class="sp-box-head"><span class="sp-no">04</span><h2>Server</h2></div>
|
||||||
<div class="sp-help-topic" id="backup"><h3>Backup and moving a domain</h3><p>Full backup downloads one archive of the whole instance: <code>data/</code> (the database, DKIM keys, application credentials and the queue), <code>docker-compose.yml</code>, <code>.env</code> and <code>certs/</code>. It holds secrets, so leave Encrypt with a password ticked: the file is sealed as <code>.spbk</code>, the password is not stored, and without it nothing can be recovered. To restore, unpack it into an empty project directory (decrypt a <code>.spbk</code> first with <code>selfpost-backup -decrypt</code>) and start the same SelfPost version that made it; the panel refuses data from another version. The reverse-proxy configuration and <code>mail.log</code> are not included. One domain moves on its own: Export domain on its settings page writes a <code>.json</code> (or encrypted <code>.spde</code>) with its DKIM key and its applications' passwords, and Import a domain here reads it into another instance, so the published DNS needs no change.</p></div>
|
<div class="sp-help-topic" id="backup"><h3>Backup and moving a domain</h3><p>Full backup downloads one archive of the whole instance: <code>data/</code> (the database, DKIM keys and application credentials), <code>docker-compose.yml</code>, <code>.env</code> and <code>certs/</code>. The Postfix queue is not in it. The backup is taken while the server is running, and Postfix keeps moving messages through the queue, so a copy could hold a message twice or miss it; mail still waiting for delivery is therefore not carried over, and a restored server starts with an empty queue. To keep the queue, stop the container and archive the project directory instead. The private key in <code>certs/</code> has to be readable by the panel: give it to group 998 on the host (<code>chgrp 998</code>, <code>chmod 640</code>). Until then the panel refuses to make a backup and names the file, rather than hand over an archive without the key. It holds secrets, so leave Encrypt with a password ticked: the file is sealed as <code>.spbk</code>, the password is not stored, and without it nothing can be recovered. To restore, unpack it into an empty project directory (decrypt a <code>.spbk</code> first with <code>selfpost-backup -decrypt</code>) and start the same SelfPost version that made it; the panel refuses data from another version. The reverse-proxy configuration and <code>mail.log</code> are not included. One domain moves on its own: Export domain on its settings page writes a <code>.json</code> (or encrypted <code>.spde</code>) with its DKIM key and its applications' passwords, and Import a domain here reads it into another instance, so the published DNS needs no change.</p></div>
|
||||||
<div class="sp-help-topic" id="users"><h3>Users and roles</h3><p>The two roles are a reach, not a rank: every user is an administrator and signs in with a username and password. Global reaches the whole server — Overview, Server, the queue and every domain — and is the only role that adds or deletes domains and users. Domain reaches only the domains assigned to it, outbound and inbound separately (the same name on both lists is two assignments); <em>All</em> includes domains added later, and a domain user needs at least one domain or <em>All</em> on either list. The only global user cannot be demoted or deleted, and you cannot delete the account you are signed in with. A new password set here signs that user out of their other sessions.</p></div>
|
<div class="sp-help-topic" id="users"><h3>Users and roles</h3><p>The two roles are a reach, not a rank: every user is an administrator and signs in with a username and password. Global reaches the whole server — Overview, Server, the queue and every domain — and is the only role that adds or deletes domains and users. Domain reaches only the domains assigned to it, outbound and inbound separately (the same name on both lists is two assignments); <em>All</em> includes domains added later, and a domain user needs at least one domain or <em>All</em> on either list. The only global user cannot be demoted or deleted, and you cannot delete the account you are signed in with. A new password set here signs that user out of their other sessions.</p></div>
|
||||||
<div class="sp-help-topic" id="settings"><h3>Settings</h3><p>Settings holds what is true of the whole instance. Log retention is how many days the Outbound log keeps its rows, from 7 to 365 and 90 by default; a sweep every six hours deletes older ones, and it is the main driver of how fast <code>/data</code> grows. It does not change the rotation of <code>mail.log</code>, which is daily with about two weeks kept. The sending rate limit shown under it is level 1, the per-client-IP ceiling set in <code>.env</code>: changing it needs a restart, and no level-2 limit can exceed it.</p></div>
|
<div class="sp-help-topic" id="settings"><h3>Settings</h3><p>Settings holds what is true of the whole instance. Log retention is how many days the Outbound log keeps its rows, from 7 to 365 and 90 by default; a sweep every six hours deletes older ones, and it is the main driver of how fast <code>/data</code> grows. With the inbound relay on there is a second number, for the Inbound log, with the same range. It does not change the rotation of <code>mail.log</code>, which is daily with about two weeks kept. The sending rate limit shown under it is level 1, the per-client-IP ceiling set in <code>.env</code>: changing it needs a restart, and no level-2 limit can exceed it.</p></div>
|
||||||
</div>
|
</div>
|
||||||
<div class="box">
|
<div class="box">
|
||||||
<div class="sp-box-head"><span class="sp-no">05</span><h2>Account</h2></div>
|
<div class="sp-box-head"><span class="sp-no">05</span><h2>Account</h2></div>
|
||||||
|
|||||||
@@ -8,7 +8,7 @@
|
|||||||
</div>
|
</div>
|
||||||
|
|
||||||
<div class="box">
|
<div class="box">
|
||||||
<div class="sp-box-head"><span class="sp-no">01</span><h2>Add an entry</h2><a class="sp-help" href="help.html#inbound" title="How the lists are applied"><i class="ti ti-help-circle"></i></a></div>
|
<div class="sp-box-head"><span class="sp-no">01</span><h2>Add an entry</h2><a class="sp-help" href="help.html#filter-lists" title="How the lists are applied"><i class="ti ti-help-circle"></i></a></div>
|
||||||
<div class="sp-box-body">
|
<div class="sp-box-body">
|
||||||
<form class="sp-form" method="post" action="#">
|
<form class="sp-form" method="post" action="#">
|
||||||
<div class="columns">
|
<div class="columns">
|
||||||
@@ -18,7 +18,7 @@
|
|||||||
</div>
|
</div>
|
||||||
<div class="column is-3">
|
<div class="column is-3">
|
||||||
<div class="field"><label class="label" for="etype">Entry is</label>
|
<div class="field"><label class="label" for="etype">Entry is</label>
|
||||||
<div class="control"><div class="select is-fullwidth"><select id="etype" name="entry_type"><option>a sender address</option><option>a sender domain</option><option>a client IP or CIDR</option></select></div></div></div>
|
<div class="control"><div class="select is-fullwidth"><select id="etype" name="entry_type"><option>a sender address</option><option>a sender domain</option><option>a client address or network</option></select></div></div></div>
|
||||||
</div>
|
</div>
|
||||||
<div class="column is-6">
|
<div class="column is-6">
|
||||||
<div class="field"><label class="label" for="value">Value</label>
|
<div class="field"><label class="label" for="value">Value</label>
|
||||||
@@ -37,11 +37,11 @@
|
|||||||
<div class="box">
|
<div class="box">
|
||||||
<div class="sp-box-head"><span class="sp-no">02</span><h2>Deny</h2><div class="sp-end">3 entries · wins over Allow</div></div>
|
<div class="sp-box-head"><span class="sp-no">02</span><h2>Deny</h2><div class="sp-end">3 entries · wins over Allow</div></div>
|
||||||
<div class="table-container"><table class="table is-fullwidth is-hoverable">
|
<div class="table-container"><table class="table is-fullwidth is-hoverable">
|
||||||
<thead><tr><th>Value</th><th>Note</th><th>Added</th><th></th></tr></thead>
|
<thead><tr><th>Value</th><th>Kind</th><th>Matched</th><th></th></tr></thead>
|
||||||
<tbody>
|
<tbody>
|
||||||
<tr><td class="sp-mono sp-small">@bulk.example <span class="tag is-light">domain</span></td><td class="sp-muted">newsletter nobody subscribed to</td><td class="sp-muted sp-nowrap sp-small">2026-09-02</td><td class="sp-actions"><a class="has-text-danger" href="#">Remove</a></td></tr>
|
<tr><td class="sp-mono sp-small">bulk.example</td><td><button type="button" class="tag is-light" popovertarget="deny-1">domain</button><div class="sp-sheet" id="deny-1" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-light">domain</span><h2>bulk.example</h2><button type="button" popovertarget="deny-1" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Note</th><td>newsletter nobody subscribed to</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-09-02</td></tr><tr><th>Matched</th><td>41 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-09-21 13:57</td></tr></tbody></table></div></div></div></div></td><td class="sp-nowrap">41</td><td class="sp-actions"><form method="post" action="#"><button type="submit" class="has-text-danger">Remove</button></form></td></tr>
|
||||||
<tr><td class="sp-mono sp-small">198.51.100.0/24 <span class="tag is-light">ip</span></td><td class="sp-muted">dictionary attack on lists.example.org</td><td class="sp-muted sp-nowrap sp-small">2026-08-19</td><td class="sp-actions"><a class="has-text-danger" href="#">Remove</a></td></tr>
|
<tr><td class="sp-mono sp-small">198.51.100.0/24</td><td><button type="button" class="tag is-light" popovertarget="deny-2">ip</button><div class="sp-sheet" id="deny-2" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-light">ip</span><h2>198.51.100.0/24</h2><button type="button" popovertarget="deny-2" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Note</th><td>dictionary attack on lists.example.org</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-08-19</td></tr><tr><th>Matched</th><td>1203 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-09-21 09:12</td></tr></tbody></table></div></div></div></div></td><td class="sp-nowrap">1203</td><td class="sp-actions"><form method="post" action="#"><button type="submit" class="has-text-danger">Remove</button></form></td></tr>
|
||||||
<tr><td class="sp-mono sp-small">win@lottery.example <span class="tag is-light">address</span></td><td class="sp-muted">—</td><td class="sp-muted sp-nowrap sp-small">2026-07-30</td><td class="sp-actions"><a class="has-text-danger" href="#">Remove</a></td></tr>
|
<tr><td class="sp-mono sp-small">win@lottery.example</td><td><button type="button" class="tag is-light" popovertarget="deny-3">address</button><div class="sp-sheet" id="deny-3" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-light">address</span><h2>win@lottery.example</h2><button type="button" popovertarget="deny-3" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Note</th><td>—</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-07-30</td></tr><tr><th>Matched</th><td>never in the last 90 days</td></tr></tbody></table></div></div></div></div></td><td class="sp-muted sp-nowrap">never</td><td class="sp-actions"><form method="post" action="#"><button type="submit" class="has-text-danger">Remove</button></form></td></tr>
|
||||||
</tbody>
|
</tbody>
|
||||||
</table></div>
|
</table></div>
|
||||||
</div>
|
</div>
|
||||||
@@ -50,10 +50,10 @@
|
|||||||
<div class="box">
|
<div class="box">
|
||||||
<div class="sp-box-head"><span class="sp-no">03</span><h2>Allow</h2><div class="sp-end">2 entries</div></div>
|
<div class="sp-box-head"><span class="sp-no">03</span><h2>Allow</h2><div class="sp-end">2 entries</div></div>
|
||||||
<div class="table-container"><table class="table is-fullwidth is-hoverable">
|
<div class="table-container"><table class="table is-fullwidth is-hoverable">
|
||||||
<thead><tr><th>Value</th><th>Note</th><th>Added</th><th></th></tr></thead>
|
<thead><tr><th>Value</th><th>Kind</th><th>Matched</th><th></th></tr></thead>
|
||||||
<tbody>
|
<tbody>
|
||||||
<tr><td class="sp-mono sp-small">@partner.example <span class="tag is-light">domain</span></td><td class="sp-muted">invoices were scored as spam</td><td class="sp-muted sp-nowrap sp-small">2026-09-11</td><td class="sp-actions"><a class="has-text-danger" href="#">Remove</a></td></tr>
|
<tr><td class="sp-mono sp-small">partner.example</td><td><button type="button" class="tag is-light" popovertarget="allow-1">domain</button><div class="sp-sheet" id="allow-1" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-light">domain</span><h2>partner.example</h2><button type="button" popovertarget="allow-1" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Note</th><td>invoices were scored as spam</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-09-11</td></tr><tr><th>Matched</th><td>7 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-09-19 16:40</td></tr></tbody></table></div></div></div></div></td><td class="sp-nowrap">7</td><td class="sp-actions"><form method="post" action="#"><button type="submit" class="has-text-danger">Remove</button></form></td></tr>
|
||||||
<tr><td class="sp-mono sp-small">noreply@github.example <span class="tag is-light">address</span></td><td class="sp-muted">security advisories</td><td class="sp-muted sp-nowrap sp-small">2026-06-04</td><td class="sp-actions"><a class="has-text-danger" href="#">Remove</a></td></tr>
|
<tr><td class="sp-mono sp-small">noreply@github.example</td><td><button type="button" class="tag is-light" popovertarget="allow-2">address</button><div class="sp-sheet" id="allow-2" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-light">address</span><h2>noreply@github.example</h2><button type="button" popovertarget="allow-2" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Note</th><td>security advisories</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-06-04</td></tr><tr><th>Matched</th><td>18 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-09-21 13:30</td></tr></tbody></table></div></div></div></div></td><td class="sp-nowrap">18</td><td class="sp-actions"><form method="post" action="#"><button type="submit" class="has-text-danger">Remove</button></form></td></tr>
|
||||||
</tbody>
|
</tbody>
|
||||||
</table></div>
|
</table></div>
|
||||||
</div>
|
</div>
|
||||||
@@ -64,10 +64,10 @@
|
|||||||
<div class="sp-box-head"><span class="sp-no">04</span><h2>Where the lists go</h2><div class="sp-end"><span class="tag is-success is-light">applied</span></div></div>
|
<div class="sp-box-head"><span class="sp-no">04</span><h2>Where the lists go</h2><div class="sp-end"><span class="tag is-success is-light">applied</span></div></div>
|
||||||
<div class="sp-box-body">
|
<div class="sp-box-body">
|
||||||
<dl class="sp-facts">
|
<dl class="sp-facts">
|
||||||
<div><dt>Applied by</dt><dd>rspamd, from map files</dd></div>
|
<div><dt>Applied by</dt><dd>rspamd, from map files it rereads within seconds of a change</dd></div>
|
||||||
<div><dt>Files</dt><dd class="sp-mono sp-small">/data/antispam/allow.map<br>/data/antispam/deny.map</dd></div>
|
<div><dt>Files</dt><dd><span class="sp-mono">/data/antispam/</span> — six, one for each list and kind of entry</dd></div>
|
||||||
<div><dt>Last written</dt><dd>2026-09-11 08:40 UTC</dd></div>
|
<div><dt>Last written</dt><dd>2026-09-11 08:40 UTC</dd></div>
|
||||||
<div><dt>When the filter is off</dt><dd class="sp-muted">The lists are kept and editable, but nothing reads them until <code>INBOUND_ANTISPAM_MILTER</code> is set.</dd></div>
|
<div><dt>When both lists match</dt><dd>Deny wins, whichever entries they are.</dd></div>
|
||||||
</dl>
|
</dl>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|||||||
@@ -3,7 +3,7 @@
|
|||||||
<div class="sp-grow">
|
<div class="sp-grow">
|
||||||
<p class="sp-kicker">Inbound</p>
|
<p class="sp-kicker">Inbound</p>
|
||||||
<h1 class="title is-3">Inbound log</h1>
|
<h1 class="title is-3">Inbound log</h1>
|
||||||
<p class="sp-lead">What port 25 decided about each message, one row per recipient. Accepts come from the inbound journal, rejects from <code>mail.log</code>; the filter's reasons appear when the spam filter is on.</p>
|
<p class="sp-lead">What port 25 decided about each message, one row per recipient. Accepted mail is recorded as it passes, refused mail is read from <code>mail.log</code>; the filter's verdict appears when the spam filter is on. Rows older than 90 days are deleted — change that in <a href="settings.html">Settings</a>.</p>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
@@ -17,17 +17,18 @@
|
|||||||
<input class="input" type="date" name="to" value="2026-09-21" aria-label="To date">
|
<input class="input" type="date" name="to" value="2026-09-21" aria-label="To date">
|
||||||
<button type="submit" class="button"><i class="ti ti-filter"></i>Filter</button>
|
<button type="submit" class="button"><i class="ti ti-filter"></i>Filter</button>
|
||||||
</form>
|
</form>
|
||||||
|
<a class="sp-help" href="help.html#inbound-log" title="What the decisions mean"><i class="ti ti-help-circle"></i></a>
|
||||||
</div>
|
</div>
|
||||||
<div class="table-container"><table class="table is-fullwidth is-hoverable">
|
<div class="table-container"><table class="table is-fullwidth is-hoverable">
|
||||||
<thead><tr><th>Time, UTC</th><th>From · client</th><th>To</th><th>Subject</th><th>Decision</th><th>Why</th><th></th></tr></thead>
|
<thead><tr><th>Time, UTC</th><th>From</th><th>To</th><th>Subject</th><th>Decision</th><th></th></tr></thead>
|
||||||
<tbody>
|
<tbody>
|
||||||
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 14:00:18</td><td>t.berg@<wbr>uni.example<br><span class="sp-mono sp-muted sp-small">192.0.2.77</span></td><td>dev@<wbr>lists.example.org</td><td class="sp-clip">Re: [dev] patch series v3</td><td><span class="tag is-success is-light">accept</span></td><td class="sp-muted sp-small">rspamd · score 0.4</td><td class="sp-actions"><a href="in-filter-lists.html">Deny sender</a></td></tr>
|
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 14:00:18</td><td>t.berg@<wbr>uni.example<br><span class="sp-mono sp-muted sp-small">192.0.2.77</span></td><td>dev@<wbr>lists.example.org</td><td class="sp-clip">Re: [dev] patch series v3</td><td><button type="button" class="tag is-success is-light" popovertarget="why-1">accept</button><div class="sp-sheet" id="why-1" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-success is-light">accept</span><h2>Why</h2><button type="button" popovertarget="why-1" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Verdict</th><td>no action, score 0.40 / 15.00</td></tr><tr><th>Subject</th><td>Re: [dev] patch series v3</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"><a href="in-filter-lists.html">Deny sender</a></td></tr>
|
||||||
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:59:02</td><td>invoice@<wbr>partner-billing.example<br><span class="sp-mono sp-muted sp-small">203.0.113.140</span></td><td>owner@<wbr>lists.example.org</td><td class="sp-clip">Outstanding invoice — action required</td><td><span class="tag is-primary is-light">quarantine</span></td><td class="sp-muted sp-small">rspamd · score 9.8 · PHISHING, FORGED_SENDER</td><td class="sp-actions"><a href="in-quarantine-message.html">Review</a></td></tr>
|
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:59:02</td><td>invoice@<wbr>partner-billing.example<br><span class="sp-mono sp-muted sp-small">203.0.113.140</span></td><td>owner@<wbr>lists.example.org</td><td class="sp-clip">Outstanding invoice — action required</td><td><button type="button" class="tag is-success is-light" popovertarget="why-2">accept</button><div class="sp-sheet" id="why-2" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-success is-light">accept</span><h2>Why</h2><button type="button" popovertarget="why-2" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Verdict</th><td>add header, score 9.80 / 15.00</td></tr><tr><th>Symbols</th><td class="sp-mono sp-small">PHISHING(5.00)<br>FORGED_SENDER(3.00)<br>R_SPF_FAIL(1.00)</td></tr><tr><th>Subject</th><td>Outstanding invoice — action required</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"><a href="in-filter-lists.html">Deny sender</a></td></tr>
|
||||||
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:57:41</td><td>promo@<wbr>bulk.example<br><span class="sp-mono sp-muted sp-small">198.51.100.90</span></td><td>sales@<wbr>lists.example.org</td><td class="sp-clip sp-muted">—</td><td><span class="tag is-danger is-light">reject</span></td><td class="sp-muted sp-small">postfix · 550 5.1.1 unknown recipient</td><td class="sp-actions"><a href="in-filter-lists.html">Deny sender</a></td></tr>
|
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:57:41</td><td>promo@<wbr>bulk.example<br><span class="sp-mono sp-muted sp-small">198.51.100.90</span></td><td>sales@<wbr>lists.example.org</td><td class="sp-clip sp-muted">—</td><td><button type="button" class="tag is-danger is-light" popovertarget="why-3">reject</button><div class="sp-sheet" id="why-3" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-danger is-light">reject</span><h2>Why</h2><button type="button" popovertarget="why-3" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>postfix</td></tr><tr><th>Reply</th><td class="sp-mono sp-small">550 5.1.1 <sales@lists.example.org>: Recipient address rejected: User unknown in relay recipient table</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"></td></tr>
|
||||||
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:51:30</td><td>billing@<wbr>partner.example<br><span class="sp-mono sp-muted sp-small">198.51.100.12</span></td><td>owner@<wbr>lists.example.org</td><td class="sp-clip">Invoice 2026-0917</td><td><span class="tag is-danger is-light">reject</span></td><td class="sp-muted sp-small">rspamd · score 16.2 · BAYES_SPAM</td><td class="sp-actions"><a href="in-filter-lists.html">Allow sender</a></td></tr>
|
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:51:30</td><td>billing@<wbr>partner.example<br><span class="sp-mono sp-muted sp-small">198.51.100.12</span></td><td>owner@<wbr>lists.example.org</td><td class="sp-clip sp-muted">—</td><td><button type="button" class="tag is-danger is-light" popovertarget="why-4">reject</button><div class="sp-sheet" id="why-4" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-danger is-light">reject</span><h2>Why</h2><button type="button" popovertarget="why-4" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Reply</th><td class="sp-mono sp-small">5.7.1 Spam message rejected</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"><a href="in-filter-lists.html">Allow sender</a></td></tr>
|
||||||
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:44:02</td><td>hr@<wbr>partner.example<br><span class="sp-mono sp-muted sp-small">192.0.2.31</span></td><td>jobs@<wbr>acme.io</td><td class="sp-clip">Application: backend engineer</td><td><span class="tag is-warning is-light">tempfail</span></td><td class="sp-muted sp-small">selfpost · upstream 10.0.4.12 timed out</td><td class="sp-actions"></td></tr>
|
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:44:02</td><td>hr@<wbr>partner.example<br><span class="sp-mono sp-muted sp-small">192.0.2.31</span></td><td>jobs@<wbr>acme.io</td><td class="sp-clip sp-muted">—</td><td><button type="button" class="tag is-warning is-light" popovertarget="why-5">tempfail</button><div class="sp-sheet" id="why-5" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-warning is-light">tempfail</span><h2>Why</h2><button type="button" popovertarget="why-5" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Reply</th><td class="sp-mono sp-small">4.7.1 Try again later</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"><a href="in-filter-lists.html">Allow sender</a></td></tr>
|
||||||
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:30:55</td><td>noreply@<wbr>github.example<br><span class="sp-mono sp-muted sp-small">192.0.2.200</span></td><td>security@<wbr>lists.example.org</td><td class="sp-clip">[advisory] new report</td><td><span class="tag is-success is-light">accept</span></td><td class="sp-muted sp-small">rspamd · allow list</td><td class="sp-actions"></td></tr>
|
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">09-21 13:30:55</td><td>noreply@<wbr>github.example<br><span class="sp-mono sp-muted sp-small">192.0.2.200</span></td><td>security@<wbr>lists.example.org</td><td class="sp-clip">[advisory] new report</td><td><button type="button" class="tag is-success is-light" popovertarget="why-6">accept</button><div class="sp-sheet" id="why-6" popover><div class="sp-sheet-in"><div class="sp-sheet-head"><span class="tag is-success is-light">accept</span><h2>Why</h2><button type="button" popovertarget="why-6" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody><tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Verdict</th><td>no action, score -98.10 / 15.00</td></tr><tr><th>Symbols</th><td class="sp-mono sp-small">SELFPOST_ALLOW_FROM(-100.00)<br>R_DKIM_ALLOW(-0.20)</td></tr><tr><th>Subject</th><td>[advisory] new report</td></tr></tbody></table></div></div></div></div></td><td class="sp-actions"><a href="in-filter-lists.html">Deny sender</a></td></tr>
|
||||||
</tbody>
|
</tbody>
|
||||||
</table></div>
|
</table></div>
|
||||||
<div class="sp-box-foot"><span>Page 1 of 38</span><a href="#">Older →</a><span class="sp-end">Allow / Deny sender opens <a href="in-filter-lists.html">Filter lists</a> with the entry filled in · global administrator only</span></div>
|
<div class="sp-box-foot"><span>Page 1 of 38</span><a href="#">Older →</a><span class="sp-end">A decision opens its reason · Allow / Deny sender opens <a href="in-filter-lists.html">Filter lists</a> with the entry filled in</span></div>
|
||||||
</div>
|
</div>
|
||||||
|
|||||||
@@ -16,6 +16,9 @@
|
|||||||
<div class="field"><label class="label" for="ret">Keep outbound log rows, days</label>
|
<div class="field"><label class="label" for="ret">Keep outbound log rows, days</label>
|
||||||
<div class="control"><input class="input" id="ret" name="send_log_retention_days" type="number" min="7" max="365" value="30" required></div>
|
<div class="control"><input class="input" id="ret" name="send_log_retention_days" type="number" min="7" max="365" value="30" required></div>
|
||||||
<p class="help">The main driver of <code>/data</code> growth. Does not affect <code>mail.log</code> rotation (fourteen daily files).</p></div>
|
<p class="help">The main driver of <code>/data</code> growth. Does not affect <code>mail.log</code> rotation (fourteen daily files).</p></div>
|
||||||
|
<div class="field"><label class="label" for="inret">Keep inbound log rows, days</label>
|
||||||
|
<div class="control"><input class="input" id="inret" name="inbound_spam_log_retention_days" type="number" min="7" max="365" value="90" required></div>
|
||||||
|
<p class="help">The inbound log also never holds more than 10 000 rows: refused mail is logged at the rate strangers knock on port 25.</p></div>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|||||||
@@ -270,3 +270,27 @@ code { background: var(--sp-well); color: var(--sp-ink); border-radius: 3px; pad
|
|||||||
.sp-record-head small { margin-left: 0; text-align: left; flex-basis: 100%; }
|
.sp-record-head small { margin-left: 0; text-align: left; flex-basis: 100%; }
|
||||||
.sp-postmark { display: none; }
|
.sp-postmark { display: none; }
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/* sheet: what does not fit a table row, opened over the page from the row's
|
||||||
|
tag. It is the browser's own popover (the tag is a button with
|
||||||
|
popovertarget), so it needs no script: Esc, a click outside and the close
|
||||||
|
button all shut it, and focus returns to the tag. It rises from the bottom
|
||||||
|
edge across the whole window - the one component that is not laid out by the
|
||||||
|
page, hence the widths - with its content, a table of name and value, in a
|
||||||
|
column of reading width in the middle. */
|
||||||
|
.sp-sheet {
|
||||||
|
inset: auto 0 0 0; width: auto; margin: 0; padding: 0; border: 0; border-top: 3px solid var(--sp-brick);
|
||||||
|
max-height: 75vh; overflow: auto; background: #fff; color: inherit; box-shadow: 0 -12px 40px rgba(40, 20, 15, .22);
|
||||||
|
}
|
||||||
|
.sp-sheet-in { max-width: 44rem; margin: 0 auto; padding: 0 1.25rem; }
|
||||||
|
.sp-sheet::backdrop { background: rgba(40, 20, 15, .38); }
|
||||||
|
.sp-sheet-head { display: flex; align-items: center; gap: .7rem; padding: 1rem 0 .8rem; border-bottom: 1px solid var(--sp-rule); }
|
||||||
|
.sp-sheet-head h2 { font-size: 1.15rem; font-weight: 600; margin-right: auto; }
|
||||||
|
.sp-sheet-head button { border: 0; background: none; cursor: pointer; font-size: 1.3rem; color: var(--sp-muted); }
|
||||||
|
.sp-sheet-body { padding: .4rem 0 1.4rem; }
|
||||||
|
/* The sheet's content is a table of name and value: the name is a label, set like the dt of facts. */
|
||||||
|
.sp-sheet .table th { font-family: var(--sp-mono); font-size: var(--sp-fs-xs); font-weight: 500; letter-spacing: .08em; text-transform: uppercase; color: var(--sp-muted); white-space: nowrap; vertical-align: top; width: 1%; padding-right: 2rem; }
|
||||||
|
.sp-sheet .table td { overflow-wrap: anywhere; }
|
||||||
|
/* A tag that opens a sheet is a button: it keeps the tag's look and says it can
|
||||||
|
be pressed with the dotted underline links to more detail carry. */
|
||||||
|
button.tag { border: 0; cursor: pointer; text-decoration: underline dotted; text-underline-offset: 3px; }
|
||||||
|
|||||||
+3
-2
@@ -107,7 +107,7 @@ than the default «meaningful step» rule in [§ Commits and release build](#com
|
|||||||
2. Git tag `vX.Y.Z` and publish the GitHub Release only on explicit request (see
|
2. Git tag `vX.Y.Z` and publish the GitHub Release only on explicit request (see
|
||||||
[§ Release image](#release-image)).
|
[§ Release image](#release-image)).
|
||||||
|
|
||||||
**Planned version cuts** (from pin `2.0.0`; adjust if semver changes mid-track):
|
**Planned version cuts** (from pin `2.1.0`; adjust if semver changes mid-track):
|
||||||
|
|
||||||
| Stage | ID | Cut |
|
| Stage | ID | Cut |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
@@ -124,7 +124,8 @@ than the default «meaningful step» rule in [§ Commits and release build](#com
|
|||||||
| 11 | panel-redesign plan: component kit, stages, reorder (docs only, owner-requested cut) | `1.9.4` (PATCH) |
|
| 11 | panel-redesign plan: component kit, stages, reorder (docs only, owner-requested cut) | `1.9.4` (PATCH) |
|
||||||
| 12 | GHCR release fix: keep per-arch images, sweep orphaned versions (CI only) | `1.9.5` (PATCH) |
|
| 12 | GHCR release fix: keep per-arch images, sweep orphaned versions (CI only) | `1.9.5` (PATCH) |
|
||||||
| 13 | panel-redesign | `2.0.0` (MAJOR, no compatibility) |
|
| 13 | panel-redesign | `2.0.0` (MAJOR, no compatibility) |
|
||||||
| 14 | inbound-antispam-panel | `2.1.0` (planned; was `1.10.0`, reordered 2026-10-09) |
|
| 14 | DMARC ingest drops and bounds, full backup from the panel, no queue in a backup | `2.0.1` (PATCH) |
|
||||||
|
| 15 | inbound-antispam-panel | `2.1.0` (was `1.10.0`, reordered 2026-10-09) |
|
||||||
|
|
||||||
Docs-only prep (checklists in plans, no product code) uses the same per-step
|
Docs-only prep (checklists in plans, no product code) uses the same per-step
|
||||||
commit rule but **no** version cut until the next product stage ships.
|
commit rule but **no** version cut until the next product stage ships.
|
||||||
|
|||||||
+96
-8
@@ -183,7 +183,7 @@ cat ./data/setup-token
|
|||||||
#### Fixed image tag
|
#### Fixed image tag
|
||||||
|
|
||||||
`deploy/docker-compose.yml` pins an explicit version (`ghcr.io/mixeme/selfpost:X.Y.Z`),
|
`deploy/docker-compose.yml` pins an explicit version (`ghcr.io/mixeme/selfpost:X.Y.Z`),
|
||||||
deliberately never `:latest`. The current pin is `2.0.0`. Intermediate
|
deliberately never `:latest`. The current pin is `2.1.0`. Intermediate
|
||||||
CHANGELOG sections (`0.2.0`…`0.6.0`) record development cuts from before that
|
CHANGELOG sections (`0.2.0`…`0.6.0`) record development cuts from before that
|
||||||
image was published. Pinning matters because of the backup version check (see
|
image was published. Pinning matters because of the backup version check (see
|
||||||
[Full backup and restore](#full-backup-and-restore)): the panel binary's
|
[Full backup and restore](#full-backup-and-restore)): the panel binary's
|
||||||
@@ -209,6 +209,8 @@ expected to set; defaults match the code exactly.
|
|||||||
| `DMARC_MESSAGE_SIZE_LIMIT` | Maximum report message size in bytes when DMARC ingest is on. | `5242880` (5 MiB) | `.env` |
|
| `DMARC_MESSAGE_SIZE_LIMIT` | Maximum report message size in bytes when DMARC ingest is on. | `5242880` (5 MiB) | `.env` |
|
||||||
| `INBOUND_ANTISPAM_MILTER` | Optional milter on the inbound relay listener only (requires `INBOUND_RELAY_ENABLE=true`; not 465/587 or DMARC-only port 25). Empty = off. Format `inet:host:port` or `unix:/path`. Example with [deploy/antispam/docker-compose.antispam.yml](../deploy/antispam/docker-compose.antispam.yml): `inet:antispam:11332`. The panel reads it to report on the *Spam filter* box of the inbound pages and to check that the filter answers (Server › Health). | *(empty)* | `.env` |
|
| `INBOUND_ANTISPAM_MILTER` | Optional milter on the inbound relay listener only (requires `INBOUND_RELAY_ENABLE=true`; not 465/587 or DMARC-only port 25). Empty = off. Format `inet:host:port` or `unix:/path`. Example with [deploy/antispam/docker-compose.antispam.yml](../deploy/antispam/docker-compose.antispam.yml): `inet:antispam:11332`. The panel reads it to report on the *Spam filter* box of the inbound pages and to check that the filter answers (Server › Health). | *(empty)* | `.env` |
|
||||||
| `INBOUND_ANTISPAM_MILTER_ACTION` | What Postfix does if that milter is down: `accept` (fail-open) or `tempfail` (defer). The *Spam filter* box states which. | `accept` | `.env` |
|
| `INBOUND_ANTISPAM_MILTER_ACTION` | What Postfix does if that milter is down: `accept` (fail-open) or `tempfail` (defer). The *Spam filter* box states which. | `accept` | `.env` |
|
||||||
|
| `INBOUND_SPAM_LOG_RETENTION_DAYS` | How long the inbound journal keeps a row, 7-365 days; it also never holds more than 10 000 rows. Seeds the setting on the first start with the inbound relay on. | `90` | `.env` |
|
||||||
|
| `INBOUND_JOURNAL_MILTER_SOCKET` | Unix socket of the panel's inbound journal milter, the last milter on port 25 while the inbound relay is on. It records accepted inbound mail and never refuses any; there is no reason to change it. | `/run/selfpost/inbound-journal.sock` | compose |
|
||||||
| `INBOUND_RATE_LIMIT_MESSAGES_PER_IP` | Coarse per-client-IP cap on inbound smtpd (`smtpd_client_message_rate_limit`). Uses the same window as `RATE_LIMIT_WINDOW_SECONDS`. | `20` | `.env` |
|
| `INBOUND_RATE_LIMIT_MESSAGES_PER_IP` | Coarse per-client-IP cap on inbound smtpd (`smtpd_client_message_rate_limit`). Uses the same window as `RATE_LIMIT_WINDOW_SECONDS`. | `20` | `.env` |
|
||||||
| `INBOUND_MESSAGE_SIZE_LIMIT` | Maximum message size in bytes on inbound smtpd (`message_size_limit`). When both inbound relay and DMARC ingest are on, the shared listener uses the **greater** of this and `DMARC_MESSAGE_SIZE_LIMIT`. | `26214400` (25 MiB) | `.env` |
|
| `INBOUND_MESSAGE_SIZE_LIMIT` | Maximum message size in bytes on inbound smtpd (`message_size_limit`). When both inbound relay and DMARC ingest are on, the shared listener uses the **greater** of this and `DMARC_MESSAGE_SIZE_LIMIT`. | `26214400` (25 MiB) | `.env` |
|
||||||
| `RATE_LIMIT_MESSAGES_PER_IP` | Level-1 backstop: maximum messages one client IP may submit per window (Postfix `smtpd_client_message_rate_limit`). See [Rate limiting — level 1](#rate-limiting--level-1-ip-backstop). | `100` | `.env` |
|
| `RATE_LIMIT_MESSAGES_PER_IP` | Level-1 backstop: maximum messages one client IP may submit per window (Postfix `smtpd_client_message_rate_limit`). See [Rate limiting — level 1](#rate-limiting--level-1-ip-backstop). | `100` | `.env` |
|
||||||
@@ -542,10 +544,14 @@ journal-milter (level 2) is down. There is no per-IP bypass.
|
|||||||
### Full backup and restore
|
### Full backup and restore
|
||||||
|
|
||||||
**Full backup** is a self-contained project archive: `data/` (SQLite, all
|
**Full backup** is a self-contained project archive: `data/` (SQLite, all
|
||||||
domains' DKIM keys, all applications' SASL credentials, the Postfix queue,
|
domains' DKIM keys, all applications' SASL credentials, `manifest.json` with
|
||||||
`manifest.json` with the version that created it), plus `docker-compose.yml`,
|
the version that created it), plus `docker-compose.yml`, `.env`, and `certs/`
|
||||||
`.env`, and `certs/` from the operator directory next to `./data`. Delivery
|
from the operator directory next to `./data`. Delivery logs under `data/log/`
|
||||||
logs under `data/log/` are excluded. The base compose file mounts the project
|
are excluded, and so is the Postfix queue: the archive is taken while the
|
||||||
|
instance runs, and a copy of a queue Postfix is working on can hold a message
|
||||||
|
twice or not at all. Mail still waiting for delivery is therefore not carried
|
||||||
|
over, and a restored instance starts with an empty queue; to keep the queue,
|
||||||
|
stop the instance and use `tar` (methods 3 and 4 below). The base compose file mounts the project
|
||||||
directory read-only at `/selfpost-deploy` so the panel and CLI can read those
|
directory read-only at `/selfpost-deploy` so the panel and CLI can read those
|
||||||
deploy files — without that mount, *Full backup* refuses with an error.
|
deploy files — without that mount, *Full backup* refuses with an error.
|
||||||
|
|
||||||
@@ -556,12 +562,25 @@ instance keeps running, what goes into the archive, and what happens on restore.
|
|||||||
|
|
||||||
**1. Panel — while the container is running (no downtime).** *Server › Backup*
|
**1. Panel — while the container is running (no downtime).** *Server › Backup*
|
||||||
→ *Full backup*. *Encrypt with a password* is ticked; untick it for a plain
|
→ *Full backup*. *Encrypt with a password* is ticked; untick it for a plain
|
||||||
archive. Same archive as the CLI below.
|
archive. Same archive as the CLI below. The panel runs as its own user, not as
|
||||||
|
root, so **the private key in `certs/` has to be readable by it**: give the
|
||||||
|
file to the container's `selfpost` group, which has the fixed id 998 and holds
|
||||||
|
both the panel and Postfix —
|
||||||
|
|
||||||
|
```sh
|
||||||
|
chgrp 998 certs/privkey.pem && chmod 640 certs/privkey.pem
|
||||||
|
```
|
||||||
|
|
||||||
|
— and have whatever renews the certificate do the same (a certbot deploy hook,
|
||||||
|
for one). A key readable by root alone is enough for Postfix, but then the
|
||||||
|
panel refuses to make a backup and names the file: it does not hand over an
|
||||||
|
archive without the key.
|
||||||
|
|
||||||
**2. `selfpost-backup` CLI — while the container is running (no downtime).**
|
**2. `selfpost-backup` CLI — while the container is running (no downtime).**
|
||||||
The panel button and this command call the same code path: a consistent SQLite
|
The panel button and this command call the same code path: a consistent SQLite
|
||||||
snapshot via `VACUUM INTO`, then a gzip tar of `data/` (minus diagnostics) plus
|
snapshot via `VACUUM INTO`, then a gzip tar of `data/` (minus diagnostics) plus
|
||||||
deploy files.
|
deploy files. `docker exec` runs the command as root, so it reads the private
|
||||||
|
key whatever its permissions.
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
docker exec <container> selfpost-backup > selfpost-backup.tar.gz
|
docker exec <container> selfpost-backup > selfpost-backup.tar.gz
|
||||||
@@ -616,6 +635,8 @@ methods avoid it by stopping first.
|
|||||||
| Self-contained archive | Yes | Yes | No — deploy files separate |
|
| Self-contained archive | Yes | Yes | No — deploy files separate |
|
||||||
| SQLite consistency | `VACUUM INTO` snapshot | Safe when stopped | Safe when stopped |
|
| SQLite consistency | `VACUUM INTO` snapshot | Safe when stopped | Safe when stopped |
|
||||||
| `data/log/` | Excluded | Included (optional exclude) | Included (optional exclude) |
|
| `data/log/` | Excluded | Included (optional exclude) | Included (optional exclude) |
|
||||||
|
| Postfix queue | No | Yes | Yes |
|
||||||
|
| `certs/` private key | Yes (the panel needs it readable by group 998) | Yes | No — deploy files separate |
|
||||||
| `data/manifest.json` | Written (version stamp) | Absent | Absent |
|
| `data/manifest.json` | Written (version stamp) | Absent | Absent |
|
||||||
| Restore version check | Yes — must match image tag | No | No |
|
| Restore version check | Yes — must match image tag | No | No |
|
||||||
| Post-restore *Resync* | Yes, on first boot | No | No |
|
| Post-restore *Resync* | Yes, on first boot | No | No |
|
||||||
@@ -691,6 +712,15 @@ needed) for the new hostname or proxy, then `docker compose up -d`. The archive
|
|||||||
carries `certs/` from the old host — re-issue certificates when the hostname or
|
carries `certs/` from the old host — re-issue certificates when the hostname or
|
||||||
IP changes. Set up the reverse-proxy vhost separately (not in the backup).
|
IP changes. Set up the reverse-proxy vhost separately (not in the backup).
|
||||||
|
|
||||||
|
The inbound log and the filter lists are in the database and so in the
|
||||||
|
backup, and so are the list files under `data/antispam/`; the panel writes
|
||||||
|
those files again from the database on the first start after a restore. What
|
||||||
|
the backup does not hold is the spam filter's own side: the `antispam/`
|
||||||
|
directory with the compose fragment and the rspamd configuration. Like the
|
||||||
|
reverse-proxy fragments it is part of the installation, not of its state -
|
||||||
|
copy it from [deploy/antispam/](../deploy/antispam/) again, or the restored
|
||||||
|
instance runs with no filter.
|
||||||
|
|
||||||
**Restoring an encrypted (`.spbk`) backup** needs a running container to
|
**Restoring an encrypted (`.spbk`) backup** needs a running container to
|
||||||
decrypt it first — any container with the `selfpost-backup` CLI works; decryption
|
decrypt it first — any container with the `selfpost-backup` CLI works; decryption
|
||||||
does not read `/data` and performs no version check. Start one normally
|
does not read `/data` and performs no version check. Start one normally
|
||||||
@@ -822,6 +852,58 @@ only sees SelfPost. Default action is fail-open (`accept`) so a down sidecar
|
|||||||
does not block backup-MX; set `INBOUND_ANTISPAM_MILTER_ACTION=tempfail` to
|
does not block backup-MX; set `INBOUND_ANTISPAM_MILTER_ACTION=tempfail` to
|
||||||
defer instead.
|
defer instead.
|
||||||
|
|
||||||
|
The fragment mounts one file into the sidecar,
|
||||||
|
[deploy/antispam/rspamd/milter_headers.conf](../deploy/antispam/rspamd/milter_headers.conf):
|
||||||
|
it makes rspamd write its verdict - the action and the score with its symbols
|
||||||
|
- into two headers of every message it lets through. SelfPost records them
|
||||||
|
with the message, and the upstream receives them with the mail. Copy the
|
||||||
|
`antispam/rspamd/` directory next to the fragment; without the file accepted
|
||||||
|
mail is still recorded, with no verdict.
|
||||||
|
|
||||||
|
The same file makes rspamd **sign** each message, in a third header,
|
||||||
|
`X-SelfPost-Filter`. The filter is attached so that mail still flows when it
|
||||||
|
is down, and then a sender could write a verdict of his own into those
|
||||||
|
headers; SelfPost therefore records a verdict only next to a signature it can
|
||||||
|
recompute. The key is `data/antispam/filter.key`: SelfPost creates it on its
|
||||||
|
first start with the inbound relay on, and the sidecar reads it from the
|
||||||
|
directory it already mounts - there is nothing to configure. If Inbound › Log
|
||||||
|
shows *selfpost* for mail the filter did judge, the sidecar is running without
|
||||||
|
this file: copy `antispam/rspamd/` again and restart the sidecar. Nothing of
|
||||||
|
rspamd's needs clearing, then or after a change of the lists or of the key: it
|
||||||
|
rereads the list files by itself and reads the key anew for every message. A
|
||||||
|
restart is needed only for a change of its own configuration - the files of
|
||||||
|
`antispam/rspamd/`.
|
||||||
|
|
||||||
|
The same directory holds
|
||||||
|
[multimap.conf](../deploy/antispam/rspamd/multimap.conf), the rules for the
|
||||||
|
**allow and deny lists** kept in the panel. SelfPost does not apply the lists
|
||||||
|
itself: it writes them as six files under `data/antispam/` (`allow_ip.map`,
|
||||||
|
`allow_from.map`, `allow_domain.map` and the three `deny_` ones), the fragment
|
||||||
|
mounts that directory into the sidecar, and rspamd rereads a file within
|
||||||
|
seconds of a change - no restart. An entry is a sender address, a sender
|
||||||
|
domain (that domain, not its subdomains) or a client address or network. A
|
||||||
|
denied sender is refused outright. An allowed one is given a weight that no
|
||||||
|
spam score outweighs, so it is accepted - unless a deny entry matches the same
|
||||||
|
message: **deny wins**, whichever entries they are. **A sender address or
|
||||||
|
domain on the allow list is not proof of anything**: it is the envelope
|
||||||
|
sender, which the sending server states and nobody checks, so whoever knows an
|
||||||
|
allowed address can send as it. Allow a client address or network where you
|
||||||
|
can. The files are readable by
|
||||||
|
anyone on the host; they hold addresses, not secrets.
|
||||||
|
|
||||||
|
In the panel the lists are **Inbound › Filter lists** (global role), and what
|
||||||
|
port 25 decided about each message is **Inbound › Log**: accepted mail with the
|
||||||
|
filter's verdict, refused mail with the reply it was given. A row of the log
|
||||||
|
leads to the lists with its sender filled in. Beside each entry of a list is
|
||||||
|
how often it decided a message, counted from the log - so over the days the
|
||||||
|
log keeps - and *never* marks an entry that may have outlived its use. The
|
||||||
|
count depends on two things in
|
||||||
|
[multimap.conf](../deploy/antispam/rspamd/multimap.conf): the message of the
|
||||||
|
deny rules and the names of the allow symbols; change them and every entry
|
||||||
|
reads *never*. A user with the *domain* role
|
||||||
|
sees the log of the inbound domains assigned to them, and not the lists. How
|
||||||
|
long the log keeps a row is set under **Server › Settings**.
|
||||||
|
|
||||||
The panel only reports this setting; it does not change it. Every inbound
|
The panel only reports this setting; it does not change it. Every inbound
|
||||||
domain page has a **Spam filter** box — on or off, what happens to mail while
|
domain page has a **Spam filter** box — on or off, what happens to mail while
|
||||||
the filter is down, and that the filter belongs to the server, not to the
|
the filter is down, and that the filter belongs to the server, not to the
|
||||||
@@ -891,7 +973,13 @@ sees the *Ingest* box — whether ingest works, the last report received, report
|
|||||||
kept this week and parse failures. Open a domain's roll-up from the list or
|
kept this week and parse failures. Open a domain's roll-up from the list or
|
||||||
from *DMARC reports* in the domain page's menu. A user with the *domain* role
|
from *DMARC reports* in the domain page's menu. A user with the *domain* role
|
||||||
sees the reports of the domains assigned to them, without the *Ingest* box.
|
sees the reports of the domains assigned to them, without the *Ingest* box.
|
||||||
Summaries are pruned (500 kept, 90 days max).
|
Summaries are pruned (500 kept for each domain, 90 days max).
|
||||||
|
|
||||||
|
A message that is not a report the relay keeps — unreadable as an aggregate
|
||||||
|
report, about a domain not sent from here, or delivered to another domain's
|
||||||
|
report address — is dropped without a retry and without a bounce. It adds one
|
||||||
|
to *parse failures*, and the reason is on the message's delivery line in the mail log
|
||||||
|
(*Server › System log*, `dropped: …`).
|
||||||
|
|
||||||
**Not an open relay.** The inbound smtpd offers no SASL. With DMARC ingest
|
**Not an open relay.** The inbound smtpd offers no SASL. With DMARC ingest
|
||||||
alone, `check_recipient_access` permits only configured report addresses;
|
alone, `check_recipient_access` permits only configured report addresses;
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# Plan: inbound-antispam-panel
|
# Plan: inbound-antispam-panel
|
||||||
|
|
||||||
**Status:** agreed
|
**Status:** shipped in `2.1.0` (2026-10-12)
|
||||||
**Date:** 2026-08-19
|
**Date:** 2026-08-19
|
||||||
**Version:** `2.1.0` MINOR (opt-in; no change to outbound-only or inbound-without-filter paths). Was `1.10.0`; on 2026-10-09 the owner put the panel redesign first, so this ships on the 2.0 baseline and its screens are built from the redesign's component kit ([panel-redesign.md](panel-redesign.md) § Component kit).
|
**Version:** `2.1.0` MINOR (opt-in; no change to outbound-only or inbound-without-filter paths). Was `1.10.0`; on 2026-10-09 the owner put the panel redesign first, so this ships on the 2.0 baseline and its screens are built from the redesign's component kit ([panel-redesign.md](panel-redesign.md) § Component kit).
|
||||||
|
|
||||||
@@ -57,8 +57,14 @@ smtpd_milters = { antispam, inbound-journal }
|
|||||||
```
|
```
|
||||||
|
|
||||||
Antispam runs first; inbound-journal runs at end-of-message on the **accept**
|
Antispam runs first; inbound-journal runs at end-of-message on the **accept**
|
||||||
path and reads rspamd-added headers (`X-Rspamd-Action`, `X-Rspamd-Score`, symbol
|
path and reads rspamd-added headers (`X-Rspamd-Action`, and `X-Spamd-Result`
|
||||||
summary when present).
|
for the score and symbols - rspamd has no `X-Rspamd-Score`; the sidecar adds
|
||||||
|
both through `deploy/antispam/rspamd/milter_headers.conf`). As built they are
|
||||||
|
read only next to rspamd's signature of the message, `X-SelfPost-Filter`
|
||||||
|
(owner, 2026-10-12: "А rspamd не может дописывать какой-то заголовок, по
|
||||||
|
которому мы будем знать, что это заголовки нашего спам фильтра?") - an HMAC
|
||||||
|
of the queue id, since a fixed mark could be written by a sender as easily as
|
||||||
|
the verdict itself.
|
||||||
|
|
||||||
**Antispam rejects** do not reach end-of-message on the journal milter. v1 also
|
**Antispam rejects** do not reach end-of-message on the journal milter. v1 also
|
||||||
tails `mail.log` for inbound `postfix/smtpd` **`milter-reject`** / **`reject`**
|
tails `mail.log` for inbound `postfix/smtpd` **`milter-reject`** / **`reject`**
|
||||||
@@ -73,7 +79,9 @@ antispam hook). Without an external filter:
|
|||||||
|
|
||||||
- **accept** rows — message relayed (engine `selfpost`, detail empty);
|
- **accept** rows — message relayed (engine `selfpost`, detail empty);
|
||||||
- **reject** / **tempfail** rows — from the mail.log tailer (Postfix policy,
|
- **reject** / **tempfail** rows — from the mail.log tailer (Postfix policy,
|
||||||
size limit, unknown recipient, etc.).
|
unknown recipient, a sender domain that does not exist. As built, a refusal
|
||||||
|
whose log line names no recipient - the size limit answered at MAIL FROM -
|
||||||
|
and the per-client rate limit, which Postfix logs as a warning, make no row).
|
||||||
|
|
||||||
Panel copy explains that rspamd symbols appear only when the antispam milter is
|
Panel copy explains that rspamd symbols appear only when the antispam milter is
|
||||||
configured.
|
configured.
|
||||||
@@ -83,6 +91,25 @@ configured.
|
|||||||
**Global administrator only** for journal and lists — same as inbound relay. No
|
**Global administrator only** for journal and lists — same as inbound relay. No
|
||||||
domain-admin access in v1.
|
domain-admin access in v1.
|
||||||
|
|
||||||
|
*As built (2026-10-11):* the lists are the global role's; the **log follows
|
||||||
|
the delegation of Inbound** that 2.0 introduced after this was written - a
|
||||||
|
user with the `domain` role sees the rows of the inbound domains assigned to
|
||||||
|
them and nothing else, without the *Allow / Deny sender* links. That is what
|
||||||
|
the accepted `in-log` mockup draws (no "global role" in its head). The owner
|
||||||
|
was told and can turn it back into a global-only page.
|
||||||
|
|
||||||
|
**Inbound › Log accepted by the owner on 2026-10-12** ("Этот экран
|
||||||
|
утверждаем"), from screenshots of the page built from its fixture, after his
|
||||||
|
changes to it: the sender column is headed *From*; *Why* is not a column but
|
||||||
|
a sheet over the page, opened by the decision tag, holding a centred table of
|
||||||
|
who decided, the verdict and symbols or the reply, and the subject whole
|
||||||
|
(kept on the sheet at his word: the table clips it); a message Postfix itself
|
||||||
|
refused offers no *Deny sender*. Inbound › Filter lists was redrawn at his word
|
||||||
|
the same day - kind in a column of its own, the note on the entry's sheet,
|
||||||
|
matches counted (§ Filter list usage) - and **accepted by him on 2026-10-12**
|
||||||
|
("Да"), after a deny entry he added in the panel refused a test message with
|
||||||
|
the list's reply and the journal recorded it.
|
||||||
|
|
||||||
### Retention
|
### Retention
|
||||||
|
|
||||||
Mirror send-log retention:
|
Mirror send-log retention:
|
||||||
@@ -101,13 +128,53 @@ Mirror send-log retention:
|
|||||||
| Entry types | sender address, sender domain (`@domain` suffix), client IP / CIDR |
|
| Entry types | sender address, sender domain (`@domain` suffix), client IP / CIDR |
|
||||||
| Precedence | **deny overrides allow** |
|
| Precedence | **deny overrides allow** |
|
||||||
| Scope | instance-wide (not per inbound domain in v1) |
|
| Scope | instance-wide (not per inbound domain in v1) |
|
||||||
| Where applied | **rspamd map files** under `/data/antispam/` (allow.map, deny.map); documented compose fragment mounts them into the sidecar |
|
| Where applied | **rspamd map files** under `/data/antispam/`, one per list and entry type (`allow_ip.map`, `allow_from.map`, `allow_domain.map`, `deny_…`) - an IP map and a sender map are different map types to rspamd; documented compose fragment mounts them into the sidecar. Deny is a prefilter reject, allow a weight of -100 (as built: two prefilters do not give "deny overrides allow") |
|
||||||
| SelfPost pre-check | **out of v1** — lists live in rspamd only |
|
| SelfPost pre-check | **out of v1** — lists live in rspamd only |
|
||||||
| Edit access | global administrator only |
|
| Edit access | global administrator only |
|
||||||
| When inactive | lists editable in the panel but UI warns that sync applies only when `INBOUND_ANTISPAM_MILTER` is set; `postfix reload` / rspamd reload documented in guide |
|
| When inactive | lists editable in the panel but UI warns that sync applies only when `INBOUND_ANTISPAM_MILTER` is set; `postfix reload` / rspamd reload documented in guide |
|
||||||
|
|
||||||
List change → atomic map rewrite (same pattern as Postfix maps) → rspamd reload
|
List change → atomic map rewrite (same pattern as Postfix maps). No reload:
|
||||||
via documented operator step or sidecar `SIGHUP` in the compose fragment.
|
rspamd rereads a changed map file by itself (about ten seconds on 3.11).
|
||||||
|
|
||||||
|
### Filter list usage (owner, 2026-10-12)
|
||||||
|
|
||||||
|
Each entry shows how often it decided a message (*Matched*, or *never*) and,
|
||||||
|
on its sheet, when it last did - so that entries nothing matches any more can
|
||||||
|
be removed. **The count is read off the journal on every view of the page;
|
||||||
|
nothing is stored for it.** A journal row counts for an entry when the list
|
||||||
|
decided the message - a rejected row whose reply is the deny rule's message,
|
||||||
|
an accepted row whose verdict carries the allow rule's symbol
|
||||||
|
(`SELFPOST_ALLOW_IP` / `_FROM` / `_DOMAIN`, which the journal milter always
|
||||||
|
keeps) - and the entry fits the row's sender or client address the way rspamd
|
||||||
|
matches it (`inbound.Lists.Usage`).
|
||||||
|
|
||||||
|
What follows from counting this way, and is accepted:
|
||||||
|
|
||||||
|
- the count reaches back as far as the journal does: its retention (90 days by
|
||||||
|
default), which is what the sheet says;
|
||||||
|
- a row counts for every fitting entry of the deciding list, since the journal
|
||||||
|
does not say which of two the filter met first;
|
||||||
|
- a message refused at end-of-message is one row whatever its recipient count,
|
||||||
|
so a count is of messages, roughly.
|
||||||
|
|
||||||
|
**When to move to a counter kept with each entry** (a migration adding hits
|
||||||
|
and last-hit to `inbound_spam_list`, bumped where the journal row is written;
|
||||||
|
the owner asked for the condition to be written down). Any one of:
|
||||||
|
|
||||||
|
1. **The journal is cut by its 10 000-row cap and not by age.** Then it holds
|
||||||
|
fewer days than its retention, the sheet's "in the last 90 days" is no
|
||||||
|
longer true, and *never* stops being evidence that an entry is stale. The
|
||||||
|
sign: the log tailer's prune line keeps reporting rows removed while the
|
||||||
|
oldest row of Inbound › Log is younger than the retention.
|
||||||
|
2. **The retention is set short** (towards its minimum of 7 days) for reasons
|
||||||
|
of its own, privacy or disk. A week without a match does not make an entry
|
||||||
|
stale.
|
||||||
|
3. **Counts are wanted that outlive the journal** - since the entry was added.
|
||||||
|
4. **The page is slow.** A view reads every journal row a list decided and
|
||||||
|
tries it against every entry. With the cap that is at most 10 000 rows
|
||||||
|
times the number of entries; past a few hundred entries, measure it.
|
||||||
|
|
||||||
|
Until then no migration is spent on it.
|
||||||
|
|
||||||
### Activation
|
### Activation
|
||||||
|
|
||||||
@@ -200,7 +267,9 @@ flowchart LR
|
|||||||
## Security
|
## Security
|
||||||
|
|
||||||
- Same validation whitelists as inbound relay ([security.md](../security.md)).
|
- Same validation whitelists as inbound relay ([security.md](../security.md)).
|
||||||
- Journal rows contain PII — retention and global-admin-only access.
|
- Journal rows contain PII — retention, and access by reach: the global role
|
||||||
|
reads all of it, a `domain` user the rows of assigned inbound domains (as
|
||||||
|
built; see § RBAC).
|
||||||
- Inbound journal milter fail-open must not block mail (backup-MX role).
|
- Inbound journal milter fail-open must not block mail (backup-MX role).
|
||||||
- List map writes injection-safe; deny wins over allow.
|
- List map writes injection-safe; deny wins over allow.
|
||||||
|
|
||||||
@@ -243,14 +312,14 @@ Target version cut: **`2.1.0`** (MINOR), after `2.0.0`. One commit per step;
|
|||||||
[development.md](../development.md) § Plan checklists.
|
[development.md](../development.md) § Plan checklists.
|
||||||
|
|
||||||
- [x] Agree journal fields, retention, and RBAC (this plan § Decisions) — **Composer**
|
- [x] Agree journal fields, retention, and RBAC (this plan § Decisions) — **Composer**
|
||||||
- [ ] Migration `0002_inbound_spam_log.sql` on the 2.0 baseline — **Composer**
|
- [x] Migration `0002_inbound_spam_log.sql` on the 2.0 baseline — **Composer**
|
||||||
- [ ] Inbound journal-milter + Postfix wiring — **Composer**
|
- [x] Inbound journal-milter + Postfix wiring — **Composer**
|
||||||
- [ ] mail.log tailer: inbound reject rows — **Composer**
|
- [x] mail.log tailer: inbound reject rows — **Composer**
|
||||||
- [ ] List CRUD + validation + atomic rspamd map sync — **Composer**
|
- [x] List CRUD + validation + atomic rspamd map sync — **Composer**
|
||||||
- [ ] Panel: Inbound › Log and Inbound › Filter lists from the kit partials, mockups `in-log` / `in-filter-lists` (add client IP and engine columns mockup-first), evidence pairs — **Composer**
|
- [x] Panel: Inbound › Log and Inbound › Filter lists from the kit partials, mockups `in-log` / `in-filter-lists` (add client IP and engine columns mockup-first), evidence pairs — **Composer**
|
||||||
- [ ] Backup includes `/data/antispam/` — **Composer**
|
- [x] Backup includes `/data/antispam/` — **Composer** (it always did: a backup walks all of `/data`; now held by a test, and the guide says the `antispam/` fragment directory is not in it)
|
||||||
- [ ] Unit + handler tests — **Composer**
|
- [x] Unit + handler tests — **Composer** (written with each step: store, milter, tailer, lists and their usage, view fixtures held to the outlines, handlers for both roles)
|
||||||
- [ ] [guide.md](../guide.md), [architecture.md](../architecture.md) — **Composer**
|
- [x] [guide.md](../guide.md), [architecture.md](../architecture.md) — **Composer** (kept up with each step; [security.md](../security.md) follows the security review)
|
||||||
- [ ] `go vet`, `go test`, e2e if applicable — **Composer**
|
- [x] `go vet`, `go test`, e2e if applicable — **Composer** (on the test server for every step; last on `7192c38`: vet ok, 22 packages ok, e2e ok, which relays, refuses, reads the log page and edits a list through the panel)
|
||||||
- [ ] Technical review of inbound antispam changes — **Opus**
|
- [x] Technical review of inbound antispam changes — **Opus** (2026-10-12; findings fixed: client address of a connection's later messages, the lists' writer under a lock, the tailer's queue-id memory, hold rows written twice, the cap by time; and, outside the feature, the level-2 limit and client-address list skipped after a connection's first message)
|
||||||
- [ ] Security review inbound antispam path — **Fable**
|
- [x] Security review inbound antispam path — **Fable** (2026-10-12; no Critical or High; HELO-forged journal rows and the six-hourly row cap fixed; a sender's own verdict while the filter is down closed by the filter's signature, the owner's idea; accepted risks recorded in [security.md](../security.md))
|
||||||
|
|||||||
+44
-6
@@ -30,7 +30,7 @@ in `git log` and [CHANGELOG.md](../CHANGELOG.md).
|
|||||||
|---|---|---|---|---|
|
|---|---|---|---|---|
|
||||||
| preflight | Installation check page (`/preflight`) | candidate | — | [plans/preflight.md](plans/preflight.md) |
|
| preflight | Installation check page (`/preflight`) | candidate | — | [plans/preflight.md](plans/preflight.md) |
|
||||||
| contributing | `CONTRIBUTING.md` | candidate | — | — |
|
| contributing | `CONTRIBUTING.md` | candidate | — | — |
|
||||||
| inbound-antispam-panel | Inbound antispam journal + allow/deny lists | agreed | 1/12 | [plans/inbound-antispam-panel.md](plans/inbound-antispam-panel.md) |
|
| inbound-antispam-panel | Inbound antispam journal + allow/deny lists | shipped `2.1.0` | 12/12 | [plans/inbound-antispam-panel.md](plans/inbound-antispam-panel.md) |
|
||||||
| inbound-quarantine | Inbound spam quarantine (hold / review / release) | candidate | — | [plans/inbound-quarantine.md](plans/inbound-quarantine.md) |
|
| inbound-quarantine | Inbound spam quarantine (hold / review / release) | candidate | — | [plans/inbound-quarantine.md](plans/inbound-quarantine.md) |
|
||||||
| csrf-tokens | Per-session CSRF tokens on state-changing forms | candidate | — | — |
|
| csrf-tokens | Per-session CSRF tokens on state-changing forms | candidate | — | — |
|
||||||
| template-data-typing | Typed structs for template data instead of `map[string]any` | candidate | — | — |
|
| template-data-typing | Typed structs for template data instead of `map[string]any` | candidate | — | — |
|
||||||
@@ -39,15 +39,18 @@ in `git log` and [CHANGELOG.md](../CHANGELOG.md).
|
|||||||
| panel-notifications | E-mail notifications about important events | candidate | — | — |
|
| panel-notifications | E-mail notifications about important events | candidate | — | — |
|
||||||
| delivery-log-storage | Keep a message's `mail.log` lines with its send-log row | candidate | — | — |
|
| delivery-log-storage | Keep a message's `mail.log` lines with its send-log row | candidate | — | — |
|
||||||
| password-reset | Password reset by e-mail | candidate | — | — |
|
| password-reset | Password reset by e-mail | candidate | — | — |
|
||||||
|
| phone-desktop-mode | A phone browser in "desktop site" mode gets the phone navigation | candidate | — | — |
|
||||||
| panel-redesign | Panel redesign (accepted mockups, design contract) | shipped `2.0.0` | 27/27 | [plans/panel-redesign.md](plans/panel-redesign.md) |
|
| panel-redesign | Panel redesign (accepted mockups, design contract) | shipped `2.0.0` | 27/27 | [plans/panel-redesign.md](plans/panel-redesign.md) |
|
||||||
| schema-squash | Squash SQLite migrations into a 2.x baseline | built in panel-redesign (stage 1) | — | [plans/panel-redesign.md](plans/panel-redesign.md) |
|
| schema-squash | Squash SQLite migrations into a 2.x baseline | built in panel-redesign (stage 1) | — | [plans/panel-redesign.md](plans/panel-redesign.md) |
|
||||||
|
|
||||||
**Recommended order** (not binding; owner, 2026-10-09: feature order follows
|
**Recommended order** (not binding; owner, 2026-10-09: feature order follows
|
||||||
development convenience): **panel-redesign** shipped as `2.0.0` (2026-10-11); the next stage is
|
development convenience): **panel-redesign** shipped as `2.0.0` (2026-10-11) and
|
||||||
**inbound-antispam-panel** (`2.1.0`), built in the new component kit —
|
**inbound-antispam-panel** as `2.1.0` (2026-10-12), built in the new component
|
||||||
styling its two screens once instead of twice. panel-docs shipped in
|
kit — styling its two screens once instead of twice. Nothing is agreed after
|
||||||
|
it: every remaining item is a candidate. panel-docs shipped in
|
||||||
[CHANGELOG.md](../CHANGELOG.md) `[1.8.0]`; dmarc-reports in `[1.7.0]`
|
[CHANGELOG.md](../CHANGELOG.md) `[1.8.0]`; dmarc-reports in `[1.7.0]`
|
||||||
(security review of the ingest path pending); domain-stats-auto-ratelimit in
|
(its ingest path first ran in `2.0.0` and was reviewed on 2026-10-11, see
|
||||||
|
[security.md](security.md)); domain-stats-auto-ratelimit in
|
||||||
`[1.6.0]`; send-log-retention in `[1.5.0]`; inbound-relay in `[1.4.0]`;
|
`[1.6.0]`; send-log-retention in `[1.5.0]`; inbound-relay in `[1.4.0]`;
|
||||||
queue-retries in `[1.3.1]`; the 2026-08-13 full-tree review follow-ups are in
|
queue-retries in `[1.3.1]`; the 2026-08-13 full-tree review follow-ups are in
|
||||||
`[1.3.0]`. Candidates need explicit agreement before they join the queue.
|
`[1.3.0]`. Candidates need explicit agreement before they join the queue.
|
||||||
@@ -133,7 +136,7 @@ hook is set.
|
|||||||
**Dependencies / risks:** inbound relay `[1.4.0]`; rspamd map/header coupling;
|
**Dependencies / risks:** inbound relay `[1.4.0]`; rspamd map/header coupling;
|
||||||
journal PII; global-admin-only RBAC.
|
journal PII; global-admin-only RBAC.
|
||||||
|
|
||||||
**Version:** `2.1.0` MINOR (was `1.10.0`; reordered after the redesign on 2026-10-09); **agreed**.
|
**Version:** shipped `2.1.0` (2026-10-12), MINOR (was `1.10.0`; reordered after the redesign on 2026-10-09).
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -405,6 +408,41 @@ goes and today Account saves it without the password.
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## phone-desktop-mode
|
||||||
|
|
||||||
|
**Problem (owner, 2026-10-11):** the panel opened on a phone with the
|
||||||
|
browser's "desktop site" switched on does not become the desktop panel.
|
||||||
|
Seen on Server › Health at `selfpost.mixfed.ru`: the page body is laid out in
|
||||||
|
columns (Machine and Processes side by side, three cards in the next row),
|
||||||
|
while the navigation is the phone one - not a bar but a stack with every
|
||||||
|
section open (Outbound, Inbound, Server, the user menu), filling the first
|
||||||
|
screen before the page starts. The content also sits flush against the left
|
||||||
|
edge of the screen.
|
||||||
|
|
||||||
|
**Goal:** find out why, then decide what the panel should be in that mode -
|
||||||
|
the desktop panel, or a phone panel that is at least consistent.
|
||||||
|
|
||||||
|
**Where to look first (a guess, not checked):** "desktop site" on Android
|
||||||
|
lays a page out about 980 px wide. Bulma turns the navigation into a bar at
|
||||||
|
1024 px and puts columns side by side from 769 px, so 980 px falls between
|
||||||
|
the two: desktop columns under a touch navigation. `panel.css` has its own
|
||||||
|
breakpoints at 768, 1023 and 1407 px. Check the width the phone really
|
||||||
|
reports, why every dropdown is open rather than folded behind the burger,
|
||||||
|
and where the left padding went.
|
||||||
|
|
||||||
|
**Boundary:** the navigation and page gutter between 769 and 1023 px, which
|
||||||
|
is also a tablet in portrait and a narrow desktop window - the same defect is
|
||||||
|
expected there. A design change is mockup-first
|
||||||
|
([plans/panel-redesign.md](plans/panel-redesign.md) § Enforcement).
|
||||||
|
|
||||||
|
**Done when:** the cause is written down here, and the panel at that width
|
||||||
|
has one coherent layout, checked on a phone in "desktop site" mode and in a
|
||||||
|
980 px window.
|
||||||
|
|
||||||
|
**Version:** `2.x` PATCH or MINOR; `candidate`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## panel-redesign
|
## panel-redesign
|
||||||
|
|
||||||
**Goal:** a panel an operator opens a few times a year and still finds their
|
**Goal:** a panel an operator opens a few times a year and still finds their
|
||||||
|
|||||||
@@ -17,8 +17,8 @@ When you add or review a migration, update this file in the same change.
|
|||||||
|
|
||||||
| Field | Value |
|
| Field | Value |
|
||||||
|---|---|
|
|---|---|
|
||||||
| Chain head | `user_version = 1` |
|
| Chain head | `user_version = 2` |
|
||||||
| Files | `0001_init.sql` (the 2.0 baseline) |
|
| Files | `0001_init.sql` (the 2.0 baseline), `0002_inbound_spam_log.sql` |
|
||||||
| File mark | `PRAGMA application_id = 0x53503230` ("SP20") |
|
| File mark | `PRAGMA application_id = 0x53503230` ("SP20") |
|
||||||
| Database file | `/data/selfpost.db` (bind mount) |
|
| Database file | `/data/selfpost.db` (bind mount) |
|
||||||
| Compatibility | 2.x MINOR releases must boot a `2.0.0` data directory. **Nothing is carried over from 1.x.** |
|
| Compatibility | 2.x MINOR releases must boot a `2.0.0` data directory. **Nothing is carried over from 1.x.** |
|
||||||
@@ -60,10 +60,9 @@ What that means in practice:
|
|||||||
|
|
||||||
**Implication:** once `2.0.0` is released, `0001_init.sql` is never edited
|
**Implication:** once `2.0.0` is released, `0001_init.sql` is never edited
|
||||||
again and no file is deleted, renamed or reordered — every 2.x database has
|
again and no file is deleted, renamed or reordered — every 2.x database has
|
||||||
already run it. Until that release the baseline is still being written:
|
already run it. `2.0.0` and `2.0.1` shipped with the baseline alone; the
|
||||||
later steps of the redesign and the features queued behind it (the next one
|
features after them add files, the first being `0002_inbound_spam_log.sql` of
|
||||||
is `0002_inbound_spam_log.sql` of inbound-antispam-panel, `2.1.0`) add files
|
inbound-antispam-panel (`2.1.0`).
|
||||||
after it.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -72,13 +71,14 @@ after it.
|
|||||||
| Ver | File | Shipped | Kind | Summary |
|
| Ver | File | Shipped | Kind | Summary |
|
||||||
|-----|------|---------|------|---------|
|
|-----|------|---------|------|---------|
|
||||||
| 1 | `0001_init.sql` | 2.0.0 | DDL | The whole 2.0 schema, below |
|
| 1 | `0001_init.sql` | 2.0.0 | DDL | The whole 2.0 schema, below |
|
||||||
|
| 2 | `0002_inbound_spam_log.sql` | 2.1.0 | DDL | `inbound_spam_log`, `inbound_spam_list` and two indexes; nothing existing changes |
|
||||||
|
|
||||||
**Kind:** *DDL* — schema only; *data* — `INSERT`/`UPDATE` that must stay correct
|
**Kind:** *DDL* — schema only; *data* — `INSERT`/`UPDATE` that must stay correct
|
||||||
for operators upgrading from older 2.x images.
|
for operators upgrading from older 2.x images.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Schema at head (v1)
|
## Schema at head (v2)
|
||||||
|
|
||||||
| Table | Role |
|
| Table | Role |
|
||||||
|-------|------|
|
|-------|------|
|
||||||
@@ -98,9 +98,12 @@ for operators upgrading from older 2.x images.
|
|||||||
| `user_inbound_domains` | Inbound domains assigned to a `domain` user — granted separately from the outbound ones |
|
| `user_inbound_domains` | Inbound domains assigned to a `domain` user — granted separately from the outbound ones |
|
||||||
| `dmarc_reports` | Parsed aggregate report summaries |
|
| `dmarc_reports` | Parsed aggregate report summaries |
|
||||||
| `dmarc_report_records` | Per-source rows inside a report |
|
| `dmarc_report_records` | Per-source rows inside a report |
|
||||||
|
| `inbound_spam_log` | Inbound antispam journal: one row per recipient of a message offered on port 25, with the decision (`accept`, `reject`, `tempfail`, `quarantine`) and who made it (`rspamd`, `postfix`, `selfpost`). Metadata only (v2) |
|
||||||
|
| `inbound_spam_list` | Allow / deny entries for the external filter (`address`, `domain`, `ip`), instance-wide; unique per list (v2) |
|
||||||
|
|
||||||
`TestBaselineSchema` (`internal/store/schema_test.go`) pins every table, column
|
`TestBaselineSchema` (`internal/store/schema_test.go`) pins every table, column
|
||||||
and index of this list.
|
and index of this list; `TestOpenMigratesA20Database` opens a database that
|
||||||
|
holds the baseline alone.
|
||||||
|
|
||||||
### What changed against the last 1.x schema
|
### What changed against the last 1.x schema
|
||||||
|
|
||||||
@@ -129,7 +132,7 @@ for the domain. Nothing refers to a user: deleting one changes no domain.
|
|||||||
|
|
||||||
1. Add `internal/store/migrations/00NN_short_name.sql` (next number).
|
1. Add `internal/store/migrations/00NN_short_name.sql` (next number).
|
||||||
2. Update the table in [Migration chain](#migration-chain), the snapshot at the
|
2. Update the table in [Migration chain](#migration-chain), the snapshot at the
|
||||||
top, and the affected rows of [Schema at head](#schema-at-head-v1).
|
top, and the affected rows of [Schema at head](#schema-at-head-v2).
|
||||||
3. Update `TestBaselineSchema` if tables, columns or indexes change, and add a
|
3. Update `TestBaselineSchema` if tables, columns or indexes change, and add a
|
||||||
test that opens a database at the previous version and migrates it.
|
test that opens a database at the previous version and migrates it.
|
||||||
4. A *data* migration states in a comment which 2.x versions it is written for.
|
4. A *data* migration states in a comment which 2.x versions it is written for.
|
||||||
|
|||||||
+105
-4
@@ -26,7 +26,31 @@ the user form did not end that user's sessions, internal errors reached forms a
|
|||||||
(see [Authentication and sessions](#authentication-and-sessions) and [Errors
|
(see [Authentication and sessions](#authentication-and-sessions) and [Errors
|
||||||
shown to users](#errors-shown-to-users)); two deliberate behaviours are
|
shown to users](#errors-shown-to-users)); two deliberate behaviours are
|
||||||
recorded under Accepted risks (upstream of a delegated inbound domain, profile
|
recorded under Accepted risks (upstream of a delegated inbound domain, profile
|
||||||
saved without the password).
|
saved without the password). The 2026-10-11 review (Fable) covered the DMARC
|
||||||
|
report path from port 25 to the report pages, which ran for the first time in
|
||||||
|
2.0.0 - the 2026-08-19 pass read code that Postfix could not yet start: no
|
||||||
|
Critical or High finding; a report's size was bounded only as bytes (now 5 000
|
||||||
|
records, 255 bytes a field), the retention cap was one for the whole server
|
||||||
|
(now per domain), the delivery command could run a hundred at once (now one),
|
||||||
|
and the reason logged for a dropped message carried the sender's bytes as they
|
||||||
|
came (now cut and reduced to printable ASCII). The 2026-10-12 reviews of
|
||||||
|
inbound-antispam-panel (`2.1.0`: the inbound journal milter, refusals read from
|
||||||
|
`mail.log`, the filter lists and their pages) were two, by different models
|
||||||
|
over the same diff. The technical one (Opus) found, outside the feature and
|
||||||
|
there since `1.9.0`, that **the level-2 rate limit and an application's
|
||||||
|
client-address list held only for the first message of a connection** - see
|
||||||
|
[Mail path](#mail-path-security-relevant) - which was reproduced on the test
|
||||||
|
stand and fixed. The security one (Fable) found no Critical or High: a client's
|
||||||
|
HELO name could supply the sender, recipient and reply of a journal row and so
|
||||||
|
make a refusal look like the deny list's (the reply now ends at Postfix's own
|
||||||
|
`; from=<`); the journal's row cap was applied only every six hours, between
|
||||||
|
which refusals were written at whatever rate a stranger sent recipients (the
|
||||||
|
tailer now cuts to the cap every 500 rows); and two behaviours are recorded
|
||||||
|
under Accepted risks (the allow list keyed on an address anyone can claim, the
|
||||||
|
list files readable by anyone). A third it named - with the filter down, a
|
||||||
|
sender's own verdict headers were recorded as the filter's - was closed the
|
||||||
|
same day at the owner's suggestion: the filter signs each message and a
|
||||||
|
verdict is read only next to its signature.
|
||||||
(2) **Accepted risks** —
|
(2) **Accepted risks** —
|
||||||
deliberate departures beyond the mandatory, recorded so the decision is not
|
deliberate departures beyond the mandatory, recorded so the decision is not
|
||||||
lost.
|
lost.
|
||||||
@@ -113,6 +137,8 @@ assigned outbound domains and inbound domains **separately**, each list with
|
|||||||
| Outbound: domain page, applications, Domain settings (report address, rate limit, export), log | every domain | assigned outbound domains |
|
| Outbound: domain page, applications, Domain settings (report address, rate limit, export), log | every domain | assigned outbound domains |
|
||||||
| Outbound › DMARC reports | every report, with the server's ingest statistics | reports of assigned domains, without the ingest box |
|
| Outbound › DMARC reports | every report, with the server's ingest statistics | reports of assigned domains, without the ingest box |
|
||||||
| Inbound: domain page, upstream, recipients | every inbound domain | assigned inbound domains |
|
| Inbound: domain page, upstream, recipients | every inbound domain | assigned inbound domains |
|
||||||
|
| Inbound › Log | every row, including those of a domain deleted since | rows of assigned inbound domains (by name), without the links into the filter lists |
|
||||||
|
| Inbound › Filter lists; the inbound log's retention | yes | no — 404 |
|
||||||
| Add or delete a domain, either direction; import a domain | yes | no |
|
| Add or delete a domain, either direction; import a domain | yes | no |
|
||||||
| Account, Help | yes | yes |
|
| Account, Help | yes | yes |
|
||||||
|
|
||||||
@@ -171,6 +197,41 @@ operator can act on it.
|
|||||||
refuse unknown addresses at RCPT (no backscatter). OpenDKIM is not attached
|
refuse unknown addresses at RCPT (no backscatter). OpenDKIM is not attached
|
||||||
on inbound. An optional antispam milter is inbound-only; default action is
|
on inbound. An optional antispam milter is inbound-only; default action is
|
||||||
fail-open (`accept`).
|
fail-open (`accept`).
|
||||||
|
- **The panel's milters judge every message of a connection.** Postfix
|
||||||
|
introduces a client to a milter once per connection, and the milter library
|
||||||
|
replaces the milter's session after any reply that is not "continue". Until
|
||||||
|
`2.1.0` the journal milter knew the client's address only from that
|
||||||
|
introduction, and both of its checks passed a message that came without one:
|
||||||
|
a second `MAIL FROM` after a refused one, or any message after the first on
|
||||||
|
one connection, got past the level-2 limit and past an application's
|
||||||
|
client-address list. Now the address comes with every `MAIL FROM`
|
||||||
|
(`milter_mail_macros` carries `{client_addr}`), the level-2 limit does not
|
||||||
|
ask for it at all, an application with a client-address list is refused when
|
||||||
|
no address is known, and neither milter replies anything but "continue" to a
|
||||||
|
message it lets through. `test/e2e` sends two `MAIL FROM` on one connection.
|
||||||
|
- **The inbound journal is written from what strangers send.** The inbound
|
||||||
|
journal milter is the last on port 25, read-only and attached fail-open; it
|
||||||
|
keeps of the filter's verdict only what its patterns recognise. **A verdict
|
||||||
|
is read only next to the filter's signature.** The filter is attached so
|
||||||
|
that mail flows when it is down, and then nothing removes verdict headers a
|
||||||
|
sender wrote himself. So rspamd adds a third header, `X-SelfPost-Filter`:
|
||||||
|
the HMAC-SHA256 of the message's queue id under a key SelfPost generates
|
||||||
|
(`data/antispam/filter.key`) and the sidecar reads from the directory it
|
||||||
|
mounts. The journal milter recomputes it at end-of-message; with no
|
||||||
|
signature, or one that does not fit, the row is that of mail nobody judged.
|
||||||
|
The signature goes on to the upstream with the mail and is of no use there:
|
||||||
|
it fits one queue id. Refusals are
|
||||||
|
parsed from `mail.log` from the head of the line, with the reply ending at
|
||||||
|
Postfix's own `; from=<` so that the HELO name after it - the client's text,
|
||||||
|
logged as written - cannot stand in for the sender, the recipient or the
|
||||||
|
reply; an address with a quote, a bracket or a space in it is not kept.
|
||||||
|
Only a recipient at an inbound domain makes a row. The journal is cut to
|
||||||
|
10 000 rows every 500 rows the tailer writes, and by age every six hours.
|
||||||
|
- **Filter lists** reach another container as files: values are whitelisted on
|
||||||
|
the way in (address, domain, canonical IP or network; no `/0`, no
|
||||||
|
IPv4-mapped network), the writer refuses a stored value with whitespace, a
|
||||||
|
newline or `#`, and a change of the table and the rewrite of the six files
|
||||||
|
are one step under a lock.
|
||||||
- TLS is mandatory before credentials are transmitted (465 wrapper / 587
|
- TLS is mandatory before credentials are transmitted (465 wrapper / 587
|
||||||
`encrypt`).
|
`encrypt`).
|
||||||
- `TRUSTED_PROXY_CIDR` — only explicitly trusted proxies may supply
|
- `TRUSTED_PROXY_CIDR` — only explicitly trusted proxies may supply
|
||||||
@@ -274,20 +335,60 @@ deferred item from the roadmap.
|
|||||||
reporters are arbitrary third parties and the mailbox address is published in
|
reporters are arbitrary third parties and the mailbox address is published in
|
||||||
DNS. The ingest already constrains the meaningful part — a report cannot be
|
DNS. The ingest already constrains the meaningful part — a report cannot be
|
||||||
filed under a domain this instance does not host, and the claimed domain is
|
filed under a domain this instance does not host, and the claimed domain is
|
||||||
cross-checked against the `+tag` of the delivery address — and the data is
|
cross-checked against the `+tag` of the delivery address (effective since
|
||||||
advisory statistics only; nothing enforces off it. The trigger to revisit is
|
2.0.0: before it the recipient never reached the command) — and the data is
|
||||||
|
advisory statistics only; nothing enforces off it. What a stranger can do is
|
||||||
|
limited to a domain that chose a SelfPost-hosted address, through that
|
||||||
|
address: file reports under it, replace one whose reporter and id they know,
|
||||||
|
and fill that domain's 500 — not another domain's. A message that is not a
|
||||||
|
report the relay keeps is dropped, not bounced (the envelope sender is the
|
||||||
|
author's to write); a genuine report the parser cannot read is lost the same
|
||||||
|
way, with a count and a line in the mail log. The trigger to revisit is
|
||||||
any feature that lets report contents drive an automatic action (policy
|
any feature that lets report contents drive an automatic action (policy
|
||||||
changes, rate limiting, alerting an operator into a config change).
|
changes, rate limiting, alerting an operator into a config change).
|
||||||
- **The application auth-IP allow-list is enforced fail-open.** The check runs
|
- **The application auth-IP allow-list is enforced fail-open.** The check runs
|
||||||
in the journal milter (`internal/milter/authip.go`) with
|
in the journal milter (`internal/milter/authip.go`) with
|
||||||
`default_action=accept` and skips on store errors, so if the milter is down
|
`default_action=accept` and skips on store errors, so if the milter is down
|
||||||
the restriction is not applied; the SASL login itself always succeeds. This
|
the restriction is not applied; the SASL login itself always succeeds.
|
||||||
|
(While the milter is up the list holds for every message, and an unknown
|
||||||
|
client address is refused - see Mail path; what is accepted here is the
|
||||||
|
milter being down.) This
|
||||||
is documented in [guide.md](guide.md) as **not an authentication boundary** —
|
is documented in [guide.md](guide.md) as **not an authentication boundary** —
|
||||||
it is a blast-radius limiter for a leaked application password, on the same
|
it is a blast-radius limiter for a leaked application password, on the same
|
||||||
fail-open path as level-2 rate limiting, and a milter outage already stops
|
fail-open path as level-2 rate limiting, and a milter outage already stops
|
||||||
the journal from recording sends at all. The trigger to revisit is a
|
the journal from recording sends at all. The trigger to revisit is a
|
||||||
requirement for the allow-list to hold as a security boundary — at which
|
requirement for the allow-list to hold as a security boundary — at which
|
||||||
point it must move to the SASL/smtpd layer and fail closed.
|
point it must move to the SASL/smtpd layer and fail closed.
|
||||||
|
- **The key the spam filter signs with is readable by anyone on the host.**
|
||||||
|
The inbound journal reads the filter's verdict only next to the filter's
|
||||||
|
signature (see Mail path), and the key for it, `data/antispam/filter.key`,
|
||||||
|
is 0644 like the list files beside it: the filter is another user in another
|
||||||
|
container. Someone with a shell on the host could therefore sign a verdict
|
||||||
|
of his own - into a monitoring page, on a host where he could do far more.
|
||||||
|
Senders on the internet, whom the signature is for, cannot read it. Trigger
|
||||||
|
to revisit: the same as for the list files - a shared host.
|
||||||
|
- **An allow entry for a sender address or domain trusts what the sending
|
||||||
|
server says.** The filter matches the envelope sender, which is not
|
||||||
|
authenticated: anyone who knows or guesses an allowed address can send as it
|
||||||
|
and is given the entry's weight, which no spam score outweighs. The page's
|
||||||
|
Help and the guide say so and recommend an entry for the client's address or
|
||||||
|
network instead. Deny entries have no such problem - claiming to be a denied
|
||||||
|
sender only gets mail refused - and deny wins over allow. Trigger to
|
||||||
|
revisit: allow entries being used as the main way to receive mail from a
|
||||||
|
sender, at which point the rule should also require the sender's SPF or
|
||||||
|
DKIM to pass.
|
||||||
|
- **The filter lists are files readable by anyone on the host.**
|
||||||
|
`/data/antispam` is 0755 and its six files 0644, unlike the rest of `/data`:
|
||||||
|
the filter runs under another user in another container. They hold sender
|
||||||
|
addresses, domains and networks - who the operator blocks and whom he lets
|
||||||
|
through - but no message data. The inbound journal itself is in the
|
||||||
|
database, 0640. Trigger to revisit: a shared host, where the sidecar should
|
||||||
|
instead be given the files through a group it shares with the panel.
|
||||||
|
- **A row of the inbound log belongs to a domain by name.** A `domain` user
|
||||||
|
sees the rows whose inbound domain has the name of one assigned to them, so
|
||||||
|
an inbound domain that is deleted and added again shows its earlier rows to
|
||||||
|
whoever it is assigned to next. Accepted: the journal is of a mail domain,
|
||||||
|
and the rows age out with the retention.
|
||||||
- **Level-2 in-flight reservations can outlive a dropped SMTP session until TTL.**
|
- **Level-2 in-flight reservations can outlive a dropped SMTP session until TTL.**
|
||||||
The journal milter cannot observe connection-close directly through
|
The journal milter cannot observe connection-close directly through
|
||||||
go-milter, so if a client disconnects without an explicit `Abort`, its
|
go-milter, so if a client disconnects without an explicit `Abort`, its
|
||||||
|
|||||||
+84
-10
@@ -1,16 +1,16 @@
|
|||||||
// Package backup implements SelfPost's full-server backup and the restore
|
// Package backup implements SelfPost's full-server backup and the restore
|
||||||
// version guard (architecture.md § Persistence). A full backup is a
|
// version guard (architecture.md § Persistence). A full backup is a
|
||||||
// gzip-compressed tar of the consolidated persistent state under /data — the
|
// gzip-compressed tar of the consolidated persistent state under /data — the
|
||||||
// SQLite database (as a consistent snapshot), the per-domain DKIM keys, the
|
// SQLite database (as a consistent snapshot), the per-domain DKIM keys and the
|
||||||
// SASL database, and the Postfix queue — plus docker-compose.yml, .env, and
|
// SASL database — plus docker-compose.yml, .env, and certs/ from the
|
||||||
// certs/ from the operator's deploy directory, and a manifest recording the
|
// operator's deploy directory, and a manifest recording the SelfPost version
|
||||||
// SelfPost version that produced it. Postfix delivery logs under log/ are
|
// that produced it. Postfix delivery logs under log/ are excluded
|
||||||
// excluded (diagnostics, not state).
|
// (diagnostics, not state), and so is the Postfix queue (see QueueDir).
|
||||||
//
|
//
|
||||||
// Restore is not a separate code path in the panel: a backup is extracted into
|
// Restore is not a separate code path in the panel: a backup is extracted into
|
||||||
// the operator's project directory (data/, docker-compose.yml, .env, certs/)
|
// the operator's project directory (data/, docker-compose.yml, .env, certs/)
|
||||||
// before first start. The archive carries everything needed to bring the
|
// before first start. The archive carries everything needed to bring the
|
||||||
// instance back — DKIM keys, sasldb2, sender map, queue, and deploy files —
|
// instance back — DKIM keys, sasldb2, sender map, and deploy files —
|
||||||
// so the operator only adjusts hostname or proxy settings on a new host. The
|
// so the operator only adjusts hostname or proxy settings on a new host. The
|
||||||
// restore-specific steps the panel runs are CheckRestore, which refuses to
|
// restore-specific steps the panel runs are CheckRestore, which refuses to
|
||||||
// boot if the manifest's version does not match the running binary so
|
// boot if the manifest's version does not match the running binary so
|
||||||
@@ -28,6 +28,7 @@ import (
|
|||||||
"compress/gzip"
|
"compress/gzip"
|
||||||
"database/sql"
|
"database/sql"
|
||||||
"encoding/json"
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
"io"
|
"io"
|
||||||
"io/fs"
|
"io/fs"
|
||||||
@@ -81,6 +82,18 @@ type Params struct {
|
|||||||
OnWarn func(string)
|
OnWarn func(string)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// QueueDir is the Postfix queue, relative to the data directory. No backup
|
||||||
|
// holds it (owner, 2026-10-11). An archive is made while the instance runs,
|
||||||
|
// and Postfix moves messages between the queue's directories all the time: a
|
||||||
|
// copy taken then can hold a message twice or not at all, so what it would
|
||||||
|
// bring back is not the queue. Nor could the panel read it - Postfix keeps
|
||||||
|
// every queue file readable by its own user alone, which is why the panel's
|
||||||
|
// backup failed outright from 1.3.0 to 2.0.0. What is not carried over is mail
|
||||||
|
// still waiting for delivery; a start without a queue directory builds an
|
||||||
|
// empty one. Whoever needs the queue stops the instance and archives the
|
||||||
|
// project directory (guide § Taking a full backup).
|
||||||
|
const QueueDir = "postfix/queue"
|
||||||
|
|
||||||
// excludedFromArchive lists the data-directory entries a backup never carries.
|
// excludedFromArchive lists the data-directory entries a backup never carries.
|
||||||
// The live database files are replaced by a consistent VACUUM INTO snapshot
|
// The live database files are replaced by a consistent VACUUM INTO snapshot
|
||||||
// written under the canonical name; the setup token is transient bootstrap
|
// written under the canonical name; the setup token is transient bootstrap
|
||||||
@@ -145,7 +158,7 @@ func Create(w io.Writer, p Params) error {
|
|||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
|
||||||
if err := addTree(tw, p.DataDir, DataArchivePrefix, excludedFromArchive); err != nil {
|
if err := addTree(tw, p.DataDir, DataArchivePrefix, excludedFromArchive, skipQueue); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -181,6 +194,52 @@ func validateDeployRoot(deployRoot string) error {
|
|||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// skipFunc says whether the entry at path (rel to the root being walked) is
|
||||||
|
// left out of the archive. A directory left out is not descended into.
|
||||||
|
type skipFunc func(path, rel string, d fs.DirEntry) bool
|
||||||
|
|
||||||
|
// skipQueue leaves the Postfix queue out of every archive.
|
||||||
|
func skipQueue(_, rel string, _ fs.DirEntry) bool {
|
||||||
|
return filepath.ToSlash(rel) == QueueDir
|
||||||
|
}
|
||||||
|
|
||||||
|
// Check reads through everything Create would put into the archive, without
|
||||||
|
// writing one, and returns the first thing that cannot be read. The panel
|
||||||
|
// streams its archive to the browser, so once Create has begun a failure can
|
||||||
|
// only cut the download short; this is run first, while a refusal can still be
|
||||||
|
// shown as a page.
|
||||||
|
//
|
||||||
|
// The panel is not root, and nothing is left out on that account: its archive
|
||||||
|
// is the one selfpost-backup makes, or it is not made. What it most likely
|
||||||
|
// cannot read is the private key under certs/ - see UnreadableCert.
|
||||||
|
func Check(p Params) error {
|
||||||
|
if p.DataDir == "" || p.DBPath == "" {
|
||||||
|
return fmt.Errorf("backup: DataDir and DBPath are required")
|
||||||
|
}
|
||||||
|
if err := ValidateDeployRoot(p.DeployRoot); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := addTree(nil, p.DataDir, DataArchivePrefix, excludedFromArchive, skipQueue); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return addDeployFiles(nil, p.DeployRoot, nil)
|
||||||
|
}
|
||||||
|
|
||||||
|
// UnreadableCert reports whether err, from Check or Create, is a file under
|
||||||
|
// the deploy root's certs/ that the caller is not allowed to read, and names
|
||||||
|
// it relative to the deploy root ("certs/privkey.pem").
|
||||||
|
func UnreadableCert(err error, deployRoot string) (name string, ok bool) {
|
||||||
|
var pathErr *fs.PathError
|
||||||
|
if !errors.As(err, &pathErr) || !errors.Is(err, fs.ErrPermission) {
|
||||||
|
return "", false
|
||||||
|
}
|
||||||
|
rel, relErr := filepath.Rel(filepath.Join(deployRoot, CertsDirName), pathErr.Path)
|
||||||
|
if relErr != nil || rel == ".." || strings.HasPrefix(filepath.ToSlash(rel), "../") {
|
||||||
|
return "", false
|
||||||
|
}
|
||||||
|
return CertsDirName + "/" + filepath.ToSlash(rel), true
|
||||||
|
}
|
||||||
|
|
||||||
func addDeployFiles(tw *tar.Writer, deployRoot string, onWarn func(string)) error {
|
func addDeployFiles(tw *tar.Writer, deployRoot string, onWarn func(string)) error {
|
||||||
for _, name := range []string{ComposeFileName, EnvFileName} {
|
for _, name := range []string{ComposeFileName, EnvFileName} {
|
||||||
src := filepath.Join(deployRoot, name)
|
src := filepath.Join(deployRoot, name)
|
||||||
@@ -206,14 +265,16 @@ func addDeployFiles(tw *tar.Writer, deployRoot string, onWarn func(string)) erro
|
|||||||
}
|
}
|
||||||
return fmt.Errorf("backup: stat %s: %w", CertsDirName, err)
|
return fmt.Errorf("backup: stat %s: %w", CertsDirName, err)
|
||||||
}
|
}
|
||||||
return addTree(tw, certsDir, CertsDirName+"/", nil)
|
return addTree(tw, certsDir, CertsDirName+"/", nil, nil)
|
||||||
}
|
}
|
||||||
|
|
||||||
// addTree walks root and adds every regular file (and directory, to preserve
|
// addTree walks root and adds every regular file (and directory, to preserve
|
||||||
// empty ones and modes) to tw under archivePrefix + path relative to root.
|
// empty ones and modes) to tw under archivePrefix + path relative to root.
|
||||||
// When exclude is non-nil, top-level names relative to root are skipped.
|
// When exclude is non-nil, top-level names relative to root are skipped; skip,
|
||||||
|
// when non-nil, leaves out whatever it says. With a nil tw nothing is written:
|
||||||
|
// every file is opened and closed, which is how Check finds what cannot be read.
|
||||||
// Non-regular, non-directory entries (symlinks, sockets) are skipped.
|
// Non-regular, non-directory entries (symlinks, sockets) are skipped.
|
||||||
func addTree(tw *tar.Writer, root, archivePrefix string, exclude map[string]bool) error {
|
func addTree(tw *tar.Writer, root, archivePrefix string, exclude map[string]bool, skip skipFunc) error {
|
||||||
return filepath.WalkDir(root, func(path string, d fs.DirEntry, err error) error {
|
return filepath.WalkDir(root, func(path string, d fs.DirEntry, err error) error {
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return err
|
||||||
@@ -236,12 +297,22 @@ func addTree(tw *tar.Writer, root, archivePrefix string, exclude map[string]bool
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if skip != nil && skip(path, rel, d) {
|
||||||
|
if d.IsDir() {
|
||||||
|
return fs.SkipDir
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
info, err := d.Info()
|
info, err := d.Info()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
switch {
|
switch {
|
||||||
case d.IsDir():
|
case d.IsDir():
|
||||||
|
if tw == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
hdr := &tar.Header{
|
hdr := &tar.Header{
|
||||||
Typeflag: tar.TypeDir,
|
Typeflag: tar.TypeDir,
|
||||||
Name: name + "/",
|
Name: name + "/",
|
||||||
@@ -282,6 +353,9 @@ func writeTarFile(tw *tar.Writer, name string, mode fs.FileMode, srcPath string)
|
|||||||
return fmt.Errorf("backup: open %s: %w", srcPath, err)
|
return fmt.Errorf("backup: open %s: %w", srcPath, err)
|
||||||
}
|
}
|
||||||
defer f.Close()
|
defer f.Close()
|
||||||
|
if tw == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
info, err := f.Stat()
|
info, err := f.Stat()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
|||||||
@@ -5,7 +5,9 @@ import (
|
|||||||
"bytes"
|
"bytes"
|
||||||
"compress/gzip"
|
"compress/gzip"
|
||||||
"encoding/json"
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
"io"
|
"io"
|
||||||
|
"io/fs"
|
||||||
"os"
|
"os"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
"strings"
|
"strings"
|
||||||
@@ -40,6 +42,7 @@ func seedDataDir(t *testing.T) (dataDir, dbPath, deployRoot string) {
|
|||||||
writeFile(t, filepath.Join(dataDir, "opendkim", "keys", "example.com", "selfpost.private"), "PRIVATE KEY")
|
writeFile(t, filepath.Join(dataDir, "opendkim", "keys", "example.com", "selfpost.private"), "PRIVATE KEY")
|
||||||
writeFile(t, filepath.Join(dataDir, "sasl", "sasldb2"), "SASLDB")
|
writeFile(t, filepath.Join(dataDir, "sasl", "sasldb2"), "SASLDB")
|
||||||
writeFile(t, filepath.Join(dataDir, "postfix", "sender_login_maps"), "@example.com login")
|
writeFile(t, filepath.Join(dataDir, "postfix", "sender_login_maps"), "@example.com login")
|
||||||
|
writeFile(t, filepath.Join(dataDir, "antispam", "deny_domain.map"), "evil.example\n")
|
||||||
writeFile(t, filepath.Join(dataDir, "postfix", "queue", "deferred", "sample"), "queue-file")
|
writeFile(t, filepath.Join(dataDir, "postfix", "queue", "deferred", "sample"), "queue-file")
|
||||||
// Transient files that must NOT be archived.
|
// Transient files that must NOT be archived.
|
||||||
writeFile(t, filepath.Join(dataDir, "setup-token"), "secret-token")
|
writeFile(t, filepath.Join(dataDir, "setup-token"), "secret-token")
|
||||||
@@ -112,7 +115,10 @@ func TestCreateIncludesStateExcludesTransient(t *testing.T) {
|
|||||||
DataArchivePrefix + "opendkim/keys/example.com/selfpost.private",
|
DataArchivePrefix + "opendkim/keys/example.com/selfpost.private",
|
||||||
DataArchivePrefix + "sasl/sasldb2",
|
DataArchivePrefix + "sasl/sasldb2",
|
||||||
DataArchivePrefix + "postfix/sender_login_maps",
|
DataArchivePrefix + "postfix/sender_login_maps",
|
||||||
DataArchivePrefix + "postfix/queue/deferred/sample",
|
// The filter lists as the spam filter reads them. They are rewritten
|
||||||
|
// from the database on the first start after a restore; carrying them
|
||||||
|
// means the filter has its lists before the panel is up.
|
||||||
|
DataArchivePrefix + "antispam/deny_domain.map",
|
||||||
ComposeFileName,
|
ComposeFileName,
|
||||||
EnvFileName,
|
EnvFileName,
|
||||||
CertsDirName + "/fullchain.pem",
|
CertsDirName + "/fullchain.pem",
|
||||||
@@ -129,6 +135,7 @@ func TestCreateIncludesStateExcludesTransient(t *testing.T) {
|
|||||||
DataArchivePrefix + "selfpost.db-shm",
|
DataArchivePrefix + "selfpost.db-shm",
|
||||||
DataArchivePrefix + "log/mail.log",
|
DataArchivePrefix + "log/mail.log",
|
||||||
DataArchivePrefix + "log/mail.log.1",
|
DataArchivePrefix + "log/mail.log.1",
|
||||||
|
DataArchivePrefix + "postfix/queue/deferred/sample",
|
||||||
} {
|
} {
|
||||||
if _, ok := files[name]; ok {
|
if _, ok := files[name]; ok {
|
||||||
t.Errorf("archive should not contain %s", name)
|
t.Errorf("archive should not contain %s", name)
|
||||||
@@ -235,3 +242,70 @@ func TestCheckRestoreWrongFormatRejected(t *testing.T) {
|
|||||||
t.Error("CheckRestore wrong format reported a restore")
|
t.Error("CheckRestore wrong format reported a restore")
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// No archive holds the Postfix queue; the rest is there, the private key
|
||||||
|
// included, and Check agrees that it can be made.
|
||||||
|
func TestCreateLeavesTheQueueOut(t *testing.T) {
|
||||||
|
dataDir, dbPath, deployRoot := seedDataDir(t)
|
||||||
|
p := Params{DataDir: dataDir, DBPath: dbPath, Version: "1.2.3", DeployRoot: deployRoot}
|
||||||
|
|
||||||
|
if err := Check(p); err != nil {
|
||||||
|
t.Fatalf("Check: %v", err)
|
||||||
|
}
|
||||||
|
var buf bytes.Buffer
|
||||||
|
if err := Create(&buf, p); err != nil {
|
||||||
|
t.Fatalf("Create: %v", err)
|
||||||
|
}
|
||||||
|
files := readArchive(t, buf.Bytes())
|
||||||
|
for name := range files {
|
||||||
|
if strings.HasPrefix(name, DataArchivePrefix+QueueDir) {
|
||||||
|
t.Errorf("the panel's archive holds %s", name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, name := range []string{
|
||||||
|
DataArchivePrefix + "selfpost.db",
|
||||||
|
DataArchivePrefix + "opendkim/keys/example.com/selfpost.private",
|
||||||
|
DataArchivePrefix + "sasl/sasldb2",
|
||||||
|
DataArchivePrefix + "postfix/sender_login_maps",
|
||||||
|
ComposeFileName, EnvFileName,
|
||||||
|
CertsDirName + "/fullchain.pem",
|
||||||
|
CertsDirName + "/privkey.pem",
|
||||||
|
} {
|
||||||
|
if _, ok := files[name]; !ok {
|
||||||
|
t.Errorf("archive missing %s", name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A file under certs/ that may not be read is told apart from any other
|
||||||
|
// failure, so the panel can say what to do about it.
|
||||||
|
func TestUnreadableCert(t *testing.T) {
|
||||||
|
root := filepath.Join("srv", "deploy")
|
||||||
|
denied := func(path string) error {
|
||||||
|
return fmt.Errorf("backup: open %s: %w", path, &fs.PathError{Op: "open", Path: path, Err: fs.ErrPermission})
|
||||||
|
}
|
||||||
|
if name, ok := UnreadableCert(denied(filepath.Join(root, "certs", "privkey.pem")), root); !ok || name != "certs/privkey.pem" {
|
||||||
|
t.Errorf("certs/privkey.pem: got %q, %v", name, ok)
|
||||||
|
}
|
||||||
|
if name, ok := UnreadableCert(denied(filepath.Join(root, ".env")), root); ok {
|
||||||
|
t.Errorf(".env was taken for a certificate file: %q", name)
|
||||||
|
}
|
||||||
|
if name, ok := UnreadableCert(denied(filepath.Join("data", "postfix", "x")), root); ok {
|
||||||
|
t.Errorf("a data file was taken for a certificate file: %q", name)
|
||||||
|
}
|
||||||
|
missing := &fs.PathError{Op: "open", Path: filepath.Join(root, "certs", "privkey.pem"), Err: fs.ErrNotExist}
|
||||||
|
if _, ok := UnreadableCert(missing, root); ok {
|
||||||
|
t.Error("a missing file was taken for one that may not be read")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Check finds what Create would stumble over, before anything is written.
|
||||||
|
func TestCheckReportsWhatCannotBeRead(t *testing.T) {
|
||||||
|
dataDir, dbPath, deployRoot := seedDataDir(t)
|
||||||
|
if err := os.Remove(filepath.Join(deployRoot, EnvFileName)); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := Check(Params{DataDir: dataDir, DBPath: dbPath, Version: "1.2.3", DeployRoot: deployRoot}); err == nil {
|
||||||
|
t.Fatal("Check passed with .env missing from the deploy root")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -0,0 +1,109 @@
|
|||||||
|
package dmarc
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/mixeme/selfpost/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
func reportXML(orgName, domain, records string) string {
|
||||||
|
return `<?xml version="1.0"?><feedback><report_metadata><org_name>` + orgName +
|
||||||
|
`</org_name><email>a@example.net</email><report_id>1</report_id>` +
|
||||||
|
`<date_range><begin>1723593600</begin><end>1723680000</end></date_range></report_metadata>` +
|
||||||
|
`<policy_published><domain>` + domain + `</domain><p>none</p></policy_published>` + records + `</feedback>`
|
||||||
|
}
|
||||||
|
|
||||||
|
const oneRecord = `<record><row><source_ip>203.0.113.10</source_ip><count>3</count>` +
|
||||||
|
`<policy_evaluated><disposition>none</disposition><dkim>pass</dkim><spf>pass</spf></policy_evaluated></row>` +
|
||||||
|
`<identifiers><header_from>example.com</header_from></identifiers></record>`
|
||||||
|
|
||||||
|
// The XML is a stranger's: what it may make the relay store is bounded.
|
||||||
|
func TestParseAggregateBoundsWhatAReportHolds(t *testing.T) {
|
||||||
|
long := strings.Repeat("é", 4000)
|
||||||
|
got, err := ParseAggregate([]byte(reportXML(long, "example.com", oneRecord)))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(got.Reporter) > maxFieldLen || !strings.HasPrefix(long, got.Reporter) || got.Reporter == "" {
|
||||||
|
t.Fatalf("reporter name is %d bytes, want the start of the name in at most %d", len(got.Reporter), maxFieldLen)
|
||||||
|
}
|
||||||
|
if strings.ContainsRune(got.Reporter, '�') {
|
||||||
|
t.Fatal("the name was cut in the middle of a character")
|
||||||
|
}
|
||||||
|
|
||||||
|
huge := `<record><row><source_ip>203.0.113.10</source_ip><count>9223372036854775807</count>` +
|
||||||
|
`<policy_evaluated><dkim>pass</dkim></policy_evaluated></row></record>`
|
||||||
|
got, err = ParseAggregate([]byte(reportXML("r", "example.com", huge+huge)))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if got.PassCount != 2*maxRecordCount {
|
||||||
|
t.Fatalf("pass count = %d, want two records clamped to %d each", got.PassCount, maxRecordCount)
|
||||||
|
}
|
||||||
|
|
||||||
|
if _, err := ParseAggregate([]byte(reportXML("r", "example.com", strings.Repeat(oneRecord, maxReportRecords)))); err != nil {
|
||||||
|
t.Fatalf("a report of %d records was refused: %v", maxReportRecords, err)
|
||||||
|
}
|
||||||
|
if _, err := ParseAggregate([]byte(reportXML("r", "example.com", strings.Repeat(oneRecord, maxReportRecords+1)))); err == nil {
|
||||||
|
t.Fatalf("a report of %d records was accepted", maxReportRecords+1)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A report over the bound is one the relay does not keep: dropped, not retried.
|
||||||
|
func TestIngestMessageRejectsAReportWithTooManyRecords(t *testing.T) {
|
||||||
|
st, err := store.Open(t.TempDir() + "/test.db")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer st.Close()
|
||||||
|
if _, err := st.AddDomain("example.com", "selfpost"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
xml := reportXML("r", "example.com", strings.Repeat(oneRecord, maxReportRecords+1))
|
||||||
|
err = IngestMessage(st, strings.NewReader(gzipMessage(t, xml)), "dmarc-reports+example.com@mail.example.com", time.Now().UTC())
|
||||||
|
if !errors.Is(err, ErrRejected) {
|
||||||
|
t.Fatalf("err = %v, want ErrRejected", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Reports filed under one domain do not push another domain's reports out.
|
||||||
|
func TestIngestKeepsEachDomainsReportsApart(t *testing.T) {
|
||||||
|
st, err := store.Open(t.TempDir() + "/test.db")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer st.Close()
|
||||||
|
now := time.Now().UTC()
|
||||||
|
if _, err := st.InsertDMARCReport(store.DMARCReport{
|
||||||
|
Domain: "quiet.example", Reporter: "genuine", ReportID: "1", PeriodBegin: now, PeriodEnd: now, ReceivedAt: now.Add(-time.Hour),
|
||||||
|
}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
for i := 0; i < store.DMARCReportsMaxKeep+20; i++ {
|
||||||
|
if _, err := st.InsertDMARCReport(store.DMARCReport{
|
||||||
|
Domain: "flooded.example", Reporter: "stranger", ReportID: strings.Repeat("x", 1+i), PeriodBegin: now, PeriodEnd: now, ReceivedAt: now,
|
||||||
|
}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err := st.PruneDMARCReports("flooded.example"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
quiet, err := st.ListDMARCReports(store.DMARCReportScope{Domains: []string{"quiet.example"}}, 10)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(quiet) != 1 {
|
||||||
|
t.Fatalf("the other domain has %d reports after the flood, want its 1", len(quiet))
|
||||||
|
}
|
||||||
|
flooded, err := st.ListDMARCReports(store.DMARCReportScope{Domains: []string{"flooded.example"}}, 10000)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(flooded) > store.DMARCReportsMaxKeep {
|
||||||
|
t.Fatalf("the flooded domain keeps %d reports, want at most %d", len(flooded), store.DMARCReportsMaxKeep)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -3,6 +3,7 @@ package dmarc
|
|||||||
import (
|
import (
|
||||||
"bytes"
|
"bytes"
|
||||||
"encoding/base64"
|
"encoding/base64"
|
||||||
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
"io"
|
"io"
|
||||||
"mime"
|
"mime"
|
||||||
@@ -16,6 +17,15 @@ import (
|
|||||||
"github.com/mixeme/selfpost/internal/store"
|
"github.com/mixeme/selfpost/internal/store"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
// ErrRejected marks a message that is not a report this relay keeps and never
|
||||||
|
// will be: it cannot be read as an aggregate report, or it reports on a domain
|
||||||
|
// that is not sent from here, or it came to another domain's address. Sending
|
||||||
|
// it again changes nothing, so the caller drops it. Any other error is the
|
||||||
|
// store's and worth another try.
|
||||||
|
var ErrRejected = errors.New("dmarc report rejected")
|
||||||
|
|
||||||
|
func rejected(err error) error { return fmt.Errorf("%w: %w", ErrRejected, err) }
|
||||||
|
|
||||||
// IngestMessage parses a raw RFC 5322 message from r and stores any DMARC
|
// IngestMessage parses a raw RFC 5322 message from r and stores any DMARC
|
||||||
// aggregate attachment it finds.
|
// aggregate attachment it finds.
|
||||||
//
|
//
|
||||||
@@ -34,11 +44,11 @@ func IngestMessage(st *store.Store, r io.Reader, recipient string, receivedAt ti
|
|||||||
}
|
}
|
||||||
payload, err := extractAggregatePayload(raw)
|
payload, err := extractAggregatePayload(raw)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return rejected(err)
|
||||||
}
|
}
|
||||||
parsed, err := ParseAggregate(payload)
|
parsed, err := ParseAggregate(payload)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return rejected(err)
|
||||||
}
|
}
|
||||||
recipient = strings.ToLower(strings.TrimSpace(recipient))
|
recipient = strings.ToLower(strings.TrimSpace(recipient))
|
||||||
if err := checkReportDomain(st, parsed.Domain, recipient); err != nil {
|
if err := checkReportDomain(st, parsed.Domain, recipient); err != nil {
|
||||||
@@ -74,7 +84,7 @@ func IngestMessage(st *store.Store, r io.Reader, recipient string, receivedAt ti
|
|||||||
if _, err := st.InsertDMARCReport(rep); err != nil {
|
if _, err := st.InsertDMARCReport(rep); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
return st.PruneDMARCReports()
|
return st.PruneDMARCReports(rep.Domain)
|
||||||
}
|
}
|
||||||
|
|
||||||
// checkReportDomain refuses a report whose policy domain is not one of this
|
// checkReportDomain refuses a report whose policy domain is not one of this
|
||||||
@@ -82,17 +92,17 @@ func IngestMessage(st *store.Store, r io.Reader, recipient string, receivedAt ti
|
|||||||
// delivered to.
|
// delivered to.
|
||||||
func checkReportDomain(st *store.Store, domain, recipient string) error {
|
func checkReportDomain(st *store.Store, domain, recipient string) error {
|
||||||
if domain == "" {
|
if domain == "" {
|
||||||
return fmt.Errorf("dmarc report: no policy domain")
|
return rejected(fmt.Errorf("no policy domain"))
|
||||||
}
|
}
|
||||||
if tagged := TaggedDomain(recipient); tagged != "" && tagged != domain {
|
if tagged := TaggedDomain(recipient); tagged != "" && tagged != domain {
|
||||||
return fmt.Errorf("dmarc report: domain %q does not match the address it was sent to (%s)", domain, recipient)
|
return rejected(fmt.Errorf("domain %q does not match the address it was sent to (%s)", domain, recipient))
|
||||||
}
|
}
|
||||||
ok, err := st.DomainExists(domain)
|
ok, err := st.DomainExists(domain)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
if !ok {
|
if !ok {
|
||||||
return fmt.Errorf("dmarc report: %q is not a domain configured on this relay", domain)
|
return rejected(fmt.Errorf("%q is not a domain configured on this relay", domain))
|
||||||
}
|
}
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|||||||
+50
-15
@@ -9,6 +9,7 @@ import (
|
|||||||
"io"
|
"io"
|
||||||
"strings"
|
"strings"
|
||||||
"time"
|
"time"
|
||||||
|
"unicode/utf8"
|
||||||
)
|
)
|
||||||
|
|
||||||
// ParsedReport is the structured form of a DMARC aggregate XML payload.
|
// ParsedReport is the structured form of a DMARC aggregate XML payload.
|
||||||
@@ -74,6 +75,31 @@ type feedbackXML struct {
|
|||||||
} `xml:"record"`
|
} `xml:"record"`
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// What a report may hold. The XML comes from anyone who knows a report address,
|
||||||
|
// and eight megabytes of it (maxAggregateXML) fit some 160 000 empty records or
|
||||||
|
// one name of eight megabytes - every one of them a row or a table cell later.
|
||||||
|
// A real report has tens of records, a few thousand from the largest
|
||||||
|
// receivers, and no field longer than a line.
|
||||||
|
const (
|
||||||
|
maxReportRecords = 5000
|
||||||
|
maxFieldLen = 255
|
||||||
|
maxRecordCount = 1_000_000_000
|
||||||
|
)
|
||||||
|
|
||||||
|
// field trims a text value of the report and cuts it to maxFieldLen bytes, on
|
||||||
|
// a character boundary.
|
||||||
|
func field(s string) string {
|
||||||
|
s = strings.TrimSpace(s)
|
||||||
|
if len(s) <= maxFieldLen {
|
||||||
|
return s
|
||||||
|
}
|
||||||
|
s = s[:maxFieldLen]
|
||||||
|
for len(s) > 0 && !utf8.ValidString(s) {
|
||||||
|
s = s[:len(s)-1]
|
||||||
|
}
|
||||||
|
return s
|
||||||
|
}
|
||||||
|
|
||||||
// ParseAggregate decodes gzip- or zip-compressed, or raw, DMARC aggregate XML.
|
// ParseAggregate decodes gzip- or zip-compressed, or raw, DMARC aggregate XML.
|
||||||
func ParseAggregate(raw []byte) (ParsedReport, error) {
|
func ParseAggregate(raw []byte) (ParsedReport, error) {
|
||||||
data, err := decompress(raw)
|
data, err := decompress(raw)
|
||||||
@@ -87,34 +113,43 @@ func ParseAggregate(raw []byte) (ParsedReport, error) {
|
|||||||
if doc.Metadata.ReportID == "" || doc.Policy.Domain == "" {
|
if doc.Metadata.ReportID == "" || doc.Policy.Domain == "" {
|
||||||
return ParsedReport{}, fmt.Errorf("dmarc xml: missing report_id or domain")
|
return ParsedReport{}, fmt.Errorf("dmarc xml: missing report_id or domain")
|
||||||
}
|
}
|
||||||
|
if len(doc.Records) > maxReportRecords {
|
||||||
|
return ParsedReport{}, fmt.Errorf("dmarc xml: %d records, more than the %d a report may hold", len(doc.Records), maxReportRecords)
|
||||||
|
}
|
||||||
|
// Cut, not refused: a domain name is at most 253 bytes, so a longer one
|
||||||
|
// names nothing this relay sends for and is turned away as such.
|
||||||
out := ParsedReport{
|
out := ParsedReport{
|
||||||
Reporter: strings.TrimSpace(doc.Metadata.OrgName),
|
Reporter: field(doc.Metadata.OrgName),
|
||||||
ReportID: strings.TrimSpace(doc.Metadata.ReportID),
|
ReportID: field(doc.Metadata.ReportID),
|
||||||
ContactEmail: strings.TrimSpace(doc.Metadata.Email),
|
ContactEmail: field(doc.Metadata.Email),
|
||||||
PeriodBegin: time.Unix(doc.Metadata.DateRange.Begin, 0).UTC(),
|
PeriodBegin: time.Unix(doc.Metadata.DateRange.Begin, 0).UTC(),
|
||||||
PeriodEnd: time.Unix(doc.Metadata.DateRange.End, 0).UTC(),
|
PeriodEnd: time.Unix(doc.Metadata.DateRange.End, 0).UTC(),
|
||||||
Domain: strings.ToLower(strings.TrimSpace(doc.Policy.Domain)),
|
Domain: strings.ToLower(field(doc.Policy.Domain)),
|
||||||
PolicyP: strings.TrimSpace(doc.Policy.P),
|
PolicyP: field(doc.Policy.P),
|
||||||
PolicySP: strings.TrimSpace(doc.Policy.SP),
|
PolicySP: field(doc.Policy.SP),
|
||||||
PolicyPct: doc.Policy.Pct,
|
PolicyPct: doc.Policy.Pct,
|
||||||
PolicyADKIM: strings.TrimSpace(doc.Policy.ADKIM),
|
PolicyADKIM: field(doc.Policy.ADKIM),
|
||||||
PolicyASPF: strings.TrimSpace(doc.Policy.ASPF),
|
PolicyASPF: field(doc.Policy.ASPF),
|
||||||
}
|
}
|
||||||
if out.PolicyPct == 0 {
|
if out.PolicyPct == 0 {
|
||||||
out.PolicyPct = 100
|
out.PolicyPct = 100
|
||||||
}
|
}
|
||||||
for _, rec := range doc.Records {
|
for _, rec := range doc.Records {
|
||||||
row := ParsedRecord{
|
row := ParsedRecord{
|
||||||
SourceIP: strings.TrimSpace(rec.Row.SourceIP),
|
SourceIP: field(rec.Row.SourceIP),
|
||||||
Count: rec.Row.Count,
|
Count: rec.Row.Count,
|
||||||
Disposition: strings.TrimSpace(rec.Row.Policy.Disposition),
|
Disposition: field(rec.Row.Policy.Disposition),
|
||||||
SPFResult: strings.TrimSpace(rec.Row.Policy.SPF),
|
SPFResult: field(rec.Row.Policy.SPF),
|
||||||
DKIMResult: strings.TrimSpace(rec.Row.Policy.DKIM),
|
DKIMResult: field(rec.Row.Policy.DKIM),
|
||||||
HeaderFrom: strings.TrimSpace(rec.Identifiers.HeaderFrom),
|
HeaderFrom: field(rec.Identifiers.HeaderFrom),
|
||||||
}
|
}
|
||||||
if row.Count <= 0 {
|
if row.Count <= 0 {
|
||||||
row.Count = 1
|
row.Count = 1
|
||||||
}
|
}
|
||||||
|
// Bounded so that the sums of a report cannot wrap around.
|
||||||
|
if row.Count > maxRecordCount {
|
||||||
|
row.Count = maxRecordCount
|
||||||
|
}
|
||||||
if dmarcRecordPasses(row) {
|
if dmarcRecordPasses(row) {
|
||||||
out.PassCount += row.Count
|
out.PassCount += row.Count
|
||||||
} else {
|
} else {
|
||||||
@@ -172,12 +207,12 @@ func unzip(raw []byte) ([]byte, error) {
|
|||||||
}
|
}
|
||||||
rc, err := f.Open()
|
rc, err := f.Open()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, fmt.Errorf("dmarc zip open %s: %w", f.Name, err)
|
return nil, fmt.Errorf("dmarc zip open %q: %w", field(f.Name), err)
|
||||||
}
|
}
|
||||||
data, err := io.ReadAll(io.LimitReader(rc, maxAggregateXML))
|
data, err := io.ReadAll(io.LimitReader(rc, maxAggregateXML))
|
||||||
rc.Close()
|
rc.Close()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, fmt.Errorf("dmarc zip read %s: %w", f.Name, err)
|
return nil, fmt.Errorf("dmarc zip read %q: %w", field(f.Name), err)
|
||||||
}
|
}
|
||||||
return data, nil
|
return data, nil
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -4,6 +4,7 @@ import (
|
|||||||
"archive/zip"
|
"archive/zip"
|
||||||
"bytes"
|
"bytes"
|
||||||
"compress/gzip"
|
"compress/gzip"
|
||||||
|
"errors"
|
||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
@@ -126,8 +127,9 @@ func TestIngestMessageRejectsUnknownDomain(t *testing.T) {
|
|||||||
}
|
}
|
||||||
defer st.Close()
|
defer st.Close()
|
||||||
|
|
||||||
if err := IngestMessage(st, bytes.NewBufferString(gzipMessage(t, sampleXML)), "dmarc-reports@mail.example.com", time.Now().UTC()); err == nil {
|
err = IngestMessage(st, bytes.NewBufferString(gzipMessage(t, sampleXML)), "dmarc-reports@mail.example.com", time.Now().UTC())
|
||||||
t.Fatal("report for an unconfigured domain was accepted")
|
if !errors.Is(err, ErrRejected) {
|
||||||
|
t.Fatalf("report for an unconfigured domain: err = %v, want ErrRejected", err)
|
||||||
}
|
}
|
||||||
list, err := st.ListDMARCReports(store.DMARCReportScope{AllDomains: true}, 10)
|
list, err := st.ListDMARCReports(store.DMARCReportScope{AllDomains: true}, 10)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -155,8 +157,47 @@ func TestIngestMessageRejectsTagMismatch(t *testing.T) {
|
|||||||
|
|
||||||
err = IngestMessage(st, bytes.NewBufferString(gzipMessage(t, sampleXML)),
|
err = IngestMessage(st, bytes.NewBufferString(gzipMessage(t, sampleXML)),
|
||||||
"dmarc-reports+other.example@mail.example.com", time.Now().UTC())
|
"dmarc-reports+other.example@mail.example.com", time.Now().UTC())
|
||||||
if err == nil {
|
if !errors.Is(err, ErrRejected) {
|
||||||
t.Fatal("report was accepted at an address tagged for another domain")
|
t.Fatalf("report at an address tagged for another domain: err = %v, want ErrRejected", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// What cannot be read as a report is rejected, not put off: Postfix would
|
||||||
|
// offer the same bytes again for days.
|
||||||
|
func TestIngestMessageRejectsWhatIsNotAReport(t *testing.T) {
|
||||||
|
st, err := store.Open(t.TempDir() + "/test.db")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer st.Close()
|
||||||
|
if _, err := st.AddDomain("example.com", "selfpost"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
for name, msg := range map[string]string{
|
||||||
|
"plain text": "From: someone@example.net\r\nSubject: hello\r\n\r\nnot a report\r\n",
|
||||||
|
"no headers": "garbage",
|
||||||
|
"broken gzip": "From: a@example.net\r\nContent-Type: application/gzip\r\n\r\nnot gzip at all",
|
||||||
|
} {
|
||||||
|
err := IngestMessage(st, bytes.NewBufferString(msg), "dmarc-reports+example.com@mail.example.com", time.Now().UTC())
|
||||||
|
if !errors.Is(err, ErrRejected) {
|
||||||
|
t.Errorf("%s: err = %v, want ErrRejected", name, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A failing store is not the report's fault: the message is to be tried again.
|
||||||
|
func TestIngestMessageStoreFailureIsNotARejection(t *testing.T) {
|
||||||
|
st, err := store.Open(t.TempDir() + "/test.db")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := st.AddDomain("example.com", "selfpost"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
st.Close()
|
||||||
|
err = IngestMessage(st, bytes.NewBufferString(gzipMessage(t, sampleXML)), "dmarc-reports+example.com@mail.example.com", time.Now().UTC())
|
||||||
|
if err == nil || errors.Is(err, ErrRejected) {
|
||||||
|
t.Fatalf("closed store: err = %v, want an error that is not ErrRejected", err)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,447 @@
|
|||||||
|
package inbound
|
||||||
|
|
||||||
|
import (
|
||||||
|
"crypto/rand"
|
||||||
|
"encoding/hex"
|
||||||
|
"fmt"
|
||||||
|
"net/netip"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"sort"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
"time"
|
||||||
|
"unicode/utf8"
|
||||||
|
|
||||||
|
"github.com/mixeme/selfpost/internal/atomicfile"
|
||||||
|
"github.com/mixeme/selfpost/internal/configsafe"
|
||||||
|
"github.com/mixeme/selfpost/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
// ListStore is the slice of the store the filter lists need. *store.Store
|
||||||
|
// satisfies it.
|
||||||
|
type ListStore interface {
|
||||||
|
AddSpamListEntry(list, kind, value, note string) (store.SpamListEntry, error)
|
||||||
|
ListSpamListEntries() ([]store.SpamListEntry, error)
|
||||||
|
DeleteSpamListEntry(id int64) error
|
||||||
|
ListSpamListHits(marks ...string) ([]store.SpamListHit, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Lists owns the allow and deny lists of the external spam filter: the rows in
|
||||||
|
// SQLite and the map files the filter reads. SelfPost does not consult the
|
||||||
|
// lists itself - it writes them out, six files in one directory, one entry a
|
||||||
|
// line, and the filter (rspamd, deploy/antispam/rspamd/multimap.conf) rereads
|
||||||
|
// a file when it changes. Which list wins when both match is the filter's
|
||||||
|
// configuration: deny.
|
||||||
|
type Lists struct {
|
||||||
|
store ListStore
|
||||||
|
dir string
|
||||||
|
// mu makes a change of the table and the rewrite of the files one step.
|
||||||
|
// Two requests at once could otherwise write the files in the other order
|
||||||
|
// than they changed the table - the first to read the entries being the
|
||||||
|
// last to write them - and leave the filter with an entry the page no
|
||||||
|
// longer lists. One Lists serves the panel, so a mutex is enough.
|
||||||
|
mu sync.Mutex
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewLists builds the filter lists over the store and the map directory
|
||||||
|
// (/data/antispam).
|
||||||
|
func NewLists(st ListStore, dir string) *Lists {
|
||||||
|
return &Lists{store: st, dir: dir}
|
||||||
|
}
|
||||||
|
|
||||||
|
// noteMaxRunes caps the operator's comment on an entry.
|
||||||
|
const noteMaxRunes = 200
|
||||||
|
|
||||||
|
// mapFiles names the file of every (list, kind). An IP map and a sender map
|
||||||
|
// are different things to the filter, and a domain entry is matched against
|
||||||
|
// another part of the sender than an address, so each has its own file.
|
||||||
|
var mapFiles = map[[2]string]string{
|
||||||
|
{store.SpamListAllow, store.SpamEntryIP}: "allow_ip.map",
|
||||||
|
{store.SpamListAllow, store.SpamEntryAddress}: "allow_from.map",
|
||||||
|
{store.SpamListAllow, store.SpamEntryDomain}: "allow_domain.map",
|
||||||
|
{store.SpamListDeny, store.SpamEntryIP}: "deny_ip.map",
|
||||||
|
{store.SpamListDeny, store.SpamEntryAddress}: "deny_from.map",
|
||||||
|
{store.SpamListDeny, store.SpamEntryDomain}: "deny_domain.map",
|
||||||
|
}
|
||||||
|
|
||||||
|
// Entries returns every entry of both lists.
|
||||||
|
func (l *Lists) Entries() ([]store.SpamListEntry, error) {
|
||||||
|
return l.store.ListSpamListEntries()
|
||||||
|
}
|
||||||
|
|
||||||
|
// Add validates an entry, stores it normalised and rewrites the maps. A
|
||||||
|
// refusal is a *ValidationError; an entry already on that list is
|
||||||
|
// store.ErrSpamListEntryExists.
|
||||||
|
func (l *Lists) Add(list, kind, rawValue, rawNote string) (store.SpamListEntry, error) {
|
||||||
|
if list != store.SpamListAllow && list != store.SpamListDeny {
|
||||||
|
return store.SpamListEntry{}, invalid("choose the allow or the deny list")
|
||||||
|
}
|
||||||
|
value, err := normalizeListValue(kind, rawValue)
|
||||||
|
if err != nil {
|
||||||
|
return store.SpamListEntry{}, err
|
||||||
|
}
|
||||||
|
note, err := normalizeNote(rawNote)
|
||||||
|
if err != nil {
|
||||||
|
return store.SpamListEntry{}, err
|
||||||
|
}
|
||||||
|
l.mu.Lock()
|
||||||
|
defer l.mu.Unlock()
|
||||||
|
e, err := l.store.AddSpamListEntry(list, kind, value, note)
|
||||||
|
if err != nil {
|
||||||
|
return store.SpamListEntry{}, err
|
||||||
|
}
|
||||||
|
if err := l.sync(); err != nil {
|
||||||
|
// An entry the filter never got is worse than no entry: the page
|
||||||
|
// would show a rule that does nothing. The files are written again
|
||||||
|
// after the row is taken back, since some of the six may already
|
||||||
|
// hold it; if that fails too, the next change or start puts it right.
|
||||||
|
_ = l.store.DeleteSpamListEntry(e.ID)
|
||||||
|
_ = l.sync()
|
||||||
|
return store.SpamListEntry{}, err
|
||||||
|
}
|
||||||
|
return e, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Delete removes an entry and rewrites the maps. If the maps cannot be
|
||||||
|
// written the entry is put back: an entry that is gone from the page and
|
||||||
|
// still in the filter's file could not even be removed a second time.
|
||||||
|
func (l *Lists) Delete(id int64) error {
|
||||||
|
l.mu.Lock()
|
||||||
|
defer l.mu.Unlock()
|
||||||
|
entries, err := l.store.ListSpamListEntries()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
var gone *store.SpamListEntry
|
||||||
|
for i := range entries {
|
||||||
|
if entries[i].ID == id {
|
||||||
|
gone = &entries[i]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err := l.store.DeleteSpamListEntry(id); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := l.sync(); err != nil {
|
||||||
|
if gone != nil {
|
||||||
|
_, _ = l.store.AddSpamListEntry(gone.List, gone.Kind, gone.Value, gone.Note)
|
||||||
|
}
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Sync writes all six map files from SQLite, each atomically, so the filter
|
||||||
|
// reads a whole old file or a whole new one. Every file is written even when
|
||||||
|
// empty: the filter's configuration names all six.
|
||||||
|
func (l *Lists) Sync() error {
|
||||||
|
l.mu.Lock()
|
||||||
|
defer l.mu.Unlock()
|
||||||
|
return l.sync()
|
||||||
|
}
|
||||||
|
|
||||||
|
// sync is Sync for a caller that holds mu.
|
||||||
|
func (l *Lists) sync() error {
|
||||||
|
entries, err := l.store.ListSpamListEntries()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
lines := make(map[string][]string, len(mapFiles))
|
||||||
|
for _, name := range mapFiles {
|
||||||
|
lines[name] = nil
|
||||||
|
}
|
||||||
|
for _, e := range entries {
|
||||||
|
name, ok := mapFiles[[2]string{e.List, e.Kind}]
|
||||||
|
if !ok {
|
||||||
|
return fmt.Errorf("filter list entry %d: unknown list %q or kind %q", e.ID, e.List, e.Kind)
|
||||||
|
}
|
||||||
|
// Values were validated on the way in; this is the writer's own guard
|
||||||
|
// against a row that got into the table some other way. In a map file a
|
||||||
|
// newline ends the entry, whitespace starts its value and # a comment.
|
||||||
|
if err := configsafe.Token("filter list entry", e.Value, " \t\r\n#"); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
lines[name] = append(lines[name], e.Value)
|
||||||
|
}
|
||||||
|
// The filter runs under another user in another container: the directory
|
||||||
|
// and the files are readable by anyone. They hold addresses, not secrets.
|
||||||
|
if err := os.MkdirAll(l.dir, 0o755); err != nil {
|
||||||
|
return fmt.Errorf("create %s: %w", l.dir, err)
|
||||||
|
}
|
||||||
|
for name, vals := range lines {
|
||||||
|
sort.Strings(vals)
|
||||||
|
var b strings.Builder
|
||||||
|
b.WriteString("# Written by SelfPost (Inbound > Filter lists). Do not edit: the next change in the panel replaces this file.\n")
|
||||||
|
for _, v := range vals {
|
||||||
|
b.WriteString(v)
|
||||||
|
b.WriteByte('\n')
|
||||||
|
}
|
||||||
|
if err := atomicfile.Write(filepath.Join(l.dir, name), []byte(b.String()), 0o644); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// LastWritten is when the maps were last written: the newest modification
|
||||||
|
// time of the six files, the zero time while none exists (the filter lists
|
||||||
|
// have never been touched, or the directory is gone).
|
||||||
|
func (l *Lists) LastWritten() time.Time {
|
||||||
|
var newest time.Time
|
||||||
|
for _, name := range mapFiles {
|
||||||
|
if fi, err := os.Stat(filepath.Join(l.dir, name)); err == nil && fi.ModTime().After(newest) {
|
||||||
|
newest = fi.ModTime()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return newest
|
||||||
|
}
|
||||||
|
|
||||||
|
// Listable reports whether value could be added to a list as an entry of kind.
|
||||||
|
// The Inbound log asks before it offers a row's sender to the lists: an
|
||||||
|
// envelope sender may be spelt in ways the lists do not take.
|
||||||
|
func Listable(kind, value string) bool {
|
||||||
|
_, err := normalizeListValue(kind, value)
|
||||||
|
return err == nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// normalizeListValue validates one entry and returns the form it is stored and
|
||||||
|
// written in: lower case for an address or a domain, the canonical text of an
|
||||||
|
// address or a network for an IP.
|
||||||
|
func normalizeListValue(kind, raw string) (string, error) {
|
||||||
|
v := strings.ToLower(strings.TrimSpace(raw))
|
||||||
|
switch kind {
|
||||||
|
case store.SpamEntryAddress:
|
||||||
|
if v == "" {
|
||||||
|
return "", invalid("sender address is required")
|
||||||
|
}
|
||||||
|
if len(v) > 254 {
|
||||||
|
return "", invalid("sender address must be at most 254 characters")
|
||||||
|
}
|
||||||
|
at := strings.LastIndexByte(v, '@')
|
||||||
|
if at <= 0 || at >= len(v)-1 {
|
||||||
|
return "", invalid("%q is not a valid email address", v)
|
||||||
|
}
|
||||||
|
local, domain := v[:at], v[at+1:]
|
||||||
|
if local[0] == '.' || local[len(local)-1] == '.' || len(local) > 64 {
|
||||||
|
return "", invalid("%q: invalid local part", v)
|
||||||
|
}
|
||||||
|
for i := 0; i < len(local); i++ {
|
||||||
|
c := local[i]
|
||||||
|
lower := c >= 'a' && c <= 'z'
|
||||||
|
digit := c >= '0' && c <= '9'
|
||||||
|
// '=' for the rewritten senders of forwarders (SRS0=…).
|
||||||
|
if !lower && !digit && !strings.ContainsRune("._+-=", rune(c)) {
|
||||||
|
return "", invalid("%q: local part contains invalid characters", v)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err := checkDomain(domain); err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
return v, nil
|
||||||
|
case store.SpamEntryDomain:
|
||||||
|
// "@example.com" is how a domain is often written in such a list.
|
||||||
|
v = strings.TrimPrefix(v, "@")
|
||||||
|
if err := checkDomain(v); err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
return v, nil
|
||||||
|
case store.SpamEntryIP:
|
||||||
|
if v == "" {
|
||||||
|
return "", invalid("address or network is required")
|
||||||
|
}
|
||||||
|
if strings.Contains(v, "/") {
|
||||||
|
p, err := netip.ParsePrefix(v)
|
||||||
|
if err != nil {
|
||||||
|
return "", invalid("%q is not a network in CIDR form (e.g. 192.0.2.0/24)", v)
|
||||||
|
}
|
||||||
|
if p.Addr().Is4In6() {
|
||||||
|
return "", invalid("%q: write an IPv4 network as IPv4 (e.g. 192.0.2.0/24)", v)
|
||||||
|
}
|
||||||
|
if p.Bits() == 0 {
|
||||||
|
return "", invalid("%q is every address there is", v)
|
||||||
|
}
|
||||||
|
// A network of one address is that address: 192.0.2.7/32 and
|
||||||
|
// 192.0.2.7 are one entry, not two that look different.
|
||||||
|
if p.Bits() == p.Addr().BitLen() {
|
||||||
|
return p.Addr().String(), nil
|
||||||
|
}
|
||||||
|
return p.Masked().String(), nil
|
||||||
|
}
|
||||||
|
a, err := netip.ParseAddr(v)
|
||||||
|
if err != nil || a.Zone() != "" {
|
||||||
|
return "", invalid("%q is not an IP address or a network in CIDR form", v)
|
||||||
|
}
|
||||||
|
return a.Unmap().String(), nil
|
||||||
|
default:
|
||||||
|
return "", invalid("choose what the entry is: a sender address, a sender domain or a client address")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// normalizeNote trims the operator's comment and refuses what a single line of
|
||||||
|
// text cannot be.
|
||||||
|
func normalizeNote(raw string) (string, error) {
|
||||||
|
note := strings.TrimSpace(raw)
|
||||||
|
if utf8.RuneCountInString(note) > noteMaxRunes {
|
||||||
|
return "", invalid("note must be at most %d characters", noteMaxRunes)
|
||||||
|
}
|
||||||
|
for _, r := range note {
|
||||||
|
if r < 0x20 || r == 0x7f || r == utf8.RuneError {
|
||||||
|
return "", invalid("note must be one line of plain text")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return note, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// What a list leaves in the journal when it decides a message - the two ends
|
||||||
|
// of deploy/antispam/rspamd/multimap.conf this package depends on. A deny rule
|
||||||
|
// refuses with a message, which the log tailer files as the reply of a
|
||||||
|
// rejected row. An allow rule adds a symbol named after the kind of entry,
|
||||||
|
// which the journal milter keeps in the verdict of an accepted row.
|
||||||
|
const (
|
||||||
|
denyReplyMark = "this server's deny list"
|
||||||
|
allowMark = "SELFPOST_ALLOW_"
|
||||||
|
)
|
||||||
|
|
||||||
|
var allowSymbol = map[string]string{
|
||||||
|
store.SpamEntryIP: "SELFPOST_ALLOW_IP(",
|
||||||
|
store.SpamEntryAddress: "SELFPOST_ALLOW_FROM(",
|
||||||
|
store.SpamEntryDomain: "SELFPOST_ALLOW_DOMAIN(",
|
||||||
|
}
|
||||||
|
|
||||||
|
// Usage is how often an entry decided a message, as far back as the journal
|
||||||
|
// reaches, and when it last did. The zero value is an entry that never did.
|
||||||
|
type Usage struct {
|
||||||
|
Hits int
|
||||||
|
Last time.Time
|
||||||
|
}
|
||||||
|
|
||||||
|
// Usage counts, for every entry, the journal rows it decided. Nothing is
|
||||||
|
// stored for this: the journal already says which list decided a message and
|
||||||
|
// who the message was from, and an entry is matched against that the way the
|
||||||
|
// filter matches it - an address whole, a domain as the sender's domain, an
|
||||||
|
// address or network against the client's. So the count reaches back as far
|
||||||
|
// as the journal does (its retention, or less when it is full), and a row
|
||||||
|
// counts for every entry of the deciding list that fits it: the journal does
|
||||||
|
// not say which of two fitting entries the filter met first.
|
||||||
|
//
|
||||||
|
// A counter kept with each entry would not depend on the journal; the plan
|
||||||
|
// (docs/plans/inbound-antispam-panel.md, Filter list usage) says when that is
|
||||||
|
// worth a migration.
|
||||||
|
func (l *Lists) Usage() (map[int64]Usage, error) {
|
||||||
|
entries, err := l.store.ListSpamListEntries()
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
out := make(map[int64]Usage, len(entries))
|
||||||
|
if len(entries) == 0 {
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
hits, err := l.store.ListSpamListHits(denyReplyMark, allowMark)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
type matcher struct {
|
||||||
|
id int64
|
||||||
|
list string
|
||||||
|
kind string
|
||||||
|
value string
|
||||||
|
prefix netip.Prefix
|
||||||
|
}
|
||||||
|
matchers := make([]matcher, 0, len(entries))
|
||||||
|
for _, e := range entries {
|
||||||
|
m := matcher{id: e.ID, list: e.List, kind: e.Kind, value: e.Value}
|
||||||
|
if e.Kind == store.SpamEntryIP {
|
||||||
|
if p, err := netip.ParsePrefix(e.Value); err == nil {
|
||||||
|
m.prefix = p
|
||||||
|
} else if a, err := netip.ParseAddr(e.Value); err == nil {
|
||||||
|
m.prefix = netip.PrefixFrom(a, a.BitLen())
|
||||||
|
} else {
|
||||||
|
continue // not an entry this package wrote
|
||||||
|
}
|
||||||
|
}
|
||||||
|
matchers = append(matchers, m)
|
||||||
|
}
|
||||||
|
for _, h := range hits {
|
||||||
|
from := strings.ToLower(h.From)
|
||||||
|
domain := ""
|
||||||
|
if i := strings.LastIndexByte(from, '@'); i >= 0 {
|
||||||
|
domain = from[i+1:]
|
||||||
|
}
|
||||||
|
client, clientErr := netip.ParseAddr(h.ClientIP)
|
||||||
|
client = client.Unmap()
|
||||||
|
denied := h.Decision != store.SpamDecisionAccept && strings.Contains(h.Detail, denyReplyMark)
|
||||||
|
for _, m := range matchers {
|
||||||
|
switch m.list {
|
||||||
|
case store.SpamListDeny:
|
||||||
|
if !denied {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
case store.SpamListAllow:
|
||||||
|
if h.Decision != store.SpamDecisionAccept || !strings.Contains(h.Detail, allowSymbol[m.kind]) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
}
|
||||||
|
fits := false
|
||||||
|
switch m.kind {
|
||||||
|
case store.SpamEntryAddress:
|
||||||
|
fits = from == m.value
|
||||||
|
case store.SpamEntryDomain:
|
||||||
|
fits = domain == m.value
|
||||||
|
case store.SpamEntryIP:
|
||||||
|
fits = clientErr == nil && m.prefix.Contains(client)
|
||||||
|
}
|
||||||
|
if !fits {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
u := out[m.id]
|
||||||
|
u.Hits++
|
||||||
|
if h.CreatedAt.After(u.Last) {
|
||||||
|
u.Last = h.CreatedAt
|
||||||
|
}
|
||||||
|
out[m.id] = u
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// FilterKeyFile is the file, beside the list files, that holds the key the
|
||||||
|
// spam filter signs messages with.
|
||||||
|
const FilterKeyFile = "filter.key"
|
||||||
|
|
||||||
|
// FilterKey returns the key the spam filter signs each message with, creating
|
||||||
|
// it on first use: 32 random bytes as 64 hex characters on one line, in the
|
||||||
|
// directory the filter's container mounts.
|
||||||
|
//
|
||||||
|
// The filter is attached so that mail flows when it is down. Then nothing
|
||||||
|
// removes verdict headers a sender wrote himself, and the inbound journal
|
||||||
|
// would record his verdict as the filter's. So the filter signs each message
|
||||||
|
// with this key and the journal reads a verdict only next to a signature it
|
||||||
|
// can recompute (internal/milter, FilterSignature).
|
||||||
|
//
|
||||||
|
// The file is readable by anyone on the host, like the list files and for the
|
||||||
|
// same reason: the filter is another user in another container. What the key
|
||||||
|
// protects is the truth of a monitoring page, against senders on the internet;
|
||||||
|
// someone with a shell on the host is not who it is kept from.
|
||||||
|
func FilterKey(dir string) (string, error) {
|
||||||
|
path := filepath.Join(dir, FilterKeyFile)
|
||||||
|
if raw, err := os.ReadFile(path); err == nil {
|
||||||
|
if key := strings.TrimSpace(string(raw)); len(key) >= 32 && !strings.ContainsAny(key, " \t\r\n") {
|
||||||
|
return key, nil
|
||||||
|
}
|
||||||
|
// Empty or cut short: not a key anyone should sign with. Replaced below.
|
||||||
|
} else if !os.IsNotExist(err) {
|
||||||
|
return "", fmt.Errorf("read %s: %w", path, err)
|
||||||
|
}
|
||||||
|
if err := os.MkdirAll(dir, 0o755); err != nil {
|
||||||
|
return "", fmt.Errorf("create %s: %w", dir, err)
|
||||||
|
}
|
||||||
|
buf := make([]byte, 32)
|
||||||
|
if _, err := rand.Read(buf); err != nil {
|
||||||
|
return "", fmt.Errorf("generate the filter key: %w", err)
|
||||||
|
}
|
||||||
|
key := hex.EncodeToString(buf)
|
||||||
|
if err := atomicfile.Write(path, []byte(key+"\n"), 0o644); err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
return key, nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,440 @@
|
|||||||
|
package inbound
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/mixeme/selfpost/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
func testLists(t *testing.T) (*Lists, *store.Store, string) {
|
||||||
|
t.Helper()
|
||||||
|
st, err := store.Open(filepath.Join(t.TempDir(), "test.db"))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
t.Cleanup(func() { st.Close() })
|
||||||
|
dir := filepath.Join(t.TempDir(), "antispam")
|
||||||
|
return NewLists(st, dir), st, dir
|
||||||
|
}
|
||||||
|
|
||||||
|
// mapLines reads a map file without its comment line.
|
||||||
|
func mapLines(t *testing.T, dir, name string) []string {
|
||||||
|
t.Helper()
|
||||||
|
raw, err := os.ReadFile(filepath.Join(dir, name))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("read %s: %v", name, err)
|
||||||
|
}
|
||||||
|
var out []string
|
||||||
|
for _, line := range strings.Split(strings.TrimRight(string(raw), "\n"), "\n") {
|
||||||
|
if !strings.HasPrefix(line, "#") {
|
||||||
|
out = append(out, line)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestNormalizeListValue(t *testing.T) {
|
||||||
|
for _, c := range []struct{ kind, raw, want string }{
|
||||||
|
{store.SpamEntryAddress, " Billing@Vendor.Example ", "billing@vendor.example"},
|
||||||
|
{store.SpamEntryAddress, "srs0=ab12=cd=example.org=user@forwarder.example", "srs0=ab12=cd=example.org=user@forwarder.example"},
|
||||||
|
{store.SpamEntryDomain, "@Evil.Example", "evil.example"},
|
||||||
|
{store.SpamEntryDomain, "sub.evil.example", "sub.evil.example"},
|
||||||
|
{store.SpamEntryIP, "192.0.2.7", "192.0.2.7"},
|
||||||
|
{store.SpamEntryIP, "192.0.2.77/24", "192.0.2.0/24"},
|
||||||
|
{store.SpamEntryIP, "192.0.2.7/32", "192.0.2.7"},
|
||||||
|
{store.SpamEntryIP, "2001:db8::1/128", "2001:db8::1"},
|
||||||
|
{store.SpamEntryIP, "::ffff:192.0.2.7", "192.0.2.7"},
|
||||||
|
{store.SpamEntryIP, "2001:DB8:0:0::1", "2001:db8::1"},
|
||||||
|
{store.SpamEntryIP, "2001:db8:1::5/48", "2001:db8:1::/48"},
|
||||||
|
} {
|
||||||
|
got, err := normalizeListValue(c.kind, c.raw)
|
||||||
|
if err != nil || got != c.want {
|
||||||
|
t.Errorf("normalizeListValue(%s, %q) = %q, %v; want %q", c.kind, c.raw, got, err, c.want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, c := range []struct{ kind, raw string }{
|
||||||
|
{store.SpamEntryAddress, ""},
|
||||||
|
{store.SpamEntryAddress, "no-at-sign"},
|
||||||
|
{store.SpamEntryAddress, "@example.com"},
|
||||||
|
{store.SpamEntryAddress, "a@localhost"},
|
||||||
|
{store.SpamEntryAddress, "a b@example.com"},
|
||||||
|
{store.SpamEntryAddress, "a#b@example.com"},
|
||||||
|
{store.SpamEntryAddress, "a\nb@example.com"},
|
||||||
|
{store.SpamEntryAddress, ".a@example.com"},
|
||||||
|
{store.SpamEntryDomain, ""},
|
||||||
|
{store.SpamEntryDomain, "example"},
|
||||||
|
{store.SpamEntryDomain, "exa mple.com"},
|
||||||
|
{store.SpamEntryDomain, "example.com # all of it"},
|
||||||
|
{store.SpamEntryDomain, "user@example.com"},
|
||||||
|
{store.SpamEntryIP, ""},
|
||||||
|
{store.SpamEntryIP, "example.com"},
|
||||||
|
{store.SpamEntryIP, "192.0.2.300"},
|
||||||
|
{store.SpamEntryIP, "192.0.2.0/33"},
|
||||||
|
{store.SpamEntryIP, "0.0.0.0/0"},
|
||||||
|
{store.SpamEntryIP, "::/0"},
|
||||||
|
{store.SpamEntryIP, "fe80::1%eth0"},
|
||||||
|
{store.SpamEntryIP, "::ffff:192.0.2.0/120"},
|
||||||
|
{store.SpamEntryIP, "192.0.2.7 192.0.2.8"},
|
||||||
|
{"subject", "pills"},
|
||||||
|
{"", "x"},
|
||||||
|
} {
|
||||||
|
got, err := normalizeListValue(c.kind, c.raw)
|
||||||
|
var ve *ValidationError
|
||||||
|
if err == nil || !errors.As(err, &ve) {
|
||||||
|
t.Errorf("normalizeListValue(%q, %q) = %q, %v; want a ValidationError", c.kind, c.raw, got, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestListsAddWritesTheMaps(t *testing.T) {
|
||||||
|
l, _, dir := testLists(t)
|
||||||
|
for _, e := range []struct{ list, kind, value string }{
|
||||||
|
{store.SpamListAllow, store.SpamEntryAddress, "Billing@Vendor.Example"},
|
||||||
|
{store.SpamListAllow, store.SpamEntryAddress, "accounts@vendor.example"},
|
||||||
|
{store.SpamListDeny, store.SpamEntryDomain, "@evil.example"},
|
||||||
|
{store.SpamListDeny, store.SpamEntryIP, "198.51.100.9/24"},
|
||||||
|
// The same value on both lists is two entries: deny wins in the filter.
|
||||||
|
{store.SpamListDeny, store.SpamEntryAddress, "billing@vendor.example"},
|
||||||
|
} {
|
||||||
|
if _, err := l.Add(e.list, e.kind, e.value, " why "); err != nil {
|
||||||
|
t.Fatalf("Add(%s, %s, %q): %v", e.list, e.kind, e.value, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
want := map[string]string{
|
||||||
|
"allow_from.map": "accounts@vendor.example billing@vendor.example",
|
||||||
|
"allow_domain.map": "",
|
||||||
|
"allow_ip.map": "",
|
||||||
|
"deny_from.map": "billing@vendor.example",
|
||||||
|
"deny_domain.map": "evil.example",
|
||||||
|
"deny_ip.map": "198.51.100.0/24",
|
||||||
|
}
|
||||||
|
for name, lines := range want {
|
||||||
|
if got := strings.Join(mapLines(t, dir, name), " "); got != lines {
|
||||||
|
t.Errorf("%s = %q, want %q", name, got, lines)
|
||||||
|
}
|
||||||
|
// The filter is another user in another container.
|
||||||
|
if info, err := os.Stat(filepath.Join(dir, name)); err != nil {
|
||||||
|
t.Errorf("stat %s: %v", name, err)
|
||||||
|
} else if info.Mode().Perm()&0o004 == 0 && os.PathSeparator == '/' {
|
||||||
|
t.Errorf("%s is mode %o, not readable by the filter", name, info.Mode().Perm())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if files, _ := os.ReadDir(dir); len(files) != len(want) {
|
||||||
|
t.Errorf("the directory holds %d files, want %d (no temp file left behind)", len(files), len(want))
|
||||||
|
}
|
||||||
|
|
||||||
|
entries, err := l.Entries()
|
||||||
|
if err != nil || len(entries) != 5 {
|
||||||
|
t.Fatalf("Entries = %d, %v", len(entries), err)
|
||||||
|
}
|
||||||
|
if entries[0].Note != "why" {
|
||||||
|
t.Errorf("note = %q, want it trimmed", entries[0].Note)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestListsRefusals(t *testing.T) {
|
||||||
|
l, _, dir := testLists(t)
|
||||||
|
if _, err := l.Add(store.SpamListAllow, store.SpamEntryIP, "192.0.2.7", ""); err != nil {
|
||||||
|
t.Fatalf("Add: %v", err)
|
||||||
|
}
|
||||||
|
// Written differently, the same entry.
|
||||||
|
if _, err := l.Add(store.SpamListAllow, store.SpamEntryIP, "::ffff:192.0.2.7", ""); !errors.Is(err, store.ErrSpamListEntryExists) {
|
||||||
|
t.Errorf("the same address again: %v, want ErrSpamListEntryExists", err)
|
||||||
|
}
|
||||||
|
var ve *ValidationError
|
||||||
|
for name, add := range map[string]func() error{
|
||||||
|
"an unknown list": func() error { _, err := l.Add("block", store.SpamEntryIP, "192.0.2.8", ""); return err },
|
||||||
|
"a bad value": func() error { _, err := l.Add(store.SpamListDeny, store.SpamEntryIP, "nope", ""); return err },
|
||||||
|
"a long note": func() error {
|
||||||
|
_, err := l.Add(store.SpamListDeny, store.SpamEntryIP, "192.0.2.8", strings.Repeat("x", noteMaxRunes+1))
|
||||||
|
return err
|
||||||
|
},
|
||||||
|
"a note of two lines": func() error {
|
||||||
|
_, err := l.Add(store.SpamListDeny, store.SpamEntryIP, "192.0.2.8", "one\ntwo")
|
||||||
|
return err
|
||||||
|
},
|
||||||
|
} {
|
||||||
|
if err := add(); !errors.As(err, &ve) {
|
||||||
|
t.Errorf("%s: %v, want a ValidationError", name, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if got := strings.Join(mapLines(t, dir, "deny_ip.map"), " "); got != "" {
|
||||||
|
t.Errorf("deny_ip.map = %q after refusals only", got)
|
||||||
|
}
|
||||||
|
if entries, _ := l.Entries(); len(entries) != 1 {
|
||||||
|
t.Errorf("entries = %d, want the first one alone", len(entries))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestListsDelete(t *testing.T) {
|
||||||
|
l, _, dir := testLists(t)
|
||||||
|
e, err := l.Add(store.SpamListDeny, store.SpamEntryDomain, "evil.example", "")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Add: %v", err)
|
||||||
|
}
|
||||||
|
if err := l.Delete(e.ID); err != nil {
|
||||||
|
t.Fatalf("Delete: %v", err)
|
||||||
|
}
|
||||||
|
if got := mapLines(t, dir, "deny_domain.map"); len(got) != 0 {
|
||||||
|
t.Errorf("deny_domain.map = %v after the delete", got)
|
||||||
|
}
|
||||||
|
if err := l.Delete(e.ID); !errors.Is(err, store.ErrSpamListEntryNotFound) {
|
||||||
|
t.Errorf("a second delete: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// An entry the filter never got must not stay on the page.
|
||||||
|
func TestListsAddRollsBackWhenTheMapsCannotBeWritten(t *testing.T) {
|
||||||
|
st, err := store.Open(filepath.Join(t.TempDir(), "test.db"))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer st.Close()
|
||||||
|
// A file where the directory should be.
|
||||||
|
blocked := filepath.Join(t.TempDir(), "antispam")
|
||||||
|
if err := os.WriteFile(blocked, nil, 0o600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
l := NewLists(st, blocked)
|
||||||
|
if _, err := l.Add(store.SpamListDeny, store.SpamEntryDomain, "evil.example", ""); err == nil {
|
||||||
|
t.Fatal("Add succeeded with nowhere to write the maps")
|
||||||
|
}
|
||||||
|
if entries, _ := l.Entries(); len(entries) != 0 {
|
||||||
|
t.Errorf("the entry stayed: %+v", entries)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The writer's own guard: a row that got into the table some other way is not
|
||||||
|
// written into a map.
|
||||||
|
func TestListsSyncRefusesAnUnsafeRow(t *testing.T) {
|
||||||
|
l, st, dir := testLists(t)
|
||||||
|
if _, err := st.AddSpamListEntry(store.SpamListDeny, store.SpamEntryDomain, "evil.example\ngood.example", ""); err != nil {
|
||||||
|
t.Fatalf("AddSpamListEntry: %v", err)
|
||||||
|
}
|
||||||
|
if err := l.Sync(); err == nil {
|
||||||
|
t.Fatal("Sync wrote a value with a newline in it")
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(filepath.Join(dir, "deny_domain.map")); err == nil {
|
||||||
|
t.Error("a map file was written")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// With nothing on the lists all six files are there, empty.
|
||||||
|
func TestListsSyncWritesEmptyMaps(t *testing.T) {
|
||||||
|
l, _, dir := testLists(t)
|
||||||
|
if err := l.Sync(); err != nil {
|
||||||
|
t.Fatalf("Sync: %v", err)
|
||||||
|
}
|
||||||
|
for _, name := range mapFiles {
|
||||||
|
if got := mapLines(t, dir, name); len(got) != 0 {
|
||||||
|
t.Errorf("%s = %v", name, got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestListsLastWritten(t *testing.T) {
|
||||||
|
l, _, dir := testLists(t)
|
||||||
|
if got := l.LastWritten(); !got.IsZero() {
|
||||||
|
t.Fatalf("LastWritten before anything was written = %v", got)
|
||||||
|
}
|
||||||
|
if err := l.Sync(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if got := l.LastWritten(); got.IsZero() || time.Since(got) > time.Minute {
|
||||||
|
t.Errorf("LastWritten after a sync = %v", got)
|
||||||
|
}
|
||||||
|
|
||||||
|
// The newest of the six counts, whichever it is.
|
||||||
|
old := time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC)
|
||||||
|
for _, name := range mapFiles {
|
||||||
|
if err := os.Chtimes(filepath.Join(dir, name), old, old); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
newest := old.Add(48 * time.Hour)
|
||||||
|
if err := os.Chtimes(filepath.Join(dir, "deny_domain.map"), newest, newest); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if got := l.LastWritten(); !got.Equal(newest) {
|
||||||
|
t.Errorf("LastWritten = %v, want %v", got, newest)
|
||||||
|
}
|
||||||
|
|
||||||
|
// A file that is not one of the six does not count.
|
||||||
|
other := newest.Add(time.Hour)
|
||||||
|
if err := os.WriteFile(filepath.Join(dir, "notes.txt"), nil, 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.Chtimes(filepath.Join(dir, "notes.txt"), other, other); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if got := l.LastWritten(); !got.Equal(newest) {
|
||||||
|
t.Errorf("LastWritten counts a stray file: %v", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// How often an entry decided a message is read off the journal: the list that
|
||||||
|
// decided, and whether the entry fits who the message was from.
|
||||||
|
func TestListsUsage(t *testing.T) {
|
||||||
|
l, st, _ := testLists(t)
|
||||||
|
add := func(list, kind, value string) int64 {
|
||||||
|
t.Helper()
|
||||||
|
e, err := l.Add(list, kind, value, "")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Add(%s, %s, %s): %v", list, kind, value, err)
|
||||||
|
}
|
||||||
|
return e.ID
|
||||||
|
}
|
||||||
|
denyDomain := add(store.SpamListDeny, store.SpamEntryDomain, "evil.example")
|
||||||
|
denyAddr := add(store.SpamListDeny, store.SpamEntryAddress, "spammer@evil.example")
|
||||||
|
denyNet := add(store.SpamListDeny, store.SpamEntryIP, "198.51.100.0/24")
|
||||||
|
denyV6 := add(store.SpamListDeny, store.SpamEntryIP, "2001:db8::/32")
|
||||||
|
allowAddr := add(store.SpamListAllow, store.SpamEntryAddress, "billing@vendor.example")
|
||||||
|
allowDomain := add(store.SpamListAllow, store.SpamEntryDomain, "vendor.example")
|
||||||
|
allowIP := add(store.SpamListAllow, store.SpamEntryIP, "203.0.113.5")
|
||||||
|
unused := add(store.SpamListAllow, store.SpamEntryDomain, "quiet.example")
|
||||||
|
|
||||||
|
const denied = "5.7.1 Refused by this server's deny list"
|
||||||
|
at := time.Date(2026, 10, 1, 0, 0, 0, 0, time.UTC)
|
||||||
|
hour := 0
|
||||||
|
row := func(from, ip, decision, engine, detail string) {
|
||||||
|
t.Helper()
|
||||||
|
hour++
|
||||||
|
err := st.InsertInboundSpam(store.InboundSpamEntry{
|
||||||
|
InboundDomain: "in.example.com", To: "a@in.example.com", From: from, ClientIP: ip,
|
||||||
|
Decision: decision, Engine: engine, Detail: detail, At: at.Add(time.Duration(hour) * time.Hour),
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// hour 1: the address and its domain are both on the deny list, and both fit.
|
||||||
|
row("Spammer@Evil.Example", "192.0.2.9", store.SpamDecisionReject, store.SpamEngineRspamd, denied)
|
||||||
|
// hour 2: another address of the domain.
|
||||||
|
row("other@evil.example", "192.0.2.9", store.SpamDecisionReject, store.SpamEngineRspamd, denied)
|
||||||
|
// hour 3, 4: the networks, v4 (as Postfix may log it, mapped) and v6.
|
||||||
|
row("x@elsewhere.example", "::ffff:198.51.100.77", store.SpamDecisionReject, store.SpamEngineRspamd, denied)
|
||||||
|
row("x@elsewhere.example", "2001:db8::1", store.SpamDecisionReject, store.SpamEngineRspamd, denied)
|
||||||
|
// hour 5: a subdomain is not the domain.
|
||||||
|
row("x@sub.evil.example", "192.0.2.9", store.SpamDecisionReject, store.SpamEngineRspamd, denied)
|
||||||
|
// hour 6: allowed by its address and its domain - two symbols, two entries.
|
||||||
|
row("billing@vendor.example", "192.0.2.50", store.SpamDecisionAccept, store.SpamEngineRspamd,
|
||||||
|
"no action, score -199.00 / 15.00; SELFPOST_ALLOW_DOMAIN(-100.00) SELFPOST_ALLOW_FROM(-100.00)")
|
||||||
|
// hour 7: allowed by its domain alone.
|
||||||
|
row("accounts@vendor.example", "192.0.2.50", store.SpamDecisionAccept, store.SpamEngineRspamd,
|
||||||
|
"no action, score -99.00 / 15.00; SELFPOST_ALLOW_DOMAIN(-100.00)")
|
||||||
|
// hour 8: allowed by the client address; the sender's domain is not why.
|
||||||
|
row("x@quiet.example", "203.0.113.5", store.SpamDecisionAccept, store.SpamEngineRspamd,
|
||||||
|
"no action, score -99.00 / 15.00; SELFPOST_ALLOW_IP(-100.00)")
|
||||||
|
// Nobody's: the filter's own refusal, an accept with no list symbol, a
|
||||||
|
// refusal by Postfix, and a sender who spells the mark in his own address.
|
||||||
|
row("spammer@evil.example", "192.0.2.9", store.SpamDecisionReject, store.SpamEngineRspamd, "5.7.1 Spam message rejected")
|
||||||
|
row("billing@vendor.example", "192.0.2.50", store.SpamDecisionAccept, store.SpamEngineRspamd, "no action, score 0.10 / 15.00")
|
||||||
|
row("spammer@evil.example", "198.51.100.1", store.SpamDecisionReject, store.SpamEnginePostfix, "550 5.1.1 <nobody@in.example.com>: this server's deny list")
|
||||||
|
row("billing@vendor.example", "203.0.113.5", store.SpamDecisionAccept, store.SpamEngineSelfPost, "")
|
||||||
|
|
||||||
|
usage, err := l.Usage()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Usage: %v", err)
|
||||||
|
}
|
||||||
|
for name, c := range map[string]struct {
|
||||||
|
id int64
|
||||||
|
hits int
|
||||||
|
last int // hour
|
||||||
|
}{
|
||||||
|
"deny domain": {denyDomain, 2, 2}, "deny address": {denyAddr, 1, 1},
|
||||||
|
"deny network": {denyNet, 1, 3}, "deny v6 network": {denyV6, 1, 4},
|
||||||
|
"allow address": {allowAddr, 1, 6}, "allow domain": {allowDomain, 2, 7}, "allow ip": {allowIP, 1, 8},
|
||||||
|
"unused": {unused, 0, 0},
|
||||||
|
} {
|
||||||
|
got := usage[c.id]
|
||||||
|
wantLast := time.Time{}
|
||||||
|
if c.hits > 0 {
|
||||||
|
wantLast = at.Add(time.Duration(c.last) * time.Hour)
|
||||||
|
}
|
||||||
|
if got.Hits != c.hits || !got.Last.Equal(wantLast) {
|
||||||
|
t.Errorf("%s: %d hits, last %v; want %d, %v", name, got.Hits, got.Last, c.hits, wantLast)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestListsUsageWithNoEntries(t *testing.T) {
|
||||||
|
l, _, _ := testLists(t)
|
||||||
|
if usage, err := l.Usage(); err != nil || len(usage) != 0 {
|
||||||
|
t.Errorf("Usage = %v, %v", usage, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The same address written as a network of one is the same entry.
|
||||||
|
func TestListsOneAddressNetworkIsTheAddress(t *testing.T) {
|
||||||
|
l, _, dir := testLists(t)
|
||||||
|
if _, err := l.Add(store.SpamListDeny, store.SpamEntryIP, "192.0.2.7", ""); err != nil {
|
||||||
|
t.Fatalf("Add: %v", err)
|
||||||
|
}
|
||||||
|
if _, err := l.Add(store.SpamListDeny, store.SpamEntryIP, "192.0.2.7/32", ""); !errors.Is(err, store.ErrSpamListEntryExists) {
|
||||||
|
t.Errorf("192.0.2.7/32 after 192.0.2.7: %v, want ErrSpamListEntryExists", err)
|
||||||
|
}
|
||||||
|
if got := strings.Join(mapLines(t, dir, "deny_ip.map"), " "); got != "192.0.2.7" {
|
||||||
|
t.Errorf("deny_ip.map = %q", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestListable(t *testing.T) {
|
||||||
|
for value, want := range map[string]bool{
|
||||||
|
"billing@vendor.example": true, "srs0=ab=cd=example.org=user@forwarder.example": true,
|
||||||
|
"o'brien@example.org": false, "a#b@example.org": false, "": false, "not-an-address": false,
|
||||||
|
} {
|
||||||
|
if got := Listable(store.SpamEntryAddress, value); got != want {
|
||||||
|
t.Errorf("Listable(address, %q) = %v, want %v", value, got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The key the filter signs with is made once and then read, and is readable by
|
||||||
|
// the filter, which is another user in another container.
|
||||||
|
func TestFilterKey(t *testing.T) {
|
||||||
|
dir := filepath.Join(t.TempDir(), "antispam")
|
||||||
|
key, err := FilterKey(dir)
|
||||||
|
if err != nil || len(key) != 64 || strings.Trim(key, "0123456789abcdef") != "" {
|
||||||
|
t.Fatalf("FilterKey = %q, %v", key, err)
|
||||||
|
}
|
||||||
|
again, err := FilterKey(dir)
|
||||||
|
if err != nil || again != key {
|
||||||
|
t.Errorf("a second call = %q, %v; want the same key", again, err)
|
||||||
|
}
|
||||||
|
raw, err := os.ReadFile(filepath.Join(dir, FilterKeyFile))
|
||||||
|
if err != nil || string(raw) != key+"\n" {
|
||||||
|
t.Errorf("file = %q, %v", raw, err)
|
||||||
|
}
|
||||||
|
if info, _ := os.Stat(filepath.Join(dir, FilterKeyFile)); os.PathSeparator == '/' && info.Mode().Perm() != 0o644 {
|
||||||
|
t.Errorf("mode = %o", info.Mode().Perm())
|
||||||
|
}
|
||||||
|
|
||||||
|
// A file that is not a key is replaced, not signed with.
|
||||||
|
if err := os.WriteFile(filepath.Join(dir, FilterKeyFile), []byte("short\n"), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
fresh, err := FilterKey(dir)
|
||||||
|
if err != nil || len(fresh) != 64 || fresh == key {
|
||||||
|
t.Errorf("after a damaged file: %q, %v", fresh, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// The lists leave it alone.
|
||||||
|
st, err := store.Open(filepath.Join(t.TempDir(), "test.db"))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer st.Close()
|
||||||
|
if err := NewLists(st, dir).Sync(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if kept, _ := FilterKey(dir); kept != fresh {
|
||||||
|
t.Errorf("the key changed when the lists were written")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,300 @@
|
|||||||
|
package logtail
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"log"
|
||||||
|
"regexp"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/mixeme/selfpost/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
// InboundStore is what the tailer needs of the inbound antispam journal:
|
||||||
|
// writing the rows for refused mail, knowing which domains are inbound, and
|
||||||
|
// pruning. *store.Store satisfies it.
|
||||||
|
type InboundStore interface {
|
||||||
|
InsertInboundSpam(e store.InboundSpamEntry) error
|
||||||
|
InsertInboundSpamOnce(e store.InboundSpamEntry) (bool, error)
|
||||||
|
IsInboundDomain(name string) (bool, error)
|
||||||
|
PruneInboundSpam(cutoff time.Time, keep int) (int64, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Inbound turns on the inbound half of the tailer: journal rows for mail that
|
||||||
|
// port 25 refused, and the journal's retention. Run takes nil when the inbound
|
||||||
|
// relay is off.
|
||||||
|
type Inbound struct {
|
||||||
|
Store InboundStore
|
||||||
|
Retention RetentionDays
|
||||||
|
}
|
||||||
|
|
||||||
|
// The inbound journal milter writes a row for mail that was accepted; mail
|
||||||
|
// that was refused never reaches it, and what is known about it is one line
|
||||||
|
// of mail.log:
|
||||||
|
//
|
||||||
|
// postfix/inbound/smtpd[41]: NOQUEUE: reject: RCPT from mx.example.net[203.0.113.9]: 550 5.1.1 <nobody@in.example.com>: Recipient address rejected: User unknown in relay recipient table; from=<a@example.net> to=<nobody@in.example.com> proto=ESMTP helo=<mx.example.net>
|
||||||
|
// postfix/cleanup[52]: 4A1B2C3D: milter-reject: END-OF-MESSAGE from mx.example.net[203.0.113.9]: 5.7.1 Spam message rejected; from=<a@example.net> to=<b@in.example.com> proto=ESMTP helo=<mx.example.net>
|
||||||
|
//
|
||||||
|
// A message the filter holds (milter-hold) is not read here: it goes on to the
|
||||||
|
// inbound journal milter, which writes its row with the subject.
|
||||||
|
//
|
||||||
|
// The first is Postfix's own policy, written by the port-25 smtpd, which
|
||||||
|
// build/postfix-config.sh names postfix/inbound so that its lines are told
|
||||||
|
// from those of the submission ports. The second is the filter's verdict at
|
||||||
|
// end-of-message, which cleanup(8) writes under one name for every listener -
|
||||||
|
// so it counts only for a queue id the port-25 smtpd was seen to open
|
||||||
|
// (inboundClientRe).
|
||||||
|
//
|
||||||
|
// Both are matched from the head of the line, after the timestamp and host
|
||||||
|
// name: the tail of such a line is the sender's own words (HELO name,
|
||||||
|
// addresses), and a pattern that could start anywhere would let him write
|
||||||
|
// journal rows.
|
||||||
|
//
|
||||||
|
// For the same reason the reply is the shortest text before "; from=<", and
|
||||||
|
// the HELO name - the last thing on the line, which Postfix logs as the client
|
||||||
|
// wrote it, spaces and brackets included - is whatever is left. Taking the
|
||||||
|
// longest reply instead would let a HELO name that itself spells
|
||||||
|
// "; from=<…> to=<…> proto=…" supply the sender, the recipient and the reply
|
||||||
|
// of the row (security review, 2026-10-12).
|
||||||
|
var (
|
||||||
|
inboundRejectRe = regexp.MustCompile(`^\S+ postfix/(inbound/smtpd|cleanup)\[\d+\]: ([0-9A-Za-z]+): (reject|milter-reject|milter-discard): \S+ from [^\[\s]*\[([0-9A-Fa-f:.]+)\]: (.*?); from=<([^>]*)> to=<([^>]*)> proto=\S+(?: helo=<.*>)?$`)
|
||||||
|
inboundClientRe = regexp.MustCompile(`^\S+ postfix/inbound/smtpd\[\d+\]: ([0-9A-Za-z]+): client=`)
|
||||||
|
// A queue id stops being port 25's when the message is queued (qmgr takes
|
||||||
|
// it: the filter let it through) or when another listener opens a message
|
||||||
|
// under the same id - Postfix reuses ids.
|
||||||
|
inboundForgetRe = regexp.MustCompile(`^\S+ postfix/(?:qmgr\[\d+\]: ([0-9A-Za-z]+): from=|(?:[a-z0-9-]+/)?smtpd\[\d+\]: ([0-9A-Za-z]+): client=)`)
|
||||||
|
)
|
||||||
|
|
||||||
|
// A row holds one line of reason; Postfix's reply text carries the far end's
|
||||||
|
// bytes and is cut and reduced to printable ASCII like any of them.
|
||||||
|
const inboundDetailMax = 255
|
||||||
|
|
||||||
|
// inboundQueueIDs is how many port-25 queue ids are remembered while waiting
|
||||||
|
// for cleanup's verdict line on them. An id is forgotten when the verdict
|
||||||
|
// comes, when the message is queued, or when another listener reuses it; the
|
||||||
|
// cap bounds what is left - messages the client abandoned - and a flood.
|
||||||
|
const inboundQueueIDs = 4096
|
||||||
|
|
||||||
|
// inboundPruneEvery is how many rows the tailer writes before it has the
|
||||||
|
// journal cut back to its row cap, without waiting for the six-hourly prune.
|
||||||
|
// Refusals are written at the rate strangers send RCPT TO, which no limit of
|
||||||
|
// Postfix's counts, so between two prunes the journal would otherwise grow as
|
||||||
|
// fast as someone cared to make it.
|
||||||
|
const inboundPruneEvery = 500
|
||||||
|
|
||||||
|
// inboundJournal is the per-run state of the inbound half.
|
||||||
|
type inboundJournal struct {
|
||||||
|
st InboundStore
|
||||||
|
// port25 holds the queue ids the port-25 smtpd opened and nobody has
|
||||||
|
// accounted for yet, each with the number of its arrival; order is those
|
||||||
|
// arrivals, for dropping the oldest. An id that was forgotten and opened
|
||||||
|
// again is in order twice: the number tells the stale entry from the live
|
||||||
|
// one when the stale one comes up for dropping.
|
||||||
|
port25 map[string]uint64
|
||||||
|
order []queued
|
||||||
|
seq uint64
|
||||||
|
// written counts the rows since the journal was last cut to its cap.
|
||||||
|
written int
|
||||||
|
}
|
||||||
|
|
||||||
|
type queued struct {
|
||||||
|
id string
|
||||||
|
seq uint64
|
||||||
|
}
|
||||||
|
|
||||||
|
func newInboundJournal(st InboundStore) *inboundJournal {
|
||||||
|
return &inboundJournal{st: st, port25: make(map[string]uint64)}
|
||||||
|
}
|
||||||
|
|
||||||
|
// handle looks at one mail.log line and writes a journal row when it is a
|
||||||
|
// refusal of mail for an inbound domain. Errors are logged; the tailer goes on.
|
||||||
|
//
|
||||||
|
// catchingUp says the line is one of those the tailer reads when it starts,
|
||||||
|
// from where it last saved its place to the end of the file. It saves its
|
||||||
|
// place every few seconds, so after an unclean stop some of those lines were
|
||||||
|
// read before; a row is then written only if it is not there already. Lines
|
||||||
|
// read as they are written are always new, and two identical refusals in one
|
||||||
|
// second stay two rows.
|
||||||
|
func (j *inboundJournal) handle(line string, now time.Time, catchingUp bool) {
|
||||||
|
at, rest := lineTime(line, now)
|
||||||
|
if m := inboundClientRe.FindStringSubmatch(rest); m != nil {
|
||||||
|
j.remember(m[1])
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if m := inboundForgetRe.FindStringSubmatch(rest); m != nil {
|
||||||
|
delete(j.port25, m[1]+m[2])
|
||||||
|
return
|
||||||
|
}
|
||||||
|
m := inboundRejectRe.FindStringSubmatch(rest)
|
||||||
|
if m == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
program, queueID, kind, clientIP, reason, from, to := m[1], m[2], m[3], m[4], m[5], m[6], m[7]
|
||||||
|
// The sender's address is his own text too, and with a quoted local part it
|
||||||
|
// may hold "> to=<…> proto=…" - which would end the sender early and supply
|
||||||
|
// the recipient. Such a line shows it: the sender is left with an unclosed
|
||||||
|
// quote. An address with a quote, a bracket or a space in it is not one the
|
||||||
|
// journal keeps.
|
||||||
|
if strings.ContainsAny(from, "\"<> ") || strings.ContainsAny(to, "\"<> ") {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if program == "cleanup" {
|
||||||
|
if _, ok := j.port25[queueID]; !ok {
|
||||||
|
return // a submission-port message, or one opened before this run
|
||||||
|
}
|
||||||
|
delete(j.port25, queueID)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Only mail for an inbound domain is the relay's. "Relay access denied"
|
||||||
|
// for a stranger's domain is every scanner on the internet, and a line
|
||||||
|
// with no recipient (size limit at MAIL) belongs to no domain.
|
||||||
|
domain := ""
|
||||||
|
if i := strings.LastIndexByte(to, '@'); i >= 0 {
|
||||||
|
domain = strings.ToLower(to[i+1:])
|
||||||
|
}
|
||||||
|
if domain == "" {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
ok, err := j.st.IsInboundDomain(domain)
|
||||||
|
if err != nil {
|
||||||
|
log.Printf("log-tailer: inbound journal: look up %s: %v", domain, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if !ok {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
e := store.InboundSpamEntry{
|
||||||
|
InboundDomain: domain,
|
||||||
|
ClientIP: clientIP,
|
||||||
|
From: from,
|
||||||
|
To: to,
|
||||||
|
Decision: store.SpamDecisionReject,
|
||||||
|
Engine: store.SpamEngineRspamd,
|
||||||
|
Detail: printable(reason, inboundDetailMax),
|
||||||
|
At: at,
|
||||||
|
}
|
||||||
|
if kind == "reject" {
|
||||||
|
e.Engine = store.SpamEnginePostfix
|
||||||
|
}
|
||||||
|
// "450 4.7.1 …" from Postfix, "4.7.1 …" from a milter: either way a reply
|
||||||
|
// that starts with 4 asks the sender to come back.
|
||||||
|
if e.Decision == store.SpamDecisionReject && strings.HasPrefix(reason, "4") {
|
||||||
|
e.Decision = store.SpamDecisionTempfail
|
||||||
|
}
|
||||||
|
if catchingUp {
|
||||||
|
if wrote, err := j.st.InsertInboundSpamOnce(e); err != nil {
|
||||||
|
log.Printf("log-tailer: inbound journal: record %s -> %s: %v", from, to, err)
|
||||||
|
return
|
||||||
|
} else if !wrote {
|
||||||
|
return // read before the last stop
|
||||||
|
}
|
||||||
|
} else if err := j.st.InsertInboundSpam(e); err != nil {
|
||||||
|
log.Printf("log-tailer: inbound journal: record %s -> %s: %v", from, to, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if j.written++; j.written >= inboundPruneEvery {
|
||||||
|
j.written = 0
|
||||||
|
// The zero time removes nothing by age: this is the cap alone.
|
||||||
|
if _, err := j.st.PruneInboundSpam(time.Time{}, inboundSpamLogMaxRows); err != nil {
|
||||||
|
log.Printf("log-tailer: inbound journal: cut to %d rows: %v", inboundSpamLogMaxRows, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (j *inboundJournal) remember(queueID string) {
|
||||||
|
if _, ok := j.port25[queueID]; ok {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
for len(j.order) >= inboundQueueIDs {
|
||||||
|
oldest := j.order[0]
|
||||||
|
j.order = j.order[1:]
|
||||||
|
if j.port25[oldest.id] == oldest.seq {
|
||||||
|
delete(j.port25, oldest.id)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
j.seq++
|
||||||
|
j.port25[queueID] = j.seq
|
||||||
|
j.order = append(j.order, queued{queueID, j.seq})
|
||||||
|
}
|
||||||
|
|
||||||
|
// printable cuts s to max bytes and replaces everything outside printable
|
||||||
|
// ASCII.
|
||||||
|
func printable(s string, max int) string {
|
||||||
|
if len(s) > max {
|
||||||
|
s = s[:max]
|
||||||
|
}
|
||||||
|
b := []byte(s)
|
||||||
|
for i, c := range b {
|
||||||
|
if c < 0x20 || c > 0x7e {
|
||||||
|
b[i] = '?'
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return string(b)
|
||||||
|
}
|
||||||
|
|
||||||
|
// lineTime reads the time at the head of a mail.log line and returns it with
|
||||||
|
// the rest of the line. A refusal is dated by its line and not by when the
|
||||||
|
// tailer got to it: after the panel was down the two are hours apart. The
|
||||||
|
// syslog format carries neither year nor zone - the year is taken from now
|
||||||
|
// (a stamp that would then lie in the future is last year's), the zone is the
|
||||||
|
// container's, which is what postlogd wrote it in. A line with no readable
|
||||||
|
// stamp is dated now.
|
||||||
|
func lineTime(line string, now time.Time) (time.Time, string) {
|
||||||
|
if m := isoStampRe.FindStringSubmatch(line); m != nil {
|
||||||
|
rest := strings.TrimSpace(line[len(m[0]):])
|
||||||
|
if t, err := time.Parse(time.RFC3339Nano, strings.TrimSpace(m[0])); err == nil {
|
||||||
|
return t.UTC(), rest
|
||||||
|
}
|
||||||
|
return now, rest
|
||||||
|
}
|
||||||
|
if m := syslogStampRe.FindStringSubmatch(line); m != nil {
|
||||||
|
rest := strings.TrimSpace(line[len(m[0]):])
|
||||||
|
stamp := strings.Join(strings.Fields(m[1]), " ")
|
||||||
|
t, err := time.ParseInLocation("Jan 2 15:04:05", stamp, time.Local)
|
||||||
|
if err != nil {
|
||||||
|
return now, rest
|
||||||
|
}
|
||||||
|
t = t.AddDate(now.In(time.Local).Year(), 0, 0)
|
||||||
|
if t.After(now.Add(24 * time.Hour)) {
|
||||||
|
t = t.AddDate(-1, 0, 0)
|
||||||
|
}
|
||||||
|
return t.UTC(), rest
|
||||||
|
}
|
||||||
|
return now, line
|
||||||
|
}
|
||||||
|
|
||||||
|
// The journal keeps at most this many rows whatever their age: refusals are
|
||||||
|
// written at the rate strangers knock on port 25.
|
||||||
|
const inboundSpamLogMaxRows = 10000
|
||||||
|
|
||||||
|
// inboundRetentionLoop prunes the inbound journal at once and then on the
|
||||||
|
// cadence of the send log.
|
||||||
|
func inboundRetentionLoop(ctx context.Context, in *Inbound) {
|
||||||
|
prune := func() {
|
||||||
|
days := in.Retention()
|
||||||
|
if days <= 0 {
|
||||||
|
days = store.InboundSpamLogRetentionDaysDefault
|
||||||
|
}
|
||||||
|
n, err := in.Store.PruneInboundSpam(time.Now().UTC().AddDate(0, 0, -days), inboundSpamLogMaxRows)
|
||||||
|
if err != nil {
|
||||||
|
log.Printf("log-tailer: inbound journal prune: %v", err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if n > 0 {
|
||||||
|
log.Printf("log-tailer: pruned %d inbound journal rows (older than %d days or beyond %d rows)", n, days, inboundSpamLogMaxRows)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
prune()
|
||||||
|
t := time.NewTicker(retentionInterval)
|
||||||
|
defer t.Stop()
|
||||||
|
for {
|
||||||
|
select {
|
||||||
|
case <-ctx.Done():
|
||||||
|
return
|
||||||
|
case <-t.C:
|
||||||
|
prune()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,333 @@
|
|||||||
|
package logtail
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/mixeme/selfpost/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
type fakeInboundStore struct {
|
||||||
|
domains map[string]bool
|
||||||
|
rows []store.InboundSpamEntry
|
||||||
|
lookupErr error
|
||||||
|
pruned []int
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeInboundStore) InsertInboundSpam(e store.InboundSpamEntry) error {
|
||||||
|
f.rows = append(f.rows, e)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// InsertInboundSpamOnce writes unless an equal row is there, like the store's.
|
||||||
|
func (f *fakeInboundStore) InsertInboundSpamOnce(e store.InboundSpamEntry) (bool, error) {
|
||||||
|
for _, r := range f.rows {
|
||||||
|
if r == e {
|
||||||
|
return false, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
f.rows = append(f.rows, e)
|
||||||
|
return true, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeInboundStore) IsInboundDomain(name string) (bool, error) {
|
||||||
|
if f.lookupErr != nil {
|
||||||
|
return false, f.lookupErr
|
||||||
|
}
|
||||||
|
return f.domains[name], nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeInboundStore) PruneInboundSpam(_ time.Time, keep int) (int64, error) {
|
||||||
|
f.pruned = append(f.pruned, keep)
|
||||||
|
return 0, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func inboundFixture() (*inboundJournal, *fakeInboundStore) {
|
||||||
|
st := &fakeInboundStore{domains: map[string]bool{"in.example.com": true}}
|
||||||
|
return newInboundJournal(st), st
|
||||||
|
}
|
||||||
|
|
||||||
|
var tailNow = time.Date(2026, 10, 11, 18, 0, 0, 0, time.UTC)
|
||||||
|
|
||||||
|
const (
|
||||||
|
head = "2026-10-11T12:00:05.123456+00:00 mail "
|
||||||
|
unknownRcp = head + "postfix/inbound/smtpd[41]: NOQUEUE: reject: RCPT from mx.example.net[203.0.113.9]: 550 5.1.1 <nobody@in.example.com>: Recipient address rejected: User unknown in relay recipient table; from=<a@example.net> to=<nobody@in.example.com> proto=ESMTP helo=<mx.example.net>"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Postfix's own refusal of a recipient at an inbound domain is a row, dated by
|
||||||
|
// its line.
|
||||||
|
func TestInboundRejectByPostfix(t *testing.T) {
|
||||||
|
j, st := inboundFixture()
|
||||||
|
j.handle(unknownRcp, tailNow, false)
|
||||||
|
if len(st.rows) != 1 {
|
||||||
|
t.Fatalf("rows = %+v", st.rows)
|
||||||
|
}
|
||||||
|
want := store.InboundSpamEntry{
|
||||||
|
InboundDomain: "in.example.com", ClientIP: "203.0.113.9",
|
||||||
|
From: "a@example.net", To: "nobody@in.example.com",
|
||||||
|
Decision: store.SpamDecisionReject, Engine: store.SpamEnginePostfix,
|
||||||
|
Detail: "550 5.1.1 <nobody@in.example.com>: Recipient address rejected: User unknown in relay recipient table",
|
||||||
|
At: time.Date(2026, 10, 11, 12, 0, 5, 123456000, time.UTC),
|
||||||
|
}
|
||||||
|
if st.rows[0] != want {
|
||||||
|
t.Errorf("row = %+v\nwant %+v", st.rows[0], want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInboundTempfailByPostfix(t *testing.T) {
|
||||||
|
j, st := inboundFixture()
|
||||||
|
j.handle(head+"postfix/inbound/smtpd[41]: NOQUEUE: reject: RCPT from unknown[2001:db8::9]: 450 4.1.8 <a@nx.example>: Sender address rejected: Domain not found; from=<a@nx.example> to=<B@IN.example.com> proto=ESMTP helo=<x>", tailNow, false)
|
||||||
|
if len(st.rows) != 1 || st.rows[0].Decision != store.SpamDecisionTempfail || st.rows[0].ClientIP != "2001:db8::9" || st.rows[0].InboundDomain != "in.example.com" {
|
||||||
|
t.Errorf("rows = %+v", st.rows)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The filter's verdict is written by cleanup, under one name for every
|
||||||
|
// listener: it is the journal's only for a queue id port 25 opened.
|
||||||
|
func TestInboundMilterVerdictNeedsAPort25QueueID(t *testing.T) {
|
||||||
|
const verdict = "postfix/cleanup[52]: 4A1B2C3D: milter-reject: END-OF-MESSAGE from mx.example.net[203.0.113.9]: 5.7.1 Spam message rejected; from=<a@example.net> to=<b@in.example.com> proto=ESMTP helo=<mx.example.net>"
|
||||||
|
|
||||||
|
j, st := inboundFixture()
|
||||||
|
// Opened by a submission port: postfix/smtpd, not postfix/inbound/smtpd.
|
||||||
|
j.handle(head+"postfix/smtpd[20]: 4A1B2C3D: client=app.example.com[203.0.113.4], sasl_method=PLAIN, sasl_username=app", tailNow, false)
|
||||||
|
j.handle(head+verdict, tailNow, false)
|
||||||
|
if len(st.rows) != 0 {
|
||||||
|
t.Fatalf("a submission-port verdict was journalled: %+v", st.rows)
|
||||||
|
}
|
||||||
|
|
||||||
|
j.handle(head+"postfix/inbound/smtpd[41]: 4A1B2C3D: client=mx.example.net[203.0.113.9]", tailNow, false)
|
||||||
|
j.handle(head+verdict, tailNow, false)
|
||||||
|
if len(st.rows) != 1 {
|
||||||
|
t.Fatalf("rows = %+v", st.rows)
|
||||||
|
}
|
||||||
|
got := st.rows[0]
|
||||||
|
if got.Decision != store.SpamDecisionReject || got.Engine != store.SpamEngineRspamd || got.Detail != "5.7.1 Spam message rejected" {
|
||||||
|
t.Errorf("row = %+v", got)
|
||||||
|
}
|
||||||
|
// One verdict per message: the id is forgotten with it.
|
||||||
|
j.handle(head+verdict, tailNow, false)
|
||||||
|
if len(st.rows) != 1 {
|
||||||
|
t.Errorf("the same queue id was journalled twice")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInboundMilterDecisions(t *testing.T) {
|
||||||
|
for kind, c := range map[string]struct{ reply, decision string }{
|
||||||
|
"milter-reject": {"4.7.1 Try again later", store.SpamDecisionTempfail},
|
||||||
|
"milter-discard": {"milter triggers DISCARD action", store.SpamDecisionReject},
|
||||||
|
} {
|
||||||
|
j, st := inboundFixture()
|
||||||
|
j.handle(head+"postfix/inbound/smtpd[41]: Q1: client=mx.example.net[203.0.113.9]", tailNow, false)
|
||||||
|
j.handle(head+"postfix/cleanup[52]: Q1: "+kind+": END-OF-MESSAGE from mx.example.net[203.0.113.9]: "+c.reply+"; from=<a@example.net> to=<b@in.example.com> proto=ESMTP helo=<mx.example.net>", tailNow, false)
|
||||||
|
if len(st.rows) != 1 || st.rows[0].Decision != c.decision || st.rows[0].Engine != store.SpamEngineRspamd {
|
||||||
|
t.Errorf("%s: rows = %+v", kind, st.rows)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// What is not the relay's mail, or is not a refusal at all, leaves no row.
|
||||||
|
func TestInboundIgnores(t *testing.T) {
|
||||||
|
for name, line := range map[string]string{
|
||||||
|
"another domain (relay access denied)": head + "postfix/inbound/smtpd[41]: NOQUEUE: reject: RCPT from x[203.0.113.9]: 554 5.7.1 <c@elsewhere.example>: Relay access denied; from=<a@example.net> to=<c@elsewhere.example> proto=ESMTP helo=<x>",
|
||||||
|
"no recipient (size limit at MAIL)": head + "postfix/inbound/smtpd[41]: NOQUEUE: reject: MAIL from x[203.0.113.9]: 552 5.3.4 Message size exceeds fixed limit; proto=ESMTP helo=<x>",
|
||||||
|
"a submission port": head + "postfix/smtpd[20]: NOQUEUE: reject: RCPT from x[203.0.113.9]: 553 5.7.1 <b@in.example.com>: Sender address rejected: not owned by user app; from=<a@example.net> to=<b@in.example.com> proto=ESMTP helo=<x>",
|
||||||
|
"a delivery line": head + "postfix/smtp[26]: 41E862C00D9E: to=<b@in.example.com>, relay=mx[203.0.113.9]:25, dsn=2.0.0, status=sent (250 OK)",
|
||||||
|
"a connect line": head + "postfix/inbound/smtpd[41]: connect from mx.example.net[203.0.113.9]",
|
||||||
|
// The HELO name is the sender's; a line cannot be started from inside one.
|
||||||
|
"a refusal spelled inside another line": head + "postfix/inbound/smtpd[41]: warning: hostname x does not resolve: mail postfix/inbound/smtpd[1]: NOQUEUE: reject: RCPT from x[203.0.113.9]: 550 no; from=<a@example.net> to=<b@in.example.com> proto=ESMTP helo=<x>",
|
||||||
|
"an address with a bracket in it": head + "postfix/inbound/smtpd[41]: NOQUEUE: reject: RCPT from x[203.0.113.9]: 554 5.7.1 <b@elsewhere.example>: Relay access denied; from=<\"a> to=<v@in.example.com> proto=X helo=<y\"@example.net> to=<b@elsewhere.example> proto=ESMTP helo=<x>",
|
||||||
|
} {
|
||||||
|
j, st := inboundFixture()
|
||||||
|
j.handle(line, tailNow, false)
|
||||||
|
if len(st.rows) != 0 {
|
||||||
|
t.Errorf("%s: journalled %+v", name, st.rows)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInboundDetailIsCutAndPrintable(t *testing.T) {
|
||||||
|
j, st := inboundFixture()
|
||||||
|
j.handle(head+"postfix/inbound/smtpd[41]: NOQUEUE: reject: RCPT from x[203.0.113.9]: 550 5.7.1 caf\xc3\xa9 \x1b[31m"+strings.Repeat("x", 400)+"; from=<a@example.net> to=<b@in.example.com> proto=ESMTP helo=<x>", tailNow, false)
|
||||||
|
if len(st.rows) != 1 {
|
||||||
|
t.Fatalf("rows = %+v", st.rows)
|
||||||
|
}
|
||||||
|
d := st.rows[0].Detail
|
||||||
|
if len(d) != inboundDetailMax || !strings.HasPrefix(d, "550 5.7.1 caf?? ?[31mxxx") {
|
||||||
|
t.Errorf("detail (%d bytes) = %q", len(d), d)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInboundLookupFailureLeavesNoRow(t *testing.T) {
|
||||||
|
j, st := inboundFixture()
|
||||||
|
st.lookupErr = errors.New("boom")
|
||||||
|
j.handle(unknownRcp, tailNow, false)
|
||||||
|
if len(st.rows) != 0 {
|
||||||
|
t.Errorf("rows = %+v", st.rows)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A message the filter holds goes on to the inbound journal milter, which
|
||||||
|
// writes its row; a second one from the log would be the same message twice.
|
||||||
|
func TestInboundHoldIsTheMiltersRow(t *testing.T) {
|
||||||
|
j, st := inboundFixture()
|
||||||
|
j.handle(head+"postfix/inbound/smtpd[41]: Q1: client=mx.example.net[203.0.113.9]", tailNow, false)
|
||||||
|
j.handle(head+"postfix/cleanup[52]: Q1: milter-hold: END-OF-MESSAGE from mx.example.net[203.0.113.9]: milter triggers HOLD action; from=<a@example.net> to=<b@in.example.com> proto=ESMTP helo=<mx.example.net>", tailNow, false)
|
||||||
|
if len(st.rows) != 0 {
|
||||||
|
t.Errorf("rows = %+v", st.rows)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The HELO name is the last thing on the line and is logged as the client
|
||||||
|
// wrote it. It cannot supply the row: the reply ends at the first "; from=<",
|
||||||
|
// which is Postfix's own.
|
||||||
|
func TestInboundHeloCannotWriteTheRow(t *testing.T) {
|
||||||
|
j, st := inboundFixture()
|
||||||
|
j.handle(head+"postfix/inbound/smtpd[41]: NOQUEUE: reject: RCPT from x[203.0.113.9]: 550 5.1.1 <nobody@in.example.com>: Recipient address rejected: User unknown in relay recipient table; from=<real@example.net> to=<nobody@in.example.com> proto=ESMTP helo=<x 5.7.1 Refused by this server's deny list; from=<victim@bank.example> to=<ceo@in.example.com> proto=ESMTP helo=<y>", tailNow, false)
|
||||||
|
if len(st.rows) != 1 {
|
||||||
|
t.Fatalf("rows = %+v", st.rows)
|
||||||
|
}
|
||||||
|
got := st.rows[0]
|
||||||
|
if got.From != "real@example.net" || got.To != "nobody@in.example.com" ||
|
||||||
|
got.Detail != "550 5.1.1 <nobody@in.example.com>: Recipient address rejected: User unknown in relay recipient table" {
|
||||||
|
t.Errorf("row = %+v", got)
|
||||||
|
}
|
||||||
|
|
||||||
|
// A HELO name with a closing bracket in it does not lose the line either.
|
||||||
|
j, st = inboundFixture()
|
||||||
|
j.handle(head+"postfix/inbound/smtpd[41]: NOQUEUE: reject: RCPT from x[203.0.113.9]: 550 5.1.1 <nobody@in.example.com>: no; from=<real@example.net> to=<nobody@in.example.com> proto=ESMTP helo=<a>b>", tailNow, false)
|
||||||
|
if len(st.rows) != 1 || st.rows[0].From != "real@example.net" {
|
||||||
|
t.Errorf("rows = %+v", st.rows)
|
||||||
|
}
|
||||||
|
|
||||||
|
// The same on a verdict line: the filter's reply is not extended by the
|
||||||
|
// HELO name, so a stranger cannot make a refusal look like the deny list's.
|
||||||
|
j, st = inboundFixture()
|
||||||
|
j.handle(head+"postfix/inbound/smtpd[41]: Q1: client=x[203.0.113.9]", tailNow, false)
|
||||||
|
j.handle(head+"postfix/cleanup[52]: Q1: milter-reject: END-OF-MESSAGE from x[203.0.113.9]: 5.7.1 Spam message rejected; from=<real@example.net> to=<b@in.example.com> proto=ESMTP helo=<x this server's deny list; from=<spoofed@example.org> to=<b@in.example.com> proto=ESMTP helo=<y>", tailNow, false)
|
||||||
|
if len(st.rows) != 1 || st.rows[0].Detail != "5.7.1 Spam message rejected" || st.rows[0].From != "real@example.net" {
|
||||||
|
t.Errorf("rows = %+v", st.rows)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A queue id is port 25's only until the message is queued or the id is
|
||||||
|
// opened again by another listener - Postfix reuses ids.
|
||||||
|
func TestInboundQueueIDIsForgotten(t *testing.T) {
|
||||||
|
const verdict = "postfix/cleanup[52]: Q1: milter-reject: END-OF-MESSAGE from app[203.0.113.4]: 4.7.1 Service unavailable - try again later; from=<a@example.net> to=<b@in.example.com> proto=ESMTP helo=<app>"
|
||||||
|
for name, forget := range map[string]string{
|
||||||
|
"queued": "postfix/qmgr[58]: Q1: from=<a@example.net>, size=600, nrcpt=1 (queue active)",
|
||||||
|
"reused by the submission port": "postfix/smtpd[20]: Q1: client=app.example.com[203.0.113.4], sasl_method=PLAIN, sasl_username=app",
|
||||||
|
} {
|
||||||
|
j, st := inboundFixture()
|
||||||
|
j.handle(head+"postfix/inbound/smtpd[41]: Q1: client=mx.example.net[203.0.113.9]", tailNow, false)
|
||||||
|
j.handle(head+forget, tailNow, false)
|
||||||
|
j.handle(head+verdict, tailNow, false)
|
||||||
|
if len(st.rows) != 0 || len(j.port25) != 0 {
|
||||||
|
t.Errorf("%s: rows %+v, remembered %v", name, st.rows, j.port25)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// An id that was forgotten and opened again is not dropped by its own stale
|
||||||
|
// place in the queue of arrivals.
|
||||||
|
func TestInboundStaleArrivalDoesNotDropALiveID(t *testing.T) {
|
||||||
|
j, _ := inboundFixture()
|
||||||
|
j.remember("Q1")
|
||||||
|
delete(j.port25, "Q1") // its verdict came
|
||||||
|
for i := 0; i < inboundQueueIDs-2; i++ {
|
||||||
|
j.remember("F" + time.Duration(i).String())
|
||||||
|
}
|
||||||
|
j.remember("Q1") // reused; order is now full, with the stale Q1 first
|
||||||
|
j.remember("LAST")
|
||||||
|
if _, ok := j.port25["Q1"]; !ok {
|
||||||
|
t.Error("the live Q1 was dropped by its stale arrival")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Between the six-hourly prunes the tailer has the journal cut to its cap
|
||||||
|
// itself, every so many rows: refusals come at the rate strangers send them.
|
||||||
|
func TestInboundCutsTheJournalAsItWrites(t *testing.T) {
|
||||||
|
j, st := inboundFixture()
|
||||||
|
for i := 0; i < inboundPruneEvery*2+10; i++ {
|
||||||
|
j.handle(unknownRcp, tailNow, false)
|
||||||
|
}
|
||||||
|
if len(st.pruned) != 2 || st.pruned[0] != inboundSpamLogMaxRows {
|
||||||
|
t.Errorf("cuts = %v, want two, to %d rows", st.pruned, inboundSpamLogMaxRows)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The queue ids waiting for a verdict are bounded.
|
||||||
|
func TestInboundQueueIDsAreBounded(t *testing.T) {
|
||||||
|
j, _ := inboundFixture()
|
||||||
|
for i := 0; i < inboundQueueIDs+10; i++ {
|
||||||
|
j.remember("Q" + strings.Repeat("A", i%7) + string(rune('0'+i%10)) + time.Duration(i).String())
|
||||||
|
}
|
||||||
|
if len(j.port25) != inboundQueueIDs || len(j.order) != inboundQueueIDs {
|
||||||
|
t.Errorf("remembered %d ids (%d in order), want %d", len(j.port25), len(j.order), inboundQueueIDs)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLineTime(t *testing.T) {
|
||||||
|
now := time.Date(2026, 1, 2, 10, 0, 0, 0, time.UTC)
|
||||||
|
for _, c := range []struct {
|
||||||
|
line string
|
||||||
|
want time.Time
|
||||||
|
rest string
|
||||||
|
}{
|
||||||
|
{"2026-01-02T09:00:00.5+03:00 mail postfix/x: y", time.Date(2026, 1, 2, 6, 0, 0, 500000000, time.UTC), "mail postfix/x: y"},
|
||||||
|
{"Jan 2 09:30:00 mail postfix/x: y", time.Date(2026, 1, 2, 9, 30, 0, 0, time.Local).UTC(), "mail postfix/x: y"},
|
||||||
|
// December read in January is last year's.
|
||||||
|
{"Dec 31 23:59:00 mail postfix/x: y", time.Date(2025, 12, 31, 23, 59, 0, 0, time.Local).UTC(), "mail postfix/x: y"},
|
||||||
|
{"no stamp here", now, "no stamp here"},
|
||||||
|
} {
|
||||||
|
got, rest := lineTime(c.line, now)
|
||||||
|
if !got.Equal(c.want) || rest != c.rest {
|
||||||
|
t.Errorf("lineTime(%q) = %v, %q; want %v, %q", c.line, got, rest, c.want, c.rest)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInboundRetentionPrunesAtStart(t *testing.T) {
|
||||||
|
old := retentionInterval
|
||||||
|
retentionInterval = time.Hour
|
||||||
|
defer func() { retentionInterval = old }()
|
||||||
|
|
||||||
|
st := &fakeInboundStore{}
|
||||||
|
ctx, cancel := contextWithCancel()
|
||||||
|
done := make(chan struct{})
|
||||||
|
go func() {
|
||||||
|
inboundRetentionLoop(ctx, &Inbound{Store: st, Retention: func() int { return 0 }})
|
||||||
|
close(done)
|
||||||
|
}()
|
||||||
|
deadline := time.Now().Add(2 * time.Second)
|
||||||
|
for len(st.pruned) == 0 && time.Now().Before(deadline) {
|
||||||
|
time.Sleep(10 * time.Millisecond)
|
||||||
|
}
|
||||||
|
cancel()
|
||||||
|
<-done
|
||||||
|
if len(st.pruned) != 1 || st.pruned[0] != inboundSpamLogMaxRows {
|
||||||
|
t.Errorf("prune calls = %v", st.pruned)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func contextWithCancel() (context.Context, context.CancelFunc) {
|
||||||
|
return context.WithCancel(context.Background())
|
||||||
|
}
|
||||||
|
|
||||||
|
// The lines the tailer reads when it starts may be ones it read before its
|
||||||
|
// last stop: a refusal among them is one row, not two. Lines read as they are
|
||||||
|
// written are always new.
|
||||||
|
func TestInboundCatchUpDoesNotWriteARowTwice(t *testing.T) {
|
||||||
|
j, st := inboundFixture()
|
||||||
|
j.handle(unknownRcp, tailNow, false) // before the stop
|
||||||
|
j.handle(unknownRcp, tailNow, true) // the same line, read again on start
|
||||||
|
if len(st.rows) != 1 {
|
||||||
|
t.Fatalf("rows = %d, want the one refusal", len(st.rows))
|
||||||
|
}
|
||||||
|
j.handle(unknownRcp, tailNow, false) // a second refusal in the same second
|
||||||
|
if len(st.rows) != 2 {
|
||||||
|
t.Errorf("rows = %d, want two: a line read as it is written is new", len(st.rows))
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -107,9 +107,16 @@ func parseDelivery(line string) (queueID, recipient, status string, ok bool) {
|
|||||||
// Run follows path and updates send-log statuses until ctx is cancelled, while
|
// Run follows path and updates send-log statuses until ctx is cancelled, while
|
||||||
// a background sweep prunes rows older than retention(). Reading resumes at
|
// a background sweep prunes rows older than retention(). Reading resumes at
|
||||||
// the offset the previous run persisted, so a restart parses the delivery lines
|
// the offset the previous run persisted, so a restart parses the delivery lines
|
||||||
// written while the panel was down. It returns nil on a clean shutdown.
|
// written while the panel was down. With inbound set it also writes the
|
||||||
func Run(ctx context.Context, path string, st StatusStore, retention RetentionDays) error {
|
// inbound journal's rows for refused mail and prunes that journal
|
||||||
|
// (inbound.go). It returns nil on a clean shutdown.
|
||||||
|
func Run(ctx context.Context, path string, st StatusStore, retention RetentionDays, inbound *Inbound) error {
|
||||||
go retentionLoop(ctx, st, retention)
|
go retentionLoop(ctx, st, retention)
|
||||||
|
var journal *inboundJournal
|
||||||
|
if inbound != nil {
|
||||||
|
go inboundRetentionLoop(ctx, inbound)
|
||||||
|
journal = newInboundJournal(inbound.Store)
|
||||||
|
}
|
||||||
|
|
||||||
// The reconcile sweep must not run against a backlog the tailer has not
|
// The reconcile sweep must not run against a backlog the tailer has not
|
||||||
// read yet: on a restart the log holds the very lines that resolve the rows
|
// read yet: on a restart the log holds the very lines that resolve the rows
|
||||||
@@ -119,6 +126,15 @@ func Run(ctx context.Context, path string, st StatusStore, retention RetentionDa
|
|||||||
go reconcileLoop(ctx, st, caughtUp)
|
go reconcileLoop(ctx, st, caughtUp)
|
||||||
|
|
||||||
return follow(ctx, path, &tracker{st: st, path: path}, caughtUp, func(line string) {
|
return follow(ctx, path, &tracker{st: st, path: path}, caughtUp, func(line string) {
|
||||||
|
if journal != nil {
|
||||||
|
catchingUp := true
|
||||||
|
select {
|
||||||
|
case <-caughtUp:
|
||||||
|
catchingUp = false
|
||||||
|
default:
|
||||||
|
}
|
||||||
|
journal.handle(line, time.Now().UTC(), catchingUp)
|
||||||
|
}
|
||||||
queueID, recipient, status, ok := parseDelivery(line)
|
queueID, recipient, status, ok := parseDelivery(line)
|
||||||
if !ok {
|
if !ok {
|
||||||
return
|
return
|
||||||
|
|||||||
@@ -212,7 +212,7 @@ func TestFollowTailsAndRotates(t *testing.T) {
|
|||||||
cs := &captureStore{}
|
cs := &captureStore{}
|
||||||
ctx, cancel := context.WithCancel(context.Background())
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
done := make(chan error, 1)
|
done := make(chan error, 1)
|
||||||
go func() { done <- Run(ctx, path, cs, func() int { return 90 }) }()
|
go func() { done <- Run(ctx, path, cs, func() int { return 90 }, nil) }()
|
||||||
|
|
||||||
// Give follow() time to open at EOF (it seeks to end immediately on start,
|
// Give follow() time to open at EOF (it seeks to end immediately on start,
|
||||||
// so the seed line above is ignored), then append a delivery line.
|
// so the seed line above is ignored), then append a delivery line.
|
||||||
@@ -295,7 +295,7 @@ func startRun(t *testing.T, path string, cs *captureStore) func() {
|
|||||||
t.Helper()
|
t.Helper()
|
||||||
ctx, cancel := context.WithCancel(context.Background())
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
done := make(chan error, 1)
|
done := make(chan error, 1)
|
||||||
go func() { done <- Run(ctx, path, cs, func() int { return 90 }) }()
|
go func() { done <- Run(ctx, path, cs, func() int { return 90 }, nil) }()
|
||||||
// follow() opens and seeks on start; give it a moment before the caller
|
// follow() opens and seeks on start; give it a moment before the caller
|
||||||
// appends, so the append is not raced by the initial open.
|
// appends, so the append is not raced by the initial open.
|
||||||
time.Sleep(50 * time.Millisecond)
|
time.Sleep(50 * time.Millisecond)
|
||||||
|
|||||||
@@ -20,8 +20,10 @@ const (
|
|||||||
// which os.SameFile cannot answer across a restart.
|
// which os.SameFile cannot answer across a restart.
|
||||||
fingerprintSize = 512
|
fingerprintSize = 512
|
||||||
// persistInterval throttles the offset write. Losing up to this much
|
// persistInterval throttles the offset write. Losing up to this much
|
||||||
// progress on a crash only means re-parsing a few lines (UpdateStatus is
|
// progress on a crash means re-parsing a few lines, which is much cheaper
|
||||||
// idempotent), which is much cheaper than a database write per poll tick.
|
// than a database write per poll tick. For a delivery status that changes
|
||||||
|
// nothing (UpdateStatus is idempotent), and a refusal among them is not
|
||||||
|
// written to the inbound journal a second time (inboundJournal.handle).
|
||||||
persistInterval = 5 * time.Second
|
persistInterval = 5 * time.Second
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -46,8 +48,9 @@ type tracker struct {
|
|||||||
// state whose fingerprint still matches means continue from it, parsing the
|
// state whose fingerprint still matches means continue from it, parsing the
|
||||||
// tail written while the panel was down; anything else means the file is not
|
// tail written while the panel was down; anything else means the file is not
|
||||||
// the one the offset referred to (rotated, recreated or truncated in the
|
// the one the offset referred to (rotated, recreated or truncated in the
|
||||||
// meantime), so read it from the start. Re-parsing lines already seen is
|
// meantime), so read it from the start. Re-parsing lines already seen writes
|
||||||
// harmless: UpdateStatus writes the same status onto the same row.
|
// the same status onto the same send-log row, and no second inbound journal
|
||||||
|
// row for a refusal among them.
|
||||||
func (t *tracker) resume(f *os.File) int64 {
|
func (t *tracker) resume(f *os.File) int64 {
|
||||||
size, err := fileSize(f)
|
size, err := fileSize(f)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
|||||||
@@ -8,11 +8,12 @@ import (
|
|||||||
)
|
)
|
||||||
|
|
||||||
// authIPAllowed reports whether the authenticated application may submit from
|
// authIPAllowed reports whether the authenticated application may submit from
|
||||||
// the connecting client IP. When the application has no IP restriction, or the
|
// the connecting client IP. When the application has no IP restriction the
|
||||||
// client IP is not known, the check passes. Store errors are fail-open (see
|
// check passes. When it has one and the client's address is not known, the
|
||||||
// overLimit).
|
// check fails: an address list that an unknown address satisfies is no list.
|
||||||
|
// Store errors are fail-open (see overLimit).
|
||||||
func (s *session) authIPAllowed() bool {
|
func (s *session) authIPAllowed() bool {
|
||||||
if s.login == "" || s.clientIP == "" {
|
if s.login == "" {
|
||||||
return true
|
return true
|
||||||
}
|
}
|
||||||
a, err := s.rec.ApplicationByLogin(s.login)
|
a, err := s.rec.ApplicationByLogin(s.login)
|
||||||
@@ -26,6 +27,10 @@ func (s *session) authIPAllowed() bool {
|
|||||||
if a.AllowsAuthFromIP(s.clientIP) {
|
if a.AllowsAuthFromIP(s.clientIP) {
|
||||||
return true
|
return true
|
||||||
}
|
}
|
||||||
|
if s.clientIP == "" {
|
||||||
|
log.Printf("journal-milter: application %q refused — its client IPs are restricted and Postfix gave no client address", s.login)
|
||||||
|
return false
|
||||||
|
}
|
||||||
log.Printf("journal-milter: application %q refused from %s — client IP not allowed", s.login, s.clientIP)
|
log.Printf("journal-milter: application %q refused from %s — client IP not allowed", s.login, s.clientIP)
|
||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,290 @@
|
|||||||
|
package milter
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"crypto/hmac"
|
||||||
|
"crypto/sha256"
|
||||||
|
"encoding/hex"
|
||||||
|
"log"
|
||||||
|
"net"
|
||||||
|
"net/textproto"
|
||||||
|
"regexp"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"github.com/emersion/go-milter"
|
||||||
|
|
||||||
|
"github.com/mixeme/selfpost/internal/mailhdr"
|
||||||
|
"github.com/mixeme/selfpost/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
// InboundStore is what the inbound journal milter needs: the journal itself
|
||||||
|
// and the list of inbound domains, since port 25 also takes mail that is not
|
||||||
|
// the relay's. *store.Store satisfies it; tests substitute a fake.
|
||||||
|
type InboundStore interface {
|
||||||
|
InsertInboundSpam(e store.InboundSpamEntry) error
|
||||||
|
IsInboundDomain(name string) (bool, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
// inboundSession is the inbound journal milter: the last milter on port 25,
|
||||||
|
// after the optional external filter. It writes one journal row per recipient
|
||||||
|
// of a message that reached end-of-message - that is, one the filter did not
|
||||||
|
// refuse; refusals never get this far and are read from mail.log instead.
|
||||||
|
//
|
||||||
|
// Like the send-log milter it only watches: every callback continues, and
|
||||||
|
// Postfix attaches the socket with default_action=accept, so
|
||||||
|
// neither an error here nor the panel being down holds up inbound mail.
|
||||||
|
type inboundSession struct {
|
||||||
|
milter.NoOpMilter
|
||||||
|
rec InboundStore
|
||||||
|
// key is what the filter signs each message with (FilterSignature), or
|
||||||
|
// empty when no filter is attached. A verdict is read only next to a
|
||||||
|
// signature that fits: the filter is attached so that mail flows when it is
|
||||||
|
// down, and then the verdict headers of a message are whatever its sender
|
||||||
|
// wrote.
|
||||||
|
key string
|
||||||
|
|
||||||
|
// clientIP is taken at Connect and again from {client_addr} with every
|
||||||
|
// MAIL FROM: a session the library made after a non-continue reply never
|
||||||
|
// sees Connect (see session.MailFrom in milter.go).
|
||||||
|
clientIP string
|
||||||
|
|
||||||
|
from string
|
||||||
|
rcpts []string
|
||||||
|
subject string
|
||||||
|
action string // X-Rspamd-Action
|
||||||
|
result string // X-Spamd-Result
|
||||||
|
seen bool // any verdict header of the filter
|
||||||
|
sig string // X-SelfPost-Filter
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *inboundSession) Connect(host, family string, port uint16, addr net.IP, m *milter.Modifier) (milter.Response, error) {
|
||||||
|
if addr != nil {
|
||||||
|
s.clientIP = addr.String()
|
||||||
|
}
|
||||||
|
return milter.RespContinue, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// MailFrom starts a message: a connection may carry several.
|
||||||
|
func (s *inboundSession) MailFrom(from string, m *milter.Modifier) (milter.Response, error) {
|
||||||
|
s.reset()
|
||||||
|
if ip := clientAddr(m); ip != "" {
|
||||||
|
s.clientIP = ip
|
||||||
|
}
|
||||||
|
s.from = cleanAddress(from)
|
||||||
|
return milter.RespContinue, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *inboundSession) RcptTo(rcpt string, m *milter.Modifier) (milter.Response, error) {
|
||||||
|
s.rcpts = append(s.rcpts, cleanAddress(rcpt))
|
||||||
|
return milter.RespContinue, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Header keeps the first Subject, the filter's verdict and its signature.
|
||||||
|
// Postfix hands the message to its milters one after another at
|
||||||
|
// end-of-message, each seeing what the one before added, which is how the
|
||||||
|
// filter's headers arrive here. The filter replaces headers of these names, so
|
||||||
|
// the last one read is its own - when it is up. Whether it was is decided at
|
||||||
|
// end-of-message, by the signature.
|
||||||
|
func (s *inboundSession) Header(name, value string, m *milter.Modifier) (milter.Response, error) {
|
||||||
|
switch textproto.CanonicalMIMEHeaderKey(name) {
|
||||||
|
case "Subject":
|
||||||
|
if s.subject == "" {
|
||||||
|
s.subject = mailhdr.DecodeSubject(value)
|
||||||
|
}
|
||||||
|
case "X-Rspamd-Action":
|
||||||
|
s.action, s.seen = value, true
|
||||||
|
case "X-Spamd-Result":
|
||||||
|
s.result, s.seen = value, true
|
||||||
|
case signatureHeader:
|
||||||
|
s.sig = strings.TrimSpace(value)
|
||||||
|
}
|
||||||
|
return milter.RespContinue, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// SignatureHeader is the header the filter signs a message in
|
||||||
|
// (deploy/antispam/rspamd/milter_headers.conf writes it).
|
||||||
|
const SignatureHeader = "X-SelfPost-Filter"
|
||||||
|
|
||||||
|
var signatureHeader = textproto.CanonicalMIMEHeaderKey(SignatureHeader)
|
||||||
|
|
||||||
|
// FilterSignature is what the filter writes into SignatureHeader for a message:
|
||||||
|
// the HMAC-SHA256 of its queue id under the key SelfPost shares with the
|
||||||
|
// filter, in hex. The key is used as the text it is stored as. A signature
|
||||||
|
// fits one queue id, so the header going on to the upstream with the mail
|
||||||
|
// gives nothing to whoever reads it there.
|
||||||
|
func FilterSignature(key, queueID string) string {
|
||||||
|
mac := hmac.New(sha256.New, []byte(key))
|
||||||
|
mac.Write([]byte(queueID))
|
||||||
|
return hex.EncodeToString(mac.Sum(nil))
|
||||||
|
}
|
||||||
|
|
||||||
|
// signed reports whether the message carries the filter's signature for its
|
||||||
|
// queue id. No key, no queue id or no signature is "no".
|
||||||
|
func (s *inboundSession) signed(queueID string) bool {
|
||||||
|
if s.key == "" || queueID == "" || s.sig == "" {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
return hmac.Equal([]byte(strings.ToLower(s.sig)), []byte(FilterSignature(s.key, queueID)))
|
||||||
|
}
|
||||||
|
|
||||||
|
// Body fires at end-of-message: the message is about to be queued for its
|
||||||
|
// upstream. "Continue" lets it through as "accept" would, and keeps this
|
||||||
|
// session for the connection's next message.
|
||||||
|
func (s *inboundSession) Body(m *milter.Modifier) (milter.Response, error) {
|
||||||
|
// The queue id is known only now, and with it whether the verdict headers
|
||||||
|
// read on the way here are the filter's.
|
||||||
|
if !s.signed(macro(m, "i")) {
|
||||||
|
s.seen = false
|
||||||
|
}
|
||||||
|
s.record()
|
||||||
|
s.reset()
|
||||||
|
return milter.RespContinue, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *inboundSession) Abort(m *milter.Modifier) error {
|
||||||
|
s.reset()
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *inboundSession) reset() {
|
||||||
|
s.from, s.rcpts, s.subject = "", nil, ""
|
||||||
|
s.action, s.result, s.seen, s.sig = "", "", false, ""
|
||||||
|
}
|
||||||
|
|
||||||
|
// record writes one row per recipient at an inbound domain. Other recipients
|
||||||
|
// of port 25 (a DMARC report address at the server's own name) are not the
|
||||||
|
// relay's mail and get no row. Failures are logged, never propagated.
|
||||||
|
func (s *inboundSession) record() {
|
||||||
|
e := store.InboundSpamEntry{
|
||||||
|
ClientIP: s.clientIP,
|
||||||
|
From: s.from,
|
||||||
|
Subject: s.subject,
|
||||||
|
Decision: store.SpamDecisionAccept,
|
||||||
|
Engine: store.SpamEngineSelfPost,
|
||||||
|
}
|
||||||
|
if s.seen {
|
||||||
|
action, detail := rspamdVerdict(s.action, s.result)
|
||||||
|
e.Engine, e.Detail = store.SpamEngineRspamd, detail
|
||||||
|
if action == "quarantine" {
|
||||||
|
e.Decision = store.SpamDecisionQuarantine
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, to := range s.rcpts {
|
||||||
|
domain := domainOf(to)
|
||||||
|
ok, err := s.rec.IsInboundDomain(domain)
|
||||||
|
if err != nil {
|
||||||
|
log.Printf("inbound-journal: look up %s: %v", domain, err)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if !ok {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
e.InboundDomain, e.To = domain, to
|
||||||
|
if err := s.rec.InsertInboundSpam(e); err != nil {
|
||||||
|
log.Printf("inbound-journal: record %s -> %s: %v", s.from, to, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// What the journal keeps of a verdict: enough to see why, on one line.
|
||||||
|
const (
|
||||||
|
verdictMaxSymbols = 6
|
||||||
|
verdictMaxBytes = 255
|
||||||
|
)
|
||||||
|
|
||||||
|
// ListSymbolPrefix starts the names of the symbols the filter lists' rules add
|
||||||
|
// (deploy/antispam/rspamd/multimap.conf): SELFPOST_ALLOW_IP, _FROM, _DOMAIN.
|
||||||
|
const ListSymbolPrefix = "SELFPOST_"
|
||||||
|
|
||||||
|
var (
|
||||||
|
// "default: False [7.20 / 15.00]" - the score and the reject threshold.
|
||||||
|
rspamdScore = regexp.MustCompile(`\[(-?[0-9]+(?:\.[0-9]+)?) / (-?[0-9]+(?:\.[0-9]+)?)\]`)
|
||||||
|
// "BAYES_SPAM(5.10)[100.00%]" - a symbol and its weight.
|
||||||
|
rspamdSymbol = regexp.MustCompile(`([A-Z][A-Z0-9_]{1,63})\((-?[0-9]+(?:\.[0-9]+)?)\)`)
|
||||||
|
)
|
||||||
|
|
||||||
|
// rspamdVerdict reduces the filter's headers to an action and a one-line
|
||||||
|
// summary: the action, the score against the threshold, and the first few
|
||||||
|
// symbols that carried weight. It reads only what it recognises - the headers
|
||||||
|
// are read only next to the filter's signature, and even so nothing of them is
|
||||||
|
// kept verbatim except through these patterns.
|
||||||
|
func rspamdVerdict(actionHeader, resultHeader string) (action, detail string) {
|
||||||
|
action = strings.ToLower(strings.TrimSpace(actionHeader))
|
||||||
|
for _, r := range action {
|
||||||
|
if (r < 'a' || r > 'z') && r != ' ' {
|
||||||
|
action = ""
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(action) > 32 {
|
||||||
|
action = ""
|
||||||
|
}
|
||||||
|
|
||||||
|
var parts []string
|
||||||
|
if action != "" {
|
||||||
|
parts = append(parts, action)
|
||||||
|
}
|
||||||
|
if m := rspamdScore.FindStringSubmatch(resultHeader); m != nil {
|
||||||
|
parts = append(parts, "score "+m[1]+" / "+m[2])
|
||||||
|
}
|
||||||
|
detail = strings.Join(parts, ", ")
|
||||||
|
|
||||||
|
// The filter lists' own symbols come first and are kept whatever else there
|
||||||
|
// is: Filter lists counts how often an entry decided by finding them here,
|
||||||
|
// and rspamd writes symbols heaviest first, which puts an allow entry's
|
||||||
|
// -100 last, behind any six others.
|
||||||
|
var symbols, rest []string
|
||||||
|
for _, m := range rspamdSymbol.FindAllStringSubmatch(resultHeader, -1) {
|
||||||
|
if strings.Trim(m[2], "-0.") == "" { // weight zero: informational
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if strings.HasPrefix(m[1], ListSymbolPrefix) {
|
||||||
|
if len(symbols) < verdictMaxSymbols {
|
||||||
|
symbols = append(symbols, m[1]+"("+m[2]+")")
|
||||||
|
}
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
rest = append(rest, m[1]+"("+m[2]+")")
|
||||||
|
}
|
||||||
|
for _, sym := range rest {
|
||||||
|
if len(symbols) == verdictMaxSymbols {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
symbols = append(symbols, sym)
|
||||||
|
}
|
||||||
|
if len(symbols) > 0 {
|
||||||
|
if detail != "" {
|
||||||
|
detail += "; "
|
||||||
|
}
|
||||||
|
detail += strings.Join(symbols, " ")
|
||||||
|
}
|
||||||
|
if len(detail) > verdictMaxBytes {
|
||||||
|
detail = detail[:verdictMaxBytes]
|
||||||
|
}
|
||||||
|
return action, detail
|
||||||
|
}
|
||||||
|
|
||||||
|
// ServeInbound runs the inbound journal milter on ln until ctx is cancelled.
|
||||||
|
// key is what the external filter in front of it signs messages with, or
|
||||||
|
// empty when none is attached (INBOUND_ANTISPAM_MILTER): no verdict is then
|
||||||
|
// read at all. It returns nil on a clean shutdown.
|
||||||
|
func ServeInbound(ctx context.Context, ln net.Listener, rec InboundStore, key string) error {
|
||||||
|
srv := &milter.Server{
|
||||||
|
NewMilter: func() milter.Milter { return &inboundSession{rec: rec, key: key} },
|
||||||
|
Actions: 0, // read-only
|
||||||
|
Protocol: milter.OptNoBody, // headers and end-of-message, not the body
|
||||||
|
}
|
||||||
|
|
||||||
|
go func() {
|
||||||
|
<-ctx.Done()
|
||||||
|
_ = srv.Close()
|
||||||
|
}()
|
||||||
|
|
||||||
|
if err := srv.Serve(ln); err != nil {
|
||||||
|
if ctx.Err() != nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,350 @@
|
|||||||
|
package milter
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"net"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/emersion/go-milter"
|
||||||
|
|
||||||
|
"github.com/mixeme/selfpost/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
// fakeInbound is the journal and the inbound-domain list of one test.
|
||||||
|
type fakeInbound struct {
|
||||||
|
domains map[string]bool
|
||||||
|
rows []store.InboundSpamEntry
|
||||||
|
insertErr error
|
||||||
|
lookupErr error
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeInbound) InsertInboundSpam(e store.InboundSpamEntry) error {
|
||||||
|
if f.insertErr != nil {
|
||||||
|
return f.insertErr
|
||||||
|
}
|
||||||
|
f.rows = append(f.rows, e)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeInbound) IsInboundDomain(name string) (bool, error) {
|
||||||
|
if f.lookupErr != nil {
|
||||||
|
return false, f.lookupErr
|
||||||
|
}
|
||||||
|
return f.domains[name], nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// A header of the kind the documented rspamd sidecar adds.
|
||||||
|
const spamdResult = "default: False [7.20 / 15.00];\n\tBAYES_SPAM(5.10)[100.00%];\n\tMIME_GOOD(-0.10)[text/plain];\n\tARC_NA(0.00)[];\n\tRCVD_COUNT_ZERO(0.00)[0];\n\tR_DKIM_NA(2.20)[]"
|
||||||
|
|
||||||
|
// message replays one message through s: envelope, headers in order, end.
|
||||||
|
func message(t *testing.T, s *inboundSession, from string, rcpts []string, headers [][2]string) milter.Response {
|
||||||
|
t.Helper()
|
||||||
|
if _, err := s.MailFrom(from, mods(nil)); err != nil {
|
||||||
|
t.Fatalf("MailFrom: %v", err)
|
||||||
|
}
|
||||||
|
for _, r := range rcpts {
|
||||||
|
if _, err := s.RcptTo(r, mods(nil)); err != nil {
|
||||||
|
t.Fatalf("RcptTo: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, h := range headers {
|
||||||
|
if _, err := s.Header(h[0], h[1], mods(nil)); err != nil {
|
||||||
|
t.Fatalf("Header: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
resp, err := s.Body(mods(map[string]string{"i": "ABC123"}))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Body: %v", err)
|
||||||
|
}
|
||||||
|
return resp
|
||||||
|
}
|
||||||
|
|
||||||
|
// testKey is the key the tests' filter signs with; the messages they replay
|
||||||
|
// have the queue id ABC123.
|
||||||
|
const testKey = "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef"
|
||||||
|
|
||||||
|
var testSig = [2]string{SignatureHeader, FilterSignature(testKey, "ABC123")}
|
||||||
|
|
||||||
|
func inboundSessionFor(t *testing.T, rec InboundStore, filter bool) *inboundSession {
|
||||||
|
t.Helper()
|
||||||
|
s := &inboundSession{rec: rec}
|
||||||
|
if filter {
|
||||||
|
s.key = testKey
|
||||||
|
}
|
||||||
|
if _, err := s.Connect("mx.example.net", "tcp4", 0, net.ParseIP("203.0.113.7"), mods(nil)); err != nil {
|
||||||
|
t.Fatalf("Connect: %v", err)
|
||||||
|
}
|
||||||
|
return s
|
||||||
|
}
|
||||||
|
|
||||||
|
// With no filter attached an accepted message is a plain row, one per
|
||||||
|
// recipient, decided by nobody.
|
||||||
|
func TestInboundRecordsAcceptedMessage(t *testing.T) {
|
||||||
|
rec := &fakeInbound{domains: map[string]bool{"in.example.com": true}}
|
||||||
|
s := inboundSessionFor(t, rec, false)
|
||||||
|
resp := message(t, s, "<sender@example.net> SIZE=1200",
|
||||||
|
[]string{"<a@in.example.com>", "b@IN.example.com"},
|
||||||
|
[][2]string{{"Subject", "=?utf-8?Q?Sch=C3=B6n?="}})
|
||||||
|
if resp != milter.RespContinue {
|
||||||
|
t.Errorf("end-of-message response = %v, want continue", resp)
|
||||||
|
}
|
||||||
|
if len(rec.rows) != 2 {
|
||||||
|
t.Fatalf("rows = %+v, want one per recipient", rec.rows)
|
||||||
|
}
|
||||||
|
want := store.InboundSpamEntry{
|
||||||
|
InboundDomain: "in.example.com", ClientIP: "203.0.113.7",
|
||||||
|
From: "sender@example.net", To: "a@in.example.com", Subject: "Schön",
|
||||||
|
Decision: store.SpamDecisionAccept, Engine: store.SpamEngineSelfPost,
|
||||||
|
}
|
||||||
|
if rec.rows[0] != want {
|
||||||
|
t.Errorf("row = %+v\nwant %+v", rec.rows[0], want)
|
||||||
|
}
|
||||||
|
if rec.rows[1].To != "b@IN.example.com" || rec.rows[1].InboundDomain != "in.example.com" {
|
||||||
|
t.Errorf("second row = %+v", rec.rows[1])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Port 25 also takes DMARC reports for the server's own name. They are not
|
||||||
|
// the relay's mail and the journal says nothing about them.
|
||||||
|
func TestInboundSkipsRecipientsOutsideInboundDomains(t *testing.T) {
|
||||||
|
rec := &fakeInbound{domains: map[string]bool{"in.example.com": true}}
|
||||||
|
s := inboundSessionFor(t, rec, false)
|
||||||
|
message(t, s, "noreply-dmarc@google.com",
|
||||||
|
[]string{"dmarc-example.com@mail.example.org", "a@in.example.com"}, nil)
|
||||||
|
if len(rec.rows) != 1 || rec.rows[0].To != "a@in.example.com" {
|
||||||
|
t.Errorf("rows = %+v, want the inbound recipient alone", rec.rows)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// With the filter attached its verdict headers become the engine and detail.
|
||||||
|
func TestInboundReadsTheFiltersVerdict(t *testing.T) {
|
||||||
|
rec := &fakeInbound{domains: map[string]bool{"in.example.com": true}}
|
||||||
|
s := inboundSessionFor(t, rec, true)
|
||||||
|
message(t, s, "sender@example.net", []string{"a@in.example.com"}, [][2]string{
|
||||||
|
{"Subject", "Cheap pills"},
|
||||||
|
{"X-Rspamd-Action", "add header"},
|
||||||
|
{"X-Spamd-Result", spamdResult},
|
||||||
|
testSig,
|
||||||
|
})
|
||||||
|
if len(rec.rows) != 1 {
|
||||||
|
t.Fatalf("rows = %+v", rec.rows)
|
||||||
|
}
|
||||||
|
got := rec.rows[0]
|
||||||
|
if got.Engine != store.SpamEngineRspamd || got.Decision != store.SpamDecisionAccept {
|
||||||
|
t.Errorf("engine %q, decision %q", got.Engine, got.Decision)
|
||||||
|
}
|
||||||
|
if want := "add header, score 7.20 / 15.00; BAYES_SPAM(5.10) MIME_GOOD(-0.10) R_DKIM_NA(2.20)"; got.Detail != want {
|
||||||
|
t.Errorf("detail = %q\nwant %q", got.Detail, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInboundQuarantineAction(t *testing.T) {
|
||||||
|
rec := &fakeInbound{domains: map[string]bool{"in.example.com": true}}
|
||||||
|
s := inboundSessionFor(t, rec, true)
|
||||||
|
message(t, s, "sender@example.net", []string{"a@in.example.com"},
|
||||||
|
[][2]string{{"x-rspamd-action", "quarantine"}, {"x-selfpost-filter", strings.ToUpper(testSig[1])}})
|
||||||
|
if len(rec.rows) != 1 || rec.rows[0].Decision != store.SpamDecisionQuarantine || rec.rows[0].Detail != "quarantine" {
|
||||||
|
t.Errorf("rows = %+v", rec.rows)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Without a filter in front, a verdict header is the sender's own writing.
|
||||||
|
func TestInboundIgnoresVerdictHeadersWithoutAFilter(t *testing.T) {
|
||||||
|
rec := &fakeInbound{domains: map[string]bool{"in.example.com": true}}
|
||||||
|
s := inboundSessionFor(t, rec, false)
|
||||||
|
message(t, s, "sender@example.net", []string{"a@in.example.com"}, [][2]string{
|
||||||
|
{"X-Rspamd-Action", "no action"},
|
||||||
|
{"X-Spamd-Result", spamdResult},
|
||||||
|
})
|
||||||
|
if len(rec.rows) != 1 || rec.rows[0].Engine != store.SpamEngineSelfPost || rec.rows[0].Detail != "" {
|
||||||
|
t.Errorf("rows = %+v", rec.rows)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The filter is attached fail-open, so with it down the headers are again the
|
||||||
|
// sender's: nothing of them reaches the journal except through the patterns.
|
||||||
|
func TestRspamdVerdictKeepsOnlyWhatItRecognises(t *testing.T) {
|
||||||
|
action, detail := rspamdVerdict("<script>alert(1)</script>", "default: False [1 / 15]; <b>HTML</b> lower_case(9.9) \x1b[31m")
|
||||||
|
if action != "" {
|
||||||
|
t.Errorf("action = %q, want it dropped", action)
|
||||||
|
}
|
||||||
|
if detail != "score 1 / 15" {
|
||||||
|
t.Errorf("detail = %q", detail)
|
||||||
|
}
|
||||||
|
|
||||||
|
var many strings.Builder
|
||||||
|
for i := 0; i < 400; i++ {
|
||||||
|
many.WriteString("SYMBOL_WITH_A_LONG_NAME_" + strings.Repeat("X", 30) + "(1.00)[];")
|
||||||
|
}
|
||||||
|
if _, detail := rspamdVerdict("reject", many.String()); len(detail) > verdictMaxBytes || strings.Count(detail, "(") > verdictMaxSymbols {
|
||||||
|
t.Errorf("detail of %d bytes with %d symbols", len(detail), strings.Count(detail, "("))
|
||||||
|
}
|
||||||
|
if action, _ := rspamdVerdict(strings.Repeat("a", 33), ""); action != "" {
|
||||||
|
t.Errorf("an action of 33 letters was kept")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A second message on the same connection starts clean.
|
||||||
|
func TestInboundResetsBetweenMessages(t *testing.T) {
|
||||||
|
rec := &fakeInbound{domains: map[string]bool{"in.example.com": true}}
|
||||||
|
s := inboundSessionFor(t, rec, true)
|
||||||
|
message(t, s, "one@example.net", []string{"a@in.example.com"}, [][2]string{
|
||||||
|
{"Subject", "First"}, {"X-Rspamd-Action", "add header"}, testSig,
|
||||||
|
})
|
||||||
|
message(t, s, "two@example.net", []string{"b@in.example.com"}, nil)
|
||||||
|
if len(rec.rows) != 2 {
|
||||||
|
t.Fatalf("rows = %+v", rec.rows)
|
||||||
|
}
|
||||||
|
want := store.InboundSpamEntry{
|
||||||
|
InboundDomain: "in.example.com", ClientIP: "203.0.113.7",
|
||||||
|
From: "two@example.net", To: "b@in.example.com",
|
||||||
|
Decision: store.SpamDecisionAccept, Engine: store.SpamEngineSelfPost,
|
||||||
|
}
|
||||||
|
if rec.rows[1] != want {
|
||||||
|
t.Errorf("second row = %+v\nwant %+v", rec.rows[1], want)
|
||||||
|
}
|
||||||
|
|
||||||
|
// An aborted transaction leaves nothing behind for the next one either.
|
||||||
|
if _, err := s.MailFrom("three@example.net", mods(nil)); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := s.RcptTo("c@in.example.com", mods(nil)); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := s.Abort(mods(nil)); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(rec.rows) != 2 || len(s.rcpts) != 0 {
|
||||||
|
t.Errorf("after an abort: rows %d, pending recipients %v", len(rec.rows), s.rcpts)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The journal failing is never the mail's problem.
|
||||||
|
func TestInboundAcceptsWhenTheStoreFails(t *testing.T) {
|
||||||
|
for name, rec := range map[string]*fakeInbound{
|
||||||
|
"insert": {domains: map[string]bool{"in.example.com": true}, insertErr: errors.New("boom")},
|
||||||
|
"lookup": {lookupErr: errors.New("boom")},
|
||||||
|
} {
|
||||||
|
s := inboundSessionFor(t, rec, false)
|
||||||
|
if resp := message(t, s, "sender@example.net", []string{"a@in.example.com"}, nil); resp != milter.RespContinue {
|
||||||
|
t.Errorf("%s failure: response = %v, want continue", name, resp)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// rspamd writes symbols heaviest first, so an allow entry's -100 comes last.
|
||||||
|
// Filter lists counts an entry's matches by finding that symbol in the journal:
|
||||||
|
// it is kept whatever else there is.
|
||||||
|
func TestRspamdVerdictKeepsTheListSymbols(t *testing.T) {
|
||||||
|
result := "default: False [-91.00 / 15.00]; SYM_A(3.00)[]; SYM_B(2.00)[]; SYM_C(1.50)[]; SYM_D(1.00)[]; SYM_E(0.80)[]; SYM_F(0.50)[]; SYM_G(0.20)[]; ARC_NA(0.00)[]; SELFPOST_ALLOW_FROM(-100.00)[billing@vendor.example]"
|
||||||
|
_, detail := rspamdVerdict("no action", result)
|
||||||
|
want := "no action, score -91.00 / 15.00; SELFPOST_ALLOW_FROM(-100.00) SYM_A(3.00) SYM_B(2.00) SYM_C(1.50) SYM_D(1.00) SYM_E(0.80)"
|
||||||
|
if detail != want {
|
||||||
|
t.Errorf("detail = %q\nwant %q", detail, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The client address of a connection's later messages comes from the MAIL
|
||||||
|
// macro: a session the library made after a non-continue reply saw no Connect.
|
||||||
|
func TestInboundClientAddressFromTheMailMacro(t *testing.T) {
|
||||||
|
rec := &fakeInbound{domains: map[string]bool{"in.example.com": true}}
|
||||||
|
s := &inboundSession{rec: rec} // no Connect
|
||||||
|
if _, err := s.MailFrom("sender@example.net", mods(map[string]string{"{client_addr}": "IPv6:2001:db8::7"})); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := s.RcptTo("a@in.example.com", mods(nil)); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := s.Body(mods(nil)); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(rec.rows) != 1 || rec.rows[0].ClientIP != "2001:db8::7" {
|
||||||
|
t.Errorf("rows = %+v", rec.rows)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// No callback of the inbound milter answers anything but "continue": it must
|
||||||
|
// never be the reason a message is held up, and a non-continue reply would
|
||||||
|
// cost it its session.
|
||||||
|
func TestInboundOnlyEverContinues(t *testing.T) {
|
||||||
|
rec := &fakeInbound{domains: map[string]bool{"in.example.com": true}}
|
||||||
|
s := &inboundSession{rec: rec, key: testKey}
|
||||||
|
check := func(stage string, resp milter.Response, err error) {
|
||||||
|
t.Helper()
|
||||||
|
if err != nil || resp != milter.RespContinue {
|
||||||
|
t.Errorf("%s: %v, %v", stage, resp, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
resp, err := s.Connect("mx", "tcp4", 0, net.ParseIP("203.0.113.7"), mods(nil))
|
||||||
|
check("connect", resp, err)
|
||||||
|
resp, err = s.MailFrom("a@example.net", mods(nil))
|
||||||
|
check("mail", resp, err)
|
||||||
|
resp, err = s.RcptTo("a@in.example.com", mods(nil))
|
||||||
|
check("rcpt", resp, err)
|
||||||
|
resp, err = s.Header("X-Rspamd-Action", "reject", mods(nil))
|
||||||
|
check("header", resp, err)
|
||||||
|
resp, err = s.Body(mods(nil))
|
||||||
|
check("end of message", resp, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// The filter is attached so that mail flows when it is down, and then the
|
||||||
|
// verdict headers of a message are its sender's own. A verdict is recorded only
|
||||||
|
// next to the filter's signature for this message's queue id.
|
||||||
|
func TestInboundReadsAVerdictOnlyWithTheFiltersSignature(t *testing.T) {
|
||||||
|
verdict := [][2]string{{"X-Rspamd-Action", "quarantine"}, {"X-Spamd-Result", spamdResult}}
|
||||||
|
for name, c := range map[string]struct {
|
||||||
|
extra [][2]string
|
||||||
|
signed bool
|
||||||
|
}{
|
||||||
|
"signed": {[][2]string{testSig}, true},
|
||||||
|
"no signature": {nil, false},
|
||||||
|
"a made-up signature": {[][2]string{{SignatureHeader, strings.Repeat("ab", 32)}}, false},
|
||||||
|
"another message's signature": {[][2]string{{SignatureHeader, FilterSignature(testKey, "OTHER1")}}, false},
|
||||||
|
"a signature under another key": {[][2]string{{SignatureHeader, FilterSignature(strings.Repeat("f", 64), "ABC123")}}, false},
|
||||||
|
"the sender's first, then the real": {[][2]string{{SignatureHeader, "forged"}, testSig}, true},
|
||||||
|
} {
|
||||||
|
rec := &fakeInbound{domains: map[string]bool{"in.example.com": true}}
|
||||||
|
s := inboundSessionFor(t, rec, true)
|
||||||
|
message(t, s, "sender@example.net", []string{"a@in.example.com"}, append(append([][2]string{}, verdict...), c.extra...))
|
||||||
|
if len(rec.rows) != 1 {
|
||||||
|
t.Fatalf("%s: rows = %+v", name, rec.rows)
|
||||||
|
}
|
||||||
|
got := rec.rows[0]
|
||||||
|
if c.signed && (got.Engine != store.SpamEngineRspamd || got.Decision != store.SpamDecisionQuarantine) {
|
||||||
|
t.Errorf("%s: a signed verdict was not read: %+v", name, got)
|
||||||
|
}
|
||||||
|
if !c.signed && (got.Engine != store.SpamEngineSelfPost || got.Decision != store.SpamDecisionAccept || got.Detail != "") {
|
||||||
|
t.Errorf("%s: an unsigned verdict was recorded: %+v", name, got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A signature is of no use to the next message on the connection.
|
||||||
|
rec := &fakeInbound{domains: map[string]bool{"in.example.com": true}}
|
||||||
|
s := inboundSessionFor(t, rec, true)
|
||||||
|
message(t, s, "one@example.net", []string{"a@in.example.com"}, append(append([][2]string{}, verdict...), testSig))
|
||||||
|
message(t, s, "two@example.net", []string{"a@in.example.com"}, verdict)
|
||||||
|
if len(rec.rows) != 2 || rec.rows[1].Engine != store.SpamEngineSelfPost {
|
||||||
|
t.Errorf("rows = %+v", rec.rows)
|
||||||
|
}
|
||||||
|
|
||||||
|
// With no queue id nothing can be checked, so nothing is trusted.
|
||||||
|
rec = &fakeInbound{domains: map[string]bool{"in.example.com": true}}
|
||||||
|
s = inboundSessionFor(t, rec, true)
|
||||||
|
_, _ = s.MailFrom("sender@example.net", mods(nil))
|
||||||
|
_, _ = s.RcptTo("a@in.example.com", mods(nil))
|
||||||
|
_, _ = s.Header("X-Rspamd-Action", "reject", mods(nil))
|
||||||
|
_, _ = s.Header(SignatureHeader, FilterSignature(testKey, ""), mods(nil))
|
||||||
|
_, _ = s.Body(mods(nil))
|
||||||
|
if len(rec.rows) != 1 || rec.rows[0].Engine != store.SpamEngineSelfPost {
|
||||||
|
t.Errorf("no queue id: rows = %+v", rec.rows)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// What rspamd 3.11 wrote for this key and queue id in a throwaway container
|
||||||
|
// (dev trial, 2026-10-12): the two ends must compute the same thing.
|
||||||
|
func TestFilterSignatureIsWhatTheFilterWrites(t *testing.T) {
|
||||||
|
if got := FilterSignature(testKey, "4A1B2C3D5E"); got != "2aa1ca6dd1545d3f16c8c0c29ba59bfc28c050224ac2c5c953bd2617f73b6de2" {
|
||||||
|
t.Errorf("FilterSignature = %s", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -53,7 +53,11 @@ type session struct {
|
|||||||
// valid zero value (no in-flight accounting).
|
// valid zero value (no in-flight accounting).
|
||||||
flight *inflight
|
flight *inflight
|
||||||
|
|
||||||
clientIP string // captured once per connection
|
// clientIP is the client's address. Postfix gives it at Connect, once per
|
||||||
|
// connection, and again with every MAIL FROM as the {client_addr} macro
|
||||||
|
// (milter_mail_macros in build/postfix-config.sh). The second is the one
|
||||||
|
// that can be relied on: see MailFrom.
|
||||||
|
clientIP string
|
||||||
|
|
||||||
login string
|
login string
|
||||||
from string
|
from string
|
||||||
@@ -62,9 +66,8 @@ type session struct {
|
|||||||
reserved []*reservation // level-2 slots held by the current message
|
reserved []*reservation // level-2 slots held by the current message
|
||||||
}
|
}
|
||||||
|
|
||||||
// Connect captures the client IP, which comes from the addr parameter rather
|
// Connect captures the client IP of a connection's first message. It is not
|
||||||
// than a macro (the {client_addr} macro was empty in testing). It is the
|
// enough on its own - see MailFrom.
|
||||||
// rate-limit key; here it is recorded for completeness.
|
|
||||||
func (s *session) Connect(host, family string, port uint16, addr net.IP, m *milter.Modifier) (milter.Response, error) {
|
func (s *session) Connect(host, family string, port uint16, addr net.IP, m *milter.Modifier) (milter.Response, error) {
|
||||||
if addr != nil {
|
if addr != nil {
|
||||||
s.clientIP = addr.String()
|
s.clientIP = addr.String()
|
||||||
@@ -79,8 +82,21 @@ func (s *session) Connect(host, family string, port uint16, addr net.IP, m *milt
|
|||||||
// limit is enforced here: over the limit, the message is refused with a 4xx
|
// limit is enforced here: over the limit, the message is refused with a 4xx
|
||||||
// tempfail before recipients are even offered (guide § Rate limiting).
|
// tempfail before recipients are even offered (guide § Rate limiting).
|
||||||
// Enforcement is fail-open — see overLimit.
|
// Enforcement is fail-open — see overLimit.
|
||||||
|
//
|
||||||
|
// The client address is read again here, from the {client_addr} macro. The
|
||||||
|
// milter library replaces the session with a new one after every reply that is
|
||||||
|
// not "continue" - a refusal, or the accept that used to end a message - and
|
||||||
|
// Postfix does not send Connect again on the same connection. A session made
|
||||||
|
// that way never saw the client, so every message of a connection after the
|
||||||
|
// first, and every MAIL FROM after a refused one, used to be judged with no
|
||||||
|
// client address: the level-2 limit and an application's client-address list
|
||||||
|
// both let it through. (Found on 2026-10-12; test/e2e sends two MAIL FROM on
|
||||||
|
// one connection.)
|
||||||
func (s *session) MailFrom(from string, m *milter.Modifier) (milter.Response, error) {
|
func (s *session) MailFrom(from string, m *milter.Modifier) (milter.Response, error) {
|
||||||
s.releaseReservations() // a previous transaction that ended without EOM/ABORT
|
s.releaseReservations() // a previous transaction that ended without EOM/ABORT
|
||||||
|
if ip := clientAddr(m); ip != "" {
|
||||||
|
s.clientIP = ip
|
||||||
|
}
|
||||||
s.from = cleanAddress(from)
|
s.from = cleanAddress(from)
|
||||||
s.login = macro(m, "auth_authen")
|
s.login = macro(m, "auth_authen")
|
||||||
s.rcpts = nil
|
s.rcpts = nil
|
||||||
@@ -113,13 +129,15 @@ func (s *session) Header(name, value string, m *milter.Modifier) (milter.Respons
|
|||||||
|
|
||||||
// Body fires at end-of-message, when the queue-id macro {i} is set and the
|
// Body fires at end-of-message, when the queue-id macro {i} is set and the
|
||||||
// message is about to be committed to the queue. This is where the "queued"
|
// message is about to be committed to the queue. This is where the "queued"
|
||||||
// rows are written. We accept (this milter is done) without ever rejecting.
|
// rows are written. The reply is "continue", which at end-of-message lets the
|
||||||
|
// message through just as "accept" does - and, unlike it, leaves this session
|
||||||
|
// in place for the connection's next message (see MailFrom).
|
||||||
func (s *session) Body(m *milter.Modifier) (milter.Response, error) {
|
func (s *session) Body(m *milter.Modifier) (milter.Response, error) {
|
||||||
s.record(macro(m, "i"))
|
s.record(macro(m, "i"))
|
||||||
// The rows are in the send log now, so the stored count sees this message
|
// The rows are in the send log now, so the stored count sees this message
|
||||||
// and its level-2 slots are no longer needed.
|
// and its level-2 slots are no longer needed.
|
||||||
s.releaseReservations()
|
s.releaseReservations()
|
||||||
return milter.RespAccept, nil
|
return milter.RespContinue, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// Abort ends the current transaction without an end-of-message (client RSET, or
|
// Abort ends the current transaction without an end-of-message (client RSET, or
|
||||||
@@ -143,6 +161,21 @@ func macro(m *milter.Modifier, name string) string {
|
|||||||
return m.Macros["{"+name+"}"]
|
return m.Macros["{"+name+"}"]
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// clientAddr is the client's address from the {client_addr} macro, or "" when
|
||||||
|
// Postfix did not send one that reads as an address. An IPv6 address may come
|
||||||
|
// in Sendmail's form, with "IPv6:" in front.
|
||||||
|
func clientAddr(m *milter.Modifier) string {
|
||||||
|
raw := strings.TrimSpace(macro(m, "client_addr"))
|
||||||
|
if len(raw) > 5 && strings.EqualFold(raw[:5], "IPv6:") {
|
||||||
|
raw = raw[5:]
|
||||||
|
}
|
||||||
|
ip := net.ParseIP(raw)
|
||||||
|
if ip == nil {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
return ip.String()
|
||||||
|
}
|
||||||
|
|
||||||
// record writes one send-log row per recipient. Failures are logged, never
|
// record writes one send-log row per recipient. Failures are logged, never
|
||||||
// propagated: journalling must not affect mail acceptance (architecture.md §
|
// propagated: journalling must not affect mail acceptance (architecture.md §
|
||||||
// Mail path).
|
// Mail path).
|
||||||
|
|||||||
@@ -166,8 +166,10 @@ func TestBodyAcceptsEvenWhenRecorderFails(t *testing.T) {
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("Body returned error, must fail open: %v", err)
|
t.Fatalf("Body returned error, must fail open: %v", err)
|
||||||
}
|
}
|
||||||
if resp != milter.RespAccept {
|
// "Continue" at end-of-message lets the message through, and keeps the
|
||||||
t.Fatalf("Body response = %v, want Accept", resp)
|
// session for the connection's next message.
|
||||||
|
if resp != milter.RespContinue {
|
||||||
|
t.Fatalf("Body response = %v, want Continue", resp)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -389,21 +391,77 @@ func TestRateLimitFailsOpenOnCountError(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestRateLimitNoIPKeyDoesNotApply(t *testing.T) {
|
// A ceiling is on a domain or an application, not on an address: a message that
|
||||||
|
// arrives with no client address is counted like any other. It used to be let
|
||||||
|
// through, which is how the second MAIL FROM of a connection got past a limit.
|
||||||
|
func TestRateLimitAppliesWithoutAClientIP(t *testing.T) {
|
||||||
rec := &fakeRecorder{
|
rec := &fakeRecorder{
|
||||||
limits: map[string]store.RateLimit{
|
limits: map[string]store.RateLimit{
|
||||||
store.RateLimitScopeDomain + "|example.com": domainLimit(),
|
store.RateLimitScopeDomain + "|example.com": domainLimit(),
|
||||||
},
|
},
|
||||||
counts: map[string]int64{store.RateLimitScopeDomain + "|example.com": 999},
|
counts: map[string]int64{store.RateLimitScopeDomain + "|example.com": 999},
|
||||||
}
|
}
|
||||||
// A session with no client IP (e.g. local submission) cannot be keyed.
|
|
||||||
s := &session{rec: rec}
|
s := &session{rec: rec}
|
||||||
resp, err := s.MailFrom("a@example.com", mods(map[string]string{"auth_authen": "app1"}))
|
resp, err := s.MailFrom("a@example.com", mods(map[string]string{"auth_authen": "app1"}))
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("MailFrom: %v", err)
|
t.Fatalf("MailFrom: %v", err)
|
||||||
}
|
}
|
||||||
if resp != milter.RespContinue {
|
if resp != milter.RespTempFail {
|
||||||
t.Fatalf("no-IP session = %v, want Continue", resp)
|
t.Fatalf("over the limit with no client IP = %v, want TempFail", resp)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The milter library hands the milter a fresh session after every reply that
|
||||||
|
// is not "continue", and Postfix sends Connect once per connection. Such a
|
||||||
|
// session knows the client only from the {client_addr} macro of MAIL FROM, and
|
||||||
|
// that has to be enough for an application's client-address list.
|
||||||
|
func TestAuthIPRestrictHoldsForASessionThatNeverSawConnect(t *testing.T) {
|
||||||
|
rec := &fakeRecorder{apps: map[string]store.Application{
|
||||||
|
"app1": {Login: "app1", AuthIPRestrict: true, AuthAllowedIPs: []string{"198.51.100.1"}},
|
||||||
|
}}
|
||||||
|
for name, c := range map[string]struct {
|
||||||
|
macros map[string]string
|
||||||
|
want milter.Response
|
||||||
|
}{
|
||||||
|
"an unlisted address in the macro": {map[string]string{"auth_authen": "app1", "{client_addr}": limitIP}, milter.RespTempFail},
|
||||||
|
"the listed address in the macro": {map[string]string{"auth_authen": "app1", "{client_addr}": "198.51.100.1"}, milter.RespContinue},
|
||||||
|
// No address at all is not a pass: a list an unknown address satisfies is no list.
|
||||||
|
"no client address": {map[string]string{"auth_authen": "app1"}, milter.RespTempFail},
|
||||||
|
"not an address": {map[string]string{"auth_authen": "app1", "{client_addr}": "unknown"}, milter.RespTempFail},
|
||||||
|
} {
|
||||||
|
s := &session{rec: rec} // no Connect
|
||||||
|
resp, err := s.MailFrom("a@example.com", mods(c.macros))
|
||||||
|
if err != nil || resp != c.want {
|
||||||
|
t.Errorf("%s: %v, %v; want %v", name, resp, err, c.want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// An application with no list is not held up by a missing address.
|
||||||
|
open := &fakeRecorder{apps: map[string]store.Application{"app1": {Login: "app1"}}}
|
||||||
|
s := &session{rec: open}
|
||||||
|
if resp, _ := s.MailFrom("a@example.com", mods(map[string]string{"auth_authen": "app1"})); resp != milter.RespContinue {
|
||||||
|
t.Errorf("no list, no address: %v, want Continue", resp)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The macro wins over what Connect said, and reads both spellings of IPv6.
|
||||||
|
func TestClientAddrFromTheMailMacro(t *testing.T) {
|
||||||
|
for raw, want := range map[string]string{
|
||||||
|
"203.0.113.7": "203.0.113.7", "2001:db8::1": "2001:db8::1", "IPv6:2001:DB8::1": "2001:db8::1",
|
||||||
|
"ipv6:::1": "::1", "": "", "unknown": "", "IPv6:": "",
|
||||||
|
} {
|
||||||
|
if got := clientAddr(mods(map[string]string{"{client_addr}": raw})); got != want {
|
||||||
|
t.Errorf("clientAddr(%q) = %q, want %q", raw, got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
s := &session{rec: &fakeRecorder{}}
|
||||||
|
_, _ = s.Connect("h", "tcp4", 0, net.ParseIP("192.0.2.1"), mods(nil))
|
||||||
|
_, _ = s.MailFrom("a@example.com", mods(map[string]string{"client_addr": "192.0.2.2"}))
|
||||||
|
if s.clientIP != "192.0.2.2" {
|
||||||
|
t.Errorf("clientIP = %q, want the macro's", s.clientIP)
|
||||||
|
}
|
||||||
|
_, _ = s.MailFrom("a@example.com", mods(nil))
|
||||||
|
if s.clientIP != "192.0.2.2" {
|
||||||
|
t.Errorf("clientIP = %q after a MAIL FROM with no macro, want it kept", s.clientIP)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -26,10 +26,9 @@ import (
|
|||||||
// reaches the send log (or is abandoned) — see inflight for why the stored
|
// reaches the send log (or is abandoned) — see inflight for why the stored
|
||||||
// count alone is not enough.
|
// count alone is not enough.
|
||||||
func (s *session) overLimit() bool {
|
func (s *session) overLimit() bool {
|
||||||
if s.clientIP == "" {
|
// A ceiling is on a domain or an application, whatever address the mail
|
||||||
return false // no client IP to key on; level-2 does not apply
|
// comes from, so the client address is not asked for here. It used to be,
|
||||||
}
|
// and a message that arrived without one was let through uncounted.
|
||||||
|
|
||||||
if s.login != "" {
|
if s.login != "" {
|
||||||
rl, ok, err := s.rec.RateLimit(store.RateLimitScopeApp, s.login)
|
rl, ok, err := s.rec.RateLimit(store.RateLimitScopeApp, s.login)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
|||||||
@@ -131,14 +131,18 @@ func (s *Store) InsertDMARCReport(rep DMARCReport) (int64, error) {
|
|||||||
return id, nil
|
return id, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// PruneDMARCReports enforces count and age caps.
|
// PruneDMARCReports enforces the age cap for every domain and the count cap
|
||||||
func (s *Store) PruneDMARCReports() error {
|
// for domain. The count is kept per domain on purpose: reports are accepted
|
||||||
|
// from anyone who knows a domain's report address, and with one cap for the
|
||||||
|
// whole instance a stranger filing reports under one domain would push every
|
||||||
|
// other domain's reports out.
|
||||||
|
func (s *Store) PruneDMARCReports(domain string) error {
|
||||||
cutoff := time.Now().UTC().AddDate(0, 0, -DMARCReportsMaxAgeDays).Format(time.RFC3339)
|
cutoff := time.Now().UTC().AddDate(0, 0, -DMARCReportsMaxAgeDays).Format(time.RFC3339)
|
||||||
if _, err := s.db.Exec(`DELETE FROM dmarc_reports WHERE received_at < ?`, cutoff); err != nil {
|
if _, err := s.db.Exec(`DELETE FROM dmarc_reports WHERE received_at < ?`, cutoff); err != nil {
|
||||||
return fmt.Errorf("prune dmarc by age: %w", err)
|
return fmt.Errorf("prune dmarc by age: %w", err)
|
||||||
}
|
}
|
||||||
var count int
|
var count int
|
||||||
if err := s.db.QueryRow(`SELECT COUNT(*) FROM dmarc_reports`).Scan(&count); err != nil {
|
if err := s.db.QueryRow(`SELECT COUNT(*) FROM dmarc_reports WHERE domain = ?`, domain).Scan(&count); err != nil {
|
||||||
return fmt.Errorf("count dmarc reports: %w", err)
|
return fmt.Errorf("count dmarc reports: %w", err)
|
||||||
}
|
}
|
||||||
if count <= DMARCReportsMaxKeep {
|
if count <= DMARCReportsMaxKeep {
|
||||||
@@ -147,8 +151,8 @@ func (s *Store) PruneDMARCReports() error {
|
|||||||
excess := count - DMARCReportsMaxKeep
|
excess := count - DMARCReportsMaxKeep
|
||||||
_, err := s.db.Exec(`
|
_, err := s.db.Exec(`
|
||||||
DELETE FROM dmarc_reports WHERE id IN (
|
DELETE FROM dmarc_reports WHERE id IN (
|
||||||
SELECT id FROM dmarc_reports ORDER BY received_at ASC LIMIT ?
|
SELECT id FROM dmarc_reports WHERE domain = ? ORDER BY received_at ASC LIMIT ?
|
||||||
)`, excess)
|
)`, domain, excess)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("prune dmarc by count: %w", err)
|
return fmt.Errorf("prune dmarc by count: %w", err)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -58,7 +58,7 @@ func TestPruneDMARCReports(t *testing.T) {
|
|||||||
t.Fatal(err)
|
t.Fatal(err)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if err := st.PruneDMARCReports(); err != nil {
|
if err := st.PruneDMARCReports("example.com"); err != nil {
|
||||||
t.Fatal(err)
|
t.Fatal(err)
|
||||||
}
|
}
|
||||||
list, _ := st.ListDMARCReports(DMARCReportScope{AllDomains: true}, 100)
|
list, _ := st.ListDMARCReports(DMARCReportScope{AllDomains: true}, 100)
|
||||||
|
|||||||
@@ -0,0 +1,379 @@
|
|||||||
|
package store
|
||||||
|
|
||||||
|
import (
|
||||||
|
"database/sql"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Decisions of the inbound antispam journal. Kept in sync with the CHECK
|
||||||
|
// constraint of inbound_spam_log.
|
||||||
|
const (
|
||||||
|
SpamDecisionAccept = "accept"
|
||||||
|
SpamDecisionReject = "reject"
|
||||||
|
SpamDecisionTempfail = "tempfail"
|
||||||
|
SpamDecisionQuarantine = "quarantine"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Who made a decision of the inbound antispam journal. Kept in sync with the
|
||||||
|
// CHECK constraint of inbound_spam_log.
|
||||||
|
const (
|
||||||
|
SpamEngineRspamd = "rspamd" // the external filter
|
||||||
|
SpamEnginePostfix = "postfix" // Postfix's own policy: an unknown recipient, a sender domain that does not exist
|
||||||
|
SpamEngineSelfPost = "selfpost" // nobody: relayed with no filter attached
|
||||||
|
)
|
||||||
|
|
||||||
|
// InboundSpamEntry is one row of the inbound antispam journal: what happened
|
||||||
|
// to a message offered on port 25 for one recipient. Metadata only - the
|
||||||
|
// message itself is relayed, not stored. Subject and Detail may be empty.
|
||||||
|
type InboundSpamEntry struct {
|
||||||
|
InboundDomain string
|
||||||
|
ClientIP string
|
||||||
|
From string
|
||||||
|
To string
|
||||||
|
Subject string
|
||||||
|
Decision string
|
||||||
|
Engine string
|
||||||
|
Detail string
|
||||||
|
// At is when it happened; zero means now. The log tailer dates a refusal
|
||||||
|
// by its mail.log line, which it may read long after.
|
||||||
|
At time.Time
|
||||||
|
}
|
||||||
|
|
||||||
|
// InsertInboundSpam records one journal row. It is called from the inbound
|
||||||
|
// journal milter's hot path and from the log tailer, so it returns any error
|
||||||
|
// for the caller to log; neither of them may hold up mail over it.
|
||||||
|
func (s *Store) InsertInboundSpam(e InboundSpamEntry) error {
|
||||||
|
at := e.At
|
||||||
|
if at.IsZero() {
|
||||||
|
at = time.Now()
|
||||||
|
}
|
||||||
|
_, err := s.db.Exec(
|
||||||
|
`INSERT INTO inbound_spam_log
|
||||||
|
(inbound_domain, client_ip, from_addr, to_addr, subject, decision, engine, detail, created_at)
|
||||||
|
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)`,
|
||||||
|
e.InboundDomain, e.ClientIP, e.From, e.To, e.Subject, e.Decision, e.Engine, e.Detail,
|
||||||
|
at.UTC().Format(time.RFC3339),
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("insert inbound_spam_log: %w", err)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// InsertInboundSpamOnce records a journal row unless the same row is there
|
||||||
|
// already - same moment, client, sender, recipient, decision, engine and
|
||||||
|
// detail - and says whether it wrote. The log tailer uses it for the lines it
|
||||||
|
// reads when it starts: after an unclean stop those are lines it may have read
|
||||||
|
// before, and a refusal read twice is otherwise two rows. e.At must be set; a
|
||||||
|
// row dated "now" is never the same as another.
|
||||||
|
func (s *Store) InsertInboundSpamOnce(e InboundSpamEntry) (bool, error) {
|
||||||
|
at := e.At.UTC().Format(time.RFC3339)
|
||||||
|
res, err := s.db.Exec(
|
||||||
|
`INSERT INTO inbound_spam_log
|
||||||
|
(inbound_domain, client_ip, from_addr, to_addr, subject, decision, engine, detail, created_at)
|
||||||
|
SELECT ?, ?, ?, ?, ?, ?, ?, ?, ?
|
||||||
|
WHERE NOT EXISTS (
|
||||||
|
SELECT 1 FROM inbound_spam_log
|
||||||
|
WHERE created_at = ? AND client_ip = ? AND from_addr = ? AND to_addr = ?
|
||||||
|
AND decision = ? AND engine = ? AND detail = ?)`,
|
||||||
|
e.InboundDomain, e.ClientIP, e.From, e.To, e.Subject, e.Decision, e.Engine, e.Detail, at,
|
||||||
|
at, e.ClientIP, e.From, e.To, e.Decision, e.Engine, e.Detail,
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
return false, fmt.Errorf("insert inbound_spam_log once: %w", err)
|
||||||
|
}
|
||||||
|
n, _ := res.RowsAffected()
|
||||||
|
return n > 0, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// PruneInboundSpam deletes journal rows older than cutoff and then, if more
|
||||||
|
// than keep rows are left, the oldest of those. It returns how many went.
|
||||||
|
func (s *Store) PruneInboundSpam(cutoff time.Time, keep int) (int64, error) {
|
||||||
|
res, err := s.db.Exec("DELETE FROM inbound_spam_log WHERE created_at < ?", cutoff.UTC().Format(time.RFC3339))
|
||||||
|
if err != nil {
|
||||||
|
return 0, fmt.Errorf("prune inbound_spam_log by age: %w", err)
|
||||||
|
}
|
||||||
|
n, _ := res.RowsAffected()
|
||||||
|
// Oldest by when it happened, not by id: the tailer dates a refusal by its
|
||||||
|
// log line, so after the panel was down the newest ids are old rows.
|
||||||
|
res, err = s.db.Exec(
|
||||||
|
`DELETE FROM inbound_spam_log WHERE id IN
|
||||||
|
(SELECT id FROM inbound_spam_log ORDER BY created_at DESC, id DESC LIMIT -1 OFFSET ?)`, keep)
|
||||||
|
if err != nil {
|
||||||
|
return n, fmt.Errorf("prune inbound_spam_log by count: %w", err)
|
||||||
|
}
|
||||||
|
m, _ := res.RowsAffected()
|
||||||
|
return n + m, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// InboundSpamRow is one row of the inbound antispam journal as it is read
|
||||||
|
// back for the panel: the entry as it was written, with its id and when it
|
||||||
|
// happened.
|
||||||
|
type InboundSpamRow struct {
|
||||||
|
ID int64
|
||||||
|
InboundDomain string
|
||||||
|
ClientIP string
|
||||||
|
From string
|
||||||
|
To string
|
||||||
|
Subject string
|
||||||
|
Decision string
|
||||||
|
Engine string
|
||||||
|
Detail string
|
||||||
|
CreatedAt time.Time
|
||||||
|
}
|
||||||
|
|
||||||
|
// InboundSpamFilter narrows QueryInboundSpam and CountInboundSpam. Like
|
||||||
|
// SendLogFilter it carries two kinds of narrowing that behave in opposite ways
|
||||||
|
// on purpose.
|
||||||
|
//
|
||||||
|
// Domain, Decision, Since and Until are the operator's own filters, chosen in
|
||||||
|
// the UI: an empty field matches everything. Since and Until are days, both
|
||||||
|
// inclusive and read as UTC days whatever the time of day they carry.
|
||||||
|
//
|
||||||
|
// Domains is the authorization scope, which no query parameter may widen: the
|
||||||
|
// exhaustive set of inbound domain names the caller may read, applied as an IN
|
||||||
|
// constraint. nil lifts it (the global role reads the whole journal, including
|
||||||
|
// rows of a domain deleted since); an empty set is a caller who reaches
|
||||||
|
// nothing and gets nothing. Unlike SendLogFilter, forgetting the scope is
|
||||||
|
// therefore the permissive case - the caller states "all" by leaving it out -
|
||||||
|
// so every caller that serves someone other than the global role must set it.
|
||||||
|
type InboundSpamFilter struct {
|
||||||
|
Domain string
|
||||||
|
Decision string
|
||||||
|
Since time.Time
|
||||||
|
Until time.Time
|
||||||
|
Domains []string
|
||||||
|
}
|
||||||
|
|
||||||
|
// QueryInboundSpam returns journal rows matching filter, newest first, for
|
||||||
|
// the Inbound log's server-side pagination. A refusal is dated by the mail.log
|
||||||
|
// line the tailer read it from, which can be older than rows written since, so
|
||||||
|
// the order is by time and not by id.
|
||||||
|
func (s *Store) QueryInboundSpam(filter InboundSpamFilter, limit, offset int) ([]InboundSpamRow, error) {
|
||||||
|
where, args := inboundSpamWhere(filter)
|
||||||
|
args = append(args, limit, offset)
|
||||||
|
rows, err := s.db.Query(
|
||||||
|
`SELECT id, inbound_domain, client_ip, from_addr, to_addr, subject, decision, engine, detail, created_at
|
||||||
|
FROM inbound_spam_log`+where+`
|
||||||
|
ORDER BY created_at DESC, id DESC
|
||||||
|
LIMIT ? OFFSET ?`,
|
||||||
|
args...,
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("query inbound_spam_log: %w", err)
|
||||||
|
}
|
||||||
|
defer rows.Close()
|
||||||
|
|
||||||
|
var out []InboundSpamRow
|
||||||
|
for rows.Next() {
|
||||||
|
var (
|
||||||
|
row InboundSpamRow
|
||||||
|
created string
|
||||||
|
)
|
||||||
|
if err := rows.Scan(&row.ID, &row.InboundDomain, &row.ClientIP, &row.From, &row.To,
|
||||||
|
&row.Subject, &row.Decision, &row.Engine, &row.Detail, &created); err != nil {
|
||||||
|
return nil, fmt.Errorf("scan inbound_spam_log row: %w", err)
|
||||||
|
}
|
||||||
|
row.CreatedAt, _ = time.Parse(time.RFC3339, created)
|
||||||
|
out = append(out, row)
|
||||||
|
}
|
||||||
|
return out, rows.Err()
|
||||||
|
}
|
||||||
|
|
||||||
|
// CountInboundSpam returns how many journal rows match filter, for the page
|
||||||
|
// numbers under the Inbound log.
|
||||||
|
func (s *Store) CountInboundSpam(filter InboundSpamFilter) (int64, error) {
|
||||||
|
where, args := inboundSpamWhere(filter)
|
||||||
|
var n int64
|
||||||
|
if err := s.db.QueryRow(`SELECT COUNT(*) FROM inbound_spam_log`+where, args...).Scan(&n); err != nil {
|
||||||
|
return 0, fmt.Errorf("count inbound_spam_log: %w", err)
|
||||||
|
}
|
||||||
|
return n, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func inboundSpamWhere(f InboundSpamFilter) (string, []any) {
|
||||||
|
var clauses []string
|
||||||
|
var args []any
|
||||||
|
if f.Domains != nil {
|
||||||
|
if len(f.Domains) == 0 {
|
||||||
|
return " WHERE 1 = 0", nil
|
||||||
|
}
|
||||||
|
marks := make([]string, len(f.Domains))
|
||||||
|
for i, name := range f.Domains {
|
||||||
|
marks[i] = "?"
|
||||||
|
args = append(args, name)
|
||||||
|
}
|
||||||
|
clauses = append(clauses, "inbound_domain IN ("+strings.Join(marks, ", ")+")")
|
||||||
|
}
|
||||||
|
if f.Domain != "" {
|
||||||
|
clauses = append(clauses, "inbound_domain = ?")
|
||||||
|
args = append(args, f.Domain)
|
||||||
|
}
|
||||||
|
if f.Decision != "" {
|
||||||
|
clauses = append(clauses, "decision = ?")
|
||||||
|
args = append(args, f.Decision)
|
||||||
|
}
|
||||||
|
// created_at is RFC3339 UTC, so comparing it as text against the same
|
||||||
|
// format is comparing moments: from midnight of the first day up to, not
|
||||||
|
// including, midnight after the last.
|
||||||
|
if !f.Since.IsZero() {
|
||||||
|
y, m, d := f.Since.UTC().Date()
|
||||||
|
clauses = append(clauses, "created_at >= ?")
|
||||||
|
args = append(args, time.Date(y, m, d, 0, 0, 0, 0, time.UTC).Format(time.RFC3339))
|
||||||
|
}
|
||||||
|
if !f.Until.IsZero() {
|
||||||
|
y, m, d := f.Until.UTC().Date()
|
||||||
|
clauses = append(clauses, "created_at < ?")
|
||||||
|
args = append(args, time.Date(y, m, d+1, 0, 0, 0, 0, time.UTC).Format(time.RFC3339))
|
||||||
|
}
|
||||||
|
if len(clauses) == 0 {
|
||||||
|
return "", nil
|
||||||
|
}
|
||||||
|
return " WHERE " + strings.Join(clauses, " AND "), args
|
||||||
|
}
|
||||||
|
|
||||||
|
// IsInboundDomain reports whether name is a configured inbound domain. Port 25
|
||||||
|
// also takes mail that is not the relay's - DMARC reports for the server's own
|
||||||
|
// name - and the journal keeps rows for inbound domains only.
|
||||||
|
func (s *Store) IsInboundDomain(name string) (bool, error) {
|
||||||
|
var one int
|
||||||
|
err := s.db.QueryRow("SELECT 1 FROM inbound_domains WHERE name = ?", strings.ToLower(name)).Scan(&one)
|
||||||
|
if errors.Is(err, sql.ErrNoRows) {
|
||||||
|
return false, nil
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
return false, fmt.Errorf("look up inbound domain: %w", err)
|
||||||
|
}
|
||||||
|
return true, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Lists and entry kinds of the filter lists. Kept in sync with the CHECK
|
||||||
|
// constraints of inbound_spam_list.
|
||||||
|
const (
|
||||||
|
SpamListAllow = "allow"
|
||||||
|
SpamListDeny = "deny"
|
||||||
|
|
||||||
|
SpamEntryAddress = "address" // sender address
|
||||||
|
SpamEntryDomain = "domain" // sender domain
|
||||||
|
SpamEntryIP = "ip" // client address or network
|
||||||
|
)
|
||||||
|
|
||||||
|
// ErrSpamListEntryExists is returned when the entry is already on that list.
|
||||||
|
var ErrSpamListEntryExists = errors.New("filter list entry already exists")
|
||||||
|
|
||||||
|
// ErrSpamListEntryNotFound is returned when a filter list entry id does not exist.
|
||||||
|
var ErrSpamListEntryNotFound = errors.New("filter list entry not found")
|
||||||
|
|
||||||
|
// SpamListEntry is one allow or deny entry for the external filter. Value is
|
||||||
|
// stored normalised by the caller (internal/inbound), which is what makes the
|
||||||
|
// uniqueness of an entry on its list mean anything.
|
||||||
|
type SpamListEntry struct {
|
||||||
|
ID int64
|
||||||
|
List string
|
||||||
|
Kind string
|
||||||
|
Value string
|
||||||
|
Note string
|
||||||
|
CreatedAt time.Time
|
||||||
|
}
|
||||||
|
|
||||||
|
// AddSpamListEntry inserts an entry. The caller must have validated it.
|
||||||
|
func (s *Store) AddSpamListEntry(list, kind, value, note string) (SpamListEntry, error) {
|
||||||
|
now := time.Now().UTC()
|
||||||
|
res, err := s.db.Exec(
|
||||||
|
"INSERT INTO inbound_spam_list (list_type, entry_type, value, note, created_at) VALUES (?, ?, ?, ?, ?)",
|
||||||
|
list, kind, value, note, now.Format(time.RFC3339),
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
if isUniqueViolation(err) {
|
||||||
|
return SpamListEntry{}, ErrSpamListEntryExists
|
||||||
|
}
|
||||||
|
return SpamListEntry{}, fmt.Errorf("insert inbound_spam_list: %w", err)
|
||||||
|
}
|
||||||
|
id, err := res.LastInsertId()
|
||||||
|
if err != nil {
|
||||||
|
return SpamListEntry{}, fmt.Errorf("filter list entry id: %w", err)
|
||||||
|
}
|
||||||
|
return SpamListEntry{ID: id, List: list, Kind: kind, Value: value, Note: note, CreatedAt: now.Truncate(time.Second)}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// ListSpamListEntries returns every entry of both lists: by list, then kind,
|
||||||
|
// then value.
|
||||||
|
func (s *Store) ListSpamListEntries() ([]SpamListEntry, error) {
|
||||||
|
rows, err := s.db.Query(
|
||||||
|
"SELECT id, list_type, entry_type, value, note, created_at FROM inbound_spam_list ORDER BY list_type, entry_type, value")
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("list inbound_spam_list: %w", err)
|
||||||
|
}
|
||||||
|
defer rows.Close()
|
||||||
|
var out []SpamListEntry
|
||||||
|
for rows.Next() {
|
||||||
|
var e SpamListEntry
|
||||||
|
var created string
|
||||||
|
if err := rows.Scan(&e.ID, &e.List, &e.Kind, &e.Value, &e.Note, &created); err != nil {
|
||||||
|
return nil, fmt.Errorf("scan inbound_spam_list: %w", err)
|
||||||
|
}
|
||||||
|
e.CreatedAt, _ = time.Parse(time.RFC3339, created)
|
||||||
|
out = append(out, e)
|
||||||
|
}
|
||||||
|
return out, rows.Err()
|
||||||
|
}
|
||||||
|
|
||||||
|
// DeleteSpamListEntry removes an entry by id.
|
||||||
|
func (s *Store) DeleteSpamListEntry(id int64) error {
|
||||||
|
res, err := s.db.Exec("DELETE FROM inbound_spam_list WHERE id = ?", id)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("delete inbound_spam_list: %w", err)
|
||||||
|
}
|
||||||
|
if n, _ := res.RowsAffected(); n == 0 {
|
||||||
|
return ErrSpamListEntryNotFound
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// SpamListHit is what the journal holds about one message a filter list
|
||||||
|
// decided: who it was from, which server offered it, when, and the detail that
|
||||||
|
// says which list it was.
|
||||||
|
type SpamListHit struct {
|
||||||
|
ClientIP string
|
||||||
|
From string
|
||||||
|
Decision string
|
||||||
|
Detail string
|
||||||
|
CreatedAt time.Time
|
||||||
|
}
|
||||||
|
|
||||||
|
// ListSpamListHits returns the journal rows of the filter in whose detail one
|
||||||
|
// of the marks occurs - the marks the filter lists leave (internal/inbound).
|
||||||
|
// Which entry a row belongs to is for the caller to work out; here the rows
|
||||||
|
// are only narrowed, so that a page view does not read the whole journal.
|
||||||
|
func (s *Store) ListSpamListHits(marks ...string) ([]SpamListHit, error) {
|
||||||
|
if len(marks) == 0 {
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
likes := make([]string, len(marks))
|
||||||
|
args := []any{SpamEngineRspamd}
|
||||||
|
for i, m := range marks {
|
||||||
|
likes[i] = "instr(detail, ?) > 0"
|
||||||
|
args = append(args, m)
|
||||||
|
}
|
||||||
|
rows, err := s.db.Query(
|
||||||
|
`SELECT client_ip, from_addr, decision, detail, created_at FROM inbound_spam_log
|
||||||
|
WHERE engine = ? AND (`+strings.Join(likes, " OR ")+`)`, args...)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("query filter list hits: %w", err)
|
||||||
|
}
|
||||||
|
defer rows.Close()
|
||||||
|
var out []SpamListHit
|
||||||
|
for rows.Next() {
|
||||||
|
var h SpamListHit
|
||||||
|
var created string
|
||||||
|
if err := rows.Scan(&h.ClientIP, &h.From, &h.Decision, &h.Detail, &created); err != nil {
|
||||||
|
return nil, fmt.Errorf("scan filter list hit: %w", err)
|
||||||
|
}
|
||||||
|
h.CreatedAt, _ = time.Parse(time.RFC3339, created)
|
||||||
|
out = append(out, h)
|
||||||
|
}
|
||||||
|
return out, rows.Err()
|
||||||
|
}
|
||||||
@@ -0,0 +1,338 @@
|
|||||||
|
package store
|
||||||
|
|
||||||
|
import (
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestInsertInboundSpam(t *testing.T) {
|
||||||
|
st := openTestStore(t)
|
||||||
|
e := InboundSpamEntry{
|
||||||
|
InboundDomain: "in.example.com", ClientIP: "203.0.113.7",
|
||||||
|
From: "sender@example.net", To: "a@in.example.com", Subject: "Hello",
|
||||||
|
Decision: SpamDecisionAccept, Engine: SpamEngineRspamd, Detail: "no action, score 0.10 / 15.00",
|
||||||
|
}
|
||||||
|
if err := st.InsertInboundSpam(e); err != nil {
|
||||||
|
t.Fatalf("InsertInboundSpam: %v", err)
|
||||||
|
}
|
||||||
|
var got InboundSpamEntry
|
||||||
|
var created string
|
||||||
|
err := st.db.QueryRow(
|
||||||
|
"SELECT inbound_domain, client_ip, from_addr, to_addr, subject, decision, engine, detail, created_at FROM inbound_spam_log",
|
||||||
|
).Scan(&got.InboundDomain, &got.ClientIP, &got.From, &got.To, &got.Subject, &got.Decision, &got.Engine, &got.Detail, &created)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("read back: %v", err)
|
||||||
|
}
|
||||||
|
if got != e {
|
||||||
|
t.Errorf("row = %+v\nwant %+v", got, e)
|
||||||
|
}
|
||||||
|
if at, err := time.Parse(time.RFC3339, created); err != nil || time.Since(at) > time.Minute {
|
||||||
|
t.Errorf("created_at = %q (%v)", created, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Every decision and engine the code names is one the table takes.
|
||||||
|
for _, d := range []string{SpamDecisionAccept, SpamDecisionReject, SpamDecisionTempfail, SpamDecisionQuarantine} {
|
||||||
|
for _, eng := range []string{SpamEngineRspamd, SpamEnginePostfix, SpamEngineSelfPost} {
|
||||||
|
e.Decision, e.Engine = d, eng
|
||||||
|
if err := st.InsertInboundSpam(e); err != nil {
|
||||||
|
t.Errorf("decision %s by %s: %v", d, eng, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
e.Decision = "bounce"
|
||||||
|
if err := st.InsertInboundSpam(e); err == nil {
|
||||||
|
t.Error("an unknown decision was stored")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestIsInboundDomain(t *testing.T) {
|
||||||
|
st := openTestStore(t)
|
||||||
|
if _, err := st.AddInboundDomain("in.example.com"); err != nil {
|
||||||
|
t.Fatalf("AddInboundDomain: %v", err)
|
||||||
|
}
|
||||||
|
for name, want := range map[string]bool{
|
||||||
|
"in.example.com": true, "IN.Example.COM": true,
|
||||||
|
"example.com": false, "": false, "other.example.com": false,
|
||||||
|
} {
|
||||||
|
if got, err := st.IsInboundDomain(name); err != nil || got != want {
|
||||||
|
t.Errorf("IsInboundDomain(%q) = %v, %v; want %v", name, got, err, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInsertInboundSpamKeepsItsOwnTime(t *testing.T) {
|
||||||
|
st := openTestStore(t)
|
||||||
|
at := time.Date(2026, 10, 1, 9, 30, 0, 0, time.FixedZone("msk", 3*3600))
|
||||||
|
err := st.InsertInboundSpam(InboundSpamEntry{
|
||||||
|
InboundDomain: "in.example.com", ClientIP: "203.0.113.7", To: "a@in.example.com",
|
||||||
|
Decision: SpamDecisionReject, Engine: SpamEnginePostfix, At: at,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("InsertInboundSpam: %v", err)
|
||||||
|
}
|
||||||
|
var created string
|
||||||
|
if err := st.db.QueryRow("SELECT created_at FROM inbound_spam_log").Scan(&created); err != nil || created != "2026-10-01T06:30:00Z" {
|
||||||
|
t.Errorf("created_at = %q, %v", created, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPruneInboundSpam(t *testing.T) {
|
||||||
|
st := openTestStore(t)
|
||||||
|
now := time.Now().UTC()
|
||||||
|
add := func(to string, age time.Duration) {
|
||||||
|
t.Helper()
|
||||||
|
err := st.InsertInboundSpam(InboundSpamEntry{
|
||||||
|
InboundDomain: "in.example.com", ClientIP: "203.0.113.7", To: to,
|
||||||
|
Decision: SpamDecisionReject, Engine: SpamEnginePostfix, At: now.Add(-age),
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("InsertInboundSpam: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
left := func() string {
|
||||||
|
t.Helper()
|
||||||
|
rows, err := st.db.Query("SELECT to_addr FROM inbound_spam_log ORDER BY id")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer rows.Close()
|
||||||
|
var out string
|
||||||
|
for rows.Next() {
|
||||||
|
var to string
|
||||||
|
if err := rows.Scan(&to); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
out += to + " "
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
add("old", 100*24*time.Hour)
|
||||||
|
add("a", 3*time.Hour)
|
||||||
|
add("b", 2*time.Hour)
|
||||||
|
add("c", time.Hour)
|
||||||
|
|
||||||
|
// Age first: nothing else goes while the rest fits.
|
||||||
|
if n, err := st.PruneInboundSpam(now.AddDate(0, 0, -90), 10); err != nil || n != 1 {
|
||||||
|
t.Fatalf("prune by age = %d, %v", n, err)
|
||||||
|
}
|
||||||
|
if got := left(); got != "a b c " {
|
||||||
|
t.Fatalf("left = %q", got)
|
||||||
|
}
|
||||||
|
// Then the cap, oldest first.
|
||||||
|
if n, err := st.PruneInboundSpam(now.AddDate(0, 0, -90), 2); err != nil || n != 1 {
|
||||||
|
t.Fatalf("prune by count = %d, %v", n, err)
|
||||||
|
}
|
||||||
|
if got := left(); got != "b c " {
|
||||||
|
t.Fatalf("left = %q", got)
|
||||||
|
}
|
||||||
|
if n, err := st.PruneInboundSpam(now.AddDate(0, 0, -90), 2); err != nil || n != 0 {
|
||||||
|
t.Errorf("a second prune removed %d, %v", n, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInboundSpamLogRetentionDays(t *testing.T) {
|
||||||
|
st := openTestStore(t)
|
||||||
|
if days, err := st.GetInboundSpamLogRetentionDays(30); err != nil || days != 30 {
|
||||||
|
t.Errorf("unset, env 30: %d, %v", days, err)
|
||||||
|
}
|
||||||
|
if days, err := st.GetInboundSpamLogRetentionDays(3); err != nil || days != InboundSpamLogRetentionDaysDefault {
|
||||||
|
t.Errorf("unset, env out of range: %d, %v", days, err)
|
||||||
|
}
|
||||||
|
if err := st.EnsureInboundSpamLogRetentionDays(30); err != nil {
|
||||||
|
t.Fatalf("Ensure: %v", err)
|
||||||
|
}
|
||||||
|
if err := st.EnsureInboundSpamLogRetentionDays(60); err != nil {
|
||||||
|
t.Fatalf("Ensure again: %v", err)
|
||||||
|
}
|
||||||
|
if days, _ := st.GetInboundSpamLogRetentionDays(90); days != 30 {
|
||||||
|
t.Errorf("after Ensure(30), Ensure(60): %d, want the first seed kept", days)
|
||||||
|
}
|
||||||
|
if err := st.SetInboundSpamLogRetentionDays(6); err == nil {
|
||||||
|
t.Error("6 days was accepted")
|
||||||
|
}
|
||||||
|
if err := st.SetInboundSpamLogRetentionDays(365); err != nil {
|
||||||
|
t.Errorf("365 days: %v", err)
|
||||||
|
}
|
||||||
|
if days, _ := st.GetInboundSpamLogRetentionDays(90); days != 365 {
|
||||||
|
t.Errorf("after Set(365): %d", days)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestQueryInboundSpamFilters(t *testing.T) {
|
||||||
|
st := openTestStore(t)
|
||||||
|
day := func(d, h int) time.Time { return time.Date(2026, 9, d, h, 0, 0, 0, time.UTC) }
|
||||||
|
add := func(domain, to, decision string, at time.Time) {
|
||||||
|
t.Helper()
|
||||||
|
err := st.InsertInboundSpam(InboundSpamEntry{
|
||||||
|
InboundDomain: domain, ClientIP: "203.0.113.7", From: "s@example.net", To: to,
|
||||||
|
Subject: "S " + to, Decision: decision, Engine: SpamEngineRspamd, Detail: "d", At: at,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("InsertInboundSpam: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
add("a.example", "1@a.example", SpamDecisionAccept, day(20, 23))
|
||||||
|
add("a.example", "2@a.example", SpamDecisionReject, day(21, 0))
|
||||||
|
add("b.example", "3@b.example", SpamDecisionAccept, day(21, 12))
|
||||||
|
add("b.example", "4@b.example", SpamDecisionTempfail, day(22, 0))
|
||||||
|
// Written last but dated earliest: the tailer reads a refusal after the fact.
|
||||||
|
add("a.example", "5@a.example", SpamDecisionReject, day(19, 8))
|
||||||
|
|
||||||
|
to := func(f InboundSpamFilter) string {
|
||||||
|
t.Helper()
|
||||||
|
rows, err := st.QueryInboundSpam(f, 50, 0)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("QueryInboundSpam(%+v): %v", f, err)
|
||||||
|
}
|
||||||
|
n, err := st.CountInboundSpam(f)
|
||||||
|
if err != nil || n != int64(len(rows)) {
|
||||||
|
t.Fatalf("CountInboundSpam(%+v) = %d, %v; the query returned %d", f, n, err, len(rows))
|
||||||
|
}
|
||||||
|
var out string
|
||||||
|
for _, r := range rows {
|
||||||
|
out += r.To[:1]
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
// Newest first by time, not by the order of writing.
|
||||||
|
if got := to(InboundSpamFilter{}); got != "43215" {
|
||||||
|
t.Errorf("no filter = %q, want 43215", got)
|
||||||
|
}
|
||||||
|
if got := to(InboundSpamFilter{Domain: "a.example"}); got != "215" {
|
||||||
|
t.Errorf("domain = %q, want 215", got)
|
||||||
|
}
|
||||||
|
if got := to(InboundSpamFilter{Decision: SpamDecisionReject}); got != "25" {
|
||||||
|
t.Errorf("decision = %q, want 25", got)
|
||||||
|
}
|
||||||
|
// Days are whole UTC days and both ends are in: 21 st is 00:00 up to 23:59.
|
||||||
|
if got := to(InboundSpamFilter{Since: day(21, 15), Until: day(21, 3)}); got != "32" {
|
||||||
|
t.Errorf("one day = %q, want 32", got)
|
||||||
|
}
|
||||||
|
if got := to(InboundSpamFilter{Since: day(22, 0)}); got != "4" {
|
||||||
|
t.Errorf("since = %q, want 4", got)
|
||||||
|
}
|
||||||
|
if got := to(InboundSpamFilter{Until: day(20, 0)}); got != "15" {
|
||||||
|
t.Errorf("until = %q, want 15", got)
|
||||||
|
}
|
||||||
|
// A time in another zone is read as the UTC day it falls on.
|
||||||
|
msk := time.FixedZone("msk", 3*3600)
|
||||||
|
if got := to(InboundSpamFilter{Since: time.Date(2026, 9, 22, 2, 0, 0, 0, msk)}); got != "432" {
|
||||||
|
t.Errorf("since in another zone = %q, want 432 (it is the 21st in UTC)", got)
|
||||||
|
}
|
||||||
|
if got := to(InboundSpamFilter{Domain: "b.example", Decision: SpamDecisionAccept, Since: day(21, 0), Until: day(21, 0)}); got != "3" {
|
||||||
|
t.Errorf("all filters = %q, want 3", got)
|
||||||
|
}
|
||||||
|
|
||||||
|
// The scope: nil is everything, a set is its domains, an empty set nothing,
|
||||||
|
// and a chosen domain outside the set is nothing too.
|
||||||
|
if got := to(InboundSpamFilter{Domains: []string{"b.example"}}); got != "43" {
|
||||||
|
t.Errorf("scope = %q, want 43", got)
|
||||||
|
}
|
||||||
|
if got := to(InboundSpamFilter{Domains: []string{}}); got != "" {
|
||||||
|
t.Errorf("empty scope = %q, want nothing", got)
|
||||||
|
}
|
||||||
|
if got := to(InboundSpamFilter{Domains: []string{"b.example"}, Domain: "a.example"}); got != "" {
|
||||||
|
t.Errorf("a chosen domain outside the scope = %q, want nothing", got)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Paging.
|
||||||
|
rows, err := st.QueryInboundSpam(InboundSpamFilter{}, 2, 2)
|
||||||
|
if err != nil || len(rows) != 2 || rows[0].To != "2@a.example" || rows[1].To != "1@a.example" {
|
||||||
|
t.Errorf("page 2 of 2 rows = %+v, %v", rows, err)
|
||||||
|
}
|
||||||
|
if rows[0].ID == 0 || !rows[0].CreatedAt.Equal(day(21, 0)) || rows[0].Subject != "S 2@a.example" ||
|
||||||
|
rows[0].Engine != SpamEngineRspamd || rows[0].Detail != "d" || rows[0].ClientIP != "203.0.113.7" || rows[0].From != "s@example.net" {
|
||||||
|
t.Errorf("a row read back = %+v", rows[0])
|
||||||
|
}
|
||||||
|
|
||||||
|
// A value is data, not SQL.
|
||||||
|
if got := to(InboundSpamFilter{Domain: "a.example' OR '1'='1"}); got != "" {
|
||||||
|
t.Errorf("injected domain = %q", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestListSpamListHits(t *testing.T) {
|
||||||
|
st := openTestStore(t)
|
||||||
|
add := func(engine, decision, detail string) {
|
||||||
|
t.Helper()
|
||||||
|
err := st.InsertInboundSpam(InboundSpamEntry{
|
||||||
|
InboundDomain: "in.example.com", ClientIP: "203.0.113.7", From: "a@example.net", To: "a@in.example.com",
|
||||||
|
Decision: decision, Engine: engine, Detail: detail, At: time.Date(2026, 10, 1, 9, 0, 0, 0, time.UTC),
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
add(SpamEngineRspamd, SpamDecisionReject, "5.7.1 Refused by this server's deny list")
|
||||||
|
add(SpamEngineRspamd, SpamDecisionAccept, "no action, score -99.00 / 15.00; SELFPOST_ALLOW_IP(-100.00)")
|
||||||
|
add(SpamEngineRspamd, SpamDecisionReject, "5.7.1 Spam message rejected")
|
||||||
|
add(SpamEnginePostfix, SpamDecisionReject, "550 5.1.1 this server's deny list")
|
||||||
|
// a mark with the characters LIKE would treat as wildcards is text
|
||||||
|
add(SpamEngineRspamd, SpamDecisionAccept, "no action; SELFPOSTXALLOWX(1.00)")
|
||||||
|
|
||||||
|
hits, err := st.ListSpamListHits("this server's deny list", "SELFPOST_ALLOW_")
|
||||||
|
if err != nil || len(hits) != 2 {
|
||||||
|
t.Fatalf("hits = %+v, %v; want the two rows a list decided", hits, err)
|
||||||
|
}
|
||||||
|
h := hits[0]
|
||||||
|
if h.From != "a@example.net" || h.ClientIP != "203.0.113.7" || h.Decision != SpamDecisionReject || h.CreatedAt.Hour() != 9 {
|
||||||
|
t.Errorf("hit = %+v", h)
|
||||||
|
}
|
||||||
|
if hits, err := st.ListSpamListHits(); err != nil || hits != nil {
|
||||||
|
t.Errorf("no marks: %v, %v", hits, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The cap removes the oldest by when it happened. After the panel was down the
|
||||||
|
// tailer writes old refusals with the newest ids, and they are not what stays.
|
||||||
|
func TestPruneInboundSpamCapIsByTime(t *testing.T) {
|
||||||
|
st := openTestStore(t)
|
||||||
|
now := time.Now().UTC()
|
||||||
|
for _, r := range []struct {
|
||||||
|
to string
|
||||||
|
age time.Duration
|
||||||
|
}{{"new-1", time.Minute}, {"new-2", 2 * time.Minute}, {"backdated-old", 5 * time.Hour}} {
|
||||||
|
err := st.InsertInboundSpam(InboundSpamEntry{
|
||||||
|
InboundDomain: "in.example.com", ClientIP: "203.0.113.7", To: r.to,
|
||||||
|
Decision: SpamDecisionReject, Engine: SpamEnginePostfix, At: now.Add(-r.age),
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if n, err := st.PruneInboundSpam(time.Time{}, 2); err != nil || n != 1 {
|
||||||
|
t.Fatalf("prune = %d, %v", n, err)
|
||||||
|
}
|
||||||
|
rows, err := st.QueryInboundSpam(InboundSpamFilter{}, 10, 0)
|
||||||
|
if err != nil || len(rows) != 2 || rows[0].To != "new-1" || rows[1].To != "new-2" {
|
||||||
|
t.Errorf("left = %+v, %v; want the two newest by time", rows, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInsertInboundSpamOnce(t *testing.T) {
|
||||||
|
st := openTestStore(t)
|
||||||
|
e := InboundSpamEntry{
|
||||||
|
InboundDomain: "in.example.com", ClientIP: "203.0.113.7", From: "a@example.net", To: "nobody@in.example.com",
|
||||||
|
Decision: SpamDecisionReject, Engine: SpamEnginePostfix, Detail: "550 5.1.1 no",
|
||||||
|
At: time.Date(2026, 10, 12, 9, 0, 5, 0, time.UTC),
|
||||||
|
}
|
||||||
|
for i, want := range []bool{true, false, false} {
|
||||||
|
if wrote, err := st.InsertInboundSpamOnce(e); err != nil || wrote != want {
|
||||||
|
t.Fatalf("attempt %d: wrote %v, %v; want %v", i+1, wrote, err, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Anything that tells two refusals apart makes a second row.
|
||||||
|
later, other := e, e
|
||||||
|
later.At = e.At.Add(time.Second)
|
||||||
|
other.To = "somebody@in.example.com"
|
||||||
|
for name, v := range map[string]InboundSpamEntry{"a second later": later, "another recipient": other} {
|
||||||
|
if wrote, err := st.InsertInboundSpamOnce(v); err != nil || !wrote {
|
||||||
|
t.Errorf("%s: wrote %v, %v", name, wrote, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if n, err := st.CountInboundSpam(InboundSpamFilter{}); err != nil || n != 3 {
|
||||||
|
t.Errorf("rows = %d, %v; want 3", n, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,40 @@
|
|||||||
|
-- Inbound antispam journal and filter lists (inbound-antispam-panel, 2.1.0).
|
||||||
|
-- The first migration after the 2.0 baseline: DDL only, so a 2.0.x database
|
||||||
|
-- gains two empty tables and nothing in it changes.
|
||||||
|
|
||||||
|
-- What happened to a message offered on port 25, one row per recipient.
|
||||||
|
-- Metadata only - SelfPost relays inbound mail and stores none of it. Accepted
|
||||||
|
-- messages are written by the inbound journal milter, refused ones by the
|
||||||
|
-- mail.log tailer, which is why a row can lack a subject (refused before DATA)
|
||||||
|
-- and why engine says who decided: the external filter, Postfix's own policy,
|
||||||
|
-- or nobody (selfpost: relayed with no filter attached).
|
||||||
|
CREATE TABLE inbound_spam_log (
|
||||||
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
|
inbound_domain TEXT NOT NULL,
|
||||||
|
client_ip TEXT NOT NULL,
|
||||||
|
from_addr TEXT NOT NULL,
|
||||||
|
to_addr TEXT NOT NULL,
|
||||||
|
subject TEXT NOT NULL,
|
||||||
|
decision TEXT NOT NULL CHECK (decision IN ('accept', 'reject', 'tempfail', 'quarantine')),
|
||||||
|
engine TEXT NOT NULL CHECK (engine IN ('rspamd', 'postfix', 'selfpost')),
|
||||||
|
detail TEXT NOT NULL,
|
||||||
|
created_at TEXT NOT NULL
|
||||||
|
);
|
||||||
|
-- Retention prunes by age; the Log page lists newest first, for the whole
|
||||||
|
-- server or for one domain.
|
||||||
|
CREATE INDEX idx_inbound_spam_log_created_at ON inbound_spam_log (created_at);
|
||||||
|
CREATE INDEX idx_inbound_spam_log_domain_created_at ON inbound_spam_log (inbound_domain, created_at);
|
||||||
|
|
||||||
|
-- Allow / deny entries for the external filter, instance-wide. The panel
|
||||||
|
-- writes them out as map files under /data/antispam/; SelfPost itself does not
|
||||||
|
-- consult them. value is stored normalised, so the same entry cannot be added
|
||||||
|
-- twice to one list.
|
||||||
|
CREATE TABLE inbound_spam_list (
|
||||||
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
|
list_type TEXT NOT NULL CHECK (list_type IN ('allow', 'deny')),
|
||||||
|
entry_type TEXT NOT NULL CHECK (entry_type IN ('address', 'domain', 'ip')),
|
||||||
|
value TEXT NOT NULL,
|
||||||
|
note TEXT NOT NULL DEFAULT '',
|
||||||
|
created_at TEXT NOT NULL,
|
||||||
|
UNIQUE (list_type, entry_type, value)
|
||||||
|
);
|
||||||
@@ -55,6 +55,8 @@ func TestBaselineSchema(t *testing.T) {
|
|||||||
"user_inbound_domains": "user_id inbound_domain_id",
|
"user_inbound_domains": "user_id inbound_domain_id",
|
||||||
"dmarc_reports": "id domain reporter report_id period_begin period_end received_at contact_email policy_p policy_sp policy_pct policy_adkim policy_aspf pass_count fail_count recipient",
|
"dmarc_reports": "id domain reporter report_id period_begin period_end received_at contact_email policy_p policy_sp policy_pct policy_adkim policy_aspf pass_count fail_count recipient",
|
||||||
"dmarc_report_records": "id report_row_id source_ip count disposition spf_result dkim_result header_from",
|
"dmarc_report_records": "id report_row_id source_ip count disposition spf_result dkim_result header_from",
|
||||||
|
"inbound_spam_log": "id inbound_domain client_ip from_addr to_addr subject decision engine detail created_at",
|
||||||
|
"inbound_spam_list": "id list_type entry_type value note created_at",
|
||||||
}
|
}
|
||||||
rows, err := st.db.Query("SELECT name FROM sqlite_master WHERE type = 'table' AND name NOT LIKE 'sqlite_%' ORDER BY name")
|
rows, err := st.db.Query("SELECT name FROM sqlite_master WHERE type = 'table' AND name NOT LIKE 'sqlite_%' ORDER BY name")
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -99,6 +101,7 @@ func TestBaselineSchema(t *testing.T) {
|
|||||||
rows.Close()
|
rows.Close()
|
||||||
wantIdx := []string{
|
wantIdx := []string{
|
||||||
"idx_dmarc_report_records_report", "idx_dmarc_reports_domain", "idx_dmarc_reports_received",
|
"idx_dmarc_report_records_report", "idx_dmarc_reports_domain", "idx_dmarc_reports_received",
|
||||||
|
"idx_inbound_spam_log_created_at", "idx_inbound_spam_log_domain_created_at",
|
||||||
"idx_send_log_app_login_created_at", "idx_send_log_created_at", "idx_send_log_domain", "idx_send_log_queue_id",
|
"idx_send_log_app_login_created_at", "idx_send_log_created_at", "idx_send_log_domain", "idx_send_log_queue_id",
|
||||||
"idx_sessions_expires_at",
|
"idx_sessions_expires_at",
|
||||||
}
|
}
|
||||||
@@ -108,8 +111,8 @@ func TestBaselineSchema(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
var version, appID int64
|
var version, appID int64
|
||||||
if err := st.db.QueryRow("PRAGMA user_version").Scan(&version); err != nil || version != 1 {
|
if err := st.db.QueryRow("PRAGMA user_version").Scan(&version); err != nil || version != 2 {
|
||||||
t.Errorf("user_version = %d, %v; want 1", version, err)
|
t.Errorf("user_version = %d, %v; want 2", version, err)
|
||||||
}
|
}
|
||||||
if err := st.db.QueryRow("PRAGMA application_id").Scan(&appID); err != nil || appID != applicationID {
|
if err := st.db.QueryRow("PRAGMA application_id").Scan(&appID); err != nil || appID != applicationID {
|
||||||
t.Errorf("application_id = %#x, %v; want %#x", appID, err, applicationID)
|
t.Errorf("application_id = %#x, %v; want %#x", appID, err, applicationID)
|
||||||
@@ -156,6 +159,96 @@ func TestReopenKeepsTheDatabase(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// A 2.0.x data directory - the baseline and nothing after it - is brought to
|
||||||
|
// the head by opening it, with what it holds left alone.
|
||||||
|
func TestOpenMigratesA20Database(t *testing.T) {
|
||||||
|
path := filepath.Join(t.TempDir(), "test.db")
|
||||||
|
baseline, err := migrationsFS.ReadFile("migrations/0001_init.sql")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("read the baseline: %v", err)
|
||||||
|
}
|
||||||
|
db, err := sql.Open("sqlite", "file:"+path)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("create a 2.0 db: %v", err)
|
||||||
|
}
|
||||||
|
for _, stmt := range []string{
|
||||||
|
string(baseline),
|
||||||
|
"PRAGMA user_version = 1",
|
||||||
|
"INSERT INTO users (username, password_hash, role, created_at) VALUES ('admin', 'hash', 'global', '2026-10-11T00:00:00Z')",
|
||||||
|
} {
|
||||||
|
if _, err := db.Exec(stmt); err != nil {
|
||||||
|
t.Fatalf("prepare a 2.0 db: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
db.Close()
|
||||||
|
|
||||||
|
st, err := Open(path)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("open a 2.0 database: %v", err)
|
||||||
|
}
|
||||||
|
defer st.Close()
|
||||||
|
var version int
|
||||||
|
if err := st.db.QueryRow("PRAGMA user_version").Scan(&version); err != nil || version != 2 {
|
||||||
|
t.Errorf("user_version = %d, %v; want 2", version, err)
|
||||||
|
}
|
||||||
|
if ok, err := st.UserExists(); err != nil || !ok {
|
||||||
|
t.Errorf("the user of the 2.0 database: exists = %v, %v", ok, err)
|
||||||
|
}
|
||||||
|
for _, table := range []string{"inbound_spam_log", "inbound_spam_list"} {
|
||||||
|
if len(tableColumns(t, st, table)) == 0 {
|
||||||
|
t.Errorf("table %s was not created", table)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// What the antispam tables refuse: a decision, engine or list the panel does
|
||||||
|
// not know, and the same entry twice in one list.
|
||||||
|
func TestInboundSpamConstraints(t *testing.T) {
|
||||||
|
st := openTestStore(t)
|
||||||
|
logRow := func(decision, engine string) error {
|
||||||
|
_, err := st.db.Exec(
|
||||||
|
"INSERT INTO inbound_spam_log (inbound_domain, client_ip, from_addr, to_addr, subject, decision, engine, detail, created_at)"+
|
||||||
|
" VALUES ('in.example.com', '192.0.2.1', 'a@example.net', 'b@in.example.com', '', ?, ?, '', '2026-10-11T00:00:00Z')",
|
||||||
|
decision, engine)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := logRow("reject", "postfix"); err != nil {
|
||||||
|
t.Fatalf("a reject by postfix: %v", err)
|
||||||
|
}
|
||||||
|
if err := logRow("bounce", "postfix"); err == nil {
|
||||||
|
t.Error("decision bounce was accepted")
|
||||||
|
}
|
||||||
|
if err := logRow("accept", "spamassassin"); err == nil {
|
||||||
|
t.Error("engine spamassassin was accepted")
|
||||||
|
}
|
||||||
|
|
||||||
|
entry := func(list, kind, value string) error {
|
||||||
|
_, err := st.db.Exec(
|
||||||
|
"INSERT INTO inbound_spam_list (list_type, entry_type, value, created_at) VALUES (?, ?, ?, '2026-10-11T00:00:00Z')",
|
||||||
|
list, kind, value)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := entry("allow", "address", "billing@vendor.example"); err != nil {
|
||||||
|
t.Fatalf("an allow entry: %v", err)
|
||||||
|
}
|
||||||
|
if err := entry("allow", "address", "billing@vendor.example"); err == nil {
|
||||||
|
t.Error("the same entry was added to one list twice")
|
||||||
|
}
|
||||||
|
if err := entry("deny", "address", "billing@vendor.example"); err != nil {
|
||||||
|
t.Errorf("the same value on the other list: %v", err)
|
||||||
|
}
|
||||||
|
if err := entry("block", "address", "x@example.net"); err == nil {
|
||||||
|
t.Error("list type block was accepted")
|
||||||
|
}
|
||||||
|
if err := entry("deny", "subject", "pills"); err == nil {
|
||||||
|
t.Error("entry type subject was accepted")
|
||||||
|
}
|
||||||
|
var note string
|
||||||
|
if err := st.db.QueryRow("SELECT note FROM inbound_spam_list WHERE list_type = 'allow'").Scan(¬e); err != nil || note != "" {
|
||||||
|
t.Errorf("note of an entry added without one = %q, %v", note, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// A 1.x data directory must be refused, loudly. Its user_version (10 at the
|
// A 1.x data directory must be refused, loudly. Its user_version (10 at the
|
||||||
// last 1.x release) would otherwise read as "the baseline is already applied",
|
// last 1.x release) would otherwise read as "the baseline is already applied",
|
||||||
// and the panel would start on tables it does not know.
|
// and the panel would start on tables it does not know.
|
||||||
|
|||||||
@@ -77,6 +77,72 @@ func (s *Store) EnsureSendLogRetentionDays(envDefault int) error {
|
|||||||
return s.SetSendLogRetentionDays(sendLogRetentionFallback(envDefault))
|
return s.SetSendLogRetentionDays(sendLogRetentionFallback(envDefault))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
const (
|
||||||
|
// InboundSpamLogRetentionDaysKey is the settings table key for the
|
||||||
|
// inbound antispam journal's retention. Same range as the send log's.
|
||||||
|
InboundSpamLogRetentionDaysKey = "inbound_spam_log_retention_days"
|
||||||
|
InboundSpamLogRetentionDaysMin = 7
|
||||||
|
InboundSpamLogRetentionDaysMax = 365
|
||||||
|
// InboundSpamLogRetentionDaysDefault matches
|
||||||
|
// INBOUND_SPAM_LOG_RETENTION_DAYS when unset.
|
||||||
|
InboundSpamLogRetentionDaysDefault = 90
|
||||||
|
)
|
||||||
|
|
||||||
|
// ErrInboundSpamLogRetentionDaysOutOfRange is returned when the inbound
|
||||||
|
// journal's retention is outside the allowed panel range.
|
||||||
|
var ErrInboundSpamLogRetentionDaysOutOfRange = errors.New("inbound journal retention days out of range")
|
||||||
|
|
||||||
|
// ValidateInboundSpamLogRetentionDays checks the panel-allowed window.
|
||||||
|
func ValidateInboundSpamLogRetentionDays(days int) error {
|
||||||
|
if days < InboundSpamLogRetentionDaysMin || days > InboundSpamLogRetentionDaysMax {
|
||||||
|
return fmt.Errorf("%w: must be between %d and %d days", ErrInboundSpamLogRetentionDaysOutOfRange, InboundSpamLogRetentionDaysMin, InboundSpamLogRetentionDaysMax)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func inboundSpamLogRetentionFallback(envDefault int) int {
|
||||||
|
if ValidateInboundSpamLogRetentionDays(envDefault) == nil {
|
||||||
|
return envDefault
|
||||||
|
}
|
||||||
|
return InboundSpamLogRetentionDaysDefault
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetInboundSpamLogRetentionDays returns the effective retention window of
|
||||||
|
// the inbound journal. When the setting is missing or invalid, envDefault is
|
||||||
|
// used (INBOUND_SPAM_LOG_RETENTION_DAYS).
|
||||||
|
func (s *Store) GetInboundSpamLogRetentionDays(envDefault int) (int, error) {
|
||||||
|
raw, err := s.GetSetting(InboundSpamLogRetentionDaysKey)
|
||||||
|
if err != nil {
|
||||||
|
return 0, err
|
||||||
|
}
|
||||||
|
days, err := strconv.Atoi(raw)
|
||||||
|
if err != nil || ValidateInboundSpamLogRetentionDays(days) != nil {
|
||||||
|
return inboundSpamLogRetentionFallback(envDefault), nil
|
||||||
|
}
|
||||||
|
return days, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// SetInboundSpamLogRetentionDays persists the inbound journal's retention.
|
||||||
|
func (s *Store) SetInboundSpamLogRetentionDays(days int) error {
|
||||||
|
if err := ValidateInboundSpamLogRetentionDays(days); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return s.SetSetting(InboundSpamLogRetentionDaysKey, strconv.Itoa(days))
|
||||||
|
}
|
||||||
|
|
||||||
|
// EnsureInboundSpamLogRetentionDays seeds the setting from envDefault when it
|
||||||
|
// has never been written.
|
||||||
|
func (s *Store) EnsureInboundSpamLogRetentionDays(envDefault int) error {
|
||||||
|
raw, err := s.GetSetting(InboundSpamLogRetentionDaysKey)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if raw != "" {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return s.SetInboundSpamLogRetentionDays(inboundSpamLogRetentionFallback(envDefault))
|
||||||
|
}
|
||||||
|
|
||||||
// GetSetting returns a settings value or empty string when missing.
|
// GetSetting returns a settings value or empty string when missing.
|
||||||
func (s *Store) GetSetting(key string) (string, error) {
|
func (s *Store) GetSetting(key string) (string, error) {
|
||||||
var value string
|
var value string
|
||||||
|
|||||||
@@ -56,6 +56,9 @@ var routes = struct{ always, inbound, dmarc []string }{
|
|||||||
"POST /inbound/domains/{id}/upstream",
|
"POST /inbound/domains/{id}/upstream",
|
||||||
"POST /inbound/domains/{id}/recipients",
|
"POST /inbound/domains/{id}/recipients",
|
||||||
"GET /inbound/domains/{id}/delete", "POST /inbound/domains/{id}/delete",
|
"GET /inbound/domains/{id}/delete", "POST /inbound/domains/{id}/delete",
|
||||||
|
"GET /inbound/log",
|
||||||
|
"GET /inbound/filter-lists", "POST /inbound/filter-lists",
|
||||||
|
"POST /inbound/filter-lists/{id}/delete",
|
||||||
},
|
},
|
||||||
dmarc: []string{
|
dmarc: []string{
|
||||||
"GET /outbound/dmarc", "GET /outbound/dmarc/domains/{id}", "GET /outbound/dmarc/reports/{id}",
|
"GET /outbound/dmarc", "GET /outbound/dmarc/domains/{id}", "GET /outbound/dmarc/reports/{id}",
|
||||||
|
|||||||
@@ -50,6 +50,9 @@ type Config struct {
|
|||||||
// SendLogRetentionEnvDefault is SEND_LOG_RETENTION_DAYS at panel start; used
|
// SendLogRetentionEnvDefault is SEND_LOG_RETENTION_DAYS at panel start; used
|
||||||
// as bootstrap and fallback when the settings row is missing or invalid.
|
// as bootstrap and fallback when the settings row is missing or invalid.
|
||||||
SendLogRetentionEnvDefault int
|
SendLogRetentionEnvDefault int
|
||||||
|
// InboundSpamLogRetentionEnvDefault is INBOUND_SPAM_LOG_RETENTION_DAYS at
|
||||||
|
// panel start, the same bootstrap and fallback for the inbound journal.
|
||||||
|
InboundSpamLogRetentionEnvDefault int
|
||||||
}
|
}
|
||||||
|
|
||||||
// Handlers holds dependencies for authenticated panel routes.
|
// Handlers holds dependencies for authenticated panel routes.
|
||||||
@@ -58,6 +61,7 @@ type Handlers struct {
|
|||||||
domains *domain.Service
|
domains *domain.Service
|
||||||
apps *app.Service
|
apps *app.Service
|
||||||
inbound *inbound.Service
|
inbound *inbound.Service
|
||||||
|
lists *inbound.Lists
|
||||||
dmarc *dmarc.Service
|
dmarc *dmarc.Service
|
||||||
cfg Config
|
cfg Config
|
||||||
view *view.Engine
|
view *view.Engine
|
||||||
@@ -72,6 +76,7 @@ func New(
|
|||||||
domains *domain.Service,
|
domains *domain.Service,
|
||||||
apps *app.Service,
|
apps *app.Service,
|
||||||
inboundSvc *inbound.Service,
|
inboundSvc *inbound.Service,
|
||||||
|
lists *inbound.Lists,
|
||||||
dmarcSvc *dmarc.Service,
|
dmarcSvc *dmarc.Service,
|
||||||
cfg Config,
|
cfg Config,
|
||||||
v *view.Engine,
|
v *view.Engine,
|
||||||
@@ -84,6 +89,7 @@ func New(
|
|||||||
domains: domains,
|
domains: domains,
|
||||||
apps: apps,
|
apps: apps,
|
||||||
inbound: inboundSvc,
|
inbound: inboundSvc,
|
||||||
|
lists: lists,
|
||||||
dmarc: dmarcSvc,
|
dmarc: dmarcSvc,
|
||||||
cfg: cfg,
|
cfg: cfg,
|
||||||
view: v,
|
view: v,
|
||||||
|
|||||||
@@ -70,11 +70,30 @@ func (h *Handlers) HandleBackup(w http.ResponseWriter, r *http.Request) {
|
|||||||
h.renderBackupPageWith(w, r, http.StatusBadRequest, "", pwErr)
|
h.renderBackupPageWith(w, r, http.StatusBadRequest, "", pwErr)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
params := backup.Params{
|
||||||
|
DataDir: h.cfg.DataDir,
|
||||||
|
DBPath: h.cfg.DBPath,
|
||||||
|
Version: h.cfg.Version,
|
||||||
|
DeployRoot: h.cfg.DeployRoot,
|
||||||
|
OnWarn: func(msg string) {
|
||||||
|
logf("panel: full backup: %s", msg)
|
||||||
|
},
|
||||||
|
}
|
||||||
if err := backup.ValidateDeployRoot(h.cfg.DeployRoot); err != nil {
|
if err := backup.ValidateDeployRoot(h.cfg.DeployRoot); err != nil {
|
||||||
logf("panel: full backup: %v", err)
|
logf("panel: full backup: %v", err)
|
||||||
h.renderBackupPageWith(w, r, http.StatusBadRequest, "", deployBackupErr(err))
|
h.renderBackupPageWith(w, r, http.StatusBadRequest, "", deployBackupErr(err))
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
// Everything is read through once before the first byte goes out. Past the
|
||||||
|
// headers a failure can only cut the download short - which is what the
|
||||||
|
// button did from 1.3.0 to 2.0.0, with a 200 and ten bytes, because the
|
||||||
|
// queue was not the panel's to read. Nothing is skipped to make the archive
|
||||||
|
// possible: it is the one selfpost-backup makes, or it is refused.
|
||||||
|
if err := backup.Check(params); err != nil {
|
||||||
|
logf("panel: full backup: %v", err)
|
||||||
|
h.renderBackupPageWith(w, r, http.StatusInternalServerError, "", unreadableBackupErr(err, h.cfg.DeployRoot))
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
stamp := time.Now().UTC().Format("20060102-150405")
|
stamp := time.Now().UTC().Format("20060102-150405")
|
||||||
filename := fmt.Sprintf("selfpost-backup-%s.tar.gz", stamp)
|
filename := fmt.Sprintf("selfpost-backup-%s.tar.gz", stamp)
|
||||||
@@ -109,15 +128,7 @@ func (h *Handlers) HandleBackup(w http.ResponseWriter, r *http.Request) {
|
|||||||
sink = env
|
sink = env
|
||||||
}
|
}
|
||||||
|
|
||||||
if err := backup.Create(sink, backup.Params{
|
if err := backup.Create(sink, params); err != nil {
|
||||||
DataDir: h.cfg.DataDir,
|
|
||||||
DBPath: h.cfg.DBPath,
|
|
||||||
Version: h.cfg.Version,
|
|
||||||
DeployRoot: h.cfg.DeployRoot,
|
|
||||||
OnWarn: func(msg string) {
|
|
||||||
logf("panel: full backup: %s", msg)
|
|
||||||
},
|
|
||||||
}); err != nil {
|
|
||||||
logf("panel: full backup failed: %v", err)
|
logf("panel: full backup failed: %v", err)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
@@ -319,6 +330,18 @@ func decryptErrorMessage(err error) string {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// unreadableBackupErr phrases what backup.Check found. The usual case is the
|
||||||
|
// private key: the panel runs as its own user, and a key readable by root
|
||||||
|
// alone is not its to read. The way out is on the host, so the message says
|
||||||
|
// what to do there (guide § Taking a full backup).
|
||||||
|
func unreadableBackupErr(err error, deployRoot string) string {
|
||||||
|
if name, ok := backup.UnreadableCert(err, deployRoot); ok {
|
||||||
|
return "The panel is not allowed to read " + name + ", and a backup without it would not bring the server back. " +
|
||||||
|
"On the host, give the file to group 998 and make it group-readable (chgrp 998, chmod 640), then try again. Nothing was downloaded."
|
||||||
|
}
|
||||||
|
return "Could not read everything a backup holds. Nothing was downloaded; the reason is in the panel's log."
|
||||||
|
}
|
||||||
|
|
||||||
// deployBackupErr phrases a pre-flight backup failure for the operator.
|
// deployBackupErr phrases a pre-flight backup failure for the operator.
|
||||||
func deployBackupErr(err error) string {
|
func deployBackupErr(err error) string {
|
||||||
if strings.Contains(err.Error(), "DeployRoot") || strings.Contains(err.Error(), "deploy root") {
|
if strings.Contains(err.Error(), "DeployRoot") || strings.Contains(err.Error(), "deploy root") {
|
||||||
|
|||||||
@@ -49,7 +49,7 @@ func TestDMARCHubShowsIngestAndRecentReports(t *testing.T) {
|
|||||||
body := getBody(t, s.h.HandleDMARCList, "/outbound/dmarc")
|
body := getBody(t, s.h.HandleDMARCList, "/outbound/dmarc")
|
||||||
has(t, "hub", body,
|
has(t, "hub", body,
|
||||||
`<h1 class="title is-3">DMARC reports</h1>`, `<h2>Ingest</h2>`, `<span class="tag is-success is-light">ok</span>`,
|
`<h1 class="title is-3">DMARC reports</h1>`, `<h2>Ingest</h2>`, `<span class="tag is-success is-light">ok</span>`,
|
||||||
`1 kept · 0 parse failures`, `Hosted addresses`, `dmarc-reports+<domain>@mail.example.org`, `500 reports or 90 days`,
|
`1 kept · 0 parse failures`, `Hosted addresses`, `dmarc-reports+<domain>@mail.example.org`, `500 reports a domain or 90 days`,
|
||||||
`<h2>Recent reports</h2>`, `<a href="/outbound/dmarc/domains/1">example.org</a>`, `Outlook.com`,
|
`<h2>Recent reports</h2>`, `<a href="/outbound/dmarc/domains/1">example.org</a>`, `Outlook.com`,
|
||||||
"<td>1\u00a0903</td>", `<span class="tag is-warning is-light">7</span>`, `href="/outbound/dmarc/reports/`+idStr(s.reportID)+`">View</a>`)
|
"<td>1\u00a0903</td>", `<span class="tag is-warning is-light">7</span>`, `href="/outbound/dmarc/reports/`+idStr(s.reportID)+`">View</a>`)
|
||||||
lacks(t, "hub", body, `None yet`)
|
lacks(t, "hub", body, `None yet`)
|
||||||
|
|||||||
@@ -0,0 +1,247 @@
|
|||||||
|
package handlers
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"net/http"
|
||||||
|
"strconv"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/mixeme/selfpost/internal/inbound"
|
||||||
|
"github.com/mixeme/selfpost/internal/store"
|
||||||
|
"github.com/mixeme/selfpost/internal/web/view"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Inbound › Log and Inbound › Filter lists. The log is for everyone who reaches
|
||||||
|
// an inbound domain, with the rows of their domains only (requireInbound); the
|
||||||
|
// lists are instance-wide and so the global role's (requireInboundGlobal), and
|
||||||
|
// answer 404 to anyone else, like the rest of what only that role may do.
|
||||||
|
|
||||||
|
const inboundLogPageSize = 50
|
||||||
|
|
||||||
|
// logDay is the form of a day in the log's date filters and in <input
|
||||||
|
// type="date">.
|
||||||
|
const logDay = "2006-01-02"
|
||||||
|
|
||||||
|
// HandleInboundLog renders the Inbound log: the journal of port 25's decisions,
|
||||||
|
// filtered server-side by domain, decision and day, a page at a time. A filter
|
||||||
|
// value that is not one the page offers is dropped rather than refused, and the
|
||||||
|
// domains a domain administrator may read are decided here, before the query
|
||||||
|
// runs - the dropdown only offers them.
|
||||||
|
func (h *Handlers) HandleInboundLog(w http.ResponseWriter, r *http.Request) {
|
||||||
|
p, ok := h.requireInbound(w, r)
|
||||||
|
if !ok {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
assigned, err := h.assignedInboundDomains(p)
|
||||||
|
if err != nil {
|
||||||
|
logf("panel: inbound log: %v", err)
|
||||||
|
http.Error(w, "internal error", http.StatusInternalServerError)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
names := make([]string, 0, len(assigned))
|
||||||
|
offered := make(map[string]bool, len(assigned))
|
||||||
|
for _, d := range assigned {
|
||||||
|
names = append(names, d.Name)
|
||||||
|
offered[d.Name] = true
|
||||||
|
}
|
||||||
|
|
||||||
|
q := r.URL.Query()
|
||||||
|
form := view.InLogFilter{}
|
||||||
|
query := store.InboundSpamFilter{}
|
||||||
|
if d := q.Get("domain"); offered[d] {
|
||||||
|
form.Domain, query.Domain = d, d
|
||||||
|
}
|
||||||
|
switch d := q.Get("decision"); d {
|
||||||
|
case store.SpamDecisionAccept, store.SpamDecisionReject, store.SpamDecisionTempfail, store.SpamDecisionQuarantine:
|
||||||
|
form.Decision, query.Decision = d, d
|
||||||
|
}
|
||||||
|
if t, err := time.Parse(logDay, q.Get("from")); err == nil {
|
||||||
|
form.From, query.Since = t.Format(logDay), t
|
||||||
|
}
|
||||||
|
if t, err := time.Parse(logDay, q.Get("to")); err == nil {
|
||||||
|
form.To, query.Until = t.Format(logDay), t
|
||||||
|
}
|
||||||
|
// A global administrator reads the whole journal, including rows left
|
||||||
|
// behind by a domain that has since been deleted; anyone else only what is
|
||||||
|
// assigned to them, however empty that is.
|
||||||
|
if !p.IsGlobal() {
|
||||||
|
query.Domains = names
|
||||||
|
}
|
||||||
|
|
||||||
|
total, err := h.store.CountInboundSpam(query)
|
||||||
|
if err != nil {
|
||||||
|
logf("panel: inbound log: %v", err)
|
||||||
|
http.Error(w, "internal error", http.StatusInternalServerError)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
lastPage := 1
|
||||||
|
if total > 0 {
|
||||||
|
lastPage = int((total + inboundLogPageSize - 1) / inboundLogPageSize)
|
||||||
|
}
|
||||||
|
page := min(parsePage(q.Get("p")), lastPage)
|
||||||
|
rows, err := h.store.QueryInboundSpam(query, inboundLogPageSize, (page-1)*inboundLogPageSize)
|
||||||
|
if err != nil {
|
||||||
|
logf("panel: inbound log: %v", err)
|
||||||
|
http.Error(w, "internal error", http.StatusInternalServerError)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
out := view.NewInLog(h.shellMeta(r), h.inboundSpamLogRetentionDays(), names, form)
|
||||||
|
table := make([]view.InLogRow, len(rows))
|
||||||
|
for i, row := range rows {
|
||||||
|
table[i] = out.NewRow(view.InLogRowInput{
|
||||||
|
ID: row.ID, Listable: inbound.Listable(store.SpamEntryAddress, row.From), At: row.CreatedAt, From: row.From, Client: row.ClientIP, To: row.To, Subject: row.Subject,
|
||||||
|
Decision: row.Decision, Engine: row.Engine, Detail: row.Detail,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
h.view.Render(w, http.StatusOK, "in-log", out.WithRows(table, page, lastPage))
|
||||||
|
}
|
||||||
|
|
||||||
|
// inboundSpamLogRetentionDays returns the effective inbound journal retention
|
||||||
|
// window.
|
||||||
|
func (h *Handlers) inboundSpamLogRetentionDays() int {
|
||||||
|
days, err := h.store.GetInboundSpamLogRetentionDays(h.cfg.InboundSpamLogRetentionEnvDefault)
|
||||||
|
if err != nil {
|
||||||
|
logf("panel: inbound log retention: %v", err)
|
||||||
|
if err := store.ValidateInboundSpamLogRetentionDays(h.cfg.InboundSpamLogRetentionEnvDefault); err == nil {
|
||||||
|
return h.cfg.InboundSpamLogRetentionEnvDefault
|
||||||
|
}
|
||||||
|
return store.InboundSpamLogRetentionDaysDefault
|
||||||
|
}
|
||||||
|
return days
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- Filter lists
|
||||||
|
|
||||||
|
func (h *Handlers) requireFilterLists(w http.ResponseWriter, r *http.Request) bool {
|
||||||
|
if _, ok := h.requireInboundGlobal(w, r); !ok {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
if h.lists == nil {
|
||||||
|
http.NotFound(w, r)
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
// filterListForm is what the add form shows: the list and kind it offers (only
|
||||||
|
// values it knows are kept) and the text of the two fields.
|
||||||
|
type filterListForm struct {
|
||||||
|
list, kind, value, note string
|
||||||
|
}
|
||||||
|
|
||||||
|
// HandleFilterLists shows the lists. The query fills the add form in - the
|
||||||
|
// Inbound log's Allow / Deny sender links lead here - and only a list and a
|
||||||
|
// kind the form offers are taken; the value is just put in the field.
|
||||||
|
func (h *Handlers) HandleFilterLists(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if !h.requireFilterLists(w, r) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
q := r.URL.Query()
|
||||||
|
form := filterListForm{value: q.Get("value")}
|
||||||
|
switch l := q.Get("list"); l {
|
||||||
|
case store.SpamListAllow, store.SpamListDeny:
|
||||||
|
form.list = l
|
||||||
|
}
|
||||||
|
switch k := q.Get("kind"); k {
|
||||||
|
case store.SpamEntryAddress, store.SpamEntryDomain, store.SpamEntryIP:
|
||||||
|
form.kind = k
|
||||||
|
}
|
||||||
|
h.renderFilterLists(w, r, http.StatusOK, filterListsFlash(r), "", form)
|
||||||
|
}
|
||||||
|
|
||||||
|
func filterListsFlash(r *http.Request) string {
|
||||||
|
switch {
|
||||||
|
case r.URL.Query().Has("added"):
|
||||||
|
return "Entry added."
|
||||||
|
case r.URL.Query().Has("removed"):
|
||||||
|
return "Entry removed."
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|
||||||
|
func (h *Handlers) renderFilterLists(w http.ResponseWriter, r *http.Request, status int, flash, formErr string, form filterListForm) {
|
||||||
|
entries, err := h.lists.Entries()
|
||||||
|
if err != nil {
|
||||||
|
logf("panel: filter lists: %v", err)
|
||||||
|
http.Error(w, "internal error", http.StatusInternalServerError)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
// How often each entry decided a message is counted from the journal. A
|
||||||
|
// failure there is not a reason to hide the lists: they are shown without
|
||||||
|
// the counts' worth - every entry "never" - and the log says why.
|
||||||
|
usage, err := h.lists.Usage()
|
||||||
|
if err != nil {
|
||||||
|
logf("panel: filter lists: usage: %v", err)
|
||||||
|
}
|
||||||
|
on, _, _ := h.inboundFilter()
|
||||||
|
in := make([]view.FilterEntryInput, len(entries))
|
||||||
|
for i, e := range entries {
|
||||||
|
in[i] = view.FilterEntryInput{ID: e.ID, List: e.List, Kind: e.Kind, Value: e.Value, Note: e.Note, Added: e.CreatedAt,
|
||||||
|
Hits: usage[e.ID].Hits, LastHit: usage[e.ID].Last}
|
||||||
|
}
|
||||||
|
page := view.NewFilterLists(h.shellMeta(r), view.FilterListsInput{
|
||||||
|
FilterOn: on, LastWritten: h.lists.LastWritten(), RetentionDays: h.inboundSpamLogRetentionDays()}).
|
||||||
|
WithEntries(in).
|
||||||
|
WithResult(flash, formErr)
|
||||||
|
// A blank list or kind leaves the form on its first choice.
|
||||||
|
if form.list == "" {
|
||||||
|
form.list = store.SpamListDeny
|
||||||
|
}
|
||||||
|
if form.kind == "" {
|
||||||
|
form.kind = store.SpamEntryAddress
|
||||||
|
}
|
||||||
|
h.view.Render(w, status, "in-filter-lists", page.WithForm(form.list, form.kind, form.value, form.note))
|
||||||
|
}
|
||||||
|
|
||||||
|
// HandleFilterListAdd adds an entry and rewrites the map files. A refusal - an
|
||||||
|
// entry the lists do not take, or one already on the list - shows the page again
|
||||||
|
// with the message and what was typed.
|
||||||
|
func (h *Handlers) HandleFilterListAdd(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if !h.requireFilterLists(w, r) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err := r.ParseForm(); err != nil {
|
||||||
|
h.renderFilterLists(w, r, http.StatusBadRequest, "", "Invalid form submission.", filterListForm{})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
form := filterListForm{
|
||||||
|
list: r.PostFormValue("list_type"), kind: r.PostFormValue("entry_type"),
|
||||||
|
value: r.PostFormValue("value"), note: r.PostFormValue("note"),
|
||||||
|
}
|
||||||
|
_, err := h.lists.Add(form.list, form.kind, form.value, form.note)
|
||||||
|
var refused *inbound.ValidationError
|
||||||
|
switch {
|
||||||
|
case err == nil:
|
||||||
|
http.Redirect(w, r, "/inbound/filter-lists?added=1", http.StatusSeeOther)
|
||||||
|
case errors.Is(err, store.ErrSpamListEntryExists):
|
||||||
|
h.renderFilterLists(w, r, http.StatusUnprocessableEntity, "", "That entry is already on the list.", form)
|
||||||
|
case errors.As(err, &refused):
|
||||||
|
h.renderFilterLists(w, r, http.StatusUnprocessableEntity, "", refused.Error(), form)
|
||||||
|
default:
|
||||||
|
logf("panel: add filter list entry: %v", err)
|
||||||
|
h.renderFilterLists(w, r, http.StatusInternalServerError, "", "Could not add the entry. Check the logs and try again.", form)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// HandleFilterListDelete removes an entry and rewrites the map files.
|
||||||
|
func (h *Handlers) HandleFilterListDelete(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if !h.requireFilterLists(w, r) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
id, err := strconv.ParseInt(r.PathValue("id"), 10, 64)
|
||||||
|
if err != nil || id <= 0 {
|
||||||
|
http.NotFound(w, r)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err := h.lists.Delete(id); err != nil {
|
||||||
|
if errors.Is(err, store.ErrSpamListEntryNotFound) {
|
||||||
|
http.NotFound(w, r)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
logf("panel: delete filter list entry %d: %v", id, err)
|
||||||
|
http.Error(w, "internal error", http.StatusInternalServerError)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
http.Redirect(w, r, "/inbound/filter-lists?removed=1", http.StatusSeeOther)
|
||||||
|
}
|
||||||
@@ -0,0 +1,674 @@
|
|||||||
|
package handlers
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"net/url"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/mixeme/selfpost/internal/inbound"
|
||||||
|
"github.com/mixeme/selfpost/internal/store"
|
||||||
|
"github.com/mixeme/selfpost/internal/web/auth"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Inbound › Log and Inbound › Filter lists, rendered from the real journal and
|
||||||
|
// lists by the handlers. Their markup is held to the mockups by the guards in
|
||||||
|
// internal/web/view; what is checked here is what the handlers read and who may.
|
||||||
|
|
||||||
|
// inboundLogStand is an inbound relay with two domains, a journal row for each
|
||||||
|
// and the lists over a directory the test can read the map files from.
|
||||||
|
type inboundLogStand struct {
|
||||||
|
h *Handlers
|
||||||
|
st *store.Store
|
||||||
|
dir string
|
||||||
|
a, b store.InboundDomain
|
||||||
|
}
|
||||||
|
|
||||||
|
func newInboundLogStand(t *testing.T) inboundLogStand {
|
||||||
|
t.Helper()
|
||||||
|
h, st := inboundHandlers(t)
|
||||||
|
s := inboundLogStand{h: h, st: st, dir: filepath.Join(t.TempDir(), "antispam")}
|
||||||
|
h.lists = inbound.NewLists(st, s.dir)
|
||||||
|
var err error
|
||||||
|
if s.a, err = st.AddInboundDomain("a.example.com"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if s.b, err = st.AddInboundDomain("b.example.com"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
s.row("a.example.com", "alice@a.example.com", "Hello A", store.SpamDecisionAccept, store.SpamEngineRspamd)
|
||||||
|
s.row("b.example.com", "bob@b.example.com", "Hello B", store.SpamDecisionAccept, store.SpamEngineRspamd)
|
||||||
|
return s
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s inboundLogStand) row(domain, to, subject, decision, engine string) {
|
||||||
|
s.rowAt(domain, to, subject, decision, engine, time.Time{})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s inboundLogStand) rowAt(domain, to, subject, decision, engine string, at time.Time) {
|
||||||
|
err := s.st.InsertInboundSpam(store.InboundSpamEntry{
|
||||||
|
InboundDomain: domain, ClientIP: "203.0.113.9", From: "sender@example.net", To: to, Subject: subject,
|
||||||
|
Decision: decision, Engine: engine, Detail: "detail of " + subject, At: at,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
panic(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// get runs a handler over a GET as p.
|
||||||
|
func (s inboundLogStand) get(h http.HandlerFunc, target string, p auth.Principal) *httptest.ResponseRecorder {
|
||||||
|
rec := httptest.NewRecorder()
|
||||||
|
h(rec, auth.RequestWithPrincipal(httptest.NewRequest(http.MethodGet, target, nil), p))
|
||||||
|
return rec
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s inboundLogStand) owner(t *testing.T, username string, domains ...store.InboundDomain) auth.Principal {
|
||||||
|
t.Helper()
|
||||||
|
var ids []int64
|
||||||
|
for _, d := range domains {
|
||||||
|
ids = append(ids, d.ID)
|
||||||
|
}
|
||||||
|
return userWith(t, s.st, username, store.RoleDomain, store.Reach{InboundDomainIDs: ids})
|
||||||
|
}
|
||||||
|
|
||||||
|
// The global role reads the whole journal, is offered every domain, and is given
|
||||||
|
// the way to correct a decision and to change how long rows are kept.
|
||||||
|
func TestInboundLogForTheGlobalRole(t *testing.T) {
|
||||||
|
s := newInboundLogStand(t)
|
||||||
|
s.row("a.example.com", "carol@a.example.com", "Quarantined", store.SpamDecisionQuarantine, store.SpamEngineRspamd)
|
||||||
|
if err := s.st.SetInboundSpamLogRetentionDays(30); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
rec := s.get(s.h.HandleInboundLog, "/inbound/log", globalPrincipal)
|
||||||
|
if rec.Code != http.StatusOK {
|
||||||
|
t.Fatalf("GET /inbound/log = %d:\n%s", rec.Code, rec.Body.String())
|
||||||
|
}
|
||||||
|
body := rec.Body.String()
|
||||||
|
has(t, "log", body,
|
||||||
|
`<h1 class="title is-3">Inbound log</h1>`,
|
||||||
|
`Rows older than 30 days are deleted — change that in <a href="/server/settings">Settings</a>.`,
|
||||||
|
`alice@<wbr>a.example.com`, `bob@<wbr>b.example.com`, `Hello A`, `Hello B`,
|
||||||
|
`<option value="a.example.com">a.example.com</option>`, `<option value="b.example.com">b.example.com</option>`,
|
||||||
|
`class="tag is-success is-light" popovertarget="why-`, `class="tag is-primary is-light" popovertarget="why-`,
|
||||||
|
`<span class="sp-mono sp-muted sp-small">203.0.113.9</span>`,
|
||||||
|
`<tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Verdict</th><td>detail of Hello A</td></tr>`,
|
||||||
|
`<a href="/inbound/filter-lists?list=deny&kind=address&value=sender%40example.net">Deny sender</a>`,
|
||||||
|
`<a href="/inbound/filter-lists?list=allow&kind=address&value=sender%40example.net">Allow sender</a>`,
|
||||||
|
`Page 1 of 1`, `Allow / Deny sender opens <a href="/inbound/filter-lists">Filter lists</a>`,
|
||||||
|
)
|
||||||
|
// Newest first: the last row written is the first on the page.
|
||||||
|
if strings.Index(body, `Quarantined`) > strings.Index(body, `Hello B`) || strings.Index(body, `Hello B`) > strings.Index(body, `Hello A`) {
|
||||||
|
t.Error("the rows are not newest first")
|
||||||
|
}
|
||||||
|
lacks(t, "log", body, `data-poll`, `hx-get`)
|
||||||
|
}
|
||||||
|
|
||||||
|
// A domain administrator reads the rows of the inbound domains assigned to them,
|
||||||
|
// is offered those domains alone, and is not given the lists or the setting.
|
||||||
|
func TestInboundLogForADomainAdministrator(t *testing.T) {
|
||||||
|
s := newInboundLogStand(t)
|
||||||
|
p := s.owner(t, "owns-a", s.a)
|
||||||
|
body := s.get(s.h.HandleInboundLog, "/inbound/log", p).Body.String()
|
||||||
|
has(t, "log", body, `Hello A`, `<option value="a.example.com">a.example.com</option>`, `Rows older than 90 days are deleted.`)
|
||||||
|
lacks(t, "log", body, `Hello B`, `b.example.com`, `href="/server/settings"`, `/inbound/filter-lists`,
|
||||||
|
`Deny sender`, `Allow sender`, `Allow / Deny`)
|
||||||
|
|
||||||
|
// A domain filter outside their reach is dropped, not honoured.
|
||||||
|
body = s.get(s.h.HandleInboundLog, "/inbound/log?domain=b.example.com", p).Body.String()
|
||||||
|
has(t, "log", body, `Hello A`)
|
||||||
|
lacks(t, "log", body, `Hello B`, `b.example.com`)
|
||||||
|
|
||||||
|
// One with All sees every domain, one with none sees no log at all.
|
||||||
|
all := userWith(t, s.st, "owns-all", store.RoleDomain, store.Reach{AllInbound: true})
|
||||||
|
body = s.get(s.h.HandleInboundLog, "/inbound/log", all).Body.String()
|
||||||
|
has(t, "log", body, `Hello A`, `Hello B`)
|
||||||
|
lacks(t, "log", body, `Deny sender`)
|
||||||
|
|
||||||
|
outboundOnly := userWith(t, s.st, "outbound-only", store.RoleDomain, store.Reach{AllDomains: true})
|
||||||
|
if rec := s.get(s.h.HandleInboundLog, "/inbound/log", outboundOnly); rec.Code != http.StatusNotFound {
|
||||||
|
t.Errorf("an outbound-only administrator got %d, want 404", rec.Code)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A domain administrator whose assigned domain has been deleted since has nothing
|
||||||
|
// to read, not everything.
|
||||||
|
func TestInboundLogOfADeletedDomainIsEmptyForItsAdministrator(t *testing.T) {
|
||||||
|
s := newInboundLogStand(t)
|
||||||
|
p := s.owner(t, "owns-a", s.a)
|
||||||
|
both := s.owner(t, "owns-both", s.a, s.b)
|
||||||
|
if err := s.st.DeleteInboundDomain(s.a.ID); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
body := s.get(s.h.HandleInboundLog, "/inbound/log", p).Body.String()
|
||||||
|
has(t, "log", body, `No messages logged yet.`)
|
||||||
|
lacks(t, "log", body, `Hello A`, `Hello B`)
|
||||||
|
body = s.get(s.h.HandleInboundLog, "/inbound/log", both).Body.String()
|
||||||
|
has(t, "log", body, `Hello B`)
|
||||||
|
lacks(t, "log", body, `Hello A`)
|
||||||
|
// The global role still reads what the deleted domain left in the journal.
|
||||||
|
has(t, "log", s.get(s.h.HandleInboundLog, "/inbound/log", globalPrincipal).Body.String(), `Hello A`, `Hello B`)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInboundLogFilters(t *testing.T) {
|
||||||
|
s := newInboundLogStand(t)
|
||||||
|
day := func(d, h int) time.Time { return time.Date(2026, 9, d, h, 0, 0, 0, time.UTC) }
|
||||||
|
s.rowAt("a.example.com", "r1@a.example.com", "Rejected on the 10th", store.SpamDecisionReject, store.SpamEnginePostfix, day(10, 23))
|
||||||
|
s.rowAt("a.example.com", "r2@a.example.com", "Rejected on the 11th", store.SpamDecisionReject, store.SpamEnginePostfix, day(11, 0))
|
||||||
|
s.rowAt("b.example.com", "r3@b.example.com", "Tempfail on the 11th", store.SpamDecisionTempfail, store.SpamEngineRspamd, day(11, 23))
|
||||||
|
s.rowAt("a.example.com", "r4@a.example.com", "Accepted on the 12th", store.SpamDecisionAccept, store.SpamEngineRspamd, day(12, 0))
|
||||||
|
|
||||||
|
rows := func(query string) string {
|
||||||
|
t.Helper()
|
||||||
|
rec := s.get(s.h.HandleInboundLog, "/inbound/log"+query, globalPrincipal)
|
||||||
|
if rec.Code != http.StatusOK {
|
||||||
|
t.Fatalf("GET /inbound/log%s = %d", query, rec.Code)
|
||||||
|
}
|
||||||
|
var got []string
|
||||||
|
body := rec.Body.String()
|
||||||
|
for _, n := range []string{"10th", "11th", "12th"} {
|
||||||
|
for _, kind := range []string{"Rejected", "Tempfail", "Accepted"} {
|
||||||
|
if strings.Contains(body, kind+" on the "+n) {
|
||||||
|
got = append(got, kind[:1]+n[:2])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return strings.Join(got, " ")
|
||||||
|
}
|
||||||
|
for query, want := range map[string]string{
|
||||||
|
"?decision=reject": "R10 R11",
|
||||||
|
"?domain=b.example.com": "T11",
|
||||||
|
"?from=2026-09-11&to=2026-09-11": "R11 T11",
|
||||||
|
"?from=2026-09-12": "A12",
|
||||||
|
"?to=2026-09-10": "R10",
|
||||||
|
"?domain=a.example.com&decision=reject&from=2026-09-11": "R11",
|
||||||
|
// A value that is not one the page offers is ignored.
|
||||||
|
"?decision=bounce": "R10 R11 T11 A12",
|
||||||
|
"?from=yesterday&to=2026-13-45": "R10 R11 T11 A12",
|
||||||
|
"?domain=nobody.example.com": "R10 R11 T11 A12",
|
||||||
|
"?decision=reject%27+OR+%271%27%3D%271": "R10 R11 T11 A12",
|
||||||
|
"?domain=a.example.com&domain=b.example.com": "R10 R11 A12",
|
||||||
|
} {
|
||||||
|
if got := rows(query); got != want {
|
||||||
|
t.Errorf("GET /inbound/log%s shows %q, want %q", query, got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The choice made is the one shown, and the page leads on with it.
|
||||||
|
body := s.get(s.h.HandleInboundLog, "/inbound/log?domain=a.example.com&decision=reject&from=2026-09-11&to=2026-09-12", globalPrincipal).Body.String()
|
||||||
|
has(t, "log", body, `<option value="a.example.com" selected>`, `<option value="reject" selected>`,
|
||||||
|
`name="from" value="2026-09-11"`, `name="to" value="2026-09-12"`)
|
||||||
|
body = s.get(s.h.HandleInboundLog, "/inbound/log?decision=quarantine", globalPrincipal).Body.String()
|
||||||
|
has(t, "log", body, `No messages match these filters.`)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInboundLogPages(t *testing.T) {
|
||||||
|
s := newInboundLogStand(t)
|
||||||
|
for i := 0; i < inboundLogPageSize; i++ {
|
||||||
|
s.row("a.example.com", fmt.Sprintf("u%d@a.example.com", i), fmt.Sprintf("Message %02d", i), store.SpamDecisionAccept, store.SpamEngineRspamd)
|
||||||
|
}
|
||||||
|
// 51 rows with the one the stand wrote: 50 and 1.
|
||||||
|
first := s.get(s.h.HandleInboundLog, "/inbound/log?domain=a.example.com", globalPrincipal).Body.String()
|
||||||
|
has(t, "page 1", first, `Page 1 of 2`, `<a href="/inbound/log?domain=a.example.com&p=2">Older →</a>`, `Message 49`, `Message 00`)
|
||||||
|
lacks(t, "page 1", first, `← Newer`, `Hello A`)
|
||||||
|
if got := strings.Count(first, `<tr><td`); got != inboundLogPageSize {
|
||||||
|
t.Errorf("page 1 has %d rows, want %d", got, inboundLogPageSize)
|
||||||
|
}
|
||||||
|
|
||||||
|
second := s.get(s.h.HandleInboundLog, "/inbound/log?domain=a.example.com&p=2", globalPrincipal).Body.String()
|
||||||
|
has(t, "page 2", second, `Page 2 of 2`, `<a href="/inbound/log?domain=a.example.com">← Newer</a>`, `Hello A`)
|
||||||
|
lacks(t, "page 2", second, `Older →`, `Message 49`, `Message 00`)
|
||||||
|
if got := strings.Count(second, `<tr><td`); got != 1 {
|
||||||
|
t.Errorf("page 2 has %d rows, want 1", got)
|
||||||
|
}
|
||||||
|
|
||||||
|
// A page past the end, or nonsense, is the last page, or the first.
|
||||||
|
has(t, "page 9", s.get(s.h.HandleInboundLog, "/inbound/log?domain=a.example.com&p=9", globalPrincipal).Body.String(), `Page 2 of 2`, `Hello A`)
|
||||||
|
has(t, "page x", s.get(s.h.HandleInboundLog, "/inbound/log?domain=a.example.com&p=x", globalPrincipal).Body.String(), `Page 1 of 2`)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInboundLogEmptyAndOff(t *testing.T) {
|
||||||
|
h, _ := inboundHandlers(t)
|
||||||
|
s := inboundLogStand{h: h}
|
||||||
|
body := s.get(h.HandleInboundLog, "/inbound/log", globalPrincipal).Body.String()
|
||||||
|
has(t, "empty log", body, `No messages logged yet.`)
|
||||||
|
lacks(t, "empty log", body, `<table`, `Page 1`)
|
||||||
|
|
||||||
|
h.cfg.InboundEnabled = false
|
||||||
|
if rec := s.get(h.HandleInboundLog, "/inbound/log", globalPrincipal); rec.Code != http.StatusNotFound {
|
||||||
|
t.Errorf("with the inbound relay off GET /inbound/log = %d, want 404", rec.Code)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// What a sender or a server wrote is text on the page.
|
||||||
|
func TestInboundLogEscapesWhatSendersWrote(t *testing.T) {
|
||||||
|
s := newInboundLogStand(t)
|
||||||
|
err := s.st.InsertInboundSpam(store.InboundSpamEntry{
|
||||||
|
InboundDomain: "a.example.com", ClientIP: "203.0.113.9", From: `"><img src=x onerror=alert(1)>@e.example`,
|
||||||
|
To: "x@a.example.com", Subject: `<script>alert(3)</script>`, Decision: store.SpamDecisionReject,
|
||||||
|
Engine: store.SpamEnginePostfix, Detail: `550 <b>no</b>`,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
body := s.get(s.h.HandleInboundLog, "/inbound/log", globalPrincipal).Body.String()
|
||||||
|
lacks(t, "log", body, `<script>alert`, `<img src=x`, `<b>no</b>`)
|
||||||
|
has(t, "log", body, `<script>alert(3)</script>`)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- Filter lists
|
||||||
|
|
||||||
|
func (s inboundLogStand) post(h http.HandlerFunc, target string, form url.Values, p auth.Principal) *httptest.ResponseRecorder {
|
||||||
|
req := httptest.NewRequest(http.MethodPost, target, strings.NewReader(form.Encode()))
|
||||||
|
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
|
||||||
|
if rest, ok := strings.CutPrefix(target, "/inbound/filter-lists/"); ok {
|
||||||
|
id, _, _ := strings.Cut(rest, "/")
|
||||||
|
req.SetPathValue("id", id)
|
||||||
|
}
|
||||||
|
rec := httptest.NewRecorder()
|
||||||
|
h(rec, auth.RequestWithPrincipal(req, p))
|
||||||
|
return rec
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s inboundLogStand) mapFile(t *testing.T, name string) string {
|
||||||
|
t.Helper()
|
||||||
|
b, err := os.ReadFile(filepath.Join(s.dir, name))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("read %s: %v", name, err)
|
||||||
|
}
|
||||||
|
return string(b)
|
||||||
|
}
|
||||||
|
|
||||||
|
func entry(list, kind, value, note string) url.Values {
|
||||||
|
return url.Values{"list_type": {list}, "entry_type": {kind}, "value": {value}, "note": {note}}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The lists are the global role's: everything about them is 404 to a domain
|
||||||
|
// administrator and while the inbound relay is off, and nothing is written.
|
||||||
|
func TestFilterListsAreForTheGlobalRoleOnly(t *testing.T) {
|
||||||
|
s := newInboundLogStand(t)
|
||||||
|
e, err := s.h.lists.Add("deny", "domain", "bulk.example", "")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
ops := s.owner(t, "owns-a", s.a)
|
||||||
|
all := userWith(t, s.st, "owns-all", store.RoleDomain, store.Reach{AllInbound: true, AllDomains: true})
|
||||||
|
for _, p := range []auth.Principal{ops, all} {
|
||||||
|
if rec := s.get(s.h.HandleFilterLists, "/inbound/filter-lists", p); rec.Code != http.StatusNotFound {
|
||||||
|
t.Errorf("GET as %s = %d, want 404", p.Username, rec.Code)
|
||||||
|
}
|
||||||
|
if rec := s.post(s.h.HandleFilterListAdd, "/inbound/filter-lists", entry("deny", "domain", "evil.example", ""), p); rec.Code != http.StatusNotFound {
|
||||||
|
t.Errorf("POST as %s = %d, want 404", p.Username, rec.Code)
|
||||||
|
}
|
||||||
|
if rec := s.post(s.h.HandleFilterListDelete, "/inbound/filter-lists/"+itoa(e.ID)+"/delete", nil, p); rec.Code != http.StatusNotFound {
|
||||||
|
t.Errorf("POST delete as %s = %d, want 404", p.Username, rec.Code)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if list, _ := s.h.lists.Entries(); len(list) != 1 {
|
||||||
|
t.Errorf("a domain administrator changed the lists: %+v", list)
|
||||||
|
}
|
||||||
|
if got := s.mapFile(t, "deny_domain.map"); strings.Contains(got, "evil.example") || !strings.Contains(got, "bulk.example") {
|
||||||
|
t.Errorf("deny_domain.map = %q", got)
|
||||||
|
}
|
||||||
|
|
||||||
|
s.h.cfg.InboundEnabled = false
|
||||||
|
if rec := s.get(s.h.HandleFilterLists, "/inbound/filter-lists", globalPrincipal); rec.Code != http.StatusNotFound {
|
||||||
|
t.Errorf("with the inbound relay off GET = %d, want 404", rec.Code)
|
||||||
|
}
|
||||||
|
if rec := s.post(s.h.HandleFilterListAdd, "/inbound/filter-lists", entry("deny", "domain", "evil.example", ""), globalPrincipal); rec.Code != http.StatusNotFound {
|
||||||
|
t.Errorf("with the inbound relay off POST = %d, want 404", rec.Code)
|
||||||
|
}
|
||||||
|
if rec := s.post(s.h.HandleFilterListDelete, "/inbound/filter-lists/"+itoa(e.ID)+"/delete", nil, globalPrincipal); rec.Code != http.StatusNotFound {
|
||||||
|
t.Errorf("with the inbound relay off POST delete = %d, want 404", rec.Code)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Nothing to write the lists to is nothing to show.
|
||||||
|
s.h.cfg.InboundEnabled, s.h.lists = true, nil
|
||||||
|
if rec := s.get(s.h.HandleFilterLists, "/inbound/filter-lists", globalPrincipal); rec.Code != http.StatusNotFound {
|
||||||
|
t.Errorf("without lists GET = %d, want 404", rec.Code)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestFilterListsAddAndRemove(t *testing.T) {
|
||||||
|
s := newInboundLogStand(t)
|
||||||
|
empty := s.get(s.h.HandleFilterLists, "/inbound/filter-lists", globalPrincipal).Body.String()
|
||||||
|
has(t, "empty lists", empty, `<h1 class="title is-3">Filter lists</h1>`, `<h2>Deny</h2><div class="sp-end">0 entries · wins over Allow</div>`,
|
||||||
|
`<h2>Allow</h2><div class="sp-end">0 entries</div>`, `No sender is turned away by hand.`, `No sender is let through by hand.`)
|
||||||
|
lacks(t, "empty lists", empty, `<table`)
|
||||||
|
|
||||||
|
// The domain is normalised, written to the file of its list and kind, and the
|
||||||
|
// page shows it.
|
||||||
|
rec := s.post(s.h.HandleFilterListAdd, "/inbound/filter-lists", entry("deny", "domain", " @Bulk.Example ", " newsletter "), globalPrincipal)
|
||||||
|
if rec.Code != http.StatusSeeOther || rec.Header().Get("Location") != "/inbound/filter-lists?added=1" {
|
||||||
|
t.Fatalf("add = %d to %q:\n%s", rec.Code, rec.Header().Get("Location"), rec.Body.String())
|
||||||
|
}
|
||||||
|
if got := s.mapFile(t, "deny_domain.map"); !strings.Contains(got, "\nbulk.example\n") {
|
||||||
|
t.Errorf("deny_domain.map = %q", got)
|
||||||
|
}
|
||||||
|
if got := s.mapFile(t, "allow_domain.map"); strings.Contains(got, "bulk.example") {
|
||||||
|
t.Errorf("allow_domain.map = %q", got)
|
||||||
|
}
|
||||||
|
rec = s.post(s.h.HandleFilterListAdd, "/inbound/filter-lists", entry("allow", "ip", "198.51.100.77/24", ""), globalPrincipal)
|
||||||
|
if rec.Code != http.StatusSeeOther {
|
||||||
|
t.Fatalf("add ip = %d:\n%s", rec.Code, rec.Body.String())
|
||||||
|
}
|
||||||
|
if got := s.mapFile(t, "allow_ip.map"); !strings.Contains(got, "\n198.51.100.0/24\n") {
|
||||||
|
t.Errorf("allow_ip.map = %q", got)
|
||||||
|
}
|
||||||
|
|
||||||
|
page := s.get(s.h.HandleFilterLists, "/inbound/filter-lists?added=1", globalPrincipal).Body.String()
|
||||||
|
has(t, "lists", page,
|
||||||
|
`<div class="notification is-success is-light">`, `Entry added.`,
|
||||||
|
`<h2>Deny</h2><div class="sp-end">1 entry · wins over Allow</div>`, `<h2>Allow</h2><div class="sp-end">1 entry</div>`,
|
||||||
|
`<td class="sp-mono sp-small">bulk.example</td><td><button type="button" class="tag is-light" popovertarget="entry-`,
|
||||||
|
`<tr><th>Note</th><td>newsletter</td></tr><tr><th>Added</th><td class="sp-mono sp-small">`+time.Now().UTC().Format("2006-01-02")+`</td></tr><tr><th>Matched</th><td>never in the last 90 days</td></tr>`,
|
||||||
|
`<td class="sp-mono sp-small">198.51.100.0/24</td>`, `<tr><th>Note</th><td>—</td></tr>`,
|
||||||
|
`<td class="sp-muted sp-nowrap">never</td>`,
|
||||||
|
)
|
||||||
|
lacks(t, "lists", page, `No sender is turned away`)
|
||||||
|
|
||||||
|
// Remove the deny entry: gone from the page and from the file, the allow
|
||||||
|
// entry untouched.
|
||||||
|
entries, _ := s.h.lists.Entries()
|
||||||
|
var denyID int64
|
||||||
|
for _, e := range entries {
|
||||||
|
if e.List == "deny" {
|
||||||
|
denyID = e.ID
|
||||||
|
}
|
||||||
|
}
|
||||||
|
has(t, "lists", page, `action="/inbound/filter-lists/`+itoa(denyID)+`/delete"`)
|
||||||
|
rec = s.post(s.h.HandleFilterListDelete, "/inbound/filter-lists/"+itoa(denyID)+"/delete", nil, globalPrincipal)
|
||||||
|
if rec.Code != http.StatusSeeOther || rec.Header().Get("Location") != "/inbound/filter-lists?removed=1" {
|
||||||
|
t.Fatalf("delete = %d to %q", rec.Code, rec.Header().Get("Location"))
|
||||||
|
}
|
||||||
|
if got := s.mapFile(t, "deny_domain.map"); strings.Contains(got, "bulk.example") {
|
||||||
|
t.Errorf("deny_domain.map still holds the entry: %q", got)
|
||||||
|
}
|
||||||
|
if got := s.mapFile(t, "allow_ip.map"); !strings.Contains(got, "198.51.100.0/24") {
|
||||||
|
t.Errorf("allow_ip.map lost its entry: %q", got)
|
||||||
|
}
|
||||||
|
page = s.get(s.h.HandleFilterLists, "/inbound/filter-lists?removed=1", globalPrincipal).Body.String()
|
||||||
|
has(t, "lists", page, `Entry removed.`, `<h2>Deny</h2><div class="sp-end">0 entries`)
|
||||||
|
lacks(t, "lists", page, `bulk.example <span`)
|
||||||
|
|
||||||
|
// An entry that is not there, or an id that is not one.
|
||||||
|
for _, id := range []string{itoa(denyID), "9999", "x", "0", "-1"} {
|
||||||
|
if rec := s.post(s.h.HandleFilterListDelete, "/inbound/filter-lists/"+id+"/delete", nil, globalPrincipal); rec.Code != http.StatusNotFound {
|
||||||
|
t.Errorf("delete %q = %d, want 404", id, rec.Code)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A refusal comes back with 422 on the same page, the message between the head
|
||||||
|
// and the form and what was typed kept in it; nothing is written.
|
||||||
|
func TestFilterListsRefusals(t *testing.T) {
|
||||||
|
s := newInboundLogStand(t)
|
||||||
|
if rec := s.post(s.h.HandleFilterListAdd, "/inbound/filter-lists", entry("deny", "address", "win@lottery.example", ""), globalPrincipal); rec.Code != http.StatusSeeOther {
|
||||||
|
t.Fatalf("add = %d", rec.Code)
|
||||||
|
}
|
||||||
|
before := s.mapFile(t, "deny_from.map")
|
||||||
|
|
||||||
|
for _, c := range []struct {
|
||||||
|
name string
|
||||||
|
form url.Values
|
||||||
|
want string
|
||||||
|
}{
|
||||||
|
{"duplicate", entry("deny", "address", "WIN@lottery.example", "again"), "That entry is already on the list."},
|
||||||
|
{"not an address", entry("deny", "address", "not-an-address", ""), "is not a valid email address"},
|
||||||
|
{"not a network", entry("allow", "ip", "192.0.2.0/99", "kept note"), "is not a network in CIDR form"},
|
||||||
|
{"no kind", entry("deny", "everything", "x", ""), "choose what the entry is"},
|
||||||
|
{"no list", entry("maybe", "domain", "x.example", ""), "choose the allow or the deny list"},
|
||||||
|
{"empty", entry("deny", "domain", " ", ""), "domain is required"},
|
||||||
|
{"long note", entry("deny", "domain", "x.example", strings.Repeat("n", 201)), "note must be at most 200 characters"},
|
||||||
|
} {
|
||||||
|
rec := s.post(s.h.HandleFilterListAdd, "/inbound/filter-lists", c.form, globalPrincipal)
|
||||||
|
if rec.Code != http.StatusUnprocessableEntity {
|
||||||
|
t.Errorf("%s: %d, want 422:\n%s", c.name, rec.Code, rec.Body.String())
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
body := rec.Body.String()
|
||||||
|
has(t, c.name, body, `<div class="notification is-danger is-light">`, c.want, `value="`+c.form.Get("value")+`"`)
|
||||||
|
if c.form.Get("list_type") == "allow" || c.form.Get("list_type") == "deny" {
|
||||||
|
has(t, c.name, body, `<option value="`+c.form.Get("list_type")+`" selected>`)
|
||||||
|
}
|
||||||
|
if c.form.Get("note") != "" && len(c.form.Get("note")) < 100 {
|
||||||
|
has(t, c.name, body, `value="`+c.form.Get("note")+`"`)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if got := s.mapFile(t, "deny_from.map"); got != before {
|
||||||
|
t.Errorf("a refused entry changed deny_from.map:\n%s", got)
|
||||||
|
}
|
||||||
|
if e, _ := s.h.lists.Entries(); len(e) != 1 {
|
||||||
|
t.Errorf("a refused entry was stored: %+v", e)
|
||||||
|
}
|
||||||
|
|
||||||
|
// What was typed is escaped when it comes back.
|
||||||
|
rec := s.post(s.h.HandleFilterListAdd, "/inbound/filter-lists", entry("deny", "address", `"><script>1</script>`, `<i>x</i>`), globalPrincipal)
|
||||||
|
if rec.Code != http.StatusUnprocessableEntity {
|
||||||
|
t.Fatalf("hostile entry = %d", rec.Code)
|
||||||
|
}
|
||||||
|
lacks(t, "refused form", rec.Body.String(), `<script>1`, `<i>x</i>`)
|
||||||
|
}
|
||||||
|
|
||||||
|
// The log's Allow / Deny sender links fill the form in. Only a list and a kind
|
||||||
|
// the form offers are taken; the value is text.
|
||||||
|
func TestFilterListsPrefillFromTheLog(t *testing.T) {
|
||||||
|
s := newInboundLogStand(t)
|
||||||
|
get := func(query string) string {
|
||||||
|
rec := s.get(s.h.HandleFilterLists, "/inbound/filter-lists"+query, globalPrincipal)
|
||||||
|
if rec.Code != http.StatusOK {
|
||||||
|
t.Fatalf("GET %s = %d", query, rec.Code)
|
||||||
|
}
|
||||||
|
return rec.Body.String()
|
||||||
|
}
|
||||||
|
body := get("?list=allow&kind=address&value=billing%40partner.example")
|
||||||
|
has(t, "prefilled", body, `<option value="allow" selected>`, `<option value="address" selected>`,
|
||||||
|
`id="value" name="value" value="billing@partner.example"`)
|
||||||
|
lacks(t, "prefilled", body, `<option value="deny" selected>`)
|
||||||
|
|
||||||
|
body = get("?list=allow&kind=domain&value=partner.example")
|
||||||
|
has(t, "prefilled", body, `<option value="allow" selected>`, `<option value="domain" selected>`, `value="partner.example"`)
|
||||||
|
|
||||||
|
// Unknown choices leave the form on its first; the value is still put in.
|
||||||
|
body = get("?list=everything&kind=whatever&value=x%40y.example")
|
||||||
|
has(t, "unknown choices", body, `<option value="deny" selected>`, `<option value="address" selected>`, `value="x@y.example"`)
|
||||||
|
|
||||||
|
// The value cannot leave its attribute.
|
||||||
|
body = get(`?value=` + url.QueryEscape(`"><script>alert(1)</script>`))
|
||||||
|
lacks(t, "hostile value", body, `<script>alert`)
|
||||||
|
has(t, "hostile value", body, `value=""><script>alert(1)</script>"`)
|
||||||
|
|
||||||
|
// Nothing in the query is a flash.
|
||||||
|
lacks(t, "plain", get(""), `notification`)
|
||||||
|
}
|
||||||
|
|
||||||
|
// The log's links and the form agree: what the log links to is what the form
|
||||||
|
// accepts, for every kind of sender it can name.
|
||||||
|
func TestFilterListsAcceptWhatTheLogLinksTo(t *testing.T) {
|
||||||
|
s := newInboundLogStand(t)
|
||||||
|
log := s.get(s.h.HandleInboundLog, "/inbound/log", globalPrincipal).Body.String()
|
||||||
|
has(t, "log", log, `href="/inbound/filter-lists?list=deny&kind=address&value=sender%40example.net"`)
|
||||||
|
body := s.get(s.h.HandleFilterLists, "/inbound/filter-lists?list=deny&kind=address&value=sender%40example.net", globalPrincipal).Body.String()
|
||||||
|
has(t, "form", body, `value="sender@example.net"`)
|
||||||
|
if rec := s.post(s.h.HandleFilterListAdd, "/inbound/filter-lists", entry("deny", "address", "sender@example.net", ""), globalPrincipal); rec.Code != http.StatusSeeOther {
|
||||||
|
t.Errorf("the entry the log leads to was refused: %d", rec.Code)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Box 04 says whether anything reads the lists and when they were last written.
|
||||||
|
func TestFilterListsWhereTheyGo(t *testing.T) {
|
||||||
|
s := newInboundLogStand(t)
|
||||||
|
|
||||||
|
// Filter off: the lists are kept, the box says nothing reads them, and the
|
||||||
|
// files have never been written.
|
||||||
|
body := s.get(s.h.HandleFilterLists, "/inbound/filter-lists", globalPrincipal).Body.String()
|
||||||
|
has(t, "filter off", body, `<h2>Where the lists go</h2>`, `<span class="tag is-light">not applied</span>`,
|
||||||
|
`Nothing — the spam filter is off. The lists are kept and can be edited; nothing reads them until <code>INBOUND_ANTISPAM_MILTER</code> is set.`,
|
||||||
|
`<dt>Last written</dt><dd>never</dd>`, `Deny wins, whichever entries they are.`)
|
||||||
|
lacks(t, "filter off", body, `is-success is-light">applied`, `rspamd, from map files`)
|
||||||
|
|
||||||
|
// Filter on, and the files written.
|
||||||
|
s.h.cfg.InboundAntispamMilter = "inet:antispam:11332"
|
||||||
|
if err := s.h.lists.Sync(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
stamp := time.Date(2026, 9, 11, 8, 40, 0, 0, time.UTC)
|
||||||
|
for _, name := range []string{"allow_ip.map", "allow_from.map", "allow_domain.map", "deny_ip.map", "deny_from.map", "deny_domain.map"} {
|
||||||
|
if err := os.Chtimes(filepath.Join(s.dir, name), stamp, stamp); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
body = s.get(s.h.HandleFilterLists, "/inbound/filter-lists", globalPrincipal).Body.String()
|
||||||
|
has(t, "filter on", body, `<span class="tag is-success is-light">applied</span>`,
|
||||||
|
`rspamd, from map files it rereads within seconds of a change`, `<dt>Last written</dt><dd>2026-09-11 08:40 UTC</dd>`,
|
||||||
|
`/data/antispam/`)
|
||||||
|
lacks(t, "filter on", body, `not applied`, `Nothing — the spam filter is off`, `INBOUND_ANTISPAM_MILTER`)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- Settings and Help
|
||||||
|
|
||||||
|
func settingsWithInbound(t *testing.T, on bool) *Handlers {
|
||||||
|
t.Helper()
|
||||||
|
h, _ := settingsServer(t)
|
||||||
|
h.cfg.InboundEnabled = on
|
||||||
|
h.cfg.InboundSpamLogRetentionEnvDefault = 90
|
||||||
|
return h
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSettingsShowsAndSavesTheInboundRetention(t *testing.T) {
|
||||||
|
h := settingsWithInbound(t, true)
|
||||||
|
body := getBody(t, h.HandleServerSettings, "/server/settings")
|
||||||
|
has(t, "settings", body, `Keep outbound log rows, days`, `Keep inbound log rows, days`,
|
||||||
|
`name="inbound_spam_log_retention_days"`, `value="90"`,
|
||||||
|
`The inbound log also never holds more than 10 000 rows`)
|
||||||
|
|
||||||
|
if err := h.store.SetInboundSpamLogRetentionDays(45); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
has(t, "settings", getBody(t, h.HandleServerSettings, "/server/settings"), `name="inbound_spam_log_retention_days" type="number" min="7" max="365" step="1" value="45"`)
|
||||||
|
|
||||||
|
rec := postFormAs(h.HandleServerSettings, globalPrincipal, "/server/settings", nil,
|
||||||
|
url.Values{"send_log_retention_days": {"120"}, "inbound_spam_log_retention_days": {"200"}})
|
||||||
|
if rec.Code != http.StatusSeeOther || rec.Header().Get("Location") != "/server/settings?saved=1" {
|
||||||
|
t.Fatalf("POST /server/settings = %d to %q:\n%s", rec.Code, rec.Header().Get("Location"), rec.Body.String())
|
||||||
|
}
|
||||||
|
if got, _ := h.store.GetSendLogRetentionDays(90); got != 120 {
|
||||||
|
t.Errorf("outbound retention = %d, want 120", got)
|
||||||
|
}
|
||||||
|
if got, _ := h.store.GetInboundSpamLogRetentionDays(90); got != 200 {
|
||||||
|
t.Errorf("inbound retention = %d, want 200", got)
|
||||||
|
}
|
||||||
|
// The Inbound log's lead reads the same setting.
|
||||||
|
if got := h.inboundSpamLogRetentionDays(); got != 200 {
|
||||||
|
t.Errorf("inboundSpamLogRetentionDays = %d, want 200", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSettingsRefuseAnInboundRetentionOutOfRange(t *testing.T) {
|
||||||
|
h := settingsWithInbound(t, true)
|
||||||
|
for _, bad := range []string{"3", "366", "nine", ""} {
|
||||||
|
rec := postFormAs(h.HandleServerSettings, globalPrincipal, "/server/settings", nil,
|
||||||
|
url.Values{"send_log_retention_days": {"120"}, "inbound_spam_log_retention_days": {bad}})
|
||||||
|
if rec.Code != http.StatusBadRequest {
|
||||||
|
t.Fatalf("POST (inbound %q) = %d, want 400:\n%s", bad, rec.Code, rec.Body.String())
|
||||||
|
}
|
||||||
|
if bad == "3" && !strings.Contains(rec.Body.String(), "between 7 and 365") {
|
||||||
|
t.Errorf("no range in the refusal:\n%s", rec.Body.String())
|
||||||
|
}
|
||||||
|
if bad == "nine" && !strings.Contains(rec.Body.String(), "Inbound log retention must be a whole number of days.") {
|
||||||
|
t.Errorf("no sentence in the refusal:\n%s", rec.Body.String())
|
||||||
|
}
|
||||||
|
// The outbound value that was fine is shown as typed, the bad one as it was.
|
||||||
|
has(t, "refused form", rec.Body.String(), `name="send_log_retention_days" type="number" min="7" max="365" step="1" value="120"`)
|
||||||
|
}
|
||||||
|
// Nothing was saved, not even the outbound value that was fine.
|
||||||
|
if got, _ := h.store.GetSendLogRetentionDays(90); got != 90 {
|
||||||
|
t.Errorf("a refused form changed the outbound retention to %d", got)
|
||||||
|
}
|
||||||
|
if got, _ := h.store.GetInboundSpamLogRetentionDays(90); got != 90 {
|
||||||
|
t.Errorf("a refused form changed the inbound retention to %d", got)
|
||||||
|
}
|
||||||
|
// A bad outbound value refuses the whole form too.
|
||||||
|
rec := postFormAs(h.HandleServerSettings, globalPrincipal, "/server/settings", nil,
|
||||||
|
url.Values{"send_log_retention_days": {"3"}, "inbound_spam_log_retention_days": {"200"}})
|
||||||
|
if rec.Code != http.StatusBadRequest {
|
||||||
|
t.Fatalf("POST (outbound 3) = %d", rec.Code)
|
||||||
|
}
|
||||||
|
if got, _ := h.store.GetInboundSpamLogRetentionDays(90); got != 90 {
|
||||||
|
t.Errorf("the inbound retention was saved by a refused form: %d", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// With the relay off there is no inbound journal to keep: the field is not on the
|
||||||
|
// page and a value posted for it is not read.
|
||||||
|
func TestSettingsHaveNoInboundRetentionWithTheRelayOff(t *testing.T) {
|
||||||
|
h := settingsWithInbound(t, false)
|
||||||
|
body := getBody(t, h.HandleServerSettings, "/server/settings")
|
||||||
|
has(t, "settings", body, `Keep outbound log rows, days`)
|
||||||
|
lacks(t, "settings", body, `inbound_spam_log_retention_days`, `Keep inbound log rows`)
|
||||||
|
|
||||||
|
rec := postFormAs(h.HandleServerSettings, globalPrincipal, "/server/settings", nil,
|
||||||
|
url.Values{"send_log_retention_days": {"120"}, "inbound_spam_log_retention_days": {"3"}})
|
||||||
|
if rec.Code != http.StatusSeeOther {
|
||||||
|
t.Fatalf("POST = %d:\n%s", rec.Code, rec.Body.String())
|
||||||
|
}
|
||||||
|
if got, _ := h.store.GetSendLogRetentionDays(90); got != 120 {
|
||||||
|
t.Errorf("outbound retention = %d, want 120", got)
|
||||||
|
}
|
||||||
|
if raw, _ := h.store.GetSetting(store.InboundSpamLogRetentionDaysKey); raw != "" {
|
||||||
|
t.Errorf("an inbound retention was saved with the relay off: %q", raw)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestHelpHasTheInboundLogAndFilterListsTopics(t *testing.T) {
|
||||||
|
h, _ := inboundHandlers(t)
|
||||||
|
rec := send(h.HandleHelp, &globalPrincipal, "GET", "/help", nil, nil)
|
||||||
|
has(t, "help", rec.Body.String(),
|
||||||
|
`id="inbound-log"`, `id="filter-lists"`, `href="#inbound-log"`, `href="#filter-lists"`,
|
||||||
|
`With the inbound relay on there is a second number, for the Inbound log, with the same range.`)
|
||||||
|
// Both pages' help links land on them.
|
||||||
|
s := inboundLogStand{h: h, st: h.store, dir: filepath.Join(t.TempDir(), "antispam")}
|
||||||
|
h.lists = inbound.NewLists(h.store, s.dir)
|
||||||
|
has(t, "log", s.get(h.HandleInboundLog, "/inbound/log", globalPrincipal).Body.String(), `href="/help#inbound-log"`)
|
||||||
|
has(t, "lists", s.get(h.HandleFilterLists, "/inbound/filter-lists", globalPrincipal).Body.String(), `href="/help#filter-lists"`)
|
||||||
|
}
|
||||||
|
|
||||||
|
// An entry's count comes from the journal: a row counts for the entry that
|
||||||
|
// fits it on the list that decided it.
|
||||||
|
func TestFilterListsCountMatchesFromTheJournal(t *testing.T) {
|
||||||
|
s := newInboundLogStand(t)
|
||||||
|
for _, e := range [][3]string{{"deny", "domain", "bulk.example"}, {"allow", "address", "billing@vendor.example"}, {"deny", "ip", "203.0.113.0/24"}} {
|
||||||
|
if _, err := s.h.lists.Add(e[0], e[1], e[2], ""); err != nil {
|
||||||
|
t.Fatalf("Add %v: %v", e, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
at := time.Date(2026, 10, 1, 9, 30, 0, 0, time.UTC)
|
||||||
|
for i, row := range []store.InboundSpamEntry{
|
||||||
|
{From: "promo@bulk.example", ClientIP: "198.51.100.9", Decision: store.SpamDecisionReject, Engine: store.SpamEngineRspamd, Detail: "5.7.1 Refused by this server's deny list"},
|
||||||
|
{From: "other@BULK.example", ClientIP: "198.51.100.9", Decision: store.SpamDecisionReject, Engine: store.SpamEngineRspamd, Detail: "5.7.1 Refused by this server's deny list"},
|
||||||
|
{From: "billing@vendor.example", ClientIP: "192.0.2.5", Decision: store.SpamDecisionAccept, Engine: store.SpamEngineRspamd, Detail: "no action, score -99.00 / 15.00; SELFPOST_ALLOW_FROM(-100.00)"},
|
||||||
|
// refused by the filter for its own reasons: nobody's match
|
||||||
|
{From: "promo@bulk.example", ClientIP: "198.51.100.9", Decision: store.SpamDecisionReject, Engine: store.SpamEngineRspamd, Detail: "5.7.1 Spam message rejected"},
|
||||||
|
} {
|
||||||
|
row.InboundDomain, row.To, row.At = "a.example.com", "alice@a.example.com", at.Add(time.Duration(i)*time.Hour)
|
||||||
|
if err := s.st.InsertInboundSpam(row); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
page := s.get(s.h.HandleFilterLists, "/inbound/filter-lists", globalPrincipal).Body.String()
|
||||||
|
has(t, "lists", page,
|
||||||
|
`<tr><th>Matched</th><td>2 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-10-01 10:30</td></tr>`,
|
||||||
|
`<td class="sp-nowrap">2</td>`,
|
||||||
|
`<tr><th>Matched</th><td>1 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-10-01 11:30</td></tr>`,
|
||||||
|
`<td class="sp-nowrap">1</td>`,
|
||||||
|
// the network nobody came from
|
||||||
|
`<td class="sp-muted sp-nowrap">never</td>`,
|
||||||
|
)
|
||||||
|
if got := strings.Count(page, `<td class="sp-muted sp-nowrap">never</td>`); got != 1 {
|
||||||
|
t.Errorf("%d entries say never, want the one network", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -10,7 +10,8 @@ import (
|
|||||||
)
|
)
|
||||||
|
|
||||||
// Server › Settings holds what is true of the whole instance, whoever is
|
// Server › Settings holds what is true of the whole instance, whoever is
|
||||||
// signed in: log retention and the read-only view of the rate limits. A
|
// signed in: log retention (the outbound log's, and the inbound log's while the
|
||||||
|
// inbound relay is on) and the read-only view of the rate limits. A
|
||||||
// user's own name, e-mail, password and DMARC default are on Account
|
// user's own name, e-mail, password and DMARC default are on Account
|
||||||
// (handlers_account.go). Before placing a setting here, ask whose it is.
|
// (handlers_account.go). Before placing a setting here, ask whose it is.
|
||||||
|
|
||||||
@@ -21,7 +22,7 @@ func (h *Handlers) HandleServerSettings(w http.ResponseWriter, r *http.Request)
|
|||||||
}
|
}
|
||||||
switch r.Method {
|
switch r.Method {
|
||||||
case http.MethodGet:
|
case http.MethodGet:
|
||||||
h.renderServerSettings(w, r, http.StatusOK, "", h.sendLogRetentionDays())
|
h.renderServerSettings(w, r, http.StatusOK, "", h.currentRetentions())
|
||||||
case http.MethodPost:
|
case http.MethodPost:
|
||||||
h.submitServerSettings(w, r)
|
h.submitServerSettings(w, r)
|
||||||
default:
|
default:
|
||||||
@@ -30,8 +31,26 @@ func (h *Handlers) HandleServerSettings(w http.ResponseWriter, r *http.Request)
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func (h *Handlers) renderServerSettings(w http.ResponseWriter, r *http.Request, status int, formErr string, formRetention int) {
|
// retentions are the two journal retentions the Settings form shows: the
|
||||||
page := view.NewSettings(h.shellMeta(r), strconv.Itoa(formRetention), view.FormatRate(h.l1Messages(), h.l1Window()))
|
// outbound log's, and the inbound log's while the inbound relay is on.
|
||||||
|
type retentions struct {
|
||||||
|
outbound, inbound int
|
||||||
|
}
|
||||||
|
|
||||||
|
func (h *Handlers) currentRetentions() retentions {
|
||||||
|
cur := retentions{outbound: h.sendLogRetentionDays()}
|
||||||
|
if h.cfg.InboundEnabled {
|
||||||
|
cur.inbound = h.inboundSpamLogRetentionDays()
|
||||||
|
}
|
||||||
|
return cur
|
||||||
|
}
|
||||||
|
|
||||||
|
func (h *Handlers) renderServerSettings(w http.ResponseWriter, r *http.Request, status int, formErr string, form retentions) {
|
||||||
|
page := view.NewSettings(h.shellMeta(r), strconv.Itoa(form.outbound), view.FormatRate(h.l1Messages(), h.l1Window()))
|
||||||
|
// The inbound field exists while there is an inbound journal to keep.
|
||||||
|
if h.cfg.InboundEnabled {
|
||||||
|
page.WithInbound(strconv.Itoa(form.inbound))
|
||||||
|
}
|
||||||
flash := ""
|
flash := ""
|
||||||
if r.URL.Query().Has("saved") {
|
if r.URL.Query().Has("saved") {
|
||||||
flash = "Settings saved."
|
flash = "Settings saved."
|
||||||
@@ -41,7 +60,7 @@ func (h *Handlers) renderServerSettings(w http.ResponseWriter, r *http.Request,
|
|||||||
}
|
}
|
||||||
|
|
||||||
func (h *Handlers) submitServerSettings(w http.ResponseWriter, r *http.Request) {
|
func (h *Handlers) submitServerSettings(w http.ResponseWriter, r *http.Request) {
|
||||||
current := h.sendLogRetentionDays()
|
current := h.currentRetentions()
|
||||||
if err := r.ParseForm(); err != nil {
|
if err := r.ParseForm(); err != nil {
|
||||||
h.renderServerSettings(w, r, http.StatusBadRequest, "Invalid form submission.", current)
|
h.renderServerSettings(w, r, http.StatusBadRequest, "Invalid form submission.", current)
|
||||||
return
|
return
|
||||||
@@ -51,17 +70,40 @@ func (h *Handlers) submitServerSettings(w http.ResponseWriter, r *http.Request)
|
|||||||
h.renderServerSettings(w, r, http.StatusBadRequest, "Send log retention must be a whole number of days.", current)
|
h.renderServerSettings(w, r, http.StatusBadRequest, "Send log retention must be a whole number of days.", current)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
form := retentions{outbound: days, inbound: current.inbound}
|
||||||
if err := store.ValidateSendLogRetentionDays(days); err != nil {
|
if err := store.ValidateSendLogRetentionDays(days); err != nil {
|
||||||
h.renderServerSettings(w, r, http.StatusBadRequest, err.Error(), days)
|
h.renderServerSettings(w, r, http.StatusBadRequest, err.Error(), form)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
if h.cfg.InboundEnabled {
|
||||||
|
inboundDays, err := strconv.Atoi(strings.TrimSpace(r.PostFormValue("inbound_spam_log_retention_days")))
|
||||||
|
if err != nil {
|
||||||
|
h.renderServerSettings(w, r, http.StatusBadRequest, "Inbound log retention must be a whole number of days.", form)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
form.inbound = inboundDays
|
||||||
|
if err := store.ValidateInboundSpamLogRetentionDays(inboundDays); err != nil {
|
||||||
|
h.renderServerSettings(w, r, http.StatusBadRequest, err.Error(), form)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
if err := h.store.SetSendLogRetentionDays(days); err != nil {
|
if err := h.store.SetSendLogRetentionDays(days); err != nil {
|
||||||
logf("panel: settings: set send-log retention failed: %v", err)
|
logf("panel: settings: set send-log retention failed: %v", err)
|
||||||
h.renderServerSettings(w, r, http.StatusInternalServerError,
|
h.renderServerSettings(w, r, http.StatusInternalServerError,
|
||||||
"Could not save send log retention. Please check the logs and try again.", days)
|
"Could not save send log retention. Please check the logs and try again.", form)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
if h.cfg.InboundEnabled {
|
||||||
|
if err := h.store.SetInboundSpamLogRetentionDays(form.inbound); err != nil {
|
||||||
|
logf("panel: settings: set inbound-log retention failed: %v", err)
|
||||||
|
h.renderServerSettings(w, r, http.StatusInternalServerError,
|
||||||
|
"Could not save inbound log retention. Please check the logs and try again.", form)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
logf("panel: instance settings updated (send-log retention: %d days, inbound-log retention: %d days)", days, form.inbound)
|
||||||
|
} else {
|
||||||
logf("panel: instance settings updated (send-log retention: %d days)", days)
|
logf("panel: instance settings updated (send-log retention: %d days)", days)
|
||||||
|
}
|
||||||
http.Redirect(w, r, "/server/settings?saved=1", http.StatusSeeOther)
|
http.Redirect(w, r, "/server/settings?saved=1", http.StatusSeeOther)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -56,6 +56,10 @@ type Settings struct {
|
|||||||
|
|
||||||
Retention Box
|
Retention Box
|
||||||
RetentionDays string
|
RetentionDays string
|
||||||
|
// Inbound is whether the inbound relay is on, which gives the retention box
|
||||||
|
// its second field; InboundRetentionDays is what that field shows.
|
||||||
|
Inbound bool
|
||||||
|
InboundRetentionDays string
|
||||||
|
|
||||||
RateLimits Box
|
RateLimits Box
|
||||||
RateFacts []Fact
|
RateFacts []Fact
|
||||||
@@ -86,6 +90,14 @@ func NewSettings(m Meta, retention, level1 string) *Settings {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// WithInbound adds the retention of the Inbound log to the Log retention box.
|
||||||
|
// days is what the field shows, like retention in NewSettings. The field exists
|
||||||
|
// only while the inbound relay is on: with it off there is no journal to keep.
|
||||||
|
func (s *Settings) WithInbound(days string) *Settings {
|
||||||
|
s.Inbound, s.InboundRetentionDays = true, days
|
||||||
|
return s
|
||||||
|
}
|
||||||
|
|
||||||
// WithResult shows the result of the last action, or its refusal, between the
|
// WithResult shows the result of the last action, or its refusal, between the
|
||||||
// head and the form.
|
// head and the form.
|
||||||
func (s *Settings) WithResult(flash, formErr string) *Settings {
|
func (s *Settings) WithResult(flash, formErr string) *Settings {
|
||||||
|
|||||||
@@ -105,18 +105,41 @@ type Tag struct {
|
|||||||
Label string
|
Label string
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Sheet is what does not fit a table row, opened over the page from the row's
|
||||||
|
// tag (partials "sheet_tag" and "sheet"; used by the Inbound log, where it says
|
||||||
|
// why a message was decided the way it was). It is the browser's own popover:
|
||||||
|
// the tag is a button that names the sheet by ID, so ID is unique on the page.
|
||||||
|
// Its content is a table of name and value.
|
||||||
|
type Sheet struct {
|
||||||
|
ID string
|
||||||
|
Tag Tag
|
||||||
|
Title string
|
||||||
|
Rows []SheetRow
|
||||||
|
}
|
||||||
|
|
||||||
|
// SheetRow is one name and its value in a sheet. A value of several lines is
|
||||||
|
// one line each; Mono is for what is read as written - a reply, a list of
|
||||||
|
// symbols.
|
||||||
|
type SheetRow struct {
|
||||||
|
Label string
|
||||||
|
Lines []string
|
||||||
|
Mono bool
|
||||||
|
}
|
||||||
|
|
||||||
// statusClass is the one mapping of status to Bulma tag classes — box 03 of the
|
// statusClass is the one mapping of status to Bulma tag classes — box 03 of the
|
||||||
// kit page. A page passes the status it knows (a DKIM check says ok, a message
|
// kit page. A page passes the status it knows (a DKIM check says ok, a message
|
||||||
// says deferred) and never picks a colour.
|
// says deferred) and never picks a colour.
|
||||||
var statusClass = map[string]string{
|
var statusClass = map[string]string{
|
||||||
"ok": "tag is-success is-light", "published": "tag is-success is-light",
|
"ok": "tag is-success is-light", "published": "tag is-success is-light",
|
||||||
"running": "tag is-success is-light", "sent": "tag is-success is-light",
|
"running": "tag is-success is-light", "sent": "tag is-success is-light",
|
||||||
|
"accept": "tag is-success is-light",
|
||||||
|
|
||||||
"warn": "tag is-warning is-light", "deferred": "tag is-warning is-light",
|
"warn": "tag is-warning is-light", "deferred": "tag is-warning is-light",
|
||||||
"quiet": "tag is-warning is-light",
|
"quiet": "tag is-warning is-light", "tempfail": "tag is-warning is-light",
|
||||||
|
|
||||||
"error": "tag is-danger is-light", "fail": "tag is-danger is-light",
|
"error": "tag is-danger is-light", "fail": "tag is-danger is-light",
|
||||||
"mismatch": "tag is-danger is-light", "bounced": "tag is-danger is-light",
|
"mismatch": "tag is-danger is-light", "bounced": "tag is-danger is-light",
|
||||||
|
"reject": "tag is-danger is-light",
|
||||||
|
|
||||||
"quarantine": "tag is-primary is-light", "held": "tag is-primary is-light",
|
"quarantine": "tag is-primary is-light", "held": "tag is-primary is-light",
|
||||||
"global": "tag is-primary is-light",
|
"global": "tag is-primary is-light",
|
||||||
@@ -503,6 +526,9 @@ type Shell struct {
|
|||||||
ShowDMARC bool
|
ShowDMARC bool
|
||||||
ShowInbound bool
|
ShowInbound bool
|
||||||
ShowServer bool
|
ShowServer bool
|
||||||
|
// ShowFilterLists is the Filter lists entry of Inbound: the lists are the
|
||||||
|
// instance's, so only the global role has them.
|
||||||
|
ShowFilterLists bool
|
||||||
}
|
}
|
||||||
|
|
||||||
// Frame is what layout.html is executed with: the shell and the page's own
|
// Frame is what layout.html is executed with: the shell and the page's own
|
||||||
@@ -523,6 +549,8 @@ func (e *Engine) shell(m Meta, copyright, sourceURL string) Shell {
|
|||||||
ShowQueue: m.IsGlobal,
|
ShowQueue: m.IsGlobal,
|
||||||
ShowOutbound: m.IsGlobal || m.HasOutbound,
|
ShowOutbound: m.IsGlobal || m.HasOutbound,
|
||||||
ShowInbound: e.inboundEnabled && (m.IsGlobal || m.HasInbound),
|
ShowInbound: e.inboundEnabled && (m.IsGlobal || m.HasInbound),
|
||||||
|
|
||||||
|
ShowFilterLists: e.inboundEnabled && m.IsGlobal,
|
||||||
}
|
}
|
||||||
s.ShowDMARC = e.dmarcEnabled && s.ShowOutbound
|
s.ShowDMARC = e.dmarcEnabled && s.ShowOutbound
|
||||||
switch {
|
switch {
|
||||||
|
|||||||
@@ -72,6 +72,7 @@ func TestStatusTagMapsEveryStatusOfTheKitTable(t *testing.T) {
|
|||||||
"running": "tag is-success is-light", "sent": "tag is-success is-light",
|
"running": "tag is-success is-light", "sent": "tag is-success is-light",
|
||||||
"warn": "tag is-warning is-light", "deferred": "tag is-warning is-light",
|
"warn": "tag is-warning is-light", "deferred": "tag is-warning is-light",
|
||||||
"quiet": "tag is-warning is-light",
|
"quiet": "tag is-warning is-light",
|
||||||
|
"accept": "tag is-success is-light", "tempfail": "tag is-warning is-light", "reject": "tag is-danger is-light",
|
||||||
"error": "tag is-danger is-light", "fail": "tag is-danger is-light",
|
"error": "tag is-danger is-light", "fail": "tag is-danger is-light",
|
||||||
"mismatch": "tag is-danger is-light", "bounced": "tag is-danger is-light",
|
"mismatch": "tag is-danger is-light", "bounced": "tag is-danger is-light",
|
||||||
"quarantine": "tag is-primary is-light", "global": "tag is-primary is-light",
|
"quarantine": "tag is-primary is-light", "global": "tag is-primary is-light",
|
||||||
@@ -413,7 +414,7 @@ func TestShellVisibilityFollowsRoleAndFlags(t *testing.T) {
|
|||||||
wantIn bool
|
wantIn bool
|
||||||
}{
|
}{
|
||||||
{"global, everything on", on, Meta{IsGlobal: true},
|
{"global, everything on", on, Meta{IsGlobal: true},
|
||||||
Shell{Home: "/overview", ShowOverview: true, ShowOutbound: true, ShowQueue: true, ShowDMARC: true, ShowInbound: true, ShowServer: true}, true},
|
Shell{Home: "/overview", ShowOverview: true, ShowOutbound: true, ShowQueue: true, ShowDMARC: true, ShowInbound: true, ShowServer: true, ShowFilterLists: true}, true},
|
||||||
{"global, inbound and DMARC off", off, Meta{IsGlobal: true},
|
{"global, inbound and DMARC off", off, Meta{IsGlobal: true},
|
||||||
Shell{Home: "/overview", ShowOverview: true, ShowOutbound: true, ShowQueue: true, ShowServer: true}, true},
|
Shell{Home: "/overview", ShowOverview: true, ShowOutbound: true, ShowQueue: true, ShowServer: true}, true},
|
||||||
{"domain admin with both reaches", on, Meta{HasOutbound: true, HasInbound: true},
|
{"domain admin with both reaches", on, Meta{HasOutbound: true, HasInbound: true},
|
||||||
@@ -446,19 +447,19 @@ func renderShell(t *testing.T, e *Engine, m Meta) string {
|
|||||||
return rec.Body.String()
|
return rec.Body.String()
|
||||||
}
|
}
|
||||||
|
|
||||||
// The menu has the 2.0 entries and none of the ones that are not in 2.0 (plan
|
// The menu has the entries that are built and none of the ones that are not (plan
|
||||||
// § Routes): no Inbound Log, Queue, Quarantine or Filter lists, no Preflight.
|
// § Routes): no Inbound Queue or Quarantine, no Preflight.
|
||||||
func TestMenuHoldsTheTwoDotZeroEntries(t *testing.T) {
|
func TestMenuHoldsTheTwoDotZeroEntries(t *testing.T) {
|
||||||
out := renderShell(t, kitEngine(t), Meta{Title: "t", User: "admin", IsGlobal: true})
|
out := renderShell(t, kitEngine(t), Meta{Title: "t", User: "admin", IsGlobal: true})
|
||||||
for _, href := range []string{
|
for _, href := range []string{
|
||||||
"/overview", "/outbound/domains", "/outbound/log", "/outbound/queue", "/outbound/dmarc",
|
"/overview", "/outbound/domains", "/outbound/log", "/outbound/queue", "/outbound/dmarc",
|
||||||
"/inbound/domains", "/server/health", "/server/log", "/server/backup", "/server/users", "/server/settings",
|
"/inbound/domains", "/inbound/log", "/inbound/filter-lists", "/server/health", "/server/log", "/server/backup", "/server/users", "/server/settings",
|
||||||
"/account", "/help", "/license",
|
"/account", "/help", "/license",
|
||||||
} {
|
} {
|
||||||
mustContain(t, out, `href="`+href+`"`)
|
mustContain(t, out, `href="`+href+`"`)
|
||||||
}
|
}
|
||||||
for _, gone := range []string{
|
for _, gone := range []string{
|
||||||
"/inbound/log", "/inbound/queue", "/inbound/quarantine", "/inbound/filter-lists", "/server/preflight",
|
"/inbound/queue", "/inbound/quarantine", "/server/preflight",
|
||||||
`href="/status"`, `href="/domains"`, `href="/users"`, `href="/settings"`,
|
`href="/status"`, `href="/domains"`, `href="/users"`, `href="/settings"`,
|
||||||
} {
|
} {
|
||||||
mustNotContain(t, out, gone)
|
mustNotContain(t, out, gone)
|
||||||
|
|||||||
@@ -113,7 +113,7 @@ func NewDMARCHub(m Meta, in IngestInput) *DMARCHub {
|
|||||||
last,
|
last,
|
||||||
{Label: "This week", Value: Plain(FormatCount(int64(in.KeptThisWeek)) + " kept · " + plural(int64(in.ParseFailures), "parse failure", "parse failures"))},
|
{Label: "This week", Value: Plain(FormatCount(int64(in.KeptThisWeek)) + " kept · " + plural(int64(in.ParseFailures), "parse failure", "parse failures"))},
|
||||||
{Label: "Hosted addresses", Value: Plain(in.Hosted), Mono: true, Small: true},
|
{Label: "Hosted addresses", Value: Plain(in.Hosted), Mono: true, Small: true},
|
||||||
{Label: "Retention", Value: Plain(FormatCount(int64(in.RetentionMax)) + " reports or " + plural(int64(in.RetentionDays), "day", "days"))},
|
{Label: "Retention", Value: Plain(FormatCount(int64(in.RetentionMax)) + " reports a domain or " + plural(int64(in.RetentionDays), "day", "days"))},
|
||||||
},
|
},
|
||||||
Recent: Box{No: "02", Title: "Recent reports"},
|
Recent: Box{No: "02", Title: "Recent reports"},
|
||||||
Empty: EmptyState{Icon: "ti-report-analytics", Text: Plain("No reports yet.")},
|
Empty: EmptyState{Icon: "ti-report-analytics", Text: Plain("No reports yet.")},
|
||||||
|
|||||||
@@ -12,7 +12,7 @@ func TestDMARCHubShowsIngestAndTheLatestReports(t *testing.T) {
|
|||||||
pageHas(t, "DMARC reports", out,
|
pageHas(t, "DMARC reports", out,
|
||||||
`<h1 class="title is-3">DMARC reports</h1>`, `<code>rua=</code>`,
|
`<h1 class="title is-3">DMARC reports</h1>`, `<code>rua=</code>`,
|
||||||
`<h2>Ingest</h2>`, `<span class="tag is-success is-light">ok</span>`,
|
`<h2>Ingest</h2>`, `<span class="tag is-success is-light">ok</span>`,
|
||||||
`<dd>2026-09-21 06:12 UTC</dd>`, `<dd>14 kept · 0 parse failures</dd>`, `<dt>Hosted addresses</dt>`, `<dd class="sp-mono sp-small">dmarc-reports+<domain>@mail.example.org</dd>`, `<dd>500 reports or 90 days</dd>`,
|
`<dd>2026-09-21 06:12 UTC</dd>`, `<dd>14 kept · 0 parse failures</dd>`, `<dt>Hosted addresses</dt>`, `<dd class="sp-mono sp-small">dmarc-reports+<domain>@mail.example.org</dd>`, `<dd>500 reports a domain or 90 days</dd>`,
|
||||||
`<th>Received, UTC</th>`, `09-21 06:12`, `<a href="/outbound/dmarc/domains/1">example.org</a>`,
|
`<th>Received, UTC</th>`, `09-21 06:12`, `<a href="/outbound/dmarc/domains/1">example.org</a>`,
|
||||||
"<td>1 903</td>", `<span class="tag is-warning is-light">7</span>`, `<td>0</td><td class="sp-actions"><a href="/outbound/dmarc/reports/41">View</a>`,
|
"<td>1 903</td>", `<span class="tag is-warning is-light">7</span>`, `<td>0</td><td class="sp-actions"><a href="/outbound/dmarc/reports/41">View</a>`,
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -38,6 +38,8 @@ var pageFixtures = map[string]func() any{
|
|||||||
"in-domains": func() any { return inDomainsFixture() },
|
"in-domains": func() any { return inDomainsFixture() },
|
||||||
"in-domain": func() any { return inDomainFixture() },
|
"in-domain": func() any { return inDomainFixture() },
|
||||||
"in-domain-delete": func() any { return inDomainDeleteFixture() },
|
"in-domain-delete": func() any { return inDomainDeleteFixture() },
|
||||||
|
"in-log": func() any { return inLogFixture() },
|
||||||
|
"in-filter-lists": func() any { return inFilterListsFixture() },
|
||||||
|
|
||||||
"out-log": func() any { return outLogFixture() },
|
"out-log": func() any { return outLogFixture() },
|
||||||
"out-message": func() any { return outMessageFixture() },
|
"out-message": func() any { return outMessageFixture() },
|
||||||
@@ -131,7 +133,7 @@ func accountFixture() *Account {
|
|||||||
// settingsFixture is the mockup's Settings: thirty days of log, a limit of 600
|
// settingsFixture is the mockup's Settings: thirty days of log, a limit of 600
|
||||||
// messages an hour.
|
// messages an hour.
|
||||||
func settingsFixture() *Settings {
|
func settingsFixture() *Settings {
|
||||||
return NewSettings(admin(), "30", FormatRate(600, 3600))
|
return NewSettings(admin(), "30", FormatRate(600, 3600)).WithInbound("90")
|
||||||
}
|
}
|
||||||
|
|
||||||
// outDomainsFixture is the mockup's Outbound domains: three domains, the first
|
// outDomainsFixture is the mockup's Outbound domains: three domains, the first
|
||||||
@@ -411,3 +413,50 @@ func userFixture() *UserForm {
|
|||||||
func userDeleteFixture() *UserDelete {
|
func userDeleteFixture() *UserDelete {
|
||||||
return NewUserDelete(admin(), UserDeleteInput{ID: 3, Username: "acme-ops", AllOutbound: true, Inbound: []string{"acme.io"}})
|
return NewUserDelete(admin(), UserDeleteInput{ID: 3, Username: "acme-ops", AllOutbound: true, Inbound: []string{"acme.io"}})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// inLogFixture is the mockup's Inbound log: six rows of a week's mail, as the
|
||||||
|
// global role sees them - an accepted message scored as spam and one let through
|
||||||
|
// by an Allow entry, refusals by Postfix and by the filter, and a tempfail.
|
||||||
|
func inLogFixture() *InLog {
|
||||||
|
p := NewInLog(admin(), 90, []string{"lists.example.org", "acme.io"},
|
||||||
|
InLogFilter{From: "2026-09-14", To: "2026-09-21"})
|
||||||
|
var id int64
|
||||||
|
row := func(clock, from, client, to, subject, decision, engine, detail string) InLogRow {
|
||||||
|
at, err := time.Parse("2006-01-02 15:04:05", "2026-09-21 "+clock)
|
||||||
|
if err != nil {
|
||||||
|
panic(err)
|
||||||
|
}
|
||||||
|
id++
|
||||||
|
return p.NewRow(InLogRowInput{ID: id, Listable: true, At: at, From: from, Client: client, To: to, Subject: subject,
|
||||||
|
Decision: decision, Engine: engine, Detail: detail})
|
||||||
|
}
|
||||||
|
return p.WithRows([]InLogRow{
|
||||||
|
row("14:00:18", "t.berg@uni.example", "192.0.2.77", "dev@lists.example.org", "Re: [dev] patch series v3",
|
||||||
|
"accept", "rspamd", "no action, score 0.40 / 15.00"),
|
||||||
|
row("13:59:02", "invoice@partner-billing.example", "203.0.113.140", "owner@lists.example.org", "Outstanding invoice — action required",
|
||||||
|
"accept", "rspamd", "add header, score 9.80 / 15.00; PHISHING(5.00) FORGED_SENDER(3.00) R_SPF_FAIL(1.00)"),
|
||||||
|
row("13:57:41", "promo@bulk.example", "198.51.100.90", "sales@lists.example.org", "",
|
||||||
|
"reject", "postfix", "550 5.1.1 <sales@lists.example.org>: Recipient address rejected: User unknown in relay recipient table"),
|
||||||
|
row("13:51:30", "billing@partner.example", "198.51.100.12", "owner@lists.example.org", "",
|
||||||
|
"reject", "rspamd", "5.7.1 Spam message rejected"),
|
||||||
|
row("13:44:02", "hr@partner.example", "192.0.2.31", "jobs@acme.io", "",
|
||||||
|
"tempfail", "rspamd", "4.7.1 Try again later"),
|
||||||
|
row("13:30:55", "noreply@github.example", "192.0.2.200", "security@lists.example.org", "[advisory] new report",
|
||||||
|
"accept", "rspamd", "no action, score -98.10 / 15.00; SELFPOST_ALLOW_FROM(-100.00) R_DKIM_ALLOW(-0.20)"),
|
||||||
|
}, 1, 38)
|
||||||
|
}
|
||||||
|
|
||||||
|
// inFilterListsFixture is the mockup's Filter lists: three senders denied, two
|
||||||
|
// allowed, and a filter that reads them.
|
||||||
|
func inFilterListsFixture() *FilterLists {
|
||||||
|
day := func(m time.Month, d int) time.Time { return time.Date(2026, m, d, 0, 0, 0, 0, time.UTC) }
|
||||||
|
at := func(d, h, m int) time.Time { return time.Date(2026, 9, d, h, m, 0, 0, time.UTC) }
|
||||||
|
return NewFilterLists(admin(), FilterListsInput{FilterOn: true, LastWritten: time.Date(2026, 9, 11, 8, 40, 0, 0, time.UTC), RetentionDays: 90}).
|
||||||
|
WithEntries([]FilterEntryInput{
|
||||||
|
{ID: 1, List: "deny", Kind: "domain", Value: "bulk.example", Note: "newsletter nobody subscribed to", Added: day(9, 2), Hits: 41, LastHit: at(21, 13, 57)},
|
||||||
|
{ID: 2, List: "deny", Kind: "ip", Value: "198.51.100.0/24", Note: "dictionary attack on lists.example.org", Added: day(8, 19), Hits: 1203, LastHit: at(21, 9, 12)},
|
||||||
|
{ID: 3, List: "deny", Kind: "address", Value: "win@lottery.example", Added: day(7, 30)},
|
||||||
|
{ID: 4, List: "allow", Kind: "domain", Value: "partner.example", Note: "invoices were scored as spam", Added: day(9, 11), Hits: 7, LastHit: at(19, 16, 40)},
|
||||||
|
{ID: 5, List: "allow", Kind: "address", Value: "noreply@github.example", Note: "security advisories", Added: day(6, 4), Hits: 18, LastHit: at(21, 13, 30)},
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|||||||
@@ -46,8 +46,6 @@ const mockupDir = "../../../docs/assets/panel-redesign/panel"
|
|||||||
// with its own plan (panel-redesign.md § Room for roadmap items). They are the
|
// with its own plan (panel-redesign.md § Room for roadmap items). They are the
|
||||||
// only keys of outlines.json the finished panel may lack.
|
// only keys of outlines.json the finished panel may lack.
|
||||||
var deferredScreens = map[string]string{
|
var deferredScreens = map[string]string{
|
||||||
"in-log": "inbound-antispam-panel, 2.1.0",
|
|
||||||
"in-filter-lists": "inbound-antispam-panel, 2.1.0",
|
|
||||||
"in-queue": "queue as a table / inbound queue, undecided",
|
"in-queue": "queue as a table / inbound queue, undecided",
|
||||||
"in-quarantine": "inbound-quarantine, candidate",
|
"in-quarantine": "inbound-quarantine, candidate",
|
||||||
"in-quarantine-message": "inbound-quarantine, candidate",
|
"in-quarantine-message": "inbound-quarantine, candidate",
|
||||||
|
|||||||
@@ -52,6 +52,9 @@ const (
|
|||||||
HelpUpstream = "upstream"
|
HelpUpstream = "upstream"
|
||||||
HelpRecipients = "recipients"
|
HelpRecipients = "recipients"
|
||||||
HelpFilter = "filter"
|
HelpFilter = "filter"
|
||||||
|
// HelpInboundLog and HelpFilterLists: the plain "log" is the Outbound log's.
|
||||||
|
HelpInboundLog = "inbound-log"
|
||||||
|
HelpFilterLists = "filter-lists"
|
||||||
// Server
|
// Server
|
||||||
HelpBackup = "backup"
|
HelpBackup = "backup"
|
||||||
HelpUsers = "users"
|
HelpUsers = "users"
|
||||||
@@ -109,6 +112,8 @@ func NewHelp(m Meta, inbound bool) *Help {
|
|||||||
{Label: "Upstream", Icon: "ti-arrow-forward-up", Href: "#" + HelpUpstream},
|
{Label: "Upstream", Icon: "ti-arrow-forward-up", Href: "#" + HelpUpstream},
|
||||||
{Label: "Valid recipients", Icon: "ti-address-book", Href: "#" + HelpRecipients},
|
{Label: "Valid recipients", Icon: "ti-address-book", Href: "#" + HelpRecipients},
|
||||||
{Label: "Spam filter", Icon: "ti-shield-check", Href: "#" + HelpFilter},
|
{Label: "Spam filter", Icon: "ti-shield-check", Href: "#" + HelpFilter},
|
||||||
|
{Label: "Inbound log", Icon: "ti-list-details", Href: "#" + HelpInboundLog},
|
||||||
|
{Label: "Filter lists", Icon: "ti-filter", Href: "#" + HelpFilterLists},
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
if m.IsGlobal {
|
if m.IsGlobal {
|
||||||
@@ -184,13 +189,35 @@ var inboundTopics = []HelpTopic{
|
|||||||
Rich("The spam filter belongs to the server, not to a domain. The server's administrator attaches it in the server's ", Code(".env"),
|
Rich("The spam filter belongs to the server, not to a domain. The server's administrator attaches it in the server's ", Code(".env"),
|
||||||
", and from then on every inbound domain is screened by it before mail is handed on; the box shows on or off and has nothing to switch. Off means mail goes to the upstream unfiltered. While the filter is down, mail is either accepted unfiltered (the default) or deferred, so the sending server tries again later, as the server's setting says; the box states which."),
|
", and from then on every inbound domain is screened by it before mail is handed on; the box shows on or off and has nothing to switch. Off means mail goes to the upstream unfiltered. While the filter is down, mail is either accepted unfiltered (the default) or deferred, so the sending server tries again later, as the server's setting says; the box states which."),
|
||||||
}},
|
}},
|
||||||
|
{ID: HelpInboundLog, Title: "Inbound log", Body: []Text{
|
||||||
|
Rich("One row per recipient of a message offered on port 25 for an inbound domain, with what was decided: ",
|
||||||
|
TagOf("accept", ""), " — handed on to the upstream; ", TagOf("reject", ""), " — refused for good; ",
|
||||||
|
TagOf("tempfail", ""), " — the sender was asked to come back later; ", TagOf("quarantine", ""), " — held by the filter. ",
|
||||||
|
Em("Why"), " names who decided. ", Code("rspamd"), " is the spam filter, with its action, the score against the reject threshold and the heaviest symbols; ",
|
||||||
|
Code("postfix"), " is the relay's own rule — an unknown recipient, a sender domain that does not exist — with its reply; ",
|
||||||
|
Code("selfpost"), " means nobody judged the message: it was relayed with no filter attached. Accept says what this server did, not what the upstream did with the message afterwards.",
|
||||||
|
Br(), Br(),
|
||||||
|
"Accepted mail is recorded as it passes; refused mail is read from ", Code("mail.log"),
|
||||||
|
", so its row has no subject, and a message refused after its text was sent is one row however many recipients it had. A refusal to relay for a domain that is not an inbound one is every scanner on the internet and is not recorded. Rows are kept for the number of days set in Settings, and never more than 10 000."),
|
||||||
|
}},
|
||||||
|
{ID: HelpFilterLists, Title: "Filter lists", Body: []Text{
|
||||||
|
Rich("Two lists for the whole server that correct the spam filter by hand. An entry is a sender address, a sender domain — that domain, not its subdomains — or a client address or network in CIDR form. A sender on the Deny list is refused outright. A sender on the Allow list is given a weight no spam score outweighs, so the filter lets it through. The sender's address is whatever the sending server says it is - anyone can claim to be a sender you allow - so allow a client address or network where you can, and a sender address only for mail you would rather receive forged than lose. When entries of both lists match one message, ",
|
||||||
|
Strong("Deny wins"), ".",
|
||||||
|
Br(), Br(),
|
||||||
|
"SelfPost does not apply the lists itself: it writes them as files under ", Code("/data/antispam/"),
|
||||||
|
", and rspamd rereads a file within seconds of a change. With the spam filter off the lists are kept and can be edited, but nothing reads them. The sender matched is the envelope sender — the one in the log — not the From line a mail program shows. Matched counts the messages in the Inbound log that the entry decided, so it reaches back as far as the log does; never means not once in that time, and such an entry is one to consider removing."),
|
||||||
|
}},
|
||||||
}
|
}
|
||||||
|
|
||||||
var serverTopics = []HelpTopic{
|
var serverTopics = []HelpTopic{
|
||||||
{ID: HelpBackup, Title: "Backup and moving a domain", Body: []Text{
|
{ID: HelpBackup, Title: "Backup and moving a domain", Body: []Text{
|
||||||
Rich("Full backup downloads one archive of the whole instance: ", Code("data/"), " (the database, DKIM keys, application credentials and the queue), ",
|
Rich("Full backup downloads one archive of the whole instance: ", Code("data/"), " (the database, DKIM keys and application credentials), ",
|
||||||
Code("docker-compose.yml"), ", ", Code(".env"), " and ", Code("certs/"),
|
Code("docker-compose.yml"), ", ", Code(".env"), " and ", Code("certs/"),
|
||||||
". It holds secrets, so leave Encrypt with a password ticked: the file is sealed as ", Code(".spbk"),
|
". The Postfix queue is not in it. The backup is taken while the server is running, and Postfix keeps moving messages through the queue, "+
|
||||||
|
"so a copy could hold a message twice or miss it; mail still waiting for delivery is therefore not carried over, and a restored server "+
|
||||||
|
"starts with an empty queue. To keep the queue, stop the container and archive the project directory instead. The private key in ", Code("certs/"),
|
||||||
|
" has to be readable by the panel: give it to group 998 on the host (", Code("chgrp 998"), ", ", Code("chmod 640"),
|
||||||
|
"). Until then the panel refuses to make a backup and names the file, rather than hand over an archive without the key. It holds secrets, so leave Encrypt with a password ticked: the file is sealed as ", Code(".spbk"),
|
||||||
", the password is not stored, and without it nothing can be recovered. To restore, unpack it into an empty project directory (decrypt a ",
|
", the password is not stored, and without it nothing can be recovered. To restore, unpack it into an empty project directory (decrypt a ",
|
||||||
Code(".spbk"), " first with ", Code("selfpost-backup -decrypt"),
|
Code(".spbk"), " first with ", Code("selfpost-backup -decrypt"),
|
||||||
") and start the same SelfPost version that made it; the panel refuses data from another version. The reverse-proxy configuration and ",
|
") and start the same SelfPost version that made it; the panel refuses data from another version. The reverse-proxy configuration and ",
|
||||||
@@ -204,7 +231,7 @@ var serverTopics = []HelpTopic{
|
|||||||
}},
|
}},
|
||||||
{ID: HelpSettings, Title: "Settings", Body: []Text{
|
{ID: HelpSettings, Title: "Settings", Body: []Text{
|
||||||
Rich("Settings holds what is true of the whole instance. Log retention is how many days the Outbound log keeps its rows, from 7 to 365 and 90 by default; a sweep every six hours deletes older ones, and it is the main driver of how fast ",
|
Rich("Settings holds what is true of the whole instance. Log retention is how many days the Outbound log keeps its rows, from 7 to 365 and 90 by default; a sweep every six hours deletes older ones, and it is the main driver of how fast ",
|
||||||
Code("/data"), " grows. It does not change the rotation of ", Code("mail.log"),
|
Code("/data"), " grows. With the inbound relay on there is a second number, for the Inbound log, with the same range. It does not change the rotation of ", Code("mail.log"),
|
||||||
", which is daily with about two weeks kept. The sending rate limit shown under it is level 1, the per-client-IP ceiling set in ",
|
", which is daily with about two weeks kept. The sending rate limit shown under it is level 1, the per-client-IP ceiling set in ",
|
||||||
Code(".env"), ": changing it needs a restart, and no level-2 limit can exceed it."),
|
Code(".env"), ": changing it needs a restart, and no level-2 limit can exceed it."),
|
||||||
}},
|
}},
|
||||||
|
|||||||
@@ -164,3 +164,30 @@ func TestHelpLinksStayInsideWhatTheirViewerIsShown(t *testing.T) {
|
|||||||
landsOn("DMARC hub", hub.Recent.Help)
|
landsOn("DMARC hub", hub.Recent.Help)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// The Inbound log and Filter lists have their topics, after the spam filter's, in
|
||||||
|
// the menu and on the page, for everyone who is shown Inbound; the Settings topic
|
||||||
|
// mentions the second retention.
|
||||||
|
func TestHelpHasTheInboundLogAndFilterListsTopics(t *testing.T) {
|
||||||
|
out := renderSignedIn(t, "help", NewHelp(admin(), true))
|
||||||
|
pageHas(t, "Help", out,
|
||||||
|
`<li><a href="#inbound-log"><i class="ti ti-list-details"></i>Inbound log</a></li>`,
|
||||||
|
`<li><a href="#filter-lists"><i class="ti ti-filter"></i>Filter lists</a></li>`,
|
||||||
|
`<div class="sp-help-topic" id="inbound-log"><h3>Inbound log</h3>`,
|
||||||
|
`<span class="tag is-primary is-light">quarantine</span> — held by the filter. <em>Why</em> names who decided.`,
|
||||||
|
`a message refused after its text was sent is one row however many recipients it had.`,
|
||||||
|
`and never more than 10 000.`,
|
||||||
|
`<div class="sp-help-topic" id="filter-lists"><h3>Filter lists</h3>`,
|
||||||
|
`<strong>Deny wins</strong>.`, `files under <code>/data/antispam/</code>`,
|
||||||
|
`With the inbound relay on there is a second number, for the Inbound log, with the same range.`)
|
||||||
|
for _, v := range helpViewers {
|
||||||
|
out := renderSignedIn(t, "help", NewHelp(v.meta, v.inbound))
|
||||||
|
shown := strings.Contains(out, `id="inbound-log"`)
|
||||||
|
if want := v.inbound && (v.meta.IsGlobal || v.meta.HasInbound); shown != want {
|
||||||
|
t.Errorf("%s: inbound log topic shown = %v, want %v", v.name, shown, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if HelpInboundLog != "inbound-log" || HelpFilterLists != "filter-lists" {
|
||||||
|
t.Errorf("anchors %q and %q", HelpInboundLog, HelpFilterLists)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -19,7 +19,7 @@ func TestInDomainShowsTheMockupsBoxes(t *testing.T) {
|
|||||||
`<h2>Spam filter</h2>`, `href="#filter"`, `href="/inbound/domains/1/delete"`, `Delete domain`,
|
`<h2>Spam filter</h2>`, `href="#filter"`, `href="/inbound/domains/1/delete"`, `Delete domain`,
|
||||||
)
|
)
|
||||||
// What has no source in this build is not drawn.
|
// What has no source in this build is not drawn.
|
||||||
pageLacks(t, "Inbound domain", out, `rspamd`, `quarantine`, `Quarantine`, `Log for this domain`, `/inbound/log`, `/inbound/quarantine`, `Allow and deny`)
|
pageLacks(t, "Inbound domain", out, `rspamd`, `quarantine`, `Quarantine`, `Log for this domain`, `/inbound/quarantine`, `Allow and deny`)
|
||||||
}
|
}
|
||||||
|
|
||||||
// The Spam filter box states what the server is set up with, and what of it the
|
// The Spam filter box states what the server is set up with, and what of it the
|
||||||
|
|||||||
@@ -0,0 +1,202 @@
|
|||||||
|
package view
|
||||||
|
|
||||||
|
// The data of Inbound › Filter lists (templates/in_filter_lists.html, mockup
|
||||||
|
// in-filter-lists.html): the allow and deny lists that correct the external spam
|
||||||
|
// filter by hand. The lists are instance-wide, so the page is the global role's.
|
||||||
|
//
|
||||||
|
// A page is a typed struct that embeds Meta, built by a constructor and filled
|
||||||
|
// by With… methods; the handler reads the lists, the view decides what is said
|
||||||
|
// and how it is worded.
|
||||||
|
|
||||||
|
import (
|
||||||
|
"net/url"
|
||||||
|
"strconv"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
const inboundFilterLists = "/inbound/filter-lists"
|
||||||
|
|
||||||
|
// FilterListsHref is the address of the page with the add form filled in: list
|
||||||
|
// is "allow" or "deny", kind "address", "domain" or "ip". The Inbound log links
|
||||||
|
// its rows here.
|
||||||
|
func FilterListsHref(list, kind, value string) string {
|
||||||
|
return inboundFilterLists + "?list=" + url.QueryEscape(list) + "&kind=" + url.QueryEscape(kind) + "&value=" + url.QueryEscape(value)
|
||||||
|
}
|
||||||
|
|
||||||
|
// FilterLists is the data of the page: the form that adds an entry, the two
|
||||||
|
// lists side by side and what happens to them after that.
|
||||||
|
type FilterLists struct {
|
||||||
|
Meta
|
||||||
|
Head Head
|
||||||
|
Flash *Flash
|
||||||
|
|
||||||
|
Add Box
|
||||||
|
Action string
|
||||||
|
ListOptions []Option
|
||||||
|
KindOptions []Option
|
||||||
|
// List, Kind, Value and Note are what the form shows: blank, prefilled from
|
||||||
|
// the Inbound log, or what was typed into a form that was refused.
|
||||||
|
List string
|
||||||
|
Kind string
|
||||||
|
Value string
|
||||||
|
Note string
|
||||||
|
|
||||||
|
Deny FilterList
|
||||||
|
Allow FilterList
|
||||||
|
|
||||||
|
Where Box
|
||||||
|
WhereFacts []Fact
|
||||||
|
|
||||||
|
retentionDays int
|
||||||
|
}
|
||||||
|
|
||||||
|
// FilterList is one of the two boxes of entries.
|
||||||
|
type FilterList struct {
|
||||||
|
Box Box
|
||||||
|
Rows []FilterRow
|
||||||
|
Empty EmptyState
|
||||||
|
}
|
||||||
|
|
||||||
|
// FilterRow is one entry of a list: the value, the kind of entry as a tag that
|
||||||
|
// opens the entry's sheet (the note, the day it was added, how often it decided
|
||||||
|
// a message and when last), how often it decided, and the way to remove it.
|
||||||
|
// Matched is empty for an entry that never did; the table then says "never".
|
||||||
|
type FilterRow struct {
|
||||||
|
Value string
|
||||||
|
Entry Sheet
|
||||||
|
Matched string
|
||||||
|
Remove RowAction
|
||||||
|
}
|
||||||
|
|
||||||
|
// FilterEntryInput is what the page knows of one entry of a list. Hits and
|
||||||
|
// LastHit are counted from the Inbound log, so they reach back as far as it
|
||||||
|
// does.
|
||||||
|
type FilterEntryInput struct {
|
||||||
|
ID int64
|
||||||
|
List string
|
||||||
|
Kind string
|
||||||
|
Value string
|
||||||
|
Note string
|
||||||
|
Added time.Time
|
||||||
|
Hits int
|
||||||
|
LastHit time.Time
|
||||||
|
}
|
||||||
|
|
||||||
|
// FilterListsInput is the state of the filter the lists are written for: whether
|
||||||
|
// a filter is attached at all, and when the map files it reads were last written.
|
||||||
|
type FilterListsInput struct {
|
||||||
|
FilterOn bool
|
||||||
|
LastWritten time.Time
|
||||||
|
// RetentionDays is how long the Inbound log keeps a row: how far back an
|
||||||
|
// entry's matches are counted.
|
||||||
|
RetentionDays int
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewFilterLists builds the page. The form starts on the Deny list for a sender
|
||||||
|
// address, the commonest correction.
|
||||||
|
func NewFilterLists(m Meta, in FilterListsInput) *FilterLists {
|
||||||
|
m.Title, m.Section, m.Page = "Filter lists", "inbound", "filter-lists"
|
||||||
|
p := &FilterLists{
|
||||||
|
Meta: m,
|
||||||
|
Head: Head{
|
||||||
|
Kicker: "Inbound · global role",
|
||||||
|
Title: "Filter lists",
|
||||||
|
Lead: Plain("Correct the spam filter by hand: senders it must always let through, and senders it must always turn away. One pair of lists for the whole instance."),
|
||||||
|
},
|
||||||
|
Add: Box{No: "01", Title: "Add an entry", Help: topicLink(HelpFilterLists, "How the lists are applied")},
|
||||||
|
Action: inboundFilterLists,
|
||||||
|
ListOptions: []Option{
|
||||||
|
{Value: "deny", Label: "Deny — always turn away"},
|
||||||
|
{Value: "allow", Label: "Allow — always let through"},
|
||||||
|
},
|
||||||
|
KindOptions: []Option{
|
||||||
|
{Value: "address", Label: "a sender address"},
|
||||||
|
{Value: "domain", Label: "a sender domain"},
|
||||||
|
{Value: "ip", Label: "a client address or network"},
|
||||||
|
},
|
||||||
|
List: "deny",
|
||||||
|
Kind: "address",
|
||||||
|
|
||||||
|
Deny: FilterList{Box: Box{No: "02", Title: "Deny"}, Empty: EmptyState{Icon: "ti-filter", Text: Plain("No sender is turned away by hand.")}},
|
||||||
|
Allow: FilterList{Box: Box{No: "03", Title: "Allow"}, Empty: EmptyState{Icon: "ti-filter", Text: Plain("No sender is let through by hand.")}},
|
||||||
|
Where: Box{No: "04", Title: "Where the lists go"},
|
||||||
|
}
|
||||||
|
p.Deny.Box.End = Plain(plural(0, "entry", "entries") + " · wins over Allow")
|
||||||
|
p.Allow.Box.End = Plain(plural(0, "entry", "entries"))
|
||||||
|
p.retentionDays = in.RetentionDays
|
||||||
|
p.setWhere(in)
|
||||||
|
return p
|
||||||
|
}
|
||||||
|
|
||||||
|
// setWhere writes box 04: the tag that says whether anything reads the lists,
|
||||||
|
// and the four facts about where they go. With no filter attached the lists are
|
||||||
|
// still kept and editable, and the box says so instead of promising an effect.
|
||||||
|
func (p *FilterLists) setWhere(in FilterListsInput) {
|
||||||
|
applied := Plain("rspamd, from map files it rereads within seconds of a change")
|
||||||
|
p.Where.End = Rich(TagOf("ok", "applied"))
|
||||||
|
if !in.FilterOn {
|
||||||
|
p.Where.End = Rich(TagOf("off", "not applied"))
|
||||||
|
applied = Rich("Nothing — the spam filter is off. The lists are kept and can be edited; nothing reads them until ",
|
||||||
|
Code("INBOUND_ANTISPAM_MILTER"), " is set.")
|
||||||
|
}
|
||||||
|
written := "never"
|
||||||
|
if !in.LastWritten.IsZero() {
|
||||||
|
written = in.LastWritten.UTC().Format("2006-01-02 15:04") + " UTC"
|
||||||
|
}
|
||||||
|
p.WhereFacts = []Fact{
|
||||||
|
{Label: "Applied by", Value: applied},
|
||||||
|
{Label: "Files", Value: Rich(Mono("/data/antispam/"), " — six, one for each list and kind of entry")},
|
||||||
|
{Label: "Last written", Value: Plain(written)},
|
||||||
|
{Label: "When both lists match", Value: Plain("Deny wins, whichever entries they are.")},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// WithEntries sets the two tables, each counted in its box head.
|
||||||
|
func (p *FilterLists) WithEntries(entries []FilterEntryInput) *FilterLists {
|
||||||
|
p.Deny.Rows, p.Allow.Rows = nil, nil
|
||||||
|
for _, e := range entries {
|
||||||
|
id := strconv.FormatInt(e.ID, 10)
|
||||||
|
window := "in the last " + plural(int64(p.retentionDays), "day", "days")
|
||||||
|
sheet := []SheetRow{
|
||||||
|
{Label: "Note", Lines: []string{orDash(e.Note)}},
|
||||||
|
{Label: "Added", Lines: []string{e.Added.UTC().Format("2006-01-02")}, Mono: true},
|
||||||
|
}
|
||||||
|
row := FilterRow{
|
||||||
|
Value: e.Value,
|
||||||
|
Remove: RowAction{Label: "Remove", Post: inboundFilterLists + "/" + id + "/delete", Danger: true},
|
||||||
|
}
|
||||||
|
if e.Hits > 0 {
|
||||||
|
row.Matched = strconv.Itoa(e.Hits)
|
||||||
|
sheet = append(sheet,
|
||||||
|
SheetRow{Label: "Matched", Lines: []string{row.Matched + " " + window}},
|
||||||
|
SheetRow{Label: "Last match", Lines: []string{e.LastHit.UTC().Format("2006-01-02 15:04")}, Mono: true})
|
||||||
|
} else {
|
||||||
|
sheet = append(sheet, SheetRow{Label: "Matched", Lines: []string{"never " + window}})
|
||||||
|
}
|
||||||
|
row.Entry = Sheet{ID: "entry-" + id, Tag: Tag{Label: e.Kind}, Title: e.Value, Rows: sheet}
|
||||||
|
switch e.List {
|
||||||
|
case "deny":
|
||||||
|
p.Deny.Rows = append(p.Deny.Rows, row)
|
||||||
|
case "allow":
|
||||||
|
p.Allow.Rows = append(p.Allow.Rows, row)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
p.Deny.Box.End = Plain(plural(int64(len(p.Deny.Rows)), "entry", "entries") + " · wins over Allow")
|
||||||
|
p.Allow.Box.End = Plain(plural(int64(len(p.Allow.Rows)), "entry", "entries"))
|
||||||
|
return p
|
||||||
|
}
|
||||||
|
|
||||||
|
// WithForm sets what the add form shows. The handler passes only what it has
|
||||||
|
// checked to be a list and a kind the form offers, or what was submitted to a
|
||||||
|
// form it is showing again.
|
||||||
|
func (p *FilterLists) WithForm(list, kind, value, note string) *FilterLists {
|
||||||
|
p.List, p.Kind, p.Value, p.Note = list, kind, value, note
|
||||||
|
return p
|
||||||
|
}
|
||||||
|
|
||||||
|
// WithResult shows the result of the last action, or the refusal of the add
|
||||||
|
// form, between the head and the first box.
|
||||||
|
func (p *FilterLists) WithResult(flash, formErr string) *FilterLists {
|
||||||
|
p.Flash = resultFlash(flash, formErr)
|
||||||
|
return p
|
||||||
|
}
|
||||||
@@ -0,0 +1,234 @@
|
|||||||
|
package view
|
||||||
|
|
||||||
|
// The data of the Inbound log (templates/in_log.html, mockup in-log.html): what
|
||||||
|
// port 25 decided about each message offered for an inbound domain, one row per
|
||||||
|
// recipient. Unlike the Outbound log it does not poll: a decision does not change
|
||||||
|
// after it is made, and the page is read to find one row, not watched.
|
||||||
|
//
|
||||||
|
// A page is a typed struct that embeds Meta, built by a constructor and filled
|
||||||
|
// by With… methods; the handler reads the journal, the view decides what is
|
||||||
|
// said and how it is worded.
|
||||||
|
|
||||||
|
import (
|
||||||
|
"net/url"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
const inboundLog = "/inbound/log"
|
||||||
|
|
||||||
|
// InLogFilter is what the filters of the Inbound log are set to, as the strings
|
||||||
|
// the form sends: a domain name, a decision and two days (2006-01-02). Empty
|
||||||
|
// means no narrowing.
|
||||||
|
type InLogFilter struct {
|
||||||
|
Domain string
|
||||||
|
Decision string
|
||||||
|
From string
|
||||||
|
To string
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f InLogFilter) active() bool { return f != InLogFilter{} }
|
||||||
|
|
||||||
|
// query is the query string of a page of the log: the filters and the page, each
|
||||||
|
// left out when it says nothing, so the first unfiltered page is the bare path.
|
||||||
|
func (f InLogFilter) query(page int) string {
|
||||||
|
q := url.Values{}
|
||||||
|
for name, v := range map[string]string{"domain": f.Domain, "decision": f.Decision, "from": f.From, "to": f.To} {
|
||||||
|
if v != "" {
|
||||||
|
q.Set(name, v)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if page > 1 {
|
||||||
|
q.Set("p", strconv.Itoa(page))
|
||||||
|
}
|
||||||
|
if len(q) == 0 {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
return "?" + q.Encode()
|
||||||
|
}
|
||||||
|
|
||||||
|
// InLogHref is the address of a page of the Inbound log.
|
||||||
|
func InLogHref(f InLogFilter, page int) string { return inboundLog + f.query(page) }
|
||||||
|
|
||||||
|
// InLog is the data of the Inbound log: the decisions about mail for the inbound
|
||||||
|
// domains the viewer reaches, filtered by domain, decision and day, a page at a
|
||||||
|
// time.
|
||||||
|
type InLog struct {
|
||||||
|
Meta
|
||||||
|
Head Head
|
||||||
|
|
||||||
|
Messages Box
|
||||||
|
Rows []InLogRow
|
||||||
|
Empty EmptyState
|
||||||
|
Foot Foot
|
||||||
|
|
||||||
|
filter InLogFilter
|
||||||
|
}
|
||||||
|
|
||||||
|
// InLogRow is one recipient of one message: when it was decided, who the mail
|
||||||
|
// was from (and the address of the server that offered it) and for, its subject,
|
||||||
|
// the decision as a tag that opens the sheet saying who decided and why, and —
|
||||||
|
// for the global role, who alone edits the filter lists — the way to correct
|
||||||
|
// the decision.
|
||||||
|
type InLogRow struct {
|
||||||
|
Time string
|
||||||
|
From string
|
||||||
|
Client string
|
||||||
|
To string
|
||||||
|
Subject string
|
||||||
|
Why Sheet
|
||||||
|
Action *Anchor
|
||||||
|
}
|
||||||
|
|
||||||
|
// InLogRowInput is what the journal holds about one row. Subject, Detail and
|
||||||
|
// From may be empty: refused mail is read from mail.log, which has no subject,
|
||||||
|
// and a filter that said nothing has no detail.
|
||||||
|
type InLogRowInput struct {
|
||||||
|
ID int64
|
||||||
|
// Listable says the sender is an address the filter lists would take. A
|
||||||
|
// row whose sender they would refuse offers no way into them.
|
||||||
|
Listable bool
|
||||||
|
At time.Time
|
||||||
|
From string
|
||||||
|
Client string
|
||||||
|
To string
|
||||||
|
Subject string
|
||||||
|
Decision string
|
||||||
|
Engine string
|
||||||
|
Detail string
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewInLog builds the log for a viewer. domains are the choices of the domain
|
||||||
|
// filter (only the inbound domains the viewer reaches) and filter is what is in
|
||||||
|
// force. retentionDays is how long rows are kept; the global role, who can
|
||||||
|
// change it and who edits the lists, is read from m.
|
||||||
|
func NewInLog(m Meta, retentionDays int, domains []string, filter InLogFilter) *InLog {
|
||||||
|
m.Title, m.Section, m.Page = "Inbound log", "inbound", "log"
|
||||||
|
lead := Rich("What port 25 decided about each message, one row per recipient. Accepted mail is recorded as it passes, refused mail is read from ",
|
||||||
|
Code("mail.log"), "; the filter's verdict appears when the spam filter is on. Rows older than "+plural(int64(retentionDays), "day", "days")+" are deleted")
|
||||||
|
if m.IsGlobal {
|
||||||
|
lead = append(lead, Inline{Text: " — change that in "}, Link("/server/settings", "Settings"))
|
||||||
|
}
|
||||||
|
lead = append(lead, Inline{Text: "."})
|
||||||
|
|
||||||
|
domainOptions := []Option{{Value: "", Label: "All domains"}}
|
||||||
|
for _, d := range domains {
|
||||||
|
domainOptions = append(domainOptions, Option{Value: d, Label: d})
|
||||||
|
}
|
||||||
|
decisionOptions := []Option{{Value: "", Label: "Any decision"}}
|
||||||
|
for _, d := range []string{"accept", "reject", "tempfail", "quarantine"} {
|
||||||
|
decisionOptions = append(decisionOptions, Option{Value: d, Label: d})
|
||||||
|
}
|
||||||
|
p := &InLog{
|
||||||
|
Meta: m,
|
||||||
|
Head: Head{Kicker: "Inbound", Title: "Inbound log", Lead: lead},
|
||||||
|
Messages: Box{No: "01", Title: "Messages", Help: topicLink(HelpInboundLog, "What the decisions mean"), Filter: &Filter{
|
||||||
|
Action: inboundLog,
|
||||||
|
Selects: []Select{
|
||||||
|
{Name: "domain", Label: "Domain", Options: domainOptions, Selected: filter.Domain},
|
||||||
|
{Name: "decision", Label: "Decision", Options: decisionOptions, Selected: filter.Decision},
|
||||||
|
},
|
||||||
|
Dates: []DateInput{
|
||||||
|
{Name: "from", Label: "From date", Value: filter.From},
|
||||||
|
{Name: "to", Label: "To date", Value: filter.To},
|
||||||
|
},
|
||||||
|
}},
|
||||||
|
Empty: EmptyState{Icon: "ti-list-details", Text: Plain("No messages logged yet.")},
|
||||||
|
filter: filter,
|
||||||
|
}
|
||||||
|
if filter.active() {
|
||||||
|
p.Empty.Text = Plain("No messages match these filters.")
|
||||||
|
}
|
||||||
|
return p
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewRow words one row of the journal for the viewer of the page.
|
||||||
|
func (p *InLog) NewRow(in InLogRowInput) InLogRow {
|
||||||
|
r := InLogRow{
|
||||||
|
Time: FormatLogTime(in.At), From: orDash(in.From), Client: in.Client, To: in.To, Subject: in.Subject,
|
||||||
|
Why: Sheet{
|
||||||
|
ID: "why-" + strconv.FormatInt(in.ID, 10), Tag: Tag{Status: in.Decision}, Title: "Why",
|
||||||
|
Rows: whyRows(in.Decision, in.Engine, in.Detail, in.Subject),
|
||||||
|
},
|
||||||
|
}
|
||||||
|
if p.IsGlobal && in.Listable {
|
||||||
|
r.Action = senderAction(in.Decision, in.Engine, in.From)
|
||||||
|
}
|
||||||
|
return r
|
||||||
|
}
|
||||||
|
|
||||||
|
// whyRows is the content of a row's sheet: who decided, what they said, and
|
||||||
|
// the subject whole, which the table clips. What was said has one of two
|
||||||
|
// shapes, and the decision says which. Mail that passed was written by the
|
||||||
|
// inbound journal milter with the filter's verdict - an action and a score,
|
||||||
|
// then the symbols after a semicolon. Mail that was refused or put off was
|
||||||
|
// read from mail.log by the tailer, with the reply it was given.
|
||||||
|
func whyRows(decision, engine, detail, subject string) []SheetRow {
|
||||||
|
rows := []SheetRow{{Label: "Decided by", Lines: []string{engine}}}
|
||||||
|
switch {
|
||||||
|
case detail == "":
|
||||||
|
case decision == "reject" || decision == "tempfail":
|
||||||
|
rows = append(rows, SheetRow{Label: "Reply", Lines: []string{detail}, Mono: true})
|
||||||
|
default:
|
||||||
|
verdict, symbols, _ := strings.Cut(detail, "; ")
|
||||||
|
rows = append(rows, SheetRow{Label: "Verdict", Lines: []string{verdict}})
|
||||||
|
if symbols != "" {
|
||||||
|
rows = append(rows, SheetRow{Label: "Symbols", Lines: strings.Fields(symbols), Mono: true})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if subject != "" {
|
||||||
|
rows = append(rows, SheetRow{Label: "Subject", Lines: []string{subject}})
|
||||||
|
}
|
||||||
|
return rows
|
||||||
|
}
|
||||||
|
|
||||||
|
// senderAction is the way to correct a decision from its row, if there is one:
|
||||||
|
// a message that got through is the one to deny its sender for, and a message
|
||||||
|
// the filter turned away, put off or held is the one to allow its sender for.
|
||||||
|
// A refusal by Postfix's own policy (an unknown recipient, say) offers nothing:
|
||||||
|
// the message is refused already and the lists would not change what Postfix
|
||||||
|
// thinks of the recipient. A row with no sender has nobody to list.
|
||||||
|
func senderAction(decision, engine, from string) *Anchor {
|
||||||
|
if from == "" {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
list := ""
|
||||||
|
switch {
|
||||||
|
case decision == "accept":
|
||||||
|
list = "deny"
|
||||||
|
case engine == "rspamd":
|
||||||
|
list = "allow"
|
||||||
|
default:
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
label := "Deny sender"
|
||||||
|
if list == "allow" {
|
||||||
|
label = "Allow sender"
|
||||||
|
}
|
||||||
|
return &Anchor{Href: FilterListsHref(list, "address", from), Label: label}
|
||||||
|
}
|
||||||
|
|
||||||
|
// WithRows sets the table for page of last, and the foot under it: where the
|
||||||
|
// viewer is, and the way to the newer and older rows. For the global role the
|
||||||
|
// foot also says where the sender links lead.
|
||||||
|
func (p *InLog) WithRows(rows []InLogRow, page, last int) *InLog {
|
||||||
|
p.Rows = rows
|
||||||
|
if len(rows) == 0 {
|
||||||
|
return p
|
||||||
|
}
|
||||||
|
text := Rich("Page " + strconv.Itoa(page) + " of " + strconv.Itoa(last))
|
||||||
|
if page > 1 {
|
||||||
|
text = append(text, Inline{Text: " · "}, Link(InLogHref(p.filter, page-1), "← Newer"))
|
||||||
|
}
|
||||||
|
p.Foot.Text = text
|
||||||
|
if page < last {
|
||||||
|
p.Foot.Link = &Anchor{Href: InLogHref(p.filter, page+1), Label: "Older →"}
|
||||||
|
}
|
||||||
|
if p.IsGlobal {
|
||||||
|
p.Foot.End = Rich("A decision opens its reason · Allow / Deny sender opens ", Link(inboundFilterLists, "Filter lists"), " with the entry filled in")
|
||||||
|
} else {
|
||||||
|
p.Foot.End = Plain("A decision opens its reason")
|
||||||
|
}
|
||||||
|
return p
|
||||||
|
}
|
||||||
@@ -0,0 +1,310 @@
|
|||||||
|
package view
|
||||||
|
|
||||||
|
import (
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// ---- The log
|
||||||
|
|
||||||
|
// The log is the whole of what the journal holds about the viewer's inbound mail:
|
||||||
|
// the filters offer what the viewer reaches, the head says how long rows live and
|
||||||
|
// where to change it, and the global role is given the way to correct a decision.
|
||||||
|
func TestInLogShowsTheRowsAndTheirFilters(t *testing.T) {
|
||||||
|
out := renderSignedIn(t, "in-log", inLogFixture())
|
||||||
|
pageHas(t, "Inbound log", out,
|
||||||
|
`<h1 class="title is-3">Inbound log</h1>`,
|
||||||
|
`refused mail is read from <code>mail.log</code>; the filter's verdict appears when the spam filter is on. Rows older than 90 days are deleted — change that in <a href="/server/settings">Settings</a>.`,
|
||||||
|
`<form class="sp-end" method="get" action="/inbound/log">`,
|
||||||
|
`<select name="domain" aria-label="Domain"><option value="" selected>All domains</option><option value="lists.example.org">lists.example.org</option><option value="acme.io">acme.io</option></select>`,
|
||||||
|
`<select name="decision" aria-label="Decision"><option value="" selected>Any decision</option><option value="accept">accept</option><option value="reject">reject</option><option value="tempfail">tempfail</option><option value="quarantine">quarantine</option></select>`,
|
||||||
|
`<input class="input" type="date" name="from" value="2026-09-14" aria-label="From date">`,
|
||||||
|
`<input class="input" type="date" name="to" value="2026-09-21" aria-label="To date">`,
|
||||||
|
`href="/help#inbound-log" title="What the decisions mean"`,
|
||||||
|
`<th>Time, UTC</th><th>From</th><th>To</th><th>Subject</th><th>Decision</th><th></th></tr>`,
|
||||||
|
`09-21 14:00:18`,
|
||||||
|
`t.berg@<wbr>uni.example<br><span class="sp-mono sp-muted sp-small">192.0.2.77</span>`,
|
||||||
|
`dev@<wbr>lists.example.org`, `title="Re: [dev] patch series v3"`,
|
||||||
|
`<td class="sp-clip sp-muted">—</td>`,
|
||||||
|
// the decision is a tag that opens the sheet with who decided and why
|
||||||
|
`<button type="button" class="tag is-success is-light" popovertarget="why-1">accept</button><div class="sp-sheet" id="why-1" popover>`,
|
||||||
|
`<div class="sp-sheet-head"><span class="tag is-success is-light">accept</span><h2>Why</h2><button type="button" popovertarget="why-1" popovertargetaction="hide" aria-label="Close">`,
|
||||||
|
`<button type="button" class="tag is-danger is-light" popovertarget="why-3">reject</button>`,
|
||||||
|
`<button type="button" class="tag is-warning is-light" popovertarget="why-5">tempfail</button>`,
|
||||||
|
`<tr><th>Decided by</th><td>rspamd</td></tr><tr><th>Verdict</th><td>no action, score 0.40 / 15.00</td></tr><tr><th>Subject</th><td>Re: [dev] patch series v3</td></tr>`,
|
||||||
|
`<tr><th>Verdict</th><td>add header, score 9.80 / 15.00</td></tr><tr><th>Symbols</th><td class="sp-mono sp-small">PHISHING(5.00)<br>FORGED_SENDER(3.00)<br>R_SPF_FAIL(1.00)</td></tr>`,
|
||||||
|
`<tr><th>Decided by</th><td>postfix</td></tr><tr><th>Reply</th><td class="sp-mono sp-small">550 5.1.1 <sales@lists.example.org>: Recipient address rejected: User unknown in relay recipient table</td></tr></tbody>`,
|
||||||
|
`Page 1 of 38`, `<a href="/inbound/log?from=2026-09-14&p=2&to=2026-09-21">Older →</a>`,
|
||||||
|
`A decision opens its reason · Allow / Deny sender opens <a href="/inbound/filter-lists">Filter lists</a> with the entry filled in`,
|
||||||
|
)
|
||||||
|
pageLacks(t, "Inbound log", out, `← Newer`, `data-poll`, `hx-get`, `<th>Why</th>`)
|
||||||
|
// A message Postfix itself refused offers no way into the lists: one Deny
|
||||||
|
// for each of the three accepted messages, one Allow for each the filter
|
||||||
|
// refused or put off.
|
||||||
|
if deny, allow := strings.Count(out, `>Deny sender</a>`), strings.Count(out, `>Allow sender</a>`); deny != 3 || allow != 2 {
|
||||||
|
t.Errorf("the fixture's six rows offer Deny %d times and Allow %d times, want 3 and 2", deny, allow)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Without the global role there is nowhere to change the retention, and
|
||||||
|
// nowhere to send a sender.
|
||||||
|
domain := NewInLog(Meta{User: "ops", HasInbound: true}, 7, []string{"acme.io"}, InLogFilter{})
|
||||||
|
out = renderSignedIn(t, "in-log", domain.WithRows([]InLogRow{domain.NewRow(InLogRowInput{
|
||||||
|
At: time.Date(2026, 9, 21, 14, 0, 0, 0, time.UTC), From: "a@example.net", To: "b@acme.io",
|
||||||
|
Decision: "accept", Engine: "rspamd"})}, 1, 1))
|
||||||
|
pageHas(t, "Inbound log", out, `Rows older than 7 days are deleted.`, `rspamd</td>`)
|
||||||
|
pageLacks(t, "Inbound log", out, `/server/settings`, `/inbound/filter-lists`, `Deny sender`, `Allow sender`, `Allow / Deny`)
|
||||||
|
}
|
||||||
|
|
||||||
|
// The lead's day count reads right for one day too, though the setting's range
|
||||||
|
// starts at seven: the word follows the number, not the other way round.
|
||||||
|
func TestInLogLeadCountsDays(t *testing.T) {
|
||||||
|
out := renderSignedIn(t, "in-log", NewInLog(Meta{User: "ops", HasInbound: true}, 1, nil, InLogFilter{}))
|
||||||
|
pageHas(t, "Inbound log", out, `Rows older than 1 day are deleted.`)
|
||||||
|
}
|
||||||
|
|
||||||
|
// The filter in force is the one selected, and the paging and the empty state
|
||||||
|
// carry and name it.
|
||||||
|
func TestInLogCarriesItsFilters(t *testing.T) {
|
||||||
|
f := InLogFilter{Domain: "acme.io", Decision: "reject", From: "2026-09-01", To: "2026-09-30"}
|
||||||
|
p := NewInLog(admin(), 90, []string{"lists.example.org", "acme.io"}, f)
|
||||||
|
p.WithRows([]InLogRow{p.NewRow(InLogRowInput{At: time.Date(2026, 9, 2, 1, 2, 3, 0, time.UTC),
|
||||||
|
From: "a@example.net", To: "b@acme.io", Decision: "reject", Engine: "postfix"})}, 3, 5)
|
||||||
|
out := renderSignedIn(t, "in-log", p)
|
||||||
|
pageHas(t, "Inbound log", out,
|
||||||
|
`<option value="acme.io" selected>acme.io</option>`, `<option value="reject" selected>reject</option>`,
|
||||||
|
`name="from" value="2026-09-01"`, `name="to" value="2026-09-30"`,
|
||||||
|
`Page 3 of 5 · <a href="/inbound/log?decision=reject&domain=acme.io&from=2026-09-01&p=2&to=2026-09-30">← Newer</a>`,
|
||||||
|
`<a href="/inbound/log?decision=reject&domain=acme.io&from=2026-09-01&p=4&to=2026-09-30">Older →</a>`,
|
||||||
|
)
|
||||||
|
if got := InLogHref(InLogFilter{}, 1); got != "/inbound/log" {
|
||||||
|
t.Errorf("the first unfiltered page is %q", got)
|
||||||
|
}
|
||||||
|
if got := InLogHref(InLogFilter{Domain: "a b&c"}, 2); got != "/inbound/log?domain=a+b%26c&p=2" {
|
||||||
|
t.Errorf("a filter is not encoded: %q", got)
|
||||||
|
}
|
||||||
|
|
||||||
|
out = renderSignedIn(t, "in-log", NewInLog(admin(), 90, nil, f).WithRows(nil, 1, 1))
|
||||||
|
pageHas(t, "Inbound log", out, `No messages match these filters.`)
|
||||||
|
out = renderSignedIn(t, "in-log", NewInLog(admin(), 90, nil, InLogFilter{}).WithRows(nil, 1, 1))
|
||||||
|
pageHas(t, "Inbound log", out, `No messages logged yet.`)
|
||||||
|
pageLacks(t, "Inbound log", out, `<table`, `Page 1`, `sp-box-foot`)
|
||||||
|
}
|
||||||
|
|
||||||
|
// The last page has nothing older.
|
||||||
|
func TestInLogLastPage(t *testing.T) {
|
||||||
|
p := NewInLog(admin(), 90, nil, InLogFilter{})
|
||||||
|
row := p.NewRow(InLogRowInput{At: time.Now(), From: "a@example.net", To: "b@acme.io", Decision: "accept", Engine: "selfpost"})
|
||||||
|
out := renderSignedIn(t, "in-log", p.WithRows([]InLogRow{row}, 2, 2))
|
||||||
|
pageHas(t, "Inbound log", out, `Page 2 of 2`, `← Newer</a>`)
|
||||||
|
pageLacks(t, "Inbound log", out, `Older →`)
|
||||||
|
}
|
||||||
|
|
||||||
|
// A row says who decided, and what they said when they said something; a message
|
||||||
|
// refused before its sender was known has no sender to show or to list.
|
||||||
|
func TestInLogRowWording(t *testing.T) {
|
||||||
|
p := NewInLog(admin(), 90, nil, InLogFilter{})
|
||||||
|
at := time.Date(2026, 9, 21, 14, 0, 18, 0, time.UTC)
|
||||||
|
r := p.NewRow(InLogRowInput{At: at, From: "a@example.net", To: "b@acme.io", Decision: "accept", Engine: "selfpost"})
|
||||||
|
if len(r.Why.Rows) != 1 || r.Why.Rows[0].Lines[0] != "selfpost" || r.Time != "09-21 14:00:18" || r.Client != "" {
|
||||||
|
t.Errorf("row = %+v", r)
|
||||||
|
}
|
||||||
|
r = p.NewRow(InLogRowInput{ID: 7, At: at, To: "b@acme.io", Decision: "reject", Engine: "postfix", Detail: "550 5.1.1 no"})
|
||||||
|
if r.Why.ID != "why-7" || r.Why.Tag.Status != "reject" || len(r.Why.Rows) != 2 || r.Why.Rows[1].Label != "Reply" || r.From != "—" || r.Action != nil {
|
||||||
|
t.Errorf("row = %+v", r)
|
||||||
|
}
|
||||||
|
out := renderSignedIn(t, "in-log", p.WithRows([]InLogRow{r}, 1, 1))
|
||||||
|
pageHas(t, "Inbound log", out, `<td>—</td>`)
|
||||||
|
pageLacks(t, "Inbound log", out, `<br><span class="sp-mono sp-muted sp-small"></span>`)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Which decisions lead to which list, for the global role: the sender of mail
|
||||||
|
// that got through can be denied, the sender of mail the filter turned away or
|
||||||
|
// put off can be allowed, and a refusal that is Postfix's own offers neither.
|
||||||
|
func TestInLogSenderAction(t *testing.T) {
|
||||||
|
for _, c := range []struct {
|
||||||
|
decision, engine, from, label, href string
|
||||||
|
}{
|
||||||
|
{"accept", "rspamd", "a@example.net", "Deny sender", "/inbound/filter-lists?list=deny&kind=address&value=a%40example.net"},
|
||||||
|
{"accept", "selfpost", "a@example.net", "Deny sender", "/inbound/filter-lists?list=deny&kind=address&value=a%40example.net"},
|
||||||
|
{"reject", "rspamd", "a+b@example.net", "Allow sender", "/inbound/filter-lists?list=allow&kind=address&value=a%2Bb%40example.net"},
|
||||||
|
{"tempfail", "rspamd", "a@example.net", "Allow sender", "/inbound/filter-lists?list=allow&kind=address&value=a%40example.net"},
|
||||||
|
{"quarantine", "rspamd", "a@example.net", "Allow sender", "/inbound/filter-lists?list=allow&kind=address&value=a%40example.net"},
|
||||||
|
{"reject", "postfix", "a@example.net", "", ""},
|
||||||
|
{"tempfail", "postfix", "a@example.net", "", ""},
|
||||||
|
{"reject", "selfpost", "a@example.net", "", ""},
|
||||||
|
{"accept", "rspamd", "", "", ""},
|
||||||
|
{"reject", "rspamd", "", "", ""},
|
||||||
|
} {
|
||||||
|
got := senderAction(c.decision, c.engine, c.from)
|
||||||
|
switch {
|
||||||
|
case c.label == "" && got != nil:
|
||||||
|
t.Errorf("%s by %s from %q is offered %+v", c.decision, c.engine, c.from, *got)
|
||||||
|
case c.label != "" && (got == nil || got.Label != c.label || got.Href != c.href):
|
||||||
|
t.Errorf("%s by %s from %q is offered %+v, want %s -> %s", c.decision, c.engine, c.from, got, c.label, c.href)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The address in the link is data: it cannot leave the attribute.
|
||||||
|
p := NewInLog(admin(), 90, nil, InLogFilter{})
|
||||||
|
// (The handler would not call such a sender listable; the view must not
|
||||||
|
// depend on that.)
|
||||||
|
row := p.NewRow(InLogRowInput{Listable: true, At: time.Now(), From: `"><script>1</script>@x`, To: "b@acme.io", Decision: "accept", Engine: "rspamd"})
|
||||||
|
out := renderSignedIn(t, "in-log", p.WithRows([]InLogRow{row}, 1, 1))
|
||||||
|
pageLacks(t, "Inbound log", out, `<script>1`)
|
||||||
|
pageHas(t, "Inbound log", out, `value=%22%3E%3Cscript%3E1%3C%2Fscript%3E%40x`)
|
||||||
|
|
||||||
|
// A sender the lists would refuse is offered no way into them.
|
||||||
|
if row := p.NewRow(InLogRowInput{At: time.Now(), From: "o'brien@example.org", To: "b@acme.io", Decision: "accept", Engine: "rspamd"}); row.Action != nil {
|
||||||
|
t.Errorf("a sender that is not listable is offered %+v", *row.Action)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// What a sender wrote is text on the page: an address, a subject or a reply from
|
||||||
|
// a server is never markup.
|
||||||
|
func TestInLogEscapesWhatSendersWrote(t *testing.T) {
|
||||||
|
p := NewInLog(admin(), 90, nil, InLogFilter{})
|
||||||
|
row := p.NewRow(InLogRowInput{At: time.Now(), From: `<img src=x onerror=alert(1)>@e.example`, Client: `<b>1</b>`,
|
||||||
|
To: `<script>alert(2)</script>@acme.io`, Subject: `<script>alert(3)</script> "quoted"`,
|
||||||
|
Decision: "reject", Engine: "rspamd", Detail: `<i>5.7.1</i>`})
|
||||||
|
out := renderSignedIn(t, "in-log", p.WithRows([]InLogRow{row}, 1, 1))
|
||||||
|
pageLacks(t, "Inbound log", out, `<script>alert`, `<img src=x`, `<b>1</b>`, `<i>5.7.1</i>`)
|
||||||
|
pageHas(t, "Inbound log", out, `<script>alert(3)</script>`, `<i>5.7.1</i>`)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- Filter lists
|
||||||
|
|
||||||
|
// The page is the add form, the two lists with their counts, and where the lists
|
||||||
|
// go - drawn as the mockup draws it.
|
||||||
|
func TestFilterListsShowsTheFormTheListsAndWhereTheyGo(t *testing.T) {
|
||||||
|
out := renderSignedIn(t, "in-filter-lists", inFilterListsFixture())
|
||||||
|
pageHas(t, "Filter lists", out,
|
||||||
|
`<p class="sp-kicker">Inbound · global role</p>`, `<h1 class="title is-3">Filter lists</h1>`,
|
||||||
|
`<form class="sp-form" method="post" action="/inbound/filter-lists">`,
|
||||||
|
`<select id="list" name="list_type">`, `<option value="deny" selected>Deny — always turn away</option>`,
|
||||||
|
`<option value="allow">Allow — always let through</option>`,
|
||||||
|
`<select id="etype" name="entry_type">`, `<option value="address" selected>a sender address</option>`,
|
||||||
|
`<option value="domain">a sender domain</option>`, `<option value="ip">a client address or network</option>`,
|
||||||
|
`<input class="input sp-mono" id="value" name="value" value="" placeholder="promo@bulk.example"`,
|
||||||
|
`<input class="input" id="note" name="note" value=""`, `Add entry`,
|
||||||
|
`href="/help#filter-lists" title="How the lists are applied"`,
|
||||||
|
`<h2>Deny</h2><div class="sp-end">3 entries · wins over Allow</div>`,
|
||||||
|
`<h2>Allow</h2><div class="sp-end">2 entries</div>`,
|
||||||
|
`<thead><tr><th>Value</th><th>Kind</th><th>Matched</th><th></th></tr></thead>`,
|
||||||
|
// the kind is a tag that opens the entry's sheet; the count is beside it
|
||||||
|
`<td class="sp-mono sp-small">bulk.example</td><td><button type="button" class="tag is-light" popovertarget="entry-1">domain</button><div class="sp-sheet" id="entry-1" popover>`,
|
||||||
|
`<div class="sp-sheet-head"><span class="tag is-light">domain</span><h2>bulk.example</h2>`,
|
||||||
|
`<tr><th>Note</th><td>newsletter nobody subscribed to</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-09-02</td></tr><tr><th>Matched</th><td>41 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-09-21 13:57</td></tr></tbody>`,
|
||||||
|
`</div></td><td class="sp-nowrap">41</td><td class="sp-actions">`,
|
||||||
|
`<td class="sp-nowrap">1203</td>`,
|
||||||
|
// an entry that never decided anything says so, in the row and on its sheet
|
||||||
|
`<td class="sp-mono sp-small">win@lottery.example</td><td><button type="button" class="tag is-light" popovertarget="entry-3">address</button>`,
|
||||||
|
`<tr><th>Note</th><td>—</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-07-30</td></tr><tr><th>Matched</th><td>never in the last 90 days</td></tr></tbody>`,
|
||||||
|
`</div></td><td class="sp-muted sp-nowrap">never</td><td class="sp-actions">`,
|
||||||
|
`<form method="post" action="/inbound/filter-lists/3/delete"><button type="submit" class="has-text-danger">Remove</button></form>`,
|
||||||
|
`<h2>Where the lists go</h2><div class="sp-end"><span class="tag is-success is-light">applied</span></div>`,
|
||||||
|
`rspamd, from map files it rereads within seconds of a change`,
|
||||||
|
`<span class="sp-mono">/data/antispam/</span> — six, one for each list and kind of entry`,
|
||||||
|
`2026-09-11 08:40 UTC`, `Deny wins, whichever entries they are.`,
|
||||||
|
)
|
||||||
|
if got := strings.Count(out, `<table class="table is-fullwidth is-hoverable">`); got != 2 {
|
||||||
|
t.Errorf("%d list tables, want the two lists", got)
|
||||||
|
}
|
||||||
|
pageLacks(t, "Filter lists", out, `<th>Note</th><th>`, `<th>Added</th><th>`)
|
||||||
|
}
|
||||||
|
|
||||||
|
// With no filter attached the lists are still the operator's to edit, and the
|
||||||
|
// box says nothing reads them; a list with no entries says so in its box.
|
||||||
|
func TestFilterListsOffAndEmpty(t *testing.T) {
|
||||||
|
p := NewFilterLists(admin(), FilterListsInput{}).WithEntries([]FilterEntryInput{
|
||||||
|
{ID: 9, List: "allow", Kind: "ip", Value: "192.0.2.1", Added: time.Date(2026, 9, 1, 0, 0, 0, 0, time.UTC)},
|
||||||
|
})
|
||||||
|
out := renderSignedIn(t, "in-filter-lists", p)
|
||||||
|
pageHas(t, "Filter lists", out,
|
||||||
|
`<span class="tag is-light">not applied</span>`,
|
||||||
|
`Nothing — the spam filter is off. The lists are kept and can be edited; nothing reads them until <code>INBOUND_ANTISPAM_MILTER</code> is set.`,
|
||||||
|
`<dt>Last written</dt><dd>never</dd>`,
|
||||||
|
`<h2>Deny</h2><div class="sp-end">0 entries · wins over Allow</div>`,
|
||||||
|
`<h2>Allow</h2><div class="sp-end">1 entry</div>`,
|
||||||
|
`No sender is turned away by hand.`,
|
||||||
|
)
|
||||||
|
pageLacks(t, "Filter lists", out, `rspamd, from map files`, `is-success is-light">applied`, `No sender is let through by hand.`)
|
||||||
|
if got := strings.Count(out, `<table class="table is-fullwidth is-hoverable">`); got != 1 {
|
||||||
|
t.Errorf("%d list tables, want the one list that has entries", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The form shows what it is told to: prefilled from the log, or what was typed
|
||||||
|
// into a form that was refused, with the refusal between the head and the form.
|
||||||
|
func TestFilterListsFormAndRefusal(t *testing.T) {
|
||||||
|
p := inFilterListsFixture().
|
||||||
|
WithForm("allow", "domain", `x"><script>1</script>`, `a "note"`).
|
||||||
|
WithResult("", "That entry is already on the list.")
|
||||||
|
out := renderSignedIn(t, "in-filter-lists", p)
|
||||||
|
pageHas(t, "Filter lists", out,
|
||||||
|
`<option value="allow" selected>Allow — always let through</option>`, `<option value="domain" selected>a sender domain</option>`,
|
||||||
|
`value="x"><script>1</script>"`, `value="a "note""`,
|
||||||
|
`<div class="notification is-danger is-light">`, `That entry is already on the list.`)
|
||||||
|
pageLacks(t, "Filter lists", out, `<script>1`, `<option value="deny" selected>`)
|
||||||
|
if strings.Index(out, `class="notification`) < strings.Index(out, `class="sp-head"`) ||
|
||||||
|
strings.Index(out, `class="notification`) > strings.Index(out, `<form class="sp-form"`) {
|
||||||
|
t.Error("the refusal is not between the head and the form")
|
||||||
|
}
|
||||||
|
|
||||||
|
out = renderSignedIn(t, "in-filter-lists", inFilterListsFixture().WithResult("Entry added.", ""))
|
||||||
|
pageHas(t, "Filter lists", out, `<div class="notification is-success is-light">`, `Entry added.`)
|
||||||
|
}
|
||||||
|
|
||||||
|
// What a person typed into a list is text on the page.
|
||||||
|
func TestFilterListsEscapesTheEntries(t *testing.T) {
|
||||||
|
p := NewFilterLists(admin(), FilterListsInput{FilterOn: true}).WithEntries([]FilterEntryInput{
|
||||||
|
{ID: 1, List: "deny", Kind: "address", Value: `<img src=x>@a.example`, Note: `<script>alert(1)</script>`, Added: time.Now()},
|
||||||
|
})
|
||||||
|
out := renderSignedIn(t, "in-filter-lists", p)
|
||||||
|
pageLacks(t, "Filter lists", out, `<img src=x>`, `<script>alert`)
|
||||||
|
}
|
||||||
|
|
||||||
|
// The menu leads to both pages, the lists for the global role only.
|
||||||
|
func TestInboundMenuHasLogAndFilterLists(t *testing.T) {
|
||||||
|
out := renderSignedIn(t, "in-log", inLogFixture())
|
||||||
|
pageHas(t, "Inbound log", out,
|
||||||
|
`<a class="navbar-item is-active" href="/inbound/log"><i class="ti ti-list-details"></i>Log</a>`,
|
||||||
|
`<a class="navbar-item" href="/inbound/filter-lists"><i class="ti ti-filter"></i>Filter lists</a>`,
|
||||||
|
`<a href="/inbound/log" aria-current="page"><i class="ti ti-list-details"></i>Log</a><a href="/inbound/filter-lists"><i class="ti ti-filter"></i>Filter lists</a>`)
|
||||||
|
out = renderSignedIn(t, "in-filter-lists", inFilterListsFixture())
|
||||||
|
pageHas(t, "Filter lists", out,
|
||||||
|
`<a class="navbar-item is-active" href="/inbound/filter-lists">`, `<a href="/inbound/filter-lists" aria-current="page">`)
|
||||||
|
|
||||||
|
domain := NewInLog(Meta{User: "ops", HasInbound: true}, 90, nil, InLogFilter{}).WithRows(nil, 1, 1)
|
||||||
|
out = renderSignedIn(t, "in-log", domain)
|
||||||
|
pageHas(t, "Inbound log", out, `<a class="navbar-item is-active" href="/inbound/log">`, `<a href="/inbound/log" aria-current="page">`)
|
||||||
|
pageLacks(t, "Inbound log", out, `/inbound/filter-lists`)
|
||||||
|
}
|
||||||
|
|
||||||
|
// What a sheet says, from the two shapes a row's detail has: the filter's
|
||||||
|
// verdict on mail that passed, and the reply a refusal was given.
|
||||||
|
func TestWhyRows(t *testing.T) {
|
||||||
|
flat := func(rows []SheetRow) string {
|
||||||
|
var out []string
|
||||||
|
for _, r := range rows {
|
||||||
|
out = append(out, r.Label+"="+strings.Join(r.Lines, "|"))
|
||||||
|
}
|
||||||
|
return strings.Join(out, "; ")
|
||||||
|
}
|
||||||
|
for _, c := range []struct{ decision, engine, detail, subject, want string }{
|
||||||
|
{"accept", "selfpost", "", "", "Decided by=selfpost"},
|
||||||
|
{"accept", "selfpost", "", "Hello", "Decided by=selfpost; Subject=Hello"},
|
||||||
|
{"accept", "rspamd", "no action, score 0.40 / 15.00", "Hi", "Decided by=rspamd; Verdict=no action, score 0.40 / 15.00; Subject=Hi"},
|
||||||
|
{"accept", "rspamd", "add header, score 9.80 / 15.00; A(5.00) B(-0.10)", "", "Decided by=rspamd; Verdict=add header, score 9.80 / 15.00; Symbols=A(5.00)|B(-0.10)"},
|
||||||
|
// a verdict with symbols and no action or score
|
||||||
|
{"quarantine", "rspamd", "A(5.00)", "", "Decided by=rspamd; Verdict=A(5.00)"},
|
||||||
|
{"reject", "rspamd", "5.7.1 Spam message rejected", "", "Decided by=rspamd; Reply=5.7.1 Spam message rejected"},
|
||||||
|
{"tempfail", "postfix", "450 4.1.8 <a@nx.example>: Sender address rejected; see x", "", "Decided by=postfix; Reply=450 4.1.8 <a@nx.example>: Sender address rejected; see x"},
|
||||||
|
// a refusal's text is a reply whatever it starts with
|
||||||
|
{"reject", "rspamd", "milter triggers DISCARD action", "", "Decided by=rspamd; Reply=milter triggers DISCARD action"},
|
||||||
|
} {
|
||||||
|
if got := flat(whyRows(c.decision, c.engine, c.detail, c.subject)); got != c.want {
|
||||||
|
t.Errorf("whyRows(%q, %q, %q, %q)\n got: %s\nwant: %s", c.decision, c.engine, c.detail, c.subject, got, c.want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -21,6 +21,7 @@ type Kit struct {
|
|||||||
Postmarks []Postmark
|
Postmarks []Postmark
|
||||||
StatusTag Box
|
StatusTag Box
|
||||||
StatusRows []KitStatusRow
|
StatusRows []KitStatusRow
|
||||||
|
Sheet Sheet
|
||||||
BoxHead Box
|
BoxHead Box
|
||||||
BoxFoot Foot
|
BoxFoot Foot
|
||||||
DangerZone Box
|
DangerZone Box
|
||||||
@@ -91,10 +92,14 @@ func KitPage() *Kit {
|
|||||||
},
|
},
|
||||||
|
|
||||||
StatusTag: Box{No: "03", Title: "status_tag", End: Plain("One mapping, one function")},
|
StatusTag: Box{No: "03", Title: "status_tag", End: Plain("One mapping, one function")},
|
||||||
|
Sheet: Sheet{ID: "kit-sheet", Tag: Tag{Status: "reject"}, Title: "sheet", Rows: []SheetRow{
|
||||||
|
{Label: "What it holds", Lines: []string{"What does not fit the row, as name and value"}},
|
||||||
|
{Label: "How it closes", Lines: []string{"Esc", "a click outside", "the cross"}, Mono: true},
|
||||||
|
}},
|
||||||
StatusRows: []KitStatusRow{
|
StatusRows: []KitStatusRow{
|
||||||
{"ok, published, running, sent", Tag{Status: "ok"}},
|
{"ok, published, running, sent, accept", Tag{Status: "ok"}},
|
||||||
{"warn, deferred, quiet", Tag{Status: "warn"}},
|
{"warn, deferred, quiet, tempfail", Tag{Status: "warn"}},
|
||||||
{"error, fail, mismatch, bounced", Tag{Status: "fail"}},
|
{"error, fail, mismatch, bounced, reject", Tag{Status: "fail"}},
|
||||||
{"unknown, off, neutral counts", Tag{Status: "off"}},
|
{"unknown, off, neutral counts", Tag{Status: "off"}},
|
||||||
{"held: quarantine · role: global", Tag{Status: "quarantine"}},
|
{"held: quarantine · role: global", Tag{Status: "quarantine"}},
|
||||||
},
|
},
|
||||||
|
|||||||
@@ -402,3 +402,23 @@ func TestOverviewFragmentIsTheBoxAndTheHeadOutOfBand(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// With the inbound relay on the retention box has a second field, for the Inbound
|
||||||
|
// log; with it off the field is not there, since there is no journal to keep.
|
||||||
|
func TestSettingsHasTheInboundRetentionOnlyWithTheRelay(t *testing.T) {
|
||||||
|
out := renderSignedIn(t, "settings", settingsFixture())
|
||||||
|
pageHas(t, "Settings", out,
|
||||||
|
`for="inbound_spam_log_retention_days">Keep inbound log rows, days</label>`,
|
||||||
|
`name="inbound_spam_log_retention_days" type="number" min="7" max="365" step="1" value="90" required`,
|
||||||
|
`The inbound log also never holds more than 10 000 rows: refused mail is logged at the rate strangers knock on port 25.`,
|
||||||
|
`name="send_log_retention_days"`)
|
||||||
|
if got := strings.Count(out, `<form class="sp-form"`); got != 1 {
|
||||||
|
t.Errorf("Settings has %d forms, want 1", got)
|
||||||
|
}
|
||||||
|
pageLacks(t, "Settings", renderSignedIn(t, "settings", NewSettings(admin(), "30", "600 / h")),
|
||||||
|
`inbound_spam_log_retention_days`, `Keep inbound log rows`, `10 000 rows`)
|
||||||
|
|
||||||
|
// A refused value comes back as typed.
|
||||||
|
out = renderSignedIn(t, "settings", NewSettings(admin(), "30", "600 / h").WithInbound("3").WithResult("", "Inbound log retention must be between 7 and 365."))
|
||||||
|
pageHas(t, "Settings", out, `value="3"`, `between 7 and 365`)
|
||||||
|
}
|
||||||
|
|||||||
@@ -327,3 +327,27 @@ code { background: var(--sp-well); color: var(--sp-ink); border-radius: 3px; pad
|
|||||||
.sp-record-head small { margin-left: 0; text-align: left; flex-basis: 100%; }
|
.sp-record-head small { margin-left: 0; text-align: left; flex-basis: 100%; }
|
||||||
.sp-postmark { display: none; }
|
.sp-postmark { display: none; }
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/* sheet: what does not fit a table row, opened over the page from the row's
|
||||||
|
tag. It is the browser's own popover (the tag is a button with
|
||||||
|
popovertarget), so it needs no script: Esc, a click outside and the close
|
||||||
|
button all shut it, and focus returns to the tag. It rises from the bottom
|
||||||
|
edge across the whole window - the one component that is not laid out by the
|
||||||
|
page, hence the widths - with its content, a table of name and value, in a
|
||||||
|
column of reading width in the middle. */
|
||||||
|
.sp-sheet {
|
||||||
|
inset: auto 0 0 0; width: auto; margin: 0; padding: 0; border: 0; border-top: 3px solid var(--sp-brick);
|
||||||
|
max-height: 75vh; overflow: auto; background: #fff; color: inherit; box-shadow: 0 -12px 40px rgba(40, 20, 15, .22);
|
||||||
|
}
|
||||||
|
.sp-sheet-in { max-width: 44rem; margin: 0 auto; padding: 0 1.25rem; }
|
||||||
|
.sp-sheet::backdrop { background: rgba(40, 20, 15, .38); }
|
||||||
|
.sp-sheet-head { display: flex; align-items: center; gap: .7rem; padding: 1rem 0 .8rem; border-bottom: 1px solid var(--sp-rule); }
|
||||||
|
.sp-sheet-head h2 { font-size: 1.15rem; font-weight: 600; margin-right: auto; }
|
||||||
|
.sp-sheet-head button { border: 0; background: none; cursor: pointer; font-size: 1.3rem; color: var(--sp-muted); }
|
||||||
|
.sp-sheet-body { padding: .4rem 0 1.4rem; }
|
||||||
|
/* The sheet's content is a table of name and value: the name is a label, set like the dt of facts. */
|
||||||
|
.sp-sheet .table th { font-family: var(--sp-mono); font-size: var(--sp-fs-xs); font-weight: 500; letter-spacing: .08em; text-transform: uppercase; color: var(--sp-muted); white-space: nowrap; vertical-align: top; width: 1%; padding-right: 2rem; }
|
||||||
|
.sp-sheet .table td { overflow-wrap: anywhere; }
|
||||||
|
/* A tag that opens a sheet is a button: it keeps the tag's look and says it can
|
||||||
|
be pressed with the dotted underline links to more detail carry. */
|
||||||
|
button.tag { border: 0; cursor: pointer; text-decoration: underline dotted; text-underline-offset: 3px; }
|
||||||
|
|||||||
@@ -13,7 +13,7 @@
|
|||||||
<div class="column is-6">
|
<div class="column is-6">
|
||||||
{{template "box_open" .Full}}
|
{{template "box_open" .Full}}
|
||||||
{{template "body_open"}}
|
{{template "body_open"}}
|
||||||
<p class="mb-3"><code>data/</code> (database, DKIM keys, credentials, the queue), <code>docker-compose.yml</code>, <code>.env</code> and <code>certs/</code> in one archive. Restore into an empty project directory and start the <strong>same SelfPost version</strong>. The reverse-proxy vhost is not included.</p>
|
<p class="mb-3"><code>data/</code> (database, DKIM keys, credentials), <code>docker-compose.yml</code>, <code>.env</code> and <code>certs/</code> in one archive. The Postfix queue is left out: it cannot be copied reliably while the server is running. Restore into an empty project directory and start the <strong>same SelfPost version</strong>. The reverse-proxy vhost is not included.</p>
|
||||||
<form class="sp-form" method="post" action="{{.FullAction}}">
|
<form class="sp-form" method="post" action="{{.FullAction}}">
|
||||||
<div class="field"><label class="checkbox"><input type="checkbox" name="encrypt" value="1"{{if .Encrypt}} checked{{end}} data-encrypt-toggle> Encrypt with a password (<code>.spbk</code>)</label></div>
|
<div class="field"><label class="checkbox"><input type="checkbox" name="encrypt" value="1"{{if .Encrypt}} checked{{end}} data-encrypt-toggle> Encrypt with a password (<code>.spbk</code>)</label></div>
|
||||||
<div data-encrypt-fields>
|
<div data-encrypt-fields>
|
||||||
|
|||||||
@@ -27,6 +27,13 @@
|
|||||||
mapping of status to Bulma classes; the label defaults to the status. */}}
|
mapping of status to Bulma classes; the label defaults to the status. */}}
|
||||||
{{define "tag"}}<span class="{{status_tag .Status}}">{{if .Label}}{{.Label}}{{else}}{{.Status}}{{end}}</span>{{end}}
|
{{define "tag"}}<span class="{{status_tag .Status}}">{{if .Label}}{{.Label}}{{else}}{{.Status}}{{end}}</span>{{end}}
|
||||||
|
|
||||||
|
{{/* sheet_tag is a status tag that opens a sheet (input: Sheet): the tag as a
|
||||||
|
button, and the sheet it names right after it. sheet is what opens: the tag
|
||||||
|
again, a title, the cross that closes it, and a table of name and value.
|
||||||
|
The browser's popover does the rest - no script. */}}
|
||||||
|
{{define "sheet_tag"}}<button type="button" class="{{status_tag .Tag.Status}}" popovertarget="{{.ID}}">{{if .Tag.Label}}{{.Tag.Label}}{{else}}{{.Tag.Status}}{{end}}</button>{{template "sheet" .}}{{end}}
|
||||||
|
{{define "sheet"}}<div class="sp-sheet" id="{{.ID}}" popover><div class="sp-sheet-in"><div class="sp-sheet-head">{{template "tag" .Tag}}<h2>{{.Title}}</h2><button type="button" popovertarget="{{.ID}}" popovertargetaction="hide" aria-label="Close"><i class="ti ti-x"></i></button></div><div class="sp-sheet-body"><div class="table-container"><table class="table is-fullwidth"><tbody>{{range .Rows}}<tr><th>{{.Label}}</th><td{{if .Mono}} class="sp-mono sp-small"{{end}}>{{range $i, $line := .Lines}}{{if $i}}<br>{{end}}{{$line}}{{end}}</td></tr>{{end}}</tbody></table></div></div></div></div>{{end}}
|
||||||
|
|
||||||
{{/* kicker is the upper-case mono micro-label on its own (input: string).
|
{{/* kicker is the upper-case mono micro-label on its own (input: string).
|
||||||
page_head writes its own kicker as the line above the title. */}}
|
page_head writes its own kicker as the line above the title. */}}
|
||||||
{{define "kicker"}}<span class="sp-kicker">{{.}}</span>{{end}}
|
{{define "kicker"}}<span class="sp-kicker">{{.}}</span>{{end}}
|
||||||
|
|||||||
@@ -0,0 +1,64 @@
|
|||||||
|
{{/* Inbound > Filter lists (mockup in-filter-lists.html): the allow and deny
|
||||||
|
lists that correct the spam filter by hand, for the whole instance, and
|
||||||
|
where they go. Global role only. The form adds an entry; each row of a list
|
||||||
|
removes its own, says how often the entry decided a message, and opens a
|
||||||
|
sheet with the entry's note from the tag of its kind. */}}
|
||||||
|
{{define "content"}}
|
||||||
|
{{template "page_head" .Head}}
|
||||||
|
{{if .Flash}}{{template "flash" .Flash}}{{end}}
|
||||||
|
|
||||||
|
{{template "box_open" .Add}}
|
||||||
|
{{template "body_open"}}
|
||||||
|
<form class="sp-form" method="post" action="/inbound/filter-lists">
|
||||||
|
<div class="columns">
|
||||||
|
<div class="column is-3">
|
||||||
|
<div class="field"><label class="label" for="list">List</label>
|
||||||
|
<div class="control"><div class="select is-fullwidth"><select id="list" name="list_type">{{$list := .List}}{{range .ListOptions}}
|
||||||
|
<option value="{{.Value}}"{{if eq .Value $list}} selected{{end}}>{{.Label}}</option>{{end}}
|
||||||
|
</select></div></div></div>
|
||||||
|
</div>
|
||||||
|
<div class="column is-3">
|
||||||
|
<div class="field"><label class="label" for="etype">Entry is</label>
|
||||||
|
<div class="control"><div class="select is-fullwidth"><select id="etype" name="entry_type">{{$kind := .Kind}}{{range .KindOptions}}
|
||||||
|
<option value="{{.Value}}"{{if eq .Value $kind}} selected{{end}}>{{.Label}}</option>{{end}}
|
||||||
|
</select></div></div></div>
|
||||||
|
</div>
|
||||||
|
<div class="column is-6">
|
||||||
|
<div class="field"><label class="label" for="value">Value</label>
|
||||||
|
<div class="control"><input class="input sp-mono" id="value" name="value" value="{{.Value}}" placeholder="promo@bulk.example" autocomplete="off" autocapitalize="none" spellcheck="false" required></div></div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div class="field"><label class="label" for="note">Note</label>
|
||||||
|
<div class="control"><input class="input" id="note" name="note" value="{{.Note}}" placeholder="Why — for whoever reads this list next year"></div></div>
|
||||||
|
<button type="submit" class="button is-primary"><i class="ti ti-plus"></i>Add entry</button>
|
||||||
|
</form>
|
||||||
|
{{template "body_close"}}
|
||||||
|
{{template "box_close"}}
|
||||||
|
|
||||||
|
<div class="columns">
|
||||||
|
<div class="column is-6">
|
||||||
|
{{template "filter_list" .Deny}}
|
||||||
|
</div>
|
||||||
|
<div class="column is-6">
|
||||||
|
{{template "filter_list" .Allow}}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
{{template "box_open" .Where}}
|
||||||
|
{{template "body_open"}}
|
||||||
|
{{template "facts" .WhereFacts}}
|
||||||
|
{{template "body_close"}}
|
||||||
|
{{template "box_close"}}
|
||||||
|
{{end}}
|
||||||
|
|
||||||
|
{{/* filter_list is one of the two boxes of entries (input: FilterList): the
|
||||||
|
table, or what is absent when the list is empty. */}}
|
||||||
|
{{define "filter_list"}}{{template "box_open" .Box}}
|
||||||
|
{{if .Rows}}<div class="table-container"><table class="table is-fullwidth is-hoverable">
|
||||||
|
<thead><tr><th>Value</th><th>Kind</th><th>Matched</th><th></th></tr></thead>
|
||||||
|
<tbody>{{range .Rows}}
|
||||||
|
<tr><td class="sp-mono sp-small">{{.Value}}</td><td>{{template "sheet_tag" .Entry}}</td>{{if .Matched}}<td class="sp-nowrap">{{.Matched}}</td>{{else}}<td class="sp-muted sp-nowrap">never</td>{{end}}<td class="sp-actions">{{template "row_action" .Remove}}</td></tr>{{end}}
|
||||||
|
</tbody>
|
||||||
|
</table></div>
|
||||||
|
{{else}}{{template "empty_state" .Empty}}
|
||||||
|
{{end}}{{template "box_close"}}{{end}}
|
||||||
@@ -0,0 +1,23 @@
|
|||||||
|
{{/* Inbound log (mockup in-log.html): what port 25 decided about each message
|
||||||
|
offered for an inbound domain, one row per recipient, newest first,
|
||||||
|
filtered by domain, decision and day. It does not poll. The decision is a
|
||||||
|
tag that opens a sheet with who decided and why. The last cell is the way
|
||||||
|
to correct a decision in Filter lists, and only the global role, who alone
|
||||||
|
edits the lists, is given it. */}}
|
||||||
|
{{define "content"}}
|
||||||
|
{{template "page_head" .Head}}
|
||||||
|
|
||||||
|
{{template "box_open" .Messages}}
|
||||||
|
{{if .Rows}}
|
||||||
|
<div class="table-container"><table class="table is-fullwidth is-hoverable">
|
||||||
|
<thead><tr><th>Time, UTC</th><th>From</th><th>To</th><th>Subject</th><th>Decision</th><th></th></tr></thead>
|
||||||
|
<tbody>{{range .Rows}}
|
||||||
|
<tr><td class="sp-mono sp-muted sp-nowrap sp-small">{{.Time}}</td><td>{{wbr_at .From}}{{if .Client}}<br><span class="sp-mono sp-muted sp-small">{{.Client}}</span>{{end}}</td><td>{{wbr_at .To}}</td>{{if .Subject}}<td class="sp-clip" title="{{.Subject}}">{{.Subject}}</td>{{else}}<td class="sp-clip sp-muted">—</td>{{end}}<td>{{template "sheet_tag" .Why}}</td><td class="sp-actions">{{if .Action}}<a href="{{.Action.Href}}">{{.Action.Label}}</a>{{end}}</td></tr>{{end}}
|
||||||
|
</tbody>
|
||||||
|
</table></div>
|
||||||
|
{{template "box_foot" .Foot}}
|
||||||
|
{{else}}
|
||||||
|
{{template "empty_state" .Empty}}
|
||||||
|
{{end}}
|
||||||
|
{{template "box_close"}}
|
||||||
|
{{end}}
|
||||||
@@ -31,6 +31,7 @@
|
|||||||
<thead><tr><th>Status</th><th>Tag</th><th>Bulma classes</th></tr></thead>
|
<thead><tr><th>Status</th><th>Tag</th><th>Bulma classes</th></tr></thead>
|
||||||
<tbody>{{range .StatusRows}}
|
<tbody>{{range .StatusRows}}
|
||||||
<tr><td>{{.Words}}</td><td>{{template "tag" .Tag}}</td><td class="sp-mono sp-small">{{status_tag .Tag.Status}}</td></tr>{{end}}
|
<tr><td>{{.Words}}</td><td>{{template "tag" .Tag}}</td><td class="sp-mono sp-small">{{status_tag .Tag.Status}}</td></tr>{{end}}
|
||||||
|
<tr><td>a tag that opens a <code>sheet</code></td><td>{{template "sheet_tag" .Sheet}}</td><td class="sp-mono sp-small">button.tag + sp-sheet (popover)</td></tr>
|
||||||
</tbody>
|
</tbody>
|
||||||
</table></div>
|
</table></div>
|
||||||
{{template "box_close"}}
|
{{template "box_close"}}
|
||||||
|
|||||||
@@ -104,7 +104,9 @@
|
|||||||
<a class="navbar-link" href="/inbound/domains">Inbound</a>
|
<a class="navbar-link" href="/inbound/domains">Inbound</a>
|
||||||
<div class="navbar-dropdown">
|
<div class="navbar-dropdown">
|
||||||
<a class="navbar-item{{if and (eq .Section "inbound") (eq .Page "domains")}} is-active{{end}}" href="/inbound/domains"><i class="ti ti-world-download"></i>Domains</a>
|
<a class="navbar-item{{if and (eq .Section "inbound") (eq .Page "domains")}} is-active{{end}}" href="/inbound/domains"><i class="ti ti-world-download"></i>Domains</a>
|
||||||
</div>
|
<a class="navbar-item{{if and (eq .Section "inbound") (eq .Page "log")}} is-active{{end}}" href="/inbound/log"><i class="ti ti-list-details"></i>Log</a>
|
||||||
|
{{if .ShowFilterLists}} <a class="navbar-item{{if and (eq .Section "inbound") (eq .Page "filter-lists")}} is-active{{end}}" href="/inbound/filter-lists"><i class="ti ti-filter"></i>Filter lists</a>
|
||||||
|
{{end}} </div>
|
||||||
</div>
|
</div>
|
||||||
{{end}}{{if .ShowServer}} <div class="navbar-item has-dropdown is-hoverable{{if eq .Section "server"}} sp-current{{end}}">
|
{{end}}{{if .ShowServer}} <div class="navbar-item has-dropdown is-hoverable{{if eq .Section "server"}} sp-current{{end}}">
|
||||||
<a class="navbar-link" href="/server/health">Server</a>
|
<a class="navbar-link" href="/server/health">Server</a>
|
||||||
@@ -143,6 +145,6 @@
|
|||||||
pages of the current group, the current one marked. A page outside every
|
pages of the current group, the current one marked. A page outside every
|
||||||
group (the user menu, the kit page's neighbours) has none. */}}
|
group (the user menu, the kit page's neighbours) has none. */}}
|
||||||
{{define "subnav"}}{{if and .ShowOutbound (eq .Section "outbound")}}<div class="sp-subnav"><div class="container"><a href="/outbound/domains"{{if eq .Page "domains"}} aria-current="page"{{end}}><i class="ti ti-world-upload"></i>Domains</a><a href="/outbound/log"{{if eq .Page "log"}} aria-current="page"{{end}}><i class="ti ti-list-details"></i>Log</a>{{if .ShowQueue}}<a href="/outbound/queue"{{if eq .Page "queue"}} aria-current="page"{{end}}><i class="ti ti-stack-2"></i>Queue</a>{{end}}{{if .ShowDMARC}}<a href="/outbound/dmarc"{{if eq .Page "dmarc"}} aria-current="page"{{end}}><i class="ti ti-report-analytics"></i>DMARC reports</a>{{end}}</div></div>
|
{{define "subnav"}}{{if and .ShowOutbound (eq .Section "outbound")}}<div class="sp-subnav"><div class="container"><a href="/outbound/domains"{{if eq .Page "domains"}} aria-current="page"{{end}}><i class="ti ti-world-upload"></i>Domains</a><a href="/outbound/log"{{if eq .Page "log"}} aria-current="page"{{end}}><i class="ti ti-list-details"></i>Log</a>{{if .ShowQueue}}<a href="/outbound/queue"{{if eq .Page "queue"}} aria-current="page"{{end}}><i class="ti ti-stack-2"></i>Queue</a>{{end}}{{if .ShowDMARC}}<a href="/outbound/dmarc"{{if eq .Page "dmarc"}} aria-current="page"{{end}}><i class="ti ti-report-analytics"></i>DMARC reports</a>{{end}}</div></div>
|
||||||
{{else if and .ShowInbound (eq .Section "inbound")}}<div class="sp-subnav"><div class="container"><a href="/inbound/domains"{{if eq .Page "domains"}} aria-current="page"{{end}}><i class="ti ti-world-download"></i>Domains</a></div></div>
|
{{else if and .ShowInbound (eq .Section "inbound")}}<div class="sp-subnav"><div class="container"><a href="/inbound/domains"{{if eq .Page "domains"}} aria-current="page"{{end}}><i class="ti ti-world-download"></i>Domains</a><a href="/inbound/log"{{if eq .Page "log"}} aria-current="page"{{end}}><i class="ti ti-list-details"></i>Log</a>{{if .ShowFilterLists}}<a href="/inbound/filter-lists"{{if eq .Page "filter-lists"}} aria-current="page"{{end}}><i class="ti ti-filter"></i>Filter lists</a>{{end}}</div></div>
|
||||||
{{else if and .ShowServer (eq .Section "server")}}<div class="sp-subnav"><div class="container"><a href="/server/health"{{if eq .Page "health"}} aria-current="page"{{end}}><i class="ti ti-heartbeat"></i>Health</a><a href="/server/log"{{if eq .Page "log"}} aria-current="page"{{end}}><i class="ti ti-file-text"></i>System log</a><a href="/server/backup"{{if eq .Page "backup"}} aria-current="page"{{end}}><i class="ti ti-archive"></i>Backup</a><a href="/server/users"{{if eq .Page "users"}} aria-current="page"{{end}}><i class="ti ti-users"></i>Users</a><a href="/server/settings"{{if eq .Page "settings"}} aria-current="page"{{end}}><i class="ti ti-settings"></i>Settings</a></div></div>
|
{{else if and .ShowServer (eq .Section "server")}}<div class="sp-subnav"><div class="container"><a href="/server/health"{{if eq .Page "health"}} aria-current="page"{{end}}><i class="ti ti-heartbeat"></i>Health</a><a href="/server/log"{{if eq .Page "log"}} aria-current="page"{{end}}><i class="ti ti-file-text"></i>System log</a><a href="/server/backup"{{if eq .Page "backup"}} aria-current="page"{{end}}><i class="ti ti-archive"></i>Backup</a><a href="/server/users"{{if eq .Page "users"}} aria-current="page"{{end}}><i class="ti ti-users"></i>Users</a><a href="/server/settings"{{if eq .Page "settings"}} aria-current="page"{{end}}><i class="ti ti-settings"></i>Settings</a></div></div>
|
||||||
{{end}}{{end}}
|
{{end}}{{end}}
|
||||||
|
|||||||
@@ -13,6 +13,10 @@
|
|||||||
<div class="field"><label class="label" for="send_log_retention_days">Keep outbound log rows, days</label>
|
<div class="field"><label class="label" for="send_log_retention_days">Keep outbound log rows, days</label>
|
||||||
<div class="control"><input class="input" id="send_log_retention_days" name="send_log_retention_days" type="number" min="7" max="365" step="1" value="{{.RetentionDays}}" required></div>
|
<div class="control"><input class="input" id="send_log_retention_days" name="send_log_retention_days" type="number" min="7" max="365" step="1" value="{{.RetentionDays}}" required></div>
|
||||||
<p class="help">The main driver of <code>/data</code> growth. Does not affect <code>mail.log</code> rotation (fourteen daily files).</p></div>
|
<p class="help">The main driver of <code>/data</code> growth. Does not affect <code>mail.log</code> rotation (fourteen daily files).</p></div>
|
||||||
|
{{if .Inbound}}<div class="field"><label class="label" for="inbound_spam_log_retention_days">Keep inbound log rows, days</label>
|
||||||
|
<div class="control"><input class="input" id="inbound_spam_log_retention_days" name="inbound_spam_log_retention_days" type="number" min="7" max="365" step="1" value="{{.InboundRetentionDays}}" required></div>
|
||||||
|
<p class="help">The inbound log also never holds more than 10 000 rows: refused mail is logged at the rate strangers knock on port 25.</p></div>
|
||||||
|
{{end}}
|
||||||
{{template "body_close"}}
|
{{template "body_close"}}
|
||||||
{{template "box_close"}}
|
{{template "box_close"}}
|
||||||
</div>
|
</div>
|
||||||
|
|||||||
@@ -54,6 +54,8 @@ var pageFiles = map[string][]string{
|
|||||||
"in-domains": {"templates/in_domains.html"},
|
"in-domains": {"templates/in_domains.html"},
|
||||||
"in-domain": {"templates/in_domain.html"},
|
"in-domain": {"templates/in_domain.html"},
|
||||||
"in-domain-delete": {"templates/in_domain_delete.html"},
|
"in-domain-delete": {"templates/in_domain_delete.html"},
|
||||||
|
"in-log": {"templates/in_log.html"},
|
||||||
|
"in-filter-lists": {"templates/in_filter_lists.html"},
|
||||||
"out-log": {"templates/out_log.html", "templates/out_log_rows.html"},
|
"out-log": {"templates/out_log.html", "templates/out_log_rows.html"},
|
||||||
"out-message": {"templates/out_message.html"},
|
"out-message": {"templates/out_message.html"},
|
||||||
"dmarc": {"templates/dmarc.html"},
|
"dmarc": {"templates/dmarc.html"},
|
||||||
|
|||||||
+14
-2
@@ -97,6 +97,9 @@ type Config struct {
|
|||||||
DMARCEnabled bool
|
DMARCEnabled bool
|
||||||
// SendLogRetentionEnvDefault is SEND_LOG_RETENTION_DAYS at panel start.
|
// SendLogRetentionEnvDefault is SEND_LOG_RETENTION_DAYS at panel start.
|
||||||
SendLogRetentionEnvDefault int
|
SendLogRetentionEnvDefault int
|
||||||
|
// InboundSpamLogRetentionEnvDefault is INBOUND_SPAM_LOG_RETENTION_DAYS at
|
||||||
|
// panel start.
|
||||||
|
InboundSpamLogRetentionEnvDefault int
|
||||||
}
|
}
|
||||||
|
|
||||||
// Server is the panel HTTP application.
|
// Server is the panel HTTP application.
|
||||||
@@ -111,7 +114,7 @@ type Server struct {
|
|||||||
// that owns DKIM keys and the OpenDKIM tables (architecture.md § OpenDKIM);
|
// that owns DKIM keys and the OpenDKIM tables (architecture.md § OpenDKIM);
|
||||||
// apps owns application SASL accounts and the Postfix sender map
|
// apps owns application SASL accounts and the Postfix sender map
|
||||||
// (architecture.md § Mail path).
|
// (architecture.md § Mail path).
|
||||||
func New(st *store.Store, domains *domain.Service, apps *app.Service, inboundSvc *inbound.Service, dmarcSvc *dmarc.Service, cfg Config, setupTokenPath string) (*Server, error) {
|
func New(st *store.Store, domains *domain.Service, apps *app.Service, inboundSvc *inbound.Service, lists *inbound.Lists, dmarcSvc *dmarc.Service, cfg Config, setupTokenPath string) (*Server, error) {
|
||||||
v, err := view.New(cfg.Version)
|
v, err := view.New(cfg.Version)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
@@ -124,7 +127,7 @@ func New(st *store.Store, domains *domain.Service, apps *app.Service, inboundSvc
|
|||||||
SessionIdleDays: cfg.SessionIdleDays,
|
SessionIdleDays: cfg.SessionIdleDays,
|
||||||
TrustedProxyCIDRs: cfg.TrustedProxyCIDRs,
|
TrustedProxyCIDRs: cfg.TrustedProxyCIDRs,
|
||||||
}, v, setupTokenPath)
|
}, v, setupTokenPath)
|
||||||
h := handlers.New(st, domains, apps, inboundSvc, dmarcSvc, handlers.Config{
|
h := handlers.New(st, domains, apps, inboundSvc, lists, dmarcSvc, handlers.Config{
|
||||||
Hostname: cfg.Hostname,
|
Hostname: cfg.Hostname,
|
||||||
SubmissionEnabled: cfg.SubmissionEnabled,
|
SubmissionEnabled: cfg.SubmissionEnabled,
|
||||||
MailLogPath: cfg.MailLogPath,
|
MailLogPath: cfg.MailLogPath,
|
||||||
@@ -143,6 +146,8 @@ func New(st *store.Store, domains *domain.Service, apps *app.Service, inboundSvc
|
|||||||
InboundAntispamAction: cfg.InboundAntispamAction,
|
InboundAntispamAction: cfg.InboundAntispamAction,
|
||||||
DMARCEnabled: cfg.DMARCEnabled,
|
DMARCEnabled: cfg.DMARCEnabled,
|
||||||
SendLogRetentionEnvDefault: cfg.SendLogRetentionEnvDefault,
|
SendLogRetentionEnvDefault: cfg.SendLogRetentionEnvDefault,
|
||||||
|
|
||||||
|
InboundSpamLogRetentionEnvDefault: cfg.InboundSpamLogRetentionEnvDefault,
|
||||||
}, v, dnscheck.New(cfg.DNSResolvers), &health.MachineSampler{}, a)
|
}, v, dnscheck.New(cfg.DNSResolvers), &health.MachineSampler{}, a)
|
||||||
return &Server{cfg: cfg, auth: a, handlers: h}, nil
|
return &Server{cfg: cfg, auth: a, handlers: h}, nil
|
||||||
}
|
}
|
||||||
@@ -240,6 +245,13 @@ func (s *Server) muxes() (public, authed *http.ServeMux) {
|
|||||||
authed.HandleFunc("POST /inbound/domains/{id}/recipients", h.HandleInboundRecipients)
|
authed.HandleFunc("POST /inbound/domains/{id}/recipients", h.HandleInboundRecipients)
|
||||||
authed.HandleFunc("GET /inbound/domains/{id}/delete", h.HandleInboundDeleteConfirm)
|
authed.HandleFunc("GET /inbound/domains/{id}/delete", h.HandleInboundDeleteConfirm)
|
||||||
authed.HandleFunc("POST /inbound/domains/{id}/delete", h.HandleInboundDelete)
|
authed.HandleFunc("POST /inbound/domains/{id}/delete", h.HandleInboundDelete)
|
||||||
|
// The log is for everyone who reaches an inbound domain (scoped to
|
||||||
|
// theirs); the lists are the instance's, so the handlers keep the
|
||||||
|
// global role's check (requireInboundGlobal) and answer 404 to the rest.
|
||||||
|
authed.HandleFunc("GET /inbound/log", h.HandleInboundLog)
|
||||||
|
authed.HandleFunc("GET /inbound/filter-lists", h.HandleFilterLists)
|
||||||
|
authed.HandleFunc("POST /inbound/filter-lists", h.HandleFilterListAdd)
|
||||||
|
authed.HandleFunc("POST /inbound/filter-lists/{id}/delete", h.HandleFilterListDelete)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Server: everything under /server/ is for the global role, so the subtree
|
// Server: everything under /server/ is for the global role, so the subtree
|
||||||
|
|||||||
+26
-2
@@ -70,6 +70,12 @@ const (
|
|||||||
recipient = "rcpt@sink.e2e.test"
|
recipient = "rcpt@sink.e2e.test"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
// The one administrator of the run.
|
||||||
|
const (
|
||||||
|
adminUser = "e2eadmin"
|
||||||
|
adminPassword = "correct-horse-battery-staple"
|
||||||
|
)
|
||||||
|
|
||||||
// scenario carries state forward between the ordered subtests of TestE2E:
|
// scenario carries state forward between the ordered subtests of TestE2E:
|
||||||
// each later step needs something an earlier one produced (the domain id, the
|
// each later step needs something an earlier one produced (the domain id, the
|
||||||
// DKIM record, the application credentials, an observed client IP).
|
// DKIM record, the application credentials, an observed client IP).
|
||||||
@@ -131,10 +137,10 @@ func TestE2E(t *testing.T) {
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatal(err)
|
t.Fatal(err)
|
||||||
}
|
}
|
||||||
if err := p.setup(token, "e2eadmin", "correct-horse-battery-staple"); err != nil {
|
if err := p.setup(token, adminUser, adminPassword); err != nil {
|
||||||
t.Fatalf("setup: %v", err)
|
t.Fatalf("setup: %v", err)
|
||||||
}
|
}
|
||||||
if err := p.login("e2eadmin", "correct-horse-battery-staple"); err != nil {
|
if err := p.login(adminUser, adminPassword); err != nil {
|
||||||
t.Fatalf("login: %v", err)
|
t.Fatalf("login: %v", err)
|
||||||
}
|
}
|
||||||
sc.panel = p
|
sc.panel = p
|
||||||
@@ -268,6 +274,24 @@ func TestE2E(t *testing.T) {
|
|||||||
run("negative_level1_ratelimit", func(t *testing.T) {
|
run("negative_level1_ratelimit", func(t *testing.T) {
|
||||||
testLevel1RateLimit(t, sc)
|
testLevel1RateLimit(t, sc)
|
||||||
})
|
})
|
||||||
|
|
||||||
|
// From here on the installation is no longer the one the run started: it
|
||||||
|
// is restored from its own backup, then switched to accept mail on port 25.
|
||||||
|
run("full_backup_restore", func(t *testing.T) {
|
||||||
|
testFullBackupRestore(t, sc)
|
||||||
|
})
|
||||||
|
run("enable_port25_features", func(t *testing.T) {
|
||||||
|
testEnablePort25Features(t, sc)
|
||||||
|
})
|
||||||
|
run("dmarc_report_reaches_the_panel", func(t *testing.T) {
|
||||||
|
testDMARCReportReachesThePanel(t, sc)
|
||||||
|
})
|
||||||
|
run("dmarc_junk_is_dropped", func(t *testing.T) {
|
||||||
|
testDMARCJunkIsDropped(t, sc)
|
||||||
|
})
|
||||||
|
run("inbound_relay_delivers_and_refuses", func(t *testing.T) {
|
||||||
|
testInboundRelay(t, sc)
|
||||||
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
// trimZoneSuffix strips the "e2e.test" suffix a fully-qualified DKIM record
|
// trimZoneSuffix strips the "e2e.test" suffix a fully-qualified DKIM record
|
||||||
|
|||||||
@@ -52,6 +52,17 @@ func testLevel2RateLimit(t *testing.T, sc *scenario) {
|
|||||||
second.dialErr, second.authErr, second.rcptErr, second.dataErr)
|
second.dialErr, second.authErr, second.rcptErr, second.dataErr)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// The limit holds for every message of a connection, not for its first
|
||||||
|
// only. The milter library hands the milter a fresh session after any reply
|
||||||
|
// that is not "continue" - the refusal above is one - and Postfix does not
|
||||||
|
// introduce the client again, so a second MAIL FROM on the same connection
|
||||||
|
// used to arrive with no client address and was let through (found by the
|
||||||
|
// technical review of 2026-10-12; the same hole let an application past its
|
||||||
|
// client-address list).
|
||||||
|
if errs := mailTwiceOnOneConnection(login, password, from); errs[0] == nil || errs[1] == nil {
|
||||||
|
t.Fatalf("over the level-2 limit, two MAIL FROM on one connection were answered %v and %v, want both refused", errs[0], errs[1])
|
||||||
|
}
|
||||||
|
|
||||||
if err := waitFor("a rejected row for l2app in the send log", 15*time.Second, 500*time.Millisecond, func() (bool, error) {
|
if err := waitFor("a rejected row for l2app in the send log", 15*time.Second, 500*time.Millisecond, func() (bool, error) {
|
||||||
rows, err := sc.panel.sendLogRows(senderDomain, login)
|
rows, err := sc.panel.sendLogRows(senderDomain, login)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
|||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user