from-header-binding: the sender check, what it costs, its pages; guide, architecture, security
test / test (push) Canceled after 0s
test / test (push) Canceled after 0s
A message's From: line is held against the application that sends it, under the rule Postfix holds the envelope sender to. OpenDKIM chooses the key by that line, so until now an application could have mail signed as another domain of the server or as the server itself. The check (internal/milter/sender.go) - A role of the panel on a socket of its own (SENDER_CHECK_SOCKET), first in smtpd_milters and strict: default_action=tempfail. The journal milter keeps its socket and stays fail-open. postfix-wrapper waits for the new socket. - It reads no database: what an application may send as comes from Postfix's own sender_login_maps, what the server signs for from OpenDKIM's SigningTable. Both are read again when they change; one that cannot be read defers mail. - One From: field, one address, allowed by the envelope's lookup; then no other name of the SigningTable may appear anywhere in the raw line. Refused: 550 5.7.1 naming the line, reduced to printable ASCII. - A refusal is a send-log row with reason "from" and the header address (migration 0006), written by another goroutine and not waited for. The journal's refusals record their reason too (limit, client), and the notifier takes only a limit's. What it costs (docs/plans/from-header-binding.md, Statistics) - The role marks itself alive every 2 s; at start the gap since the mark is downtime, counted from Postfix's start if later and not at all when supervisord says Postfix is not running. - The log tailer counts the connections Postfix turned away: a warning that names the socket, then milter-reject "Service unavailable" from the same smtpd. Seen on the stand: the client is refused from CONNECT on and leaves at AUTH (454), so no MAIL line is ever written. - sender_check_days, a row per UTC day, kept a year. Pages - Health: the card "Sender check", the socket's row, and the cards layout - boxes numbered by importance in two stacks read in rows; the left stack polls, the three readings of the right one are swapped in by id. - Outbound log: a refused row says why in its tag (wrong From, wrong client, rate-limited); the message's page shows the From: line and what the application may send as. - Help: topic "Sender check", word for word the mockup's. Found by the stand, fixed here: the names for the second look came from the sender bindings, and a domain with a key and no application yet was not among them. e2e: negative_journal_milter_fail_open is gone - it proved that mail passes with the panel down, which is no longer so. In its place sender_check_holds_the_from_line (another domain's From:, the hostname's, a stranger's, an address off the list, another sender in the display name - refused; own addresses - delivered) and panel_down_defers_mail (4xx with the panel stopped, the container alive, mail again after, downtime and one deferred message on Health, the card warning). Test server, this tree before the last cosmetic edits (a fixture row, a fact's size, docs): go vet ok, go test 23 packages ok, make e2e ok (337 s). Locally after them: internal/web/view ok. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -97,6 +97,8 @@ RATE_WINDOW="${RATE_LIMIT_WINDOW_SECONDS:-3600}"
|
||||
# (monitoring, fail-open). Fixed container paths, matched by postfix-wrapper.sh.
|
||||
OPENDKIM_SOCK="${OPENDKIM_SOCKET:-/run/opendkim/opendkim.sock}"
|
||||
JOURNAL_SOCK="${JOURNAL_MILTER_SOCKET:-/run/selfpost/journal.sock}"
|
||||
# The panel's sender check (the From: line against the login, strict).
|
||||
SENDER_SOCK="${SENDER_CHECK_SOCKET:-/run/selfpost/sender.sock}"
|
||||
|
||||
# Persistent panel-managed sender map; texthash needs no postmap, so the
|
||||
# unprivileged panel can rewrite it and just ask for a reload.
|
||||
@@ -225,6 +227,11 @@ postconf -e \
|
||||
# monitoring only and is fail-open (default_action=accept): its failure must
|
||||
# never block the relay. Per-milter settings use Postfix 3.0+ brace syntax.
|
||||
#
|
||||
# The sender check comes first and is strict as well: it holds the From: line
|
||||
# against the login, which is what OpenDKIM then chooses the key by, so a
|
||||
# message it could not judge must not reach the signer. While the panel does
|
||||
# not answer on that socket, mail is deferred (docs/security.md).
|
||||
#
|
||||
# non_smtpd_milters is the chain for mail submitted locally with sendmail(1) -
|
||||
# the panel's own notifications, sent as postmaster@ the hostname. OpenDKIM
|
||||
# signs them with the key of the server's own name; the journal-milter is not
|
||||
@@ -233,7 +240,7 @@ postconf -e \
|
||||
postconf -e \
|
||||
"milter_protocol=6" \
|
||||
"milter_default_action=tempfail" \
|
||||
"smtpd_milters={ unix:${OPENDKIM_SOCK}, default_action=tempfail }, { unix:${JOURNAL_SOCK}, default_action=accept }" \
|
||||
"smtpd_milters={ unix:${SENDER_SOCK}, default_action=tempfail }, { unix:${OPENDKIM_SOCK}, default_action=tempfail }, { unix:${JOURNAL_SOCK}, default_action=accept }" \
|
||||
"non_smtpd_milters={ unix:${OPENDKIM_SOCK}, default_action=tempfail }"
|
||||
|
||||
# What Postfix tells a milter with MAIL FROM: its defaults plus the client's
|
||||
|
||||
@@ -2,8 +2,9 @@
|
||||
# Postfix start wrapper: solves the cold-start race where Postfix would try to
|
||||
# reach the milter sockets before they are listening.
|
||||
#
|
||||
# It blocks until the milter sockets — OpenDKIM, the panel's journal-milter
|
||||
# and, with the inbound relay on, its inbound journal milter — are present,
|
||||
# It blocks until the milter sockets — OpenDKIM, the panel's sender check and
|
||||
# journal-milter and, with the inbound relay on, its inbound journal milter —
|
||||
# are present,
|
||||
# then execs `postfix start-fg`. If they are not ready within the
|
||||
# timeout it exits non-zero WITHOUT starting Postfix, so supervisord/Docker see
|
||||
# an explicit startup failure instead of a relay running blind.
|
||||
@@ -14,10 +15,11 @@ set -eu
|
||||
|
||||
OPENDKIM_SOCK="${OPENDKIM_SOCKET:-/run/opendkim/opendkim.sock}"
|
||||
JOURNAL_SOCK="${JOURNAL_MILTER_SOCKET:-/run/selfpost/journal.sock}"
|
||||
SENDER_SOCK="${SENDER_CHECK_SOCKET:-/run/selfpost/sender.sock}"
|
||||
TIMEOUT="${MILTER_WAIT_TIMEOUT:-30}"
|
||||
INTERVAL=1
|
||||
|
||||
SOCKS="$OPENDKIM_SOCK $JOURNAL_SOCK"
|
||||
SOCKS="$OPENDKIM_SOCK $JOURNAL_SOCK $SENDER_SOCK"
|
||||
if [ "${INBOUND_RELAY_ENABLE:-false}" = "true" ]; then
|
||||
SOCKS="$SOCKS ${INBOUND_JOURNAL_MILTER_SOCKET:-/run/selfpost/inbound-journal.sock}"
|
||||
fi
|
||||
|
||||
@@ -32,6 +32,7 @@ var documentedInternal = []string{
|
||||
"SELFPOST_SETUP_TOKEN_FILE",
|
||||
"PANEL_HTTP_ADDR",
|
||||
"JOURNAL_MILTER_SOCKET",
|
||||
"SENDER_CHECK_SOCKET",
|
||||
"MAIL_LOG",
|
||||
"PANEL_COOKIE_SECURE",
|
||||
"OPENDKIM_SOCKET",
|
||||
@@ -70,6 +71,7 @@ var loadConfigKeys = []string{
|
||||
"SELFPOST_DATA_DIR",
|
||||
"PANEL_HTTP_ADDR",
|
||||
"JOURNAL_MILTER_SOCKET",
|
||||
"SENDER_CHECK_SOCKET",
|
||||
"MAIL_LOG",
|
||||
"SEND_LOG_RETENTION_DAYS",
|
||||
"SELFPOST_DB_PATH",
|
||||
@@ -104,6 +106,7 @@ var buildScriptKeys = []string{
|
||||
"RATE_LIMIT_WINDOW_SECONDS",
|
||||
"OPENDKIM_SOCKET",
|
||||
"JOURNAL_MILTER_SOCKET",
|
||||
"SENDER_CHECK_SOCKET",
|
||||
"MAIL_LOG",
|
||||
"POSTFIX_SENDER_LOGIN_MAPS",
|
||||
"POSTFIX_QUEUE_DIR",
|
||||
|
||||
@@ -114,6 +114,7 @@ func newPanel(cfg config, st *store.Store) (*web.Server, error) {
|
||||
TLSCertFile: cfg.tlsCertFile,
|
||||
OpenDKIMSocket: cfg.opendkimSocket,
|
||||
JournalSocket: cfg.journalSocket,
|
||||
SenderSocket: cfg.senderSocket,
|
||||
SessionIdleDays: cfg.sessionIdleDays,
|
||||
DNSResolvers: cfg.dnsResolvers,
|
||||
RateLimitMessagesPerIP: cfg.rateLimitMessagesPerIP,
|
||||
|
||||
+7
-3
@@ -60,6 +60,8 @@ func main() {
|
||||
type config struct {
|
||||
httpAddr string
|
||||
journalSocket string
|
||||
// The sender check's socket; the default mirrors build/postfix-config.sh.
|
||||
senderSocket string
|
||||
mailLog string
|
||||
retentionDays int
|
||||
|
||||
@@ -113,6 +115,7 @@ func loadConfig() config {
|
||||
return config{
|
||||
httpAddr: envDefault("PANEL_HTTP_ADDR", ":8080"),
|
||||
journalSocket: envDefault("JOURNAL_MILTER_SOCKET", "/run/selfpost/journal.sock"),
|
||||
senderSocket: envDefault("SENDER_CHECK_SOCKET", "/run/selfpost/sender.sock"),
|
||||
// Postfix's delivery log, under /data so the lines that resolve a
|
||||
// "queued" send-log row outlive the container. The default must match
|
||||
// maillog_file in build/postfix-config.sh.
|
||||
@@ -341,8 +344,8 @@ func run() error {
|
||||
}
|
||||
|
||||
var wg sync.WaitGroup
|
||||
// Room for every role to report a failure without blocking: six at most.
|
||||
errc := make(chan error, 6)
|
||||
// Room for every role to report a failure without blocking: seven at most.
|
||||
errc := make(chan error, 7)
|
||||
|
||||
roles := []struct {
|
||||
name string
|
||||
@@ -350,6 +353,7 @@ func run() error {
|
||||
}{
|
||||
{"http", func(ctx context.Context) error { return serveHTTP(ctx, cfg, st) }},
|
||||
{"journal-milter", func(ctx context.Context) error { return serveJournal(ctx, cfg, st) }},
|
||||
{"sender-check", func(ctx context.Context) error { return serveSenderCheck(ctx, cfg, st) }},
|
||||
{"log-tailer", func(ctx context.Context) error {
|
||||
return logtail.Run(ctx, cfg.mailLog, st, func() int {
|
||||
days, err := st.GetSendLogRetentionDays(cfg.retentionDays)
|
||||
@@ -358,7 +362,7 @@ func run() error {
|
||||
return cfg.retentionDays
|
||||
}
|
||||
return days
|
||||
}, inboundTail)
|
||||
}, inboundTail, &logtail.SenderCheck{Store: st, Socket: cfg.senderSocket})
|
||||
}},
|
||||
{"rate-limit-recalc", func(ctx context.Context) error {
|
||||
runAutoRateLimitRecalc(ctx, cfg, st)
|
||||
|
||||
@@ -0,0 +1,124 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"log"
|
||||
"os"
|
||||
"time"
|
||||
|
||||
"github.com/mixeme/selfpost/internal/domain"
|
||||
"github.com/mixeme/selfpost/internal/health"
|
||||
"github.com/mixeme/selfpost/internal/milter"
|
||||
"github.com/mixeme/selfpost/internal/postfix"
|
||||
"github.com/mixeme/selfpost/internal/store"
|
||||
)
|
||||
|
||||
// aliveEvery is how often the sender check marks itself alive. The gap a
|
||||
// restart finds since the last mark is the check's downtime, so the mark's
|
||||
// age is the error of that figure.
|
||||
const aliveEvery = 2 * time.Second
|
||||
|
||||
// serveSenderCheck runs the sender check (internal/milter/sender.go) on a
|
||||
// socket of its own. Postfix attaches that socket strictly, so while this
|
||||
// role is not serving, mail is deferred: what that costs is counted here and
|
||||
// by the log tailer, and shown on Server > Health
|
||||
// (docs/plans/from-header-binding.md § Statistics).
|
||||
func serveSenderCheck(ctx context.Context, cfg config, st *store.Store) error {
|
||||
ln, err := listenMilter(cfg.senderSocket)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
log.Printf("sender-check listening on %s", cfg.senderSocket)
|
||||
|
||||
// Statistics are the database's business and the check is not: none of
|
||||
// this may keep the check from serving.
|
||||
now := time.Now()
|
||||
mark := cfg.senderSocket + ".alive"
|
||||
if err := st.StartSenderCheckStats(now); err != nil {
|
||||
log.Printf("sender-check: statistics: %v", err)
|
||||
}
|
||||
if down := senderCheckDowntime(mark, now, postfixStarted); down > 0 {
|
||||
log.Printf("sender-check: was not answering for %s while Postfix was running", down.Round(time.Second))
|
||||
if err := st.AddSenderCheckDown(now, down); err != nil {
|
||||
log.Printf("sender-check: statistics: %v", err)
|
||||
}
|
||||
}
|
||||
go keepAlive(ctx, mark, st)
|
||||
|
||||
// The two tables the mail path itself goes by: whom Postfix lets send as
|
||||
// what, and what OpenDKIM signs.
|
||||
bind := milter.NewBindings(postfix.New(cfg.postfixDir).SenderLoginMapsPath())
|
||||
signers := milter.NewSigners(domain.NewOpenDKIM(cfg.opendkimDir).SigningTablePath())
|
||||
return milter.ServeSender(ctx, ln, bind, signers, st)
|
||||
}
|
||||
|
||||
// senderCheckDowntime is for how long the check was not serving before now
|
||||
// while Postfix was: the time since the mark at path was last set, or since
|
||||
// Postfix started if that was later. No mark means the container is new, and
|
||||
// a Postfix that is not running means mail was stopped for another reason -
|
||||
// a rollout, a reboot - and neither is the check's downtime.
|
||||
func senderCheckDowntime(path string, now time.Time, postfixStarted func() (time.Time, bool)) time.Duration {
|
||||
fi, err := os.Stat(path)
|
||||
if err != nil {
|
||||
return 0
|
||||
}
|
||||
from := fi.ModTime()
|
||||
started, running := postfixStarted()
|
||||
if !running {
|
||||
return 0
|
||||
}
|
||||
if started.After(from) {
|
||||
from = started
|
||||
}
|
||||
if d := now.Sub(from); d > 0 {
|
||||
return d
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
// postfixStarted asks supervisord since when Postfix has been running.
|
||||
func postfixStarted() (time.Time, bool) {
|
||||
procs, err := health.Processes()
|
||||
if err != nil {
|
||||
return time.Time{}, false
|
||||
}
|
||||
for _, p := range procs {
|
||||
if p.Name != "postfix" || p.State != "RUNNING" {
|
||||
continue
|
||||
}
|
||||
if up, ok := health.Uptime(p.Detail); ok {
|
||||
return time.Now().Add(-up), true
|
||||
}
|
||||
}
|
||||
return time.Time{}, false
|
||||
}
|
||||
|
||||
// keepAlive sets the mark until ctx ends, and once a day drops the days of
|
||||
// the statistics that are older than a year.
|
||||
func keepAlive(ctx context.Context, path string, st *store.Store) {
|
||||
touch := func() {
|
||||
now := time.Now()
|
||||
if err := os.Chtimes(path, now, now); err != nil {
|
||||
if err := os.WriteFile(path, nil, 0o644); err != nil {
|
||||
log.Printf("sender-check: alive mark: %v", err)
|
||||
}
|
||||
}
|
||||
}
|
||||
touch()
|
||||
tick := time.NewTicker(aliveEvery)
|
||||
defer tick.Stop()
|
||||
prune := time.NewTicker(24 * time.Hour)
|
||||
defer prune.Stop()
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case <-tick.C:
|
||||
touch()
|
||||
case <-prune.C:
|
||||
if err := st.PruneSenderCheckDays(time.Now()); err != nil {
|
||||
log.Printf("sender-check: statistics: %v", err)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,47 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// The check's downtime is the time since its last mark, under a Postfix that
|
||||
// was running: a new container has no mark, and a Postfix that is not up, or
|
||||
// came up later than the mark, was not kept waiting by the check for all of
|
||||
// that time.
|
||||
func TestSenderCheckDowntime(t *testing.T) {
|
||||
now := time.Date(2026, 10, 12, 18, 0, 0, 0, time.UTC)
|
||||
mark := filepath.Join(t.TempDir(), "sender.sock.alive")
|
||||
set := func(at time.Time) {
|
||||
t.Helper()
|
||||
if err := os.WriteFile(mark, nil, 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.Chtimes(mark, at, at); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
postfix := func(started time.Time, running bool) func() (time.Time, bool) {
|
||||
return func() (time.Time, bool) { return started, running }
|
||||
}
|
||||
|
||||
if d := senderCheckDowntime(mark, now, postfix(now.Add(-time.Hour), true)); d != 0 {
|
||||
t.Errorf("no mark: %s", d)
|
||||
}
|
||||
set(now.Add(-40 * time.Second))
|
||||
if d := senderCheckDowntime(mark, now, postfix(now.Add(-time.Hour), true)); d != 40*time.Second {
|
||||
t.Errorf("Postfix up throughout: %s, want 40s", d)
|
||||
}
|
||||
if d := senderCheckDowntime(mark, now, postfix(now.Add(-10*time.Second), true)); d != 10*time.Second {
|
||||
t.Errorf("Postfix started within the gap: %s, want 10s", d)
|
||||
}
|
||||
if d := senderCheckDowntime(mark, now, postfix(time.Time{}, false)); d != 0 {
|
||||
t.Errorf("Postfix not running: %s", d)
|
||||
}
|
||||
set(now.Add(time.Minute)) // a clock set back
|
||||
if d := senderCheckDowntime(mark, now, postfix(now.Add(-time.Hour), true)); d != 0 {
|
||||
t.Errorf("a mark in the future: %s", d)
|
||||
}
|
||||
}
|
||||
+88
-15
@@ -27,13 +27,13 @@ Managed programs ([build/supervisord.conf](../build/supervisord.conf)):
|
||||
| Program | User | Priority | Role |
|
||||
|---|---|---|---|
|
||||
| `opendkim` | root → `opendkim` (`UserID` in opendkim.conf) | 100 | DKIM signing milter |
|
||||
| `panel` | panel | 200 | HTTP UI + journal-milter + log-tailer + rate-limit recalc; with the inbound relay on also the inbound journal milter |
|
||||
| `postfix` | root (wrapper) | 300 | MTA — started only after the milter sockets exist (OpenDKIM's, the journal milter's and, with the inbound relay on, the inbound journal milter's) |
|
||||
| `panel` | panel | 200 | HTTP UI + sender check + journal-milter + log-tailer + rate-limit recalc; with the inbound relay on also the inbound journal milter |
|
||||
| `postfix` | root (wrapper) | 300 | MTA — started only after the milter sockets exist (OpenDKIM's, the sender check's, the journal milter's and, with the inbound relay on, the inbound journal milter's) |
|
||||
| `postfix-reload` | root | — | On-demand `postfix reload` (autostart off) |
|
||||
| `cert-reload` | root | 400 | Daily `postfix reload` for renewed TLS certs |
|
||||
| `logrotate` | root | 400 | Periodic `mail.log` rotation |
|
||||
|
||||
Start order: OpenDKIM → panel (opens journal-milter socket) → Postfix wrapper
|
||||
Start order: OpenDKIM → panel (opens the sender check's and the journal-milter's sockets) → Postfix wrapper
|
||||
polls unix sockets (timeout `MILTER_WAIT_TIMEOUT`, default 30s) then
|
||||
`postfix start-fg`.
|
||||
|
||||
@@ -50,6 +50,7 @@ panel, and postfix are RUNNING; Docker `HEALTHCHECK` uses the same probe.
|
||||
```
|
||||
Client ──TLS+SASL──► Postfix (465 smtps, optional 587 submission)
|
||||
│
|
||||
├─► sender check (From: against the login, tempfail on failure)
|
||||
├─► OpenDKIM milter (sign, tempfail on failure)
|
||||
├─► journal-milter (send log + L2 rate limits, fail-open)
|
||||
└─► outbound MX delivery (port 25 client)
|
||||
@@ -144,7 +145,9 @@ smtp/inet` removes Debian's stock smtpd). Outbound delivery still uses the
|
||||
`INBOUND_RATE_LIMIT_MESSAGES_PER_IP` and message size uses the greater of
|
||||
`INBOUND_MESSAGE_SIZE_LIMIT` and `DMARC_MESSAGE_SIZE_LIMIT`.
|
||||
- **No open relay** — `permit_sasl_authenticated`, `reject_unauth_destination`;
|
||||
`smtpd_sender_login_maps` + `reject_sender_login_mismatch`.
|
||||
`smtpd_sender_login_maps` + `reject_sender_login_mismatch`. That holds the
|
||||
envelope sender; the `From:` header is held to the same table by the
|
||||
panel's sender check (§ Panel roles).
|
||||
- **Level-1 rate limit** — `smtpd_client_message_rate_limit` /
|
||||
`anvil_rate_time_unit` from `RATE_LIMIT_*` env vars; independent of milter.
|
||||
- **Chroot disabled** for all services (DNS/TLS inside container).
|
||||
@@ -183,7 +186,9 @@ One process, five roles (six with the inbound relay on):
|
||||
numbers; they never call `postconf` per request. If `postconf` is missing,
|
||||
the panel logs a warning and uses Postfix 3.x compiled-in defaults
|
||||
(`300s` / `4000s` / `5d` / `0`) with a muted note on the box.
|
||||
2. **journal-milter** — unix socket `JOURNAL_MILTER_SOCKET`; records From/To/
|
||||
2. **sender check** — unix socket `SENDER_CHECK_SOCKET`, first in the chain
|
||||
and **strict** (`default_action=tempfail`): § Sender check below.
|
||||
3. **journal-milter** — unix socket `JOURNAL_MILTER_SOCKET`; records From/To/
|
||||
Subject/SASL user at DATA; enforces level-2 rate limits; **fail-open**
|
||||
(`default_action=accept`) so milter failure does not stop mail. Domain
|
||||
ceilings apply to every client IP; an application ceiling **overrides** the
|
||||
@@ -195,20 +200,85 @@ One process, five roles (six with the inbound relay on):
|
||||
this process has admitted but not yet written (`internal/milter/inflight.go`),
|
||||
so concurrent sessions cannot each spend the same last slot; a reservation
|
||||
is released at end-of-message, on ABORT, or after a 10-minute TTL.
|
||||
3. **log-tailer** — follows `MAIL_LOG`, updates send-log delivery status by
|
||||
4. **log-tailer** — follows `MAIL_LOG`, updates send-log delivery status by
|
||||
queue-id. Send-log `queued → sent` transitions depend on this goroutine alone
|
||||
(`UpdateStatus` is only called from [internal/logtail](../internal/logtail/logtail.go)).
|
||||
It also keeps the lines Postfix writes under a queue id, for the message's
|
||||
page (§ Log tailer, *Stored lines*).
|
||||
4. **rate-limit recalc** — every six hours (and on demand from Domain settings and
|
||||
page (§ Log tailer, *Stored lines*), and counts the mail Postfix deferred
|
||||
because the sender check did not answer.
|
||||
5. **rate-limit recalc** — every six hours (and on demand from Domain settings and
|
||||
the application form), recomputes level-2 **Auto** rate limits from send-log statistics.
|
||||
5. **notifier** — writes to the panel's users when something needs them
|
||||
6. **notifier** — writes to the panel's users when something needs them
|
||||
(§ Notifications below).
|
||||
|
||||
Milter chain in Postfix: OpenDKIM (tempfail) then journal (accept on failure).
|
||||
Mail submitted locally with `sendmail(1)` — the notifier's — passes OpenDKIM
|
||||
alone (`non_smtpd_milters`): it is signed, and it is not an application's
|
||||
mail to be journalled or limited. Bounces Postfix generates pass no milter.
|
||||
Milter chain in Postfix: the sender check (tempfail), OpenDKIM (tempfail),
|
||||
then the journal (accept on failure). Mail submitted locally with
|
||||
`sendmail(1)` — the notifier's — passes OpenDKIM alone (`non_smtpd_milters`):
|
||||
it is signed, it is the panel's own and so needs no check of whose it is, and
|
||||
it is not an application's mail to be journalled or limited. Bounces Postfix
|
||||
generates pass no milter.
|
||||
|
||||
### Sender check
|
||||
|
||||
[internal/milter/sender.go](../internal/milter/sender.go) holds a message's
|
||||
`From:` header against the application that sends it
|
||||
([plans/from-header-binding.md](plans/from-header-binding.md)). Postfix holds
|
||||
the envelope sender against the SASL login, and OpenDKIM chooses the signing
|
||||
key by the header; until `2.4.0` nothing connected the two, and an
|
||||
application could have mail signed as another domain of the server or as the
|
||||
server itself.
|
||||
|
||||
- **The rule is the envelope's.** The check reads the file Postfix reads for
|
||||
`smtpd_sender_login_maps` and makes the same lookup: the address itself is
|
||||
bound to the login, or `@domain` is. So an application set to any address of
|
||||
its domain may write any address of that domain, and one with a list only
|
||||
the listed ones; everything else is refused - another registered domain,
|
||||
the server's name, a domain the server knows nothing of. One source for
|
||||
both senders: the two cannot disagree.
|
||||
- **How the header is read.** Exactly one `From:` field (none, or more than
|
||||
one, is refused), parsed as an address list that must hold exactly one
|
||||
address. Then a cruder second look at the raw value: OpenDKIM finds the
|
||||
address with a parser of its own, and wherever the two could read a line
|
||||
differently - a display name, a comment, a quoted string - the other
|
||||
address would have to be spelled out in it. So no `@name` may appear there
|
||||
that is another name the server signs for; those names are read from
|
||||
OpenDKIM's own `SigningTable`.
|
||||
- **The verdict** comes at the end of the headers: `550 5.7.1`, permanent,
|
||||
naming the line, as a reply code of the milter's own. What the client wrote
|
||||
is reduced to printable ASCII and cut before it is said back, logged or
|
||||
stored. A message with no SASL login is refused as well; the submission
|
||||
ports do not let one this far.
|
||||
- **Strict, and therefore small.** Postfix attaches the socket with
|
||||
`default_action=tempfail`: while the panel does not answer on it within the
|
||||
milter timeouts, mail is deferred, not accepted unchecked. The role
|
||||
therefore reads no database. Its two inputs are files the panel rewrites by
|
||||
rename and the mail path itself goes by; they are read again when their
|
||||
time or size has changed, and a file that cannot be read defers mail - a
|
||||
table from before would let a deleted application go on sending. The
|
||||
send-log row of a refusal (`status=rejected`, `reason=from`,
|
||||
`header_from`) is handed to another goroutine through a buffer and is not
|
||||
waited for.
|
||||
- **What it costs is counted.** The role marks itself alive every two
|
||||
seconds (the socket's path with `.alive`, under `/run`, so a new container
|
||||
starts without one). At start the time since the mark is the check's
|
||||
downtime - counted from Postfix's own start if that was later, and not at
|
||||
all when supervisord says Postfix is not running, since mail was then
|
||||
stopped for another reason. The log tailer counts the mail: Postfix
|
||||
writes a warning naming the socket and then, from the same smtpd process,
|
||||
`milter-reject: … 451 4.7.1 Service unavailable` from the connection on
|
||||
(the client is told `454 4.3.0` at `AUTH` and leaves), so what is counted
|
||||
is a connection turned away, once - a client brings a message when it
|
||||
connects. The reject line does not name the milter and the journal milter's
|
||||
level-2 refusal reads the same, hence the warning. Both go into
|
||||
`sender_check_days`, a
|
||||
row per UTC day kept a year, and Server › Health shows today, 30 days and
|
||||
12 months. The second figure also sees a panel that is running and not
|
||||
answering, which the mark does not. A log read twice after an unclean stop
|
||||
can count a message twice.
|
||||
- **Why it is not a process of its own.** Decided with the owner: measure
|
||||
first. The role has a socket of its own so that the journal stays
|
||||
fail-open, and moving it out later changes where it is started and nothing
|
||||
else.
|
||||
|
||||
### Notifications
|
||||
|
||||
@@ -229,7 +299,9 @@ is sent; the operator's view is guide § Notifications.
|
||||
- **A refused row is not believed whole.** Its domain is the one in
|
||||
`MAIL FROM`, which the client typed and Postfix has not yet held against
|
||||
the login when the milter refuses; the same status is written when an
|
||||
application is refused for its client address. So refusals are counted by
|
||||
application is refused for its client address or its `From:` line. Since
|
||||
`2.4.0` a row says which (`send_log.reason`) and only a limit's rows are
|
||||
taken; a row from before has no reason. So refusals are counted by
|
||||
the domain the login is registered under, only for logins that exist, and
|
||||
only while a level-2 limit is set on the login or its domain
|
||||
(`Store.LimitRefusalsSince`). One subject per sending domain, whichever of
|
||||
@@ -808,7 +880,8 @@ unsupported rather than as a missing doc:
|
||||
- **Panel paths and tuning:** `SELFPOST_DATA_DIR` (`/data`), `SELFPOST_DB_PATH`
|
||||
(`/data/selfpost.db`), `SELFPOST_SETUP_TOKEN_FILE`
|
||||
(`/data/setup-token`), `PANEL_HTTP_ADDR` (`:8080`),
|
||||
`JOURNAL_MILTER_SOCKET` (`/run/selfpost/journal.sock`), `MAIL_LOG`
|
||||
`JOURNAL_MILTER_SOCKET` (`/run/selfpost/journal.sock`),
|
||||
`SENDER_CHECK_SOCKET` (`/run/selfpost/sender.sock`), `MAIL_LOG`
|
||||
(`/data/log/mail.log` — read by the panel and written by Postfix, so a change
|
||||
here has to be matched in `build/postfix-config.sh`),
|
||||
`PANEL_COOKIE_SECURE` (`true`), `OPENDKIM_SOCKET`
|
||||
|
||||
+57
-10
@@ -377,7 +377,7 @@ and reverse DNS. A check the panel could not run shows as a warning, never as
|
||||
fine. Below the cards are the outbound domain list and, with the inbound relay
|
||||
on, the inbound one. Each card leads to its table on **Server › Health**
|
||||
(`/server/health`) — the queue card to the Outbound queue — and Health has the
|
||||
readings:
|
||||
readings, in the order of what stops mail first:
|
||||
|
||||
- **Machine** — processor (core and thread counts), memory and swap, and
|
||||
per-interface network throughput and totals, read from the kernel's
|
||||
@@ -386,7 +386,16 @@ readings:
|
||||
memory is a warning, because both delay or kill the mail path; throughput is
|
||||
only reported.
|
||||
- **Processes**, **TLS certificate** (validity and expiry) and **Milter
|
||||
sockets** (OpenDKIM and the journal-milter).
|
||||
sockets** (OpenDKIM, the sender check and the journal-milter).
|
||||
- **Sender check** — what the strict check of the `From:` line has cost
|
||||
today, in the last 30 days and in the last 12 months: for how long the
|
||||
panel did not answer while Postfix was running, and how many messages
|
||||
Postfix deferred because of it (see [Who an application may send
|
||||
as](#who-an-application-may-send-as)). A restart of the whole container is
|
||||
not counted: mail was stopped for another reason then. The card warns when
|
||||
a message was deferred in the last 24 hours. If these numbers do not stay
|
||||
at nothing, the panel is in the way of your mail more than it should be -
|
||||
report it.
|
||||
- **Hostname and reverse DNS** — compares `SELFPOST_HOSTNAME` against the PTR
|
||||
record the internet publishes for this server's IP (forward-confirmed
|
||||
reverse DNS), see [Server-level DNS](#server-level-dns-ptrrdns); use
|
||||
@@ -1076,6 +1085,38 @@ to a domain:
|
||||
level 2](#rate-limiting--level-2-domain-and-application)), *Export domain*
|
||||
and, for the global role, *Delete domain*, which asks once more.
|
||||
|
||||
#### Who an application may send as
|
||||
|
||||
An application sends as any address of its domain, or only as the addresses
|
||||
listed for it. Since `2.4.0` that rule covers both senders a message has:
|
||||
|
||||
- the **envelope sender** (`MAIL FROM`), which Postfix checks;
|
||||
- the **`From:` line** of the message - the one a mail program shows and the
|
||||
one the DKIM signature is chosen by - which the panel checks.
|
||||
|
||||
A message whose `From:` is anything else is refused with `550 5.7.1` and a
|
||||
text naming the line: another of your domains, the server's own name, an
|
||||
address at a domain that is not on this server, a line with two addresses,
|
||||
or no `From:` at all. A name this server signs for may not appear anywhere
|
||||
in the line besides the address itself, a display name included. The refusal
|
||||
is permanent, so the application does not send the message again; it is a
|
||||
row of the [Outbound log](#outbound-log) tagged `wrong From`, and the row's
|
||||
page shows what the line said.
|
||||
|
||||
What this stops working when you upgrade from an earlier version:
|
||||
|
||||
- an application that sends with the `From:` of another of your domains -
|
||||
give that domain an application of its own;
|
||||
- an application with a list whose envelope address and `From:` address
|
||||
differ - add the second address to the list;
|
||||
- a contact form that puts its visitor's address into `From:` - put it into
|
||||
`Reply-To:` and send from an address of your own, which is also what
|
||||
DMARC requires.
|
||||
|
||||
The check is strict: while the panel does not answer, Postfix replies `451`
|
||||
and applications send again later; nothing is accepted unchecked. **Server ›
|
||||
Health** counts how often that happens.
|
||||
|
||||
*Export domain* writes a single-domain file; *Import a domain* on *Server ›
|
||||
Backup* reads one back in (**global role only**) — see [Exporting and importing
|
||||
a single domain](#exporting-and-importing-a-single-domain).
|
||||
@@ -1149,8 +1190,8 @@ rate limits, and it takes exact addresses only — CIDR ranges are not parsed.
|
||||
|
||||
It is **not an authentication boundary.** The SASL login itself still succeeds;
|
||||
the check runs in the journal-milter, on the same fail-open path as level 2
|
||||
(below), so if the milter is unavailable the restriction is skipped rather than
|
||||
enforced. Use it to narrow where an application may send from, not to contain a
|
||||
(below), so if that milter does not answer the restriction is skipped rather
|
||||
than enforced. Use it to narrow where an application may send from, not to contain a
|
||||
leaked password — for that, regenerate the password.
|
||||
|
||||
**Level 2 — application** — optional override of the domain limit for one
|
||||
@@ -1173,10 +1214,12 @@ hours and on demand via **Recalculate now**.
|
||||
journal-milter and is deliberately fail-open: if the rate-limit lookup hits
|
||||
a store error, or the connecting client's IP is not available to the
|
||||
milter, level 2 is skipped and the message is accepted rather than held up.
|
||||
The same applies if the milter itself is down — Postfix is configured to
|
||||
accept rather than defer when it cannot reach it. Level 1 is the backstop
|
||||
that keeps working even when level 2 cannot run, and it is the only one of
|
||||
the two that does not depend on the panel process.
|
||||
The same applies if the journal-milter itself does not answer — Postfix is
|
||||
configured to accept rather than defer when it cannot reach it. (Since
|
||||
`2.4.0` a panel that is down altogether defers mail anyway, through the
|
||||
[sender check](#who-an-application-may-send-as) of the same process.) Level 1
|
||||
is the backstop that keeps working even when level 2 cannot run, and it is
|
||||
the only one of the two that does not depend on the panel process.
|
||||
|
||||
### Outbound log
|
||||
|
||||
@@ -1186,8 +1229,12 @@ rows of their domains. It refreshes its rows without resetting a filter being
|
||||
chosen. A row identifies its message and nothing more — time,
|
||||
sender, recipient, subject and status `queued` (accepted, not yet
|
||||
delivered), `sent` (handed off successfully), `deferred` (Postfix is
|
||||
retrying), `bounced` (final failure), or `rejected` (refused — for example
|
||||
by a [level-2 rate limit](#rate-limiting--level-2-domain-and-application));
|
||||
retrying), `bounced` (final failure), or refused before it was queued. A
|
||||
refused row says why in its tag: `rate-limited` (a [level-2 rate
|
||||
limit](#rate-limiting--level-2-domain-and-application)), `wrong client` (the
|
||||
application's client IP list) or `wrong From` (see [Who an application may
|
||||
send as](#who-an-application-may-send-as)); a row refused before `2.4.0`
|
||||
says `rejected`;
|
||||
*Details* opens that row's own page (`/outbound/log/{id}`). That page carries
|
||||
the sending domain, the application it was submitted under, the Postfix
|
||||
queue id and the journal id, beside the message's history — when it was
|
||||
|
||||
@@ -80,7 +80,8 @@ in the last 24 hours; downtime that deferred nothing warns of nothing.
|
||||
- **In the Outbound log:** a row with the reason in the status tag -
|
||||
`wrong From`, beside `rate-limited` and the tag of a refused client address.
|
||||
The built panel wrote `rejected` for all of them; it follows the mockup now.
|
||||
The status filter keeps one entry, `rejected`.
|
||||
(The mockup's status filter is not built - the log filters by domain and
|
||||
application - so there is nothing to keep in step with the tags.)
|
||||
- **On the message's page:** a fact *From: line* with the address written in
|
||||
the header, and a History step that says which address was refused, for
|
||||
which application, what that application may write, and that it was told
|
||||
@@ -101,7 +102,30 @@ the fourth was not.
|
||||
beside a tall one and the eye still goes 01, 02, 03. On a phone the boxes
|
||||
come in the order of their numbers.
|
||||
|
||||
## To settle while building
|
||||
## Settled while building
|
||||
|
||||
- **The header is read twice.** One `From:` field holding one address, by
|
||||
Go's parser; then every `@name` of the raw value is held against OpenDKIM's
|
||||
own `SigningTable`. The first build took the names for the second look from
|
||||
the sender bindings, and the e2e stand let a line through that named a
|
||||
domain with a key and no application yet: a domain is a signing name from
|
||||
the moment it is added, not from its first application.
|
||||
- **What Postfix writes when the check does not answer** (seen on the stand):
|
||||
a warning naming the socket, then `milter-reject: CONNECT … 451 4.7.1
|
||||
Service unavailable` and the same for `EHLO`; the client is told `454 4.3.0
|
||||
Try again later` at `AUTH` and leaves. No `MAIL` is ever offered, so what
|
||||
is counted is the connection turned away, once. The journal milter's
|
||||
level-2 refusal is written in the same words, which is why a reject line
|
||||
counts only after the warning.
|
||||
- **When Postfix started** is asked of supervisord (`supervisorctl status`,
|
||||
the uptime of `postfix`).
|
||||
- **The reply** is a milter reply code, `550 5.7.1 From: <line> - <why>`; what
|
||||
the client wrote is reduced to printable ASCII and cut first.
|
||||
- **The Delivery log box stays on a refused message's page**, with the one
|
||||
sentence it already had for a message that never reached the queue; the
|
||||
picture shown to the owner had left the box out.
|
||||
|
||||
## To settle while building (as first written)
|
||||
|
||||
- How the header is read so that the check and OpenDKIM cannot disagree: one
|
||||
`From:` field, one address, and a cruder second look at the raw value.
|
||||
@@ -116,11 +140,11 @@ the fourth was not.
|
||||
## Implementation checklist
|
||||
|
||||
- [x] Mockups: Health card and layout, the log row, help topic; accepted by the owner 2026-10-12 (`design:` commit)
|
||||
- [ ] Migration and store: refusal reason and header address, days of the sender check
|
||||
- [ ] The check: a role of the panel on its own socket, bindings read from Postfix's file; Postfix attaches it strictly; start order
|
||||
- [ ] Statistics: the alive mark and downtime, deferred mail from `mail.log`
|
||||
- [ ] Panel: Health (card, sockets row, layout), Outbound log tags, the message's page, help
|
||||
- [ ] guide.md, architecture.md, security.md (the accepted risk goes), schema-migrations.md
|
||||
- [ ] `go vet`, `go test`, e2e: refused for another domain's `From:`, for the hostname's and for an address off the list; accepted for its own
|
||||
- [x] Migration and store: refusal reason and header address, days of the sender check (`0006_sender_check.sql`)
|
||||
- [x] The check: a role of the panel on its own socket, bindings read from Postfix's file; Postfix attaches it strictly; start order
|
||||
- [x] Statistics: the alive mark and downtime, deferred mail from `mail.log`
|
||||
- [x] Panel: Health (card, sockets row, layout), Outbound log tags, the message's page, help
|
||||
- [x] guide.md, architecture.md, security.md (the accepted risk goes), schema-migrations.md
|
||||
- [x] `go vet`, `go test`, e2e: refused for another domain's `From:`, for the hostname's and for an address off the list; accepted for its own (on the test server: vet ok, 23 packages ok, e2e ok - `sender_check_holds_the_from_line`, `panel_down_defers_mail`)
|
||||
- [ ] Technical review — **Opus**
|
||||
- [ ] Security review — **Fable**
|
||||
|
||||
+1
-1
@@ -39,7 +39,7 @@ in `git log` and [CHANGELOG.md](../CHANGELOG.md).
|
||||
| delivery-log-storage | Keep a message's `mail.log` lines with its send-log row | shipped `2.2.0` | 8/8 | [plans/delivery-log-storage.md](plans/delivery-log-storage.md) |
|
||||
| password-reset | Password reset by e-mail | candidate | — | — |
|
||||
| phone-desktop-mode | A phone browser in "desktop site" mode gets the phone navigation | candidate | — | — |
|
||||
| from-header-binding | Hold a message's `From:` header against the application that sends it | in progress, for `2.4.0` | 1/9 | [plans/from-header-binding.md](plans/from-header-binding.md) |
|
||||
| from-header-binding | Hold a message's `From:` header against the application that sends it | in progress, for `2.4.0` | 7/9 | [plans/from-header-binding.md](plans/from-header-binding.md) |
|
||||
| panel-redesign | Panel redesign (accepted mockups, design contract) | shipped `2.0.0` | 27/27 | [plans/panel-redesign.md](plans/panel-redesign.md) |
|
||||
| schema-squash | Squash SQLite migrations into a 2.x baseline | built in panel-redesign (stage 1) | — | [plans/panel-redesign.md](plans/panel-redesign.md) |
|
||||
|
||||
|
||||
@@ -18,7 +18,7 @@ When you add or review a migration, update this file in the same change.
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Chain head | `user_version = 5` |
|
||||
| Files | `0001_init.sql` (the 2.0 baseline), `0002_inbound_spam_log.sql`, `0003_send_log_lines.sql`, `0004_notifications.sql`, `0005_send_log_rejected_index.sql` |
|
||||
| Files | `0001_init.sql` (the 2.0 baseline), `0002_inbound_spam_log.sql`, `0003_send_log_lines.sql`, `0004_notifications.sql`, `0005_send_log_rejected_index.sql`, `0006_sender_check.sql` |
|
||||
| File mark | `PRAGMA application_id = 0x53503230` ("SP20") |
|
||||
| Database file | `/data/selfpost.db` (bind mount) |
|
||||
| Compatibility | 2.x MINOR releases must boot a `2.0.0` data directory. **Nothing is carried over from 1.x.** |
|
||||
@@ -63,8 +63,9 @@ again and no file is deleted, renamed or reordered — every 2.x database has
|
||||
already run it. `2.0.0` and `2.0.1` shipped with the baseline alone; the
|
||||
features after them add files, the first being `0002_inbound_spam_log.sql` of
|
||||
inbound-antispam-panel (`2.1.0`), then `0003_send_log_lines.sql` of
|
||||
delivery-log-storage (`2.2.0`), and `0004_notifications.sql` and
|
||||
`0005_send_log_rejected_index.sql` of panel-notifications (`2.3.0`).
|
||||
delivery-log-storage (`2.2.0`), `0004_notifications.sql` and
|
||||
`0005_send_log_rejected_index.sql` of panel-notifications (`2.3.0`), and
|
||||
`0006_sender_check.sql` of from-header-binding (`2.4.0`).
|
||||
|
||||
---
|
||||
|
||||
@@ -77,6 +78,7 @@ delivery-log-storage (`2.2.0`), and `0004_notifications.sql` and
|
||||
| 3 | `0003_send_log_lines.sql` | 2.2.0 | DDL | `send_log_lines` and one index; nothing existing changes |
|
||||
| 4 | `0004_notifications.sql` | 2.3.0 | DDL | `users.notify` (default 1: every user starts with notifications on) and `notify_state` |
|
||||
| 5 | `0005_send_log_rejected_index.sql` | 2.3.0 | DDL | `idx_send_log_rejected`, a partial index of the refused rows of `send_log` by time; a file of its own because the test installation had run 0004 before the review asked for it |
|
||||
| 6 | `0006_sender_check.sql` | 2.4.0 | DDL | `send_log.reason` and `send_log.header_from` (both default `''`: a row refused earlier keeps no reason), and `sender_check_days` |
|
||||
|
||||
**Kind:** *DDL* — schema only; *data* — `INSERT`/`UPDATE` that must stay correct
|
||||
for operators upgrading from older 2.x images.
|
||||
|
||||
+38
-20
@@ -188,6 +188,24 @@ operator can act on it.
|
||||
|
||||
- **Not an open relay** — SASL only on 465/587; `reject_unauth_destination`;
|
||||
`smtpd_sender_login_maps` + `reject_sender_login_mismatch`.
|
||||
- **An application signs only as itself (`2.4.0`).** OpenDKIM chooses the
|
||||
key by the `From:` header and Postfix holds only the envelope sender
|
||||
against the login, so until `2.4.0` an application of one sending domain
|
||||
could write another registered domain - or, since `2.3.0`, the server's
|
||||
own name - into `From:` and have the message signed with that name's key
|
||||
(found by the security review of panel-notifications, 2026-10-12). The
|
||||
panel's sender check now holds the header to the table the envelope is
|
||||
held to and refuses anything else with `550 5.7.1`, a domain the server
|
||||
does not sign for included (the owner's rule: the header may say what the
|
||||
envelope may, nothing more). It is the first milter of the chain and
|
||||
strict: a message it could not judge is deferred, never passed on to the
|
||||
signer. It reads the header as one field holding one address and then
|
||||
looks at the raw value once more for any other name of OpenDKIM's
|
||||
`SigningTable`, so that a line its parser and OpenDKIM's could read
|
||||
differently is refused rather than argued about. What the client wrote
|
||||
reaches the reply, the log and the send log only as printable ASCII, cut.
|
||||
Mail the panel submits itself does not pass the check: only root,
|
||||
`postfix` and the panel can submit locally. Architecture § Sender check.
|
||||
- **Inbound relay (optional, `INBOUND_RELAY_ENABLE`)** — port 25 is not an
|
||||
open relay either: SASL is off; `smtpd_relay_restrictions` /
|
||||
`smtpd_recipient_restrictions` are `reject_unauth_destination` and
|
||||
@@ -353,11 +371,14 @@ deferred item from the roadmap.
|
||||
changes, rate limiting, alerting an operator into a config change).
|
||||
- **The application auth-IP allow-list is enforced fail-open.** The check runs
|
||||
in the journal milter (`internal/milter/authip.go`) with
|
||||
`default_action=accept` and skips on store errors, so if the milter is down
|
||||
the restriction is not applied; the SASL login itself always succeeds.
|
||||
(While the milter is up the list holds for every message, and an unknown
|
||||
client address is refused - see Mail path; what is accepted here is the
|
||||
milter being down.) This
|
||||
`default_action=accept` and skips on store errors, so if the journal milter
|
||||
does not answer the restriction is not applied; the SASL login itself
|
||||
always succeeds. (While the milter is up the list holds for every message,
|
||||
and an unknown client address is refused - see Mail path; what is accepted
|
||||
here is the milter not answering. Since `2.4.0` a panel that is down
|
||||
altogether defers mail through the sender check, which shares its process;
|
||||
what is left of this window is a journal milter that hangs or a database
|
||||
that errs while the check beside it still answers.) This
|
||||
is documented in [guide.md](guide.md) as **not an authentication boundary** —
|
||||
it is a blast-radius limiter for a leaked application password, on the same
|
||||
fail-open path as level-2 rate limiting, and a milter outage already stops
|
||||
@@ -436,21 +457,18 @@ deferred item from the roadmap.
|
||||
a dozen times a day at most by someone who flaps the zone; a domain's
|
||||
limits once a day; and no more than 60 messages leave in any hour whatever
|
||||
the rules arrive at.
|
||||
- **A notification can be forged by an application, signed as the server.**
|
||||
OpenDKIM chooses the key by the `From:` header, and Postfix holds only the
|
||||
envelope sender against the login. An authenticated application can
|
||||
therefore send a message whose header says `From: postmaster@<hostname>` and
|
||||
have it signed with the key of the server's own name - a message that
|
||||
passes DMARC and looks like the panel's. The same has always let an
|
||||
application of one sending domain write another registered domain into
|
||||
`From:` and get that domain's signature; the server's name is one more name
|
||||
it works for. Accepted for now: applications are the operator's own, and a
|
||||
forged notification can ask for nothing the panel acts on - it carries no
|
||||
link and the panel takes no mail. The fix is to hold the `From:` header
|
||||
against the login in the journal milter, which refuses mail and is a
|
||||
decision of its own ([roadmap](roadmap.md) § from-header-binding). Trigger
|
||||
to revisit: applications handed to anyone the operator does not trust with
|
||||
every domain of the instance.
|
||||
- **The sender check stops mail while the panel does not answer.** It is a
|
||||
role of the panel process, attached with `default_action=tempfail`, so a
|
||||
panel that is down or slower than the milter timeouts (15 s a command)
|
||||
defers every submission: availability of sending now depends on the panel,
|
||||
which it did not before `2.4.0`. Chosen by the owner over the alternative,
|
||||
a check that is absent whenever the panel is - that one could be switched
|
||||
off by whoever can make the panel slow. Supervisord restarts the panel at
|
||||
once and takes the container down after three failed starts, so the
|
||||
outage is seconds or the whole instance; Server › Health counts the
|
||||
downtime and the mail deferred for it. Trigger to revisit: those numbers
|
||||
not staying at nothing - then the check moves into a process of its own
|
||||
([plans/from-header-binding.md](plans/from-header-binding.md)).
|
||||
- **When the hostname is itself a sending domain,** the server's mail is
|
||||
signed with that domain's key. Whoever may export the domain has the key,
|
||||
and its applications may send as `postmaster@<hostname>` by right. The
|
||||
|
||||
@@ -144,6 +144,12 @@ func (o *OpenDKIM) Record(domainName, selector string) (DKIMRecord, error) {
|
||||
return dkimRecord(selector, domainName, &key.PublicKey)
|
||||
}
|
||||
|
||||
// SigningTablePath is the SigningTable's location: the names this server
|
||||
// signs for, which the sender check reads (internal/milter/sender.go).
|
||||
func (o *OpenDKIM) SigningTablePath() string {
|
||||
return o.signingTablePath
|
||||
}
|
||||
|
||||
// Rebuild regenerates KeyTable and SigningTable from the full domain set and
|
||||
// reloads OpenDKIM (architecture.md § OpenDKIM). Full regeneration (rather
|
||||
// than incremental edits) keeps the files a pure function of the registry, so
|
||||
|
||||
@@ -188,3 +188,21 @@ postfix FATAL Exited too quickly
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestUptime(t *testing.T) {
|
||||
for detail, want := range map[string]time.Duration{
|
||||
"pid 41, uptime 0:03:12": 3*time.Minute + 12*time.Second,
|
||||
"pid 41, uptime 18 days, 3:02:11": 18*24*time.Hour + 3*time.Hour + 2*time.Minute + 11*time.Second,
|
||||
"pid 41, uptime 1 day, 0:00:00": 24 * time.Hour,
|
||||
"pid 7, uptime 127:00:01": 127*time.Hour + time.Second,
|
||||
} {
|
||||
if got, ok := Uptime(detail); !ok || got != want {
|
||||
t.Errorf("Uptime(%q) = %s, %v; want %s", detail, got, ok, want)
|
||||
}
|
||||
}
|
||||
for _, detail := range []string{"", "Not started", "Oct 12 04:00 PM", "pid 41, uptime soon", "pid 41, uptime x days, 0:00:01", "pid 41, uptime 0:00"} {
|
||||
if got, ok := Uptime(detail); ok {
|
||||
t.Errorf("Uptime(%q) = %s, want none", detail, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -3,7 +3,9 @@ package health
|
||||
import (
|
||||
"fmt"
|
||||
"os/exec"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// supervisorConf is the supervisord configuration the panel's other control
|
||||
@@ -85,6 +87,39 @@ func parseProcesses(out string) []Process {
|
||||
return procs
|
||||
}
|
||||
|
||||
// Uptime reads for how long a running program has been up from the detail
|
||||
// supervisord prints for it: "pid 41, uptime 0:03:12", or with days in front,
|
||||
// "pid 41, uptime 18 days, 3:02:11".
|
||||
func Uptime(detail string) (time.Duration, bool) {
|
||||
_, rest, found := strings.Cut(detail, "uptime ")
|
||||
if !found {
|
||||
return 0, false
|
||||
}
|
||||
var days int
|
||||
if d, clock, hasDays := strings.Cut(rest, ", "); hasDays {
|
||||
n, _, _ := strings.Cut(d, " ")
|
||||
v, err := strconv.Atoi(n)
|
||||
if err != nil || v < 0 {
|
||||
return 0, false
|
||||
}
|
||||
days, rest = v, clock
|
||||
}
|
||||
parts := strings.Split(strings.TrimSpace(rest), ":")
|
||||
if len(parts) != 3 {
|
||||
return 0, false
|
||||
}
|
||||
var hms [3]int
|
||||
for i, p := range parts {
|
||||
v, err := strconv.Atoi(p)
|
||||
if err != nil || v < 0 {
|
||||
return 0, false
|
||||
}
|
||||
hms[i] = v
|
||||
}
|
||||
return time.Duration(days)*24*time.Hour + time.Duration(hms[0])*time.Hour +
|
||||
time.Duration(hms[1])*time.Minute + time.Duration(hms[2])*time.Second, true
|
||||
}
|
||||
|
||||
// processStatus grades a supervisord state. A one-shot program that is not
|
||||
// running is healthy; anything else that is not RUNNING means a component of
|
||||
// the mail path is down or flapping.
|
||||
|
||||
@@ -114,8 +114,10 @@ func parseDelivery(line string) (queueID, recipient, status string, ok bool) {
|
||||
// for the message's page, except those of mail that came in on port 25
|
||||
// (lines.go). With inbound set it also writes the
|
||||
// inbound journal's rows for refused mail and prunes that journal
|
||||
// (inbound.go). It returns nil on a clean shutdown.
|
||||
func Run(ctx context.Context, path string, st StatusStore, retention RetentionDays, inbound *Inbound) error {
|
||||
// (inbound.go), and with sender set it counts the mail Postfix deferred
|
||||
// because the sender check did not answer (sendercheck.go). It returns nil on
|
||||
// a clean shutdown.
|
||||
func Run(ctx context.Context, path string, st StatusStore, retention RetentionDays, inbound *Inbound, sender *SenderCheck) error {
|
||||
go retentionLoop(ctx, st, retention)
|
||||
var journal *inboundJournal
|
||||
if inbound != nil {
|
||||
@@ -130,6 +132,11 @@ func Run(ctx context.Context, path string, st StatusStore, retention RetentionDa
|
||||
caughtUp := make(chan struct{})
|
||||
go reconcileLoop(ctx, st, caughtUp)
|
||||
|
||||
var deferred *deferredCounter
|
||||
if sender != nil {
|
||||
deferred = newDeferredCounter(sender)
|
||||
}
|
||||
|
||||
keeper := newLineKeeper(st)
|
||||
return follow(ctx, path, &tracker{st: st, path: path}, caughtUp, keeper.flush, func(line string) {
|
||||
// Until the first read reaches end-of-file the lines are those written
|
||||
@@ -145,6 +152,9 @@ func Run(ctx context.Context, path string, st StatusStore, retention RetentionDa
|
||||
if journal != nil {
|
||||
journal.handle(line, now, catchingUp)
|
||||
}
|
||||
if deferred != nil {
|
||||
deferred.handle(line, now)
|
||||
}
|
||||
keeper.add(line, now, catchingUp)
|
||||
queueID, recipient, status, ok := parseDelivery(line)
|
||||
if !ok {
|
||||
|
||||
@@ -264,7 +264,7 @@ func TestFollowTailsAndRotates(t *testing.T) {
|
||||
cs := &captureStore{}
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
done := make(chan error, 1)
|
||||
go func() { done <- Run(ctx, path, cs, func() int { return 90 }, nil) }()
|
||||
go func() { done <- Run(ctx, path, cs, func() int { return 90 }, nil, nil) }()
|
||||
|
||||
// Give follow() time to open at EOF (it seeks to end immediately on start,
|
||||
// so the seed line above is ignored), then append a delivery line.
|
||||
@@ -347,7 +347,7 @@ func startRun(t *testing.T, path string, cs *captureStore) func() {
|
||||
t.Helper()
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
done := make(chan error, 1)
|
||||
go func() { done <- Run(ctx, path, cs, func() int { return 90 }, nil) }()
|
||||
go func() { done <- Run(ctx, path, cs, func() int { return 90 }, nil, nil) }()
|
||||
// follow() opens and seeks on start; give it a moment before the caller
|
||||
// appends, so the append is not raced by the initial open.
|
||||
time.Sleep(50 * time.Millisecond)
|
||||
|
||||
@@ -0,0 +1,90 @@
|
||||
package logtail
|
||||
|
||||
import (
|
||||
"log"
|
||||
"regexp"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// The sender check is attached to Postfix strictly: while the panel does not
|
||||
// answer on its socket, mail is deferred (docs/plans/from-header-binding.md).
|
||||
// How many submissions that happened to is read here, from what Postfix
|
||||
// writes about it, and shown on Server > Health. It is the figure that also
|
||||
// sees a panel which is running and not answering.
|
||||
//
|
||||
// Postfix says it in lines of one smtpd process - why, then what:
|
||||
//
|
||||
// postfix/smtpd[1183]: warning: connect to Milter service unix:/run/selfpost/sender.sock: No such file or directory
|
||||
// postfix/smtpd[1183]: NOQUEUE: milter-reject: CONNECT from unknown[10.77.0.1]: 451 4.7.1 Service unavailable - try again later; proto=SMTP
|
||||
// postfix/smtpd[1183]: NOQUEUE: milter-reject: EHLO from unknown[10.77.0.1]: 451 4.7.1 Service unavailable - try again later; proto=SMTP helo=<app>
|
||||
//
|
||||
// and the client, told "454 4.3.0 Try again later" at AUTH, goes away without
|
||||
// a line of its own. So what is counted is the connection Postfix turned
|
||||
// away, once: a client brings a message when it connects, and it will bring
|
||||
// it again.
|
||||
//
|
||||
// The reject lines do not name the milter, and the same words are written
|
||||
// when OpenDKIM does not answer or when the journal milter refuses under a
|
||||
// level-2 limit. So a process is remembered from the warning that names this
|
||||
// socket, and counted at its next reject line.
|
||||
|
||||
// SenderCheckStore is where the count goes. *store.Store satisfies it.
|
||||
type SenderCheckStore interface {
|
||||
AddSenderCheckDeferred(at time.Time, n int64) error
|
||||
}
|
||||
|
||||
// SenderCheck turns the counting on: the store and the socket Postfix was
|
||||
// given for the check (SENDER_CHECK_SOCKET), as a path.
|
||||
type SenderCheck struct {
|
||||
Store SenderCheckStore
|
||||
Socket string
|
||||
}
|
||||
|
||||
var smtpdLineRe = regexp.MustCompile(`^\S+ postfix/(?:[a-z0-9-]+/)?smtpd\[(\d+)\]: (.*)$`)
|
||||
|
||||
const (
|
||||
milterUnavailable = "Service unavailable - try again later"
|
||||
// senderCheckPids bounds the processes remembered; smtpd processes are a
|
||||
// few dozen, and a log that names more is not kept track of.
|
||||
senderCheckPids = 1024
|
||||
)
|
||||
|
||||
// deferredCounter is the per-run state of the count.
|
||||
type deferredCounter struct {
|
||||
st SenderCheckStore
|
||||
needle string // the socket as Postfix names it in a warning
|
||||
failed map[string]bool // smtpd pid -> the check did not answer its current client
|
||||
}
|
||||
|
||||
func newDeferredCounter(sc *SenderCheck) *deferredCounter {
|
||||
return &deferredCounter{st: sc.Store, needle: "unix:" + sc.Socket, failed: make(map[string]bool)}
|
||||
}
|
||||
|
||||
// handle looks at one mail.log line.
|
||||
func (c *deferredCounter) handle(line string, now time.Time) {
|
||||
at, rest := lineTime(line, now)
|
||||
m := smtpdLineRe.FindStringSubmatch(rest)
|
||||
if m == nil {
|
||||
return
|
||||
}
|
||||
pid, text := m[1], m[2]
|
||||
switch {
|
||||
case strings.HasPrefix(text, "connect from "):
|
||||
delete(c.failed, pid)
|
||||
case strings.HasPrefix(text, "warning: ") && strings.Contains(text, c.needle) && strings.Contains(strings.ToLower(text), "milter"):
|
||||
// "connect to Milter service <socket>: …" when nobody listens, and
|
||||
// "milter <socket>: can't read … reply packet header: …" when the
|
||||
// check stopped answering in the middle of a message.
|
||||
if len(c.failed) >= senderCheckPids {
|
||||
c.failed = make(map[string]bool)
|
||||
}
|
||||
c.failed[pid] = true
|
||||
case c.failed[pid] && strings.Contains(text, "milter-reject: ") && strings.Contains(text, milterUnavailable):
|
||||
// Once for the connection, however many commands the client tries.
|
||||
delete(c.failed, pid)
|
||||
if err := c.st.AddSenderCheckDeferred(at, 1); err != nil {
|
||||
log.Printf("log-tailer: sender check statistics: %v", err)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,63 @@
|
||||
package logtail
|
||||
|
||||
import (
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
type deferredStore struct {
|
||||
days map[string]int64
|
||||
}
|
||||
|
||||
func (d *deferredStore) AddSenderCheckDeferred(at time.Time, n int64) error {
|
||||
if d.days == nil {
|
||||
d.days = make(map[string]int64)
|
||||
}
|
||||
d.days[at.UTC().Format("2006-01-02")] += n
|
||||
return nil
|
||||
}
|
||||
|
||||
// The lines are Postfix 3.7's, as the e2e stand wrote them with the panel
|
||||
// stopped (2026-10-12), with the lines of two other things that use the same
|
||||
// words: a level-2 refusal of the journal milter and OpenDKIM not answering.
|
||||
func TestDeferredCounter(t *testing.T) {
|
||||
st := &deferredStore{}
|
||||
c := newDeferredCounter(&SenderCheck{Store: st, Socket: "/run/selfpost/sender.sock"})
|
||||
now := time.Date(2026, 10, 12, 18, 0, 0, 0, time.UTC)
|
||||
for _, line := range []string{
|
||||
// The journal milter's own temporary refusal: no warning, not counted.
|
||||
`2026-10-12T16:22:13.100000+00:00 mail postfix/smtpd[1149]: connect from unknown[10.77.0.1]`,
|
||||
`2026-10-12T16:22:13.200000+00:00 mail postfix/smtpd[1149]: NOQUEUE: milter-reject: MAIL from unknown[10.77.0.1]: 451 4.7.1 Service unavailable - try again later; from=<billing@sender.e2e.test> proto=ESMTP helo=<e2e-client.e2e.test> sasl_username=<l2app>`,
|
||||
|
||||
// The check does not answer: one connection, counted once.
|
||||
`2026-10-12T16:22:57.000000+00:00 mail postfix/smtpd[1183]: connect from unknown[10.77.0.1]`,
|
||||
`2026-10-12T16:22:57.100000+00:00 mail postfix/smtpd[1183]: warning: connect to Milter service unix:/run/selfpost/sender.sock: No such file or directory`,
|
||||
`2026-10-12T16:22:57.200000+00:00 mail postfix/smtpd[1183]: NOQUEUE: milter-reject: CONNECT from unknown[10.77.0.1]: 451 4.7.1 Service unavailable - try again later; proto=SMTP`,
|
||||
`2026-10-12T16:22:57.300000+00:00 mail postfix/smtpd[1183]: NOQUEUE: milter-reject: EHLO from unknown[10.77.0.1]: 451 4.7.1 Service unavailable - try again later; proto=SMTP helo=<e2e-client.e2e.test>`,
|
||||
`2026-10-12T16:22:57.400000+00:00 mail postfix/smtpd[1183]: disconnect from unknown[10.77.0.1] ehlo=1 auth=0/1 quit=1 commands=2/3`,
|
||||
|
||||
// The same process takes another client, and the check answers again:
|
||||
// a level-2 refusal there is not the check's.
|
||||
`2026-10-12T16:23:30.000000+00:00 mail postfix/smtpd[1183]: connect from unknown[10.77.0.1]`,
|
||||
`2026-10-12T16:23:30.100000+00:00 mail postfix/smtpd[1183]: NOQUEUE: milter-reject: MAIL from unknown[10.77.0.1]: 451 4.7.1 Service unavailable - try again later; from=<a@sender.e2e.test> proto=ESMTP helo=<x>`,
|
||||
|
||||
// OpenDKIM does not answer: another socket, not counted.
|
||||
`2026-10-12T16:24:00.000000+00:00 mail postfix/smtpd[1201]: connect from unknown[10.77.0.1]`,
|
||||
`2026-10-12T16:24:00.100000+00:00 mail postfix/smtpd[1201]: warning: connect to Milter service unix:/run/opendkim/opendkim.sock: Connection refused`,
|
||||
`2026-10-12T16:24:00.200000+00:00 mail postfix/smtpd[1201]: NOQUEUE: milter-reject: CONNECT from unknown[10.77.0.1]: 451 4.7.1 Service unavailable - try again later; proto=SMTP`,
|
||||
|
||||
// The check stops answering in the middle of a message, the next day.
|
||||
`2026-10-13T00:00:05.000000+00:00 mail postfix/smtpd[1300]: connect from unknown[10.77.0.1]`,
|
||||
`2026-10-13T00:00:21.000000+00:00 mail postfix/smtpd[1300]: warning: milter unix:/run/selfpost/sender.sock: can't read SMFIC_EOH reply packet header: Connection timed out`,
|
||||
`2026-10-13T00:00:21.100000+00:00 mail postfix/smtpd[1300]: 4XcB7k2Jm9z1: milter-reject: END-OF-HEADER from unknown[10.77.0.1]: 4.7.1 Service unavailable - try again later; from=<a@sender.e2e.test> to=<b@sink.e2e.test> proto=ESMTP helo=<x>`,
|
||||
|
||||
// The check's own verdict is a refusal, not a deferral, and it is
|
||||
// cleanup that writes it.
|
||||
`2026-10-13T00:01:00.000000+00:00 mail postfix/cleanup[1195]: B6B484831946: milter-reject: END-OF-MESSAGE from unknown[10.77.0.1]: 5.7.1 From: director@other.e2e.test - not an address this application sends as; from=<alerts@sender.e2e.test> to=<rcpt@sink.e2e.test> proto=ESMTP helo=<x>`,
|
||||
} {
|
||||
c.handle(line, now)
|
||||
}
|
||||
if len(st.days) != 2 || st.days["2026-10-12"] != 1 || st.days["2026-10-13"] != 1 {
|
||||
t.Fatalf("deferred per day = %v, want one on each of the two days", st.days)
|
||||
}
|
||||
}
|
||||
@@ -102,11 +102,11 @@ func (s *session) MailFrom(from string, m *milter.Modifier) (milter.Response, er
|
||||
s.rcpts = nil
|
||||
s.subject = ""
|
||||
if !s.authIPAllowed() {
|
||||
s.recordRejected()
|
||||
s.recordRejected(store.ReasonClient)
|
||||
return milter.RespTempFail, nil
|
||||
}
|
||||
if s.overLimit() {
|
||||
s.recordRejected()
|
||||
s.recordRejected(store.ReasonLimit)
|
||||
return milter.RespTempFail, nil
|
||||
}
|
||||
return milter.RespContinue, nil
|
||||
|
||||
@@ -85,14 +85,16 @@ func (s *session) releaseReservations() {
|
||||
}
|
||||
|
||||
// recordRejected writes a send-log row for a message refused by a level-2
|
||||
// limit (guide § Rate limiting — refusals are recorded too), so the rejection
|
||||
// shows up in the monitoring screen. Only MAIL-stage fields are known; the
|
||||
// write is best-effort and never affects the response.
|
||||
func (s *session) recordRejected() {
|
||||
// limit (guide § Rate limiting — refusals are recorded too) or for its client
|
||||
// address, so the rejection shows up in the monitoring screen with its reason
|
||||
// (store.Reason*). Only MAIL-stage fields are known; the write is best-effort
|
||||
// and never affects the response.
|
||||
func (s *session) recordRejected(reason string) {
|
||||
err := s.rec.InsertRejected(store.SendLogEntry{
|
||||
Domain: domainOf(s.from),
|
||||
AppLogin: s.login,
|
||||
From: s.from,
|
||||
Reason: reason,
|
||||
})
|
||||
if err != nil {
|
||||
log.Printf("journal-milter: record rejected %s: %v", s.from, err)
|
||||
|
||||
@@ -0,0 +1,399 @@
|
||||
package milter
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"log"
|
||||
"net"
|
||||
"net/mail"
|
||||
"net/textproto"
|
||||
"os"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/emersion/go-milter"
|
||||
|
||||
"github.com/mixeme/selfpost/internal/mailhdr"
|
||||
"github.com/mixeme/selfpost/internal/store"
|
||||
)
|
||||
|
||||
// The sender check: a message's From: line is held against the application
|
||||
// that sends it, under the rule Postfix holds the envelope sender to
|
||||
// (docs/plans/from-header-binding.md). OpenDKIM chooses the signing key by
|
||||
// that line, so without the check an application could have mail signed as
|
||||
// another domain of the server, or as the server itself.
|
||||
//
|
||||
// Unlike the journal milter this one is attached strictly
|
||||
// (default_action=tempfail in build/postfix-config.sh): while it does not
|
||||
// answer, Postfix defers mail. It therefore depends on as little as it can.
|
||||
// It reads no database: what an application may send as comes from the file
|
||||
// of sender bindings Postfix itself reads for the envelope, so the two cannot
|
||||
// disagree. The send-log row of a refusal is written by another goroutine and
|
||||
// is not waited for.
|
||||
|
||||
// tableFile is a table of the mail path that the panel rewrites and the check
|
||||
// reads: it is read again when it has changed. An unreadable file is an error
|
||||
// and the check then defers mail - a table from before would let a deleted
|
||||
// application go on sending, and no table would refuse everyone for good.
|
||||
type tableFile struct {
|
||||
path string
|
||||
modTime time.Time
|
||||
size int64
|
||||
loaded bool
|
||||
}
|
||||
|
||||
// changed returns the file's lines when they have to be read (again), and
|
||||
// nil when what was read last still stands.
|
||||
func (f *tableFile) changed() (lines []string, again bool, err error) {
|
||||
fi, err := os.Stat(f.path)
|
||||
if err != nil {
|
||||
return nil, false, err
|
||||
}
|
||||
if f.loaded && fi.ModTime().Equal(f.modTime) && fi.Size() == f.size {
|
||||
return nil, false, nil
|
||||
}
|
||||
raw, err := os.ReadFile(f.path)
|
||||
if err != nil {
|
||||
return nil, false, err
|
||||
}
|
||||
f.modTime, f.size, f.loaded = fi.ModTime(), fi.Size(), true
|
||||
return strings.Split(string(raw), "\n"), true, nil
|
||||
}
|
||||
|
||||
// Bindings is Postfix's sender_login_maps file as the check reads it: each
|
||||
// sender - an address, or "@domain" for any address of a domain - with the
|
||||
// logins that may use it.
|
||||
type Bindings struct {
|
||||
mu sync.Mutex
|
||||
file tableFile
|
||||
logins map[string]map[string]bool // sender -> logins
|
||||
}
|
||||
|
||||
// NewBindings reads the file at path on first use.
|
||||
func NewBindings(path string) *Bindings {
|
||||
return &Bindings{file: tableFile{path: path}}
|
||||
}
|
||||
|
||||
// fresh makes the table match the file.
|
||||
func (b *Bindings) fresh() error {
|
||||
lines, again, err := b.file.changed()
|
||||
if err != nil || !again {
|
||||
return err
|
||||
}
|
||||
logins := make(map[string]map[string]bool)
|
||||
for _, line := range lines {
|
||||
// texthash: <key><whitespace><value>, the value a comma-separated
|
||||
// list of logins (internal/postfix renderSenderLoginMaps).
|
||||
fields := strings.Fields(line)
|
||||
if len(fields) < 2 || strings.HasPrefix(fields[0], "#") {
|
||||
continue
|
||||
}
|
||||
sender := strings.ToLower(fields[0])
|
||||
set := logins[sender]
|
||||
if set == nil {
|
||||
set = make(map[string]bool)
|
||||
logins[sender] = set
|
||||
}
|
||||
for _, l := range strings.Split(strings.Join(fields[1:], ""), ",") {
|
||||
if l != "" {
|
||||
set[l] = true
|
||||
}
|
||||
}
|
||||
}
|
||||
b.logins = logins
|
||||
return nil
|
||||
}
|
||||
|
||||
// Allowed reports whether login may send as addr: the address itself is bound
|
||||
// to it, or its whole domain is. This is the lookup Postfix makes for the
|
||||
// envelope sender.
|
||||
func (b *Bindings) Allowed(login, addr string) (bool, error) {
|
||||
b.mu.Lock()
|
||||
defer b.mu.Unlock()
|
||||
if err := b.fresh(); err != nil {
|
||||
return false, err
|
||||
}
|
||||
addr = strings.ToLower(addr)
|
||||
if b.logins[addr][login] {
|
||||
return true, nil
|
||||
}
|
||||
if d := domainOf(addr); d != "" && b.logins["@"+d][login] {
|
||||
return true, nil
|
||||
}
|
||||
return false, nil
|
||||
}
|
||||
|
||||
// Signers is OpenDKIM's SigningTable as the check reads it: the names this
|
||||
// server signs for - every sending domain, with applications or without, and
|
||||
// the server's own name.
|
||||
type Signers struct {
|
||||
mu sync.Mutex
|
||||
file tableFile
|
||||
names map[string]bool
|
||||
}
|
||||
|
||||
// NewSigners reads the file at path on first use.
|
||||
func NewSigners(path string) *Signers {
|
||||
return &Signers{file: tableFile{path: path}}
|
||||
}
|
||||
|
||||
// Signs reports whether mail whose From: is at domain gets a signature.
|
||||
func (s *Signers) Signs(domain string) (bool, error) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
lines, again, err := s.file.changed()
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
if again {
|
||||
names := make(map[string]bool)
|
||||
for _, line := range lines {
|
||||
// "*@<name> <key name>" (internal/domain renderTables).
|
||||
fields := strings.Fields(line)
|
||||
if len(fields) < 2 || strings.HasPrefix(fields[0], "#") {
|
||||
continue
|
||||
}
|
||||
if d := domainOf(fields[0]); d != "" {
|
||||
names[d] = true
|
||||
}
|
||||
}
|
||||
s.names = names
|
||||
}
|
||||
return s.names[strings.ToLower(domain)], nil
|
||||
}
|
||||
|
||||
// Refusals is where the check leaves the send-log rows of what it refused.
|
||||
// *store.Store satisfies it.
|
||||
type Refusals interface {
|
||||
InsertRejected(e store.SendLogEntry) error
|
||||
}
|
||||
|
||||
// senderSession is one connection of the sender check.
|
||||
type senderSession struct {
|
||||
milter.NoOpMilter
|
||||
bind *Bindings
|
||||
signers *Signers
|
||||
refused chan<- store.SendLogEntry
|
||||
|
||||
login string
|
||||
from string
|
||||
rcpts []string
|
||||
subject string
|
||||
froms []string // the value of every From: field of the message
|
||||
}
|
||||
|
||||
// MailFrom starts a message. Like the journal milter it reads everything of a
|
||||
// message again here: the library replaces the session after a refusal.
|
||||
func (s *senderSession) MailFrom(from string, m *milter.Modifier) (milter.Response, error) {
|
||||
s.from = cleanAddress(from)
|
||||
s.login = macro(m, "auth_authen")
|
||||
s.rcpts, s.subject, s.froms = nil, "", nil
|
||||
return milter.RespContinue, nil
|
||||
}
|
||||
|
||||
func (s *senderSession) RcptTo(rcpt string, m *milter.Modifier) (milter.Response, error) {
|
||||
s.rcpts = append(s.rcpts, cleanAddress(rcpt))
|
||||
return milter.RespContinue, nil
|
||||
}
|
||||
|
||||
func (s *senderSession) Header(name, value string, m *milter.Modifier) (milter.Response, error) {
|
||||
// The name is trimmed: the obsolete syntax allows blanks before the colon.
|
||||
switch textproto.CanonicalMIMEHeaderKey(strings.TrimSpace(name)) {
|
||||
case "From":
|
||||
s.froms = append(s.froms, value)
|
||||
case "Subject":
|
||||
if s.subject == "" {
|
||||
s.subject = mailhdr.DecodeSubject(value)
|
||||
}
|
||||
}
|
||||
return milter.RespContinue, nil
|
||||
}
|
||||
|
||||
// Headers is the end of the headers, where the message is judged.
|
||||
func (s *senderSession) Headers(h textproto.MIMEHeader, m *milter.Modifier) (milter.Response, error) {
|
||||
shown, why, err := s.judge()
|
||||
if err != nil {
|
||||
// Strict: what cannot be checked is not let through.
|
||||
log.Printf("sender-check: %v - deferring mail of %q", err, s.login)
|
||||
return milter.RespTempFail, nil
|
||||
}
|
||||
if why == "" {
|
||||
return milter.RespContinue, nil
|
||||
}
|
||||
log.Printf("sender-check: application %q refused: %s (From: %s)", s.login, why, shown)
|
||||
s.record(shown)
|
||||
// A reply code of its own, so that the application is told what is wrong
|
||||
// and, the refusal being permanent, does not send the message again.
|
||||
return milter.NewResponseStr('y', "550 5.7.1 From: "+shown+" - "+why), nil
|
||||
}
|
||||
|
||||
func (s *senderSession) Abort(m *milter.Modifier) error {
|
||||
s.rcpts, s.subject, s.froms = nil, "", nil
|
||||
return nil
|
||||
}
|
||||
|
||||
// judge decides about the message at hand. why is empty when it may pass;
|
||||
// otherwise it says what is wrong, and shown is the From: line in a form that
|
||||
// is safe to print - both go into the reply to the client.
|
||||
func (s *senderSession) judge() (shown, why string, err error) {
|
||||
if s.login == "" {
|
||||
// Not reachable while the submission ports demand authentication; a
|
||||
// message nobody answers for is not signed either way.
|
||||
return "(none)", "no authenticated application", nil
|
||||
}
|
||||
switch len(s.froms) {
|
||||
case 0:
|
||||
return "(none)", "the message has no From: line", nil
|
||||
case 1:
|
||||
default:
|
||||
return printable(strings.Join(s.froms, " | ")), "the message has more than one From: line", nil
|
||||
}
|
||||
raw := s.froms[0]
|
||||
shown = printable(raw)
|
||||
list, perr := mail.ParseAddressList(raw)
|
||||
if perr != nil || len(list) != 1 || list[0].Address == "" {
|
||||
return shown, "it must be one address", nil
|
||||
}
|
||||
addr := strings.ToLower(list[0].Address)
|
||||
shown = printable(addr)
|
||||
ok, err := s.bind.Allowed(s.login, addr)
|
||||
if err != nil {
|
||||
return shown, "", fmt.Errorf("sender bindings: %w", err)
|
||||
}
|
||||
if !ok {
|
||||
return shown, "not an address this application sends as", nil
|
||||
}
|
||||
// A cruder second look. OpenDKIM finds the address in this line with a
|
||||
// parser of its own, and the key it signs with follows what that parser
|
||||
// finds. Wherever the two could read the line differently - a display
|
||||
// name, a comment, a quoted string - the other address would have to be
|
||||
// spelled out in it, so no other name this server signs for may appear.
|
||||
own := domainOf(addr)
|
||||
for _, d := range mentionedDomains(raw) {
|
||||
if d == own {
|
||||
continue
|
||||
}
|
||||
signs, err := s.signers.Signs(d)
|
||||
if err != nil {
|
||||
return shown, "", fmt.Errorf("signing table: %w", err)
|
||||
}
|
||||
if signs {
|
||||
return printable(raw), "it names another sender of this server, " + d, nil
|
||||
}
|
||||
}
|
||||
return shown, "", nil
|
||||
}
|
||||
|
||||
// record leaves a row per recipient for the Outbound log. It never waits: a
|
||||
// database that is slow must not make the check slow.
|
||||
func (s *senderSession) record(shown string) {
|
||||
if s.refused == nil {
|
||||
return
|
||||
}
|
||||
rcpts := s.rcpts
|
||||
if len(rcpts) == 0 {
|
||||
rcpts = []string{""}
|
||||
}
|
||||
for _, to := range rcpts {
|
||||
e := store.SendLogEntry{
|
||||
Domain: domainOf(s.from),
|
||||
AppLogin: s.login,
|
||||
From: s.from,
|
||||
To: to,
|
||||
Subject: s.subject,
|
||||
Reason: store.ReasonFrom,
|
||||
HeaderFrom: shown,
|
||||
}
|
||||
select {
|
||||
case s.refused <- e:
|
||||
default:
|
||||
log.Printf("sender-check: the refusal of %s -> %s is not in the log: too many are waiting to be written", s.from, to)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// mentionedDomains returns, lower-cased, what follows every "@" of v as far
|
||||
// as it reads as a host name.
|
||||
func mentionedDomains(v string) []string {
|
||||
var out []string
|
||||
for i := 0; i < len(v); i++ {
|
||||
if v[i] != '@' {
|
||||
continue
|
||||
}
|
||||
j := i + 1
|
||||
for j < len(v) && isHostByte(v[j]) {
|
||||
j++
|
||||
}
|
||||
if d := strings.Trim(strings.ToLower(v[i+1:j]), ".-"); d != "" {
|
||||
out = append(out, d)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func isHostByte(c byte) bool {
|
||||
return c >= 'a' && c <= 'z' || c >= 'A' && c <= 'Z' || c >= '0' && c <= '9' || c == '.' || c == '-'
|
||||
}
|
||||
|
||||
// printable reduces what a client wrote to printable ASCII without the
|
||||
// characters that mean something in an SMTP reply or a log line, and cuts it:
|
||||
// it is said back to the client, logged and stored.
|
||||
func printable(v string) string {
|
||||
const max = 120
|
||||
var sb strings.Builder
|
||||
for _, r := range v {
|
||||
if sb.Len() >= max {
|
||||
sb.WriteString("...")
|
||||
break
|
||||
}
|
||||
switch {
|
||||
case r == '%' || r < 0x20 || r > 0x7e:
|
||||
sb.WriteByte('?')
|
||||
default:
|
||||
sb.WriteRune(r)
|
||||
}
|
||||
}
|
||||
return strings.TrimSpace(sb.String())
|
||||
}
|
||||
|
||||
// ServeSender runs the sender check on ln until ctx is cancelled. rec may be
|
||||
// nil: refusals are then not logged.
|
||||
func ServeSender(ctx context.Context, ln net.Listener, bind *Bindings, signers *Signers, rec Refusals) error {
|
||||
var refused chan store.SendLogEntry
|
||||
if rec != nil {
|
||||
refused = make(chan store.SendLogEntry, 256)
|
||||
go func() {
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case e := <-refused:
|
||||
if err := rec.InsertRejected(e); err != nil {
|
||||
log.Printf("sender-check: record refused %s: %v", e.From, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
}()
|
||||
}
|
||||
srv := &milter.Server{
|
||||
NewMilter: func() milter.Milter {
|
||||
return &senderSession{bind: bind, signers: signers, refused: refused}
|
||||
},
|
||||
Actions: 0, // read-only
|
||||
Protocol: milter.OptNoBody, // the headers decide
|
||||
}
|
||||
|
||||
go func() {
|
||||
<-ctx.Done()
|
||||
_ = srv.Close()
|
||||
}()
|
||||
|
||||
if err := srv.Serve(ln); err != nil {
|
||||
if ctx.Err() != nil {
|
||||
return nil
|
||||
}
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,226 @@
|
||||
package milter
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/emersion/go-milter"
|
||||
|
||||
"github.com/mixeme/selfpost/internal/store"
|
||||
)
|
||||
|
||||
// senderMaps is a sender_login_maps file as internal/postfix writes it: shop
|
||||
// and news send as any address of their domains, billing only as two listed
|
||||
// addresses of example.org, and two applications share shop.example.org.
|
||||
const senderMaps = `@news.example.net news
|
||||
@shop.example.org shop,shop2
|
||||
invoices@example.org billing
|
||||
billing@example.org billing
|
||||
`
|
||||
|
||||
// signingTable is OpenDKIM's SigningTable for the same server: the three
|
||||
// sending domains, one more that has no application yet, and the server's own
|
||||
// name.
|
||||
const signingTable = `*@news.example.net selfpost._domainkey.news.example.net
|
||||
*@shop.example.org selfpost._domainkey.shop.example.org
|
||||
*@example.org selfpost._domainkey.example.org
|
||||
*@empty.example.com selfpost._domainkey.empty.example.com
|
||||
*@mail.example.org selfpost._domainkey.mail.example.org
|
||||
`
|
||||
|
||||
func signers(t *testing.T) *Signers {
|
||||
t.Helper()
|
||||
return NewSigners(writeMaps(t, signingTable))
|
||||
}
|
||||
|
||||
func writeMaps(t *testing.T, content string) string {
|
||||
t.Helper()
|
||||
p := filepath.Join(t.TempDir(), "table")
|
||||
if err := os.WriteFile(p, []byte(content), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return p
|
||||
}
|
||||
|
||||
// offer plays one message to the check and returns the reply at the end of
|
||||
// the headers, with what was left for the log.
|
||||
func offer(t *testing.T, bind *Bindings, login, envelope string, headers ...[2]string) (milter.Response, []store.SendLogEntry) {
|
||||
t.Helper()
|
||||
refused := make(chan store.SendLogEntry, 8)
|
||||
s := &senderSession{bind: bind, signers: signers(t), refused: refused}
|
||||
if _, err := s.MailFrom(envelope, mods(map[string]string{"{auth_authen}": login})); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := s.RcptTo("someone@elsewhere.example", mods(nil)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, h := range headers {
|
||||
if _, err := s.Header(h[0], h[1], mods(nil)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
resp, err := s.Headers(nil, mods(nil))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
close(refused)
|
||||
var rows []store.SendLogEntry
|
||||
for e := range refused {
|
||||
rows = append(rows, e)
|
||||
}
|
||||
return resp, rows
|
||||
}
|
||||
|
||||
func replyText(r milter.Response) string {
|
||||
m := r.Response()
|
||||
return string(rune(m.Code)) + " " + strings.TrimRight(string(m.Data), "\x00")
|
||||
}
|
||||
|
||||
func TestSenderCheck(t *testing.T) {
|
||||
bind := NewBindings(writeMaps(t, senderMaps))
|
||||
from := func(v string) [2]string { return [2]string{"From", v} }
|
||||
|
||||
cases := []struct {
|
||||
name string
|
||||
login string
|
||||
headers [][2]string
|
||||
pass bool
|
||||
wantText string // a part of the refusal
|
||||
}{
|
||||
{"own address", "shop", [][2]string{from("shop@shop.example.org")}, true, ""},
|
||||
{"own domain, any address, with a name", "shop", [][2]string{from(`"The Shop" <Orders@Shop.Example.Org>`)}, true, ""},
|
||||
{"second application of the domain", "shop2", [][2]string{from("x@shop.example.org")}, true, ""},
|
||||
{"a listed address", "billing", [][2]string{from("Invoices <invoices@example.org>")}, true, ""},
|
||||
{"header name in another case", "shop", [][2]string{{"FROM", "a@shop.example.org"}}, true, ""},
|
||||
{"its own name inside the display name", "shop", [][2]string{from(`"shop@shop.example.org" <a@shop.example.org>`)}, true, ""},
|
||||
{"an encoded display name", "shop", [][2]string{from("=?utf-8?b?0JzQsNCz0LDQt9C40L0=?= <a@shop.example.org>")}, true, ""},
|
||||
|
||||
{"another registered domain", "shop", [][2]string{from("director@example.org")}, false, "not an address this application sends as"},
|
||||
{"the server's own name", "shop", [][2]string{from("postmaster@mail.example.org")}, false, "not an address this application sends as"},
|
||||
{"a stranger's domain", "shop", [][2]string{from("someone@gmail.example")}, false, "not an address this application sends as"},
|
||||
{"an address off the list", "billing", [][2]string{from("director@example.org")}, false, "not an address this application sends as"},
|
||||
{"a subdomain of its own", "shop", [][2]string{from("a@sub.shop.example.org")}, false, "not an address this application sends as"},
|
||||
{"no From: line", "shop", [][2]string{{"Subject", "hello"}}, false, "no From: line"},
|
||||
{"two From: lines", "shop", [][2]string{from("a@shop.example.org"), from("director@example.org")}, false, "more than one From: line"},
|
||||
{"two addresses in one line", "shop", [][2]string{from("a@shop.example.org, director@example.org")}, false, "one address"},
|
||||
{"not an address", "shop", [][2]string{from("the shop")}, false, "one address"},
|
||||
{"a group", "shop", [][2]string{from("friends: a@shop.example.org, b@shop.example.org;")}, false, "one address"},
|
||||
{"another sender in the display name", "shop", [][2]string{from(`"director@example.org" <a@shop.example.org>`)}, false, "names another sender of this server"},
|
||||
{"the server's name in a comment", "shop", [][2]string{from("a@shop.example.org (postmaster@mail.example.org)")}, false, "names another sender of this server"},
|
||||
{"a domain with a key and no application, in the display name", "shop", [][2]string{from(`"x@empty.example.com" <a@shop.example.org>`)}, false, "names another sender of this server"},
|
||||
{"a stranger's address in the display name", "shop", [][2]string{from(`"someone@gmail.example" <a@shop.example.org>`)}, true, ""},
|
||||
{"no login", "", [][2]string{from("a@shop.example.org")}, false, "no authenticated application"},
|
||||
{"an unknown login", "ghost", [][2]string{from("a@shop.example.org")}, false, "not an address this application sends as"},
|
||||
}
|
||||
for _, c := range cases {
|
||||
t.Run(c.name, func(t *testing.T) {
|
||||
resp, rows := offer(t, bind, c.login, "shop@shop.example.org", c.headers...)
|
||||
if c.pass {
|
||||
if resp != milter.RespContinue {
|
||||
t.Fatalf("refused: %s", replyText(resp))
|
||||
}
|
||||
if len(rows) != 0 {
|
||||
t.Fatalf("a passed message left rows: %+v", rows)
|
||||
}
|
||||
return
|
||||
}
|
||||
text := replyText(resp)
|
||||
if !strings.HasPrefix(text, "y 550 5.7.1 From: ") || !strings.Contains(text, c.wantText) {
|
||||
t.Fatalf("reply %q, want a 550 5.7.1 saying %q", text, c.wantText)
|
||||
}
|
||||
if len(rows) != 1 || rows[0].Reason != store.ReasonFrom || rows[0].To != "someone@elsewhere.example" || rows[0].AppLogin != c.login {
|
||||
t.Fatalf("log rows: %+v", rows)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// What the client wrote is said back to it, logged and stored: it must not
|
||||
// be able to put a line break or a control character there.
|
||||
func TestSenderCheckPrintsNothingRaw(t *testing.T) {
|
||||
bind := NewBindings(writeMaps(t, senderMaps))
|
||||
evil := "x@gmail.example\r\n250 ok %s \x00\x1b[31m " + strings.Repeat("a", 400)
|
||||
resp, rows := offer(t, bind, "shop", "shop@shop.example.org", [2]string{"From", evil})
|
||||
text := replyText(resp)
|
||||
for _, bad := range []string{"\r", "\n", "%", "\x1b"} {
|
||||
if strings.Contains(text, bad) || strings.Contains(rows[0].HeaderFrom, bad) {
|
||||
t.Fatalf("%q got through: %q / %q", bad, text, rows[0].HeaderFrom)
|
||||
}
|
||||
}
|
||||
if len(rows[0].HeaderFrom) > 130 {
|
||||
t.Fatalf("stored %d bytes of the header", len(rows[0].HeaderFrom))
|
||||
}
|
||||
}
|
||||
|
||||
// Without its file the check knows nothing, and what cannot be checked is
|
||||
// deferred - neither passed nor refused for good.
|
||||
func TestSenderCheckDefersWithoutBindings(t *testing.T) {
|
||||
bind := NewBindings(filepath.Join(t.TempDir(), "absent"))
|
||||
resp, rows := offer(t, bind, "shop", "shop@shop.example.org", [2]string{"From", "a@shop.example.org"})
|
||||
if resp != milter.RespTempFail {
|
||||
t.Fatalf("reply %s, want a temporary failure", replyText(resp))
|
||||
}
|
||||
if len(rows) != 0 {
|
||||
t.Fatalf("a deferred message left rows: %+v", rows)
|
||||
}
|
||||
}
|
||||
|
||||
// The panel rewrites the file when an application is added or deleted; the
|
||||
// check has to follow without a restart.
|
||||
func TestBindingsFollowTheFile(t *testing.T) {
|
||||
p := writeMaps(t, senderMaps)
|
||||
bind := NewBindings(p)
|
||||
if ok, err := bind.Allowed("shop", "a@shop.example.org"); err != nil || !ok {
|
||||
t.Fatalf("before: %v %v", ok, err)
|
||||
}
|
||||
if err := os.WriteFile(p, []byte("@news.example.net news\n"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// A rewrite within the clock's resolution that keeps the size would not be
|
||||
// seen; the panel replaces the file by rename, which gives it a new time.
|
||||
later := time.Now().Add(2 * time.Second)
|
||||
if err := os.Chtimes(p, later, later); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if ok, err := bind.Allowed("shop", "a@shop.example.org"); err != nil || ok {
|
||||
t.Fatalf("after the application was deleted: %v %v", ok, err)
|
||||
}
|
||||
}
|
||||
|
||||
// Without the signing table the second look cannot be taken, and a line it
|
||||
// would have to judge is deferred; a line that names nobody else is not held
|
||||
// up by it.
|
||||
func TestSenderCheckDefersWithoutSigningTable(t *testing.T) {
|
||||
bind := NewBindings(writeMaps(t, senderMaps))
|
||||
play := func(v string) milter.Response {
|
||||
s := &senderSession{bind: bind, signers: NewSigners(filepath.Join(t.TempDir(), "absent"))}
|
||||
_, _ = s.MailFrom("shop@shop.example.org", mods(map[string]string{"{auth_authen}": "shop"}))
|
||||
_, _ = s.Header("From", v, mods(nil))
|
||||
resp, _ := s.Headers(nil, mods(nil))
|
||||
return resp
|
||||
}
|
||||
if resp := play(`"x@example.org" <a@shop.example.org>`); resp != milter.RespTempFail {
|
||||
t.Fatalf("reply %s, want a temporary failure", replyText(resp))
|
||||
}
|
||||
if resp := play(`a@shop.example.org`); resp != milter.RespContinue {
|
||||
t.Fatalf("a plain own address: %s", replyText(resp))
|
||||
}
|
||||
}
|
||||
|
||||
// A second message on the connection is judged by its own headers.
|
||||
func TestSenderCheckResetsBetweenMessages(t *testing.T) {
|
||||
bind := NewBindings(writeMaps(t, senderMaps))
|
||||
s := &senderSession{bind: bind, signers: signers(t)}
|
||||
auth := mods(map[string]string{"{auth_authen}": "shop"})
|
||||
for i, v := range []string{"director@example.org", "a@shop.example.org", "director@example.org"} {
|
||||
_, _ = s.MailFrom("shop@shop.example.org", auth)
|
||||
_, _ = s.Header("From", v, mods(nil))
|
||||
resp, _ := s.Headers(nil, mods(nil))
|
||||
if passed := resp == milter.RespContinue; passed != (i == 1) {
|
||||
t.Fatalf("message %d (From: %s): passed=%v", i, v, passed)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,26 @@
|
||||
-- The sender check: a message's From: line is held against the application
|
||||
-- that sends it (from-header-binding, 2.4.0). DDL only; nothing existing
|
||||
-- changes.
|
||||
|
||||
-- Why a refused row was refused. Until now "rejected" was one status for
|
||||
-- three things, and the row did not say which:
|
||||
-- limit a level-2 rate limit
|
||||
-- client the application's list of client addresses
|
||||
-- from the From: line is not the application's
|
||||
-- A row from before this migration has '' and is shown as plainly rejected.
|
||||
ALTER TABLE send_log ADD COLUMN reason TEXT NOT NULL DEFAULT '';
|
||||
|
||||
-- For a row refused for its From: line, what the header said - cut and
|
||||
-- reduced to printable characters by the check before it is stored. from_addr
|
||||
-- stays the envelope sender, the address the application had the right to.
|
||||
ALTER TABLE send_log ADD COLUMN header_from TEXT NOT NULL DEFAULT '';
|
||||
|
||||
-- What the strict check costs, a row per UTC day that has something to say:
|
||||
-- for how many seconds the panel did not answer while Postfix was running,
|
||||
-- and how many messages Postfix deferred because the check did not answer.
|
||||
-- Kept a year. The first day counted is in settings, "sender_check_since".
|
||||
CREATE TABLE sender_check_days (
|
||||
day TEXT PRIMARY KEY,
|
||||
down_seconds INTEGER NOT NULL DEFAULT 0,
|
||||
deferred INTEGER NOT NULL DEFAULT 0
|
||||
);
|
||||
@@ -176,6 +176,9 @@ type LimitRefusal struct {
|
||||
// The same status is written when an application is refused for its client
|
||||
// address (applications.auth_ip_restrict), which is no limit at all; so a
|
||||
// login counts only while a level-2 limit is set on it or on its domain.
|
||||
// Since 2.4.0 a row says why it was refused, and one refused for its client
|
||||
// address or its From: line is left out by that; a row from before has no
|
||||
// reason and is still judged the old way.
|
||||
func (s *Store) LimitRefusalsSince(since time.Time) ([]LimitRefusal, error) {
|
||||
// The status is a literal so that the partial index of migration 0005
|
||||
// answers the question.
|
||||
@@ -184,7 +187,7 @@ func (s *Store) LimitRefusalsSince(since time.Time) ([]LimitRefusal, error) {
|
||||
FROM send_log l
|
||||
JOIN applications a ON a.login = l.app_login
|
||||
JOIN domains d ON d.id = a.domain_id
|
||||
WHERE l.status = 'rejected' AND l.created_at >= ?
|
||||
WHERE l.status = 'rejected' AND l.created_at >= ? AND l.reason IN ('', 'limit')
|
||||
AND (EXISTS (SELECT 1 FROM rate_limits r WHERE r.scope = 'application' AND r.ref_id = a.id)
|
||||
OR EXISTS (SELECT 1 FROM rate_limits r WHERE r.scope = 'domain' AND r.ref_id = d.id))
|
||||
GROUP BY d.name, a.login ORDER BY d.name, a.login`,
|
||||
|
||||
@@ -47,7 +47,7 @@ func TestBaselineSchema(t *testing.T) {
|
||||
"applications": "id domain_id login address_mode auth_ip_restrict auth_allowed_ips created_at",
|
||||
"application_addresses": "id application_id address",
|
||||
"rate_limits": "id scope ref_id max_messages window_seconds mode auto_multiplier auto_updated_at",
|
||||
"send_log": "id queue_id domain app_login from_addr to_addr subject status created_at updated_at",
|
||||
"send_log": "id queue_id domain app_login from_addr to_addr subject status created_at updated_at reason header_from",
|
||||
"logtail_state": "path fingerprint read_offset updated_at",
|
||||
"inbound_domains": "id name recipient_mode created_at",
|
||||
"inbound_transports": "inbound_domain_id host port tls_mode",
|
||||
@@ -59,6 +59,7 @@ func TestBaselineSchema(t *testing.T) {
|
||||
"inbound_spam_list": "id list_type entry_type value note created_at",
|
||||
"send_log_lines": "id queue_id logged_at line",
|
||||
"notify_state": "key phase count mark since sent_at",
|
||||
"sender_check_days": "day down_seconds deferred",
|
||||
}
|
||||
rows, err := st.db.Query("SELECT name FROM sqlite_master WHERE type = 'table' AND name NOT LIKE 'sqlite_%' ORDER BY name")
|
||||
if err != nil {
|
||||
@@ -113,8 +114,8 @@ func TestBaselineSchema(t *testing.T) {
|
||||
}
|
||||
|
||||
var version, appID int64
|
||||
if err := st.db.QueryRow("PRAGMA user_version").Scan(&version); err != nil || version != 5 {
|
||||
t.Errorf("user_version = %d, %v; want 5", version, err)
|
||||
if err := st.db.QueryRow("PRAGMA user_version").Scan(&version); err != nil || version != 6 {
|
||||
t.Errorf("user_version = %d, %v; want 6", version, err)
|
||||
}
|
||||
if err := st.db.QueryRow("PRAGMA application_id").Scan(&appID); err != nil || appID != applicationID {
|
||||
t.Errorf("application_id = %#x, %v; want %#x", appID, err, applicationID)
|
||||
@@ -190,8 +191,8 @@ func TestOpenMigratesA20Database(t *testing.T) {
|
||||
}
|
||||
defer st.Close()
|
||||
var version int
|
||||
if err := st.db.QueryRow("PRAGMA user_version").Scan(&version); err != nil || version != 5 {
|
||||
t.Errorf("user_version = %d, %v; want 5", version, err)
|
||||
if err := st.db.QueryRow("PRAGMA user_version").Scan(&version); err != nil || version != 6 {
|
||||
t.Errorf("user_version = %d, %v; want 6", version, err)
|
||||
}
|
||||
if ok, err := st.UserExists(); err != nil || !ok {
|
||||
t.Errorf("the user of the 2.0 database: exists = %v, %v", ok, err)
|
||||
|
||||
@@ -0,0 +1,146 @@
|
||||
package store
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Why a send-log row with status "rejected" was refused (send_log.reason).
|
||||
// A row written before the reason was recorded has "".
|
||||
const (
|
||||
ReasonLimit = "limit" // a level-2 rate limit
|
||||
ReasonClient = "client" // the application's list of client addresses
|
||||
ReasonFrom = "from" // the From: line is not the application's
|
||||
)
|
||||
|
||||
// Settings keys of the sender check's statistics.
|
||||
const (
|
||||
// settingSenderCheckSince is the first UTC day the statistics cover.
|
||||
settingSenderCheckSince = "sender_check_since"
|
||||
// settingSenderCheckDeferredAt is when a message was last deferred because
|
||||
// the check did not answer (RFC3339 UTC).
|
||||
settingSenderCheckDeferredAt = "sender_check_deferred_at"
|
||||
)
|
||||
|
||||
// senderCheckKeepDays is how long a day of the statistics is kept.
|
||||
const senderCheckKeepDays = 366
|
||||
|
||||
const dayLayout = "2006-01-02"
|
||||
|
||||
// SenderCheckTotals is what the strict sender check has cost over a period.
|
||||
type SenderCheckTotals struct {
|
||||
Down time.Duration // the panel not answering while Postfix was running
|
||||
Deferred int64 // messages Postfix deferred because of it
|
||||
}
|
||||
|
||||
// SenderCheckStats is the card of Server > Health.
|
||||
type SenderCheckStats struct {
|
||||
Today, Month, Year SenderCheckTotals
|
||||
// Since is the first day counted; zero before anything was.
|
||||
Since time.Time
|
||||
// DeferredAt is when a message was last deferred; zero if never.
|
||||
DeferredAt time.Time
|
||||
}
|
||||
|
||||
// StartSenderCheckStats records the first day the statistics cover, once.
|
||||
func (s *Store) StartSenderCheckStats(now time.Time) error {
|
||||
v, err := s.GetSetting(settingSenderCheckSince)
|
||||
if err != nil || v != "" {
|
||||
return err
|
||||
}
|
||||
return s.SetSetting(settingSenderCheckSince, now.UTC().Format(dayLayout))
|
||||
}
|
||||
|
||||
// AddSenderCheckDown adds to the day of at the time the panel did not answer
|
||||
// while Postfix was running.
|
||||
func (s *Store) AddSenderCheckDown(at time.Time, d time.Duration) error {
|
||||
secs := int64(d / time.Second)
|
||||
if secs <= 0 {
|
||||
return nil
|
||||
}
|
||||
_, err := s.db.Exec(
|
||||
`INSERT INTO sender_check_days (day, down_seconds) VALUES (?, ?)
|
||||
ON CONFLICT (day) DO UPDATE SET down_seconds = down_seconds + excluded.down_seconds`,
|
||||
at.UTC().Format(dayLayout), secs,
|
||||
)
|
||||
if err != nil {
|
||||
return fmt.Errorf("sender check downtime: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// AddSenderCheckDeferred counts, on the day of at, n messages deferred because
|
||||
// the check did not answer, and remembers at as the last such moment.
|
||||
func (s *Store) AddSenderCheckDeferred(at time.Time, n int64) error {
|
||||
if n <= 0 {
|
||||
return nil
|
||||
}
|
||||
_, err := s.db.Exec(
|
||||
`INSERT INTO sender_check_days (day, deferred) VALUES (?, ?)
|
||||
ON CONFLICT (day) DO UPDATE SET deferred = deferred + excluded.deferred`,
|
||||
at.UTC().Format(dayLayout), n,
|
||||
)
|
||||
if err != nil {
|
||||
return fmt.Errorf("sender check deferred: %w", err)
|
||||
}
|
||||
// Lines are read in order, but a re-read log must not move the mark back.
|
||||
last, err := s.GetSetting(settingSenderCheckDeferredAt)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
stamp := at.UTC().Format(time.RFC3339)
|
||||
if stamp > last {
|
||||
return s.SetSetting(settingSenderCheckDeferredAt, stamp)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// SenderCheckStats sums the days up to now: today, the last 30 days and the
|
||||
// last 12 months, each including today.
|
||||
func (s *Store) SenderCheckStats(now time.Time) (SenderCheckStats, error) {
|
||||
now = now.UTC()
|
||||
var out SenderCheckStats
|
||||
sum := func(days int) (SenderCheckTotals, error) {
|
||||
var secs, deferred int64
|
||||
err := s.db.QueryRow(
|
||||
`SELECT COALESCE(SUM(down_seconds), 0), COALESCE(SUM(deferred), 0)
|
||||
FROM sender_check_days WHERE day > ? AND day <= ?`,
|
||||
now.AddDate(0, 0, -days).Format(dayLayout), now.Format(dayLayout),
|
||||
).Scan(&secs, &deferred)
|
||||
if err != nil {
|
||||
return SenderCheckTotals{}, fmt.Errorf("sender check stats: %w", err)
|
||||
}
|
||||
return SenderCheckTotals{Down: time.Duration(secs) * time.Second, Deferred: deferred}, nil
|
||||
}
|
||||
var err error
|
||||
if out.Today, err = sum(1); err != nil {
|
||||
return out, err
|
||||
}
|
||||
if out.Month, err = sum(30); err != nil {
|
||||
return out, err
|
||||
}
|
||||
if out.Year, err = sum(365); err != nil {
|
||||
return out, err
|
||||
}
|
||||
since, err := s.GetSetting(settingSenderCheckSince)
|
||||
if err != nil {
|
||||
return out, err
|
||||
}
|
||||
out.Since, _ = time.Parse(dayLayout, since)
|
||||
deferredAt, err := s.GetSetting(settingSenderCheckDeferredAt)
|
||||
if err != nil {
|
||||
return out, err
|
||||
}
|
||||
out.DeferredAt, _ = time.Parse(time.RFC3339, deferredAt)
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// PruneSenderCheckDays deletes the days older than a year.
|
||||
func (s *Store) PruneSenderCheckDays(now time.Time) error {
|
||||
_, err := s.db.Exec(`DELETE FROM sender_check_days WHERE day < ?`,
|
||||
now.UTC().AddDate(0, 0, -senderCheckKeepDays).Format(dayLayout))
|
||||
if err != nil {
|
||||
return fmt.Errorf("prune sender check days: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,114 @@
|
||||
package store
|
||||
|
||||
import (
|
||||
"path/filepath"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestSenderCheckStats(t *testing.T) {
|
||||
st, err := Open(filepath.Join(t.TempDir(), "selfpost.db"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer st.Close()
|
||||
|
||||
now := time.Date(2026, 10, 20, 15, 0, 0, 0, time.UTC)
|
||||
day := func(n int) time.Time { return now.AddDate(0, 0, -n) }
|
||||
|
||||
// Nothing counted yet: zeros, and no first day.
|
||||
got, err := st.SenderCheckStats(now)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got != (SenderCheckStats{}) {
|
||||
t.Fatalf("empty statistics = %+v", got)
|
||||
}
|
||||
|
||||
// The first day is recorded once.
|
||||
if err := st.StartSenderCheckStats(day(40)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := st.StartSenderCheckStats(now); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
must := func(err error) {
|
||||
t.Helper()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
must(st.AddSenderCheckDown(now, 3*time.Second))
|
||||
must(st.AddSenderCheckDown(now, 1500*time.Millisecond)) // whole seconds: one
|
||||
must(st.AddSenderCheckDown(now, 400*time.Millisecond)) // nothing
|
||||
must(st.AddSenderCheckDeferred(now.Add(-time.Hour), 2))
|
||||
must(st.AddSenderCheckDown(day(10), 30*time.Second))
|
||||
must(st.AddSenderCheckDeferred(day(10), 1))
|
||||
must(st.AddSenderCheckDown(day(29), time.Minute))
|
||||
must(st.AddSenderCheckDown(day(30), time.Hour)) // the 31st day back: the year's only
|
||||
must(st.AddSenderCheckDeferred(day(200), 5))
|
||||
must(st.AddSenderCheckDown(day(365), 24*time.Hour)) // a year and a day: nobody's
|
||||
|
||||
got, err = st.SenderCheckStats(now)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
want := SenderCheckStats{
|
||||
Today: SenderCheckTotals{Down: 4 * time.Second, Deferred: 2},
|
||||
Month: SenderCheckTotals{Down: 94 * time.Second, Deferred: 3},
|
||||
Year: SenderCheckTotals{Down: 94*time.Second + time.Hour, Deferred: 8},
|
||||
Since: time.Date(2026, 9, 10, 0, 0, 0, 0, time.UTC),
|
||||
DeferredAt: now.Add(-time.Hour),
|
||||
}
|
||||
if got != want {
|
||||
t.Fatalf("statistics\n got %+v\nwant %+v", got, want)
|
||||
}
|
||||
|
||||
// The last deferral is the latest one seen, not the last one written: a
|
||||
// log read again after an unclean stop does not move it back.
|
||||
if got.DeferredAt != now.Add(-time.Hour) {
|
||||
t.Fatalf("an older deferral moved the mark to %s", got.DeferredAt)
|
||||
}
|
||||
|
||||
// A year on, the old days are gone.
|
||||
must(st.PruneSenderCheckDays(now.AddDate(1, 0, 0)))
|
||||
var n int
|
||||
if err := st.db.QueryRow(`SELECT COUNT(*) FROM sender_check_days`).Scan(&n); err != nil || n != 1 {
|
||||
t.Fatalf("%d days left after a year's prune (%v), want the newest alone", n, err)
|
||||
}
|
||||
}
|
||||
|
||||
// A refused row keeps why it was refused and, for its From: line, what the
|
||||
// line said; an accepted row has neither.
|
||||
func TestRefusedRowKeepsItsReason(t *testing.T) {
|
||||
st, err := Open(filepath.Join(t.TempDir(), "selfpost.db"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer st.Close()
|
||||
|
||||
if err := st.InsertRejected(SendLogEntry{Domain: "shop.example.org", AppLogin: "shop", From: "shop@shop.example.org",
|
||||
To: "k.berg@web.de", Subject: "s", Reason: ReasonFrom, HeaderFrom: "director@example.org"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := st.InsertQueued(SendLogEntry{QueueID: "4XcB7k2Jm9z1", Domain: "shop.example.org", AppLogin: "shop",
|
||||
From: "shop@shop.example.org", To: "k.berg@web.de"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
rows, err := st.QuerySendLog(SendLogFilter{AllDomains: true}, 10, 0)
|
||||
if err != nil || len(rows) != 2 {
|
||||
t.Fatalf("rows = %d, %v", len(rows), err)
|
||||
}
|
||||
queued, refused := rows[0], rows[1]
|
||||
if refused.Status != StatusRejected || refused.Reason != ReasonFrom || refused.HeaderFrom != "director@example.org" || refused.To != "k.berg@web.de" {
|
||||
t.Errorf("refused row = %+v", refused)
|
||||
}
|
||||
if queued.Reason != "" || queued.HeaderFrom != "" {
|
||||
t.Errorf("queued row carries a refusal: %+v", queued)
|
||||
}
|
||||
one, err := st.GetSendLog(refused.ID)
|
||||
if err != nil || one.Reason != ReasonFrom || one.HeaderFrom != "director@example.org" {
|
||||
t.Errorf("GetSendLog = %+v, %v", one, err)
|
||||
}
|
||||
}
|
||||
+23
-16
@@ -34,6 +34,10 @@ type SendLogEntry struct {
|
||||
From string
|
||||
To string
|
||||
Subject string
|
||||
// Reason and HeaderFrom are set on refused rows only: why the message
|
||||
// was refused (Reason*), and for ReasonFrom what its From: line said.
|
||||
Reason string
|
||||
HeaderFrom string
|
||||
}
|
||||
|
||||
// InsertQueued records an accepted message in the send log with status
|
||||
@@ -63,9 +67,9 @@ func (s *Store) InsertRejected(e SendLogEntry) error {
|
||||
now := time.Now().UTC().Format(time.RFC3339)
|
||||
_, err := s.db.Exec(
|
||||
`INSERT INTO send_log
|
||||
(queue_id, domain, app_login, from_addr, to_addr, subject, status, created_at, updated_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)`,
|
||||
e.QueueID, e.Domain, e.AppLogin, e.From, e.To, e.Subject, StatusRejected, now, now,
|
||||
(queue_id, domain, app_login, from_addr, to_addr, subject, status, reason, header_from, created_at, updated_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
|
||||
e.QueueID, e.Domain, e.AppLogin, e.From, e.To, e.Subject, StatusRejected, e.Reason, e.HeaderFrom, now, now,
|
||||
)
|
||||
if err != nil {
|
||||
return fmt.Errorf("insert rejected send_log: %w", err)
|
||||
@@ -136,15 +140,18 @@ func (s *Store) ListQueuedOlderThan(cutoff time.Time) ([]QueuedDelivery, error)
|
||||
// Persistence): a SendLogEntry plus the fields that only exist once a row has
|
||||
// been written (id, current status, timestamps).
|
||||
type SendLogRow struct {
|
||||
ID int64
|
||||
QueueID string
|
||||
Domain string
|
||||
AppLogin string
|
||||
From string
|
||||
To string
|
||||
Subject string
|
||||
Status string
|
||||
CreatedAt time.Time
|
||||
ID int64
|
||||
QueueID string
|
||||
Domain string
|
||||
AppLogin string
|
||||
From string
|
||||
To string
|
||||
Subject string
|
||||
Status string
|
||||
// Reason and HeaderFrom: see SendLogEntry.
|
||||
Reason string
|
||||
HeaderFrom string
|
||||
CreatedAt time.Time
|
||||
// UpdatedAt is when the status last changed — the log-tailer's report of
|
||||
// the delivery attempt. It equals CreatedAt while a row is still queued.
|
||||
UpdatedAt time.Time
|
||||
@@ -163,10 +170,10 @@ func (s *Store) GetSendLog(id int64) (SendLogRow, error) {
|
||||
createdAt, updatedAt string
|
||||
)
|
||||
err := s.db.QueryRow(
|
||||
`SELECT id, queue_id, domain, app_login, from_addr, to_addr, subject, status, created_at, updated_at
|
||||
`SELECT id, queue_id, domain, app_login, from_addr, to_addr, subject, status, reason, header_from, created_at, updated_at
|
||||
FROM send_log WHERE id = ?`, id,
|
||||
).Scan(&row.ID, &row.QueueID, &row.Domain, &row.AppLogin, &row.From, &row.To,
|
||||
&row.Subject, &row.Status, &createdAt, &updatedAt)
|
||||
&row.Subject, &row.Status, &row.Reason, &row.HeaderFrom, &createdAt, &updatedAt)
|
||||
if errors.Is(err, sql.ErrNoRows) {
|
||||
return SendLogRow{}, ErrSendLogNotFound
|
||||
}
|
||||
@@ -205,7 +212,7 @@ func (s *Store) QuerySendLog(filter SendLogFilter, limit, offset int) ([]SendLog
|
||||
where, args := sendLogWhere(filter)
|
||||
args = append(args, limit, offset)
|
||||
rows, err := s.db.Query(
|
||||
`SELECT id, queue_id, domain, app_login, from_addr, to_addr, subject, status, created_at, updated_at
|
||||
`SELECT id, queue_id, domain, app_login, from_addr, to_addr, subject, status, reason, header_from, created_at, updated_at
|
||||
FROM send_log`+where+`
|
||||
ORDER BY id DESC
|
||||
LIMIT ? OFFSET ?`,
|
||||
@@ -223,7 +230,7 @@ func (s *Store) QuerySendLog(filter SendLogFilter, limit, offset int) ([]SendLog
|
||||
createdAt, updatedAt string
|
||||
)
|
||||
if err := rows.Scan(&row.ID, &row.QueueID, &row.Domain, &row.AppLogin,
|
||||
&row.From, &row.To, &row.Subject, &row.Status, &createdAt, &updatedAt); err != nil {
|
||||
&row.From, &row.To, &row.Subject, &row.Status, &row.Reason, &row.HeaderFrom, &createdAt, &updatedAt); err != nil {
|
||||
return nil, fmt.Errorf("scan send_log row: %w", err)
|
||||
}
|
||||
row.CreatedAt, _ = time.Parse(time.RFC3339, createdAt)
|
||||
|
||||
@@ -28,6 +28,7 @@ type Config struct {
|
||||
TLSCertFile string
|
||||
OpenDKIMSocket string
|
||||
JournalSocket string
|
||||
SenderSocket string
|
||||
// Level-1 Postfix anvil backstop (env RATE_LIMIT_*), shown in the panel
|
||||
// and used to cap domain/app level-2 ceilings (guide § Rate limiting).
|
||||
RateLimitMessagesPerIP int
|
||||
|
||||
@@ -25,7 +25,8 @@ func socketsServer(t *testing.T) *Handlers {
|
||||
dir := t.TempDir()
|
||||
h.cfg.OpenDKIMSocket = filepath.Join(dir, "opendkim.sock")
|
||||
h.cfg.JournalSocket = filepath.Join(dir, "journal.sock")
|
||||
for _, path := range []string{h.cfg.OpenDKIMSocket, h.cfg.JournalSocket} {
|
||||
h.cfg.SenderSocket = filepath.Join(dir, "sender.sock")
|
||||
for _, path := range []string{h.cfg.OpenDKIMSocket, h.cfg.SenderSocket, h.cfg.JournalSocket} {
|
||||
l, err := net.Listen("unix", path)
|
||||
if err != nil {
|
||||
t.Skipf("unix sockets unavailable here: %v", err)
|
||||
@@ -55,19 +56,19 @@ func TestFilterOffAddsNoSocket(t *testing.T) {
|
||||
h := socketsServer(t)
|
||||
h.cfg.InboundEnabled = true
|
||||
h.cfg.InboundAntispamMilter = "" // no filter attached
|
||||
if got := socketsOf(t, h); len(got) != 2 {
|
||||
t.Fatalf("%d sockets with the filter off, want 2: %+v", len(got), got)
|
||||
if got := socketsOf(t, h); len(got) != 3 {
|
||||
t.Fatalf("%d sockets with the filter off, want 3: %+v", len(got), got)
|
||||
}
|
||||
|
||||
// A milter address with the inbound relay off is not a filter either.
|
||||
h.cfg.InboundEnabled = false
|
||||
h.cfg.InboundAntispamMilter = "inet:127.0.0.1:1"
|
||||
got := socketsOf(t, h)
|
||||
if len(got) != 2 {
|
||||
t.Fatalf("%d sockets with the inbound relay off, want 2: %+v", len(got), got)
|
||||
if len(got) != 3 {
|
||||
t.Fatalf("%d sockets with the inbound relay off, want 3: %+v", len(got), got)
|
||||
}
|
||||
card := socketCard(t, h)
|
||||
if card.Value != "2 of 2 answering" || card.Sub != "OpenDKIM, send-log" || card.Level != "" {
|
||||
if card.Value != "3 of 3 answering" || card.Sub != "OpenDKIM, sender-check, send-log" || card.Level != "" {
|
||||
t.Errorf("card = %+v", card)
|
||||
}
|
||||
}
|
||||
@@ -83,7 +84,7 @@ func TestFilterOnAnsweringIsTheThirdSocket(t *testing.T) {
|
||||
h.cfg.InboundAntispamMilter, h.cfg.InboundAntispamAction = "inet:"+l.Addr().String(), "tempfail"
|
||||
|
||||
got := socketsOf(t, h)
|
||||
if len(got) != 3 || got[2].Name != "Spam filter" || got[2].Note != "inbound" || got[2].Path != "inet:"+l.Addr().String() {
|
||||
if len(got) != 4 || got[3].Name != "Spam filter" || got[3].Note != "inbound" || got[3].Path != "inet:"+l.Addr().String() {
|
||||
t.Fatalf("sockets = %+v", got)
|
||||
}
|
||||
for _, s := range got {
|
||||
@@ -92,7 +93,7 @@ func TestFilterOnAnsweringIsTheThirdSocket(t *testing.T) {
|
||||
}
|
||||
}
|
||||
card := socketCard(t, h)
|
||||
if card.Value != "3 of 3 answering" || card.Sub != "incl. the spam filter" || card.Level != "" {
|
||||
if card.Value != "4 of 4 answering" || card.Sub != "incl. the spam filter" || card.Level != "" {
|
||||
t.Errorf("card = %+v", card)
|
||||
}
|
||||
|
||||
@@ -132,10 +133,10 @@ func TestFilterDownIsAMissingSocket(t *testing.T) {
|
||||
} {
|
||||
h.cfg.InboundAntispamMilter, h.cfg.InboundAntispamAction = dead, tt.action
|
||||
got := socketsOf(t, h)
|
||||
if len(got) != 3 {
|
||||
t.Fatalf("action %q: %d sockets, want 3", tt.action, len(got))
|
||||
if len(got) != 4 {
|
||||
t.Fatalf("action %q: %d sockets, want 4", tt.action, len(got))
|
||||
}
|
||||
f := got[2]
|
||||
f := got[3]
|
||||
refused := "Connection to " + l.Addr().String() + " refused. " + tt.detail
|
||||
if f.Present || f.Status != tt.status || f.Detail != refused {
|
||||
t.Errorf("action %q: filter = %+v, want status %q, detail %q", tt.action, f, tt.status, refused)
|
||||
@@ -146,7 +147,7 @@ func TestFilterDownIsAMissingSocket(t *testing.T) {
|
||||
t.Errorf("action %q: SocketStatus = %q, want %q", tt.action, c["SocketStatus"], tt.status)
|
||||
}
|
||||
card := socketCard(t, h)
|
||||
if card.Value != "2 of 3 answering" || card.Level != tt.level || !strings.HasSuffix(card.Sub, "Spam filter") {
|
||||
if card.Value != "3 of 4 answering" || card.Level != tt.level || !strings.HasSuffix(card.Sub, "Spam filter") {
|
||||
t.Errorf("action %q: card = %+v", tt.action, card)
|
||||
}
|
||||
|
||||
@@ -160,7 +161,7 @@ func TestFilterDownIsAMissingSocket(t *testing.T) {
|
||||
rec := httptest.NewRecorder()
|
||||
h.HandleOverviewFragment(rec, reqAs("/overview/fragment", globalPrincipal))
|
||||
page := rec.Body.String()
|
||||
for _, want := range []string{`>` + tt.stamp + `<`, `Milter sockets</a> (2 of 3 answering)`} {
|
||||
for _, want := range []string{`>` + tt.stamp + `<`, `Milter sockets</a> (3 of 4 answering)`} {
|
||||
if !strings.Contains(page, want) {
|
||||
t.Errorf("action %q: Overview is missing %q", tt.action, want)
|
||||
}
|
||||
|
||||
@@ -95,13 +95,13 @@ func TestHealthPageRendersEveryCheck(t *testing.T) {
|
||||
if strings.Contains(out, "Listening") {
|
||||
t.Error("a socket that answers repeats its detail beside the tag")
|
||||
}
|
||||
// The reload control stays outside the polled rows.
|
||||
body, conf := strings.Index(out, `id="health-machine"`), strings.Index(out, `id="configuration"`)
|
||||
if body < 0 || conf < body {
|
||||
t.Fatal("polled rows or the Configuration box missing or out of order")
|
||||
// The reload control stays outside the polled stack, in the other one.
|
||||
body, right, conf := strings.Index(out, `id="health-stack"`), strings.Index(out, `id="sockets"`), strings.Index(out, `id="configuration"`)
|
||||
if body < 0 || right < body || conf < right {
|
||||
t.Fatal("the polled stack or the Configuration box missing or out of order")
|
||||
}
|
||||
if frag := out[body:conf]; strings.Contains(frag, `action="/server/health/reload"`) || !strings.Contains(frag, `id="hostname"`) {
|
||||
t.Error("the hostname box is outside the polled rows, or the reload control inside them")
|
||||
if frag := out[body:right]; strings.Contains(frag, `action="/server/health/reload"`) || !strings.Contains(frag, `id="hostname"`) || !strings.Contains(frag, `id="sender-check"`) {
|
||||
t.Error("the hostname or sender check box is outside the polled stack, or the reload control inside it")
|
||||
}
|
||||
}
|
||||
|
||||
@@ -163,8 +163,9 @@ func TestHealthFragmentCarriesTheSameRows(t *testing.T) {
|
||||
t.Fatalf("GET /server/health/fragment = %d:\n%s", rec.Code, out)
|
||||
}
|
||||
for _, want := range []string{
|
||||
`id="health-machine" data-poll`, `hx-get="/server/health/fragment"`,
|
||||
`id="health-checks" hx-swap-oob="true"`, `id="machine"`, `id="processes"`, `id="certificate"`, `id="sockets"`, `id="hostname"`,
|
||||
`id="health-stack" data-poll`, `hx-get="/server/health/fragment"`,
|
||||
`id="sockets" hx-swap-oob="true"`, `id="certificate" hx-swap-oob="true"`, `id="notification-mail" hx-swap-oob="true"`,
|
||||
`id="machine"`, `id="processes"`, `id="sender-check"`, `id="hostname"`,
|
||||
} {
|
||||
if !strings.Contains(out, want) {
|
||||
t.Errorf("health fragment is missing %q", want)
|
||||
|
||||
@@ -104,6 +104,7 @@ func (h *Handlers) HandleDelivery(w http.ResponseWriter, r *http.Request) {
|
||||
in := view.MessageInput{
|
||||
ID: row.ID, QueueID: row.QueueID, Domain: row.Domain, App: row.AppLogin,
|
||||
From: row.From, To: row.To, Subject: row.Subject, Status: row.Status,
|
||||
Reason: row.Reason, HeaderFrom: row.HeaderFrom,
|
||||
Accepted: row.CreatedAt, Reported: row.UpdatedAt,
|
||||
// Where the row came from, so the way back returns to the page and filters
|
||||
// the operator was looking at rather than the top of an unfiltered log.
|
||||
@@ -122,13 +123,31 @@ func (h *Handlers) HandleDelivery(w http.ResponseWriter, r *http.Request) {
|
||||
break
|
||||
}
|
||||
page := view.NewOutMessage(h.shellMeta(r), in).
|
||||
WithHistory(deliveryEvents(row, h.cfg.RetryPolicy)).
|
||||
WithHistory(deliveryEvents(row, h.cfg.RetryPolicy, h.sendsAs(row.AppLogin))).
|
||||
// The mail.log lines for this message, and — when there are none — the
|
||||
// reason, which is a normal outcome rather than a failure.
|
||||
WithDeliveryLog(logLines, logNote)
|
||||
h.view.Render(w, http.StatusOK, "out-message", page)
|
||||
}
|
||||
|
||||
// sendsAs says what an application may send as, in the words of a refusal:
|
||||
// any address of its domain, or the addresses of its list. Empty when the
|
||||
// login is no application's any more.
|
||||
func (h *Handlers) sendsAs(login string) view.Text {
|
||||
a, err := h.store.GetApplicationByLogin(login)
|
||||
if err != nil {
|
||||
return nil
|
||||
}
|
||||
if a.AddressMode == store.AddressModeList {
|
||||
return view.Plain("the addresses of its list")
|
||||
}
|
||||
d, err := h.store.GetDomain(a.DomainID)
|
||||
if err != nil {
|
||||
return nil
|
||||
}
|
||||
return view.Rich("any address of ", view.Mono(d.Name))
|
||||
}
|
||||
|
||||
// deliveryEvents turns a row's two timestamps into the history the page shows.
|
||||
// The journal keeps no event table — a row is created when the message is
|
||||
// accepted and updated once when Postfix reports the attempt — so the two
|
||||
@@ -137,7 +156,10 @@ func (h *Handlers) HandleDelivery(w http.ResponseWriter, r *http.Request) {
|
||||
// six hours, then delivered" rather than as two dates in a list of fields.
|
||||
// policy supplies the human intervals for deferred and bounced copy, the same
|
||||
// strings the Mail queue card prints, so the two cannot drift.
|
||||
func deliveryEvents(row store.SendLogRow, policy postfix.RetryPolicy) []view.Step {
|
||||
//
|
||||
// sendsAs is what the row's application may send as, for the step of a message
|
||||
// refused for its From: line; empty when the application is gone.
|
||||
func deliveryEvents(row store.SendLogRow, policy postfix.RetryPolicy, sendsAs view.Text) []view.Step {
|
||||
// step states what happened and when; outcome is ok, warn, error or "" for a
|
||||
// step that reports nothing either way, and a zero time is a step that has
|
||||
// not happened yet.
|
||||
@@ -153,6 +175,22 @@ func deliveryEvents(row store.SendLogRow, policy postfix.RetryPolicy) []view.Ste
|
||||
// A rejected message has no second step, and its first one is not an
|
||||
// acceptance: the journal-milter refused it, so Postfix never queued it.
|
||||
if row.Status == store.StatusRejected {
|
||||
switch row.Reason {
|
||||
case store.ReasonFrom:
|
||||
// The one step that names things: which line, whose, and what that
|
||||
// application may write instead.
|
||||
text := view.Rich(" — ", view.Code("From: "+row.HeaderFrom), " is not an address of application ", view.Mono(row.AppLogin))
|
||||
if len(sendsAs) > 0 {
|
||||
text = append(append(text, view.Inline{Text: ", which sends as "}), sendsAs...)
|
||||
}
|
||||
text = append(text, view.Rich(". The application was told ", view.Code("550 5.7.1"), " and will not try again.")...)
|
||||
s := step(row.CreatedAt, "error", "Refused before queueing", "")
|
||||
s.Text = text
|
||||
return []view.Step{s}
|
||||
case store.ReasonClient:
|
||||
return []view.Step{step(row.CreatedAt, "error", "Refused before queueing",
|
||||
"The application accepts mail only from its listed client addresses, and this message came from another. It was never queued, so there is no queue id and Postfix never attempted delivery.")}
|
||||
}
|
||||
return []view.Step{step(row.CreatedAt, "error", "Refused before queueing",
|
||||
"The journal-milter refused the message under a rate limit. It was never queued, so there is no queue id and Postfix never attempted delivery.")}
|
||||
}
|
||||
@@ -336,7 +374,7 @@ func (h *Handlers) outLogPage(r *http.Request) (*view.OutLog, error) {
|
||||
for i, row := range rows {
|
||||
table[i] = view.OutLogRow{
|
||||
Time: view.FormatLogTime(row.CreatedAt), From: row.From, To: row.To,
|
||||
Subject: mailhdr.DecodeSubject(row.Subject), Status: view.Tag{Status: row.Status},
|
||||
Subject: mailhdr.DecodeSubject(row.Subject), Status: view.SendLogTag(row.Status, row.Reason),
|
||||
Href: out.DetailHref(row.ID, page),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -269,7 +269,7 @@ func TestHealthNotificationMailCard(t *testing.T) {
|
||||
}
|
||||
|
||||
// With both records in place the card is green. The card is part of the polled
|
||||
// fragment, swapped in by its id like the second row.
|
||||
// fragment, swapped in by its id.
|
||||
func TestHealthNotificationMailCardIsGreenAndPolled(t *testing.T) {
|
||||
h, zone, mail := serverMailStand(t)
|
||||
publishMail(zone, mail)
|
||||
@@ -279,7 +279,7 @@ func TestHealthNotificationMailCardIsGreenAndPolled(t *testing.T) {
|
||||
h.HandleHealthFragment(rec, auth.RequestWithPrincipal(req, globalPrincipal))
|
||||
out := rec.Body.String()
|
||||
for _, want := range []string{
|
||||
`id="health-mail" hx-swap-oob="true"`,
|
||||
`id="notification-mail" hx-swap-oob="true"`,
|
||||
`<h2>Notification mail</h2><div class="sp-end"><span class="tag is-success is-light">ok</span></div>`,
|
||||
`<td><span class="tag is-success is-light">published</span></td>`,
|
||||
} {
|
||||
@@ -287,7 +287,7 @@ func TestHealthNotificationMailCardIsGreenAndPolled(t *testing.T) {
|
||||
t.Errorf("health fragment is missing %q:\n%s", want, out)
|
||||
}
|
||||
}
|
||||
if strings.Contains(out[strings.Index(out, `id="health-mail"`):], "is-warning") {
|
||||
if strings.Contains(out[strings.Index(out, `id="notification-mail"`):], "is-warning") {
|
||||
t.Error("the card warns while both records are published")
|
||||
}
|
||||
}
|
||||
|
||||
@@ -4,6 +4,7 @@ import (
|
||||
"fmt"
|
||||
"net/http"
|
||||
"regexp"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
@@ -118,15 +119,13 @@ func (h *Handlers) HandleHealth(w http.ResponseWriter, r *http.Request) {
|
||||
h.view.Render(w, http.StatusOK, "health", h.healthView(r, statusFlash(r)))
|
||||
}
|
||||
|
||||
// HandleHealthFragment is the polled part of Health: the same two rows of
|
||||
// boxes the page holds, read again.
|
||||
// HandleHealthFragment is the polled part of Health: the boxes the page holds,
|
||||
// all but Configuration, read again.
|
||||
func (h *Handlers) HandleHealthFragment(w http.ResponseWriter, r *http.Request) {
|
||||
if _, ok := h.requireGlobal(w, r); !ok {
|
||||
return
|
||||
}
|
||||
page := h.healthView(r, "")
|
||||
page.Refresh = true
|
||||
h.view.RenderFragment(w, http.StatusOK, "health_body", page)
|
||||
h.view.RenderFragment(w, http.StatusOK, "health_body", h.healthView(r, "").AsFragment())
|
||||
}
|
||||
|
||||
// HandleHealthRecheck re-runs the DNS checks of the server ignoring the cache.
|
||||
@@ -144,7 +143,31 @@ func (h *Handlers) HandleHealthRecheck(w http.ResponseWriter, r *http.Request) {
|
||||
func (h *Handlers) healthView(r *http.Request, flash string) *view.Health {
|
||||
page := healthPage(h.shellMeta(r), h.healthChecks(), flash)
|
||||
mail := h.serverMail(false)
|
||||
return page.WithNotificationMail(notify.From(h.cfg.Hostname), mail.rows, mail.status)
|
||||
page.WithNotificationMail(notify.From(h.cfg.Hostname), mail.rows, mail.status)
|
||||
stats, err := h.store.SenderCheckStats(time.Now())
|
||||
if err != nil {
|
||||
logf("panel: health: sender check statistics: %v", err)
|
||||
}
|
||||
rows, since, status := senderCheckCard(stats, time.Now())
|
||||
return page.WithSenderCheck(rows, since, status)
|
||||
}
|
||||
|
||||
// senderCheckCard is the Sender check card from its statistics. It warns when
|
||||
// a message was deferred in the last 24 hours: that is mail an application had
|
||||
// to send again. Downtime that deferred nothing is only shown.
|
||||
func senderCheckCard(s store.SenderCheckStats, now time.Time) (rows []view.SenderCheckRow, since, status string) {
|
||||
row := func(period string, t store.SenderCheckTotals) view.SenderCheckRow {
|
||||
return view.SenderCheckRow{Period: period, Down: view.Downtime(t.Down), Deferred: strconv.FormatInt(t.Deferred, 10)}
|
||||
}
|
||||
rows = []view.SenderCheckRow{row("Today", s.Today), row("Last 30 days", s.Month), row("Last 12 months", s.Year)}
|
||||
if !s.Since.IsZero() {
|
||||
since = s.Since.Format("2006-01-02")
|
||||
}
|
||||
status = string(health.StatusOK)
|
||||
if !s.DeferredAt.IsZero() && now.Sub(s.DeferredAt) < 24*time.Hour {
|
||||
status = string(health.StatusWarn)
|
||||
}
|
||||
return rows, since, status
|
||||
}
|
||||
|
||||
// healthPage turns one reading of the checks (healthChecks) into the Health
|
||||
@@ -397,6 +420,7 @@ func (h *Handlers) healthChecks() map[string]any {
|
||||
cert := health.CheckCertificate(h.cfg.TLSCertFile)
|
||||
sockets := []health.Socket{
|
||||
health.CheckSocket("OpenDKIM", h.cfg.OpenDKIMSocket, true),
|
||||
health.CheckSocket("sender-check", h.cfg.SenderSocket, true),
|
||||
health.CheckSocket("send-log", h.cfg.JournalSocket, false),
|
||||
}
|
||||
if on, milter, action := h.inboundFilter(); on {
|
||||
|
||||
@@ -281,6 +281,9 @@ type Box struct {
|
||||
Help *HelpLink
|
||||
Variant string
|
||||
Poll string
|
||||
// OOB marks the box to be swapped into the page by its ID when it comes
|
||||
// in a polled fragment beside the element that asked for the fragment.
|
||||
OOB bool
|
||||
// DomainPick: see Box.
|
||||
DomainPick bool
|
||||
}
|
||||
|
||||
@@ -92,7 +92,14 @@ func overviewFixture() *Overview {
|
||||
// record. refresh renders it as the polled fragment.
|
||||
func healthFixture(refresh bool) *Health {
|
||||
h := NewHealth(admin(), "")
|
||||
h.Refresh = refresh
|
||||
if refresh {
|
||||
h.AsFragment()
|
||||
}
|
||||
h.WithSenderCheck([]SenderCheckRow{
|
||||
{Period: "Today", Down: "0 s", Deferred: "0"},
|
||||
{Period: "Last 30 days", Down: "4 s", Deferred: "0"},
|
||||
{Period: "Last 12 months", Down: "41 s", Deferred: "3"},
|
||||
}, "2026-10-13", "ok")
|
||||
h.Machine.End = Verdict("ok")
|
||||
h.MachineRows = []MachineRow{
|
||||
{Resource: "CPU", Gauge: &Gauge{Percent: 12, Text: "12%", Level: LevelOK}, Detail: []string{"2 cores · 2 threads"}},
|
||||
@@ -113,6 +120,7 @@ func healthFixture(refresh bool) *Health {
|
||||
h.Sockets.End = Verdict("ok")
|
||||
h.SocketRows = []SocketRow{
|
||||
{Name: "OpenDKIM", Path: "inet:127.0.0.1:8891", State: Tag{Status: "ok"}},
|
||||
{Name: "sender-check", Path: "unix:/run/selfpost/sender.sock", State: Tag{Status: "ok"}},
|
||||
{Name: "send-log", Path: "unix:/run/selfpost/journal.sock", State: Tag{Status: "ok"}},
|
||||
}
|
||||
h.Hostname.End = Verdict("ok")
|
||||
@@ -245,8 +253,10 @@ func outDomainDeleteFixture() *OutDomainDelete {
|
||||
func outLogFixture() *OutLog {
|
||||
p := NewOutLog(admin(), 30, true,
|
||||
[]string{"example.org", "shop.example.org", "notify.acme.io"}, []string{"prod-server", "alerts"}, "", "")
|
||||
// status is a row's status, or for a refused row "rejected:" and its reason.
|
||||
row := func(id int64, at, from, to, subject, status string) OutLogRow {
|
||||
return OutLogRow{Time: at, From: from, To: to, Subject: subject, Status: Tag{Status: status},
|
||||
status, reason, _ := strings.Cut(status, ":")
|
||||
return OutLogRow{Time: at, From: from, To: to, Subject: subject, Status: SendLogTag(status, reason),
|
||||
Href: p.DetailHref(id, 1)}
|
||||
}
|
||||
return p.WithRows([]OutLogRow{
|
||||
@@ -254,7 +264,8 @@ func outLogFixture() *OutLog {
|
||||
row(184222, "09-21 14:01:50", "shop@shop.example.org", "anna.lind@outlook.com", "Your order #48212 has shipped", "sent"),
|
||||
row(184220, "09-21 13:58:07", "alerts@example.org", "noc@example.org", "[FIRING] disk usage above 90 % on db-2", "deferred"),
|
||||
row(184211, "09-21 13:41:19", "no-reply@notify.acme.io", "j.doe@nonexistent.example", "Reset your password", "bounced"),
|
||||
row(184209, "09-21 13:40:02", "prod@example.org", "billing@partner.example", "Invoice 2026-0917", "rejected"),
|
||||
row(184231, "09-21 13:40:55", "shop@shop.example.org", "k.berg@web.de", "A message from the director", "rejected:from"),
|
||||
row(184209, "09-21 13:40:02", "prod@example.org", "billing@partner.example", "Invoice 2026-0917", "rejected:limit"),
|
||||
row(184198, "09-21 13:12:44", "shop@shop.example.org", "p.novak@seznam.cz", "We received your return request", "sent"),
|
||||
}, 1, 412)
|
||||
}
|
||||
|
||||
+71
-20
@@ -1,30 +1,38 @@
|
||||
package view
|
||||
|
||||
import (
|
||||
"strconv"
|
||||
"time"
|
||||
)
|
||||
|
||||
// The data of Server › Health (templates/health.html, mockup health.html):
|
||||
// every check behind the cards of Overview, as tables, and the home of the two
|
||||
// server actions, Re-check DNS and Reload configuration.
|
||||
//
|
||||
// Two rows of the page are polled (templates/health_body.html): the handler
|
||||
// builds the same Health for the page and for its fragment, and the fragment
|
||||
// carries Refresh so that its second row is swapped in out of band.
|
||||
// The page is two stacks of boxes numbered by importance (the `cards` layout),
|
||||
// and all but the Configuration box are polled (templates/health_body.html):
|
||||
// the handler builds the same Health for the page and for its fragment, and
|
||||
// AsFragment marks what the fragment swaps in out of band.
|
||||
|
||||
// Health is the data of the Health page.
|
||||
type Health struct {
|
||||
Meta
|
||||
Head Head
|
||||
Flash *Flash
|
||||
// Refresh is set on the polled fragment only. The fragment answers with two
|
||||
// sibling rows; the first replaces the element that polls, and the second
|
||||
// is marked to be swapped into place by its id.
|
||||
Refresh bool
|
||||
|
||||
Machine Box
|
||||
MachineRows []MachineRow
|
||||
|
||||
Head Head
|
||||
Flash *Flash
|
||||
Processes Box
|
||||
ProcessRows []ProcessRow
|
||||
ProcessError bool
|
||||
|
||||
// SenderCheck is what the strict check of the From: line has cost: for how
|
||||
// long the panel did not answer while Postfix was running, and how many
|
||||
// messages were deferred for it. SenderNote is the line under its table.
|
||||
SenderCheck Box
|
||||
SenderRows []SenderCheckRow
|
||||
SenderNote Text
|
||||
|
||||
Machine Box
|
||||
MachineRows []MachineRow
|
||||
|
||||
Certificate Box
|
||||
CertFacts []Fact
|
||||
CertDetail string
|
||||
@@ -37,8 +45,7 @@ type Health struct {
|
||||
|
||||
// NotifyMail is the card of the DNS records notifications are signed and
|
||||
// authorised with (their values are under Server › Settings); NotifyNote
|
||||
// is the line under its table. It is polled with the rest, so Refresh swaps
|
||||
// it into place on its own.
|
||||
// is the line under its table.
|
||||
NotifyMail Box
|
||||
NotifyRows []MailRow
|
||||
NotifyNote Text
|
||||
@@ -47,6 +54,13 @@ type Health struct {
|
||||
Reload Action
|
||||
}
|
||||
|
||||
// SenderCheckRow is one period of the Sender check table.
|
||||
type SenderCheckRow struct {
|
||||
Period string
|
||||
Down string
|
||||
Deferred string
|
||||
}
|
||||
|
||||
// MachineRow is one line of the Machine table. A row that is a level (CPU,
|
||||
// memory) carries a Gauge; one that is a rate (network) carries Usage text; a
|
||||
// reading that could not be taken carries neither and shows a dash. Detail is
|
||||
@@ -122,14 +136,17 @@ func NewHealth(m Meta, flash string) *Health {
|
||||
Lead: Plain("Everything behind the cards on Overview. Refreshes automatically."),
|
||||
Actions: []Action{{Label: "Re-check DNS", Icon: "ti-refresh", Post: "/server/health/recheck"}},
|
||||
},
|
||||
Machine: Box{No: "01", Title: "Machine", ID: "machine", Help: topicLink(HelpChecks, "What these readings mean")},
|
||||
Processes: Box{No: "02", Title: "Processes", ID: "processes"},
|
||||
Certificate: Box{No: "03", Title: "TLS certificate", ID: "certificate", Help: topicLink(HelpCertificate, "When it warns and what to do")},
|
||||
Sockets: Box{No: "04", Title: "Milter sockets", ID: "sockets"},
|
||||
// Numbered by importance; the odd ones are the left stack.
|
||||
Processes: Box{No: "01", Title: "Processes", ID: "processes"},
|
||||
Sockets: Box{No: "02", Title: "Milter sockets", ID: "sockets"},
|
||||
SenderCheck: Box{No: "03", Title: "Sender check", ID: "sender-check",
|
||||
Help: topicLink(HelpSenderCheck, "What is checked and what these numbers are for")},
|
||||
Certificate: Box{No: "04", Title: "TLS certificate", ID: "certificate", Help: topicLink(HelpCertificate, "When it warns and what to do")},
|
||||
Hostname: Box{No: "05", Title: "Hostname and reverse DNS", ID: "hostname", Help: topicLink(HelpRDNS, "Forward and reverse DNS")},
|
||||
NotifyMail: Box{No: "06", Title: "Notification mail", ID: "notification-mail",
|
||||
Help: topicLink(HelpNotifications, "What the panel sends and what makes it arrive")},
|
||||
Configuration: Box{No: "07", Title: "Configuration", ID: "configuration",
|
||||
Machine: Box{No: "07", Title: "Machine", ID: "machine", Help: topicLink(HelpChecks, "What these readings mean")},
|
||||
Configuration: Box{No: "08", Title: "Configuration", ID: "configuration",
|
||||
End: Plain("Safe to run at any time")},
|
||||
Reload: Action{Label: "Reload configuration", Icon: "ti-reload", Post: "/server/health/reload"},
|
||||
}
|
||||
@@ -150,5 +167,39 @@ func (h *Health) WithNotificationMail(from string, rows []MailRow, status string
|
||||
return h
|
||||
}
|
||||
|
||||
// WithSenderCheck fills the Sender check card: its three periods, the first
|
||||
// day counted ("" before any was) and the card's verdict.
|
||||
func (h *Health) WithSenderCheck(rows []SenderCheckRow, since, status string) *Health {
|
||||
h.SenderCheck.End = Verdict(status)
|
||||
h.SenderRows = rows
|
||||
h.SenderNote = Rich("The panel holds each message's ", Code("From:"),
|
||||
" against the application that sends it. While the panel does not answer, Postfix defers mail instead of accepting it unchecked. Time is counted only while Postfix itself was running.")
|
||||
if since != "" {
|
||||
h.SenderNote = append(h.SenderNote, Rich(" Counted since ", Mono(since), ".")...)
|
||||
}
|
||||
return h
|
||||
}
|
||||
|
||||
// AsFragment makes the page its polled fragment: the left stack replaces the
|
||||
// element that polls, and the three readings of the right stack are marked to
|
||||
// be swapped into place by their ids.
|
||||
func (h *Health) AsFragment() *Health {
|
||||
h.Sockets.OOB, h.Certificate.OOB, h.NotifyMail.OOB = true, true, true
|
||||
return h
|
||||
}
|
||||
|
||||
// Downtime prints a length of time the way the Sender check table does: whole
|
||||
// seconds while they say something, then minutes, then hours.
|
||||
func Downtime(d time.Duration) string {
|
||||
secs := int64(d / time.Second)
|
||||
switch {
|
||||
case secs < 60:
|
||||
return strconv.FormatInt(secs, 10) + " s"
|
||||
case secs < 3600:
|
||||
return strconv.FormatInt(secs/60, 10) + " min " + strconv.FormatInt(secs%60, 10) + " s"
|
||||
}
|
||||
return strconv.FormatInt(secs/3600, 10) + " h " + strconv.FormatInt(secs%3600/60, 10) + " min"
|
||||
}
|
||||
|
||||
// Verdict puts a check's status in the end slot of its box, as the mockup does.
|
||||
func Verdict(status string) Text { return Text{TagOf(status, "")} }
|
||||
|
||||
@@ -41,12 +41,15 @@ const (
|
||||
HelpCertificate = "certificate"
|
||||
HelpRDNS = "rdns"
|
||||
// Outbound
|
||||
HelpDNS = "dns"
|
||||
HelpApps = "apps"
|
||||
HelpLimits = "limits"
|
||||
HelpLog = "log"
|
||||
HelpQueue = "queue"
|
||||
HelpDMARC = "dmarc"
|
||||
HelpDNS = "dns"
|
||||
HelpApps = "apps"
|
||||
// HelpSenderCheck is what an application may write in From: and what the
|
||||
// strict check costs; Health and a refused message's page link to it.
|
||||
HelpSenderCheck = "sender-check"
|
||||
HelpLimits = "limits"
|
||||
HelpLog = "log"
|
||||
HelpQueue = "queue"
|
||||
HelpDMARC = "dmarc"
|
||||
// Inbound
|
||||
HelpInbound = "inbound"
|
||||
HelpUpstream = "upstream"
|
||||
@@ -103,6 +106,7 @@ func NewHelp(m Meta, inbound bool) *Help {
|
||||
add("Outbound", outboundTopics, []MenuItem{
|
||||
{Label: "DNS records", Icon: "ti-world-check", Href: "#" + HelpDNS},
|
||||
{Label: "Applications", Icon: "ti-apps", Href: "#" + HelpApps},
|
||||
{Label: "Sender check", Icon: "ti-shield-check", Href: "#" + HelpSenderCheck},
|
||||
{Label: "Rate limits", Icon: "ti-gauge", Href: "#" + HelpLimits},
|
||||
{Label: "Outbound log", Icon: "ti-list-details", Href: "#" + HelpLog},
|
||||
{Label: "Queue and retries", Icon: "ti-stack-2", Href: "#" + HelpQueue},
|
||||
@@ -161,11 +165,19 @@ var outboundTopics = []HelpTopic{
|
||||
{ID: HelpApps, Title: "Applications", Body: []Text{
|
||||
Plain("An application is one SASL login, unique across all domains. It may send as any address of its domain or only as listed addresses, and, with a list of client IPs set, only from those addresses. The password is shown once at creation or regeneration and is not stored in readable form."),
|
||||
}},
|
||||
{ID: HelpSenderCheck, Title: "Sender check", Body: []Text{
|
||||
Rich("An application sends only as itself, and that holds for both senders a message has. The envelope sender is held against the login by Postfix. The ",
|
||||
Code("From:"), " line, the one a mail program shows and the one the DKIM signature is chosen by, is held against it by the panel, under the same rule: an application set to any address of its domain may write any address of that domain, and one with a list of addresses only those. A message with any other ",
|
||||
Code("From:"), " - another of your domains, the server's own name, an address somewhere else, or none at all - is refused for good with ",
|
||||
Code("550 5.7.1"), ". It is a rejected row of the Outbound log, and its page says which address was refused. This is what keeps one application from getting mail signed with another domain's key.",
|
||||
Br(), Br(),
|
||||
"The check is strict. While the panel does not answer, Postfix does not accept mail unchecked: it replies with a temporary error and the application sends again later. Server › Health counts what that costs: for how long the panel did not answer while Postfix itself was running, and how many messages were deferred because of it. The card warns when a message was deferred in the last 24 hours. If the numbers stay at nothing, the check is where it should be; if they do not, it belongs in a process of its own."),
|
||||
}},
|
||||
{ID: HelpLimits, Title: "Rate limits", Body: []Text{
|
||||
Rich("Level 1 is a per-IP ceiling set in ", Code(".env"), ". Level 2 is optional, per domain and per application, entered by hand or computed from the last 30 days of sending, and can never exceed level 1. An application's own limit replaces its domain's."),
|
||||
}},
|
||||
{ID: HelpLog, Title: "Outbound log", Body: []Text{
|
||||
Rich("One row per message and recipient, with the status Postfix last reported: queued (accepted, nothing reported yet), sent (the receiving server took it; what the mailbox does next is not reported back), deferred (not taken yet, Postfix tries again), bounced (failed for good, or the queue lifetime ran out) and rejected (refused before queueing under a level-2 rate limit, so it never reached Postfix). A message's page lists these steps and the lines Postfix wrote about it in ",
|
||||
Rich("One row per message and recipient, with the status Postfix last reported: queued (accepted, nothing reported yet), sent (the receiving server took it; what the mailbox does next is not reported back), deferred (not taken yet, Postfix tries again), bounced (failed for good, or the queue lifetime ran out) and rejected (refused before queueing, so it never reached the queue: under a level-2 rate limit, from a client address the application does not allow, or for a ", Code("From:"), " line that is not the application's - the message's page says which). A message's page lists these steps and the lines Postfix wrote about it in ",
|
||||
Code("mail.log"), ", found by queue id. The lines are kept with the row: they stay on the page after that file has rotated, for as long as the row does. A message accepted before version 2.2.0 shows them only while they are still in the current file. Rows are deleted after the retention set in Server › Settings, 90 days by default, and their lines with them. A domain administrator sees only the rows of their own domains."),
|
||||
}},
|
||||
{ID: HelpQueue, Title: "Queue and retries", Body: []Text{
|
||||
|
||||
@@ -80,6 +80,8 @@ func KitPage() *Kit {
|
||||
LayoutFacts: []Fact{
|
||||
{Label: "list", Value: Rich("page_head, then full-width boxes: the add form first, the table second. ", Link("/outbound/domains", "Example"))},
|
||||
{Label: "detail", Value: Rich("page_head, then ", Code("columns"), ": ", Code("is-3"), " side menu + ", Code("is-9"), " boxes.")},
|
||||
{Label: "cards", Value: Rich("page_head, then ", Code("columns sp-cards"), ": two ", Code("is-6"),
|
||||
" stacks of numbered boxes read in rows, odd on the left and even on the right, each stack closed up. On a phone the boxes come in the order of their numbers. The two stacks under this box are laid out so.")},
|
||||
{Label: "form", Value: Rich("page_head, then boxes in equal ", Code("columns"), ", one submit row under them.")},
|
||||
{Label: "signed out", Value: Rich("The split brand / form screen, no navbar. ", Link("/login", "Example"))},
|
||||
},
|
||||
|
||||
@@ -168,6 +168,10 @@ type MessageInput struct {
|
||||
To string
|
||||
Subject string
|
||||
Status string
|
||||
// Reason and HeaderFrom are a refused row's: why it was refused
|
||||
// (store.Reason*) and, for its From: line, what that line said.
|
||||
Reason string
|
||||
HeaderFrom string
|
||||
Accepted time.Time
|
||||
Reported time.Time
|
||||
// BackHref is the log the message was opened from, filters and page kept.
|
||||
@@ -193,6 +197,24 @@ type OutMessage struct {
|
||||
LogEmpty EmptyState
|
||||
}
|
||||
|
||||
// SendLogTag is the tag of a send-log row. A refused row says why in the tag,
|
||||
// so that the log answers it without the message being opened: the reasons
|
||||
// are store.Reason*, and a row from before they were recorded is "rejected".
|
||||
func SendLogTag(status, reason string) Tag {
|
||||
if status != "rejected" {
|
||||
return Tag{Status: status}
|
||||
}
|
||||
switch reason {
|
||||
case "from":
|
||||
return Tag{Status: "reject", Label: "wrong From"}
|
||||
case "client":
|
||||
return Tag{Status: "reject", Label: "wrong client"}
|
||||
case "limit":
|
||||
return Tag{Status: "rate-limited"}
|
||||
}
|
||||
return Tag{Status: status}
|
||||
}
|
||||
|
||||
// dash is the value of a field a row does not have.
|
||||
const dash = "—"
|
||||
|
||||
@@ -218,6 +240,28 @@ func NewOutMessage(m Meta, in MessageInput) *OutMessage {
|
||||
}
|
||||
return Rich(Link(href, text))
|
||||
}
|
||||
tag := SendLogTag(in.Status, in.Reason)
|
||||
facts := []Fact{
|
||||
{Label: "Domain", Value: value(in.Domain, in.DomainHref)},
|
||||
{Label: "Application", Value: value(in.App, in.AppHref), Mono: true},
|
||||
}
|
||||
history := Box{No: "02", Title: "History", Help: topicLink(HelpLog, "What the statuses mean")}
|
||||
if in.Status == "rejected" {
|
||||
// A refused message was never accepted and never queued: its one date
|
||||
// is the refusal, and it has no queue id to look anything up by.
|
||||
facts = append(facts, Fact{Label: "Refused", Value: Plain(formatStamp(in.Accepted))})
|
||||
if in.Reason == "from" {
|
||||
facts = append(facts, Fact{Label: "From: line", Value: Plain(orDash(in.HeaderFrom)), Mono: true, Small: true})
|
||||
history.Help = topicLink(HelpSenderCheck, "What an application may write in From:")
|
||||
}
|
||||
facts = append(facts, Fact{Label: "Queue id", Value: Plain("none - never queued")})
|
||||
} else {
|
||||
facts = append(facts,
|
||||
Fact{Label: "Accepted", Value: Plain(formatStamp(in.Accepted))},
|
||||
Fact{Label: "Status reported", Value: Plain(formatStamp(in.Reported))},
|
||||
Fact{Label: "Queue id", Value: Plain(orDash(in.QueueID)), Mono: true})
|
||||
}
|
||||
facts = append(facts, Fact{Label: "Journal id", Value: Plain(strconv.FormatInt(in.ID, 10)), Mono: true})
|
||||
return &OutMessage{
|
||||
Meta: m,
|
||||
Head: Head{
|
||||
@@ -226,19 +270,12 @@ func NewOutMessage(m Meta, in MessageInput) *OutMessage {
|
||||
Compact: true,
|
||||
Route: Route{
|
||||
{Text: orDash(in.From)}, {Arrow: true}, {Text: orDash(in.To)},
|
||||
{Tag: &Tag{Status: in.Status}},
|
||||
{Tag: &tag},
|
||||
},
|
||||
},
|
||||
Message: Box{No: "01", Title: "Message"},
|
||||
Facts: []Fact{
|
||||
{Label: "Domain", Value: value(in.Domain, in.DomainHref)},
|
||||
{Label: "Application", Value: value(in.App, in.AppHref), Mono: true},
|
||||
{Label: "Accepted", Value: Plain(formatStamp(in.Accepted))},
|
||||
{Label: "Status reported", Value: Plain(formatStamp(in.Reported))},
|
||||
{Label: "Queue id", Value: Plain(orDash(in.QueueID)), Mono: true},
|
||||
{Label: "Journal id", Value: Plain(strconv.FormatInt(in.ID, 10)), Mono: true},
|
||||
},
|
||||
History: Box{No: "02", Title: "History", Help: topicLink(HelpLog, "What the statuses mean")},
|
||||
Message: Box{No: "01", Title: "Message"},
|
||||
Facts: facts,
|
||||
History: history,
|
||||
Delivery: Box{No: "03", Title: "Delivery log", End: Rich("Lines from ", Code("mail.log"), " with this queue id")},
|
||||
LogEmpty: EmptyState{Icon: "ti-file-text"},
|
||||
}
|
||||
|
||||
@@ -22,7 +22,7 @@ func TestOutLogShowsTheRowsAndTheirFilters(t *testing.T) {
|
||||
`<th>Time, UTC</th>`, `09-21 14:01:52`,
|
||||
`shop@<wbr>shop.example.org`, `m.keller@<wbr>gmx.de`, `title="[FIRING] disk usage above 90 % on db-2"`,
|
||||
`<span class="tag is-success is-light">sent</span>`, `<span class="tag is-warning is-light">deferred</span>`,
|
||||
`<span class="tag is-danger is-light">bounced</span>`, `<span class="tag is-light">rejected</span>`,
|
||||
`<span class="tag is-danger is-light">bounced</span>`, `<span class="tag is-light">rate-limited</span>`, `<span class="tag is-danger is-light">wrong From</span>`,
|
||||
`href="/outbound/log/184220">Details</a>`,
|
||||
`Page 1 of 412`, `<a href="/outbound/log?p=2">Older →</a>`, `New rows appear on their own`,
|
||||
)
|
||||
@@ -114,7 +114,8 @@ func TestOutMessageShowsWhatTheJournalRecorded(t *testing.T) {
|
||||
}
|
||||
|
||||
// A message without a subject is named for it, and a row with nothing to show
|
||||
// for a field shows a dash.
|
||||
// for a field shows a dash. A refused row has no queue id and says so; one
|
||||
// from before the reason was recorded is plainly "rejected".
|
||||
func TestOutMessageWithoutSubjectOrRecipient(t *testing.T) {
|
||||
at := time.Date(2026, 9, 21, 13, 58, 7, 0, time.UTC)
|
||||
p := NewOutMessage(admin(), MessageInput{ID: 9, Domain: "gone.example", App: "old", Status: "rejected", Accepted: at, Reported: at,
|
||||
@@ -123,9 +124,40 @@ func TestOutMessageWithoutSubjectOrRecipient(t *testing.T) {
|
||||
out := renderSignedIn(t, "out-message", p)
|
||||
pageHas(t, "Message", out, `<h1 class="title is-4">(no subject)</h1>`, `<span>—</span><i class="ti ti-arrow-right sp-muted"></i><span>—</span>`,
|
||||
`<span class="tag is-light">rejected</span>`, `href="/outbound/log?domain=gone.example&p=2"`,
|
||||
`<dd>gone.example</dd>`, `<dd class="sp-mono">old</dd>`, `<dd class="sp-mono">—</dd>`,
|
||||
`<dd>gone.example</dd>`, `<dd class="sp-mono">old</dd>`, `<dt>Refused</dt>`, `<dd>none - never queued</dd>`,
|
||||
`never reached the queue`)
|
||||
pageLacks(t, "Message", out, `<pre`, `<a href="/outbound/domains/`)
|
||||
pageLacks(t, "Message", out, `<pre`, `<a href="/outbound/domains/`, `Status reported`, `From: line`)
|
||||
}
|
||||
|
||||
// A row refused for its From: line says so in its tag, shows what the line
|
||||
// said, and its History links to the topic that explains the rule.
|
||||
func TestOutMessageRefusedForItsFromLine(t *testing.T) {
|
||||
at := time.Date(2026, 9, 21, 13, 40, 55, 0, time.UTC)
|
||||
p := NewOutMessage(admin(), MessageInput{ID: 184231, Domain: "shop.example.org", App: "shop",
|
||||
From: "shop@shop.example.org", To: "k.berg@web.de", Subject: "A message from the director",
|
||||
Status: "rejected", Reason: "from", HeaderFrom: "director@example.org", Accepted: at, Reported: at}).
|
||||
WithDeliveryLog(nil, "This message never reached the queue, so Postfix wrote no delivery lines for it.")
|
||||
out := renderSignedIn(t, "out-message", p)
|
||||
pageHas(t, "Message", out, `<span class="tag is-danger is-light">wrong From</span>`,
|
||||
`<dt>From: line</dt><dd class="sp-mono sp-small">director@example.org</dd>`, `<dt>Refused</dt>`,
|
||||
`<dd>none - never queued</dd>`, `href="/help#sender-check"`)
|
||||
}
|
||||
|
||||
func TestSendLogTag(t *testing.T) {
|
||||
for _, c := range []struct {
|
||||
status, reason string
|
||||
want Tag
|
||||
}{
|
||||
{"sent", "", Tag{Status: "sent"}},
|
||||
{"rejected", "", Tag{Status: "rejected"}},
|
||||
{"rejected", "limit", Tag{Status: "rate-limited"}},
|
||||
{"rejected", "from", Tag{Status: "reject", Label: "wrong From"}},
|
||||
{"rejected", "client", Tag{Status: "reject", Label: "wrong client"}},
|
||||
} {
|
||||
if got := SendLogTag(c.status, c.reason); got != c.want {
|
||||
t.Errorf("SendLogTag(%q, %q) = %+v, want %+v", c.status, c.reason, got, c.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestOutMessageEscapesWhatSendersWrote(t *testing.T) {
|
||||
|
||||
@@ -4,6 +4,7 @@ import (
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"regexp"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
@@ -158,47 +159,55 @@ func TestHealthShowsEveryCheckAndBothActions(t *testing.T) {
|
||||
`hx-get="/server/health/fragment"`,
|
||||
)
|
||||
pageLacks(t, "Health", out, `hx-swap-oob`, `/overview/fragment`)
|
||||
if got := strings.Count(out, `class="columns"`); got != 3 {
|
||||
t.Errorf("Health has %d rows of boxes, want 3", got)
|
||||
if got := strings.Count(out, `class="columns sp-cards"`); got != 1 {
|
||||
t.Errorf("Health has %d sets of cards, want 1", got)
|
||||
}
|
||||
// The Configuration box is outside the polled rows: a refresh must not re-render a control.
|
||||
if strings.Index(out, `id="configuration"`) < strings.Index(out, `id="health-checks"`) ||
|
||||
strings.Contains(out[strings.Index(out, `id="health-machine"`):strings.Index(out, `id="configuration"`)], `action="/server/health/reload"`) {
|
||||
t.Error("the reload control is inside the polled rows")
|
||||
// The boxes are numbered by importance and read in rows: the odd ones are
|
||||
// the left stack, the even ones the right.
|
||||
order := []string{`id="processes"`, `id="sender-check"`, `id="hostname"`, `id="machine"`,
|
||||
`id="sockets"`, `id="certificate"`, `id="notification-mail"`, `id="configuration"`}
|
||||
for i := 1; i < len(order); i++ {
|
||||
if strings.Index(out, order[i-1]) < 0 || strings.Index(out, order[i]) < strings.Index(out, order[i-1]) {
|
||||
t.Errorf("%s does not come after %s", order[i], order[i-1])
|
||||
}
|
||||
}
|
||||
for no, title := range []string{"Processes", "Milter sockets", "Sender check", "TLS certificate",
|
||||
"Hostname and reverse DNS", "Notification mail", "Machine", "Configuration"} {
|
||||
if want := `<span class="sp-no">0` + strconv.Itoa(no+1) + `</span><h2>` + title + `</h2>`; !strings.Contains(out, want) {
|
||||
t.Errorf("Health is missing %s", want)
|
||||
}
|
||||
}
|
||||
pageHas(t, "Health", out, `<tr><td>Last 12 months</td><td>41 s</td><td>3</td></tr>`,
|
||||
`Counted since <span class="sp-mono">2026-10-13</span>.`, `href="/help#sender-check"`)
|
||||
// The Configuration box is outside the polled stack: a refresh must not re-render a control.
|
||||
if strings.Contains(out[strings.Index(out, `id="health-stack"`):strings.Index(out, `id="sockets"`)], `action="/server/health/reload"`) {
|
||||
t.Error("the reload control is inside the polled stack")
|
||||
}
|
||||
}
|
||||
|
||||
// The fragment is the polled part of the page and nothing else: the same rows,
|
||||
// the second one marked to be swapped in by its id, and the Notification mail
|
||||
// card, a column of the last row, swapped in the same way. No shell and no
|
||||
// actions.
|
||||
func TestHealthFragmentIsThePolledRows(t *testing.T) {
|
||||
// The fragment is the polled part of the page and nothing else: the left
|
||||
// stack as it is, and the three readings of the right stack, each marked to be
|
||||
// swapped in by its id. No shell and no actions.
|
||||
func TestHealthFragmentIsThePolledBoxes(t *testing.T) {
|
||||
page := renderSignedIn(t, "health", healthFixture(false))
|
||||
frag := renderFragment(t, "health_body", healthFixture(true))
|
||||
|
||||
const card = `<div class="column is-6" id="health-mail"`
|
||||
// The part of s before the card, and the card itself up to what follows it
|
||||
// on the page (the Configuration box).
|
||||
rows := func(s string) string {
|
||||
return s[strings.Index(s, `<div class="columns" id="health-machine"`):strings.Index(s, card)]
|
||||
}
|
||||
mail := func(s string) string {
|
||||
a := strings.Index(s, card)
|
||||
if b := strings.Index(s, `id="configuration"`); b > a {
|
||||
return s[a:b]
|
||||
// The polled boxes of s: from the left stack to the Configuration box, or
|
||||
// to the end of a fragment. On the page the right stack's column opens
|
||||
// between them; the fragment has no such line.
|
||||
polled := func(s string) string {
|
||||
s = s[strings.Index(s, `<div class="column is-6" id="health-stack"`):]
|
||||
if b := strings.Index(s, `<div class="box" id="configuration"`); b > 0 {
|
||||
s = s[:b]
|
||||
}
|
||||
return s[a:]
|
||||
s = strings.ReplaceAll(s, ` hx-swap-oob="true"`, "")
|
||||
return strings.Join(strings.Fields(strings.Replace(s, `<div class="column is-6">`, "", 1)), " ")
|
||||
}
|
||||
// Same markup, apart from the out-of-band marks and what follows the card on the page.
|
||||
oob := func(s string) string { return strings.ReplaceAll(strings.TrimSpace(s), ` hx-swap-oob="true"`, "") }
|
||||
if got, want := oob(rows(frag)), oob(rows(page)); !strings.HasPrefix(want, got) {
|
||||
t.Errorf("the fragment is not the page's polled rows\nfragment: %.400s\npage: %.400s", got, want)
|
||||
if got, want := polled(frag), polled(page); got != want {
|
||||
t.Errorf("the fragment is not the page's polled boxes\nfragment: %.600s\npage: %.600s", got, want)
|
||||
}
|
||||
if got, want := oob(mail(frag)), oob(mail(page)); !strings.HasPrefix(want, got) {
|
||||
t.Errorf("the fragment's Notification mail card is not the page's\nfragment: %.400s\npage: %.400s", got, want)
|
||||
}
|
||||
pageHas(t, "fragment", frag, `id="health-machine" data-poll`, `id="health-checks" hx-swap-oob="true"`,
|
||||
`id="health-mail" hx-swap-oob="true"`, `id="notification-mail"`)
|
||||
pageHas(t, "fragment", frag, `id="health-stack" data-poll`, `id="sockets" hx-swap-oob="true"`,
|
||||
`id="certificate" hx-swap-oob="true"`, `id="notification-mail" hx-swap-oob="true"`, `id="sender-check"`)
|
||||
pageLacks(t, "fragment", frag, `<h1`, `navbar`, `action="/server/health/reload"`, `action="/server/health/recheck"`)
|
||||
}
|
||||
|
||||
@@ -257,7 +266,7 @@ func TestHealthStatesTheFixtureDoesNotDraw(t *testing.T) {
|
||||
`<div class="notification is-success is-light">`, `DNS re-checked.`,
|
||||
)
|
||||
// the flash sits between the head and the first row, as the kit puts it
|
||||
if strings.Index(out, `class="notification`) > strings.Index(out, `id="health-machine"`) ||
|
||||
if strings.Index(out, `class="notification`) > strings.Index(out, `id="health-stack"`) ||
|
||||
strings.Index(out, `class="notification`) < strings.Index(out, `class="sp-head"`) {
|
||||
t.Error("the flash is not between page_head and the first box")
|
||||
}
|
||||
|
||||
@@ -326,8 +326,24 @@ code { background: var(--sp-well); color: var(--sp-ink); border-radius: 3px; pad
|
||||
.sp-record-head { flex-wrap: wrap; }
|
||||
.sp-record-head small { margin-left: 0; text-align: left; flex-basis: 100%; }
|
||||
.sp-postmark { display: none; }
|
||||
.sp-cards { display: flex; flex-direction: column; padding-inline: .75rem; }
|
||||
.sp-cards > .column { display: contents; }
|
||||
.sp-cards .box { margin-bottom: 1.5rem; }
|
||||
.sp-cards > .column:first-child > .box:nth-child(1) { order: 1; }
|
||||
.sp-cards > .column:first-child > .box:nth-child(2) { order: 3; }
|
||||
.sp-cards > .column:first-child > .box:nth-child(3) { order: 5; }
|
||||
.sp-cards > .column:first-child > .box:nth-child(4) { order: 7; }
|
||||
.sp-cards > .column:last-child > .box:nth-child(1) { order: 2; }
|
||||
.sp-cards > .column:last-child > .box:nth-child(2) { order: 4; }
|
||||
.sp-cards > .column:last-child > .box:nth-child(3) { order: 6; }
|
||||
.sp-cards > .column:last-child > .box:nth-child(4) { order: 8; }
|
||||
}
|
||||
|
||||
/* cards: two stacks of boxes that are read in rows - the odd ones on the left,
|
||||
the even ones on the right, each stack closed up so a short box leaves no
|
||||
gap beside a tall one. On a phone the stacks dissolve and the boxes come in
|
||||
the order of their numbers (the rules above). */
|
||||
|
||||
/* sheet: what does not fit a table row, opened over the page from the row's
|
||||
tag. It is the browser's own popover (the tag is a button with
|
||||
popovertarget), so it needs no script: Esc, a click outside and the close
|
||||
|
||||
@@ -84,11 +84,19 @@
|
||||
sp-credential (a secret shown once). A box foot, a table in its
|
||||
table-container, a run of records, a log or an empty state sit directly
|
||||
inside; anything else goes in body_open … body_close. */}}
|
||||
{{define "box_open"}}<div class="box{{if eq .Variant "danger"}} sp-danger-zone{{else if eq .Variant "credential"}} sp-credential{{end}}"{{if .ID}} id="{{.ID}}"{{end}}{{if .Poll}} data-poll aria-live="polite" hx-get="{{.Poll}}" hx-trigger="load" hx-swap="outerHTML"{{end}}{{if .DomainPick}} data-domain-pick{{end}}>
|
||||
{{define "box_open"}}<div class="box{{if eq .Variant "danger"}} sp-danger-zone{{else if eq .Variant "credential"}} sp-credential{{end}}"{{if .ID}} id="{{.ID}}"{{end}}{{if .OOB}} hx-swap-oob="true"{{end}}{{if .Poll}} data-poll aria-live="polite" hx-get="{{.Poll}}" hx-trigger="load" hx-swap="outerHTML"{{end}}{{if .DomainPick}} data-domain-pick{{end}}>
|
||||
{{template "box_head" .}}
|
||||
{{end}}
|
||||
{{define "box_close"}}</div>{{end}}
|
||||
|
||||
{{/* cards_open starts the `cards` layout and cards_close ends it: two
|
||||
`column is-6` stacks of numbered boxes that are read in rows, the odd ones
|
||||
on the left and the even ones on the right, each stack closed up so that a
|
||||
short box leaves no gap beside a tall one. On a phone the stacks dissolve
|
||||
and the boxes come in the order of their numbers (panel.css). */}}
|
||||
{{define "cards_open"}}<div class="columns sp-cards">{{end}}
|
||||
{{define "cards_close"}}</div>{{end}}
|
||||
|
||||
{{/* body_open pads the content of a box (box_body); body_close ends it. */}}
|
||||
{{define "body_open"}}<div class="sp-box-body">{{end}}
|
||||
{{define "body_close"}}</div>{{end}}
|
||||
|
||||
@@ -1,17 +1,18 @@
|
||||
{{/* Server > Health (mockup health.html): every check behind the cards of
|
||||
Overview, and the one action that is not about DNS, Reload configuration. The
|
||||
two rows of tables are the polled part (health_body.html), and so is the
|
||||
Notification mail card (health_mail), which sits in the last row beside the
|
||||
static Configuration box. */}}
|
||||
Overview, and the one action that is not about DNS, Reload configuration.
|
||||
The page is the `cards` layout: two stacks of boxes numbered by importance,
|
||||
the odd ones on the left and the even ones on the right. Everything but
|
||||
the Configuration box is polled (health_body.html). */}}
|
||||
{{define "content"}}
|
||||
{{template "page_head" .Head}}
|
||||
{{if .Flash}}{{template "flash" .Flash}}{{end}}
|
||||
|
||||
{{template "health_body" .}}
|
||||
|
||||
<div class="columns">
|
||||
{{template "health_mail" .}}
|
||||
{{template "cards_open"}}
|
||||
{{template "health_stack" .}}
|
||||
<div class="column is-6">
|
||||
{{template "health_sockets" .}}
|
||||
{{template "health_certificate" .}}
|
||||
{{template "health_mail" .}}
|
||||
{{template "box_open" .Configuration}}
|
||||
{{template "body_open"}}
|
||||
<p class="mb-3">Regenerates the OpenDKIM and Postfix configuration from the database and reloads both daemons. Use it after editing the files by hand, after restoring a backup, or when the running configuration looks out of step with the domain and application lists. It does not touch the queue or the TLS certificate.</p>
|
||||
@@ -19,5 +20,5 @@
|
||||
{{template "body_close"}}
|
||||
{{template "box_close"}}
|
||||
</div>
|
||||
</div>
|
||||
{{template "cards_close"}}
|
||||
{{end}}
|
||||
|
||||
@@ -1,80 +1,83 @@
|
||||
{{/* The polled part of Health: two rows of boxes, in the page and in its
|
||||
fragment (GET /server/health/fragment) alike. panel.js polls the first row
|
||||
(data-poll, outerHTML) and htmx swaps the second into place by its id when
|
||||
the fragment marks it out of band (.Refresh) — the two rows are siblings, and
|
||||
a fragment can replace only the element that asked for it. The Notification
|
||||
mail card is the third thing the fragment swaps in by id: it sits in a row of
|
||||
the page with the static Configuration box, so it is a column of its own. */}}
|
||||
{{define "health_body"}}<div class="columns" id="health-machine" data-poll aria-live="polite" hx-get="/server/health/fragment" hx-trigger="load" hx-swap="outerHTML">
|
||||
<div class="column is-6">
|
||||
{{template "box_open" .Machine}}
|
||||
<div class="table-container"><table class="table is-fullwidth">
|
||||
<thead><tr><th>Resource</th><th>Usage</th><th>Detail</th></tr></thead>
|
||||
<tbody>{{range .MachineRows}}
|
||||
<tr><td>{{.Resource}}</td><td>{{if .Gauge}}<progress class="progress is-small {{.Gauge.Bar}} mb-1" value="{{.Gauge.Percent}}" max="100"></progress>{{.Gauge.Text}}{{else if .Usage}}<span class="sp-nowrap">{{.Usage}}</span>{{else}}<span class="sp-muted">—</span>{{end}}</td><td class="sp-muted">{{range .Detail}}<div>{{.}}</div>{{end}}</td></tr>{{end}}
|
||||
</tbody>
|
||||
</table></div>
|
||||
{{template "box_close"}}
|
||||
</div>
|
||||
<div class="column is-6">
|
||||
{{template "box_open" .Processes}}
|
||||
{{if .ProcessError}}
|
||||
{{template "body_open"}}
|
||||
<p class="has-text-danger">Could not ask supervisord for the process list.</p>
|
||||
{{template "body_close"}}
|
||||
{{else}}
|
||||
<div class="table-container"><table class="table is-fullwidth">
|
||||
<thead><tr><th>Program</th><th>State</th><th>Detail</th></tr></thead>
|
||||
<tbody>{{range .ProcessRows}}
|
||||
<tr><td>{{.Name}}</td><td>{{template "tag" .State}}</td><td class="sp-muted">{{.Detail}}</td></tr>{{end}}
|
||||
</tbody>
|
||||
</table></div>
|
||||
{{end}}
|
||||
{{template "box_close"}}
|
||||
</div>
|
||||
</div>
|
||||
{{/* The polled part of Health, in the page and in its fragment
|
||||
(GET /server/health/fragment) alike. The left stack is the element that
|
||||
polls (data-poll, outerHTML): all four of its boxes are readings. The right
|
||||
stack ends in the static Configuration box, so its three readings are
|
||||
swapped into place one by one, each box by its own id, when the fragment
|
||||
marks them out of band (Health.AsFragment) - a fragment can replace only
|
||||
the element that asked for it. */}}
|
||||
{{define "health_body"}}{{template "health_stack" .}}
|
||||
{{template "health_sockets" .}}
|
||||
{{template "health_certificate" .}}
|
||||
{{template "health_mail" .}}{{end}}
|
||||
|
||||
<div class="columns" id="health-checks"{{if .Refresh}} hx-swap-oob="true"{{end}}>
|
||||
<div class="column is-4">
|
||||
{{template "box_open" .Certificate}}
|
||||
{{template "body_open"}}
|
||||
{{if .CertFacts}}{{template "facts" .CertFacts}}{{end}}
|
||||
<p class="mt-4 {{if .CertProblem}}has-text-danger{{else}}sp-muted{{end}}">{{.CertDetail}}</p>
|
||||
{{template "body_close"}}
|
||||
{{template "box_close"}}
|
||||
</div>
|
||||
<div class="column is-4">
|
||||
{{template "box_open" .Sockets}}
|
||||
<div class="table-container"><table class="table is-fullwidth">
|
||||
<thead><tr><th>Milter</th><th>State</th></tr></thead>
|
||||
<tbody>{{range .SocketRows}}
|
||||
<tr><td>{{.Name}}{{if .Note}} <span class="sp-muted">· {{.Note}}</span>{{end}}<br><span class="sp-muted sp-mono sp-small">{{.Path}}</span>{{if .Detail}}<br><span class="sp-muted sp-small">{{.Detail}}</span>{{end}}</td><td>{{template "tag" .State}}</td></tr>{{end}}
|
||||
</tbody>
|
||||
</table></div>
|
||||
{{template "box_close"}}
|
||||
</div>
|
||||
<div class="column is-4">
|
||||
{{template "box_open" .Hostname}}
|
||||
{{template "body_open"}}
|
||||
{{template "facts" .HostFacts}}
|
||||
{{template "body_close"}}
|
||||
{{template "box_close"}}
|
||||
</div>
|
||||
</div>{{if .Refresh}}
|
||||
{{template "health_mail" .}}{{end}}{{end}}
|
||||
|
||||
{{/* health_mail is the Notification mail card (the page calls it inside its own
|
||||
row; the fragment sends it out of band). */}}
|
||||
{{define "health_mail"}}<div class="column is-6" id="health-mail"{{if .Refresh}} hx-swap-oob="true"{{end}}>
|
||||
{{template "box_open" .NotifyMail}}
|
||||
{{/* health_stack is the left stack: Processes, Sender check, Hostname and
|
||||
reverse DNS, Machine. */}}
|
||||
{{define "health_stack"}}<div class="column is-6" id="health-stack" data-poll aria-live="polite" hx-get="/server/health/fragment" hx-trigger="load" hx-swap="outerHTML">
|
||||
{{template "box_open" .Processes}}
|
||||
{{if .ProcessError}}
|
||||
{{template "body_open"}}
|
||||
<p class="has-text-danger">Could not ask supervisord for the process list.</p>
|
||||
{{template "body_close"}}
|
||||
{{else}}
|
||||
<div class="table-container"><table class="table is-fullwidth">
|
||||
<thead><tr><th>Record</th><th>State</th></tr></thead>
|
||||
<tbody>{{range .NotifyRows}}
|
||||
<tr><td>{{.Name}}<br><span class="sp-muted sp-mono sp-small">{{.Host}}</span></td><td>{{template "tag" .State}}</td></tr>{{end}}
|
||||
<thead><tr><th>Program</th><th>State</th><th>Detail</th></tr></thead>
|
||||
<tbody>{{range .ProcessRows}}
|
||||
<tr><td>{{.Name}}</td><td>{{template "tag" .State}}</td><td class="sp-muted">{{.Detail}}</td></tr>{{end}}
|
||||
</tbody>
|
||||
</table></div>
|
||||
{{end}}
|
||||
{{template "box_close"}}
|
||||
{{template "box_open" .SenderCheck}}
|
||||
<div class="table-container"><table class="table is-fullwidth">
|
||||
<thead><tr><th>Period</th><th>Panel not answering</th><th>Messages deferred</th></tr></thead>
|
||||
<tbody>{{range .SenderRows}}
|
||||
<tr><td>{{.Period}}</td><td>{{.Down}}</td><td>{{.Deferred}}</td></tr>{{end}}
|
||||
</tbody>
|
||||
</table></div>
|
||||
{{template "body_open"}}
|
||||
<p class="sp-muted">{{template "inline" .NotifyNote}}</p>
|
||||
<p class="sp-muted">{{template "inline" .SenderNote}}</p>
|
||||
{{template "body_close"}}
|
||||
{{template "box_close"}}
|
||||
{{template "box_open" .Hostname}}
|
||||
{{template "body_open"}}
|
||||
{{template "facts" .HostFacts}}
|
||||
{{template "body_close"}}
|
||||
{{template "box_close"}}
|
||||
{{template "box_open" .Machine}}
|
||||
<div class="table-container"><table class="table is-fullwidth">
|
||||
<thead><tr><th>Resource</th><th>Usage</th><th>Detail</th></tr></thead>
|
||||
<tbody>{{range .MachineRows}}
|
||||
<tr><td>{{.Resource}}</td><td>{{if .Gauge}}<progress class="progress is-small {{.Gauge.Bar}} mb-1" value="{{.Gauge.Percent}}" max="100"></progress>{{.Gauge.Text}}{{else if .Usage}}<span class="sp-nowrap">{{.Usage}}</span>{{else}}<span class="sp-muted">—</span>{{end}}</td><td class="sp-muted">{{range .Detail}}<div>{{.}}</div>{{end}}</td></tr>{{end}}
|
||||
</tbody>
|
||||
</table></div>
|
||||
{{template "box_close"}}
|
||||
</div>{{end}}
|
||||
|
||||
{{define "health_sockets"}}{{template "box_open" .Sockets}}
|
||||
<div class="table-container"><table class="table is-fullwidth">
|
||||
<thead><tr><th>Milter</th><th>State</th></tr></thead>
|
||||
<tbody>{{range .SocketRows}}
|
||||
<tr><td>{{.Name}}{{if .Note}} <span class="sp-muted">· {{.Note}}</span>{{end}}<br><span class="sp-muted sp-mono sp-small">{{.Path}}</span>{{if .Detail}}<br><span class="sp-muted sp-small">{{.Detail}}</span>{{end}}</td><td>{{template "tag" .State}}</td></tr>{{end}}
|
||||
</tbody>
|
||||
</table></div>
|
||||
{{template "box_close"}}{{end}}
|
||||
|
||||
{{define "health_certificate"}}{{template "box_open" .Certificate}}
|
||||
{{template "body_open"}}
|
||||
{{if .CertFacts}}{{template "facts" .CertFacts}}{{end}}
|
||||
<p class="mt-4 {{if .CertProblem}}has-text-danger{{else}}sp-muted{{end}}">{{.CertDetail}}</p>
|
||||
{{template "body_close"}}
|
||||
{{template "box_close"}}{{end}}
|
||||
|
||||
{{/* health_mail is the Notification mail card. */}}
|
||||
{{define "health_mail"}}{{template "box_open" .NotifyMail}}
|
||||
<div class="table-container"><table class="table is-fullwidth">
|
||||
<thead><tr><th>Record</th><th>State</th></tr></thead>
|
||||
<tbody>{{range .NotifyRows}}
|
||||
<tr><td>{{.Name}}<br><span class="sp-muted sp-mono sp-small">{{.Host}}</span></td><td>{{template "tag" .State}}</td></tr>{{end}}
|
||||
</tbody>
|
||||
</table></div>
|
||||
{{template "body_open"}}
|
||||
<p class="sp-muted">{{template "inline" .NotifyNote}}</p>
|
||||
{{template "body_close"}}
|
||||
{{template "box_close"}}{{end}}
|
||||
|
||||
@@ -15,7 +15,7 @@
|
||||
{{template "body_close"}}
|
||||
{{template "box_close"}}
|
||||
|
||||
<div class="columns">
|
||||
{{template "cards_open"}}
|
||||
<div class="column is-6">
|
||||
{{template "box_open" .Postmark}}
|
||||
{{template "body_open"}}
|
||||
@@ -58,7 +58,7 @@
|
||||
{{template "body_close"}}
|
||||
{{template "box_close"}}
|
||||
</div>
|
||||
</div>
|
||||
{{template "cards_close"}}
|
||||
|
||||
{{template "box_open" .HealthCard}}
|
||||
{{template "health_cards" .HealthCards}}
|
||||
|
||||
@@ -67,6 +67,8 @@ type Config struct {
|
||||
// only for the send log (the journal-milter fails open).
|
||||
OpenDKIMSocket string
|
||||
JournalSocket string
|
||||
// SenderSocket is the sender check's socket, attached strictly.
|
||||
SenderSocket string
|
||||
// SessionIdleDays is the sliding inactivity window after which a login
|
||||
// session expires (env PANEL_SESSION_IDLE_DAYS, plan B.1). Non-positive
|
||||
// falls back to the 7-day default.
|
||||
@@ -138,6 +140,7 @@ func New(st *store.Store, domains *domain.Service, apps *app.Service, inboundSvc
|
||||
TLSCertFile: cfg.TLSCertFile,
|
||||
OpenDKIMSocket: cfg.OpenDKIMSocket,
|
||||
JournalSocket: cfg.JournalSocket,
|
||||
SenderSocket: cfg.SenderSocket,
|
||||
RateLimitMessagesPerIP: cfg.RateLimitMessagesPerIP,
|
||||
RateLimitWindowSeconds: cfg.RateLimitWindowSeconds,
|
||||
RetryPolicy: cfg.RetryPolicy,
|
||||
|
||||
@@ -281,8 +281,11 @@ func TestE2E(t *testing.T) {
|
||||
run("negative_foreign_relay_rejected", func(t *testing.T) {
|
||||
testForeignRelayRejected(t, sc)
|
||||
})
|
||||
run("negative_journal_milter_fail_open", func(t *testing.T) {
|
||||
testJournalMilterFailOpen(t, sc)
|
||||
run("sender_check_holds_the_from_line", func(t *testing.T) {
|
||||
testSenderCheck(t, sc)
|
||||
})
|
||||
run("panel_down_defers_mail", func(t *testing.T) {
|
||||
testPanelDownDefersMail(t, sc)
|
||||
})
|
||||
run("session_survives_restart", func(t *testing.T) {
|
||||
testSessionSurvivesRestart(t, sc)
|
||||
|
||||
@@ -63,15 +63,15 @@ func testLevel2RateLimit(t *testing.T, sc *scenario) {
|
||||
t.Fatalf("over the level-2 limit, two MAIL FROM on one connection were answered %v and %v, want both refused", errs[0], errs[1])
|
||||
}
|
||||
|
||||
if err := waitFor("a rejected row for l2app in the send log", 15*time.Second, 500*time.Millisecond, func() (bool, error) {
|
||||
if err := waitFor("a rate-limited row for l2app in the send log", 15*time.Second, 500*time.Millisecond, func() (bool, error) {
|
||||
rows, err := sc.panel.sendLogRows(senderDomain, login)
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
if containsCell(rows, "rejected") {
|
||||
if containsCell(rows, "rate-limited") {
|
||||
return true, nil
|
||||
}
|
||||
return false, fmt.Errorf("no rejected row for l2app yet")
|
||||
return false, fmt.Errorf("no rate-limited row for l2app yet")
|
||||
}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -127,46 +127,6 @@ func testForeignRelayRejected(t *testing.T, sc *scenario) {
|
||||
}
|
||||
}
|
||||
|
||||
// testJournalMilterFailOpen is plan C.4 negative check 6: the journal-milter
|
||||
// is monitoring-only and fails open (architecture.md § Mail path) — stopping
|
||||
// the panel process (which owns the milter socket) must not block mail, and
|
||||
// must not crash the container (crashexit only fires on PROCESS_STATE_FATAL, a
|
||||
// clean supervisor stop is STOPPED, see build/crashexit.py).
|
||||
func testJournalMilterFailOpen(t *testing.T, sc *scenario) {
|
||||
if _, err := h.execIn("selfpost", "supervisorctl", "-c", "/etc/supervisor/supervisord.conf", "stop", "panel"); err != nil {
|
||||
t.Fatalf("stop panel: %v", err)
|
||||
}
|
||||
t.Cleanup(func() {
|
||||
_, _ = h.execIn("selfpost", "supervisorctl", "-c", "/etc/supervisor/supervisord.conf", "start", "panel")
|
||||
})
|
||||
|
||||
res := attemptSend(sendAttempt{
|
||||
authLogin: sc.appLogin, authPassword: sc.appPassword,
|
||||
from: "alerts@" + senderDomain, to: recipient,
|
||||
subject: "e2e " + uniqueToken("failopen"), body: "must still be accepted",
|
||||
})
|
||||
if !res.ok() {
|
||||
t.Fatalf("send with the journal-milter down was rejected, want fail-open accept: dial=%v auth=%v mail=%v rcpt=%v data=%v",
|
||||
res.dialErr, res.authErr, res.mailErr, res.rcptErr, res.dataErr)
|
||||
}
|
||||
|
||||
if err := checkContainerAlive(h); err != nil {
|
||||
t.Fatalf("container did not survive the panel stopping: %v", err)
|
||||
}
|
||||
|
||||
// Restart the panel (also undone by t.Cleanup, redundantly and harmlessly,
|
||||
// in case a later step needs it sooner than cleanup order guarantees).
|
||||
if _, err := h.execIn("selfpost", "supervisorctl", "-c", "/etc/supervisor/supervisord.conf", "start", "panel"); err != nil {
|
||||
t.Fatalf("restart panel: %v", err)
|
||||
}
|
||||
if err := waitFor("panel HTTP to answer again", 15*time.Second, 300*time.Millisecond, func() (bool, error) {
|
||||
_, err := sc.panel.status()
|
||||
return err == nil, err
|
||||
}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
// checkContainerAlive confirms the container is still up after a component
|
||||
// stop — the crashexit listener must not have brought it down.
|
||||
func checkContainerAlive(s *stack) error {
|
||||
|
||||
@@ -0,0 +1,182 @@
|
||||
package e2e
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"regexp"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// otherDomain is a second sending domain of the stand: registered, with a DKIM
|
||||
// key of its own, and no application of the first domain has any right to it.
|
||||
const otherDomain = "other.e2e.test"
|
||||
|
||||
// testSenderCheck is from-header-binding's "done when": an application is
|
||||
// refused for another tenant's From:, for the hostname's and for an address
|
||||
// off its list, and accepted for its own. The envelope sender is the
|
||||
// application's own every time - Postfix would refuse anything else before
|
||||
// the header was looked at.
|
||||
func testSenderCheck(t *testing.T, sc *scenario) {
|
||||
if _, err := sc.panel.addDomain(otherDomain); err != nil {
|
||||
t.Fatalf("add the second domain: %v", err)
|
||||
}
|
||||
listLogin, listPassword, err := sc.panel.addApplication(sc.domainID, "app-list", "list", "billing@"+senderDomain)
|
||||
if err != nil {
|
||||
t.Fatalf("add the application with a list: %v", err)
|
||||
}
|
||||
|
||||
type who struct{ login, password, envelope string }
|
||||
wildcard := who{sc.appLogin, sc.appPassword, "alerts@" + senderDomain}
|
||||
list := who{listLogin, listPassword, "billing@" + senderDomain}
|
||||
|
||||
for _, c := range []struct {
|
||||
name string
|
||||
as who
|
||||
header string
|
||||
pass bool
|
||||
}{
|
||||
{"own address", wildcard, "alerts@" + senderDomain, true},
|
||||
{"another address of the domain, with a name", wildcard, `"The Shop" <orders@` + senderDomain + `>`, true},
|
||||
{"the listed address", list, "billing@" + senderDomain, true},
|
||||
|
||||
{"another registered domain", wildcard, "director@" + otherDomain, false},
|
||||
{"the server's own name", wildcard, "postmaster@" + selfpostHostname, false},
|
||||
{"a domain the server knows nothing of", wildcard, "someone@elsewhere.e2e.test", false},
|
||||
{"an address off the list", list, "director@" + senderDomain, false},
|
||||
{"another sender in the display name", wildcard, `"director@` + otherDomain + `" <alerts@` + senderDomain + `>`, false},
|
||||
} {
|
||||
token := uniqueToken("from")
|
||||
res := attemptSend(sendAttempt{
|
||||
authLogin: c.as.login, authPassword: c.as.password,
|
||||
from: c.as.envelope, headerFrom: c.header, to: recipient,
|
||||
subject: "e2e " + token, body: "sender check: " + c.name,
|
||||
})
|
||||
if c.pass {
|
||||
if !res.ok() {
|
||||
t.Errorf("%s: From: %s was refused: %v", c.name, c.header, res.firstErr())
|
||||
continue
|
||||
}
|
||||
if _, err := findSinkMessage(h.stageDir, token, 30*time.Second); err != nil {
|
||||
t.Errorf("%s: accepted, and never reached the sink: %v", c.name, err)
|
||||
}
|
||||
continue
|
||||
}
|
||||
if res.ok() {
|
||||
t.Errorf("%s: From: %s was accepted", c.name, c.header)
|
||||
continue
|
||||
}
|
||||
// The verdict comes at the end of the headers, which is the end of
|
||||
// DATA for the client; it is permanent and says what is wrong.
|
||||
if e := fmt.Sprint(res.dataErr); res.dataErr == nil || !strings.HasPrefix(e, "550 ") || !strings.Contains(e, "5.7.1 From:") {
|
||||
t.Errorf("%s: want 550 5.7.1 naming the From: line at the end of DATA, got %v", c.name, res.firstErr())
|
||||
}
|
||||
if _, err := findSinkMessage(h.stageDir, token, 2*time.Second); err == nil {
|
||||
t.Errorf("%s: the refused message reached the sink", c.name)
|
||||
}
|
||||
}
|
||||
|
||||
// A refusal is a row of the Outbound log whose tag says why.
|
||||
if err := waitFor("the refusals to appear in the Outbound log", 15*time.Second, 500*time.Millisecond, func() (bool, error) {
|
||||
rows, err := sc.panel.sendLogRows(senderDomain, "")
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
if n := strings.Count(rows, `">wrong From</span>`); n < 5 {
|
||||
return false, fmt.Errorf("%d rows tagged wrong From, want 5", n)
|
||||
}
|
||||
return true, nil
|
||||
}); err != nil {
|
||||
t.Error(err)
|
||||
}
|
||||
}
|
||||
|
||||
// testPanelDownDefersMail: the sender check is the panel's and Postfix
|
||||
// attaches it strictly, so with the panel stopped mail is deferred - a
|
||||
// temporary error, nothing accepted unchecked - and the container stays up
|
||||
// (crashexit fires on PROCESS_STATE_FATAL only; a supervisor stop is STOPPED).
|
||||
// Once the panel is back it has counted both what that cost: the time it did
|
||||
// not answer under a running Postfix, and the message that was deferred.
|
||||
//
|
||||
// Until 2.4.0 this step proved the opposite, that mail passes with the panel
|
||||
// down: the journal milter still fails open, but it shares the process with
|
||||
// the check, and the check does not.
|
||||
func testPanelDownDefersMail(t *testing.T, sc *scenario) {
|
||||
supervisorctl := func(verb string) error {
|
||||
_, err := h.execIn("selfpost", "supervisorctl", "-c", "/etc/supervisor/supervisord.conf", verb, "panel")
|
||||
return err
|
||||
}
|
||||
if err := supervisorctl("stop"); err != nil {
|
||||
t.Fatalf("stop panel: %v", err)
|
||||
}
|
||||
t.Cleanup(func() { _ = supervisorctl("start") })
|
||||
|
||||
res := attemptSend(sendAttempt{
|
||||
authLogin: sc.appLogin, authPassword: sc.appPassword,
|
||||
from: "alerts@" + senderDomain, to: recipient,
|
||||
subject: "e2e " + uniqueToken("paneldown"), body: "must be deferred",
|
||||
})
|
||||
if res.ok() {
|
||||
t.Fatal("mail was accepted with the panel down: the sender check did not hold it")
|
||||
}
|
||||
// Temporary, and before any message is offered: Postfix answers 451 from
|
||||
// the connection on, and "454 4.3.0 Try again later" at AUTH, which is
|
||||
// where a client that authenticates gives up.
|
||||
if err := res.firstErr(); err == nil || !strings.HasPrefix(fmt.Sprint(err), "4") {
|
||||
t.Fatalf("want a temporary 4xx with the panel down, got %v", res.firstErr())
|
||||
}
|
||||
if err := checkContainerAlive(h); err != nil {
|
||||
t.Fatalf("container did not survive the panel stopping: %v", err)
|
||||
}
|
||||
// What Postfix wrote about it is what the panel counts from; shown so that
|
||||
// a change of Postfix's wording is seen here and not guessed at.
|
||||
if out, err := h.execIn("selfpost", "sh", "-c", "grep -i 'milter' /data/log/mail.log | tail -n 6"); err == nil {
|
||||
t.Logf("mail.log on the deferred message:\n%s", out)
|
||||
}
|
||||
time.Sleep(2 * time.Second) // downtime worth a whole second or two
|
||||
|
||||
if err := supervisorctl("start"); err != nil {
|
||||
t.Fatalf("restart panel: %v", err)
|
||||
}
|
||||
if err := waitFor("panel HTTP to answer again", 15*time.Second, 300*time.Millisecond, func() (bool, error) {
|
||||
_, err := sc.panel.status()
|
||||
return err == nil, err
|
||||
}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
// Mail flows again.
|
||||
token := uniqueToken("panelback")
|
||||
res = attemptSend(sendAttempt{
|
||||
authLogin: sc.appLogin, authPassword: sc.appPassword,
|
||||
from: "alerts@" + senderDomain, to: recipient,
|
||||
subject: "e2e " + token, body: "must be accepted again",
|
||||
})
|
||||
if !res.ok() {
|
||||
t.Fatalf("mail is still refused with the panel back: %v", res.firstErr())
|
||||
}
|
||||
|
||||
// Server > Health, "Sender check", today: some downtime, one message.
|
||||
today := regexp.MustCompile(`<tr><td>Today</td><td>([^<]*)</td><td>(\d+)</td></tr>`)
|
||||
if err := waitFor("Health to show the downtime and the deferred message", 20*time.Second, 500*time.Millisecond, func() (bool, error) {
|
||||
_, body, err := sc.panel.get("/server/health")
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
m := today.FindStringSubmatch(body)
|
||||
if m == nil {
|
||||
return false, fmt.Errorf("no Sender check row for today on Health")
|
||||
}
|
||||
deferred, _ := strconv.Atoi(m[2])
|
||||
if m[1] == "0 s" || deferred < 1 {
|
||||
return false, fmt.Errorf("today: panel not answering %q, messages deferred %d", m[1], deferred)
|
||||
}
|
||||
if !strings.Contains(body, `<h2>Sender check</h2><div class="sp-end"><span class="tag is-warning is-light">warn</span>`) {
|
||||
return false, fmt.Errorf("the card does not warn about mail deferred a moment ago")
|
||||
}
|
||||
return true, nil
|
||||
}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
@@ -15,10 +15,12 @@ const smtpsAddr = "127.0.0.1:20465"
|
||||
|
||||
// sendAttempt is one SMTP transaction against the stand. An empty authLogin
|
||||
// means "connect without AUTH" (negative checks 1/3); from/to are full
|
||||
// mailbox addresses.
|
||||
// mailbox addresses. headerFrom is the From: line of the message when it is
|
||||
// not to be the envelope sender (the sender check holds the two to one rule).
|
||||
type sendAttempt struct {
|
||||
authLogin, authPassword string
|
||||
from, to string
|
||||
headerFrom string
|
||||
subject, body string
|
||||
}
|
||||
|
||||
@@ -135,6 +137,9 @@ func waitForSMTPSReady(timeout time.Duration) error {
|
||||
// to carry a unique token so the harness can find this exact message in the
|
||||
// sink-MX's dump directory afterwards.
|
||||
func buildMessage(a sendAttempt) string {
|
||||
if a.headerFrom != "" {
|
||||
a.from = a.headerFrom
|
||||
}
|
||||
return fmt.Sprintf(
|
||||
"From: %s\r\nTo: %s\r\nSubject: %s\r\nDate: %s\r\nMIME-Version: 1.0\r\nContent-Type: text/plain; charset=utf-8\r\n\r\n%s\r\n",
|
||||
a.from, a.to, a.subject, time.Now().UTC().Format(time.RFC1123Z), a.body,
|
||||
|
||||
Reference in New Issue
Block a user