base-trixie: the image on Debian trixie, Postfix 3.10.13; Received: hidden on 465/587
test / test (push) Canceled after 0s

The runtime base is debian:trixie-slim, the build stage golang:1.26-trixie:
Postfix 3.10.13 with upstream's 2026 security fixes up to the August
batch, libmilter 8.18.1, OpenSSL 3.5. Every citation of another program in
code, scripts and documents was re-read against the trixie sources
(dev/upstream; bookworm's kept in dev/upstream-bookworm) and updated;
development.md's cited-package table holds the trixie versions.

What the re-reading changed in behaviour:
- logtail: reject lines with sasl_method/sasl_username/sasl_sender
  (smtpd_check.c:1019-1026); cleanup's "removed (canceled)" forgets a
  port-25 id (cleanup_api.c:363-364).
- internal/postfix/retry.go reads postconf's stdout only: since 3.9 a
  deprecated-parameter warning on stderr made one value too many and the
  panel fell back to the compiled-in retry policy.
- logrotate-loop.sh: logrotate 3.22 exits 1 when it ignores a config;
  the loop refuses on that output whatever the status.
- postfix-config.sh: cyrus_sasl_config_path set explicitly (bookworm's
  07_sasl_config.diff is gone); maillog_file_permissions=0640; mynetworks
  explicit; Dockerfile: SYNC_CHROOT= in /etc/default/postfix,
  opendkim-tools dropped, openssl named, systemd-standalone-sysusers so
  cron does not bring the whole of systemd.
- The selfpost group is 10998, outside the system range 100-999: in
  trixie a package took 998 and the build failed (owner's decision:
  a conflict-free id, the distribution's own ids untouched). The
  entrypoint hands /data back by name when the user or the group differs.
  Guide, backup refusal text, help (design commit 68ec8b2), e2e.
- smtpd_hide_client_session=yes on smtps and submission (owner's
  decision): Received: names no client, TLS, login or protocol; port 25
  keeps the full header. e2e checks both.
- test/upstream/opendkim-probe.sh makes keys with openssl genrsa.

Tests: on the server, make e2e exit 0 (674 s, the image built on trixie),
go vet + go test 27 packages ok. CHANGELOG [Unreleased], guide,
architecture.md, security.md, roadmap.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
mix
2026-10-15 19:31:35 +03:00
co-authored by Claude Opus 5.5
parent 68ec8b274c
commit 1c6e04b7d3
80 changed files with 970 additions and 454 deletions
+1 -1
View File
@@ -55,7 +55,7 @@ type SASLDB struct {
// realm should match SELFPOST_HOSTNAME so the account identity lines up with
// Postfix's SASL configuration.
//
// Postfix 3.7.11, src/xsasl/xsasl_cyrus_server.c, xsasl_cyrus_server_create():
// Postfix 3.10.13, src/xsasl/xsasl_cyrus_server.c, xsasl_cyrus_server_create():
// sasl_server_new() gets $myhostname and, with smtpd_sasl_local_domain empty,
// no user realm; Cyrus SASL 2.1.28, plugins/sasldb.c,
// sasldb_auxprop_lookup(), then looks a login without "@" up under the
+1 -1
View File
@@ -32,7 +32,7 @@ func invalid(format string, args ...any) error {
// (idx_applications_login_nocase); a login imported from another instance is
// not folded, since its SASL account and its client use it as it is.
//
// Postfix 3.7.11, src/smtpd/smtpd_check.c, reject_auth_sender_login_mismatch():
// Postfix 3.10.13, src/smtpd/smtpd_check.c, reject_auth_sender_login_mismatch():
// the SASL login is compared with each owner by strcasecmp_utf8().
func foldLogin(login string) string {
return strings.ToLower(login)
+1 -1
View File
@@ -91,7 +91,7 @@ type Params struct {
// bring back is not the queue. Nor could the panel read it - Postfix keeps
// every queue file readable by its own user alone, which is why the panel's
// backup failed outright from 1.3.0 to 2.0.0 (date not recorded; Postfix
// 3.7.11, conf/postfix-files: active, deferred, incoming and the other queue
// 3.10.13, conf/postfix-files: active, deferred, incoming and the other queue
// directories are $mail_owner 700). What is not carried over is mail
// still waiting for delivery; a start without a queue directory builds an
// empty one. Whoever needs the queue stops the instance and archives the
+2 -2
View File
@@ -26,7 +26,7 @@ const certWarnDays = 14
// Postfix itself serves the copy build/postfix-config.sh makes in
// /etc/postfix/tls-internal, refreshed by build/postfix-cert-reload.sh up to
// TLS_RELOAD_INTERVAL_SECONDS later, and each smtpd loads it when it starts
// (Postfix 3.7.11, src/smtpd/smtpd.c, pre_jail_init(), TLS_SERVER_INIT) - so
// (Postfix 3.10.13, src/smtpd/smtpd.c, pre_jail_init(), TLS_SERVER_INIT) - so
// the mounted file can be newer than what is served (assumptions A-65).
// CheckServedCertificate reads the copy, and names the mounted file as a
// renewal waiting while the two differ.
@@ -52,7 +52,7 @@ type Renewal struct {
}
// ServedCertPath is the certificate file Postfix serves, smtpd_tls_cert_file
// as postconf prints it (Postfix 3.7.11, postconf(1): -h, the value without
// as postconf prints it (Postfix 3.10.13, postconf(1): -h, the value without
// its name): build/postfix-config.sh's copy when the mounted files were there
// at the start, the mounted file itself otherwise. "" when postconf fails.
// It is read once, at the panel's start, as the retry policy is: main.cf is
+1 -1
View File
@@ -17,7 +17,7 @@ import (
const milterDialTimeout = time.Second
// CheckMilter probes a milter address in the two forms build/postfix-config.sh
// lets INBOUND_ANTISPAM_MILTER take (Postfix 3.7.11, src/milter/milter8.c,
// lets INBOUND_ANTISPAM_MILTER take (Postfix 3.10.13, src/milter/milter8.c,
// milter8_connect(), also accepts "local:", which that script refuses):
// unix:/path is stat-ed like any milter socket, and inet:host:port is dialled
// and the connection closed at once, with nothing written. required says what
+2 -2
View File
@@ -26,7 +26,7 @@ type Socket struct {
// stops mail leaving the server, while the journal-milter is attached with
// default_action=accept and fails open — mail still goes out, only the send
// log stops being written (build/postfix-config.sh smtpd_milters; Postfix
// 3.7.11, src/milter/milter8.c, milter8_comm_error(): tempfail answers "451
// 3.10.13, src/milter/milter8.c, milter8_comm_error(): tempfail answers "451
// 4.7.1 Service unavailable - try again later", accept answers nothing).
//
// The socket is dialled, as Postfix will dial it; a file is not a listener.
@@ -35,7 +35,7 @@ type Socket struct {
// a killed OpenDKIM or panel leaves a file that refuses every connection -
// assumptions A-38. The connection is closed before a byte is sent, which
// none of the three milters logs: OpenDKIM's libmilter reads the EOF as
// SMFIC_EOF and ends the session without a line (sendmail 8.17.1.9,
// SMFIC_EOF and ends the session without a line (libmilter 8.18.1,
// libmilter/comm.c, mi_rd_cmd(); engine.c, mi_engine()), and OpenDKIM's
// mlfi_close() has no connection to clean up (opendkim 2.11.0~beta2,
// opendkim/opendkim.c); go-milter v0.4.1 returns quietly on io.EOF
+1 -1
View File
@@ -79,7 +79,7 @@ func (e *UpstreamShared) Error() string {
// SetTransportAs validates and saves the upstream of domain id, then rebuilds
// the maps. An upstream has one TLS mode: Postfix's TLS policy is keyed by the
// next hop, not by the domain (smtp_tls_policy_maps; Postfix 3.7.11,
// next hop, not by the domain (smtp_tls_policy_maps; Postfix 3.10.13,
// README_FILES/TLS_README: the key is the verbatim next-hop), so two domains
// to one [host]:port share one mode - the map held both keys and the first by
// domain name won with a warning (audit 2026-10-12, L5). The other domains
+28 -16
View File
@@ -45,11 +45,15 @@ type Inbound struct {
// so it counts only for a queue id the port-25 smtpd was seen to open
// (inboundClientRe).
//
// Postfix 3.7.11: smtpd's refusal is src/smtpd/smtpd_check.c, log_whatsup():
// Postfix 3.10.13: smtpd's refusal is src/smtpd/smtpd_check.c, log_whatsup():
// "%s: %s: %s from %s: %s;" then " from=<..>", " to=<..>", " proto=",
// " helo=<..>", nothing after; a milter's at smtpd is src/smtpd/smtpd.c,
// " helo=<..>", then - new in 3.10 (RELEASE_NOTES, [Incompat 20241023]) -
// " sasl_method=%s", " sasl_username=%s" and " sasl_sender=%s" for a session
// that has them, nothing after; a milter's at smtpd is src/smtpd/smtpd.c,
// check_milter_reply(), whose smtpd_whatsup() appends " sasl_username=<..>"
// only to an authenticated session (SASL is off on port 25). "name[addr]" is
// only to an authenticated session. SASL is off on port 25
// (build/postfix-config.sh, smtpd_sasl_auth_enable=no), so neither tail is
// written there; the pattern takes the 3.10 one all the same. "name[addr]" is
// state->namaddr, the address without "IPv6:" (smtpd.h,
// SMTPD_BUILD_NAMADDRPORT). cleanup's is src/cleanup/cleanup_milter.c,
// cleanup_milter_apply(); smtpd hands it to the one cleanup service
@@ -64,21 +68,29 @@ type Inbound struct {
// journal rows.
//
// For the same reason the reply is the shortest text before "; from=<", and
// the HELO name - the last thing on the line, which Postfix logs as the client
// wrote it, spaces and brackets included - is whatever is left. Taking the
// longest reply instead would let a HELO name that itself spells
// the HELO name - the last thing on the line but the SASL fields above, which
// Postfix logs as the client wrote it, spaces and brackets included - is
// whatever is left. Taking the longest reply instead would let a HELO name
// that itself spells
// "; from=<…> to=<…> proto=…" supply the sender, the recipient and the reply
// of the row (security review, 2026-10-12).
var (
inboundRejectRe = regexp.MustCompile(`^\S+ postfix/(inbound/smtpd|cleanup)\[\d+\]: ([0-9A-Za-z]+): (reject|milter-reject|milter-discard): \S+ from [^\[\s]*\[([0-9A-Fa-f:.]+)\]: (.*?); from=<([^>]*)> to=<([^>]*)> proto=\S+(?: helo=<.*>)?$`)
inboundRejectRe = regexp.MustCompile(`^\S+ postfix/(inbound/smtpd|cleanup)\[\d+\]: ([0-9A-Za-z]+): (reject|milter-reject|milter-discard): \S+ from [^\[\s]*\[([0-9A-Fa-f:.]+)\]: (.*?); from=<([^>]*)> to=<([^>]*)> proto=\S+(?: helo=<.*>)?(?: sasl_(?:method|username|sender)=.*)?$`)
inboundClientRe = regexp.MustCompile(`^\S+ postfix/inbound/smtpd\[\d+\]: ([0-9A-Za-z]+): client=`)
// A queue id stops being port 25's when the message is queued (qmgr takes
// it: the filter let it through) or when another listener opens a message
// under the same id - Postfix reuses ids. Postfix 3.7.11,
// it: the filter let it through), when cleanup removes the queue file of
// a message it did not queue, or when another listener opens a message
// under the same id - Postfix reuses ids. Postfix 3.10.13,
// src/qmgr/qmgr_message.c, qmgr_message_read() calls opened()
// (src/global/opened.c: "%s: from=<%s>, size=%ld, nrcpt=%d") for a queue
// file cleanup has committed, each time it enters the active queue.
inboundForgetRe = regexp.MustCompile(`^\S+ postfix/(?:qmgr\[\d+\]: ([0-9A-Za-z]+): from=|(?:[a-z0-9-]+/)?smtpd\[\d+\]: ([0-9A-Za-z]+): client=)`)
// file cleanup has committed, each time it enters the active queue;
// src/cleanup/cleanup_api.c, cleanup_flush(): "%s: removed (%s)", with
// "canceled" or "discarded", once the queue file of a refused, discarded
// or unfinished message is gone (new in 3.10, HISTORY 20241112) - after
// the verdict line, which cleanup_milter_inspect() writes earlier in
// cleanup_flush(). Without it, the id of a message whose client went away
// in DATA stayed until inboundQueueIDs pushed it out.
inboundForgetRe = regexp.MustCompile(`^\S+ postfix/(?:qmgr\[\d+\]: ([0-9A-Za-z]+): from=|(?:[a-z0-9-]+/)?smtpd\[\d+\]: ([0-9A-Za-z]+): client=|cleanup\[\d+\]: ([0-9A-Za-z]+): removed \()`)
)
// A row holds one line of reason; Postfix's reply text carries the far end's
@@ -95,7 +107,7 @@ const inboundQueueIDs = 4096
// journal cut back to its row cap, without waiting for the six-hourly prune.
// Refusals are written at the rate strangers send RCPT TO. Postfix's one
// limit on that, smtpd_client_recipient_rate_limit, is per client address and
// off by default (postconf(5), Postfix 3.7.11) and not set here, so between
// off by default (postconf(5), Postfix 3.10.13) and not set here, so between
// two prunes the journal would otherwise grow as fast as someone cared to
// make it.
const inboundPruneEvery = 500
@@ -140,7 +152,7 @@ func (j *inboundJournal) handle(line string, now time.Time, catchingUp bool) {
return
}
if m := inboundForgetRe.FindStringSubmatch(rest); m != nil {
delete(j.port25, m[1]+m[2])
delete(j.port25, m[1]+m[2]+m[3])
return
}
m := inboundRejectRe.FindStringSubmatch(rest)
@@ -150,7 +162,7 @@ func (j *inboundJournal) handle(line string, now time.Time, catchingUp bool) {
program, queueID, kind, clientIP, reason, from, to := m[1], m[2], m[3], m[4], m[5], m[6], m[7]
// The sender's address is his own text too, and with a quoted local part it
// may hold "> to=<…> proto=…" - which would end the sender early and supply
// the recipient (Postfix 3.7.11 logs addresses in external form, the local
// the recipient (Postfix 3.10.13 logs addresses in external form, the local
// part quoted when it holds spaces or specials: postconf(5)
// info_log_address_format, src/global/info_log_addr_form.c). Such a line
// shows it: the sender is left with an unclosed quote. An address with a
@@ -198,7 +210,7 @@ func (j *inboundJournal) handle(line string, now time.Time, catchingUp bool) {
e.Engine = store.SpamEnginePostfix
}
// "450 4.7.1 …" from Postfix, "4.7.1 …" from a milter: either way a reply
// that starts with 4 asks the sender to come back. Postfix 3.7.11:
// that starts with 4 asks the sender to come back. Postfix 3.10.13:
// smtpd_check_reject() logs smtpd's whole "%d %s ..." reply through
// log_whatsup(); cleanup_milter_apply() logs a
// milter's from its fifth byte (text = resp + 4), so the enhanced code
@@ -263,7 +275,7 @@ func printable(s string, max int) string {
// tailer got to it: after the panel was down the two are hours apart. The
// syslog format carries neither year nor zone - the year is taken from now
// (a stamp that would then lie in the future is last year's), the zone is the
// container's. Postfix 3.7.11, src/util/msg_logger.c, msg_logger_print(): the
// container's. Postfix 3.10.13, src/util/msg_logger.c, msg_logger_print(): the
// stamp is made by the process that logs, with localtime(), and postlogd
// writes it unchanged (src/postlogd/postlogd.c, postlogd_service()); that
// process has the container's zone (TZ is in import_environment,
+53 -10
View File
@@ -53,8 +53,9 @@ func inboundFixture() (*inboundJournal, *fakeInboundStore) {
var tailNow = time.Date(2026, 10, 11, 18, 0, 0, 0, time.UTC)
// The mail.log lines in this file are composed by hand, after
// src/smtpd/smtpd_check.c log_whatsup() and src/cleanup/cleanup_milter.c
// cleanup_milter_apply() (Postfix 3.7.11), with the stamp Postfix 3.7 writes
// src/smtpd/smtpd_check.c log_whatsup(), src/cleanup/cleanup_milter.c
// cleanup_milter_apply() and src/cleanup/cleanup_api.c cleanup_flush()
// (Postfix 3.10.13), with the stamp Postfix 3.10 writes
// (src/util/msg_logger.c, "%b %d %H:%M:%S").
const (
head = "Oct 11 12:00:05 mail "
@@ -89,6 +90,26 @@ func TestInboundTempfailByPostfix(t *testing.T) {
}
}
// Postfix 3.10's log_whatsup() appends " sasl_method=%s", " sasl_username=%s"
// and " sasl_sender=%s" after the HELO name for a session that has them
// (src/smtpd/smtpd_check.c; RELEASE_NOTES, [Incompat 20241023]). Port 25 has
// SASL off, so it never writes them there; a line that had them, with the
// HELO name or without it, is still read for what comes before. Composed
// from that format.
func TestInboundRejectTakesTheSASLTail(t *testing.T) {
for name, line := range map[string]string{
"after the HELO name": head + "postfix/inbound/smtpd[41]: NOQUEUE: reject: RCPT from mx.example.net[203.0.113.9]: 550 5.1.1 <nobody@in.example.com>: Recipient address rejected: User unknown in relay recipient table; from=<a@example.net> to=<nobody@in.example.com> proto=ESMTP helo=<mx.example.net> sasl_method=PLAIN sasl_username=u sasl_sender=s",
"with no HELO name": head + "postfix/inbound/smtpd[41]: NOQUEUE: reject: RCPT from mx.example.net[203.0.113.9]: 550 5.1.1 <nobody@in.example.com>: Recipient address rejected: User unknown in relay recipient table; from=<a@example.net> to=<nobody@in.example.com> proto=ESMTP sasl_method=PLAIN sasl_username=u",
} {
j, st := inboundFixture()
j.handle(line, tailNow, false)
if len(st.rows) != 1 || st.rows[0].From != "a@example.net" || st.rows[0].To != "nobody@in.example.com" ||
st.rows[0].Detail != "550 5.1.1 <nobody@in.example.com>: Recipient address rejected: User unknown in relay recipient table" {
t.Errorf("%s: rows = %+v", name, st.rows)
}
}
}
// The filter's verdict is written by cleanup, under one name for every
// listener: it is the journal's only for a queue id port 25 opened.
func TestInboundMilterVerdictNeedsAPort25QueueID(t *testing.T) {
@@ -137,7 +158,7 @@ func TestInboundIgnores(t *testing.T) {
for name, line := range map[string]string{
"another domain (relay access denied)": head + "postfix/inbound/smtpd[41]: NOQUEUE: reject: RCPT from x[203.0.113.9]: 554 5.7.1 <c@elsewhere.example>: Relay access denied; from=<a@example.net> to=<c@elsewhere.example> proto=ESMTP helo=<x>",
"no recipient (size limit at MAIL)": head + "postfix/inbound/smtpd[41]: NOQUEUE: reject: MAIL from x[203.0.113.9]: 552 5.3.4 Message size exceeds fixed limit; proto=ESMTP helo=<x>",
"a submission port": head + "postfix/smtpd[20]: NOQUEUE: reject: RCPT from x[203.0.113.9]: 553 5.7.1 <b@in.example.com>: Sender address rejected: not owned by user app; from=<a@example.net> to=<b@in.example.com> proto=ESMTP helo=<x>",
"a submission port": head + "postfix/smtpd[20]: NOQUEUE: reject: RCPT from x[203.0.113.9]: 553 5.7.1 <b@in.example.com>: Sender address rejected: not owned by user app; from=<a@example.net> to=<b@in.example.com> proto=ESMTP helo=<x> sasl_method=PLAIN sasl_username=app",
"a delivery line": head + "postfix/smtp[26]: 41E862C00D9E: to=<b@in.example.com>, relay=mx[203.0.113.9]:25, dsn=2.0.0, status=sent (250 OK)",
"a connect line": head + "postfix/inbound/smtpd[41]: connect from mx.example.net[203.0.113.9]",
// The HELO name is the sender's; a line cannot be started from inside one.
@@ -176,11 +197,28 @@ func TestInboundLookupFailureLeavesNoRow(t *testing.T) {
// A message the filter holds goes on to the inbound journal milter, which
// writes its row; a second one from the log would be the same message twice.
func TestInboundHoldIsTheMiltersRow(t *testing.T) {
j, st := inboundFixture()
j.handle(head+"postfix/inbound/smtpd[41]: Q1: client=mx.example.net[203.0.113.9]", tailNow, false)
j.handle(head+"postfix/cleanup[52]: Q1: milter-hold: END-OF-MESSAGE from mx.example.net[203.0.113.9]: milter triggers HOLD action; from=<a@example.net> to=<b@in.example.com> proto=ESMTP helo=<mx.example.net>", tailNow, false)
if len(st.rows) != 0 {
t.Errorf("rows = %+v", st.rows)
// Postfix 3.10.13 logs a quarantine with the milter's reason, or
// "default_action" when milter_default_action asked for it, where 3.7
// wrote "milter triggers HOLD action" (src/cleanup/cleanup_milter.c,
// cleanup_milter_apply(), and src/smtpd/smtpd.c, check_milter_reply():
// text = reply[1] ? reply + 1 : "milter triggers HOLD action";
// src/milter/milter8.c, SMFIR_QUARANTINE "H%.*s" and milter8_comm_error()
// "Hdefault_action"). Whatever the reason says, the line is a hold.
// Composed from those formats.
for _, hold := range []string{
"postfix/cleanup[52]: Q1: milter-hold: END-OF-MESSAGE from mx.example.net[203.0.113.9]: milter triggers HOLD action; from=<a@example.net> to=<b@in.example.com> proto=ESMTP helo=<mx.example.net>",
"postfix/cleanup[52]: Q1: milter-hold: END-OF-MESSAGE from mx.example.net[203.0.113.9]: Gtube pattern; from=<a@example.net> to=<b@in.example.com> proto=ESMTP helo=<mx.example.net>",
"postfix/cleanup[52]: Q1: milter-hold: END-OF-MESSAGE from mx.example.net[203.0.113.9]: default_action; from=<a@example.net> to=<b@in.example.com> proto=ESMTP helo=<mx.example.net>",
// A reason that spells a reject is still the reason of a hold.
"postfix/cleanup[52]: Q1: milter-hold: END-OF-MESSAGE from mx.example.net[203.0.113.9]: x: milter-reject: END-OF-MESSAGE from y[192.0.2.1]: 5.7.1 no; from=<a@example.net> to=<b@in.example.com> proto=ESMTP helo=<mx.example.net>",
"postfix/inbound/smtpd[41]: Q1: milter-hold: RCPT from mx.example.net[203.0.113.9]: held for review; from=<a@example.net> to=<b@in.example.com> proto=ESMTP helo=<mx.example.net>",
} {
j, st := inboundFixture()
j.handle(head+"postfix/inbound/smtpd[41]: Q1: client=mx.example.net[203.0.113.9]", tailNow, false)
j.handle(head+hold, tailNow, false)
if len(st.rows) != 0 {
t.Errorf("%s: rows = %+v", hold, st.rows)
}
}
}
@@ -216,13 +254,18 @@ func TestInboundHeloCannotWriteTheRow(t *testing.T) {
}
}
// A queue id is port 25's only until the message is queued or the id is
// opened again by another listener - Postfix reuses ids.
// A queue id is port 25's only until the message is queued, cleanup removes
// it unqueued, or the id is opened again by another listener - Postfix reuses
// ids.
func TestInboundQueueIDIsForgotten(t *testing.T) {
const verdict = "postfix/cleanup[52]: Q1: milter-reject: END-OF-MESSAGE from app[203.0.113.4]: 4.7.1 Service unavailable - try again later; from=<a@example.net> to=<b@in.example.com> proto=ESMTP helo=<app>"
for name, forget := range map[string]string{
"queued": "postfix/qmgr[58]: Q1: from=<a@example.net>, size=600, nrcpt=1 (queue active)",
"reused by the submission port": "postfix/smtpd[20]: Q1: client=app.example.com[203.0.113.4], sasl_method=PLAIN, sasl_username=app",
// Postfix 3.10's cleanup_flush(): "%s: removed (%s)", for a message
// whose client went away in DATA, or that was discarded.
"canceled by cleanup": "postfix/cleanup[52]: Q1: removed (canceled)",
"discarded by cleanup": "postfix/cleanup[52]: Q1: removed (discarded)",
} {
j, st := inboundFixture()
j.handle(head+"postfix/inbound/smtpd[41]: Q1: client=mx.example.net[203.0.113.9]", tailNow, false)
+19 -9
View File
@@ -3,6 +3,7 @@ package logtail
import (
"log"
"regexp"
"strings"
"time"
"github.com/mixeme/selfpost/internal/store"
@@ -28,7 +29,7 @@ import (
// enable_long_queue_ids, digits and letters without vowels; neither word is
// either.
//
// Postfix 3.7.11, src/global/mail_queue.c, mail_queue_enter(), with
// Postfix 3.10.13, src/global/mail_queue.c, mail_queue_enter(), with
// src/global/mail_queue.h MQID_*: a short id is "%05X" of the microseconds
// and "%lX" of the inode (6 to 21 characters); a long one is the seconds (6+)
// and microseconds (4) in src/global/safe_ultostr.c's vowel-less alphabet, 'z',
@@ -38,7 +39,7 @@ var queueLineRe = regexp.MustCompile(`^\S+ ([^\s\[]+)\[\d+\]: ([0-9A-F]{6,24}|[0
// smtpdProgramRe tells an smtpd from the other programs, whatever listener it
// serves: "postfix/smtpd", "postfix/submission/smtpd", "postfix/inbound/smtpd".
// Postfix 3.7.11, src/global/mail_task.c, mail_task(): the tag is
// Postfix 3.10.13, src/global/mail_task.c, mail_task(): the tag is
// "<syslog_name>/<process name>"; a master.cf "-o syslog_name=" reaches it
// through mail_params_init() (src/global/mail_params.c) before
// single_server_main() calls maillog_client_init() again.
@@ -55,7 +56,7 @@ const (
// logged when it arrives and its row is written at end-of-message; an hour
// is a message body sent slowly. Postfix sets no such bound: smtpd_timeout
// limits each read, not the message (postconf(5) smtpd_timeout,
// smtpd_per_request_deadline, Postfix 3.7.11) - assumptions A-34.
// smtpd_per_request_deadline, Postfix 3.10.13) - assumptions A-34.
lineGrace = time.Hour
// lineBatch is how many lines are held before they are written without
@@ -80,13 +81,21 @@ type lineKeeper struct {
// port25 holds the queue ids the port-25 smtpd opened: mail offered by
// strangers, which never gets a send-log row. Its lines are not kept - what
// a stranger can make Postfix log is not something to store on his say-so
// and prune later. An id leaves when qmgr removes the message or when
// another listener opens a message under it (Postfix reuses ids).
// Postfix 3.7.11: smtpd writes "<id>: client=..." when it opens the queue
// and prune later. An id leaves when qmgr removes the message, when
// cleanup removes a message it did not queue, or when another listener
// opens a message under it (Postfix reuses ids).
// Postfix 3.10.13: smtpd writes "<id>: client=..." when it opens the queue
// file (src/smtpd/smtpd.c, mail_open_stream()); qmgr writes "<id>: removed"
// when it deletes it (src/qmgr/qmgr_active.c, qmgr_active_done_3_generic());
// short ids repeat (postconf(5) enable_long_queue_ids: the long ones are
// the "non-repeating" kind).
// cleanup writes "<id>: removed (canceled)" or "<id>: removed (discarded)"
// when it deletes the queue file of a message it refused, discarded or
// was not given in full (src/cleanup/cleanup_api.c, cleanup_flush(),
// msg_info("%s: removed (%s)"), new in 3.10: HISTORY 20241112), after its
// milter verdict line (cleanup_milter_inspect() runs earlier in
// cleanup_flush()) and after smtpd's line about a connection lost in DATA
// (src/smtpd/smtpd.c, smtpd_proto(), logs it before mail_reset() closes the
// cleanup stream); short ids repeat (postconf(5) enable_long_queue_ids:
// the long ones are the "non-repeating" kind).
port25 map[string]struct{}
}
@@ -121,7 +130,8 @@ func (k *lineKeeper) add(line string, now time.Time, catchingUp bool) {
}
}
if _, inbound := k.port25[queueID]; inbound {
if program == "postfix/qmgr" && text == "removed" {
if program == "postfix/qmgr" && text == "removed" ||
program == "postfix/cleanup" && strings.HasPrefix(text, "removed (") {
delete(k.port25, queueID)
}
return
+41
View File
@@ -24,6 +24,15 @@ func TestQueueLineHead(t *testing.T) {
{"port-25 smtpd", "mail postfix/inbound/smtpd[41]: 4A1B2C3D: client=mx.example.net[203.0.113.9]", "4A1B2C3D"},
{"cleanup", "mail postfix/cleanup[52]: 41E862C00D9E: message-id=<1@example.com>", "41E862C00D9E"},
{"qmgr removes it", "mail postfix/qmgr[10]: 41E862C00D9E: removed", "41E862C00D9E"},
// Postfix 3.10.13: src/cleanup/cleanup_api.c, cleanup_flush(), "%s:
// removed (%s)"; src/smtpd/smtpd.c, smtpd_proto(), "%s: %s after %s
// (%lu bytes) from %s" with the queue id or "NOQUEUE".
{"cleanup removes an unqueued message", "mail postfix/cleanup[52]: 41E862C00D9E: removed (canceled)", "41E862C00D9E"},
{"a connection lost in DATA", "mail postfix/submission/smtpd[10]: 41E862C00D9E: lost connection after DATA (5420 bytes) from unknown[203.0.113.7]", "41E862C00D9E"},
{"a connection lost before a queue id", "mail postfix/smtpd[10]: NOQUEUE: lost connection after EHLO from unknown[203.0.113.7]", ""},
// The same function's "%s: %s after %s from %s" for the other reasons.
{"too many errors with a queue id", "mail postfix/submission/smtpd[10]: 41E862C00D9E: too many errors after RCPT from unknown[203.0.113.7]", "41E862C00D9E"},
{"a timeout before a queue id", "mail postfix/smtpd[10]: NOQUEUE: timeout after MAIL from unknown[203.0.113.7]", ""},
{"a bounce is sent: the id at the head, not the new message's", "mail postfix/bounce[30]: 41E862C00D9E: sender non-delivery notification: 5F00AA12", "41E862C00D9E"},
{"long queue id", "mail postfix/qmgr[10]: 3sJ8Xh2Cgqz4Xy1: removed", "3sJ8Xh2Cgqz4Xy1"},
{"refused before queueing", "mail postfix/inbound/smtpd[41]: NOQUEUE: reject: RCPT from mx.example.net[203.0.113.9]: 550 5.1.1 <n@in.example.com>: Recipient address rejected; from=<a@example.net> to=<n@in.example.com> proto=ESMTP helo=<mx.example.net>", ""},
@@ -162,6 +171,38 @@ func TestLineKeeperLeavesOutPort25Mail(t *testing.T) {
}
}
// A port-25 message that is never queued - refused at end-of-message, or its
// client gone in DATA - ends with cleanup's "removed (canceled)" rather than
// qmgr's "removed", and its id is anybody's again from there. Postfix 3.10.13:
// src/smtpd/smtpd.c, smtpd_proto(), logs the lost connection before
// mail_reset() closes the cleanup stream; src/cleanup/cleanup_api.c,
// cleanup_flush(), logs "%s: removed (%s)" after the milter verdict. The
// lines are composed from those formats.
func TestLineKeeperForgetsPort25AtCleanupRemoval(t *testing.T) {
cs := &captureStore{}
k := newLineKeeper(cs)
now := time.Date(2026, 10, 12, 18, 0, 0, 0, time.UTC)
const bounce = "host postfix/cleanup[52]: 4A1B2C3D: message-id=<20261012180000.4A1B2C3D@mail.example.com>"
for _, line := range []string{
"host postfix/inbound/smtpd[41]: 4A1B2C3D: client=mx.example.net[198.51.100.9]",
"host postfix/inbound/smtpd[41]: 4A1B2C3D: lost connection after DATA (5420 bytes) from mx.example.net[198.51.100.9]",
"host postfix/cleanup[52]: 4A1B2C3D: removed (canceled)",
// The id comes round for a bounce Postfix generates, which has no smtpd.
bounce,
} {
k.add(line, now, false)
}
k.flush()
if got := cs.kept(); len(got) != 1 || got[0] != "4A1B2C3D|"+bounce {
t.Errorf("kept %v, want only the bounce's line", got)
}
if len(k.port25) != 0 {
t.Errorf("port-25 ids still remembered: %v", k.port25)
}
}
// The tailer keeps queue-id lines as it follows the log, and after an unclean
// stop - its saved place lies before lines it had already read - it does not
// keep them a second time.
+19 -18
View File
@@ -62,7 +62,7 @@ func RetentionInterval() time.Duration {
}
// queueIDs lists the messages Postfix currently holds, for the reconcile sweep.
// It is a var so tests can answer without a running Postfix. Postfix 3.7.11,
// It is a var so tests can answer without a running Postfix. Postfix 3.10.13,
// src/showq/showq.c, showq_service(): the listing covers the maildrop,
// active, incoming, deferred and hold queues.
var queueIDs = postfix.QueueIDs
@@ -91,7 +91,7 @@ type RetentionDays func() int
// postfix/smtp[26]: 41E862C00D9E: to=<a@example.net>, relay=…, dsn=2.0.0, status=sent (250 OK)
//
// The "<queue-id>: to=<addr>, …, status=<word>" shape is the per-recipient
// result line: Postfix 3.7.11, src/global/log_adhoc.c, log_adhoc() writes
// result line: Postfix 3.10.13, src/global/log_adhoc.c, log_adhoc() writes
// "%s: to=<%s>", ", orig_to=<%s>" (only when the address was rewritten),
// ", relay=%s", then ", dsn=%s, status=%s (%s)". It is called from sent(),
// defer_append() and bounce_append() in whichever process decides - the
@@ -150,7 +150,7 @@ func parseDelivery(line string) (queueID, recipient, status string, ok bool) {
//
// postfix/qmgr[93]: 41E862C00D9E: from=<app@example.org>, status=expired, returned to sender
//
// Postfix 3.7.11, src/qmgr/qmgr_active.c, qmgr_active_done_25_generic():
// Postfix 3.10.13, src/qmgr/qmgr_active.c, qmgr_active_done_25_generic():
// msg_info("%s: from=<%s>, status=%s, returned to sender") with "expired"
// once the queue lifetime has passed, "force-expired" after postsuper -e.
// It is the only line about the expiry, one per message, and no recipient is
@@ -253,7 +253,7 @@ func reconcileLoop(ctx context.Context, st StatusStore, caughtUp <-chan struct{}
}
// A message Postfix still works on stays queued until its first retry after
// the queue lifetime, which can be a whole backoff later. Postfix 3.7.11,
// the queue lifetime, which can be a whole backoff later. Postfix 3.10.13,
// src/qmgr/qmgr_active.c: expiry is decided when an attempt ends
// (qmgr_active_done_25_generic()), and a deferred message waits at most
// maximal_backoff_time (qmgr_active_done_3_generic()). The deferred queue
@@ -361,7 +361,7 @@ func reconcile(st StatusStore, cutoff time.Time, path string) {
// message's expiry line. A deferred line is not one: a deferred row has one
// by definition. Rows reach the sweep minutes after they were written, so
// the end of those two files is where such a line would be.
// logrotate 3.21.0, logrotate.c, prerotateSingleLog(): the first rotated
// logrotate 3.22.0, logrotate.c, prerotateSingleLog(): the first rotated
// name is "<log>.1", without the compression suffix under delaycompress
// (build/logrotate-mail.conf).
func resultInLog(path string, row store.QueuedDelivery) bool {
@@ -556,11 +556,11 @@ func QueueLines(path, queueID string, n int) ([]string, error) {
// mentionsQueueID reports whether a mail.log line is about queueID. Postfix
// writes the id followed by a colon ("… postfix/smtp[26]: 41E862C00D9E: to=…";
// Postfix 3.7.11 "%s: ..." in log_adhoc(), smtpd's "%s: client=", qmgr's
// "%s: removed"),
// and the match is anchored on the character before it so a shorter id is not
// found inside a longer one — queue ids are hexadecimal, and one being the tail
// of another is ordinary, not unlikely.
// Postfix 3.10.13 "%s: ..." in log_adhoc(), smtpd's "%s: client=", qmgr's
// "%s: removed", cleanup's "%s: removed (%s)"), and the match is anchored on
// the character before it so a shorter id is not found inside a longer one —
// queue ids are hexadecimal, and one being the tail of another is ordinary,
// not unlikely.
func mentionsQueueID(line, queueID string) bool {
needle := queueID + ":"
for i := 0; i <= len(line)-len(needle); {
@@ -582,16 +582,17 @@ func isQueueIDByte(b byte) bool {
}
// syslogStampRe is the timestamp at the head of a mail.log line (maillog_file
// in build/postfix-config.sh), the one format Postfix 3.7 writes. Postfix 3.7.11,
// src/util/msg_logger.c, msg_logger_print(): every record is stamped
// in build/postfix-config.sh), the one format Postfix 3.10 writes. Postfix
// 3.10.13, src/util/msg_logger.c, msg_logger_print(): every record is stamped
// strftime("%b %d %H:%M:%S ") in the logging process's local time (the day
// zero-padded), then the first label of the host name, and postlogd writes it
// as received (src/postlogd/postlogd.c, postlogd_service()). It carries no
// year and no zone, so the stamp shown is a wall clock and nothing more,
// which is all this column claims to be. An RFC 3339 stamp once matched
// here too, for a maillog_file_format parameter said to arrive in Postfix
// 3.9; there is none in 3.7.11, nor in the 3.12 snapshot of 2026-09-25
// (proto/postconf.proto; msg_logger.c keeps the strftime) - assumptions A-12.
// 3.9; there is none in 3.7.11, nor in 3.10.13, nor in the 3.12 snapshot of
// 2026-09-25 (proto/postconf.proto; msg_logger.c keeps the strftime) -
// assumptions A-12.
var syslogStampRe = regexp.MustCompile(`^([A-Z][a-z]{2}\s+\d{1,2} \d{2}:\d{2}:\d{2})\s`)
// SplitTimestamp separates the timestamp at the head of a mail.log line from
@@ -615,7 +616,7 @@ func SplitTimestamp(line string) (stamp, rest string) {
// first ever run); it reopens the file when it is rotated, so nothing is missed.
// The inode changes under logrotate's create, which build/logrotate-mail.conf
// uses; a file shorter than the read position is taken as truncation.
// logrotate 3.21.0, logrotate.c, rotateSingleLog(): without copytruncate the
// logrotate 3.22.0, logrotate.c, rotateSingleLog(): without copytruncate the
// file is rename()d and, with create, a new one made by createOutputFile();
// with copytruncate it is copied and truncated, which that config does not use.
//
@@ -817,14 +818,14 @@ func follow(ctx context.Context, path string, tr *tracker, caughtUp chan struct{
}
// rotatedTail is mail.log just rotated away, still open: postlogd writes to
// it until "postfix reload" makes it exit. Postfix 3.7.11,
// it until "postfix reload" makes it exit. Postfix 3.10.13,
// src/postlogd/postlogd.c, pre_jail_init(): maillog_file is opened once per
// postlogd process; src/master/master_service.c, master_restart_service(),
// closes the service's status pipe at a reload (master_status_cleanup()),
// and src/master/dgram_server.c, dgram_server_abort(), exits when it reads
// its end - the next postlogd opens the new file. logrotate runs the reload
// right after the rename and create (build/logrotate-mail.conf, postrotate;
// logrotate 3.21.0, logrotate.c, rotateLogSet(): the postrotate script runs
// logrotate 3.22.0, logrotate.c, rotateLogSet(): the postrotate script runs
// right after rotateSingleLog() has renamed and created), so the gap is short;
// rotatedGrace is far longer than it.
type rotatedTail struct {
@@ -845,7 +846,7 @@ const rotatedGrace = time.Minute
// lost to a restart and lands on the days it covered. A stretch shorter than
// blindMin is not counted: a rotation leaves the name missing for a tick
// between rename and create, and that loses nothing - the new file is read
// from its start. logrotate 3.21.0, logrotate.c, rotateSingleLog(): the
// from its start. logrotate 3.22.0, logrotate.c, rotateSingleLog(): the
// rename() and createOutputFile() follow each other in one pass, with no
// script or wait between them (postrotate runs after).
type blindness struct {
+26 -4
View File
@@ -15,12 +15,12 @@ import (
)
// The mail.log lines in this file are composed by hand, after the formats the
// code cites (Postfix 3.7.11, src/global/log_adhoc.c and the others), unless a
// code cites (Postfix 3.10.13, src/global/log_adhoc.c and the others), unless a
// comment says where one was seen; some carry no stamp, which the parsers
// take as they come. One shape Postfix does not write here: opendkim lines in
// mail.log (OpenDKIM logs through syslog(3), to the container log - build/opendkim.conf).
// qmgr's expiry line, as Postfix 3.7.11 writes it (src/qmgr/qmgr_active.c,
// qmgr's expiry line, as Postfix 3.10.13 writes it (src/qmgr/qmgr_active.c,
// qmgr_active_done_25_generic(): "%s: from=<%s>, status=%s, returned to
// sender"); composed by hand after that format. The sender is the client's
// text, so a line only resembles one when it ends as qmgr's does.
@@ -36,6 +36,11 @@ func TestParseExpiry(t *testing.T) {
// qmgr's ordinary from= line, whose sender carries the text.
{"Oct 14 09:01:02 relay postfix/qmgr[93]: 41E862C00D9E: from=<x>, status=expired, returned to sender>, size=500, nrcpt=1 (queue active)", ""},
{"Oct 14 09:01:02 relay postfix/qmgr[93]: 41E862C00D9E: removed", ""},
// cleanup's removal of a message it did not queue (Postfix 3.10.13,
// src/cleanup/cleanup_api.c, cleanup_flush(): "%s: removed (%s)")
// says nothing of what became of a queued one.
{"Oct 14 09:01:02 relay postfix/cleanup[52]: 41E862C00D9E: removed (canceled)", ""},
{"Oct 14 09:01:02 relay postfix/cleanup[52]: 41E862C00D9E: removed (discarded)", ""},
} {
id, ok := parseExpiry(c.line)
if ok != (c.id != "") || id != c.id {
@@ -194,7 +199,8 @@ func TestParseDelivery(t *testing.T) {
status: store.StatusSent,
},
{
// The stamp Postfix 3.7's postlogd writes, as on the stand.
// The stamp postlogd writes, as on the stand (Postfix 3.7;
// 3.10.13's msg_logger_print() keeps the format).
name: "syslog stamp",
line: "Oct 14 11:05:51 mail postfix/smtp[26]: 41E862C00D9E: to=<a@example.net>, relay=x, dsn=2.0.0, status=sent (250 OK)",
wantOK: true,
@@ -221,6 +227,22 @@ func TestParseDelivery(t *testing.T) {
line: "Oct 14 11:05:51 mail postfix/smtpd[10]: 5B4A2C1D3E: client=unknown[203.0.113.7], sasl_username=41E862C00D9E: to=<victim@example.net>, status=sent",
wantOK: false,
},
{
// Postfix 3.10.13, src/cleanup/cleanup_api.c, cleanup_flush():
// "%s: removed (%s)" for a transaction cleanup did not queue. It
// is no result for any recipient. Composed from that format.
name: "cleanup's removed (canceled) is not one",
line: "Oct 14 11:05:51 mail postfix/cleanup[52]: 41E862C00D9E: removed (canceled)",
wantOK: false,
},
{
// Postfix 3.10.13, src/smtpd/smtpd.c, smtpd_proto(): "%s: %s after
// %s (%lu bytes) from %s", the queue id first since 3.10. Composed
// from that format.
name: "smtpd's lost connection with a queue id is not one",
line: "Oct 14 11:05:51 mail postfix/submission/smtpd[10]: 41E862C00D9E: lost connection after DATA (5420 bytes) from unknown[203.0.113.7]",
wantOK: false,
},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
@@ -920,7 +942,7 @@ func TestSplitTimestamp(t *testing.T) {
stamp, rest string
}{
{
// An RFC 3339 stamp is not one Postfix 3.7 writes (assumptions
// An RFC 3339 stamp is not one Postfix 3.10 writes (assumptions
// A-12): it is not taken off.
name: "RFC 3339 is not a stamp Postfix writes",
line: "2026-08-03T05:15:52.219218+00:00 mail postfix/qmgr[10]: 4A1B2C3D: removed",
+1 -1
View File
@@ -7,7 +7,7 @@ import (
)
// TestMain reads mail.log stamps in UTC, whatever the zone of the machine the
// tests run on: Postfix 3.7 writes "%b %d %H:%M:%S" in the logging process's
// tests run on: Postfix 3.10 writes "%b %d %H:%M:%S" in the logging process's
// local time (src/util/msg_logger.c, msg_logger_print()) and lineTime reads it
// in time.Local, which in the container is the zone the fixtures are written
// in.
+2 -2
View File
@@ -17,7 +17,7 @@ import (
const (
// fingerprintSize is how many bytes from the head of the log identify it.
// Postfix writes a timestamped line per event (Postfix 3.7.11,
// Postfix writes a timestamped line per event (Postfix 3.10.13,
// src/util/msg_logger.c, msg_logger_print(): "%b %d %H:%M:%S" on every
// record), so the first 512 bytes are
// effectively unique per log generation — enough to tell "the file we were
@@ -130,7 +130,7 @@ func (t *tracker) resume(f *os.File) int64 {
// readRotated reads, when the log was rotated while the panel was down, the
// rest of the file the saved place belongs to: logrotate renamed it to
// mail.log.1 (delaycompress leaves it plain - logrotate 3.21.0, logrotate.c,
// mail.log.1 (delaycompress leaves it plain - logrotate 3.22.0, logrotate.c,
// prerotateSingleLog()), and what Postfix wrote there
// between the panel's stop and the rotation - deliveries, refusals, deferrals -
// was otherwise never read (audit 2026-10-12, stage 2b). It is read from the
+11 -8
View File
@@ -18,14 +18,17 @@ import (
//
// postfix/smtpd[1149]: NOQUEUE: milter-reject: MAIL from unknown[10.77.0.1]: 451 4.7.1 Service unavailable - try again later; from=<billing@sender.e2e.test> proto=ESMTP helo=<e2e-client.e2e.test> sasl_username=<l2app>
//
// Postfix 3.7.11, src/smtpd/smtpd.c, check_milter_reply(): "%s: %s: %s from
// Postfix 3.10.13, src/smtpd/smtpd.c, check_milter_reply(): "%s: %s: %s from
// %s: %s;%s" with "NOQUEUE" before the queue file is opened, and
// smtpd_whatsup() for the rest: " from=<..>", " to=<..>", " proto=..",
// " helo=<..>", then " sasl_username=<..>" when the session authenticated
// (assumptions A-20, A-21). The login is read at the end of the line, where
// Postfix puts it last; a line without one is no application's refusal - the
// journal milter refuses at MAIL FROM only an authenticated session
// (milter/ratelimit.go). The HELO name before it is the client's text and can
// (assumptions A-20, A-21). smtpd's own refusals are another function,
// src/smtpd/smtpd_check.c, log_whatsup(), whose line says "reject:", not
// "milter-reject:", and since 3.10 ends in " sasl_method=%s sasl_username=%s"
// without the brackets; it is not one of these. The login is read at the end
// of the line, where Postfix puts it last; a line without one is no
// application's refusal - the journal milter refuses at MAIL FROM only an
// authenticated session (milter/ratelimit.go). The HELO name before it is the client's text and can
// end in what looks like the field, but only in a session that has no login,
// and so no refused row to take the line; a row must also be of the line's
// client address and sender, within minutes.
@@ -52,7 +55,7 @@ type RefusalStore interface {
// The From: line as the check printed it runs to the last " - " before the
// first semicolon; the reason after it holds neither, and the check prints no
// ";" of the client's (sender.go printable), so that semicolon is Postfix's
// and what follows it - the envelope - is Postfix's own text. Postfix 3.7.11,
// and what follows it - the envelope - is Postfix's own text. Postfix 3.10.13,
// src/cleanup/cleanup_milter.c, cleanup_milter_apply(): "%s: %s: %s from
// %s[%s]: %s;" with the reply from its fifth byte (the "550 " dropped), then
// " from=<..>", " to=<..>" - the last recipient only (assumptions A-15).
@@ -60,7 +63,7 @@ var fromRefusalRe = regexp.MustCompile(`^\S+ postfix/(?:[a-z0-9-]+/)?cleanup\[\d
// clientLoginRe is smtpd's line naming the login a queue id was submitted
// under: "<id>: client=host[addr], sasl_method=PLAIN, sasl_username=<login>".
// Postfix 3.7.11, src/smtpd/smtpd.c, mail_open_stream(): ", sasl_sender=.."
// Postfix 3.10.13, src/smtpd/smtpd.c, mail_open_stream(): ", sasl_sender=.."
// follows when the client gave AUTH= on MAIL FROM, then ", orig_queue_id=.."
// and ", orig_client=.." for XFORWARD; the login ends at the comma (a login
// holds none - internal/postfix assertMapSafe; assumptions A-23).
@@ -148,7 +151,7 @@ func (r *refusalLines) handle(line string, now time.Time, start Place) {
return
}
// The address in "name[addr]" is smtpd's state->addr, which has no
// "IPv6:" (Postfix 3.7.11, src/smtpd/smtpd_peer.c, smtpd_peer_init();
// "IPv6:" (Postfix 3.10.13, src/smtpd/smtpd_peer.c, smtpd_peer_init();
// assumptions A-24).
ip := net.ParseIP(m[2])
if ip == nil {
+4 -3
View File
@@ -31,8 +31,9 @@ func (r *refusalProbe) AttachRefusalLine(clientIP, login, from string, _ time.Ti
// The mail.log lines in this file are composed by hand, with the e2e stand's
// names and addresses (smtpd.c check_milter_reply(), mail_open_stream();
// cleanup_milter.c cleanup_milter_apply(), Postfix 3.7.11), with the stamp
// Postfix 3.7 writes (src/util/msg_logger.c, "%b %d %H:%M:%S").
// smtpd_check.c log_whatsup(); cleanup_milter.c cleanup_milter_apply(),
// Postfix 3.10.13), with the stamp Postfix 3.10 writes
// (src/util/msg_logger.c, "%b %d %H:%M:%S").
//
// The NOQUEUE line of a refusal at MAIL FROM goes on its row; a line whose
// row is not written yet waits and is tried again; port 25's and other
@@ -45,7 +46,7 @@ func TestRefusalLines(t *testing.T) {
for _, other := range []string{
`Oct 14 12:00:00 mail postfix/inbound/smtpd[372]: NOQUEUE: milter-reject: MAIL from unknown[203.0.113.7]: 451 4.7.1 Service unavailable - try again later; from=<a@stranger.example> proto=ESMTP helo=<x>`,
`Oct 14 12:00:00 mail postfix/smtpd[11]: NOQUEUE: milter-reject: CONNECT from unknown[10.77.0.1]: 451 4.7.1 Service unavailable - try again later; proto=SMTP`,
`Oct 14 12:00:00 mail postfix/smtpd[11]: NOQUEUE: reject: RCPT from unknown[10.77.0.1]: 553 5.7.1 <a@x>: Sender address rejected: not owned by user app1; from=<a@x> to=<b@y> proto=ESMTP helo=<x>`,
`Oct 14 12:00:00 mail postfix/smtpd[11]: NOQUEUE: reject: RCPT from unknown[10.77.0.1]: 553 5.7.1 <a@x>: Sender address rejected: not owned by user app1; from=<a@x> to=<b@y> proto=ESMTP helo=<x> sasl_method=PLAIN sasl_username=app1`,
// No login: no application's refusal (the panel down refuses every
// session, an unauthenticated one too).
`Oct 14 12:00:00 mail postfix/smtpd[12]: NOQUEUE: milter-reject: MAIL from unknown[10.77.0.1]: 451 4.7.1 Service unavailable - try again later; from=<billing@sender.e2e.test> proto=ESMTP helo=<x>`,
+2 -2
View File
@@ -29,7 +29,7 @@ import (
// The reject lines do not: the same words are written when OpenDKIM does not
// answer and when the journal milter refuses under a level-2 limit.
//
// Postfix 3.7.11, src/milter/milter8.c: milter8_connect() warns "connect to
// Postfix 3.10.13, src/milter/milter8.c: milter8_connect() warns "connect to
// Milter service %s: %m" with the milter's configured name; then
// milter8_comm_error() answers, for default_action=tempfail, "451 4.7.1
// Service unavailable - try again later" - the very text milter8_event()
@@ -50,7 +50,7 @@ import (
//
// One cleanup process serves message after message and the first warning
// ends the message, so each such warning is a message, counted on its own.
// Postfix 3.7.11, src/milter/milter8.c, milter8_read_resp(): "milter %s:
// Postfix 3.10.13, src/milter/milter8.c, milter8_read_resp(): "milter %s:
// can't read %s reply packet header: %m", then milter8_comm_error(), which
// closes the milter's stream and leaves it in the error state for the rest of
// that message.
+1 -1
View File
@@ -39,7 +39,7 @@ func (d *deferredStore) AddSenderCheckDeferred(at time.Time, n int64, place stri
// that use the same words - a level-2 refusal of the journal milter, OpenDKIM
// not answering. Those - smtpd 1149, 1190 and 1201, the second client of
// 1183, smtpd 1300 and the OpenDKIM warning of cleanup 1339 - are composed,
// after src/smtpd/smtpd.c and src/milter/milter8.c (Postfix 3.7.11).
// after src/smtpd/smtpd.c and src/milter/milter8.c (Postfix 3.10.13).
func TestDeferredCounter(t *testing.T) {
st := &deferredStore{}
c := newDeferredCounter(&SenderCheck{Store: st, Socket: "/run/selfpost/sender.sock"})
+1 -1
View File
@@ -31,7 +31,7 @@ func TestCleanAddress(t *testing.T) {
// The journal row's recipient has to be the address Postfix later logs in
// to=<…>, or the delivery line matches no row and the sweep closes it as
// bounced: the local part unquoted, then quoted again only when it is not a
// dot-string (Postfix 3.7.11, src/global/tok822_parse.c and
// dot-string (Postfix 3.10.13, src/global/tok822_parse.c and
// quote_822_local.c). Composed by hand after those functions.
func TestRecipientAsLogged(t *testing.T) {
for _, c := range []struct{ in, want string }{
+3 -3
View File
@@ -34,7 +34,7 @@ type InboundStore interface {
// Postfix attaches the socket with default_action=accept, so
// neither an error here nor the panel being down holds up inbound mail.
//
// Postfix 3.7.11, src/milter/milter.c, milter_message(): the milters are asked
// Postfix 3.10.13, src/milter/milter.c, milter_message(): the milters are asked
// in turn while the reply is null, so one that refuses is the last asked;
// src/milter/milter8.c, milter8_comm_error(): default_action=accept gives no
// reply when this milter cannot answer.
@@ -91,7 +91,7 @@ func (s *inboundSession) RcptTo(rcpt string, m *milter.Modifier) (milter.Respons
// the last one read is its own - when it is up. Whether it was is decided at
// end-of-message, by the signature.
//
// Postfix 3.7.11, src/milter/milter.c, milter_message(), hands each milter the
// Postfix 3.10.13, src/milter/milter.c, milter_message(), hands each milter the
// queue file from the start of the content; src/milter/milter8.c,
// milter8_message(), reads it with rec_get(), which follows the pointer
// records src/cleanup/cleanup_milter.c, cleanup_add_header(), leaves after
@@ -151,7 +151,7 @@ func (s *inboundSession) signed(queueID string) bool {
// upstream. "Continue" lets it through as "accept" would, and keeps this
// session for the connection's next message.
//
// The queue id: Postfix 3.7.11, src/global/mail_params.h, DEF_MILT_EOD_MACROS
// The queue id: Postfix 3.10.13, src/global/mail_params.h, DEF_MILT_EOD_MACROS
// "i", answered by src/cleanup/cleanup_milter.c, cleanup_milter_eval(). The
// rest as session.Body in milter.go.
func (s *inboundSession) Body(m *milter.Modifier) (milter.Response, error) {
+1 -1
View File
@@ -7,7 +7,7 @@ import (
// reservationTTL bounds how long a message may stay reserved. A reservation is
// released at end-of-message or on ABORT. A client that drops the connection
// mid-transaction still produces ABORT: Postfix 3.7.11, src/smtpd/smtpd.c,
// mid-transaction still produces ABORT: Postfix 3.10.13, src/smtpd/smtpd.c,
// smtpd_proto() on a lost connection runs mail_reset(), which calls
// milter_abort() (src/milter/milter8.c, milter8_abort(), SMFIC_ABORT)
// (assumptions B-G7). What produces neither is smtpd itself dying, or the
+1 -1
View File
@@ -2,7 +2,7 @@ package milter
// The address OpenDKIM signs for, found as OpenDKIM finds it: a port of
// libopendkim's dkim_mail_parse() and its tokenizer, from Debian's
// opendkim 2.11.0~beta2-8+deb12u1 (libopendkim/dkim-mailparse.c, identical
// opendkim 2.11.0~beta2-9.1+b1 (libopendkim/dkim-mailparse.c, identical
// there to upstream's develop branch), line for line. OpenDKIM calls it on the
// value of the message's first From: field (opendkim/opendkim.c, mlfi_eoh(),
// dkimf_findheader(dfc, "from", 0): build/opendkim.conf sets no SenderHeaders),
+15 -15
View File
@@ -13,7 +13,7 @@
// Postfix is configured with default_action=accept for this milter's socket,
// so even a crash or hang fails open (architecture.md § Mail path).
//
// Postfix 3.7.11, src/milter/milter8.c, milter8_comm_error(): a milter that
// Postfix 3.10.13, src/milter/milter8.c, milter8_comm_error(): a milter that
// cannot be reached, or does not answer within milter_*_timeout, gives no
// reply with default_action=accept - the message goes on.
package milter
@@ -55,7 +55,7 @@ type Store interface {
//
// go-milter v0.4.1, session.go, Process(), case CodeMacro: every macro packet
// replaces the map; server.go, Serve(): NewMilter() for each connection.
// Postfix 3.7.11, src/milter/milter8.c, milter8_event(): a macro packet, empty
// Postfix 3.10.13, src/milter/milter8.c, milter8_event(): a macro packet, empty
// or not, goes before every event (milter.c, milter_macro_lookup(), never
// returns null).
type session struct {
@@ -69,7 +69,7 @@ type session struct {
// clientIP is the client's address. Postfix gives it at Connect, once per
// connection, and again with every MAIL FROM as the {client_addr} macro
// (milter_mail_macros in build/postfix-config.sh). The second is the one
// that can be relied on: see MailFrom. Postfix 3.7.11,
// that can be relied on: see MailFrom. Postfix 3.10.13,
// src/smtpd/smtpd_milter.c, smtpd_milter_eval(): {client_addr} is
// state->rfc_addr (see clientAddr).
clientIP string
@@ -84,7 +84,7 @@ type session struct {
// Connect captures the client IP of a connection's first message. It is not
// enough on its own - see MailFrom.
//
// Postfix 3.7.11, src/smtpd/smtpd.c, smtpd_proto(): milter_conn_event() is
// Postfix 3.10.13, src/smtpd/smtpd.c, smtpd_proto(): milter_conn_event() is
// given state->addr, the address without "IPv6:". go-milter v0.4.1,
// session.go, case CodeConn: net.ParseIP of it - nil for "unknown" and for a
// family that sends no address.
@@ -115,7 +115,7 @@ func (s *session) Connect(host, family string, port uint16, addr net.IP, m *milt
//
// go-milter v0.4.1, session.go, HandleMilterCommands(): after a reply whose
// Continue() is false (response.go: accept, discard, reject, tempfail) the
// backend is replaced by NewMilter(). Postfix 3.7.11, src/milter/milter8.c,
// backend is replaced by NewMilter(). Postfix 3.10.13, src/milter/milter8.c,
// milter8_conn_event(): CONNECT goes once per milter connection.
// {auth_authen}: src/smtpd/smtpd_milter.c, smtpd_milter_eval(), gives
// state->sasl_username, null without authentication, and then milter.c,
@@ -146,11 +146,11 @@ func (s *session) MailFrom(from string, m *milter.Modifier) (milter.Response, er
// RcptTo records each recipient. Postfix calls this once per recipient, which
// is what lets the journal keep a separate row per (queue-id, recipient).
//
// Postfix 3.7.11, src/smtpd/smtpd.c, rcpt_cmd(): milter_rcpt_event() once per
// Postfix 3.10.13, src/smtpd/smtpd.c, rcpt_cmd(): milter_rcpt_event() once per
// RCPT TO; a recipient Postfix refused goes only to a milter that asked for
// SMFIP_RCPT_REJ, which Serve's Protocol does not.
// A recipient given twice - in any case - is one row: cleanup keeps the first
// and drops the rest (Postfix 3.7.11, src/cleanup/cleanup_out_recipient.c,
// and drops the rest (Postfix 3.10.13, src/cleanup/cleanup_out_recipient.c,
// cleanup_out_recipient(), been_here() on a table made with BH_FLAG_FOLD in
// cleanup_state.c), so only one delivery line will come (assumptions A-31).
// Its key also holds the DSN NOTIFY= and ORCPT= of the RCPT TO command, which
@@ -181,7 +181,7 @@ func (s *session) Header(name, value string, m *milter.Modifier) (milter.Respons
// message through just as "accept" does - and, unlike it, leaves this session
// in place for the connection's next message (see MailFrom).
//
// go-milter v0.4.1, session.go, case CodeEOB: Body(). Postfix 3.7.11,
// go-milter v0.4.1, session.go, case CodeEOB: Body(). Postfix 3.10.13,
// src/global/mail_params.h: DEF_MILT_EOD_MACROS "i", which
// src/cleanup/cleanup_milter.c, cleanup_milter_eval(), answers with the queue
// id. src/milter/milter8.c, milter8_event(): SMFIR_CONTINUE returns the
@@ -198,7 +198,7 @@ func (s *session) Body(m *milter.Modifier) (milter.Response, error) {
// Postfix rejecting the message for its own reasons). No send-log row will be
// written, so the level-2 slots this message held must go back.
//
// Postfix 3.7.11, src/smtpd/smtpd.c: rset_cmd() and the refusals call
// Postfix 3.10.13, src/smtpd/smtpd.c: rset_cmd() and the refusals call
// mail_reset(), which calls milter_abort() (src/milter/milter8.c,
// milter8_abort(): SMFIC_ABORT); so do the end of every transaction and the
// end of the session in smtpd_proto().
@@ -214,7 +214,7 @@ func (s *session) Abort(m *milter.Modifier) error {
// single-character names (e.g. i) arrive bare. go-milter stores whatever name
// Postfix sends verbatim, so a lookup must try both forms.
//
// Postfix 3.7.11, src/milter/milter.c, milter_macro_lookup(): each name is
// Postfix 3.10.13, src/milter/milter.c, milter_macro_lookup(): each name is
// sent as milter_*_macros spells it. go-milter v0.4.1, session.go, case
// CodeMacro: m.macros[name] = value, the name as received.
func macro(m *milter.Modifier, name string) string {
@@ -228,7 +228,7 @@ func macro(m *milter.Modifier, name string) string {
// Postfix did not send one that reads as an address. An IPv6 address may come
// in Sendmail's form, with "IPv6:" in front.
//
// Postfix 3.7.11, src/smtpd/smtpd_peer.c, smtpd_peer_sockaddr_to_hostaddr():
// Postfix 3.10.13, src/smtpd/smtpd_peer.c, smtpd_peer_sockaddr_to_hostaddr():
// rfc_addr is the dotted quad, "IPv6:" + the address for IPv6, a V4-mapped
// address rewritten to IPv4, or "unknown".
func clientAddr(m *milter.Modifier) string {
@@ -270,7 +270,7 @@ func (s *session) record(queueID string) {
}
// cleanAddress is an address as the milter gets it, without the angle
// brackets: Postfix 3.7.11 hands the milter the command's argument as the
// brackets: Postfix 3.10.13 hands the milter the command's argument as the
// client wrote it, ESMTP parameters as separate strings (src/smtpd/smtpd.c,
// mail_cmd()/rcpt_cmd(), milter_argv()), and go-milter v0.4.1 passes the first
// with "<>" trimmed (session.go, case CodeMail/CodeRcpt). A quoted local part
@@ -280,7 +280,7 @@ func cleanAddress(a string) string {
}
// recipientAsLogged is a recipient in the form Postfix writes it in "to=<..>"
// and "orig_to=<..>", so a delivery line finds its row. Postfix 3.7.11 keeps
// and "orig_to=<..>", so a delivery line finds its row. Postfix 3.10.13 keeps
// the address in internal form - a quoted local part unquoted, a backslash
// dropped before the character it escapes, a blank in quotes made a space
// (src/smtpd/smtpd.c, extract_addr(), over src/global/tok822_parse.c,
@@ -315,7 +315,7 @@ func recipientAsLogged(a string) string {
return quoteLocalPart(local) + domain
}
// quoteLocalPart is quote_822_local() of Postfix 3.7.11
// quoteLocalPart is quote_822_local() of Postfix 3.10.13
// (src/global/quote_822_local.c, with QUOTE_FLAG_8BITCLEAN, the default): a
// local part that is a dot-string - atoms of no blank, control character or
// one of ()<>@,;:\"[], joined by single dots, none first or last - is left as
@@ -366,7 +366,7 @@ func domainOf(addr string) string {
// gets a fresh session bound to rec. It returns nil on a clean shutdown.
//
// go-milter v0.4.1, server.go: Close() closes the listener and Serve() then
// returns ErrServerClosed. Postfix 3.7.11, src/milter/milter8.c,
// returns ErrServerClosed. Postfix 3.10.13, src/milter/milter8.c,
// milter8_message(): SMFIP_NOBODY leaves out only the body chunks - headers,
// end-of-headers and end-of-body (milter8_eob()) still go; milter8_event()
// takes a reply code (SMFIR_REPLYCODE) whatever the action mask.
+13 -13
View File
@@ -39,7 +39,7 @@ import (
// it up (Bindings). The send-log row of a refusal is written by another
// goroutine and is not waited for.
//
// Postfix 3.7.11, src/smtpd/smtpd_check.c, reject_auth_sender_login_mismatch():
// Postfix 3.10.13, src/smtpd/smtpd_check.c, reject_auth_sender_login_mismatch():
// the SASL login is compared with an owner by strcasecmp_utf8();
// src/milter/milter8.c, milter8_comm_error(): default_action=tempfail answers
// "451 4.7.1" for a milter that does not answer. OpenDKIM 2.11.0~beta2,
@@ -103,7 +103,7 @@ func (b *Bindings) fresh() error {
return b.broken
}
// parseSenderMap reads the lines of a sender_login_maps file as Postfix 3.7.11
// parseSenderMap reads the lines of a sender_login_maps file as Postfix 3.10.13
// reads a texthash table (src/util/dict_thash.c, dict_thash_open(), over
// src/util/readlline.c, readllines()):
// - a line that starts with a blank continues the one before; an empty
@@ -112,8 +112,8 @@ func (b *Bindings) fresh() error {
// escaping the next character; a line with an unbalanced quote, or
// without a key or a value, is ignored (with a warning, in Postfix);
// - the key is folded to lower case (DICT_FLAG_FOLD_FIX, which
// src/smtpd/smtpd_check.c, pre_jail_init(), gives
// smtpd_sender_login_maps);
// src/smtpd/smtpd_check.c, smtpd_check_init(), called from
// src/smtpd/smtpd.c, pre_jail_init(), gives smtpd_sender_login_maps);
// - a key given twice makes the whole table unusable: no DICT_FLAG_DUP_*
// flag is given, so dict_thash_open() returns a surrogate that fails
// every lookup, and smtpd answers 451 (reject_server_error()). The check
@@ -194,7 +194,7 @@ func splitTableLine(line string) (key, value string, ok bool) {
}
// Allowed reports whether login may send as addr - the lookup Postfix makes
// for the envelope sender. Postfix 3.7.11, src/smtpd/smtpd_check.c,
// for the envelope sender. Postfix 3.10.13, src/smtpd/smtpd_check.c,
// reject_auth_sender_login_mismatch(), takes the owners of the first key
// mail_addr_find() finds (src/global/mail_addr_find.c, mail_addr_find_opt()
// with MA_FIND_DEFAULT): the address, then the address without its
@@ -242,7 +242,7 @@ const extensionDelimiter = "+"
// follows its first delimiter. "user+tag@example.org" is "user@example.org";
// an address whose local part begins with the delimiter has no extension.
//
// Postfix 3.7.11, src/global/strip_addr.c, strip_addr_internal(), and
// Postfix 3.10.13, src/global/strip_addr.c, strip_addr_internal(), and
// split_addr.c, split_addr_internal(): the local part, up to the last "@", is
// cut at its first delimiter, unless that is its first character.
func withoutExtension(addr string) string {
@@ -322,7 +322,7 @@ func (s *senderSession) Header(name, value string, m *milter.Modifier) (milter.R
// Headers is the end of the headers, where the message is judged.
//
// go-milter v0.4.1, session.go, case CodeEOH: Headers(). Postfix 3.7.11,
// go-milter v0.4.1, session.go, case CodeEOH: Headers(). Postfix 3.10.13,
// src/milter/milter8.c, milter8_event(), SMFIR_REPLYCODE: a "4"/"5" reply
// whose enhanced code starts with the same digit is the message's reply;
// src/cleanup/cleanup_milter.c, cleanup_milter_apply(): a "5" rejects the
@@ -347,7 +347,7 @@ func (s *senderSession) Headers(h textproto.MIMEHeader, m *milter.Modifier) (mil
// cleanup has given the message a queue id by the end of its headers
// (milter_end_of_header_macros has {i}); Postfix's lines about it - the
// refusal among them - are found by it on the message's page.
// Postfix 3.7.11, src/global/mail_params.h: DEF_MILT_EOH_MACROS "i",
// Postfix 3.10.13, src/global/mail_params.h: DEF_MILT_EOH_MACROS "i",
// answered by src/cleanup/cleanup_milter.c, cleanup_milter_eval().
s.record(shown, macro(m, "i"))
// A reply code of its own, so that the application is told what is wrong
@@ -367,7 +367,7 @@ func (s *senderSession) judge() (shown, why string, err error) {
if s.login == "" {
// Not reachable while the submission ports demand authentication; a
// message nobody answers for is not signed either way.
// Postfix 3.7.11, src/smtpd/smtpd_check.c, smtpd_check_rcpt(): with
// Postfix 3.10.13, src/smtpd/smtpd_check.c, smtpd_check_rcpt(): with
// smtpd_delay_reject (default yes) smtpd_client_restrictions
// (permit_sasl_authenticated,reject on 465 and 587) refuse every RCPT;
// src/smtpd/smtpd.c, data_cmd(): no valid recipients, no message.
@@ -416,7 +416,7 @@ func (s *senderSession) judge() (shown, why string, err error) {
// line must also be one address in RFC 5322's grammar (net/mail), at the
// domain OpenDKIM signs for; a quoted display name may say anything.
// And that address, its local part unquoted as Postfix holds the
// envelope's (Postfix 3.7.11, src/global/tok822_parse.c), is the one
// envelope's (Postfix 3.10.13, src/global/tok822_parse.c), is the one
// held against the bindings: "x"@mine.com is x@mine.com, which may be
// another application's (review of stage 2c, F1, F2).
list, perr := fromParser.ParseList(unfold(raw))
@@ -504,7 +504,7 @@ func (s *senderSession) record(shown, queueID string) {
// holds against nobody, the domain the login is bound to. Without it the row
// would belong to no domain and its administrator would never see it.
//
// Postfix 3.7.11, src/smtpd/smtpd_check.c, generic_checks():
// Postfix 3.10.13, src/smtpd/smtpd_check.c, generic_checks():
// reject_*_sender_login_mismatch run only "if (state->sender &&
// *state->sender)".
func (s *senderSession) domain() string {
@@ -518,7 +518,7 @@ func (s *senderSession) domain() string {
// characters that mean something in an SMTP reply or a log line, and cuts it:
// it is said back to the client, logged and stored.
//
// Postfix 3.7.11, src/milter/milter8.c, milter8_event(), SMFIR_REPLYCODE:
// Postfix 3.10.13, src/milter/milter8.c, milter8_event(), SMFIR_REPLYCODE:
// "%%" becomes "%" and a single "%" is dropped; src/cleanup/cleanup_milter.c,
// cleanup_milter_apply(), logs "<id>: milter-reject: END-OF-MESSAGE from
// <client>: <text>; from=<...> to=<...> ...".
@@ -573,7 +573,7 @@ func ServeSender(ctx context.Context, ln net.Listener, bind *Bindings, rec Refus
// The headers decide; and each header value with its leading space,
// as OpenDKIM asks for it (opendkim.c, mlfi_negotiate(),
// SMFIP_HDR_LEADSPC), so that both read the same bytes. Postfix
// 3.7.11, src/milter/milter8.c: milter8_negotiate() keeps the event
// 3.10.13, src/milter/milter8.c: milter8_negotiate() keeps the event
// mask a milter of any version asks for (go-milter v0.4.1 says
// version 2), and milter8_header() drops the space only for a milter
// without SMFIP_HDR_LEADSPC.
+1 -1
View File
@@ -230,7 +230,7 @@ func TestBindingsFollowTheFile(t *testing.T) {
// The owners of the first key Postfix finds decide, not any key that lists
// the login: an address bound to one application is not open to another that
// holds its whole domain (Postfix 3.7.11, smtpd_check.c,
// holds its whole domain (Postfix 3.10.13, smtpd_check.c,
// reject_auth_sender_login_mismatch(), over mail_addr_find(); assumptions
// B-PS1). Composed by hand in internal/postfix's format.
func TestBindingsFirstKeyWins(t *testing.T) {
+1 -1
View File
@@ -7,7 +7,7 @@
*
* test/upstream/mailparse-diff.sh <opendkim source dir>
*
* builds it in debian:bookworm and runs TestMailParseMatchesOpenDKIM with it.
* builds it in debian:trixie and runs TestMailParseMatchesOpenDKIM with it.
*/
#include <sys/types.h>
#include <stdio.h>
+1 -1
View File
@@ -41,7 +41,7 @@ func (p *Postfix) RebuildDMARCMaps(cfg DMARCMapsConfig) error {
return p.reload()
}
// Postfix 3.7.11, man/man5/access.5: "OK" accepts the address the key
// Postfix 3.10.13, man/man5/access.5: "OK" accepts the address the key
// matches; man/man5/transport.5: "user@domain transport:nexthop", and "a
// non-null transport field with a null nexthop field resets the nexthop
// information to the recipient domain"; relay_domains as in renderInboundMaps.
+1 -1
View File
@@ -8,7 +8,7 @@ import (
"time"
)
// ParseTime reads a Postfix time value as Postfix 3.7.11 reads it:
// ParseTime reads a Postfix time value as Postfix 3.10.13 reads it:
// src/global/conv_time.c, conv_time(), called by get_mail_conf_time() in
// src/global/mail_conf_time.c. One number - strtol(), so leading blanks and a
// "+" are taken, a negative number or one past an int is invalid - followed
+1 -1
View File
@@ -5,7 +5,7 @@ import (
"time"
)
// Postfix 3.7.11, src/global/conv_time.c, conv_time(): one number, one
// Postfix 3.10.13, src/global/conv_time.c, conv_time(): one number, one
// lower-case unit, the default unit when there is none, the rest ignored.
func TestParseTime(t *testing.T) {
cases := []struct {
+2 -2
View File
@@ -55,7 +55,7 @@ func (p *Postfix) RebuildInboundMaps(routes []InboundRoute) error {
return p.reload()
}
// Postfix 3.7.11, man/man5/postconf.5: a relay_domains table "is matched when
// Postfix 3.10.13, man/man5/postconf.5: a relay_domains table "is matched when
// a (parent) domain appears as lookup key", the value unused; relay_recipient_maps
// is used as a list, "@domain" its wild-card (src/global/mail_addr_find.c,
// MA_FIND_AT_DOMAIN, the exact domain only). man/man5/transport.5: "domain
@@ -104,7 +104,7 @@ func renderInboundMaps(routes []InboundRoute) (relay, transport, recipients, tls
// what disable the MX lookup, so the operator's explicit upstream is used as
// given; they also delimit a bare IPv6 address.
//
// Postfix 3.7.11, src/smtp/smtp_connect.c: no MX lookup for a destination that
// Postfix 3.10.13, src/smtp/smtp_connect.c: no MX lookup for a destination that
// starts with "["; src/util/host_port.c, host_port(): "[host]:port", the host
// a name or an address ("ipv6:" in front is optional), checked by
// valid_hostaddr().
+4 -4
View File
@@ -62,7 +62,7 @@ type Binding struct {
// a comma-separated list, which is how Postfix expects multiple owners of one
// sender.
//
// Postfix 3.7.11, src/smtpd/smtpd_check.c, reject_auth_sender_login_mismatch():
// Postfix 3.10.13, src/smtpd/smtpd_check.c, reject_auth_sender_login_mismatch():
// the value is split into owners with mystrtok(&cp, CHARS_COMMA_SP).
func (p *Postfix) RebuildSenderLoginMaps(bindings []Binding) error {
content, err := renderSenderLoginMaps(bindings)
@@ -105,7 +105,7 @@ func renderSenderLoginMaps(bindings []Binding) ([]byte, error) {
// One line per key as Postfix folds it: smtpd opens the map with
// DICT_FLAG_FOLD_FIX and no DICT_FLAG_DUP_* flag, so two keys that
// differ only in case would make the whole table unusable (Postfix
// 3.7.11, src/util/dict_thash.c, dict_thash_open()). The panel stores
// 3.10.13, src/util/dict_thash.c, dict_thash_open()). The panel stores
// addresses in lower case already; this keeps the file safe if a row
// ever is not.
addr := strings.ToLower(b.Address)
@@ -141,7 +141,7 @@ func appendUnique(list []string, v string) []string {
// and the backslash. A login additionally may not carry "@" — that would make it
// look like an address.
//
// Postfix 3.7.11, src/util/dict_thash.c, dict_thash_open(): the key ends at
// Postfix 3.10.13, src/util/dict_thash.c, dict_thash_open(): the key ends at
// the first blank outside quotes, a backslash escapes the next character;
// src/util/readlline.c: a newline ends the line (one that starts with a blank
// continues the one before); the owners split on comma and blank (see
@@ -180,7 +180,7 @@ func assertMapSafe(address, login string) error {
// Arguments are fixed literals — no user input is interpolated into the command,
// and it never goes through a shell (security.md).
//
// Postfix 3.7.11, conf/postfix-script, start-fg: a process that is not PID 1
// Postfix 3.10.13, conf/postfix-script, start-fg: a process that is not PID 1
// runs "$daemon_directory/master -s" as its child, and src/master/master.c
// puts that master in a session of its own (setsid()). "reload" sends the
// master SIGHUP; src/master/master.c calls master_refresh(), and
+3 -3
View File
@@ -51,7 +51,7 @@ var postqueueP = func(ctx context.Context) *exec.Cmd {
// listQueue runs `postqueue -p` and keeps at most maxQueueOutput bytes of it;
// truncated says whether there was more.
//
// Postfix 3.7.11, src/postqueue/postqueue.c, show_queue(): any user passes
// Postfix 3.10.13, src/postqueue/postqueue.c, show_queue(): any user passes
// authorized_mailq_users (DEF_SHOWQ_ACL static:anyone), and a failure is
// msg_fatal_status() - a non-zero exit with the reason on stderr;
// src/postqueue/showq_compat.c, showq_compat(): "Mail queue is empty".
@@ -107,7 +107,7 @@ func Queue() (string, error) {
// cut are still in the queue, and a backlog that large is the outage the sweep
// exists for (audit 2026-10-12, H3). QueueMessages treats its cap the same way.
//
// Postfix 3.7.11, src/showq/showq.c, showq_service(): maildrop, active,
// Postfix 3.10.13, src/showq/showq.c, showq_service(): maildrop, active,
// incoming, deferred and hold are the queues it lists.
func QueueIDs() (map[string]struct{}, error) {
out, truncated, err := listQueue()
@@ -126,7 +126,7 @@ func QueueIDs() (map[string]struct{}, error) {
//
// The id is at the start of the line, optionally flagged '*' (in the active
// queue) or '!' (on hold), then '#' when the message was forced to expire
// (Postfix 3.7.11, src/postqueue/showq_compat.c, showq_message(): the flags
// (Postfix 3.10.13, src/postqueue/showq_compat.c, showq_message(): the flags
// are appended to the id in that order; assumptions B-PQ1), and is followed
// by the message size. Requiring the
// size is what separates an entry from the listing's other left-margin lines:
+2 -2
View File
@@ -12,7 +12,7 @@ import (
// TestFakePostqueue is not a test: it is the fake `postqueue -p` the tests
// below run in place of the real one, writing SP_FAKE_POSTQUEUE_BYTES bytes of
// queue entries composed by hand after the layout of Postfix 3.7.11's
// queue entries composed by hand after the layout of Postfix 3.10.13's
// src/postqueue/showq_compat.c.
func TestFakePostqueue(t *testing.T) {
if os.Getenv("SP_FAKE_POSTQUEUE") != "1" {
@@ -76,7 +76,7 @@ func TestQueueIDsReadsAListingUnderTheCap(t *testing.T) {
// The reconcile sweep decides whether a message is still Postfix's problem, so
// the parser must pick queue ids out of a listing and nothing else out of it:
// not the header, not the byte-count trailer, and above all not a deferred
// entry's reason line. The listing is composed by hand. Postfix 3.7.11
// entry's reason line. The listing is composed by hand. Postfix 3.10.13
// (src/postqueue/showq_compat.c, showq_message()) right-aligns a reason to
// column 76, so it reaches the left margin only when it is 76 characters or
// longer; the one here is put at the margin as the hard case.
+1 -1
View File
@@ -13,7 +13,7 @@ import (
// QueuedMessage is one message Postfix is holding, as `postqueue -j` reports
// it: where it is, and since when.
//
// Postfix 3.7.11, src/postqueue/showq_json.c, format_json(): one object and a
// Postfix 3.10.13, src/postqueue/showq_json.c, format_json(): one object and a
// newline per message - "queue_name", "queue_id", "arrival_time" (seconds,
// %ld), "message_size", "forced_expire", "sender", "recipients" [{"address",
// "delay_reason"}]; the queue names are those of src/global/mail_queue.h that
+1 -1
View File
@@ -7,7 +7,7 @@ import (
)
// The listing is composed by hand: its first line in the field order of
// Postfix 3.7.11's src/postqueue/showq_json.c, format_json(); the others
// Postfix 3.10.13's src/postqueue/showq_json.c, format_json(); the others
// short of fields or broken on purpose.
func TestParseQueueJSON(t *testing.T) {
out := []byte(`{"queue_name": "deferred", "queue_id": "41E862C00D9E", "arrival_time": 1791000000, "message_size": 446, "forced_expire": false, "sender": "app@example.com", "recipients": [{"address": "a@example.net", "delay_reason": "connect to mx.example.net[203.0.113.9]:25: Connection timed out"}]}
+33 -19
View File
@@ -29,7 +29,7 @@ type RetryPolicy struct {
// build/postfix-config.sh does not set. Used when postconf is missing (unit
// tests, a binary outside the container) so the panel still starts.
//
// Postfix 3.7.11, src/global/mail_params.h: DEF_QUEUE_RUN_DELAY "300s",
// Postfix 3.10.13, src/global/mail_params.h: DEF_QUEUE_RUN_DELAY "300s",
// DEF_MIN_BACKOFF_TIME DEF_QUEUE_RUN_DELAY, DEF_MAX_BACKOFF_TIME "4000s",
// DEF_MAX_QUEUE_TIME "5d", DEF_DSN_QUEUE_TIME "5d", DEF_DELAY_WARN_TIME "0h".
func DefaultRetryPolicy() RetryPolicy {
@@ -56,7 +56,7 @@ var retryConfKeys = []string{
}
// retryConfUnits is each key's default unit: the last letter of its default
// value (Postfix 3.7.11, src/global/mail_params.h, DEF_QUEUE_RUN_DELAY
// value (Postfix 3.10.13, src/global/mail_params.h, DEF_QUEUE_RUN_DELAY
// "300s", DEF_MIN_BACKOFF_TIME, DEF_MAX_BACKOFF_TIME "4000s",
// DEF_MAX_QUEUE_TIME "5d", DEF_DSN_QUEUE_TIME "5d", DEF_DELAY_WARN_TIME
// "0h"; src/global/mail_conf_time.c, get_def_time_unit()).
@@ -66,23 +66,37 @@ var retryConfUnits = []byte{'s', 's', 's', 'd', 'd', 'h'}
// the same way logtail stubs queueIDs.
var readRetryConf = postconfRetryValues
func postconfRetryValues() ([]string, error) {
// postconfRetryCmd builds the `postconf -h -x` command. It is a var so tests
// can put a fake postconf in its place, the way postqueueP is.
var postconfRetryCmd = func() *exec.Cmd {
// Fixed argv, no user input — same pattern as Queue (security.md).
// Postfix 3.7.11, src/postconf/postconf_main.c, pcf_show_parameters():
// named parameters are printed in argument order, with -h one "%s\n"
// value each; -x expands "$name" as Postfix itself does when it reads
// the value (mail_conf_eval()).
cmd := exec.Command("postconf", "-h", "-x",
"queue_run_delay",
"minimal_backoff_time",
"maximal_backoff_time",
"maximal_queue_lifetime",
"bounce_queue_lifetime",
"delay_warning_time",
)
out, err := cmd.CombinedOutput()
return exec.Command("postconf", append([]string{"-h", "-x"}, retryConfKeys...)...)
}
// postconfRetryValues reads the values from postconf's standard output only.
// Postfix 3.10.13, src/postconf/postconf_main.c, pcf_show_parameters():
// named parameters are printed to stdout in argument order, with -h one
// "%s\n" value each, and an unknown one is a msg_warn() with no line;
// -x expands "$name" (pcf_expand_parameter_value(),
// src/postconf/postconf_lookup.c) as Postfix itself does when it reads the
// value (mail_conf_eval()). Warnings go to stderr (src/postconf/postconf.c,
// main(), msg_vstream_init(argv[0], VSTREAM_ERR)): an unknown name, and after
// the values (src/postconf/postconf.c, main()) an unused
// main.cf parameter, and since 3.9 a deprecated one, "support for parameter
// \"%s\" will be removed; instead, %s" (src/postconf/postconf_unused.c,
// pcf_flag_unused_parameters(); RELEASE_NOTES-3.9). Read together with stdout,
// as CombinedOutput did until base-trixie, such a warning was one value too
// many and the panel fell back to the compiled-in defaults.
func postconfRetryValues() ([]string, error) {
cmd := postconfRetryCmd()
var errOut strings.Builder
cmd.Stderr = &errOut
out, err := cmd.Output()
if err != nil {
return nil, fmt.Errorf("postconf -h: %w: %s", err, strings.TrimSpace(string(out)))
return nil, fmt.Errorf("postconf -h: %w: %s", err, strings.TrimSpace(errOut.String()))
}
if why := strings.TrimSpace(errOut.String()); why != "" {
log.Printf("postfix: postconf -h: %s", why)
}
lines := strings.Split(strings.TrimSpace(string(out)), "\n")
if len(lines) != len(retryConfKeys) {
@@ -123,7 +137,7 @@ func parseRetryPolicy(lines []string) (RetryPolicy, error) {
}
durs[i] = d
}
// Postfix 3.7.11, src/qmgr/qmgr.c, qmgr_post_init(): "bounce_queue_lifetime
// Postfix 3.10.13, src/qmgr/qmgr.c, qmgr_post_init(): "bounce_queue_lifetime
// is larger than maximal_queue_lifetime - adjusting" - qmgr uses the
// smaller.
if durs[4] > durs[3] {
@@ -155,7 +169,7 @@ func (p RetryPolicy) BackoffCap() string {
// sender, which an application may submit too. It bounds which stored log
// lines can be about a send-log row (store.SendLogLines).
//
// Postfix 3.7.11, src/qmgr/qmgr_active.c, qmgr_active_done_25_generic(): a
// Postfix 3.10.13, src/qmgr/qmgr_active.c, qmgr_active_done_25_generic(): a
// message expires after var_max_queue_time, or var_dsn_queue_time when its
// sender is null - whoever submitted it; qmgr.c caps the second at the
// first, and so does parseRetryPolicy, which leaves the first the longer.
+44 -3
View File
@@ -2,12 +2,15 @@ package postfix
import (
"errors"
"fmt"
"os"
"os/exec"
"testing"
"time"
)
// The postconf values in these tests are composed by hand: values main.cf
// could hold. Stock Postfix 3.7.11 prints "0h" for delay_warning_time
// could hold. Stock Postfix 3.10.13 prints "0h" for delay_warning_time
// (src/global/mail_params.h, DEF_DELAY_WARN_TIME), not "0".
func TestLoadRetryPolicyUsesPostconfValues(t *testing.T) {
old := readRetryConf
@@ -29,7 +32,7 @@ func TestLoadRetryPolicyUsesPostconfValues(t *testing.T) {
if p.MaximalQueueLifetime != 2*24*time.Hour {
t.Errorf("MaximalQueueLifetime = %v, want 2d", p.MaximalQueueLifetime)
}
// 5d over a maximal 2d: qmgr uses 2d (Postfix 3.7.11, src/qmgr/qmgr.c,
// 5d over a maximal 2d: qmgr uses 2d (Postfix 3.10.13, src/qmgr/qmgr.c,
// qmgr_post_init(), "adjusting bounce_queue_lifetime").
if p.BounceQueueLifetime != 2*24*time.Hour {
t.Errorf("BounceQueueLifetime = %v, want 2d", p.BounceQueueLifetime)
@@ -40,7 +43,7 @@ func TestLoadRetryPolicyUsesPostconfValues(t *testing.T) {
}
// A message lives at most maximal_queue_lifetime: qmgr caps
// bounce_queue_lifetime at it (Postfix 3.7.11, src/qmgr/qmgr.c,
// bounce_queue_lifetime at it (Postfix 3.10.13, src/qmgr/qmgr.c,
// qmgr_post_init()), so a larger bounce lifetime does not lengthen it.
func TestLongestLifetimeIsTheQueueLifetime(t *testing.T) {
p, err := parseRetryPolicy([]string{"300s", "300s", "4000s", "1d", "5d", "0h"})
@@ -95,3 +98,41 @@ func TestParseRetryPolicyStock(t *testing.T) {
t.Errorf("parsed = %+v", p)
}
}
// TestFakePostconf is not a test: it is the fake `postconf -h -x` the test
// below runs in place of the real one. Its warning is composed after Postfix
// 3.10.13, src/postconf/postconf_unused.c, pcf_flag_unused_parameters(),
// written to stderr after the values (src/postconf/postconf.c, main()).
func TestFakePostconf(t *testing.T) {
if os.Getenv("SP_FAKE_POSTCONF") != "1" {
t.Skip("helper process")
}
fmt.Fprint(os.Stdout, "300s\n300s\n4000s\n2d\n5d\n0h\n")
fmt.Fprint(os.Stderr, "postconf: warning: /etc/postfix/main.cf: support for parameter \"smtpd_use_tls\" will be removed; instead, specify \"smtpd_tls_security_level\"\n")
os.Exit(0)
}
// postconf's warnings are not values: since Postfix 3.9 a deprecated
// main.cf parameter draws one on every run, and read with the values it made
// one line too many, so the panel showed the compiled-in defaults.
func TestPostconfRetryValuesReadsStdoutOnly(t *testing.T) {
old := postconfRetryCmd
postconfRetryCmd = func() *exec.Cmd {
cmd := exec.Command(os.Args[0], "-test.run=^TestFakePostconf$")
cmd.Env = append(os.Environ(), "SP_FAKE_POSTCONF=1")
return cmd
}
t.Cleanup(func() { postconfRetryCmd = old })
lines, err := postconfRetryValues()
if err != nil {
t.Fatalf("postconfRetryValues: %v", err)
}
p, err := parseRetryPolicy(lines)
if err != nil {
t.Fatalf("parse %q: %v", lines, err)
}
if p.MaximalQueueLifetime != 2*24*time.Hour || p.FromDefaults {
t.Errorf("policy = %+v, want the postconf values", p)
}
}
+1 -1
View File
@@ -13,7 +13,7 @@ import (
// login is globally unique because it is the SASL identity Postfix
// authenticates (one sasldb2 across all domains, architecture.md § Mail path).
// It is unique without regard to case too (migration 0008's NOCASE index):
// Postfix 3.7.11, src/smtpd/smtpd_check.c reject_auth_sender_login_mismatch()
// Postfix 3.10.13, src/smtpd/smtpd_check.c reject_auth_sender_login_mismatch()
// compares the SASL login with the map's owners by strcasecmp_utf8.
var ErrLoginExists = errors.New("application login already exists")
+2 -2
View File
@@ -100,7 +100,7 @@ func (s *Store) InsertRejected(e SendLogEntry) error {
// wrote, and the milter's row holds it as Postfix logs it
// (milter.recipientAsLogged); but a recipient repeated in another case is
// one recipient to Postfix - cleanup drops the second, comparing folded
// (Postfix 3.7.11, src/cleanup/cleanup_out_recipient.c, been_here() with
// (Postfix 3.10.13, src/cleanup/cleanup_out_recipient.c, been_here() with
// BH_FLAG_FOLD) - and one row to the milter, spelt as first given, and
// log_adhoc() writes orig_to= only when it differs from to= other than in
// case (src/global/log_adhoc.c, strcasecmp_utf8()). Assumptions D-S20. It
@@ -191,7 +191,7 @@ func (s *Store) CloseUnreported(id int64) (int64, error) {
// ExpireMessage closes the rows of a message Postfix gave up on at the end of
// its queue lifetime: each recipient's newest row with that queue id, if it
// is still open ("queued" or "deferred"), becomes "bounced" - Postfix returns
// the message to its sender. Postfix 3.7.11, src/qmgr/qmgr_active.c,
// the message to its sender. Postfix 3.10.13, src/qmgr/qmgr_active.c,
// qmgr_active_done_25_generic(), logs expiry once per message, "<id>:
// from=<..>, status=expired, returned to sender", with no line per
// recipient: the recipients' last lines say "deferred" (assumptions A-8). It
+1 -1
View File
@@ -20,7 +20,7 @@ const socketPath = "/run/supervisor.sock"
// runOnceFor bounds RunOnce: a reload that is still running after it is
// reported as a failure. "postfix reload" checks that master runs, moves the
// incoming queue to active and sends master SIGHUP (Postfix 3.7.11,
// incoming queue to active and sends master SIGHUP (Postfix 3.10.13,
// conf/postfix-script, "reload") - well under a second on the stand.
var runOnceFor = time.Minute
+1 -1
View File
@@ -354,7 +354,7 @@ func decryptErrorMessage(err error) string {
func unreadableBackupErr(err error, deployRoot string) string {
if name, ok := backup.UnreadableCert(err, deployRoot); ok {
return "The panel is not allowed to read " + name + ", and a backup without it would not bring the server back. " +
"On the host, give the file to group 998 and make it group-readable (chgrp 998, chmod 640), then try again. Nothing was downloaded."
"On the host, give the file to group 10998 and make it group-readable (chgrp 10998, chmod 640), then try again. Nothing was downloaded."
}
if errors.Is(err, backup.ErrNoCerts) {
return "The project's certs/ holds no certificate, so a backup would bring the server back without one. " +
+1 -1
View File
@@ -232,7 +232,7 @@ var serverTopics = []HelpTopic{
". The Postfix queue is not in it. The backup is taken while the server is running, and Postfix keeps moving messages through the queue, "+
"so a copy could hold a message twice or miss it; mail still waiting for delivery is therefore not carried over, and a restored server "+
"starts with an empty queue. To keep the queue, stop the container and archive the project directory instead. The private key in ", Code("certs/"),
" has to be readable by the panel: give it to group 998 on the host (", Code("chgrp 998"), ", ", Code("chmod 640"),
" has to be readable by the panel: give it to group 10998 on the host (", Code("chgrp 10998"), ", ", Code("chmod 640"),
"). Until then the panel refuses to make a backup and names the file, rather than hand over an archive without the key. It holds secrets, so leave Encrypt with a password ticked: the file is sealed as ", Code(".spbk"),
", the password is not stored, and without it nothing can be recovered. To restore, unpack it into an empty project directory (decrypt a ",
Code(".spbk"), " first with ", Code("selfpost-backup -decrypt"),
+1 -1
View File
@@ -289,7 +289,7 @@ func NewOutMessage(m Meta, in MessageInput) *OutMessage {
}
if in.Recovered {
// cleanup's line names the message's last recipient only (Postfix
// 3.7.11, src/cleanup/cleanup_milter.c, cleanup_milter_apply(),
// 3.10.13, src/cleanup/cleanup_milter.c, cleanup_milter_apply(),
// " to=<%s>" of state->recip; assumptions A-15).
facts = append(facts, Fact{Label: "Written", Value: Rich("from ", Code("mail.log"), " after a restart: the panel stopped before it wrote the refusal. Postfix's line names the last recipient only")})
}