diff --git a/CHANGELOG.md b/CHANGELOG.md index d0329f7..9dcc8d6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,81 @@ Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); version ## [Unreleased] +### Security + +- **The image runs Debian trixie instead of bookworm** (base-trixie). + Bookworm left regular security support on 2026-07-12, and its Postfix + 3.7.11 lacks upstream's 2026 security fixes, which carry no CVE ids and + so will not reach bookworm LTS. Trixie brings Postfix 3.10.13 with the + August batch (among them a BDAT memory exhaustion, a crash by DATA after + a BDAT error, and a policy and milter bypass after an end-of-data + reject), Debian's security support until 2028, OpenSSL 3.5 and libmilter + 8.18.1. OpenDKIM's code is unchanged. The build stage is + `golang:1.26-trixie`. + +### Changed + +- **Every listener holds clients to two SMTP rules,** by Postfix 3.10's + defaults: DATA ends only at `.` + (`smtpd_forbid_bare_newline=normalize`, now on 465 and 587 too), and a + client that sends commands ahead of the replies where RFC 2920 does not + allow it gets `554 5.5.0 Error: SMTP protocol synchronization` and is + disconnected (`smtpd_forbid_unauth_pipelining=yes`). An application whose + mail library breaks either rule stops sending; the guide says so under + Ports. +- **The code's citations of other programs name the trixie sources:** + every place that relies on Postfix, Cyrus SASL, supervisor, logrotate or + libmilter was re-read against the trixie package and its comment + updated; the cited-package table in development.md and the e2e check of + it hold the trixie versions. +- **Postfix's own start, stop and reload messages reach `docker logs`** + as well as mail.log: since 3.8 postfix(1) writes to stderr even when it + is not a terminal. +- **The SASL configuration's directory is set by the image:** + bookworm's Postfix found `/etc/postfix/sasl/smtpd.conf` through a Debian + patch that trixie dropped; trixie's package writes + `cyrus_sasl_config_path=/etc/postfix/sasl` into its main.cf instead. The + image now sets it itself, so the login on 465 and 587 does not hang on + the package's file. +- **The `Received:` header no longer shows where an application runs.** + On 465 and 587 Postfix now writes the short form, `Received: by + (Postfix) id for ; `, without the + application's address, host name, HELO name, TLS details or login + (`smtpd_hide_client_session=yes`, new in Postfix 3.10; the owner's + decision). The queue id still leads to the message in *Outbound › Log* + and to the client address in mail.log. Port 25 keeps the full header, as + RFC 5321 requires; e2e checks both. +- **The container's `selfpost` group is 10998, no longer 998.** 998 lies + in the range the distribution hands out to system groups (100-999): in + the trixie image a package took it and the build failed, and on a host it + is often a system group of the host's own, which could then read the key + as well (on the test server's host it is `systemd-network`). 10998 is + outside that range. **On upgrade, give the private key in `certs/` to + the new group on the host** - `chgrp 10998` where the guide said + `chgrp 998`, in a certbot deploy hook too - or a full backup is refused + until then; Postfix reads its own copy and is not affected. The other + accounts keep the numbers their packages give them, and the entrypoint + hands everything under `/data` back to them by name on every start. +- `opendkim-tools` is no longer installed: nothing in the image ran it. + `openssl` is installed by name, for the certificate reload. + +### Fixed + +- **The retry policy shown in the panel could fall back to Postfix's + compiled-in defaults:** the panel read `postconf`'s warnings together + with its values, and since Postfix 3.9 `postconf` warns about deprecated + parameters. It reads the values alone now and logs the warning. +- **logrotate 3.22 exits 1 when it skips a config it does not trust;** + 3.21 exited 0. The rotation loop now refuses to run on that output + whatever the exit code, instead of retrying for ever. +- **The log tailer reads Postfix 3.10's new line shapes:** refusal lines + that carry `sasl_method=`, `sasl_username=` and `sasl_sender=`, and + cleanup's `removed (canceled)` for a transaction that did not complete. +- Debian's chroot sync no longer copies `/etc/resolv.conf` and the like + into the queue under `/data` at every start (`SYNC_CHROOT=` in + `/etc/default/postfix`; nothing runs chrooted). A Postfix-created + mail.log is `0640`, as the entrypoint and logrotate create it. + ## [2.6.1] - 2026-10-15 The inbound relay's port 25 ends a message only at the standard diff --git a/build/Dockerfile b/build/Dockerfile index 60cc1eb..c535ef4 100644 --- a/build/Dockerfile +++ b/build/Dockerfile @@ -6,7 +6,7 @@ # docker build -f build/Dockerfile -t selfpost:dev --build-arg VERSION=dev . # ---- build stage ------------------------------------------------------------- -FROM golang:1.26-bookworm AS build +FROM golang:1.26-trixie AS build WORKDIR /src @@ -30,61 +30,95 @@ RUN go vet ./... \ -o /out/selfpost-backup ./cmd/selfpost-backup # ---- runtime stage ----------------------------------------------------------- -FROM debian:bookworm-slim AS runtime +FROM debian:trixie-slim AS runtime ENV DEBIAN_FRONTEND=noninteractive # Preseed Postfix so its install is non-interactive and yields a working # main.cf. The real relay configuration is generated by the panel. -# (postfix 3.7.11-0+deb12u1, debian/templates: postfix/main_mailer_type and -# postfix/mailname are the questions the install asks.) -# What else arrives through Depends, and is used here: openssl, by -# ca-certificates ("Depends: openssl (>= 1.1.1)", packages.debian.org/bookworm/ -# ca-certificates) - postfix-cert-reload.sh; python3, by supervisor (supervisor -# 4.2.5-1, debian/control, ${python3:Depends}) - crashexit.py. sasl2-bin brings -# saslpasswd2 and libsasl2-modules the PLAIN and LOGIN plugins; the sasldb -# auxprop plugin is in libsasl2-modules-db, a Depends of libsasl2-2 -# (cyrus-sasl2 2.1.28+dfsg-10, debian/control and debian/*.install). db-util -# ships /usr/bin/db_dump (packages.debian.org/bookworm/amd64/db-util/filelist). +# (postfix 3.10.13-0+deb13u1, debian/templates:14 and :50: +# postfix/main_mailer_type and postfix/mailname are the questions the install +# asks; debian/postfix.postinst:156-171 writes main.cf for any answer but "No +# configuration".) +# openssl is named because postfix-cert-reload.sh runs it, not left to arrive +# with ca-certificates. python3 arrives with supervisor (supervisor 4.2.5-3, +# debian/control:29, ${python3:Depends}) - crashexit.py. sasl2-bin brings +# saslpasswd2 (cyrus-sasl2 2.1.28+dfsg1-9, debian/rules:207, +# debian/sasl2-bin.install:3, usr/sbin) and depends on db-util +# (debian/control:36); libsasl2-modules brings the PLAIN and LOGIN plugins +# (debian/libsasl2-modules.install:4 and :6); the sasldb auxprop plugin is in +# libsasl2-modules-db (debian/libsasl2-modules-db.install:1), a Depends of +# libsasl2-2 (debian/control:65). db-util is named for /usr/bin/db_dump, which +# the panel runs (internal/app/sasl.go); that db-util ships that path on +# trixie is not in a source read here - the image build and the e2e sasldb +# test say whether it does. +# opendkim-tools is not installed: nothing in the image runs its programs +# (DKIM keys are made in Go, internal/domain/dkim.go). +# systemd-standalone-sysusers is named so that cron-daemon-common, which +# logrotate brings (logrotate 3.22.0-1, debian/control: "Depends: cron | +# anacron | cron-daemon | systemd-sysv"), creates its group with it rather +# than with the whole of systemd, the first alternative of its "Depends: +# systemd | systemd-standalone-sysusers | systemd-sysusers" (dpkg -s in the +# image, 2026-10-15). Nothing here runs systemd. RUN echo "postfix postfix/mailname string localhost" | debconf-set-selections \ && echo "postfix postfix/main_mailer_type string Internet Site" | debconf-set-selections \ && apt-get update \ && apt-get install -y --no-install-recommends \ postfix \ opendkim \ - opendkim-tools \ sasl2-bin \ libsasl2-modules \ db-util \ supervisor \ logrotate \ ca-certificates \ + openssl \ curl \ + systemd-standalone-sysusers \ && rm -rf /var/lib/apt/lists/* -# Unprivileged user for the panel process. +# Debian's postfix runs its chroot sync on every "postfix check" and "postfix +# start-fg" (postfix 3.10.13-0+deb13u1, debian/patches/ +# debian-run-configure-instance-from-create-missing.patch); with no service +# chrooted (build/postfix-config.sh) it would only copy /etc/resolv.conf and +# the like into the queue under /data on each start. SYNC_CHROOT empty turns +# it off (debian/configure-instance.in:22-24 and :44; debian/README.Debian:136-138). +RUN printf 'SYNC_CHROOT=\n' > /etc/default/postfix + +# Unprivileged user for the panel process. Its ids are left to useradd: the +# entrypoint gives /data back to it by name on every start, so a number +# that differs from one image to the next changes nothing. RUN useradd --system --no-create-home --shell /usr/sbin/nologin panel # Shared group bridging the unprivileged services: the panel generates # per-domain DKIM keys, application SASL accounts (sasldb2) and the Postfix # sender map, while OpenDKIM and Postfix (different users) must read them. -# Membership in this group — plus setgid dirs under /data (set up in -# entrypoint.sh) — lets OpenDKIM read the panel-owned keys and lets Postfix read -# the sasldb2/sender map. (The supervisor control socket has a group of its -# own, `supervisor`, below.) -# The id is fixed, and it is the one the group has always had: an operator -# gives the TLS private key on the host to this group by number, so that the -# panel can put it into a full backup (guide § Taking a full backup). -# groupadd(8) refuses a --gid already in use, so a package that took 998 would +# Membership in this group - plus setgid dirs under /data (set up in +# entrypoint.sh) - lets OpenDKIM read the panel-owned keys and lets Postfix +# read the sasldb2/sender map. (The supervisor control socket has a group of +# its own, `supervisor`, below.) +# The id is fixed, because the host sees it: an operator gives the TLS +# private key on the host to this group by number, so that the panel can put +# it into a full backup (guide § Taking a full backup). It lies outside the +# range the distribution hands out to system groups, 100-999 (adduser 3.152, +# /etc/adduser.conf: "Default: FIRST_SYSTEM_GID=100, LAST_SYSTEM_GID=999"; +# passwd 1:4.17.4-2, /etc/login.defs: SYS_GID_MIN 101, SYS_GID_MAX 999 - both +# read in the image, 2026-10-15), so no package can take it, and far above +# the 1000 a host starts its users' groups at, so a host is unlikely to have +# a group of that number that would read the key too. Until 2.7.0 it was +# 998, inside that range: in the trixie image systemd took 998 for +# systemd-network and the build failed, and on the test server's host 998 +# is systemd-network too (seen 2026-10-15). +# groupadd(8) refuses a --gid already in use, so an id taken after all would # fail this build rather than share the group. -RUN groupadd --system --gid 998 selfpost \ +RUN groupadd --system --gid 10998 selfpost \ && usermod -aG selfpost panel \ && usermod -aG selfpost opendkim \ && usermod -aG selfpost postfix # The panel reads the mail queue with `postqueue -p` (and the status page's # queue card). postqueue is setgid postdrop, which normally gives it the group -# needed to reach Postfix's showq socket — but the documented deployment runs +# needed to reach Postfix's showq socket - but the documented deployment runs # with `no-new-privileges`, which disables setgid transitions, so the panel # would always see "Permission denied". Making `panel` a real member of postdrop # grants the same access without relying on a setgid escalation the hardening @@ -93,8 +127,10 @@ RUN groupadd --system --gid 998 selfpost \ # to write the maildrop directory (internal/notify/mailer.go). It does not let # the panel bypass any Postfix restriction that a local user does not already # have through the world-executable sendmail. -# (Postfix 3.7.11, conf/postfix-files: postdrop and postqueue are -# root:$setgid_group 2755. Linux, Documentation/userspace-api/no_new_privs.rst: +# (Postfix 3.10.13, conf/postfix-files:140-141: postdrop and postqueue are +# root:$setgid_group 2755; Debian sets them root:postdrop 02555, postfix +# 3.10.13-0+deb13u1, debian/postfix.postinst:134-141. Linux, +# Documentation/userspace-api/no_new_privs.rst: # with no_new_privs, execve() no longer changes the gid for a setgid file.) RUN usermod -aG postdrop panel # The supervisord control socket's group (supervisord.conf, [unix_http_server]): diff --git a/build/entrypoint.sh b/build/entrypoint.sh index 969ccec..617c874 100644 --- a/build/entrypoint.sh +++ b/build/entrypoint.sh @@ -70,8 +70,13 @@ chmod 755 /data # under /data; make sure they stay writable without disturbing anything that a # later phase deliberately hands to another service. /data/log is exempt: it is # deliberately owned by postfix (postlogd writes the delivery log there) and is -# normalised on its own below. -find /data -mindepth 1 -maxdepth 1 ! -user panel ! -name log ! -name postfix -exec chown -R panel:panel {} + +# normalised on its own below, as are the trees shared through group selfpost. +# A file's owner is a number: an entry whose user or group is not the panel's +# by that number - unpacked by root, or written under another image whose +# useradd gave the panel other ids (bookworm's 999:999, until 2.7.0) - is +# handed over whole. +find /data -mindepth 1 -maxdepth 1 ! -name log ! -name postfix ! -name opendkim ! -name sasl ! -name antispam \ + \( ! -user panel -o ! -group panel \) -exec chown -R panel:panel {} + # DKIM key tree. The panel (user `panel`) generates keys and writes # the OpenDKIM tables; OpenDKIM (user `opendkim`) must read them. Normalise the @@ -133,7 +138,7 @@ chmod 2750 /data/postfix # so the directory and its files are readable by anyone: they hold sender # addresses and networks, no secrets. mkdir -p /data/antispam -chown panel:selfpost /data/antispam +chown -R panel:selfpost /data/antispam chmod 0755 /data/antispam # Delivery log (architecture.md § Log tailer). postlogd writes it as user @@ -141,17 +146,19 @@ chmod 0755 /data/antispam # lives under /data — not the ephemeral /var/log — so the delivery lines that # resolve a "queued" send-log row survive a container recreate. # -# postlogd creates a missing log itself, but at 0600, which the unprivileged -# panel cannot read; so create it here (and re-normalise an existing one, plus -# whatever logrotate left behind) at 0640 owned postfix:selfpost. The setgid +# postlogd creates a missing log itself, by default at 0600, which the +# unprivileged panel cannot read (postfix-config.sh now asks it for 0640, +# maillog_file_permissions); so create it here (and re-normalise an existing +# one, plus whatever logrotate left behind) at 0640 owned postfix:selfpost. The setgid # directory keeps the shared group on anything created inside it later, and # 2750 keeps it group-traversable but not group-writable — logrotate refuses to # rotate a log whose directory is writable by a non-root group. -# Postfix 3.7.11: src/util/logwriter.c, logwriter_open_or_die(), creates the -# file 0644, under master's umask(077) (src/master/master.c) - 0600. logrotate -# 3.21.0, logrotate.c, findNeedRotating(): "skipping ... because parent -# directory has insecure permissions" when the directory's group is not root -# and has write permission. +# Postfix 3.10.13: src/util/logwriter.c:99, logwriter_open_or_die(), creates the +# file with logwriter_perms, 0600 (:86) unless maillog_file_permissions says +# otherwise (proto/postconf.proto:18500, default 0600; Postfix 3.9 and later). +# logrotate 3.22.0, logrotate.c:1406-1408, findNeedRotating(): "skipping ... +# because parent directory has insecure permissions" when the directory's +# group is not root and has write permission. mkdir -p /data/log [ -e /data/log/mail.log ] || : > /data/log/mail.log chown -R postfix:selfpost /data/log @@ -199,8 +206,8 @@ ln -sfn /run/syslog/log /dev/log # The queue tree, on a first start or after a restore, is already built here: # postfix-config.sh ends with "postfix check", whose check-fatal runs -# "post-install create-missing" (Postfix 3.7.11, conf/postfix-script; -# conf/postfix-files, "$queue_directory/active:d:$mail_owner:-:700:ucr"). +# "post-install create-missing" (Postfix 3.10.13, conf/postfix-script:314; +# conf/postfix-files:55, "$queue_directory/active:d:$mail_owner:-:700:ucr"). # A "postfix set-permissions" for a missing active/ stood here and never ran # (assumptions C-P34). chown -R postfix:postfix /data/postfix/queue @@ -214,10 +221,10 @@ chgrp postdrop /data/postfix/queue/public /data/postfix/queue/maildrop # It would also leave the queue directory itself and pid/ owned by postfix. # Postfix keeps those two owned by root (master, which runs as root, is the only # writer) and logs "warning: not owned by root" for each at every start. -# (Postfix 3.7.11, conf/postfix-files: $queue_directory root 755, maildrop +# (Postfix 3.10.13, conf/postfix-files:51-68: $queue_directory root 755, maildrop # $setgid_group 730, public $setgid_group 710, pid root 755, the other queue -# directories $mail_owner 700; conf/postfix-script, "check-warn": "not owned by -# root".) +# directories $mail_owner 700; conf/postfix-script:354-356, "check-warn": "not +# owned by root".) chown root /data/postfix/queue /data/postfix/queue/pid # [program:postfix]'s startsecs (build/supervisord.conf): longer than the diff --git a/build/logrotate-loop.sh b/build/logrotate-loop.sh index d528e57..d5c3760 100644 --- a/build/logrotate-loop.sh +++ b/build/logrotate-loop.sh @@ -7,25 +7,29 @@ # before the reload is lost. `create` (rather than `nocreate`) matters here # beyond timing: a postlogd-triggered recreate lands the file at 0600 owned by # postfix, which the unprivileged panel process cannot read — confirmed on a -# live container (date not recorded), and so in the source: Postfix 3.7.11, -# src/util/logwriter.c, logwriter_open_or_die(), creates it 0644 under master's -# umask(077), src/master/master.c — so logrotate must be the one to create it. -# The reload: Postfix 3.7.11, conf/postfix-script, "logrotate" (mv, then kill -# -HUP master); master_conf.c, master_config() -> master_restart_service() -> -# master_status_cleanup(), on which postlogd exits (src/master/dgram_server.c, -# dgram_server_abort() on MASTER_STATUS_FD), and the next postlogd opens the -# path anew (src/postlogd/postlogd.c, logwriter_open_or_die()). +# live container (date not recorded, Postfix 3.7.11), and so in the source: +# Postfix 3.10.13, src/util/logwriter.c:99, logwriter_open_or_die(), creates it +# with logwriter_perms, 0600 (:86) unless maillog_file_permissions says +# otherwise (Postfix 3.9 and later). build/postfix-config.sh now sets that to +# 0640, so a file postlogd makes is readable too; logrotate still creates it, +# so that the new log is there, with its mode and owner, at once. +# The reload: Postfix 3.10.13, conf/postfix-script:535-566, "logrotate" (mv, +# then kill -HUP master); master_conf.c:92, master_config() -> :147, +# master_restart_service() -> master_status_cleanup(), on which postlogd exits +# (src/master/dgram_server.c:241, dgram_server_abort(), :640 on +# MASTER_STATUS_FD), and the next postlogd opens the path anew +# (src/postlogd/postlogd.c:236, logwriter_open_or_die()). # # logrotate itself only rotates once the configured "daily" period has elapsed # (tracked in its state file), so it is safe to invoke this more often than # daily — polling merely bounds how late a legitimate rotation runs. The state # file is beside the log, under /data, not at the default -# /var/lib/logrotate/status (logrotate 3.21.0-1: debian/rules, +# /var/lib/logrotate/status (logrotate 3.22.0-1: debian/rules:11, # --with-state-file-path) in the container's own filesystem: a recreated -# container started without it, logrotate.c, newState(), records a log it has -# no state for as rotated today, and so a host that recreates the container +# container started without it, logrotate.c:456, newState(), records a log it +# has no state for as rotated today, and so a host that recreates the container # daily - an .env change, an image bump - never rotated mail.log at all (audit -# 2026-10-12, M19). (logrotate 3.21.0, logrotate.8.in: "-s, --state +# 2026-10-12, M19). (logrotate 3.22.0, logrotate.8.in:74: "-s, --state # statefile".) The directory is postfix:selfpost 2750 and logrotate runs as # root. set -eu @@ -34,13 +38,18 @@ CONFIG=/etc/logrotate.d/mail STATE=/data/log/.logrotate.status INTERVAL="${LOGROTATE_INTERVAL_SECONDS:-21600}" -# logrotate refuses configs writable by group or others and exits 0 while -# ignoring them — fail here so supervisord reports the fault. (logrotate -# 3.21.0, config.c, readConfigFile(): as root, mode & 0022 gives "Ignoring ... -# because it is writable by group or others." and returns 0, not an error; a -# config not owned by root is ignored the same way. "Potentially dangerous -# mode" is only a warning there, for any mode & 07533 other than 0400 - this -# loop chooses to treat it as fatal too.) +# logrotate refuses configs writable by group or others and goes on without +# them — fail here so supervisord reports the fault. (logrotate 3.22.0, +# config.c:1099, readConfigFile(): as root, mode & 0022 gives (:1160-1165) +# "Ignoring ... because it is writable by group or others." and returns 1; a +# config not owned by root is ignored the same way (:1168-1173). Since 3.22.0 +# that makes logrotate's exit status 1 (logrotate.c:3278-3279 and :3301; +# ChangeLog.md, 3.22.0: "in case a configuration file was skipped due to +# unsafe permissions the exit status after rotattion will be `1` (#508)"); +# 3.21.0 returned 0 there and exited 0. So the output is looked at whatever +# the status. "Potentially dangerous mode" is only a warning there +# (:1154-1157), for any mode & 07533 other than 0400 - this loop chooses to +# treat it as fatal too.) logrotate_config_ok() { mode=$(stat -c '%a' "$CONFIG") mode=${mode#0} @@ -61,16 +70,19 @@ if ! logrotate_config_ok; then fi run_logrotate() { - out=$(logrotate -s "$STATE" "$CONFIG" 2>&1) || { - echo "$out" >&2 - return 1 - } + status=0 + out=$(logrotate -s "$STATE" "$CONFIG" 2>&1) || status=$? case "$out" in *Ignoring*|*Potentially\ dangerous\ mode*) echo "$out" >&2 - logrotate_config_fatal + echo "logrotate-loop: refusing to run: logrotate did not take $CONFIG as it is (above)" >&2 + exit 1 ;; esac + if [ "$status" -ne 0 ]; then + echo "$out" >&2 + return 1 + fi return 0 } diff --git a/build/logrotate-mail.conf b/build/logrotate-mail.conf index 3cdd5e5..f409f73 100644 --- a/build/logrotate-mail.conf +++ b/build/logrotate-mail.conf @@ -4,7 +4,7 @@ # build/postfix-config.sh (maillog_file) and build/entrypoint.sh: postlogd # writes it as user postfix, the unprivileged panel reads it through the shared # selfpost group, hence create 0640 postfix selfpost rather than 0644 root root. -# (logrotate 3.21.0, logrotate.8: "create mode owner group" makes the new log +# (logrotate 3.22.0, logrotate.8:402: "create mode owner group" makes the new log # right after the rotation; "delaycompress" leaves the previous log, .1, # uncompressed until the next cycle.) /data/log/mail.log { diff --git a/build/postfix-cert-reload.sh b/build/postfix-cert-reload.sh index c35d271..59a3236 100644 --- a/build/postfix-cert-reload.sh +++ b/build/postfix-cert-reload.sh @@ -11,9 +11,9 @@ # copies - not the mounted files, which a renewer may be rewriting - are a # certificate and the key that belongs to it, renames both over the targets # and reloads Postfix. When nothing changed it does nothing. -# (Postfix 3.7.11: smtpd loads the certificate in pre_jail_init(), once per -# process, src/smtpd/smtpd.c; "postfix reload" is SIGHUP to master, -# conf/postfix-script, after which the old smtpd processes exit on the closed +# (Postfix 3.10.13: smtpd loads the certificate in pre_jail_init(), once per +# process, src/smtpd/smtpd.c:6442; "postfix reload" is SIGHUP to master, +# conf/postfix-script:277, after which the old smtpd processes exit on the closed # status pipe, src/master/master_conf.c, master_config() -> # master_restart_service(), and new ones load the files again.) # diff --git a/build/postfix-config.sh b/build/postfix-config.sh index 1626ba9..c0dad4e 100644 --- a/build/postfix-config.sh +++ b/build/postfix-config.sh @@ -24,18 +24,19 @@ set -eu # passed here is written with commas only; a value with whitespace is a bug in # this script and stops the container. # -# Postfix 3.7.11, man/man5/master.5, "-o name=value (short form)", NOTE 2: no -# whitespace in a short-form value (the long form "-o { name = value }" would -# keep it; this script does not write it). src/postconf/postconf_edit.c, +# Postfix 3.10.13, man/man5/master.5:215-226, "-o name=value (short form)": +# "The "short form" does not support whitespace around the "=" or in +# parameter values" (the long form "-o { name = value }" would keep it; this +# script does not write it). src/postconf/postconf_edit.c:415, # pcf_edit_master(): "-Pe does not accept whitespace in parameter value". -# src/smtpd/smtpd_check.c, smtpd_check_parse(), and src/milter/milter.c, +# src/smtpd/smtpd_check.c:664, smtpd_check_parse(), and src/milter/milter.c:646, # milter_new(), split their lists on CHARS_COMMA_SP: commas or whitespace. # -# master.5, SYNTAX: a line that starts with whitespace continues a logical -# line, so the "-o" line awk inserts below the header belongs to the service. -# A later postconf -F/-P re-parses a matched entry with its continuation lines -# (postconf_edit.c, pcf_gobble_cf_line()) and prints it whole again, keeping -# these options. +# master.5:41-42, SYNTAX: a line that starts with whitespace continues a +# logical line, so the "-o" line awk inserts below the header belongs to the +# service. A later postconf -F/-P re-parses a matched entry with its +# continuation lines (postconf_edit.c:134, pcf_gobble_cf_line()) and prints +# it whole again, keeping these options. master_cf_set_smtp_inet_o() { param="$1" value="$2" @@ -75,7 +76,7 @@ master_cf_set_smtp_inet_o() { # sasldb2 accounts are looked up under. The panel creates accounts under realm # $SELFPOST_HOSTNAME (SASL_REALM), so myhostname MUST match or authentication # fails. Fall back to the container hostname only outside a real deployment. -# Postfix 3.7.11, src/xsasl/xsasl_cyrus_server.c, xsasl_cyrus_server_create(): +# Postfix 3.10.13, src/xsasl/xsasl_cyrus_server.c:339, xsasl_cyrus_server_create(): # sasl_server_new() gets var_myhostname as serverFQDN; Cyrus SASL 2.1.28, # plugins/sasldb.c, sasldb_auxprop_lookup(): with no user realm the account is # looked up under serverFQDN. @@ -83,17 +84,17 @@ HOSTNAME_VALUE="${SELFPOST_HOSTNAME:-$(hostname -f 2>/dev/null || hostname)}" # TLS material supplied by the reverse-proxy through a read-only bind mount. The # relay requires TLS on 465; if these files are absent the master still starts -# but TLS handshakes on 465 fail until they appear. Postfix 3.7.11, +# but TLS handshakes on 465 fail until they appear. Postfix 3.10.13, # src/smtpd/smtpd.c: pre_jail_init() only warns when the TLS context cannot be -# made, and smtpd_proto() then drops each wrapper-mode connection ("Wrapper-mode -# request dropped ... TLS context initialization failed"); every new smtpd -# process loads the files again in pre_jail_init(). +# made, and smtpd_proto() then drops each wrapper-mode connection (:5846, +# "Wrapper-mode request dropped ... TLS context initialization failed"); every +# new smtpd process loads the files again in pre_jail_init(). # # Postfix's TLS_README advises a private key owned by root and readable by no # other user; Postfix does not enforce it (smtpd loads the key as root in -# pre_jail_init(), Postfix 3.7.11 src/smtpd/smtpd.c), but the mount cannot +# pre_jail_init(), Postfix 3.10.13 src/smtpd/smtpd.c), but the mount cannot # give that mode - it is :ro and owned by the host's user, often readable by -# group 998 for the panel's backup. So the pair is copied into a directory of +# group 10998 for the panel's backup. So the pair is copied into a directory of # Postfix's own, the key root:root 0600, before postconf and `postfix check` # (assumptions D-arch:154). TLS_CERT_SRC="${TLS_CERT_FILE:-/etc/postfix/tls/fullchain.pem}" @@ -123,11 +124,13 @@ RATE_WINDOW="${RATE_LIMIT_WINDOW_SECONDS:-3600}" # first connection, or anvil and with it the rate limit (audit 2026-10-12, # L31; review of stage 5) - so the start stops here, saying which and why. # Counts and sizes are whole numbers from min up to 2147483647, an int's top -# (Postfix 3.7.11 reads them with get_mail_conf_int*, src/global/ -# mail_conf_int.c: an int, held to the parameter's own minimum). A time value +# (Postfix 3.10.13 reads the counts with get_mail_conf_int*, src/global/ +# mail_conf_int.c, check_mail_conf_int(): an int, held to the parameter's own minimum; +# message_size_limit is a long, src/global/mail_params.h:2014, and is held to +# the int's top here all the same). A time value # is a whole number from 1 with an optional unit, s m h d or w, at most # 2147483647 seconds -# (proto/postconf.proto: "Specify a non-zero time value (an integral value +# (proto/postconf.proto:266-268: "Specify a non-zero time value (an integral value # plus an optional one-letter suffix ...). Time units: s (seconds), m # (minutes), h (hours), d (days), w (weeks)"; one unit - "1h30m" is read as # 1h, src/global/conv_time.c - so it is refused here). @@ -184,8 +187,8 @@ JOURNAL_SOCK="${JOURNAL_MILTER_SOCKET:-/run/selfpost/journal.sock}" SENDER_SOCK="${SENDER_CHECK_SOCKET:-/run/selfpost/sender.sock}" # Persistent panel-managed sender map; texthash needs no postmap, so the -# unprivileged panel can rewrite it and just ask for a reload. Postfix 3.7.11, -# html/DATABASE_README.html, "texthash": no postmap(1), and the file is read +# unprivileged panel can rewrite it and just ask for a reload. Postfix 3.10.13, +# html/DATABASE_README.html:458-464, "texthash": no postmap(1), and the file is read # once by each process - a reload makes the new processes read it again. SENDER_LOGIN_MAPS="${POSTFIX_SENDER_LOGIN_MAPS:-/data/postfix/sender_login_maps}" # Transit mail queue under /data so deferred/active messages survive container @@ -239,25 +242,37 @@ postconf -PX "cleanup/unix/message_size_limit" 2>/dev/null || true # maillog_file lives under the persistent /data bind mount (architecture.md § # Log tailer). Postfix's default maillog_file_prefixes are only /var and # /dev/stdout — without /data, `postfix check` rejects the path. -# inet_protocols=all in a bridge network without IPv6: Postfix 3.7.11, -# src/util/inet_proto.c, inet_proto_init(), keeps IPv6 when socket(PF_INET6) -# succeeds, and src/smtp/smtp_connect.c, smtp_connect_inet(), goes on to -# the next address when a connection fails - an AAAA that cannot be reached +# inet_protocols=all in a bridge network without IPv6: Postfix 3.10.13, +# src/util/inet_proto.c:200, inet_proto_init(), keeps IPv6 when socket(PF_INET6) +# succeeds, and src/smtp/smtp_connect.c:1102-1142, smtp_connect_inet(), goes on +# to the next address when a connection fails - an AAAA that cannot be reached # costs one failed attempt, then IPv4. Postfix -# 3.7.11, src/global/mail_params.h, DEF_MAILLOG_FILE_PFXS "/var, /dev/stdout"; -# src/global/maillog_client.c, maillog_client_init(): fatal "does not match any +# 3.10.13, src/global/mail_params.h:4406, DEF_MAILLOG_FILE_PFXS "/var, /dev/stdout"; +# src/global/maillog_client.c:232, maillog_client_init(): fatal "does not match any # prefix", in every Postfix program that logs, postfix(1) itself included. +# +# maillog_file_permissions=0640: the log is the panel's to read, through +# group selfpost (the setgid /data/log, build/entrypoint.sh). entrypoint.sh +# and logrotate's "create" make it 0640 themselves; this is for a file that +# Postfix makes - one removed while the container runs - which would +# otherwise be 0600 and unreadable to the panel. (Postfix 3.10.13, +# src/global/mail_params.h:4418, DEF_MAILLOG_FILE_PERMS "0600"; +# src/global/mail_params.c:997, set_logwriter_create_perms(); src/util/ +# logwriter.c:143-148: "644", "640" or "600", leading zeros ignored, anything +# else a warning and 0600; :99, logwriter_open_or_die() creates with it; +# Postfix 3.9 and later, proto/postconf.proto:18500-18509.) postconf -e \ "myhostname=${HOSTNAME_VALUE}" \ "maillog_file=${MAIL_LOG_PATH}" \ "maillog_file_prefixes=/var,/dev/stdout,/data" \ + "maillog_file_permissions=0640" \ "mydestination=" \ "relayhost=" \ "inet_interfaces=all" \ "inet_protocols=all" -# postlogd is mandatory whenever maillog_file is set (Postfix 3.7.11, -# html/MAILLOG_README.html, "Configuring logging to file"; conf/postfix-script, +# postlogd is mandatory whenever maillog_file is set (Postfix 3.10.13, +# html/MAILLOG_README.html:42, "Configuring logging to file"; conf/postfix-script:333, # "check-fatal": "missing 'postlog' service in master.cf"). Debian's # stock master.cf usually has it; pin it explicitly so a stripped/upgraded # image cannot lose the service. @@ -266,10 +281,10 @@ postconf -M "postlog/unix-dgram=postlog unix-dgram n - n - 1 postlogd" # This is an outbound relay: no local delivery, no per-user aliases. Empty # these so a misfiled recipient never gets delivered locally. An empty # mydestination is not enough: an address literal of an interface the -# container listens on - x@[127.0.0.1] - is local too (Postfix 3.7.11, +# container listens on - x@[127.0.0.1] - is local too (Postfix 3.10.13, # src/global/resolve_local.c, resolve_local(), own_inet_addr()), so local(8) # itself is replaced by a refusal, as Postfix's null-client example does -# (README_FILES/STANDARD_CONFIGURATION_README, "local_transport = error:local +# (README_FILES/STANDARD_CONFIGURATION_README:209, "local_transport = error:local # mail delivery is disabled"; assumptions C-P8). postconf -e \ "local_recipient_maps=" \ @@ -283,9 +298,9 @@ postconf -e \ "smtp_tls_note_starttls_offer=yes" # TLS server material shared by every inbound service. auth_only guarantees -# credentials are never accepted before TLS is up on any port. Postfix 3.7.11, -# src/smtpd/smtpd_sasl_proto.c, smtpd_sasl_auth_cmd(): "504 5.5.4 Encryption -# required" while auth_only is set and TLS is not up; src/smtpd/smtpd.c, +# credentials are never accepted before TLS is up on any port. Postfix 3.10.13, +# src/smtpd/smtpd_sasl_proto.c:188, smtpd_sasl_auth_cmd(): "504 5.5.4 Encryption +# required" while auth_only is set and TLS is not up; src/smtpd/smtpd.c:6546-6547, # pre_jail_init(): wrappermode implies enforce, and enforce implies auth_only. postconf -e \ "smtpd_tls_cert_file=${TLS_CERT}" \ @@ -297,12 +312,13 @@ postconf -e \ # TLS 1.2 at least, opportunistic or required - and the client's too where an # inbound domain requires TLS to its upstream (smtp_tls_mandatory_protocols, # the security review of stage 5) (audit 2026-10-12, L31; Postfix -# 3.7.11, proto/postconf.proto, smtpd_tls_protocols: since 3.6 the default is -# ">=TLSv1", and a lower bound is written ">=version"). A sender that offers -# less on port 25 sends in the clear, as one without TLS does. VRFY answers -# no question about who is accepted on port 25 (relay_recipient_maps), and -# the banner names no distribution (Debian's adds "(Debian/GNU)"; the -# upstream default is "$myhostname ESMTP $mail_name"). +# 3.10.13, proto/postconf.proto:13240-13250, smtpd_tls_protocols: since 3.6 the +# default is ">=TLSv1", and a lower bound is written ">=version"). A sender that +# offers less on port 25 sends in the clear, as one without TLS does. VRFY +# answers no question about who is accepted on port 25 (relay_recipient_maps), +# and the banner names no distribution (Debian's adds "(Debian)": postfix +# 3.10.13-0+deb13u1, debian/main.cf.in:15 "(@DEB_VENDOR@)", filled in by +# debian/rules:118; the upstream default is "$myhostname ESMTP $mail_name"). postconf -e \ "smtpd_tls_protocols=>=TLSv1.2" \ "smtpd_tls_mandatory_protocols=>=TLSv1.2" \ @@ -315,7 +331,7 @@ postconf -e \ # uses for sender_login_maps is the BARE login the panel writes into the map; # the sasldb2 lookup still resolves because Postfix hands Cyrus $myhostname as # the server realm, which equals the realm the accounts were created under. -# (Postfix 3.7.11, src/smtpd/smtpd_sasl_glue.c, smtpd_sasl_activate(): an empty +# (Postfix 3.10.13, src/smtpd/smtpd_sasl_glue.c:249, smtpd_sasl_activate(): an empty # smtpd_sasl_local_domain is a NULL user realm; see HOSTNAME_VALUE above.) postconf -e \ "smtpd_sasl_auth_enable=yes" \ @@ -330,7 +346,7 @@ postconf -e \ # up in this map with mail_addr_find(), the order being smtpd's and not # texthash's: the full address, the address without its extension, then the # "@domain" wildcard - so both address modes work from the same map (Postfix -# 3.7.11, src/smtpd/smtpd_check.c, reject_auth_sender_login_mismatch(); +# 3.10.13, src/smtpd/smtpd_check.c:3933, reject_auth_sender_login_mismatch(); # src/global/mail_addr_find.c, mail_addr_find_opt() with MA_FIND_DEFAULT). # # recipient_delimiter is said here and not left to the distribution's main.cf: @@ -347,9 +363,9 @@ postconf -e \ # Restrictions: authenticated clients only, no relay to foreign destinations, # and every authenticated sender address must be owned by its login. NO # permit_mynetworks anywhere — authorisation is by credentials, never by -# network. This is what makes an open relay impossible. (Postfix 3.7.11, -# src/smtpd/smtpd_check.c, smtpd_check_rcpt(): the relay and recipient lists -# are evaluated in turn and the first REJECT ends them.) +# network. This is what makes an open relay impossible. (Postfix 3.10.13, +# src/smtpd/smtpd_check.c:5076, smtpd_check_rcpt(), :5160-5173: the relay and +# recipient lists are evaluated in turn and the first REJECT ends them.) postconf -e \ "smtpd_helo_required=yes" \ "smtpd_relay_restrictions=permit_sasl_authenticated, reject_unauth_destination" \ @@ -357,16 +373,24 @@ postconf -e \ "smtpd_sender_restrictions=reject_sender_login_mismatch, permit" # Level-1 rate limit by client IP. Backstop that keeps working even if the -# journal-milter (level 2) is down. Postfix 3.7.11, src/smtpd/smtpd.c, -# mail_cmd(): "450 4.7.1 Error: too much mail from", except for clients in -# smtpd_client_event_limit_exceptions (default $mynetworks; Debian's debconf -# default, debian/templates "postfix/mynetworks", is loopback only). +# journal-milter (level 2) is down. Postfix 3.10.13, src/smtpd/smtpd.c:2566, +# mail_cmd(), :2624: "450 4.7.1 Error: too much mail from", except for clients +# in smtpd_client_event_limit_exceptions (:2619, hogger_list; default +# $mynetworks, src/smtpd/smtpd.c:779). # anvil_rate_time_unit is one global window for every anvil limit (postconf(5)). # "Client IP" is the address the container sees: an IPv4 client's own, but # for a client over IPv6 the compose network's gateway, through Docker's # proxy - measured on a live server, 2026-10-15, Docker 29.6.1 (assumptions # C-K5; guide, "Rate limiting - level 1"). +# +# mynetworks is said here, loopback only, and not left to the package: it is +# what the exceptions above and smtpd_forbid_bare_newline_exclusions (below) +# spare. The value is the one Debian's debconf default gives (postfix +# 3.10.13-0+deb13u1, debian/templates:124-126, "postfix/mynetworks", written +# by debian/postfix.postinst:249-262; debian/main.cf.in:28 has 127.0.0.0/8 +# alone), so on this image it changes nothing. postconf -e \ + "mynetworks=127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128" \ "smtpd_client_message_rate_limit=${RATE_MSGS}" \ "anvil_rate_time_unit=${RATE_WINDOW}s" @@ -386,12 +410,12 @@ postconf -e \ # on this chain, since they are not an application's mail. Bounces Postfix # generates itself do not pass it (internal_mail_filter_classes is empty). # -# Postfix 3.7.11: html/MILTER_README.html, "Different settings for different -# Milter applications" (the braces, Postfix 3.0 and later) and "Signing -# internally-generated bounce messages"; src/milter/milter.c, milter_*_event() -# and milter_message(): milters are asked in list order and the first non-empty -# reply ends the round; src/global/mail_params.h, DEF_INT_FILT_CLASSES -# INT_FILT_CLASS_NONE. +# Postfix 3.10.13: html/MILTER_README.html:584, "Different settings for different +# Milter applications" (the braces, Postfix 3.0 and later) and :449, "Signing +# internally-generated bounce messages"; src/milter/milter.c:399-564, +# milter_*_event() and milter_message(): milters are asked in list order and +# the first non-empty reply ends the round; src/global/mail_params.h:3599, +# DEF_INT_FILT_CLASSES INT_FILT_CLASS_NONE. postconf -e \ "milter_protocol=6" \ "milter_default_action=tempfail" \ @@ -403,7 +427,7 @@ postconf -e \ # application's client-address list) and record it (the inbound journal), and # the address given once at connect is not there for a connection's later # messages - see MailFrom in internal/milter/milter.go. The defaults: Postfix -# 3.7.11, src/global/mail_params.h, DEF_MILT_MAIL_MACROS. +# 3.10.13, src/global/mail_params.h:3532-3534, DEF_MILT_MAIL_MACROS. postconf -e \ "milter_mail_macros=i {auth_type} {auth_authen} {auth_author} {mail_addr} {mail_host} {mail_mailer} {client_addr}" @@ -413,7 +437,7 @@ postconf -e \ # journal-milter hang then resolves to accept and an OpenDKIM hang to tempfail, # but within seconds rather than minutes. Values are well above any healthy # response time (signing/DB insert are sub-second), so they never fire in normal -# operation. The defaults: Postfix 3.7.11, src/global/mail_params.h, +# operation. The defaults: Postfix 3.10.13, src/global/mail_params.h:3559-3567, # DEF_MILT_CONN_TIME "30s", DEF_MILT_CMD_TIME "30s", DEF_MILT_MSG_TIME "300s". postconf -e \ "milter_connect_timeout=${MILTER_CONNECT_TIMEOUT:-15s}" \ @@ -424,12 +448,26 @@ postconf -e \ # smtps (465, implicit/wrapper TLS) — the primary, always-on submission # service. chroot=n so smtpd can read the sasldb2 and sender map under /data # and the Cyrus config outside any chroot. -# The names smtps, submission and smtp need no /etc/services: Postfix 3.7.11, -# src/global/mail_params.h, DEF_KNOWN_TCP_PORTS "lmtp=24, smtp=25, -# smtps=submissions=465, submission=587" (src/util/find_inet.c, +# The names smtps, submission and smtp need no /etc/services: Postfix 3.10.13, +# src/global/mail_params.h:4450-4451, DEF_KNOWN_TCP_PORTS "lmtp=24, smtp=25, +# smtps=submissions=465, submission=587" (src/util/find_inet.c:94-99, # find_inet_port() via filter_known_tcp_port()). +# +# smtpd_hide_client_session=yes on both submission services (owner's +# decision, 2026-10-15): the Received: header Postfix adds names neither the +# application's HELO name, reverse name and address, nor its TLS session, +# SASL login or the protocol - only "Received: by (Postfix) id +# ", "for " when there is one recipient, and the date +# (Postfix 3.10.13, src/smtpd/smtpd.c:3495, the client part written only +# when the parameter is off; :3627-3645, the "by" line opening the header +# when it is on). proto/postconf.proto:19580-19594: "may be used for the +# port 587 and 465 MUA services"; port 25 must keep the default, RFC 5321's +# trace, and does (smtp/inet below sets nothing). The queue id stays in the +# header, so a complaint about a message still leads to its row in the +# panel's log and to smtpd's "client=" line in mail.log. postconf -M "smtps/inet=smtps inet n - n - - smtpd" postconf -P \ + "smtps/inet/smtpd_hide_client_session=yes" \ "smtps/inet/smtpd_tls_wrappermode=yes" \ "smtps/inet/smtpd_sasl_auth_enable=yes" \ "smtps/inet/smtpd_client_restrictions=permit_sasl_authenticated,reject" @@ -437,13 +475,15 @@ postconf -P \ # submission (587, STARTTLS) — optional. Same SASL/milter/limits; the only # difference is TLS is negotiated via STARTTLS, so require encryption before # auth. Added only when SUBMISSION_ENABLE=true, otherwise removed so a restart -# after disabling it does not leave the port listening. (Postfix 3.7.11, -# src/smtpd/smtpd.c, pre_jail_init(): smtpd_tls_security_level=encrypt sets -# enforce_tls, which implies smtpd_tls_auth_only; src/postconf/postconf_edit.c, -# pcf_edit_master(): -MX drops the matched entry, continuation lines included.) +# after disabling it does not leave the port listening. (Postfix 3.10.13, +# src/smtpd/smtpd.c:6525-6526, pre_jail_init(): smtpd_tls_security_level=encrypt +# sets enforce_tls, which implies smtpd_tls_auth_only (:6547); +# src/postconf/postconf_edit.c:319, pcf_edit_master(): -MX drops the matched +# entry, continuation lines included.) if [ "${SUBMISSION_ENABLE}" = "true" ]; then postconf -M "submission/inet=submission inet n - n - - smtpd" postconf -P \ + "submission/inet/smtpd_hide_client_session=yes" \ "submission/inet/smtpd_tls_security_level=encrypt" \ "submission/inet/smtpd_sasl_auth_enable=yes" \ "submission/inet/smtpd_client_restrictions=permit_sasl_authenticated,reject" @@ -452,9 +492,9 @@ else fi # --- inbound smtpd on port 25 (optional inbound relay and/or DMARC ingest) ----- -# Debian's stock master.cf enables smtp/inet (postfix 3.7.11-0+deb12u1, -# debian/patches/40_chroot_by_default.diff: "smtp inet n - y - - smtpd", -# uncommented). When both flags are off, remove +# Debian's stock master.cf enables smtp/inet (postfix 3.10.13-0+deb13u1, +# debian/patches/40_chroot_by_default.diff:16: "smtp inet n - y - - smtpd", +# uncommented; conf/master.cf:12). When both flags are off, remove # that listener so port 25 is not an inbound smtpd (outbound delivery uses # smtp/unix). When either is on: no SASL, no OpenDKIM; relay accepts only # configured domains/recipients; DMARC accepts only allow-listed report addresses. @@ -479,15 +519,15 @@ if [ "${INBOUND_RELAY_ENABLE}" = "true" ] || [ "${DMARC_REPORTS_ENABLE}" = "true # postconf -P refuses them ("unmatched service_name/type") and leaves a # bare "pipe", which dies with "fatal: missing user= command-line # attribute" on every report. They go into the service line itself. - # (Seen in 1.7.0 through 1.9.5, date not recorded. Postfix 3.7.11: - # src/postconf/postconf_master.c, pcf_edit_master_param(), -P writes - # "-o name=value" and nothing else; postconf_edit.c warns "unmatched - # service_name/type" for a service master.cf does not have; - # src/pipe/pipe.c, get_service_attr(): "missing user= command-line - # attribute".) + # (Seen in 1.7.0 through 1.9.5, date not recorded, on Postfix 3.7.11. + # Postfix 3.10.13: src/postconf/postconf_master.c:1029, + # pcf_edit_master_param(), -P writes "-o name=value" and nothing else; + # postconf_edit.c:617 warns "unmatched service_name/type" for a service + # master.cf does not have; src/pipe/pipe.c:867, get_service_attr(), + # :1018: "missing user= command-line attribute".) # ${recipient} is expanded by pipe(8), not by the shell: the command # learns which report address the message came to from its argument - # (man/man8/pipe.8, ${recipient}: as many arguments as recipients). + # (man/man8/pipe.8:289, ${recipient}: as many arguments as recipients). # One at a time (the 1 before "pipe"): reports are in no hurry, every run # writes to the database the panel writes to, and a hundred of them in # parallel - Postfix's default - is what a flood of reports would start. @@ -498,9 +538,9 @@ if [ "${INBOUND_RELAY_ENABLE}" = "true" ] || [ "${DMARC_REPORTS_ENABLE}" = "true # command reads from the container's environment is named there. # export_environment is a list split at whitespace in which Postfix # expands "$": a path with either would reach the command as another - # path, and it would quietly open a database of its own. (Postfix 3.7.11, - # man/man8/pipe.8, export_environment; src/global/mail_params.h, - # DEF_EXPORT_ENVIRON "TZ MAIL_CONFIG LANG"; src/global/clean_env.c. The + # path, and it would quietly open a database of its own. (Postfix 3.10.13, + # man/man8/pipe.8:417, export_environment; src/global/mail_params.h:2650, + # DEF_EXPORT_ENVIRON "TZ MAIL_CONFIG LANG"; src/util/clean_env.c. The # parameter is main.cf's, so every Postfix process exports these too.) case "${SELFPOST_DATA_DIR:-}${SELFPOST_DB_PATH:-}" in *[[:space:]]* | *'$'*) @@ -520,8 +560,8 @@ if [ "${INBOUND_RELAY_ENABLE}" = "true" ] || [ "${DMARC_REPORTS_ENABLE}" = "true TRANSPORT_SETTING="texthash:${DMARC_TRANSPORT_MAP}" fi # Comma, not a space, between check_recipient_access and its table: see - # master_cf_set_smtp_inet_o (master.5, NOTE 2: "or use commas instead of - # spaces"). + # master_cf_set_smtp_inet_o (master.5, "-o name=value (short form)": "use + # commas instead of spaces if the parameter supports that"). RECIPIENT_RESTRICTIONS="check_recipient_access,texthash:${DMARC_RECIPIENTS_MAP},${RECIPIENT_RESTRICTIONS}" else postconf -MX "dmarc-ingest/unix" 2>/dev/null || true @@ -539,16 +579,17 @@ if [ "${INBOUND_RELAY_ENABLE}" = "true" ] || [ "${DMARC_REPORTS_ENABLE}" = "true # relay domain too (the server's own name), so their table is a recipient # table as well; without it they are "User unknown in relay recipient # table" and no report is ever accepted while both features are on. - # (Postfix 3.7.11, src/smtpd/smtpd_check.c, smtpd_check_rcpt(): after - # the lists, check_recipient_rcpt_maps() runs unless the status is - # REJECT - an OK from check_recipient_access does not skip it.) + # (Postfix 3.10.13, src/smtpd/smtpd_check.c, smtpd_check_rcpt(), + # :5196-5200: after the lists, check_recipient_rcpt_maps() runs unless + # the status is REJECT - an OK from check_recipient_access does not + # skip it.) RELAY_RECIPIENT_SETTING="texthash:${RELAY_RECIPIENTS_MAP}" if [ "${DMARC_REPORTS_ENABLE}" = "true" ]; then RELAY_RECIPIENT_SETTING="${RELAY_RECIPIENT_SETTING} texthash:${DMARC_RECIPIENTS_MAP}" fi # The panel writes the same "[host]:port" into transport and # tls_policy: the policy lookup key is the verbatim next-hop, port and - # brackets included (Postfix 3.7.11, README_FILES/TLS_README, "TLS + # brackets included (Postfix 3.10.13, README_FILES/TLS_README:914, "TLS # policy table" note: ":smtp", ":25" and no port are different keys). postconf -e \ "relay_recipient_maps=${RELAY_RECIPIENT_SETTING}" \ @@ -566,10 +607,10 @@ if [ "${INBOUND_RELAY_ENABLE}" = "true" ] || [ "${DMARC_REPORTS_ENABLE}" = "true # domain — polluting Deliveries and letting a forged From: consume that # domain's level-2 rate-limit budget. So smtpd_milters is always overridden # here: the optional antispam milter, then - with the inbound relay on - the - # panel's inbound journal milter, or nothing at all. (Postfix 3.7.11, + # panel's inbound journal milter, or nothing at all. (Postfix 3.10.13, # man/man5/master.5: "-o" overrides the main.cf parameter for this service; - # src/smtpd/smtpd.c hands its own milter list to cleanup, milter_send(), - # and src/cleanup/cleanup_milter.c, cleanup_milter_receive(), uses it.) + # src/smtpd/smtpd.c:3475 hands its own milter list to cleanup, milter_send(), + # and src/cleanup/cleanup_milter.c:1997, cleanup_milter_receive(), uses it.) INBOUND_MILTERS="" if [ "${INBOUND_RELAY_ENABLE}" = "true" ] && [ -n "${INBOUND_ANTISPAM_MILTER}" ]; then # A glob's "*" matches anything, so the shape alone would pass @@ -602,11 +643,12 @@ if [ "${INBOUND_RELAY_ENABLE}" = "true" ] || [ "${DMARC_REPORTS_ENABLE}" = "true # never refuses, and must never hold up someone else's mail: accept when # it is down. A comma, not a space, between the two - see # master_cf_set_smtp_inet_o. - # Postfix 3.7.11: src/milter/milter.c, milter_message(), sends the - # message to each milter in list order; src/milter/milter8.c, - # milter8_event(), applies a header a milter adds (SMFIR_ADDHEADER) to - # the queue file at once, and the next milter8_message() reads that - # file with rec_get(), which follows the added records. + # Postfix 3.10.13: src/milter/milter.c:564, milter_message(), sends the + # message to each milter in list order; src/milter/milter8.c:960, + # milter8_event(), applies a header a milter adds (SMFIR_ADDHEADER, + # :1424) to the queue file at once, and the next milter8_message() + # (:2544) reads that file with rec_get() (:2612), which follows the + # added records. case "${INBOUND_JOURNAL_SOCK}" in *[!A-Za-z0-9._/-]*) echo "FATAL: INBOUND_JOURNAL_MILTER_SOCKET must be an absolute path, got: ${INBOUND_JOURNAL_SOCK}" >&2 @@ -638,23 +680,29 @@ if [ "${INBOUND_RELAY_ENABLE}" = "true" ] || [ "${DMARC_REPORTS_ENABLE}" = "true # syslog_name: this listener's lines read postfix/inbound/smtpd, the # submission ports' postfix/smtpd. The log tailer files a refusal on port 25 - # into the inbound journal and has to tell the two apart. (Postfix 3.7.11, - # src/global/mail_task.c, mail_task(): the log tag is syslog_name "/" + # into the inbound journal and has to tell the two apart. (Postfix 3.10.13, + # src/global/mail_task.c:56, mail_task(): the log tag is syslog_name "/" # program name. Only smtpd carries it: cleanup and the rest are other # services and log as postfix/.) postconf -M "smtp/inet=smtp inet n - n - - smtpd" - # smtpd_forbid_bare_newline: the 2023 SMTP-smuggling defence, which this - # Postfix leaves off. normalize takes a bare as a line end but ends - # DATA only at the standard .: a "." line that a bare - # precedes or ends is dropped, not taken as the end of the message - # (Postfix 3.7.11, src/smtpd/smtpd.c:807, the default "no" for Postfix - # < 3.9; :1656, bare_lf_mask_table, normalize = BARE_LF_FLAG_WANT_STD_EOD; - # :3649-3652, the DATA loop; RELEASE_NOTES, "Major changes with Postfix - # 3.8.5, 3.7.10": normalize is the recommended setting and 3.9's - # default). Its exclusions default to $mynetworks, loopback only here - # (above), so no client is spared. The submission ports keep the - # default: their smuggled transaction would still be the authenticated - # session's, under its bindings and milters (own-mta-builds, § 5). + # smtpd_forbid_bare_newline: the 2023 SMTP-smuggling defence. normalize + # takes a bare as a line end but ends DATA only at the standard + # .: a "." line that a bare precedes or ends is + # dropped, not taken as the end of the message (Postfix 3.10.13, + # src/smtpd/smtpd.c:1693-1694, bare_lf_mask_table, normalize = + # BARE_LF_FLAG_WANT_STD_EOD; :3712-3716, the DATA loop; RELEASE_NOTES-3.9: + # 166-167, "the recommended setting "smtpd_forbid_bare_newline + # = normalize""). Since 3.9 normalize is also the default (src/smtpd/ + # smtpd.c:830, "Postfix >= 3.9: normalize"; src/global/mail_params.h:4313, + # DEF_SMTPD_FORBID_BARE_LF "normalize"); the line stays so that port 25 + # does not depend on a default (it came in 2.6.1, while the image's + # Postfix 3.7.11 defaulted to "no"). + # Its exclusions default to $mynetworks (mail_params.h:4316), loopback + # only here (above), so no client is spared. The submission ports take + # the default, normalize as well since this Postfix: an application whose + # SMTP library ends DATA with a bare . is no longer read to its + # end there (own-mta-builds, § 5: their smuggled transaction would still + # be the authenticated session's, so they were left to the default). postconf -P \ "smtp/inet/smtpd_forbid_bare_newline=normalize" \ "smtp/inet/syslog_name=postfix/inbound" \ @@ -671,12 +719,12 @@ if [ "${INBOUND_RELAY_ENABLE}" = "true" ] || [ "${DMARC_REPORTS_ENABLE}" = "true # smtpd's -o limit is not the whole of it: cleanup writes the queue file # under its own message_size_limit, main.cf's, which is never set here and - # so is 10240000 (Postfix 3.7.11, src/cleanup/cleanup_init.c, - # cleanup_post_jail(), set_file_limit(); src/global/mail_params.h, + # so is 10240000 (Postfix 3.10.13, src/cleanup/cleanup_init.c:444, + # cleanup_post_jail(), :455, set_file_limit(); src/global/mail_params.h:2013, # DEF_MESSAGE_LIMIT), and a larger message is refused 552 after DATA # (assumptions C-P31). cleanup gets port 25's limit when it is larger; - # smtpd on 465 and 587 still holds its own, main.cf's (src/smtpd/smtpd.c, - # the DATA and BDAT size checks). + # smtpd on 465 and 587 still holds its own, main.cf's (src/smtpd/smtpd.c:3721, + # :3964, :4136, :4250, the DATA and BDAT size checks). if [ "${PORT25_SIZE}" -gt "$(postconf -h message_size_limit)" ]; then postconf -P "cleanup/unix/message_size_limit=${PORT25_SIZE}" fi @@ -697,23 +745,44 @@ fi # Inside a container the chroot buys little (the container is the isolation # boundary) and breaks DNS/TLS trust-store access, so turn it off uniformly. Our # own smtps/submission services are already n; this covers the delivery agents -# and the rest. (postfix 3.7.11-0+deb12u1: debian/patches/40_chroot_by_default.diff -# sets chroot y; debian/configure-instance.sh copies etc/resolv.conf and the -# like into the chroot, and nothing runs it in this container. chroot(2) also -# needs CAP_SYS_CHROOT, which deploy/docker-compose.yml drops with cap_drop: -# ALL - capabilities(7).) +# and the rest. (postfix 3.10.13-0+deb13u1: debian/patches/40_chroot_by_default.diff +# sets chroot y. chroot(2) also needs CAP_SYS_CHROOT, which +# deploy/docker-compose.yml drops with cap_drop: ALL - capabilities(7).) +# +# Debian's chroot setup now runs on every "postfix check" and "postfix +# start-fg": conf/post-install:587 (debian/patches/ +# debian-run-configure-instance-from-create-missing.patch) calls +# /usr/lib/postfix/configure-instance.sh at the end of create-missing, and +# that copies etc/localtime, etc/resolv.conf, etc/hosts and the like into +# $queue_directory/etc - here /data/postfix/queue/etc - whether a service is +# chrooted or not (debian/configure-instance.in:80-91, before its +# need_chroot test at :93). With nothing chrooted those copies are read by +# nobody, so the image turns the sync off with SYNC_CHROOT= in +# /etc/default/postfix (build/Dockerfile; configure-instance.in:22-24, :44, +# "[ -n "$SYNC_CHROOT" ] || exit 0"; debian/README.Debian:136-138). postconf -F "*/*/chroot=n" # --- Cyrus SASL app config for smtpd ----------------------------------------- # Tells the Cyrus library (invoked by smtpd via smtpd_sasl_path=smtpd) to verify # passwords straight from the panel-maintained sasldb2. PLAIN/LOGIN # only — both are safe because TLS is mandatory before auth on every port. -# The file is found by a Debian patch, not by upstream Postfix: postfix -# 3.7.11-0+deb12u1, debian/patches/07_sasl_config.diff, xsasl_getconfpath() in -# src/xsasl/xsasl_cyrus_log.c gives Cyrus "$config_directory/sasl:/usr/lib/sasl", -# and smtpd_sasl_path=smtpd names smtpd.conf there. The keys: Cyrus SASL +# The directory is said here, cyrus_sasl_config_path, and not left to the +# package's main.cf. Postfix 3.10.13, src/xsasl/xsasl_cyrus_server.c:227-232, +# xsasl_cyrus_server_init(): a non-empty cyrus_sasl_config_path goes to Cyrus +# with sasl_set_path(SASL_PATH_TYPE_CONFIG, ...), and :246, +# sasl_server_init(callbacks, path_info) with smtpd_sasl_path=smtpd +# (src/smtpd/smtpd_sasl_glue.c:188-189) names smtpd.conf there; the default +# is empty (src/global/mail_params.h:1690, DEF_CYRUS_CONF_PATH ""), and then +# Cyrus looks in its own path, /etc/sasl2:/etc/sasl:/usr/lib//sasl2: +# /usr/lib/sasl2 (cyrus-sasl2 2.1.28+dfsg1, debian/rules:96, --with-configdir; +# lib/common.c:210, sasl_set_path()), which this file is not in. Bookworm's +# package found it with a patch (07_sasl_config.diff, gone in 3.10.13); trixie's +# writes cyrus_sasl_config_path = /etc/postfix/sasl into a new main.cf +# (postfix 3.10.13-0+deb13u1, debian/main.cf.in:71) and into an upgraded one +# only when it is empty (debian/postfix.postinst:88-97). The keys: Cyrus SASL # 2.1.28, doc/legacy/options.html (pwcheck_method, auxprop_plugin, # sasldb_path, mech_list). +postconf -e "cyrus_sasl_config_path=/etc/postfix/sasl" mkdir -p /etc/postfix/sasl cat > /etc/postfix/sasl/smtpd.conf <&1) ec=$? diff --git a/build/postfix-reload.sh b/build/postfix-reload.sh index ddefeca..d4541da 100644 --- a/build/postfix-reload.sh +++ b/build/postfix-reload.sh @@ -6,11 +6,14 @@ # behind the milter wait (build/postfix-wrapper.sh) - and the master that # starts next reads the files as they are now, so the panel takes that as # done. Anything else is "postfix reload" itself: 0, or its failure (Postfix -# 3.7.11, conf/postfix-script, "reload": exit 1 when not running or when -# postsuper fails; src/postfix/postfix.c, a main.cf it cannot read). "master -# -t" exits 0, silently, while no master holds the lock (src/master/master.c, +# 3.10.13, conf/postfix-script:266-279, "reload": exit 1 when not running or +# when postsuper fails; src/postfix/postfix.c, a main.cf it cannot read). "master +# -t" exits 0, silently, while no master holds the lock (src/master/master.c:495-499, # test_lock). A master that stops between the two is a failure the panel -# reports - the rarer mistake (review of stage 2c, F5). +# reports - the rarer mistake (review of stage 2c, F5). Since Postfix 3.8 the +# reload also writes "refreshing the Postfix mail system" to stderr, which +# goes to the container's log (postfix-wrapper.sh, the note on 3.8's stderr +# logging); the panel reads only the exit status. d=$(postconf -h daemon_directory) || exit 1 if "$d/master" -t 2>/dev/null; then echo "postfix-reload: Postfix is not running; its start reads the files" >&2 diff --git a/build/postfix-wrapper.sh b/build/postfix-wrapper.sh index d577fa3..25094fd 100644 --- a/build/postfix-wrapper.sh +++ b/build/postfix-wrapper.sh @@ -44,23 +44,37 @@ for sock in $SOCKS; do done echo "postfix-wrapper: milter sockets ready, starting postfix" -# Postfix 3.7.11: postfix(1) execs conf/postfix-script (src/postfix/postfix.c), -# whose "start-fg" execs master only as PID 1 and otherwise runs "master -s" as -# its child; master then makes itself a session leader (src/master/master.c, -# setsid()). A signal supervisord sends this program never reaches master +# Postfix 3.10.13: postfix(1) execs conf/postfix-script (src/postfix/postfix.c:669), +# whose "start-fg" (:200-210) execs master only as PID 1 and otherwise runs +# "master -s" as its child; master then makes itself a session leader +# (src/master/master.c:441, setsid()). A signal supervisord sends this +# program never reaches master # (supervisor 4.2.5, supervisor/process.py, Subprocess.kill(): to the pid # alone without stopasgroup, and master is not in its group anyway): before, # a stop left master running - a restart of the program then found Postfix # "already running", and a stop of the container cut Postfix off with # SIGKILL (assumptions C-P42). So this shell stays, and on TERM or INT runs -# "postfix stop" (conf/postfix-script, "stop": TERM to master, then waits up -# to five seconds and kills its group; src/master/master_sig.c, -# master_sigdeath(): "terminating on signal 15"). +# "postfix stop" (conf/postfix-script:237-253, "stop": TERM to master, then +# waits up to five seconds and kills its group; src/master/master_sig.c:205, +# master_sigdeath(): "terminating on signal 15"). Debian's redirect of +# "postfix start" to systemd (debian/patches/ +# debian-re-run-startup-through-systemd.patch, conf/postfix-script:170-189) +# leaves start-fg alone and needs /run/systemd/system, which this container +# has not. +# +# Since Postfix 3.8 postfix(1) and postlog(1) - postfix-script's $INFO, $WARN +# and $FATAL - write their messages to stderr even when it is not a terminal +# (src/postfix/postfix.c:546, msg_vstream_init() no longer under isatty(); +# RELEASE_NOTES-3.8:114-115, [Incompat 20230308]), besides maillog_file. So +# "starting the Postfix mail system", "stopping ..." and check-warn's +# warnings now reach the container's log through this program's stderr +# (build/supervisord.conf) as well as /data/log/mail.log; nothing reads them +# there but a person. # # "postfix start-fg" spends its first seconds in its checks before master # runs; a stop asked then would find nothing to stop, so it is asked again # until the child is gone. "master -t" exits 0, silently, while no master -# holds the lock (src/master/master.c, test_lock). +# holds the lock (src/master/master.c:495-499, test_lock). daemon_directory=$(postconf -h daemon_directory) child= stop_asked= diff --git a/cmd/panel/dmarc_ingest.go b/cmd/panel/dmarc_ingest.go index 9a65dfc..1a74aec 100644 --- a/cmd/panel/dmarc_ingest.go +++ b/cmd/panel/dmarc_ingest.go @@ -32,7 +32,7 @@ func runDMARCIngestMode() error { // Not a report this relay keeps. It is dropped: deferring would have // Postfix try it again for days, and a bounce would go to an envelope // sender anyone can write - the report addresses are public. Postfix - // 3.7.11: exit 0 is PIPE_STAT_OK, logged "status=sent (delivered via + // 3.10.13: exit 0 is PIPE_STAT_OK, logged "status=sent (delivered via // ... service ...)" (src/global/pipe_command.c, pipe_command(); // src/pipe/pipe.c, eval_command_status()); a 4.x.x status defers until // maximal_queue_lifetime and a 5.x.x one bounces, either way a notice @@ -49,7 +49,7 @@ func runDMARCIngestMode() error { // what a delivery command wrote - about a hundred bytes - into its own line // for the message ("status=sent (delivered via dmarc-ingest service (...))"), // so the reason goes out first and bare: no prefix, no date, no recipient, -// all of which that line already has. Postfix 3.7.11, src/pipe/pipe.c, +// all of which that line already has. Postfix 3.10.13, src/pipe/pipe.c, // eval_command_status(): " (%.100s)" of the output, which // src/global/pipe_command.c takes from stdout and stderr together. postlog is // no use here: pipe(8) starts the command without the group that may write to @@ -58,7 +58,9 @@ func runDMARCIngestMode() error { // installs it set-gid, Debian's package does not - measured in the image, // 2026-10-15: /usr/sbin/postlog is -rwxr-xr-x root:root (postfix // 3.7.11-0+deb12u1), and dpkg-statoverride lists only postdrop and -// postqueue (assumptions A-43). +// postqueue (assumptions A-43). postfix 3.10.13-0+deb13u1 is the same by its +// source: debian/postfix.postinst adds statoverrides for /usr/sbin/postdrop +// and /usr/sbin/postqueue alone, and nothing in debian/ makes postlog set-gid. func logRejected(reason error) { fmt.Fprintln(os.Stderr, "dropped: "+logSafe(strings.TrimPrefix(reason.Error(), dmarc.ErrRejected.Error()+": "))) } @@ -83,7 +85,7 @@ func logSafe(s string) string { // ingestRecipient is the report address the message was delivered to. Postfix's // pipe(8) hands it over as an argument (${recipient} in the service's argv, // build/postfix-config.sh) - it sets no RECIPIENT variable, that is local(8). -// Postfix 3.7.11, src/pipe/pipe.c, deliver_message(): pipe_command() gets only +// Postfix 3.10.13, src/pipe/pipe.c, deliver_message(): pipe_command() gets only // CA_PIPE_CMD_EXPORT(export_environment), no extra environment; // src/local/command.c sets "RECIPIENT". // Without it the check that a report came to its own domain's address is diff --git a/cmd/panel/httpserver.go b/cmd/panel/httpserver.go index b72291f..9b4bc55 100644 --- a/cmd/panel/httpserver.go +++ b/cmd/panel/httpserver.go @@ -99,7 +99,7 @@ func newPanel(cfg config, st *store.Store) (*web.Server, error) { ms := newMailStack(cfg, st) // postfix-config.sh has already run (entrypoint). postconf -h is the // effective config, including a manual override; the panel keeps this - // snapshot for the process lifetime (architecture.md). Postfix 3.7.11, + // snapshot for the process lifetime (architecture.md). Postfix 3.10.13, // postconf(1): without -d or -n it shows main.cf's value or the default, // and -h drops the "name = " label; values are not $-expanded without -x. retryPolicy := postfix.LoadRetryPolicy() diff --git a/cmd/panel/main.go b/cmd/panel/main.go index 87a3cef..d1febeb 100644 --- a/cmd/panel/main.go +++ b/cmd/panel/main.go @@ -37,7 +37,7 @@ func main() { log.SetPrefix("dmarc-ingest: ") if err := runDMARCIngestMode(); err != nil { log.Printf("ingest failed: %v", err) - // EX_TEMPFAIL — ask Postfix to defer. Postfix 3.7.11, + // EX_TEMPFAIL — ask Postfix to defer. Postfix 3.10.13, // src/global/pipe_command.c, pipe_command(): a sysexits code maps // to its DSN (src/global/sys_exits.c: 75 is "4.3.0") and a 4 defers, // unless the output starts with a "D.S.N text" of its own - the @@ -223,7 +223,7 @@ func antispamAction(raw string) string { // saslRealm chooses the realm new SASL accounts live under. It mirrors the // hostname Postfix's SASL layer uses so a client authenticating with a bare -// login resolves to the right account. Postfix 3.7.11, +// login resolves to the right account. Postfix 3.10.13, // src/xsasl/xsasl_cyrus_server.c, xsasl_cyrus_server_create(): serverFQDN is // $myhostname (= SELFPOST_HOSTNAME, build/postfix-config.sh) and user_realm // NULL while smtpd_sasl_local_domain is empty; Cyrus SASL 2.1.28, diff --git a/cmd/panel/sendercheck.go b/cmd/panel/sendercheck.go index a8a8147..2fb6f4c 100644 --- a/cmd/panel/sendercheck.go +++ b/cmd/panel/sendercheck.go @@ -19,7 +19,7 @@ const aliveEvery = 2 * time.Second // serveSenderCheck runs the sender check (internal/milter/sender.go) on a // socket of its own. Postfix attaches that socket strictly -// (default_action=tempfail in build/postfix-config.sh; Postfix 3.7.11, +// (default_action=tempfail in build/postfix-config.sh; Postfix 3.10.13, // src/milter/milter8.c, milter8_comm_error(): "451 4.7.1 Service unavailable - // try again later"), so while this role is not serving, mail is deferred: // what that costs is counted here and diff --git a/deploy/antispam/rspamd/milter_headers.conf b/deploy/antispam/rspamd/milter_headers.conf index d0e72e1..d59c418 100644 --- a/deploy/antispam/rspamd/milter_headers.conf +++ b/deploy/antispam/rspamd/milter_headers.conf @@ -43,8 +43,8 @@ # text. task:get_queue_id() is the Queue-Id the proxy took from the milter # macro "{i}" or "i" (src/libserver/milter.c, rspamd_milter_macro_http(); read # back in src/libserver/protocol.c), which Postfix sends at -# end-of-message (Postfix 3.7.11, src/global/mail_params.h, DEF_MILT_EOD_MACROS -# "i"). +# end-of-message (Postfix 3.10.13, src/global/mail_params.h:3555, +# DEF_MILT_EOD_MACROS "i"). use = ["x-spamd-result", "x-rspamd-action", "selfpost-signature"]; skip_local = false; skip_authenticated = false; diff --git a/deploy/traefik/extract-cert.sh b/deploy/traefik/extract-cert.sh index dd4a360..84074e1 100755 --- a/deploy/traefik/extract-cert.sh +++ b/deploy/traefik/extract-cert.sh @@ -59,7 +59,7 @@ grep -q -- '-----BEGIN CERTIFICATE-----' "$tmp_cert" || die "the certificate of grep -q -- '-----BEGIN .*PRIVATE KEY-----' "$tmp_key" || die "the key of $DOMAIN is not PEM" chmod 0640 "$tmp_cert" "$tmp_key" -# The group the files had stays theirs: the guide gives the key to group 998 +# The group the files had stays theirs: the guide gives the key to group 10998 # so the panel can read it for a full backup, and a run from root's cron would # otherwise hand it to root's group (review of audit stage 5). for f in fullchain privkey; do diff --git a/docs/architecture.md b/docs/architecture.md index 4f25c63..8b6153c 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -191,9 +191,22 @@ smtp/inet` removes Debian's stock smtpd). Outbound delivery still uses the `anvil_rate_time_unit` from `RATE_LIMIT_*` env vars; independent of milter. - **Port 25 ends DATA only at `.`** — `smtpd_forbid_bare_newline=normalize` on the `smtp/inet` listener (Postfix - 3.7.11, `src/smtpd/smtpd.c`, the DATA loop at 3649: a `.` line next to a - bare `` is dropped, not taken as the end); a bare `` elsewhere is - read as ``. The submission ports keep Postfix's default. + 3.10.13, `src/smtpd/smtpd.c`, `receive_data_message()`, the DATA loop at + 3713: a `.` line next to a bare `` is dropped, not taken as the end); a + bare `` elsewhere is read as ``. The submission ports keep + Postfix's default, which is the same `normalize` since Postfix 3.9 + (`src/global/mail_params.h`, `DEF_SMTPD_FORBID_BARE_LF`); the image ran + Postfix 3.7.11 until `2.7.0`, whose default was `no`, hence the explicit + setting on port 25. +- **The submission ports hide the application in `Received:`** (`2.7.0`) — + `smtpd_hide_client_session=yes` on `smtps/inet` and `submission/inet`: + Postfix's header reads `Received: by (Postfix) id `, + with the recipient when there is one, and the date - no HELO name, client + name or address, TLS, login or protocol (Postfix 3.10.13, + `src/smtpd/smtpd.c:3495` and `:3627-3645`). Port 25 keeps the full + header, which RFC 5321 requires there (`proto/postconf.proto:19587-19589`). + The queue id in the header leads to the message's row in the panel's log + and to smtpd's `client=` line in mail.log. - **Chroot disabled** for all services (DNS/TLS inside container). - **TLS certs** — read-only mount at `TLS_CERT_FILE` / `TLS_KEY_FILE`, copied at start into `/etc/postfix/tls-internal`, owned by root and `0600` as @@ -234,7 +247,7 @@ milter, § Postfix above, port 25): `delay_warning_time`), reads each as Postfix does - one number and one unit, the parameter's own default unit when there is none (`5d` for the lifetimes, so a bare `3` is three days), `bounce_queue_lifetime` capped at - `maximal_queue_lifetime` as qmgr caps it (Postfix 3.7.11, + `maximal_queue_lifetime` as qmgr caps it (Postfix 3.10.13, `src/global/conv_time.c`, `src/qmgr/qmgr.c`) - and caches the snapshot on the handlers config. The Outbound queue box and a delivery's `deferred` / `bounced` history print those @@ -295,7 +308,7 @@ server itself. reason), then `@domain` - and the **first of these keys in the table decides**, its owners alone: an address listed for one application is not open to another that holds the whole domain. Logins are compared without - regard to case, as Postfix compares them (Postfix 3.7.11, + regard to case, as Postfix compares them (Postfix 3.10.13, `src/smtpd/smtpd_check.c`, `reject_auth_sender_login_mismatch()`, over `src/global/mail_addr_find.c`, `mail_addr_find_opt()`). A key written twice makes Postfix refuse the whole table, and the check defers then too. @@ -461,7 +474,7 @@ recreates the file on its first line after the reload, as root and mode `0600`, which the unprivileged panel user cannot read). `follow()` drains the old inode once more before switching descriptors, and keeps it open and reads it for a minute after: postlogd keeps writing to the renamed file until the -reload makes it exit, seconds later (Postfix 3.7.11, src/postlogd/postlogd.c; +reload makes it exit, seconds later (Postfix 3.10.13, src/postlogd/postlogd.c; src/master/dgram_server.c, `dgram_server_abort()`). The panel treats a missing log file as an empty tail, not an error. @@ -494,7 +507,7 @@ the lines stored under it - only those of the minutes around the refusal, since Postfix frees the id at once. A message the journal milter refused at `MAIL FROM` has none: the tailer reads smtpd's `NOQUEUE: milter-reject: MAIL from [
]: …; from=<…> … sasl_username=` - the login is -Postfix's last field on an authenticated session (Postfix 3.7.11, +Postfix's last field on an authenticated session (Postfix 3.10.13, `src/smtpd/smtpd.c`, `smtpd_whatsup()`), and a line without one is no application's - and puts it on the nearest refused row of that client address, login and sender within two minutes @@ -513,7 +526,7 @@ a kill is covered as well as a clean stop. **Expiry.** A message Postfix gives up on at the end of its queue lifetime has no line per recipient: qmgr writes one line for the message, `: from=<...>, status=expired, returned to sender` (`force-expired` after -`postsuper -e`; Postfix 3.7.11, `src/qmgr/qmgr_active.c`, +`postsuper -e`; Postfix 3.10.13, `src/qmgr/qmgr_active.c`, `qmgr_active_done_25_generic()`), and the recipients' last lines say `deferred`. The tailer reads that line and closes the message's open rows - each recipient's newest, `queued` or `deferred` - as `bounced`. @@ -542,7 +555,7 @@ of the id's flags (`src/postqueue/showq_compat.c`, `showq_message()`). **Which row a delivery line is.** The journal milter writes a row's recipient as Postfix will log it: the address the client gave, its quoted local part unquoted and quoted again only where it needs to be (`"ab"@x` -becomes `ab@x`, `"a b"@x` stays; Postfix 3.7.11, `tok822_parse.c`, +becomes `ab@x`, `"a b"@x` stays; Postfix 3.10.13, `tok822_parse.c`, `quote_822_local.c`), case kept, and a recipient given twice - in any case - once, since cleanup keeps one (`cleanup_out_recipient.c`, `been_here()` folding case). A delivery line names it in `orig_to=` when Postfix rewrote diff --git a/docs/assumptions-2026-10-14.md b/docs/assumptions-2026-10-14.md index 569aefa..22609e5 100644 --- a/docs/assumptions-2026-10-14.md +++ b/docs/assumptions-2026-10-14.md @@ -38,6 +38,14 @@ versions `go.mod` pins (go-milter v0.4.1, modernc.org/sqlite v1.60.1 with SQLite 3.53.4); htmx 2.0.4 as vendored. Upstream paths below are relative to the package's source tree. +Since base-trixie (`2.7.0`) the image runs Debian trixie (Postfix 3.10.13, +opendkim 2.11.0~beta2-9.1+b1, cyrus-sasl2 2.1.28+dfsg1-9, supervisor +4.2.5-3, logrotate 3.22.0-1; development.md § Code that relies on another +component's behaviour), and the citations in the code were re-read against +the trixie sources. This register is left as it was made: its line numbers +and versions are those of the bookworm sources above, which are now kept +under `dev/upstream-bookworm/` (where it says `dev/upstream/`). + Counts at the inventory, by the justification the tree had: | Part | Entries | CITED | MEASURED | GUESSED | WRONG? | diff --git a/docs/development.md b/docs/development.md index c423af4..5061e27 100644 --- a/docs/development.md +++ b/docs/development.md @@ -152,8 +152,8 @@ commit rule but **no** version cut until the next product stage ships. | **SQLite** | `modernc.org/sqlite` (pure Go, no cgo) | | **Build** | [Makefile](../Makefile): primary targets `vet`, `test`, `build`, `e2e` (also `all`, `clean`) | | **Container** | Docker + Compose v2 on the dev host and in CI | -| **Image (build stage)** | `golang:1.26-bookworm` — [build/Dockerfile](../build/Dockerfile). The tag floats to the newest 1.26 patch release, and most standard-library security fixes (`net/http`, `html/template`, …) arrive only that way: rebuild the image after each Go patch release, no edit needed. Dependencies are raised by hand (`go get …@latest`, `go mod tidy`), at least at each audit | -| **Image (runtime)** | `debian:bookworm-slim` + Postfix, OpenDKIM, supervisord, SASL, logrotate | +| **Image (build stage)** | `golang:1.26-trixie` (bookworm until `2.7.0`) — [build/Dockerfile](../build/Dockerfile). The tag floats to the newest 1.26 patch release, and most standard-library security fixes (`net/http`, `html/template`, …) arrive only that way: rebuild the image after each Go patch release, no edit needed. Dependencies are raised by hand (`go get …@latest`, `go mod tidy`), at least at each audit | +| **Image (runtime)** | `debian:trixie-slim` (bookworm until `2.7.0`) + Postfix, OpenDKIM, supervisord, SASL, logrotate | | **CI** | GitHub Actions — [.github/workflows/](../.github/workflows/) | | **Image registry** | `ghcr.io/mixeme/selfpost` | @@ -209,8 +209,8 @@ the image. ### Debian packages in the runtime image Postfix, OpenDKIM, `supervisord`, `sasl2-bin`, `logrotate`, and others come -from Debian bookworm repositories; licenses are in each package's `copyright` -file on . +from Debian trixie repositories; licenses are in each package's `copyright` +file on . The image also ships [LICENSE](../LICENSE), [NOTICE](../NOTICE), and the IBM Plex [OFL.txt](../internal/web/view/static/OFL.txt) under `/usr/share/doc/selfpost/`. The panel serves the AGPL text at `/license` and @@ -225,21 +225,22 @@ lines and reply codes, feeds OpenDKIM's tables, parses `postqueue`, through go-milter, relies on SQLite's locking, on logrotate's and Docker's behaviour. Every block of code that does so carries a comment that says what it relies on and where that is written, for the version the image runs -(the Debian bookworm package, not upstream's head): +(the Debian trixie package, not upstream's head): ```go -// Postfix 3.7.11, src/smtpd/smtpd_check.c, +// Postfix 3.10.13, src/smtpd/smtpd_check.c, // reject_auth_sender_login_mismatch(): the SASL login is compared with // each owner in sender_login_maps by strcasecmp_utf8. ``` The Debian sources of the image's versions are fetched with `apt-get source` -inside a `debian:bookworm` container (`postfix`, `opendkim`, `cyrus-sasl2`, +inside a `debian:trixie` container (`postfix`, `opendkim`, `cyrus-sasl2`, `supervisor`, `logrotate`, and `sendmail` for libmilter); `dev/upstream/` -(not in git) is where they are kept for reading. Go modules are read in the -module cache at the version `go.mod` pins. +(not in git) is where they are kept for reading, and `dev/upstream-bookworm/` +holds the bookworm ones the citations named before `2.7.0`. Go modules are +read in the module cache at the version `go.mod` pins. -The image takes `debian:bookworm-slim` and installs without pinning (pinned +The image takes `debian:trixie-slim` and installs without pinning (pinned Debian versions leave the mirrors), so a rebuild after a point or security release runs packages the citations were not read against. The versions read are these; the e2e stand compares them with `dpkg-query` in the image @@ -249,14 +250,14 @@ re-read what changed, update the citations, then this table: | Package | Version read | |---|---| -| `postfix` | `3.7.11-0+deb12u1` | -| `opendkim` | `2.11.0~beta2-8+deb12u1` | -| `libopendkim11` | `2.11.0~beta2-8+deb12u1` | -| `libmilter1.0.1` | `8.17.1.9-2+deb12u2` | -| `libsasl2-2` | `2.1.28+dfsg-10` | -| `sasl2-bin` | `2.1.28+dfsg-10` | -| `supervisor` | `4.2.5-1` | -| `logrotate` | `3.21.0-1` | +| `postfix` | `3.10.13-0+deb13u1` | +| `opendkim` | `2.11.0~beta2-9.1+b1` | +| `libopendkim11` | `2.11.0~beta2-9.1+b1` | +| `libmilter1.0.1` | `8.18.1-6+b1` | +| `libsasl2-2` | `2.1.28+dfsg1-9` | +| `sasl2-bin` | `2.1.28+dfsg1-9` | +| `supervisor` | `4.2.5-3` | +| `logrotate` | `3.22.0-1` | A document instead of source: its URL and section. Where neither answers the diff --git a/docs/guide.md b/docs/guide.md index 4ac83e5..e33f145 100644 --- a/docs/guide.md +++ b/docs/guide.md @@ -73,6 +73,20 @@ listened on: refused from outside over IPv4 and from the host's public IPv4 address; accepted and closed through `127.0.0.1`, the same proxy. IPv6 itself was not probed for this. +Since `2.7.0` the image runs Postfix 3.10, whose defaults hold every +listener - 465 and 587 as well as 25 - to two rules of SMTP: a message ends +only at the standard `.` (a client that ends DATA with a +bare newline waits for a reply that does not come), and a client that sends +commands ahead of the server's replies where RFC 2920 does not allow it is +answered `554 5.5.0 Error: SMTP protocol synchronization` and disconnected. + +Since `2.7.0` a message sent on 465 or 587 does not tell its recipients +where the application runs: the `Received:` header SelfPost adds reads +`Received: by mail.example.com (Postfix) id 4AbC12xyz for <...>; `, +without the application's address, host name or login. To trace a message +someone complains about, take that queue id to *Outbound › Log*; mail.log +keeps the client address on smtpd's `client=` line for the same id. + ### Local trial The [README quick start](../README.md#quick-start) runs a single container @@ -312,13 +326,13 @@ ln -s /etc/letsencrypt/live/mail.example.com/privkey.pem certs/privkey.pem ``` The panel reads them as its own user (Health shows the certificate, a full -backup carries both files), through the container's group 998. certbot's +backup carries both files), through the container's group 10998. certbot's documentation gives the way for a server that is not root, and certbot keeps the key's group and mode at every renewal: ```sh sudo chmod 0755 /etc/letsencrypt/live /etc/letsencrypt/archive -sudo chgrp 998 /etc/letsencrypt/archive/mail.example.com/privkey*.pem +sudo chgrp 10998 /etc/letsencrypt/archive/mail.example.com/privkey*.pem sudo chmod 0640 /etc/letsencrypt/archive/mail.example.com/privkey*.pem ``` @@ -339,7 +353,7 @@ docker compose -f docker-compose.yml -f nginx/docker-compose.nginx.yml \ The fragment mounts certbot's directory into both nginx and SelfPost, at `/etc/letsencrypt`; `./certs` holds two links to `/etc/letsencrypt/live/mail.example.com/` — a path that exists inside the -container, not on the host — and the panel reads the key through group 998, +container, not on the host — and the panel reads the key through group 10998, as in the Apache case. Then start everything: ```sh @@ -347,7 +361,7 @@ mkdir certs ln -s /etc/letsencrypt/live/mail.example.com/fullchain.pem certs/fullchain.pem ln -s /etc/letsencrypt/live/mail.example.com/privkey.pem certs/privkey.pem sudo chmod 0755 nginx/certbot-etc/live nginx/certbot-etc/archive -sudo chgrp 998 nginx/certbot-etc/archive/mail.example.com/privkey*.pem +sudo chgrp 10998 nginx/certbot-etc/archive/mail.example.com/privkey*.pem sudo chmod 0640 nginx/certbot-etc/archive/mail.example.com/privkey*.pem docker compose -f docker-compose.yml -f nginx/docker-compose.nginx.yml up -d @@ -714,11 +728,11 @@ instance keeps running, what goes into the archive, and what happens on restore. → *Full backup*. *Encrypt with a password* is ticked; untick it for a plain archive. Same archive as the CLI below. The panel runs as its own user, not as root, so **the private key in `certs/` has to be readable by it**: give the -file to the container's `selfpost` group, which has the fixed id 998 and holds +file to the container's `selfpost` group, which has the fixed id 10998 and holds both the panel and Postfix — ```sh -chgrp 998 certs/privkey.pem && chmod 640 certs/privkey.pem +chgrp 10998 certs/privkey.pem && chmod 640 certs/privkey.pem ``` — and have whatever renews the certificate do the same; certbot does it by @@ -795,7 +809,7 @@ methods avoid it by stopping first. | SQLite consistency | `VACUUM INTO` snapshot | Safe when stopped | Safe when stopped | | `data/log/` | Excluded | Included (optional exclude) | Included (optional exclude) | | Postfix queue | No | Yes | Yes | -| `certs/` private key | Yes (the panel needs it readable by group 998; links are followed) | Yes when `certs/` holds the files; links are stored as links unless `tar -h` | No — deploy files separate | +| `certs/` private key | Yes (the panel needs it readable by group 10998; links are followed) | Yes when `certs/` holds the files; links are stored as links unless `tar -h` | No — deploy files separate | | `data/manifest.json` | Written (version stamp) | Absent | Absent | | Restore version check | Yes — must match image tag | No | No | | Post-restore *Resync* | Yes, on first boot | No | No | diff --git a/docs/roadmap.md b/docs/roadmap.md index 8b7200b..752eaed 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -827,9 +827,9 @@ supervisord; `dev/upstream` refetched from trixie; the cited-package table and `checkCitedPackageVersions`; the e2e sink image; `opendkim-tools` dropped from the image (used by nothing at run time). OpenDKIM's code is the same in trixie, its citations stand. One stage, a MINOR release. Also -gained: `smtpd_hide_client_session` (3.10) is available - whether the -relay should hide the application's address from `Received:` is a product -question to decide separately. +gained: `smtpd_hide_client_session` (3.10); the owner decided on +2026-10-15 to turn it on for 465 and 587 in the same release, so the +`Received:` header no longer shows the application's address. ## panel-dkim-signing diff --git a/docs/security.md b/docs/security.md index a5d3c86..0297ab5 100644 --- a/docs/security.md +++ b/docs/security.md @@ -218,6 +218,14 @@ operator can act on it. - **Not an open relay** — SASL only on 465/587; `reject_unauth_destination`; `smtpd_sender_login_maps` + `reject_sender_login_mismatch`. +- **A recipient does not learn where an application runs (`2.7.0`).** The + `Received:` header Postfix adds on 465 and 587 carries no client name, + address, HELO name, TLS details or login (`smtpd_hide_client_session=yes`, + the owner's decision of 2026-10-15): before, every message told its + recipients the address and the internal host name of the machine the + application sent from. What a complaint needs stays with the operator - the + queue id in the header, the panel's log and smtpd's `client=` line in + mail.log. Port 25 keeps the full header, as RFC 5321 requires of an MTA. - **An application signs only as itself (`2.4.0`).** OpenDKIM chooses the key by the `From:` header and Postfix holds only the envelope sender against the login, so until `2.4.0` an application of one sending domain @@ -264,12 +272,16 @@ operator can act on it. on inbound. An optional antispam milter is inbound-only; default action is fail-open (`accept`). Since `2.6.1` the listener ends DATA only at the standard `.` (`smtpd_forbid_bare_newline=normalize`, the - 2023 SMTP-smuggling defence, which Postfix 3.7 leaves off by default): a + 2023 SMTP-smuggling defence, which Postfix 3.7 left off by default): a `.` line next to a bare newline is body, and what a smuggler would put behind it is delivered as part of the one message, not read as a second - transaction. The submission ports keep the default - there a smuggled - transaction is still the authenticated session's, under its sender - bindings and milters ([own-mta-builds](own-mta-builds-2026-10-15.md) § 5). + transaction. The submission ports keep the default. Since `2.7.0` the + image runs Postfix 3.10.13, whose default is `normalize` too (Postfix 3.9 + and later; `src/global/mail_params.h`, `DEF_SMTPD_FORBID_BARE_LF`), so + they end DATA the same way; the port-25 setting stays explicit. Before + `2.7.0` a smuggled transaction on a submission port was still the + authenticated session's, under its sender bindings and milters + ([own-mta-builds](own-mta-builds-2026-10-15.md) § 5). - **The panel's milters judge every message of a connection.** Postfix introduces a client to a milter once per connection, and the milter library replaces the milter's session after any reply that is not "continue". Until @@ -303,7 +315,7 @@ operator can act on it. parsed from `mail.log` from the head of the line, with the reply ending at Postfix's own `; from=<` so that the HELO name after it - the client's text, which Postfix logs with non-printable characters and ` <>()\";@` turned into - `?` (Postfix 3.7.11, src/smtpd/smtpd.c `helo_cmd()`) - cannot stand in for + `?` (Postfix 3.10.13, src/smtpd/smtpd.c `helo_cmd()`) - cannot stand in for the sender, the recipient or the reply; an address with a quote, a bracket or a space in it is not kept. Only a recipient at an inbound domain makes a row. The journal is cut to 10 000 rows every 500 rows the tailer writes, and by age every six hours. @@ -524,7 +536,7 @@ deferred item from the roadmap. § Roles. - **Level-2 in-flight reservations can outlive a broken SMTP session until TTL.** Postfix tells the milters ABORT when a client drops the connection - mid-transaction (Postfix 3.7.11, src/smtpd/smtpd.c `mail_reset()` -> + mid-transaction (Postfix 3.10.13, src/smtpd/smtpd.c `mail_reset()` -> `milter_abort()`), and the journal milter releases the reservation then. When that never arrives - smtpd killed, the milter connection lost - go-milter has no close callback, and the in-memory reservation is released by TTL diff --git a/internal/app/sasl.go b/internal/app/sasl.go index fa65a86..4659a0d 100644 --- a/internal/app/sasl.go +++ b/internal/app/sasl.go @@ -55,7 +55,7 @@ type SASLDB struct { // realm should match SELFPOST_HOSTNAME so the account identity lines up with // Postfix's SASL configuration. // -// Postfix 3.7.11, src/xsasl/xsasl_cyrus_server.c, xsasl_cyrus_server_create(): +// Postfix 3.10.13, src/xsasl/xsasl_cyrus_server.c, xsasl_cyrus_server_create(): // sasl_server_new() gets $myhostname and, with smtpd_sasl_local_domain empty, // no user realm; Cyrus SASL 2.1.28, plugins/sasldb.c, // sasldb_auxprop_lookup(), then looks a login without "@" up under the diff --git a/internal/app/validate.go b/internal/app/validate.go index 52f40a3..30116bd 100644 --- a/internal/app/validate.go +++ b/internal/app/validate.go @@ -32,7 +32,7 @@ func invalid(format string, args ...any) error { // (idx_applications_login_nocase); a login imported from another instance is // not folded, since its SASL account and its client use it as it is. // -// Postfix 3.7.11, src/smtpd/smtpd_check.c, reject_auth_sender_login_mismatch(): +// Postfix 3.10.13, src/smtpd/smtpd_check.c, reject_auth_sender_login_mismatch(): // the SASL login is compared with each owner by strcasecmp_utf8(). func foldLogin(login string) string { return strings.ToLower(login) diff --git a/internal/backup/backup.go b/internal/backup/backup.go index fb72472..2bc2308 100644 --- a/internal/backup/backup.go +++ b/internal/backup/backup.go @@ -91,7 +91,7 @@ type Params struct { // bring back is not the queue. Nor could the panel read it - Postfix keeps // every queue file readable by its own user alone, which is why the panel's // backup failed outright from 1.3.0 to 2.0.0 (date not recorded; Postfix -// 3.7.11, conf/postfix-files: active, deferred, incoming and the other queue +// 3.10.13, conf/postfix-files: active, deferred, incoming and the other queue // directories are $mail_owner 700). What is not carried over is mail // still waiting for delivery; a start without a queue directory builds an // empty one. Whoever needs the queue stops the instance and archives the diff --git a/internal/health/cert.go b/internal/health/cert.go index 33006b6..c613772 100644 --- a/internal/health/cert.go +++ b/internal/health/cert.go @@ -26,7 +26,7 @@ const certWarnDays = 14 // Postfix itself serves the copy build/postfix-config.sh makes in // /etc/postfix/tls-internal, refreshed by build/postfix-cert-reload.sh up to // TLS_RELOAD_INTERVAL_SECONDS later, and each smtpd loads it when it starts -// (Postfix 3.7.11, src/smtpd/smtpd.c, pre_jail_init(), TLS_SERVER_INIT) - so +// (Postfix 3.10.13, src/smtpd/smtpd.c, pre_jail_init(), TLS_SERVER_INIT) - so // the mounted file can be newer than what is served (assumptions A-65). // CheckServedCertificate reads the copy, and names the mounted file as a // renewal waiting while the two differ. @@ -52,7 +52,7 @@ type Renewal struct { } // ServedCertPath is the certificate file Postfix serves, smtpd_tls_cert_file -// as postconf prints it (Postfix 3.7.11, postconf(1): -h, the value without +// as postconf prints it (Postfix 3.10.13, postconf(1): -h, the value without // its name): build/postfix-config.sh's copy when the mounted files were there // at the start, the mounted file itself otherwise. "" when postconf fails. // It is read once, at the panel's start, as the retry policy is: main.cf is diff --git a/internal/health/milter.go b/internal/health/milter.go index 7a69ced..121dc71 100644 --- a/internal/health/milter.go +++ b/internal/health/milter.go @@ -17,7 +17,7 @@ import ( const milterDialTimeout = time.Second // CheckMilter probes a milter address in the two forms build/postfix-config.sh -// lets INBOUND_ANTISPAM_MILTER take (Postfix 3.7.11, src/milter/milter8.c, +// lets INBOUND_ANTISPAM_MILTER take (Postfix 3.10.13, src/milter/milter8.c, // milter8_connect(), also accepts "local:", which that script refuses): // unix:/path is stat-ed like any milter socket, and inet:host:port is dialled // and the connection closed at once, with nothing written. required says what diff --git a/internal/health/socket.go b/internal/health/socket.go index 1bba430..610381f 100644 --- a/internal/health/socket.go +++ b/internal/health/socket.go @@ -26,7 +26,7 @@ type Socket struct { // stops mail leaving the server, while the journal-milter is attached with // default_action=accept and fails open — mail still goes out, only the send // log stops being written (build/postfix-config.sh smtpd_milters; Postfix -// 3.7.11, src/milter/milter8.c, milter8_comm_error(): tempfail answers "451 +// 3.10.13, src/milter/milter8.c, milter8_comm_error(): tempfail answers "451 // 4.7.1 Service unavailable - try again later", accept answers nothing). // // The socket is dialled, as Postfix will dial it; a file is not a listener. @@ -35,7 +35,7 @@ type Socket struct { // a killed OpenDKIM or panel leaves a file that refuses every connection - // assumptions A-38. The connection is closed before a byte is sent, which // none of the three milters logs: OpenDKIM's libmilter reads the EOF as -// SMFIC_EOF and ends the session without a line (sendmail 8.17.1.9, +// SMFIC_EOF and ends the session without a line (libmilter 8.18.1, // libmilter/comm.c, mi_rd_cmd(); engine.c, mi_engine()), and OpenDKIM's // mlfi_close() has no connection to clean up (opendkim 2.11.0~beta2, // opendkim/opendkim.c); go-milter v0.4.1 returns quietly on io.EOF diff --git a/internal/inbound/service.go b/internal/inbound/service.go index 874f61a..09248ca 100644 --- a/internal/inbound/service.go +++ b/internal/inbound/service.go @@ -79,7 +79,7 @@ func (e *UpstreamShared) Error() string { // SetTransportAs validates and saves the upstream of domain id, then rebuilds // the maps. An upstream has one TLS mode: Postfix's TLS policy is keyed by the -// next hop, not by the domain (smtp_tls_policy_maps; Postfix 3.7.11, +// next hop, not by the domain (smtp_tls_policy_maps; Postfix 3.10.13, // README_FILES/TLS_README: the key is the verbatim next-hop), so two domains // to one [host]:port share one mode - the map held both keys and the first by // domain name won with a warning (audit 2026-10-12, L5). The other domains diff --git a/internal/logtail/inbound.go b/internal/logtail/inbound.go index cef9014..c405725 100644 --- a/internal/logtail/inbound.go +++ b/internal/logtail/inbound.go @@ -45,11 +45,15 @@ type Inbound struct { // so it counts only for a queue id the port-25 smtpd was seen to open // (inboundClientRe). // -// Postfix 3.7.11: smtpd's refusal is src/smtpd/smtpd_check.c, log_whatsup(): +// Postfix 3.10.13: smtpd's refusal is src/smtpd/smtpd_check.c, log_whatsup(): // "%s: %s: %s from %s: %s;" then " from=<..>", " to=<..>", " proto=", -// " helo=<..>", nothing after; a milter's at smtpd is src/smtpd/smtpd.c, +// " helo=<..>", then - new in 3.10 (RELEASE_NOTES, [Incompat 20241023]) - +// " sasl_method=%s", " sasl_username=%s" and " sasl_sender=%s" for a session +// that has them, nothing after; a milter's at smtpd is src/smtpd/smtpd.c, // check_milter_reply(), whose smtpd_whatsup() appends " sasl_username=<..>" -// only to an authenticated session (SASL is off on port 25). "name[addr]" is +// only to an authenticated session. SASL is off on port 25 +// (build/postfix-config.sh, smtpd_sasl_auth_enable=no), so neither tail is +// written there; the pattern takes the 3.10 one all the same. "name[addr]" is // state->namaddr, the address without "IPv6:" (smtpd.h, // SMTPD_BUILD_NAMADDRPORT). cleanup's is src/cleanup/cleanup_milter.c, // cleanup_milter_apply(); smtpd hands it to the one cleanup service @@ -64,21 +68,29 @@ type Inbound struct { // journal rows. // // For the same reason the reply is the shortest text before "; from=<", and -// the HELO name - the last thing on the line, which Postfix logs as the client -// wrote it, spaces and brackets included - is whatever is left. Taking the -// longest reply instead would let a HELO name that itself spells +// the HELO name - the last thing on the line but the SASL fields above, which +// Postfix logs as the client wrote it, spaces and brackets included - is +// whatever is left. Taking the longest reply instead would let a HELO name +// that itself spells // "; from=<…> to=<…> proto=…" supply the sender, the recipient and the reply // of the row (security review, 2026-10-12). var ( - inboundRejectRe = regexp.MustCompile(`^\S+ postfix/(inbound/smtpd|cleanup)\[\d+\]: ([0-9A-Za-z]+): (reject|milter-reject|milter-discard): \S+ from [^\[\s]*\[([0-9A-Fa-f:.]+)\]: (.*?); from=<([^>]*)> to=<([^>]*)> proto=\S+(?: helo=<.*>)?$`) + inboundRejectRe = regexp.MustCompile(`^\S+ postfix/(inbound/smtpd|cleanup)\[\d+\]: ([0-9A-Za-z]+): (reject|milter-reject|milter-discard): \S+ from [^\[\s]*\[([0-9A-Fa-f:.]+)\]: (.*?); from=<([^>]*)> to=<([^>]*)> proto=\S+(?: helo=<.*>)?(?: sasl_(?:method|username|sender)=.*)?$`) inboundClientRe = regexp.MustCompile(`^\S+ postfix/inbound/smtpd\[\d+\]: ([0-9A-Za-z]+): client=`) // A queue id stops being port 25's when the message is queued (qmgr takes - // it: the filter let it through) or when another listener opens a message - // under the same id - Postfix reuses ids. Postfix 3.7.11, + // it: the filter let it through), when cleanup removes the queue file of + // a message it did not queue, or when another listener opens a message + // under the same id - Postfix reuses ids. Postfix 3.10.13, // src/qmgr/qmgr_message.c, qmgr_message_read() calls opened() // (src/global/opened.c: "%s: from=<%s>, size=%ld, nrcpt=%d") for a queue - // file cleanup has committed, each time it enters the active queue. - inboundForgetRe = regexp.MustCompile(`^\S+ postfix/(?:qmgr\[\d+\]: ([0-9A-Za-z]+): from=|(?:[a-z0-9-]+/)?smtpd\[\d+\]: ([0-9A-Za-z]+): client=)`) + // file cleanup has committed, each time it enters the active queue; + // src/cleanup/cleanup_api.c, cleanup_flush(): "%s: removed (%s)", with + // "canceled" or "discarded", once the queue file of a refused, discarded + // or unfinished message is gone (new in 3.10, HISTORY 20241112) - after + // the verdict line, which cleanup_milter_inspect() writes earlier in + // cleanup_flush(). Without it, the id of a message whose client went away + // in DATA stayed until inboundQueueIDs pushed it out. + inboundForgetRe = regexp.MustCompile(`^\S+ postfix/(?:qmgr\[\d+\]: ([0-9A-Za-z]+): from=|(?:[a-z0-9-]+/)?smtpd\[\d+\]: ([0-9A-Za-z]+): client=|cleanup\[\d+\]: ([0-9A-Za-z]+): removed \()`) ) // A row holds one line of reason; Postfix's reply text carries the far end's @@ -95,7 +107,7 @@ const inboundQueueIDs = 4096 // journal cut back to its row cap, without waiting for the six-hourly prune. // Refusals are written at the rate strangers send RCPT TO. Postfix's one // limit on that, smtpd_client_recipient_rate_limit, is per client address and -// off by default (postconf(5), Postfix 3.7.11) and not set here, so between +// off by default (postconf(5), Postfix 3.10.13) and not set here, so between // two prunes the journal would otherwise grow as fast as someone cared to // make it. const inboundPruneEvery = 500 @@ -140,7 +152,7 @@ func (j *inboundJournal) handle(line string, now time.Time, catchingUp bool) { return } if m := inboundForgetRe.FindStringSubmatch(rest); m != nil { - delete(j.port25, m[1]+m[2]) + delete(j.port25, m[1]+m[2]+m[3]) return } m := inboundRejectRe.FindStringSubmatch(rest) @@ -150,7 +162,7 @@ func (j *inboundJournal) handle(line string, now time.Time, catchingUp bool) { program, queueID, kind, clientIP, reason, from, to := m[1], m[2], m[3], m[4], m[5], m[6], m[7] // The sender's address is his own text too, and with a quoted local part it // may hold "> to=<…> proto=…" - which would end the sender early and supply - // the recipient (Postfix 3.7.11 logs addresses in external form, the local + // the recipient (Postfix 3.10.13 logs addresses in external form, the local // part quoted when it holds spaces or specials: postconf(5) // info_log_address_format, src/global/info_log_addr_form.c). Such a line // shows it: the sender is left with an unclosed quote. An address with a @@ -198,7 +210,7 @@ func (j *inboundJournal) handle(line string, now time.Time, catchingUp bool) { e.Engine = store.SpamEnginePostfix } // "450 4.7.1 …" from Postfix, "4.7.1 …" from a milter: either way a reply - // that starts with 4 asks the sender to come back. Postfix 3.7.11: + // that starts with 4 asks the sender to come back. Postfix 3.10.13: // smtpd_check_reject() logs smtpd's whole "%d %s ..." reply through // log_whatsup(); cleanup_milter_apply() logs a // milter's from its fifth byte (text = resp + 4), so the enhanced code @@ -263,7 +275,7 @@ func printable(s string, max int) string { // tailer got to it: after the panel was down the two are hours apart. The // syslog format carries neither year nor zone - the year is taken from now // (a stamp that would then lie in the future is last year's), the zone is the -// container's. Postfix 3.7.11, src/util/msg_logger.c, msg_logger_print(): the +// container's. Postfix 3.10.13, src/util/msg_logger.c, msg_logger_print(): the // stamp is made by the process that logs, with localtime(), and postlogd // writes it unchanged (src/postlogd/postlogd.c, postlogd_service()); that // process has the container's zone (TZ is in import_environment, diff --git a/internal/logtail/inbound_test.go b/internal/logtail/inbound_test.go index acddfd7..23215bd 100644 --- a/internal/logtail/inbound_test.go +++ b/internal/logtail/inbound_test.go @@ -53,8 +53,9 @@ func inboundFixture() (*inboundJournal, *fakeInboundStore) { var tailNow = time.Date(2026, 10, 11, 18, 0, 0, 0, time.UTC) // The mail.log lines in this file are composed by hand, after -// src/smtpd/smtpd_check.c log_whatsup() and src/cleanup/cleanup_milter.c -// cleanup_milter_apply() (Postfix 3.7.11), with the stamp Postfix 3.7 writes +// src/smtpd/smtpd_check.c log_whatsup(), src/cleanup/cleanup_milter.c +// cleanup_milter_apply() and src/cleanup/cleanup_api.c cleanup_flush() +// (Postfix 3.10.13), with the stamp Postfix 3.10 writes // (src/util/msg_logger.c, "%b %d %H:%M:%S"). const ( head = "Oct 11 12:00:05 mail " @@ -89,6 +90,26 @@ func TestInboundTempfailByPostfix(t *testing.T) { } } +// Postfix 3.10's log_whatsup() appends " sasl_method=%s", " sasl_username=%s" +// and " sasl_sender=%s" after the HELO name for a session that has them +// (src/smtpd/smtpd_check.c; RELEASE_NOTES, [Incompat 20241023]). Port 25 has +// SASL off, so it never writes them there; a line that had them, with the +// HELO name or without it, is still read for what comes before. Composed +// from that format. +func TestInboundRejectTakesTheSASLTail(t *testing.T) { + for name, line := range map[string]string{ + "after the HELO name": head + "postfix/inbound/smtpd[41]: NOQUEUE: reject: RCPT from mx.example.net[203.0.113.9]: 550 5.1.1 : Recipient address rejected: User unknown in relay recipient table; from= to= proto=ESMTP helo= sasl_method=PLAIN sasl_username=u sasl_sender=s", + "with no HELO name": head + "postfix/inbound/smtpd[41]: NOQUEUE: reject: RCPT from mx.example.net[203.0.113.9]: 550 5.1.1 : Recipient address rejected: User unknown in relay recipient table; from= to= proto=ESMTP sasl_method=PLAIN sasl_username=u", + } { + j, st := inboundFixture() + j.handle(line, tailNow, false) + if len(st.rows) != 1 || st.rows[0].From != "a@example.net" || st.rows[0].To != "nobody@in.example.com" || + st.rows[0].Detail != "550 5.1.1 : Recipient address rejected: User unknown in relay recipient table" { + t.Errorf("%s: rows = %+v", name, st.rows) + } + } +} + // The filter's verdict is written by cleanup, under one name for every // listener: it is the journal's only for a queue id port 25 opened. func TestInboundMilterVerdictNeedsAPort25QueueID(t *testing.T) { @@ -137,7 +158,7 @@ func TestInboundIgnores(t *testing.T) { for name, line := range map[string]string{ "another domain (relay access denied)": head + "postfix/inbound/smtpd[41]: NOQUEUE: reject: RCPT from x[203.0.113.9]: 554 5.7.1 : Relay access denied; from= to= proto=ESMTP helo=", "no recipient (size limit at MAIL)": head + "postfix/inbound/smtpd[41]: NOQUEUE: reject: MAIL from x[203.0.113.9]: 552 5.3.4 Message size exceeds fixed limit; proto=ESMTP helo=", - "a submission port": head + "postfix/smtpd[20]: NOQUEUE: reject: RCPT from x[203.0.113.9]: 553 5.7.1 : Sender address rejected: not owned by user app; from= to= proto=ESMTP helo=", + "a submission port": head + "postfix/smtpd[20]: NOQUEUE: reject: RCPT from x[203.0.113.9]: 553 5.7.1 : Sender address rejected: not owned by user app; from= to= proto=ESMTP helo= sasl_method=PLAIN sasl_username=app", "a delivery line": head + "postfix/smtp[26]: 41E862C00D9E: to=, relay=mx[203.0.113.9]:25, dsn=2.0.0, status=sent (250 OK)", "a connect line": head + "postfix/inbound/smtpd[41]: connect from mx.example.net[203.0.113.9]", // The HELO name is the sender's; a line cannot be started from inside one. @@ -176,11 +197,28 @@ func TestInboundLookupFailureLeavesNoRow(t *testing.T) { // A message the filter holds goes on to the inbound journal milter, which // writes its row; a second one from the log would be the same message twice. func TestInboundHoldIsTheMiltersRow(t *testing.T) { - j, st := inboundFixture() - j.handle(head+"postfix/inbound/smtpd[41]: Q1: client=mx.example.net[203.0.113.9]", tailNow, false) - j.handle(head+"postfix/cleanup[52]: Q1: milter-hold: END-OF-MESSAGE from mx.example.net[203.0.113.9]: milter triggers HOLD action; from= to= proto=ESMTP helo=", tailNow, false) - if len(st.rows) != 0 { - t.Errorf("rows = %+v", st.rows) + // Postfix 3.10.13 logs a quarantine with the milter's reason, or + // "default_action" when milter_default_action asked for it, where 3.7 + // wrote "milter triggers HOLD action" (src/cleanup/cleanup_milter.c, + // cleanup_milter_apply(), and src/smtpd/smtpd.c, check_milter_reply(): + // text = reply[1] ? reply + 1 : "milter triggers HOLD action"; + // src/milter/milter8.c, SMFIR_QUARANTINE "H%.*s" and milter8_comm_error() + // "Hdefault_action"). Whatever the reason says, the line is a hold. + // Composed from those formats. + for _, hold := range []string{ + "postfix/cleanup[52]: Q1: milter-hold: END-OF-MESSAGE from mx.example.net[203.0.113.9]: milter triggers HOLD action; from= to= proto=ESMTP helo=", + "postfix/cleanup[52]: Q1: milter-hold: END-OF-MESSAGE from mx.example.net[203.0.113.9]: Gtube pattern; from= to= proto=ESMTP helo=", + "postfix/cleanup[52]: Q1: milter-hold: END-OF-MESSAGE from mx.example.net[203.0.113.9]: default_action; from= to= proto=ESMTP helo=", + // A reason that spells a reject is still the reason of a hold. + "postfix/cleanup[52]: Q1: milter-hold: END-OF-MESSAGE from mx.example.net[203.0.113.9]: x: milter-reject: END-OF-MESSAGE from y[192.0.2.1]: 5.7.1 no; from= to= proto=ESMTP helo=", + "postfix/inbound/smtpd[41]: Q1: milter-hold: RCPT from mx.example.net[203.0.113.9]: held for review; from= to= proto=ESMTP helo=", + } { + j, st := inboundFixture() + j.handle(head+"postfix/inbound/smtpd[41]: Q1: client=mx.example.net[203.0.113.9]", tailNow, false) + j.handle(head+hold, tailNow, false) + if len(st.rows) != 0 { + t.Errorf("%s: rows = %+v", hold, st.rows) + } } } @@ -216,13 +254,18 @@ func TestInboundHeloCannotWriteTheRow(t *testing.T) { } } -// A queue id is port 25's only until the message is queued or the id is -// opened again by another listener - Postfix reuses ids. +// A queue id is port 25's only until the message is queued, cleanup removes +// it unqueued, or the id is opened again by another listener - Postfix reuses +// ids. func TestInboundQueueIDIsForgotten(t *testing.T) { const verdict = "postfix/cleanup[52]: Q1: milter-reject: END-OF-MESSAGE from app[203.0.113.4]: 4.7.1 Service unavailable - try again later; from= to= proto=ESMTP helo=" for name, forget := range map[string]string{ "queued": "postfix/qmgr[58]: Q1: from=, size=600, nrcpt=1 (queue active)", "reused by the submission port": "postfix/smtpd[20]: Q1: client=app.example.com[203.0.113.4], sasl_method=PLAIN, sasl_username=app", + // Postfix 3.10's cleanup_flush(): "%s: removed (%s)", for a message + // whose client went away in DATA, or that was discarded. + "canceled by cleanup": "postfix/cleanup[52]: Q1: removed (canceled)", + "discarded by cleanup": "postfix/cleanup[52]: Q1: removed (discarded)", } { j, st := inboundFixture() j.handle(head+"postfix/inbound/smtpd[41]: Q1: client=mx.example.net[203.0.113.9]", tailNow, false) diff --git a/internal/logtail/lines.go b/internal/logtail/lines.go index 83f2c17..6de269c 100644 --- a/internal/logtail/lines.go +++ b/internal/logtail/lines.go @@ -3,6 +3,7 @@ package logtail import ( "log" "regexp" + "strings" "time" "github.com/mixeme/selfpost/internal/store" @@ -28,7 +29,7 @@ import ( // enable_long_queue_ids, digits and letters without vowels; neither word is // either. // -// Postfix 3.7.11, src/global/mail_queue.c, mail_queue_enter(), with +// Postfix 3.10.13, src/global/mail_queue.c, mail_queue_enter(), with // src/global/mail_queue.h MQID_*: a short id is "%05X" of the microseconds // and "%lX" of the inode (6 to 21 characters); a long one is the seconds (6+) // and microseconds (4) in src/global/safe_ultostr.c's vowel-less alphabet, 'z', @@ -38,7 +39,7 @@ var queueLineRe = regexp.MustCompile(`^\S+ ([^\s\[]+)\[\d+\]: ([0-9A-F]{6,24}|[0 // smtpdProgramRe tells an smtpd from the other programs, whatever listener it // serves: "postfix/smtpd", "postfix/submission/smtpd", "postfix/inbound/smtpd". -// Postfix 3.7.11, src/global/mail_task.c, mail_task(): the tag is +// Postfix 3.10.13, src/global/mail_task.c, mail_task(): the tag is // "/"; a master.cf "-o syslog_name=" reaches it // through mail_params_init() (src/global/mail_params.c) before // single_server_main() calls maillog_client_init() again. @@ -55,7 +56,7 @@ const ( // logged when it arrives and its row is written at end-of-message; an hour // is a message body sent slowly. Postfix sets no such bound: smtpd_timeout // limits each read, not the message (postconf(5) smtpd_timeout, - // smtpd_per_request_deadline, Postfix 3.7.11) - assumptions A-34. + // smtpd_per_request_deadline, Postfix 3.10.13) - assumptions A-34. lineGrace = time.Hour // lineBatch is how many lines are held before they are written without @@ -80,13 +81,21 @@ type lineKeeper struct { // port25 holds the queue ids the port-25 smtpd opened: mail offered by // strangers, which never gets a send-log row. Its lines are not kept - what // a stranger can make Postfix log is not something to store on his say-so - // and prune later. An id leaves when qmgr removes the message or when - // another listener opens a message under it (Postfix reuses ids). - // Postfix 3.7.11: smtpd writes ": client=..." when it opens the queue + // and prune later. An id leaves when qmgr removes the message, when + // cleanup removes a message it did not queue, or when another listener + // opens a message under it (Postfix reuses ids). + // Postfix 3.10.13: smtpd writes ": client=..." when it opens the queue // file (src/smtpd/smtpd.c, mail_open_stream()); qmgr writes ": removed" // when it deletes it (src/qmgr/qmgr_active.c, qmgr_active_done_3_generic()); - // short ids repeat (postconf(5) enable_long_queue_ids: the long ones are - // the "non-repeating" kind). + // cleanup writes ": removed (canceled)" or ": removed (discarded)" + // when it deletes the queue file of a message it refused, discarded or + // was not given in full (src/cleanup/cleanup_api.c, cleanup_flush(), + // msg_info("%s: removed (%s)"), new in 3.10: HISTORY 20241112), after its + // milter verdict line (cleanup_milter_inspect() runs earlier in + // cleanup_flush()) and after smtpd's line about a connection lost in DATA + // (src/smtpd/smtpd.c, smtpd_proto(), logs it before mail_reset() closes the + // cleanup stream); short ids repeat (postconf(5) enable_long_queue_ids: + // the long ones are the "non-repeating" kind). port25 map[string]struct{} } @@ -121,7 +130,8 @@ func (k *lineKeeper) add(line string, now time.Time, catchingUp bool) { } } if _, inbound := k.port25[queueID]; inbound { - if program == "postfix/qmgr" && text == "removed" { + if program == "postfix/qmgr" && text == "removed" || + program == "postfix/cleanup" && strings.HasPrefix(text, "removed (") { delete(k.port25, queueID) } return diff --git a/internal/logtail/lines_test.go b/internal/logtail/lines_test.go index 152bd66..d2c7d6a 100644 --- a/internal/logtail/lines_test.go +++ b/internal/logtail/lines_test.go @@ -24,6 +24,15 @@ func TestQueueLineHead(t *testing.T) { {"port-25 smtpd", "mail postfix/inbound/smtpd[41]: 4A1B2C3D: client=mx.example.net[203.0.113.9]", "4A1B2C3D"}, {"cleanup", "mail postfix/cleanup[52]: 41E862C00D9E: message-id=<1@example.com>", "41E862C00D9E"}, {"qmgr removes it", "mail postfix/qmgr[10]: 41E862C00D9E: removed", "41E862C00D9E"}, + // Postfix 3.10.13: src/cleanup/cleanup_api.c, cleanup_flush(), "%s: + // removed (%s)"; src/smtpd/smtpd.c, smtpd_proto(), "%s: %s after %s + // (%lu bytes) from %s" with the queue id or "NOQUEUE". + {"cleanup removes an unqueued message", "mail postfix/cleanup[52]: 41E862C00D9E: removed (canceled)", "41E862C00D9E"}, + {"a connection lost in DATA", "mail postfix/submission/smtpd[10]: 41E862C00D9E: lost connection after DATA (5420 bytes) from unknown[203.0.113.7]", "41E862C00D9E"}, + {"a connection lost before a queue id", "mail postfix/smtpd[10]: NOQUEUE: lost connection after EHLO from unknown[203.0.113.7]", ""}, + // The same function's "%s: %s after %s from %s" for the other reasons. + {"too many errors with a queue id", "mail postfix/submission/smtpd[10]: 41E862C00D9E: too many errors after RCPT from unknown[203.0.113.7]", "41E862C00D9E"}, + {"a timeout before a queue id", "mail postfix/smtpd[10]: NOQUEUE: timeout after MAIL from unknown[203.0.113.7]", ""}, {"a bounce is sent: the id at the head, not the new message's", "mail postfix/bounce[30]: 41E862C00D9E: sender non-delivery notification: 5F00AA12", "41E862C00D9E"}, {"long queue id", "mail postfix/qmgr[10]: 3sJ8Xh2Cgqz4Xy1: removed", "3sJ8Xh2Cgqz4Xy1"}, {"refused before queueing", "mail postfix/inbound/smtpd[41]: NOQUEUE: reject: RCPT from mx.example.net[203.0.113.9]: 550 5.1.1 : Recipient address rejected; from= to= proto=ESMTP helo=", ""}, @@ -162,6 +171,38 @@ func TestLineKeeperLeavesOutPort25Mail(t *testing.T) { } } +// A port-25 message that is never queued - refused at end-of-message, or its +// client gone in DATA - ends with cleanup's "removed (canceled)" rather than +// qmgr's "removed", and its id is anybody's again from there. Postfix 3.10.13: +// src/smtpd/smtpd.c, smtpd_proto(), logs the lost connection before +// mail_reset() closes the cleanup stream; src/cleanup/cleanup_api.c, +// cleanup_flush(), logs "%s: removed (%s)" after the milter verdict. The +// lines are composed from those formats. +func TestLineKeeperForgetsPort25AtCleanupRemoval(t *testing.T) { + cs := &captureStore{} + k := newLineKeeper(cs) + now := time.Date(2026, 10, 12, 18, 0, 0, 0, time.UTC) + + const bounce = "host postfix/cleanup[52]: 4A1B2C3D: message-id=<20261012180000.4A1B2C3D@mail.example.com>" + for _, line := range []string{ + "host postfix/inbound/smtpd[41]: 4A1B2C3D: client=mx.example.net[198.51.100.9]", + "host postfix/inbound/smtpd[41]: 4A1B2C3D: lost connection after DATA (5420 bytes) from mx.example.net[198.51.100.9]", + "host postfix/cleanup[52]: 4A1B2C3D: removed (canceled)", + // The id comes round for a bounce Postfix generates, which has no smtpd. + bounce, + } { + k.add(line, now, false) + } + k.flush() + + if got := cs.kept(); len(got) != 1 || got[0] != "4A1B2C3D|"+bounce { + t.Errorf("kept %v, want only the bounce's line", got) + } + if len(k.port25) != 0 { + t.Errorf("port-25 ids still remembered: %v", k.port25) + } +} + // The tailer keeps queue-id lines as it follows the log, and after an unclean // stop - its saved place lies before lines it had already read - it does not // keep them a second time. diff --git a/internal/logtail/logtail.go b/internal/logtail/logtail.go index 8e59d6d..f279d25 100644 --- a/internal/logtail/logtail.go +++ b/internal/logtail/logtail.go @@ -62,7 +62,7 @@ func RetentionInterval() time.Duration { } // queueIDs lists the messages Postfix currently holds, for the reconcile sweep. -// It is a var so tests can answer without a running Postfix. Postfix 3.7.11, +// It is a var so tests can answer without a running Postfix. Postfix 3.10.13, // src/showq/showq.c, showq_service(): the listing covers the maildrop, // active, incoming, deferred and hold queues. var queueIDs = postfix.QueueIDs @@ -91,7 +91,7 @@ type RetentionDays func() int // postfix/smtp[26]: 41E862C00D9E: to=, relay=…, dsn=2.0.0, status=sent (250 OK) // // The ": to=, …, status=" shape is the per-recipient -// result line: Postfix 3.7.11, src/global/log_adhoc.c, log_adhoc() writes +// result line: Postfix 3.10.13, src/global/log_adhoc.c, log_adhoc() writes // "%s: to=<%s>", ", orig_to=<%s>" (only when the address was rewritten), // ", relay=%s", then ", dsn=%s, status=%s (%s)". It is called from sent(), // defer_append() and bounce_append() in whichever process decides - the @@ -150,7 +150,7 @@ func parseDelivery(line string) (queueID, recipient, status string, ok bool) { // // postfix/qmgr[93]: 41E862C00D9E: from=, status=expired, returned to sender // -// Postfix 3.7.11, src/qmgr/qmgr_active.c, qmgr_active_done_25_generic(): +// Postfix 3.10.13, src/qmgr/qmgr_active.c, qmgr_active_done_25_generic(): // msg_info("%s: from=<%s>, status=%s, returned to sender") with "expired" // once the queue lifetime has passed, "force-expired" after postsuper -e. // It is the only line about the expiry, one per message, and no recipient is @@ -253,7 +253,7 @@ func reconcileLoop(ctx context.Context, st StatusStore, caughtUp <-chan struct{} } // A message Postfix still works on stays queued until its first retry after - // the queue lifetime, which can be a whole backoff later. Postfix 3.7.11, + // the queue lifetime, which can be a whole backoff later. Postfix 3.10.13, // src/qmgr/qmgr_active.c: expiry is decided when an attempt ends // (qmgr_active_done_25_generic()), and a deferred message waits at most // maximal_backoff_time (qmgr_active_done_3_generic()). The deferred queue @@ -361,7 +361,7 @@ func reconcile(st StatusStore, cutoff time.Time, path string) { // message's expiry line. A deferred line is not one: a deferred row has one // by definition. Rows reach the sweep minutes after they were written, so // the end of those two files is where such a line would be. -// logrotate 3.21.0, logrotate.c, prerotateSingleLog(): the first rotated +// logrotate 3.22.0, logrotate.c, prerotateSingleLog(): the first rotated // name is ".1", without the compression suffix under delaycompress // (build/logrotate-mail.conf). func resultInLog(path string, row store.QueuedDelivery) bool { @@ -556,11 +556,11 @@ func QueueLines(path, queueID string, n int) ([]string, error) { // mentionsQueueID reports whether a mail.log line is about queueID. Postfix // writes the id followed by a colon ("… postfix/smtp[26]: 41E862C00D9E: to=…"; -// Postfix 3.7.11 "%s: ..." in log_adhoc(), smtpd's "%s: client=", qmgr's -// "%s: removed"), -// and the match is anchored on the character before it so a shorter id is not -// found inside a longer one — queue ids are hexadecimal, and one being the tail -// of another is ordinary, not unlikely. +// Postfix 3.10.13 "%s: ..." in log_adhoc(), smtpd's "%s: client=", qmgr's +// "%s: removed", cleanup's "%s: removed (%s)"), and the match is anchored on +// the character before it so a shorter id is not found inside a longer one — +// queue ids are hexadecimal, and one being the tail of another is ordinary, +// not unlikely. func mentionsQueueID(line, queueID string) bool { needle := queueID + ":" for i := 0; i <= len(line)-len(needle); { @@ -582,16 +582,17 @@ func isQueueIDByte(b byte) bool { } // syslogStampRe is the timestamp at the head of a mail.log line (maillog_file -// in build/postfix-config.sh), the one format Postfix 3.7 writes. Postfix 3.7.11, -// src/util/msg_logger.c, msg_logger_print(): every record is stamped +// in build/postfix-config.sh), the one format Postfix 3.10 writes. Postfix +// 3.10.13, src/util/msg_logger.c, msg_logger_print(): every record is stamped // strftime("%b %d %H:%M:%S ") in the logging process's local time (the day // zero-padded), then the first label of the host name, and postlogd writes it // as received (src/postlogd/postlogd.c, postlogd_service()). It carries no // year and no zone, so the stamp shown is a wall clock and nothing more, // which is all this column claims to be. An RFC 3339 stamp once matched // here too, for a maillog_file_format parameter said to arrive in Postfix -// 3.9; there is none in 3.7.11, nor in the 3.12 snapshot of 2026-09-25 -// (proto/postconf.proto; msg_logger.c keeps the strftime) - assumptions A-12. +// 3.9; there is none in 3.7.11, nor in 3.10.13, nor in the 3.12 snapshot of +// 2026-09-25 (proto/postconf.proto; msg_logger.c keeps the strftime) - +// assumptions A-12. var syslogStampRe = regexp.MustCompile(`^([A-Z][a-z]{2}\s+\d{1,2} \d{2}:\d{2}:\d{2})\s`) // SplitTimestamp separates the timestamp at the head of a mail.log line from @@ -615,7 +616,7 @@ func SplitTimestamp(line string) (stamp, rest string) { // first ever run); it reopens the file when it is rotated, so nothing is missed. // The inode changes under logrotate's create, which build/logrotate-mail.conf // uses; a file shorter than the read position is taken as truncation. -// logrotate 3.21.0, logrotate.c, rotateSingleLog(): without copytruncate the +// logrotate 3.22.0, logrotate.c, rotateSingleLog(): without copytruncate the // file is rename()d and, with create, a new one made by createOutputFile(); // with copytruncate it is copied and truncated, which that config does not use. // @@ -817,14 +818,14 @@ func follow(ctx context.Context, path string, tr *tracker, caughtUp chan struct{ } // rotatedTail is mail.log just rotated away, still open: postlogd writes to -// it until "postfix reload" makes it exit. Postfix 3.7.11, +// it until "postfix reload" makes it exit. Postfix 3.10.13, // src/postlogd/postlogd.c, pre_jail_init(): maillog_file is opened once per // postlogd process; src/master/master_service.c, master_restart_service(), // closes the service's status pipe at a reload (master_status_cleanup()), // and src/master/dgram_server.c, dgram_server_abort(), exits when it reads // its end - the next postlogd opens the new file. logrotate runs the reload // right after the rename and create (build/logrotate-mail.conf, postrotate; -// logrotate 3.21.0, logrotate.c, rotateLogSet(): the postrotate script runs +// logrotate 3.22.0, logrotate.c, rotateLogSet(): the postrotate script runs // right after rotateSingleLog() has renamed and created), so the gap is short; // rotatedGrace is far longer than it. type rotatedTail struct { @@ -845,7 +846,7 @@ const rotatedGrace = time.Minute // lost to a restart and lands on the days it covered. A stretch shorter than // blindMin is not counted: a rotation leaves the name missing for a tick // between rename and create, and that loses nothing - the new file is read -// from its start. logrotate 3.21.0, logrotate.c, rotateSingleLog(): the +// from its start. logrotate 3.22.0, logrotate.c, rotateSingleLog(): the // rename() and createOutputFile() follow each other in one pass, with no // script or wait between them (postrotate runs after). type blindness struct { diff --git a/internal/logtail/logtail_test.go b/internal/logtail/logtail_test.go index 8db4228..4c0a897 100644 --- a/internal/logtail/logtail_test.go +++ b/internal/logtail/logtail_test.go @@ -15,12 +15,12 @@ import ( ) // The mail.log lines in this file are composed by hand, after the formats the -// code cites (Postfix 3.7.11, src/global/log_adhoc.c and the others), unless a +// code cites (Postfix 3.10.13, src/global/log_adhoc.c and the others), unless a // comment says where one was seen; some carry no stamp, which the parsers // take as they come. One shape Postfix does not write here: opendkim lines in // mail.log (OpenDKIM logs through syslog(3), to the container log - build/opendkim.conf). -// qmgr's expiry line, as Postfix 3.7.11 writes it (src/qmgr/qmgr_active.c, +// qmgr's expiry line, as Postfix 3.10.13 writes it (src/qmgr/qmgr_active.c, // qmgr_active_done_25_generic(): "%s: from=<%s>, status=%s, returned to // sender"); composed by hand after that format. The sender is the client's // text, so a line only resembles one when it ends as qmgr's does. @@ -36,6 +36,11 @@ func TestParseExpiry(t *testing.T) { // qmgr's ordinary from= line, whose sender carries the text. {"Oct 14 09:01:02 relay postfix/qmgr[93]: 41E862C00D9E: from=, status=expired, returned to sender>, size=500, nrcpt=1 (queue active)", ""}, {"Oct 14 09:01:02 relay postfix/qmgr[93]: 41E862C00D9E: removed", ""}, + // cleanup's removal of a message it did not queue (Postfix 3.10.13, + // src/cleanup/cleanup_api.c, cleanup_flush(): "%s: removed (%s)") + // says nothing of what became of a queued one. + {"Oct 14 09:01:02 relay postfix/cleanup[52]: 41E862C00D9E: removed (canceled)", ""}, + {"Oct 14 09:01:02 relay postfix/cleanup[52]: 41E862C00D9E: removed (discarded)", ""}, } { id, ok := parseExpiry(c.line) if ok != (c.id != "") || id != c.id { @@ -194,7 +199,8 @@ func TestParseDelivery(t *testing.T) { status: store.StatusSent, }, { - // The stamp Postfix 3.7's postlogd writes, as on the stand. + // The stamp postlogd writes, as on the stand (Postfix 3.7; + // 3.10.13's msg_logger_print() keeps the format). name: "syslog stamp", line: "Oct 14 11:05:51 mail postfix/smtp[26]: 41E862C00D9E: to=, relay=x, dsn=2.0.0, status=sent (250 OK)", wantOK: true, @@ -221,6 +227,22 @@ func TestParseDelivery(t *testing.T) { line: "Oct 14 11:05:51 mail postfix/smtpd[10]: 5B4A2C1D3E: client=unknown[203.0.113.7], sasl_username=41E862C00D9E: to=, status=sent", wantOK: false, }, + { + // Postfix 3.10.13, src/cleanup/cleanup_api.c, cleanup_flush(): + // "%s: removed (%s)" for a transaction cleanup did not queue. It + // is no result for any recipient. Composed from that format. + name: "cleanup's removed (canceled) is not one", + line: "Oct 14 11:05:51 mail postfix/cleanup[52]: 41E862C00D9E: removed (canceled)", + wantOK: false, + }, + { + // Postfix 3.10.13, src/smtpd/smtpd.c, smtpd_proto(): "%s: %s after + // %s (%lu bytes) from %s", the queue id first since 3.10. Composed + // from that format. + name: "smtpd's lost connection with a queue id is not one", + line: "Oct 14 11:05:51 mail postfix/submission/smtpd[10]: 41E862C00D9E: lost connection after DATA (5420 bytes) from unknown[203.0.113.7]", + wantOK: false, + }, } for _, c := range cases { t.Run(c.name, func(t *testing.T) { @@ -920,7 +942,7 @@ func TestSplitTimestamp(t *testing.T) { stamp, rest string }{ { - // An RFC 3339 stamp is not one Postfix 3.7 writes (assumptions + // An RFC 3339 stamp is not one Postfix 3.10 writes (assumptions // A-12): it is not taken off. name: "RFC 3339 is not a stamp Postfix writes", line: "2026-08-03T05:15:52.219218+00:00 mail postfix/qmgr[10]: 4A1B2C3D: removed", diff --git a/internal/logtail/main_test.go b/internal/logtail/main_test.go index af6d220..f39c313 100644 --- a/internal/logtail/main_test.go +++ b/internal/logtail/main_test.go @@ -7,7 +7,7 @@ import ( ) // TestMain reads mail.log stamps in UTC, whatever the zone of the machine the -// tests run on: Postfix 3.7 writes "%b %d %H:%M:%S" in the logging process's +// tests run on: Postfix 3.10 writes "%b %d %H:%M:%S" in the logging process's // local time (src/util/msg_logger.c, msg_logger_print()) and lineTime reads it // in time.Local, which in the container is the zone the fixtures are written // in. diff --git a/internal/logtail/offset.go b/internal/logtail/offset.go index 9fe0ac5..169e295 100644 --- a/internal/logtail/offset.go +++ b/internal/logtail/offset.go @@ -17,7 +17,7 @@ import ( const ( // fingerprintSize is how many bytes from the head of the log identify it. - // Postfix writes a timestamped line per event (Postfix 3.7.11, + // Postfix writes a timestamped line per event (Postfix 3.10.13, // src/util/msg_logger.c, msg_logger_print(): "%b %d %H:%M:%S" on every // record), so the first 512 bytes are // effectively unique per log generation — enough to tell "the file we were @@ -130,7 +130,7 @@ func (t *tracker) resume(f *os.File) int64 { // readRotated reads, when the log was rotated while the panel was down, the // rest of the file the saved place belongs to: logrotate renamed it to -// mail.log.1 (delaycompress leaves it plain - logrotate 3.21.0, logrotate.c, +// mail.log.1 (delaycompress leaves it plain - logrotate 3.22.0, logrotate.c, // prerotateSingleLog()), and what Postfix wrote there // between the panel's stop and the rotation - deliveries, refusals, deferrals - // was otherwise never read (audit 2026-10-12, stage 2b). It is read from the diff --git a/internal/logtail/refusals.go b/internal/logtail/refusals.go index c49280c..1870311 100644 --- a/internal/logtail/refusals.go +++ b/internal/logtail/refusals.go @@ -18,14 +18,17 @@ import ( // // postfix/smtpd[1149]: NOQUEUE: milter-reject: MAIL from unknown[10.77.0.1]: 451 4.7.1 Service unavailable - try again later; from= proto=ESMTP helo= sasl_username= // -// Postfix 3.7.11, src/smtpd/smtpd.c, check_milter_reply(): "%s: %s: %s from +// Postfix 3.10.13, src/smtpd/smtpd.c, check_milter_reply(): "%s: %s: %s from // %s: %s;%s" with "NOQUEUE" before the queue file is opened, and // smtpd_whatsup() for the rest: " from=<..>", " to=<..>", " proto=..", // " helo=<..>", then " sasl_username=<..>" when the session authenticated -// (assumptions A-20, A-21). The login is read at the end of the line, where -// Postfix puts it last; a line without one is no application's refusal - the -// journal milter refuses at MAIL FROM only an authenticated session -// (milter/ratelimit.go). The HELO name before it is the client's text and can +// (assumptions A-20, A-21). smtpd's own refusals are another function, +// src/smtpd/smtpd_check.c, log_whatsup(), whose line says "reject:", not +// "milter-reject:", and since 3.10 ends in " sasl_method=%s sasl_username=%s" +// without the brackets; it is not one of these. The login is read at the end +// of the line, where Postfix puts it last; a line without one is no +// application's refusal - the journal milter refuses at MAIL FROM only an +// authenticated session (milter/ratelimit.go). The HELO name before it is the client's text and can // end in what looks like the field, but only in a session that has no login, // and so no refused row to take the line; a row must also be of the line's // client address and sender, within minutes. @@ -52,7 +55,7 @@ type RefusalStore interface { // The From: line as the check printed it runs to the last " - " before the // first semicolon; the reason after it holds neither, and the check prints no // ";" of the client's (sender.go printable), so that semicolon is Postfix's -// and what follows it - the envelope - is Postfix's own text. Postfix 3.7.11, +// and what follows it - the envelope - is Postfix's own text. Postfix 3.10.13, // src/cleanup/cleanup_milter.c, cleanup_milter_apply(): "%s: %s: %s from // %s[%s]: %s;" with the reply from its fifth byte (the "550 " dropped), then // " from=<..>", " to=<..>" - the last recipient only (assumptions A-15). @@ -60,7 +63,7 @@ var fromRefusalRe = regexp.MustCompile(`^\S+ postfix/(?:[a-z0-9-]+/)?cleanup\[\d // clientLoginRe is smtpd's line naming the login a queue id was submitted // under: ": client=host[addr], sasl_method=PLAIN, sasl_username=". -// Postfix 3.7.11, src/smtpd/smtpd.c, mail_open_stream(): ", sasl_sender=.." +// Postfix 3.10.13, src/smtpd/smtpd.c, mail_open_stream(): ", sasl_sender=.." // follows when the client gave AUTH= on MAIL FROM, then ", orig_queue_id=.." // and ", orig_client=.." for XFORWARD; the login ends at the comma (a login // holds none - internal/postfix assertMapSafe; assumptions A-23). @@ -148,7 +151,7 @@ func (r *refusalLines) handle(line string, now time.Time, start Place) { return } // The address in "name[addr]" is smtpd's state->addr, which has no - // "IPv6:" (Postfix 3.7.11, src/smtpd/smtpd_peer.c, smtpd_peer_init(); + // "IPv6:" (Postfix 3.10.13, src/smtpd/smtpd_peer.c, smtpd_peer_init(); // assumptions A-24). ip := net.ParseIP(m[2]) if ip == nil { diff --git a/internal/logtail/refusals_test.go b/internal/logtail/refusals_test.go index 0efcfd5..f8e3769 100644 --- a/internal/logtail/refusals_test.go +++ b/internal/logtail/refusals_test.go @@ -31,8 +31,9 @@ func (r *refusalProbe) AttachRefusalLine(clientIP, login, from string, _ time.Ti // The mail.log lines in this file are composed by hand, with the e2e stand's // names and addresses (smtpd.c check_milter_reply(), mail_open_stream(); -// cleanup_milter.c cleanup_milter_apply(), Postfix 3.7.11), with the stamp -// Postfix 3.7 writes (src/util/msg_logger.c, "%b %d %H:%M:%S"). +// smtpd_check.c log_whatsup(); cleanup_milter.c cleanup_milter_apply(), +// Postfix 3.10.13), with the stamp Postfix 3.10 writes +// (src/util/msg_logger.c, "%b %d %H:%M:%S"). // // The NOQUEUE line of a refusal at MAIL FROM goes on its row; a line whose // row is not written yet waits and is tried again; port 25's and other @@ -45,7 +46,7 @@ func TestRefusalLines(t *testing.T) { for _, other := range []string{ `Oct 14 12:00:00 mail postfix/inbound/smtpd[372]: NOQUEUE: milter-reject: MAIL from unknown[203.0.113.7]: 451 4.7.1 Service unavailable - try again later; from= proto=ESMTP helo=`, `Oct 14 12:00:00 mail postfix/smtpd[11]: NOQUEUE: milter-reject: CONNECT from unknown[10.77.0.1]: 451 4.7.1 Service unavailable - try again later; proto=SMTP`, - `Oct 14 12:00:00 mail postfix/smtpd[11]: NOQUEUE: reject: RCPT from unknown[10.77.0.1]: 553 5.7.1 : Sender address rejected: not owned by user app1; from= to= proto=ESMTP helo=`, + `Oct 14 12:00:00 mail postfix/smtpd[11]: NOQUEUE: reject: RCPT from unknown[10.77.0.1]: 553 5.7.1 : Sender address rejected: not owned by user app1; from= to= proto=ESMTP helo= sasl_method=PLAIN sasl_username=app1`, // No login: no application's refusal (the panel down refuses every // session, an unauthenticated one too). `Oct 14 12:00:00 mail postfix/smtpd[12]: NOQUEUE: milter-reject: MAIL from unknown[10.77.0.1]: 451 4.7.1 Service unavailable - try again later; from= proto=ESMTP helo=`, diff --git a/internal/logtail/sendercheck.go b/internal/logtail/sendercheck.go index 88699ae..78fcd0a 100644 --- a/internal/logtail/sendercheck.go +++ b/internal/logtail/sendercheck.go @@ -29,7 +29,7 @@ import ( // The reject lines do not: the same words are written when OpenDKIM does not // answer and when the journal milter refuses under a level-2 limit. // -// Postfix 3.7.11, src/milter/milter8.c: milter8_connect() warns "connect to +// Postfix 3.10.13, src/milter/milter8.c: milter8_connect() warns "connect to // Milter service %s: %m" with the milter's configured name; then // milter8_comm_error() answers, for default_action=tempfail, "451 4.7.1 // Service unavailable - try again later" - the very text milter8_event() @@ -50,7 +50,7 @@ import ( // // One cleanup process serves message after message and the first warning // ends the message, so each such warning is a message, counted on its own. -// Postfix 3.7.11, src/milter/milter8.c, milter8_read_resp(): "milter %s: +// Postfix 3.10.13, src/milter/milter8.c, milter8_read_resp(): "milter %s: // can't read %s reply packet header: %m", then milter8_comm_error(), which // closes the milter's stream and leaves it in the error state for the rest of // that message. diff --git a/internal/logtail/sendercheck_test.go b/internal/logtail/sendercheck_test.go index ea94d4b..a1f1d54 100644 --- a/internal/logtail/sendercheck_test.go +++ b/internal/logtail/sendercheck_test.go @@ -39,7 +39,7 @@ func (d *deferredStore) AddSenderCheckDeferred(at time.Time, n int64, place stri // that use the same words - a level-2 refusal of the journal milter, OpenDKIM // not answering. Those - smtpd 1149, 1190 and 1201, the second client of // 1183, smtpd 1300 and the OpenDKIM warning of cleanup 1339 - are composed, -// after src/smtpd/smtpd.c and src/milter/milter8.c (Postfix 3.7.11). +// after src/smtpd/smtpd.c and src/milter/milter8.c (Postfix 3.10.13). func TestDeferredCounter(t *testing.T) { st := &deferredStore{} c := newDeferredCounter(&SenderCheck{Store: st, Socket: "/run/selfpost/sender.sock"}) diff --git a/internal/milter/address_test.go b/internal/milter/address_test.go index ada918e..8d59191 100644 --- a/internal/milter/address_test.go +++ b/internal/milter/address_test.go @@ -31,7 +31,7 @@ func TestCleanAddress(t *testing.T) { // The journal row's recipient has to be the address Postfix later logs in // to=<…>, or the delivery line matches no row and the sweep closes it as // bounced: the local part unquoted, then quoted again only when it is not a -// dot-string (Postfix 3.7.11, src/global/tok822_parse.c and +// dot-string (Postfix 3.10.13, src/global/tok822_parse.c and // quote_822_local.c). Composed by hand after those functions. func TestRecipientAsLogged(t *testing.T) { for _, c := range []struct{ in, want string }{ diff --git a/internal/milter/inbound.go b/internal/milter/inbound.go index b6e5662..9097bdb 100644 --- a/internal/milter/inbound.go +++ b/internal/milter/inbound.go @@ -34,7 +34,7 @@ type InboundStore interface { // Postfix attaches the socket with default_action=accept, so // neither an error here nor the panel being down holds up inbound mail. // -// Postfix 3.7.11, src/milter/milter.c, milter_message(): the milters are asked +// Postfix 3.10.13, src/milter/milter.c, milter_message(): the milters are asked // in turn while the reply is null, so one that refuses is the last asked; // src/milter/milter8.c, milter8_comm_error(): default_action=accept gives no // reply when this milter cannot answer. @@ -91,7 +91,7 @@ func (s *inboundSession) RcptTo(rcpt string, m *milter.Modifier) (milter.Respons // the last one read is its own - when it is up. Whether it was is decided at // end-of-message, by the signature. // -// Postfix 3.7.11, src/milter/milter.c, milter_message(), hands each milter the +// Postfix 3.10.13, src/milter/milter.c, milter_message(), hands each milter the // queue file from the start of the content; src/milter/milter8.c, // milter8_message(), reads it with rec_get(), which follows the pointer // records src/cleanup/cleanup_milter.c, cleanup_add_header(), leaves after @@ -151,7 +151,7 @@ func (s *inboundSession) signed(queueID string) bool { // upstream. "Continue" lets it through as "accept" would, and keeps this // session for the connection's next message. // -// The queue id: Postfix 3.7.11, src/global/mail_params.h, DEF_MILT_EOD_MACROS +// The queue id: Postfix 3.10.13, src/global/mail_params.h, DEF_MILT_EOD_MACROS // "i", answered by src/cleanup/cleanup_milter.c, cleanup_milter_eval(). The // rest as session.Body in milter.go. func (s *inboundSession) Body(m *milter.Modifier) (milter.Response, error) { diff --git a/internal/milter/inflight.go b/internal/milter/inflight.go index 33a8b73..ec466c9 100644 --- a/internal/milter/inflight.go +++ b/internal/milter/inflight.go @@ -7,7 +7,7 @@ import ( // reservationTTL bounds how long a message may stay reserved. A reservation is // released at end-of-message or on ABORT. A client that drops the connection -// mid-transaction still produces ABORT: Postfix 3.7.11, src/smtpd/smtpd.c, +// mid-transaction still produces ABORT: Postfix 3.10.13, src/smtpd/smtpd.c, // smtpd_proto() on a lost connection runs mail_reset(), which calls // milter_abort() (src/milter/milter8.c, milter8_abort(), SMFIC_ABORT) // (assumptions B-G7). What produces neither is smtpd itself dying, or the diff --git a/internal/milter/mailparse.go b/internal/milter/mailparse.go index eee2aab..c90941f 100644 --- a/internal/milter/mailparse.go +++ b/internal/milter/mailparse.go @@ -2,7 +2,7 @@ package milter // The address OpenDKIM signs for, found as OpenDKIM finds it: a port of // libopendkim's dkim_mail_parse() and its tokenizer, from Debian's -// opendkim 2.11.0~beta2-8+deb12u1 (libopendkim/dkim-mailparse.c, identical +// opendkim 2.11.0~beta2-9.1+b1 (libopendkim/dkim-mailparse.c, identical // there to upstream's develop branch), line for line. OpenDKIM calls it on the // value of the message's first From: field (opendkim/opendkim.c, mlfi_eoh(), // dkimf_findheader(dfc, "from", 0): build/opendkim.conf sets no SenderHeaders), diff --git a/internal/milter/milter.go b/internal/milter/milter.go index b317b72..b819b52 100644 --- a/internal/milter/milter.go +++ b/internal/milter/milter.go @@ -13,7 +13,7 @@ // Postfix is configured with default_action=accept for this milter's socket, // so even a crash or hang fails open (architecture.md § Mail path). // -// Postfix 3.7.11, src/milter/milter8.c, milter8_comm_error(): a milter that +// Postfix 3.10.13, src/milter/milter8.c, milter8_comm_error(): a milter that // cannot be reached, or does not answer within milter_*_timeout, gives no // reply with default_action=accept - the message goes on. package milter @@ -55,7 +55,7 @@ type Store interface { // // go-milter v0.4.1, session.go, Process(), case CodeMacro: every macro packet // replaces the map; server.go, Serve(): NewMilter() for each connection. -// Postfix 3.7.11, src/milter/milter8.c, milter8_event(): a macro packet, empty +// Postfix 3.10.13, src/milter/milter8.c, milter8_event(): a macro packet, empty // or not, goes before every event (milter.c, milter_macro_lookup(), never // returns null). type session struct { @@ -69,7 +69,7 @@ type session struct { // clientIP is the client's address. Postfix gives it at Connect, once per // connection, and again with every MAIL FROM as the {client_addr} macro // (milter_mail_macros in build/postfix-config.sh). The second is the one - // that can be relied on: see MailFrom. Postfix 3.7.11, + // that can be relied on: see MailFrom. Postfix 3.10.13, // src/smtpd/smtpd_milter.c, smtpd_milter_eval(): {client_addr} is // state->rfc_addr (see clientAddr). clientIP string @@ -84,7 +84,7 @@ type session struct { // Connect captures the client IP of a connection's first message. It is not // enough on its own - see MailFrom. // -// Postfix 3.7.11, src/smtpd/smtpd.c, smtpd_proto(): milter_conn_event() is +// Postfix 3.10.13, src/smtpd/smtpd.c, smtpd_proto(): milter_conn_event() is // given state->addr, the address without "IPv6:". go-milter v0.4.1, // session.go, case CodeConn: net.ParseIP of it - nil for "unknown" and for a // family that sends no address. @@ -115,7 +115,7 @@ func (s *session) Connect(host, family string, port uint16, addr net.IP, m *milt // // go-milter v0.4.1, session.go, HandleMilterCommands(): after a reply whose // Continue() is false (response.go: accept, discard, reject, tempfail) the -// backend is replaced by NewMilter(). Postfix 3.7.11, src/milter/milter8.c, +// backend is replaced by NewMilter(). Postfix 3.10.13, src/milter/milter8.c, // milter8_conn_event(): CONNECT goes once per milter connection. // {auth_authen}: src/smtpd/smtpd_milter.c, smtpd_milter_eval(), gives // state->sasl_username, null without authentication, and then milter.c, @@ -146,11 +146,11 @@ func (s *session) MailFrom(from string, m *milter.Modifier) (milter.Response, er // RcptTo records each recipient. Postfix calls this once per recipient, which // is what lets the journal keep a separate row per (queue-id, recipient). // -// Postfix 3.7.11, src/smtpd/smtpd.c, rcpt_cmd(): milter_rcpt_event() once per +// Postfix 3.10.13, src/smtpd/smtpd.c, rcpt_cmd(): milter_rcpt_event() once per // RCPT TO; a recipient Postfix refused goes only to a milter that asked for // SMFIP_RCPT_REJ, which Serve's Protocol does not. // A recipient given twice - in any case - is one row: cleanup keeps the first -// and drops the rest (Postfix 3.7.11, src/cleanup/cleanup_out_recipient.c, +// and drops the rest (Postfix 3.10.13, src/cleanup/cleanup_out_recipient.c, // cleanup_out_recipient(), been_here() on a table made with BH_FLAG_FOLD in // cleanup_state.c), so only one delivery line will come (assumptions A-31). // Its key also holds the DSN NOTIFY= and ORCPT= of the RCPT TO command, which @@ -181,7 +181,7 @@ func (s *session) Header(name, value string, m *milter.Modifier) (milter.Respons // message through just as "accept" does - and, unlike it, leaves this session // in place for the connection's next message (see MailFrom). // -// go-milter v0.4.1, session.go, case CodeEOB: Body(). Postfix 3.7.11, +// go-milter v0.4.1, session.go, case CodeEOB: Body(). Postfix 3.10.13, // src/global/mail_params.h: DEF_MILT_EOD_MACROS "i", which // src/cleanup/cleanup_milter.c, cleanup_milter_eval(), answers with the queue // id. src/milter/milter8.c, milter8_event(): SMFIR_CONTINUE returns the @@ -198,7 +198,7 @@ func (s *session) Body(m *milter.Modifier) (milter.Response, error) { // Postfix rejecting the message for its own reasons). No send-log row will be // written, so the level-2 slots this message held must go back. // -// Postfix 3.7.11, src/smtpd/smtpd.c: rset_cmd() and the refusals call +// Postfix 3.10.13, src/smtpd/smtpd.c: rset_cmd() and the refusals call // mail_reset(), which calls milter_abort() (src/milter/milter8.c, // milter8_abort(): SMFIC_ABORT); so do the end of every transaction and the // end of the session in smtpd_proto(). @@ -214,7 +214,7 @@ func (s *session) Abort(m *milter.Modifier) error { // single-character names (e.g. i) arrive bare. go-milter stores whatever name // Postfix sends verbatim, so a lookup must try both forms. // -// Postfix 3.7.11, src/milter/milter.c, milter_macro_lookup(): each name is +// Postfix 3.10.13, src/milter/milter.c, milter_macro_lookup(): each name is // sent as milter_*_macros spells it. go-milter v0.4.1, session.go, case // CodeMacro: m.macros[name] = value, the name as received. func macro(m *milter.Modifier, name string) string { @@ -228,7 +228,7 @@ func macro(m *milter.Modifier, name string) string { // Postfix did not send one that reads as an address. An IPv6 address may come // in Sendmail's form, with "IPv6:" in front. // -// Postfix 3.7.11, src/smtpd/smtpd_peer.c, smtpd_peer_sockaddr_to_hostaddr(): +// Postfix 3.10.13, src/smtpd/smtpd_peer.c, smtpd_peer_sockaddr_to_hostaddr(): // rfc_addr is the dotted quad, "IPv6:" + the address for IPv6, a V4-mapped // address rewritten to IPv4, or "unknown". func clientAddr(m *milter.Modifier) string { @@ -270,7 +270,7 @@ func (s *session) record(queueID string) { } // cleanAddress is an address as the milter gets it, without the angle -// brackets: Postfix 3.7.11 hands the milter the command's argument as the +// brackets: Postfix 3.10.13 hands the milter the command's argument as the // client wrote it, ESMTP parameters as separate strings (src/smtpd/smtpd.c, // mail_cmd()/rcpt_cmd(), milter_argv()), and go-milter v0.4.1 passes the first // with "<>" trimmed (session.go, case CodeMail/CodeRcpt). A quoted local part @@ -280,7 +280,7 @@ func cleanAddress(a string) string { } // recipientAsLogged is a recipient in the form Postfix writes it in "to=<..>" -// and "orig_to=<..>", so a delivery line finds its row. Postfix 3.7.11 keeps +// and "orig_to=<..>", so a delivery line finds its row. Postfix 3.10.13 keeps // the address in internal form - a quoted local part unquoted, a backslash // dropped before the character it escapes, a blank in quotes made a space // (src/smtpd/smtpd.c, extract_addr(), over src/global/tok822_parse.c, @@ -315,7 +315,7 @@ func recipientAsLogged(a string) string { return quoteLocalPart(local) + domain } -// quoteLocalPart is quote_822_local() of Postfix 3.7.11 +// quoteLocalPart is quote_822_local() of Postfix 3.10.13 // (src/global/quote_822_local.c, with QUOTE_FLAG_8BITCLEAN, the default): a // local part that is a dot-string - atoms of no blank, control character or // one of ()<>@,;:\"[], joined by single dots, none first or last - is left as @@ -366,7 +366,7 @@ func domainOf(addr string) string { // gets a fresh session bound to rec. It returns nil on a clean shutdown. // // go-milter v0.4.1, server.go: Close() closes the listener and Serve() then -// returns ErrServerClosed. Postfix 3.7.11, src/milter/milter8.c, +// returns ErrServerClosed. Postfix 3.10.13, src/milter/milter8.c, // milter8_message(): SMFIP_NOBODY leaves out only the body chunks - headers, // end-of-headers and end-of-body (milter8_eob()) still go; milter8_event() // takes a reply code (SMFIR_REPLYCODE) whatever the action mask. diff --git a/internal/milter/sender.go b/internal/milter/sender.go index b1dcdce..be6c719 100644 --- a/internal/milter/sender.go +++ b/internal/milter/sender.go @@ -39,7 +39,7 @@ import ( // it up (Bindings). The send-log row of a refusal is written by another // goroutine and is not waited for. // -// Postfix 3.7.11, src/smtpd/smtpd_check.c, reject_auth_sender_login_mismatch(): +// Postfix 3.10.13, src/smtpd/smtpd_check.c, reject_auth_sender_login_mismatch(): // the SASL login is compared with an owner by strcasecmp_utf8(); // src/milter/milter8.c, milter8_comm_error(): default_action=tempfail answers // "451 4.7.1" for a milter that does not answer. OpenDKIM 2.11.0~beta2, @@ -103,7 +103,7 @@ func (b *Bindings) fresh() error { return b.broken } -// parseSenderMap reads the lines of a sender_login_maps file as Postfix 3.7.11 +// parseSenderMap reads the lines of a sender_login_maps file as Postfix 3.10.13 // reads a texthash table (src/util/dict_thash.c, dict_thash_open(), over // src/util/readlline.c, readllines()): // - a line that starts with a blank continues the one before; an empty @@ -112,8 +112,8 @@ func (b *Bindings) fresh() error { // escaping the next character; a line with an unbalanced quote, or // without a key or a value, is ignored (with a warning, in Postfix); // - the key is folded to lower case (DICT_FLAG_FOLD_FIX, which -// src/smtpd/smtpd_check.c, pre_jail_init(), gives -// smtpd_sender_login_maps); +// src/smtpd/smtpd_check.c, smtpd_check_init(), called from +// src/smtpd/smtpd.c, pre_jail_init(), gives smtpd_sender_login_maps); // - a key given twice makes the whole table unusable: no DICT_FLAG_DUP_* // flag is given, so dict_thash_open() returns a surrogate that fails // every lookup, and smtpd answers 451 (reject_server_error()). The check @@ -194,7 +194,7 @@ func splitTableLine(line string) (key, value string, ok bool) { } // Allowed reports whether login may send as addr - the lookup Postfix makes -// for the envelope sender. Postfix 3.7.11, src/smtpd/smtpd_check.c, +// for the envelope sender. Postfix 3.10.13, src/smtpd/smtpd_check.c, // reject_auth_sender_login_mismatch(), takes the owners of the first key // mail_addr_find() finds (src/global/mail_addr_find.c, mail_addr_find_opt() // with MA_FIND_DEFAULT): the address, then the address without its @@ -242,7 +242,7 @@ const extensionDelimiter = "+" // follows its first delimiter. "user+tag@example.org" is "user@example.org"; // an address whose local part begins with the delimiter has no extension. // -// Postfix 3.7.11, src/global/strip_addr.c, strip_addr_internal(), and +// Postfix 3.10.13, src/global/strip_addr.c, strip_addr_internal(), and // split_addr.c, split_addr_internal(): the local part, up to the last "@", is // cut at its first delimiter, unless that is its first character. func withoutExtension(addr string) string { @@ -322,7 +322,7 @@ func (s *senderSession) Header(name, value string, m *milter.Modifier) (milter.R // Headers is the end of the headers, where the message is judged. // -// go-milter v0.4.1, session.go, case CodeEOH: Headers(). Postfix 3.7.11, +// go-milter v0.4.1, session.go, case CodeEOH: Headers(). Postfix 3.10.13, // src/milter/milter8.c, milter8_event(), SMFIR_REPLYCODE: a "4"/"5" reply // whose enhanced code starts with the same digit is the message's reply; // src/cleanup/cleanup_milter.c, cleanup_milter_apply(): a "5" rejects the @@ -347,7 +347,7 @@ func (s *senderSession) Headers(h textproto.MIMEHeader, m *milter.Modifier) (mil // cleanup has given the message a queue id by the end of its headers // (milter_end_of_header_macros has {i}); Postfix's lines about it - the // refusal among them - are found by it on the message's page. - // Postfix 3.7.11, src/global/mail_params.h: DEF_MILT_EOH_MACROS "i", + // Postfix 3.10.13, src/global/mail_params.h: DEF_MILT_EOH_MACROS "i", // answered by src/cleanup/cleanup_milter.c, cleanup_milter_eval(). s.record(shown, macro(m, "i")) // A reply code of its own, so that the application is told what is wrong @@ -367,7 +367,7 @@ func (s *senderSession) judge() (shown, why string, err error) { if s.login == "" { // Not reachable while the submission ports demand authentication; a // message nobody answers for is not signed either way. - // Postfix 3.7.11, src/smtpd/smtpd_check.c, smtpd_check_rcpt(): with + // Postfix 3.10.13, src/smtpd/smtpd_check.c, smtpd_check_rcpt(): with // smtpd_delay_reject (default yes) smtpd_client_restrictions // (permit_sasl_authenticated,reject on 465 and 587) refuse every RCPT; // src/smtpd/smtpd.c, data_cmd(): no valid recipients, no message. @@ -416,7 +416,7 @@ func (s *senderSession) judge() (shown, why string, err error) { // line must also be one address in RFC 5322's grammar (net/mail), at the // domain OpenDKIM signs for; a quoted display name may say anything. // And that address, its local part unquoted as Postfix holds the - // envelope's (Postfix 3.7.11, src/global/tok822_parse.c), is the one + // envelope's (Postfix 3.10.13, src/global/tok822_parse.c), is the one // held against the bindings: "x"@mine.com is x@mine.com, which may be // another application's (review of stage 2c, F1, F2). list, perr := fromParser.ParseList(unfold(raw)) @@ -504,7 +504,7 @@ func (s *senderSession) record(shown, queueID string) { // holds against nobody, the domain the login is bound to. Without it the row // would belong to no domain and its administrator would never see it. // -// Postfix 3.7.11, src/smtpd/smtpd_check.c, generic_checks(): +// Postfix 3.10.13, src/smtpd/smtpd_check.c, generic_checks(): // reject_*_sender_login_mismatch run only "if (state->sender && // *state->sender)". func (s *senderSession) domain() string { @@ -518,7 +518,7 @@ func (s *senderSession) domain() string { // characters that mean something in an SMTP reply or a log line, and cuts it: // it is said back to the client, logged and stored. // -// Postfix 3.7.11, src/milter/milter8.c, milter8_event(), SMFIR_REPLYCODE: +// Postfix 3.10.13, src/milter/milter8.c, milter8_event(), SMFIR_REPLYCODE: // "%%" becomes "%" and a single "%" is dropped; src/cleanup/cleanup_milter.c, // cleanup_milter_apply(), logs ": milter-reject: END-OF-MESSAGE from // : ; from=<...> to=<...> ...". @@ -573,7 +573,7 @@ func ServeSender(ctx context.Context, ln net.Listener, bind *Bindings, rec Refus // The headers decide; and each header value with its leading space, // as OpenDKIM asks for it (opendkim.c, mlfi_negotiate(), // SMFIP_HDR_LEADSPC), so that both read the same bytes. Postfix - // 3.7.11, src/milter/milter8.c: milter8_negotiate() keeps the event + // 3.10.13, src/milter/milter8.c: milter8_negotiate() keeps the event // mask a milter of any version asks for (go-milter v0.4.1 says // version 2), and milter8_header() drops the space only for a milter // without SMFIP_HDR_LEADSPC. diff --git a/internal/milter/sender_test.go b/internal/milter/sender_test.go index f268d3c..0a4c3cd 100644 --- a/internal/milter/sender_test.go +++ b/internal/milter/sender_test.go @@ -230,7 +230,7 @@ func TestBindingsFollowTheFile(t *testing.T) { // The owners of the first key Postfix finds decide, not any key that lists // the login: an address bound to one application is not open to another that -// holds its whole domain (Postfix 3.7.11, smtpd_check.c, +// holds its whole domain (Postfix 3.10.13, smtpd_check.c, // reject_auth_sender_login_mismatch(), over mail_addr_find(); assumptions // B-PS1). Composed by hand in internal/postfix's format. func TestBindingsFirstKeyWins(t *testing.T) { diff --git a/internal/milter/testdata/mailparse_harness.c b/internal/milter/testdata/mailparse_harness.c index 09448c3..12a0f9b 100644 --- a/internal/milter/testdata/mailparse_harness.c +++ b/internal/milter/testdata/mailparse_harness.c @@ -7,7 +7,7 @@ * * test/upstream/mailparse-diff.sh * - * builds it in debian:bookworm and runs TestMailParseMatchesOpenDKIM with it. + * builds it in debian:trixie and runs TestMailParseMatchesOpenDKIM with it. */ #include #include diff --git a/internal/postfix/dmarc.go b/internal/postfix/dmarc.go index 8b3970f..f9b9a58 100644 --- a/internal/postfix/dmarc.go +++ b/internal/postfix/dmarc.go @@ -41,7 +41,7 @@ func (p *Postfix) RebuildDMARCMaps(cfg DMARCMapsConfig) error { return p.reload() } -// Postfix 3.7.11, man/man5/access.5: "OK" accepts the address the key +// Postfix 3.10.13, man/man5/access.5: "OK" accepts the address the key // matches; man/man5/transport.5: "user@domain transport:nexthop", and "a // non-null transport field with a null nexthop field resets the nexthop // information to the recipient domain"; relay_domains as in renderInboundMaps. diff --git a/internal/postfix/duration.go b/internal/postfix/duration.go index 147c0ef..d0edf68 100644 --- a/internal/postfix/duration.go +++ b/internal/postfix/duration.go @@ -8,7 +8,7 @@ import ( "time" ) -// ParseTime reads a Postfix time value as Postfix 3.7.11 reads it: +// ParseTime reads a Postfix time value as Postfix 3.10.13 reads it: // src/global/conv_time.c, conv_time(), called by get_mail_conf_time() in // src/global/mail_conf_time.c. One number - strtol(), so leading blanks and a // "+" are taken, a negative number or one past an int is invalid - followed diff --git a/internal/postfix/duration_test.go b/internal/postfix/duration_test.go index 8ca4067..877a7ab 100644 --- a/internal/postfix/duration_test.go +++ b/internal/postfix/duration_test.go @@ -5,7 +5,7 @@ import ( "time" ) -// Postfix 3.7.11, src/global/conv_time.c, conv_time(): one number, one +// Postfix 3.10.13, src/global/conv_time.c, conv_time(): one number, one // lower-case unit, the default unit when there is none, the rest ignored. func TestParseTime(t *testing.T) { cases := []struct { diff --git a/internal/postfix/inbound.go b/internal/postfix/inbound.go index 37b59e6..89087c5 100644 --- a/internal/postfix/inbound.go +++ b/internal/postfix/inbound.go @@ -55,7 +55,7 @@ func (p *Postfix) RebuildInboundMaps(routes []InboundRoute) error { return p.reload() } -// Postfix 3.7.11, man/man5/postconf.5: a relay_domains table "is matched when +// Postfix 3.10.13, man/man5/postconf.5: a relay_domains table "is matched when // a (parent) domain appears as lookup key", the value unused; relay_recipient_maps // is used as a list, "@domain" its wild-card (src/global/mail_addr_find.c, // MA_FIND_AT_DOMAIN, the exact domain only). man/man5/transport.5: "domain @@ -104,7 +104,7 @@ func renderInboundMaps(routes []InboundRoute) (relay, transport, recipients, tls // what disable the MX lookup, so the operator's explicit upstream is used as // given; they also delimit a bare IPv6 address. // -// Postfix 3.7.11, src/smtp/smtp_connect.c: no MX lookup for a destination that +// Postfix 3.10.13, src/smtp/smtp_connect.c: no MX lookup for a destination that // starts with "["; src/util/host_port.c, host_port(): "[host]:port", the host // a name or an address ("ipv6:" in front is optional), checked by // valid_hostaddr(). diff --git a/internal/postfix/postfix.go b/internal/postfix/postfix.go index 9affbf5..8b9c6af 100644 --- a/internal/postfix/postfix.go +++ b/internal/postfix/postfix.go @@ -62,7 +62,7 @@ type Binding struct { // a comma-separated list, which is how Postfix expects multiple owners of one // sender. // -// Postfix 3.7.11, src/smtpd/smtpd_check.c, reject_auth_sender_login_mismatch(): +// Postfix 3.10.13, src/smtpd/smtpd_check.c, reject_auth_sender_login_mismatch(): // the value is split into owners with mystrtok(&cp, CHARS_COMMA_SP). func (p *Postfix) RebuildSenderLoginMaps(bindings []Binding) error { content, err := renderSenderLoginMaps(bindings) @@ -105,7 +105,7 @@ func renderSenderLoginMaps(bindings []Binding) ([]byte, error) { // One line per key as Postfix folds it: smtpd opens the map with // DICT_FLAG_FOLD_FIX and no DICT_FLAG_DUP_* flag, so two keys that // differ only in case would make the whole table unusable (Postfix - // 3.7.11, src/util/dict_thash.c, dict_thash_open()). The panel stores + // 3.10.13, src/util/dict_thash.c, dict_thash_open()). The panel stores // addresses in lower case already; this keeps the file safe if a row // ever is not. addr := strings.ToLower(b.Address) @@ -141,7 +141,7 @@ func appendUnique(list []string, v string) []string { // and the backslash. A login additionally may not carry "@" — that would make it // look like an address. // -// Postfix 3.7.11, src/util/dict_thash.c, dict_thash_open(): the key ends at +// Postfix 3.10.13, src/util/dict_thash.c, dict_thash_open(): the key ends at // the first blank outside quotes, a backslash escapes the next character; // src/util/readlline.c: a newline ends the line (one that starts with a blank // continues the one before); the owners split on comma and blank (see @@ -180,7 +180,7 @@ func assertMapSafe(address, login string) error { // Arguments are fixed literals — no user input is interpolated into the command, // and it never goes through a shell (security.md). // -// Postfix 3.7.11, conf/postfix-script, start-fg: a process that is not PID 1 +// Postfix 3.10.13, conf/postfix-script, start-fg: a process that is not PID 1 // runs "$daemon_directory/master -s" as its child, and src/master/master.c // puts that master in a session of its own (setsid()). "reload" sends the // master SIGHUP; src/master/master.c calls master_refresh(), and diff --git a/internal/postfix/queue.go b/internal/postfix/queue.go index 89ea0a2..23321d0 100644 --- a/internal/postfix/queue.go +++ b/internal/postfix/queue.go @@ -51,7 +51,7 @@ var postqueueP = func(ctx context.Context) *exec.Cmd { // listQueue runs `postqueue -p` and keeps at most maxQueueOutput bytes of it; // truncated says whether there was more. // -// Postfix 3.7.11, src/postqueue/postqueue.c, show_queue(): any user passes +// Postfix 3.10.13, src/postqueue/postqueue.c, show_queue(): any user passes // authorized_mailq_users (DEF_SHOWQ_ACL static:anyone), and a failure is // msg_fatal_status() - a non-zero exit with the reason on stderr; // src/postqueue/showq_compat.c, showq_compat(): "Mail queue is empty". @@ -107,7 +107,7 @@ func Queue() (string, error) { // cut are still in the queue, and a backlog that large is the outage the sweep // exists for (audit 2026-10-12, H3). QueueMessages treats its cap the same way. // -// Postfix 3.7.11, src/showq/showq.c, showq_service(): maildrop, active, +// Postfix 3.10.13, src/showq/showq.c, showq_service(): maildrop, active, // incoming, deferred and hold are the queues it lists. func QueueIDs() (map[string]struct{}, error) { out, truncated, err := listQueue() @@ -126,7 +126,7 @@ func QueueIDs() (map[string]struct{}, error) { // // The id is at the start of the line, optionally flagged '*' (in the active // queue) or '!' (on hold), then '#' when the message was forced to expire -// (Postfix 3.7.11, src/postqueue/showq_compat.c, showq_message(): the flags +// (Postfix 3.10.13, src/postqueue/showq_compat.c, showq_message(): the flags // are appended to the id in that order; assumptions B-PQ1), and is followed // by the message size. Requiring the // size is what separates an entry from the listing's other left-margin lines: diff --git a/internal/postfix/queue_test.go b/internal/postfix/queue_test.go index a51be99..94c957b 100644 --- a/internal/postfix/queue_test.go +++ b/internal/postfix/queue_test.go @@ -12,7 +12,7 @@ import ( // TestFakePostqueue is not a test: it is the fake `postqueue -p` the tests // below run in place of the real one, writing SP_FAKE_POSTQUEUE_BYTES bytes of -// queue entries composed by hand after the layout of Postfix 3.7.11's +// queue entries composed by hand after the layout of Postfix 3.10.13's // src/postqueue/showq_compat.c. func TestFakePostqueue(t *testing.T) { if os.Getenv("SP_FAKE_POSTQUEUE") != "1" { @@ -76,7 +76,7 @@ func TestQueueIDsReadsAListingUnderTheCap(t *testing.T) { // The reconcile sweep decides whether a message is still Postfix's problem, so // the parser must pick queue ids out of a listing and nothing else out of it: // not the header, not the byte-count trailer, and above all not a deferred -// entry's reason line. The listing is composed by hand. Postfix 3.7.11 +// entry's reason line. The listing is composed by hand. Postfix 3.10.13 // (src/postqueue/showq_compat.c, showq_message()) right-aligns a reason to // column 76, so it reaches the left margin only when it is 76 characters or // longer; the one here is put at the margin as the hard case. diff --git a/internal/postfix/queuejson.go b/internal/postfix/queuejson.go index 4efe0c5..5a428e4 100644 --- a/internal/postfix/queuejson.go +++ b/internal/postfix/queuejson.go @@ -13,7 +13,7 @@ import ( // QueuedMessage is one message Postfix is holding, as `postqueue -j` reports // it: where it is, and since when. // -// Postfix 3.7.11, src/postqueue/showq_json.c, format_json(): one object and a +// Postfix 3.10.13, src/postqueue/showq_json.c, format_json(): one object and a // newline per message - "queue_name", "queue_id", "arrival_time" (seconds, // %ld), "message_size", "forced_expire", "sender", "recipients" [{"address", // "delay_reason"}]; the queue names are those of src/global/mail_queue.h that diff --git a/internal/postfix/queuejson_test.go b/internal/postfix/queuejson_test.go index 6eccc4a..b0e328e 100644 --- a/internal/postfix/queuejson_test.go +++ b/internal/postfix/queuejson_test.go @@ -7,7 +7,7 @@ import ( ) // The listing is composed by hand: its first line in the field order of -// Postfix 3.7.11's src/postqueue/showq_json.c, format_json(); the others +// Postfix 3.10.13's src/postqueue/showq_json.c, format_json(); the others // short of fields or broken on purpose. func TestParseQueueJSON(t *testing.T) { out := []byte(`{"queue_name": "deferred", "queue_id": "41E862C00D9E", "arrival_time": 1791000000, "message_size": 446, "forced_expire": false, "sender": "app@example.com", "recipients": [{"address": "a@example.net", "delay_reason": "connect to mx.example.net[203.0.113.9]:25: Connection timed out"}]} diff --git a/internal/postfix/retry.go b/internal/postfix/retry.go index b2f106c..2702af3 100644 --- a/internal/postfix/retry.go +++ b/internal/postfix/retry.go @@ -29,7 +29,7 @@ type RetryPolicy struct { // build/postfix-config.sh does not set. Used when postconf is missing (unit // tests, a binary outside the container) so the panel still starts. // -// Postfix 3.7.11, src/global/mail_params.h: DEF_QUEUE_RUN_DELAY "300s", +// Postfix 3.10.13, src/global/mail_params.h: DEF_QUEUE_RUN_DELAY "300s", // DEF_MIN_BACKOFF_TIME DEF_QUEUE_RUN_DELAY, DEF_MAX_BACKOFF_TIME "4000s", // DEF_MAX_QUEUE_TIME "5d", DEF_DSN_QUEUE_TIME "5d", DEF_DELAY_WARN_TIME "0h". func DefaultRetryPolicy() RetryPolicy { @@ -56,7 +56,7 @@ var retryConfKeys = []string{ } // retryConfUnits is each key's default unit: the last letter of its default -// value (Postfix 3.7.11, src/global/mail_params.h, DEF_QUEUE_RUN_DELAY +// value (Postfix 3.10.13, src/global/mail_params.h, DEF_QUEUE_RUN_DELAY // "300s", DEF_MIN_BACKOFF_TIME, DEF_MAX_BACKOFF_TIME "4000s", // DEF_MAX_QUEUE_TIME "5d", DEF_DSN_QUEUE_TIME "5d", DEF_DELAY_WARN_TIME // "0h"; src/global/mail_conf_time.c, get_def_time_unit()). @@ -66,23 +66,37 @@ var retryConfUnits = []byte{'s', 's', 's', 'd', 'd', 'h'} // the same way logtail stubs queueIDs. var readRetryConf = postconfRetryValues -func postconfRetryValues() ([]string, error) { +// postconfRetryCmd builds the `postconf -h -x` command. It is a var so tests +// can put a fake postconf in its place, the way postqueueP is. +var postconfRetryCmd = func() *exec.Cmd { // Fixed argv, no user input — same pattern as Queue (security.md). - // Postfix 3.7.11, src/postconf/postconf_main.c, pcf_show_parameters(): - // named parameters are printed in argument order, with -h one "%s\n" - // value each; -x expands "$name" as Postfix itself does when it reads - // the value (mail_conf_eval()). - cmd := exec.Command("postconf", "-h", "-x", - "queue_run_delay", - "minimal_backoff_time", - "maximal_backoff_time", - "maximal_queue_lifetime", - "bounce_queue_lifetime", - "delay_warning_time", - ) - out, err := cmd.CombinedOutput() + return exec.Command("postconf", append([]string{"-h", "-x"}, retryConfKeys...)...) +} + +// postconfRetryValues reads the values from postconf's standard output only. +// Postfix 3.10.13, src/postconf/postconf_main.c, pcf_show_parameters(): +// named parameters are printed to stdout in argument order, with -h one +// "%s\n" value each, and an unknown one is a msg_warn() with no line; +// -x expands "$name" (pcf_expand_parameter_value(), +// src/postconf/postconf_lookup.c) as Postfix itself does when it reads the +// value (mail_conf_eval()). Warnings go to stderr (src/postconf/postconf.c, +// main(), msg_vstream_init(argv[0], VSTREAM_ERR)): an unknown name, and after +// the values (src/postconf/postconf.c, main()) an unused +// main.cf parameter, and since 3.9 a deprecated one, "support for parameter +// \"%s\" will be removed; instead, %s" (src/postconf/postconf_unused.c, +// pcf_flag_unused_parameters(); RELEASE_NOTES-3.9). Read together with stdout, +// as CombinedOutput did until base-trixie, such a warning was one value too +// many and the panel fell back to the compiled-in defaults. +func postconfRetryValues() ([]string, error) { + cmd := postconfRetryCmd() + var errOut strings.Builder + cmd.Stderr = &errOut + out, err := cmd.Output() if err != nil { - return nil, fmt.Errorf("postconf -h: %w: %s", err, strings.TrimSpace(string(out))) + return nil, fmt.Errorf("postconf -h: %w: %s", err, strings.TrimSpace(errOut.String())) + } + if why := strings.TrimSpace(errOut.String()); why != "" { + log.Printf("postfix: postconf -h: %s", why) } lines := strings.Split(strings.TrimSpace(string(out)), "\n") if len(lines) != len(retryConfKeys) { @@ -123,7 +137,7 @@ func parseRetryPolicy(lines []string) (RetryPolicy, error) { } durs[i] = d } - // Postfix 3.7.11, src/qmgr/qmgr.c, qmgr_post_init(): "bounce_queue_lifetime + // Postfix 3.10.13, src/qmgr/qmgr.c, qmgr_post_init(): "bounce_queue_lifetime // is larger than maximal_queue_lifetime - adjusting" - qmgr uses the // smaller. if durs[4] > durs[3] { @@ -155,7 +169,7 @@ func (p RetryPolicy) BackoffCap() string { // sender, which an application may submit too. It bounds which stored log // lines can be about a send-log row (store.SendLogLines). // -// Postfix 3.7.11, src/qmgr/qmgr_active.c, qmgr_active_done_25_generic(): a +// Postfix 3.10.13, src/qmgr/qmgr_active.c, qmgr_active_done_25_generic(): a // message expires after var_max_queue_time, or var_dsn_queue_time when its // sender is null - whoever submitted it; qmgr.c caps the second at the // first, and so does parseRetryPolicy, which leaves the first the longer. diff --git a/internal/postfix/retry_test.go b/internal/postfix/retry_test.go index 295b331..ae927e0 100644 --- a/internal/postfix/retry_test.go +++ b/internal/postfix/retry_test.go @@ -2,12 +2,15 @@ package postfix import ( "errors" + "fmt" + "os" + "os/exec" "testing" "time" ) // The postconf values in these tests are composed by hand: values main.cf -// could hold. Stock Postfix 3.7.11 prints "0h" for delay_warning_time +// could hold. Stock Postfix 3.10.13 prints "0h" for delay_warning_time // (src/global/mail_params.h, DEF_DELAY_WARN_TIME), not "0". func TestLoadRetryPolicyUsesPostconfValues(t *testing.T) { old := readRetryConf @@ -29,7 +32,7 @@ func TestLoadRetryPolicyUsesPostconfValues(t *testing.T) { if p.MaximalQueueLifetime != 2*24*time.Hour { t.Errorf("MaximalQueueLifetime = %v, want 2d", p.MaximalQueueLifetime) } - // 5d over a maximal 2d: qmgr uses 2d (Postfix 3.7.11, src/qmgr/qmgr.c, + // 5d over a maximal 2d: qmgr uses 2d (Postfix 3.10.13, src/qmgr/qmgr.c, // qmgr_post_init(), "adjusting bounce_queue_lifetime"). if p.BounceQueueLifetime != 2*24*time.Hour { t.Errorf("BounceQueueLifetime = %v, want 2d", p.BounceQueueLifetime) @@ -40,7 +43,7 @@ func TestLoadRetryPolicyUsesPostconfValues(t *testing.T) { } // A message lives at most maximal_queue_lifetime: qmgr caps -// bounce_queue_lifetime at it (Postfix 3.7.11, src/qmgr/qmgr.c, +// bounce_queue_lifetime at it (Postfix 3.10.13, src/qmgr/qmgr.c, // qmgr_post_init()), so a larger bounce lifetime does not lengthen it. func TestLongestLifetimeIsTheQueueLifetime(t *testing.T) { p, err := parseRetryPolicy([]string{"300s", "300s", "4000s", "1d", "5d", "0h"}) @@ -95,3 +98,41 @@ func TestParseRetryPolicyStock(t *testing.T) { t.Errorf("parsed = %+v", p) } } + +// TestFakePostconf is not a test: it is the fake `postconf -h -x` the test +// below runs in place of the real one. Its warning is composed after Postfix +// 3.10.13, src/postconf/postconf_unused.c, pcf_flag_unused_parameters(), +// written to stderr after the values (src/postconf/postconf.c, main()). +func TestFakePostconf(t *testing.T) { + if os.Getenv("SP_FAKE_POSTCONF") != "1" { + t.Skip("helper process") + } + fmt.Fprint(os.Stdout, "300s\n300s\n4000s\n2d\n5d\n0h\n") + fmt.Fprint(os.Stderr, "postconf: warning: /etc/postfix/main.cf: support for parameter \"smtpd_use_tls\" will be removed; instead, specify \"smtpd_tls_security_level\"\n") + os.Exit(0) +} + +// postconf's warnings are not values: since Postfix 3.9 a deprecated +// main.cf parameter draws one on every run, and read with the values it made +// one line too many, so the panel showed the compiled-in defaults. +func TestPostconfRetryValuesReadsStdoutOnly(t *testing.T) { + old := postconfRetryCmd + postconfRetryCmd = func() *exec.Cmd { + cmd := exec.Command(os.Args[0], "-test.run=^TestFakePostconf$") + cmd.Env = append(os.Environ(), "SP_FAKE_POSTCONF=1") + return cmd + } + t.Cleanup(func() { postconfRetryCmd = old }) + + lines, err := postconfRetryValues() + if err != nil { + t.Fatalf("postconfRetryValues: %v", err) + } + p, err := parseRetryPolicy(lines) + if err != nil { + t.Fatalf("parse %q: %v", lines, err) + } + if p.MaximalQueueLifetime != 2*24*time.Hour || p.FromDefaults { + t.Errorf("policy = %+v, want the postconf values", p) + } +} diff --git a/internal/store/applications.go b/internal/store/applications.go index cb335ad..7e5b2b6 100644 --- a/internal/store/applications.go +++ b/internal/store/applications.go @@ -13,7 +13,7 @@ import ( // login is globally unique because it is the SASL identity Postfix // authenticates (one sasldb2 across all domains, architecture.md § Mail path). // It is unique without regard to case too (migration 0008's NOCASE index): -// Postfix 3.7.11, src/smtpd/smtpd_check.c reject_auth_sender_login_mismatch() +// Postfix 3.10.13, src/smtpd/smtpd_check.c reject_auth_sender_login_mismatch() // compares the SASL login with the map's owners by strcasecmp_utf8. var ErrLoginExists = errors.New("application login already exists") diff --git a/internal/store/sendlog.go b/internal/store/sendlog.go index d71252e..c937267 100644 --- a/internal/store/sendlog.go +++ b/internal/store/sendlog.go @@ -100,7 +100,7 @@ func (s *Store) InsertRejected(e SendLogEntry) error { // wrote, and the milter's row holds it as Postfix logs it // (milter.recipientAsLogged); but a recipient repeated in another case is // one recipient to Postfix - cleanup drops the second, comparing folded -// (Postfix 3.7.11, src/cleanup/cleanup_out_recipient.c, been_here() with +// (Postfix 3.10.13, src/cleanup/cleanup_out_recipient.c, been_here() with // BH_FLAG_FOLD) - and one row to the milter, spelt as first given, and // log_adhoc() writes orig_to= only when it differs from to= other than in // case (src/global/log_adhoc.c, strcasecmp_utf8()). Assumptions D-S20. It @@ -191,7 +191,7 @@ func (s *Store) CloseUnreported(id int64) (int64, error) { // ExpireMessage closes the rows of a message Postfix gave up on at the end of // its queue lifetime: each recipient's newest row with that queue id, if it // is still open ("queued" or "deferred"), becomes "bounced" - Postfix returns -// the message to its sender. Postfix 3.7.11, src/qmgr/qmgr_active.c, +// the message to its sender. Postfix 3.10.13, src/qmgr/qmgr_active.c, // qmgr_active_done_25_generic(), logs expiry once per message, ": // from=<..>, status=expired, returned to sender", with no line per // recipient: the recipients' last lines say "deferred" (assumptions A-8). It diff --git a/internal/supervisor/rpc.go b/internal/supervisor/rpc.go index ff36574..a6ae74c 100644 --- a/internal/supervisor/rpc.go +++ b/internal/supervisor/rpc.go @@ -20,7 +20,7 @@ const socketPath = "/run/supervisor.sock" // runOnceFor bounds RunOnce: a reload that is still running after it is // reported as a failure. "postfix reload" checks that master runs, moves the -// incoming queue to active and sends master SIGHUP (Postfix 3.7.11, +// incoming queue to active and sends master SIGHUP (Postfix 3.10.13, // conf/postfix-script, "reload") - well under a second on the stand. var runOnceFor = time.Minute diff --git a/internal/web/handlers/handlers_backup.go b/internal/web/handlers/handlers_backup.go index 722a1a3..62d133e 100644 --- a/internal/web/handlers/handlers_backup.go +++ b/internal/web/handlers/handlers_backup.go @@ -354,7 +354,7 @@ func decryptErrorMessage(err error) string { func unreadableBackupErr(err error, deployRoot string) string { if name, ok := backup.UnreadableCert(err, deployRoot); ok { return "The panel is not allowed to read " + name + ", and a backup without it would not bring the server back. " + - "On the host, give the file to group 998 and make it group-readable (chgrp 998, chmod 640), then try again. Nothing was downloaded." + "On the host, give the file to group 10998 and make it group-readable (chgrp 10998, chmod 640), then try again. Nothing was downloaded." } if errors.Is(err, backup.ErrNoCerts) { return "The project's certs/ holds no certificate, so a backup would bring the server back without one. " + diff --git a/internal/web/view/help.go b/internal/web/view/help.go index cd1eb56..e68bbcf 100644 --- a/internal/web/view/help.go +++ b/internal/web/view/help.go @@ -232,7 +232,7 @@ var serverTopics = []HelpTopic{ ". The Postfix queue is not in it. The backup is taken while the server is running, and Postfix keeps moving messages through the queue, "+ "so a copy could hold a message twice or miss it; mail still waiting for delivery is therefore not carried over, and a restored server "+ "starts with an empty queue. To keep the queue, stop the container and archive the project directory instead. The private key in ", Code("certs/"), - " has to be readable by the panel: give it to group 998 on the host (", Code("chgrp 998"), ", ", Code("chmod 640"), + " has to be readable by the panel: give it to group 10998 on the host (", Code("chgrp 10998"), ", ", Code("chmod 640"), "). Until then the panel refuses to make a backup and names the file, rather than hand over an archive without the key. It holds secrets, so leave Encrypt with a password ticked: the file is sealed as ", Code(".spbk"), ", the password is not stored, and without it nothing can be recovered. To restore, unpack it into an empty project directory (decrypt a ", Code(".spbk"), " first with ", Code("selfpost-backup -decrypt"), diff --git a/internal/web/view/outbound_log.go b/internal/web/view/outbound_log.go index 6476255..d5d39f1 100644 --- a/internal/web/view/outbound_log.go +++ b/internal/web/view/outbound_log.go @@ -289,7 +289,7 @@ func NewOutMessage(m Meta, in MessageInput) *OutMessage { } if in.Recovered { // cleanup's line names the message's last recipient only (Postfix - // 3.7.11, src/cleanup/cleanup_milter.c, cleanup_milter_apply(), + // 3.10.13, src/cleanup/cleanup_milter.c, cleanup_milter_apply(), // " to=<%s>" of state->recip; assumptions A-15). facts = append(facts, Fact{Label: "Written", Value: Rich("from ", Code("mail.log"), " after a restart: the panel stopped before it wrote the refusal. Postfix's line names the last recipient only")}) } diff --git a/test/e2e/delivery_log_test.go b/test/e2e/delivery_log_test.go index 10edf3c..3d140f9 100644 --- a/test/e2e/delivery_log_test.go +++ b/test/e2e/delivery_log_test.go @@ -55,7 +55,7 @@ func testDeliveryLogSurvivesRotation(t *testing.T, sc *scenario) { // The rotation ends with `postfix reload`, which fails while Postfix is // still starting - and this step follows one that restarts the container. - // (Postfix 3.7.11, conf/postfix-script: "reload" and "status" both test + // (Postfix 3.10.13, conf/postfix-script: "reload" and "status" both test // "master -t" and exit 1 while master does not hold its pid lock.) if err := waitFor("Postfix to be running", 30*time.Second, 500*time.Millisecond, func() (bool, error) { if _, err := h.execIn("selfpost", "postfix", "status"); err != nil { @@ -82,7 +82,7 @@ func testDeliveryLogSurvivesRotation(t *testing.T, sc *scenario) { // testExpiredMessageIsClosed: a message Postfix gives up on is closed as // bounced. Postfix logs expiry once per message, ": from=<..>, // status=expired, returned to sender" (here "force-expired"), and no line per -// recipient (Postfix 3.7.11, src/qmgr/qmgr_active.c, +// recipient (Postfix 3.10.13, src/qmgr/qmgr_active.c, // qmgr_active_done_25_generic(); assumptions A-8). The queue lifetime cannot // be waited out, so the message is deferred by hand - defer_transports=smtp, // under which qmgr hands the recipient to the retry service, "4.3.2 deferred @@ -154,7 +154,7 @@ print("" if r is None else r[0]+" "+r[1])`, token) // testRecipientsAsPostfixLogsThem: each recipient's row reaches "sent". The // row holds the recipient as Postfix logs it - "a b" stays quoted, "ab" loses // its quotes - and a recipient given again in other capitals is one row, as -// cleanup keeps one (Postfix 3.7.11, src/global/quote_822_local.c, +// cleanup keeps one (Postfix 3.10.13, src/global/quote_822_local.c, // src/cleanup/cleanup_out_recipient.c; assumptions B-G5, A-31). Before, the // first was cut at its blank and its row closed as bounced by the sweep. func testRecipientsAsPostfixLogsThem(t *testing.T, sc *scenario) { @@ -184,7 +184,7 @@ print("; ".join(r[0]+" "+r[1] for r in rows))`, token) // testAddressLiteralIsNotLocal: an address literal of an interface the // container listens on is a local destination to Postfix even with an empty -// mydestination (Postfix 3.7.11, src/global/resolve_local.c, +// mydestination (Postfix 3.10.13, src/global/resolve_local.c, // resolve_local()). Local delivery is Postfix's refusal now // (local_transport=error:..., assumptions C-P8), and smtpd refuses such a // recipient at RCPT with its text: smtpd_reject_unlisted_recipient (on by diff --git a/test/e2e/mail_helpers.go b/test/e2e/mail_helpers.go index 870b9e3..fbef5f9 100644 --- a/test/e2e/mail_helpers.go +++ b/test/e2e/mail_helpers.go @@ -3,9 +3,11 @@ package e2e import ( "bytes" "fmt" + "net/mail" "os" "path/filepath" "regexp" + "strings" "time" "github.com/emersion/go-msgauth/dkim" @@ -73,12 +75,50 @@ func verifyDKIM(raw []byte, domain string) error { return fmt.Errorf("no DKIM signature found for domain %s (got %d signature(s))", domain, len(verifications)) } +// postfixReceived returns the Received: headers of raw that a Postfix smtpd +// wrote - "(Postfix)" after the host name, the only form it writes (Postfix +// 3.10.13, src/smtpd/smtpd.c:3627-3645, "by %s (%s)" with var_mail_name). +// In the stand only SelfPost's Postfix writes one: the sink is smtp-sink, +// which adds none, and the tests compose their messages without any. +func postfixReceived(raw []byte) ([]string, error) { + msg, err := mail.ReadMessage(bytes.NewReader(raw)) + if err != nil { + return nil, fmt.Errorf("parse the delivered message: %w", err) + } + var out []string + for _, v := range msg.Header["Received"] { + if strings.Contains(v, "(Postfix)") { + out = append(out, v) + } + } + return out, nil +} + +// checkReceivedHidden: the submission ports set smtpd_hide_client_session +// (build/postfix-config.sh), so SelfPost's Received: header is the redacted +// one - it opens with "by", names no client and no protocol (Postfix 3.10.13, +// src/smtpd/smtpd.c:3495, the "from" part written only when the parameter +// is off, and with_protocol emptied with it). +func checkReceivedHidden(raw []byte) error { + recv, err := postfixReceived(raw) + if err != nil { + return err + } + if len(recv) != 1 { + return fmt.Errorf("want one Received: header from SelfPost's Postfix, got %d: %q", len(recv), recv) + } + if !strings.HasPrefix(recv[0], "by ") || strings.Contains(recv[0], " with ") { + return fmt.Errorf("the Received: header names the client session: %q", recv[0]) + } + return nil +} + // connectFromPattern matches Postfix's smtpd "connect from ...[ADDR]" log // line, which is how the harness learns the source address Postfix itself // observed for a just-made connection (needed to configure a level-2 rate // limit's allowed-IPs list — the address a host-published port is seen as // inside the container depends on Docker's NAT and isn't worth hard-coding). -// Postfix 3.7.11, src/smtpd/smtpd.c, smtpd_service(): msg_info("connect from +// Postfix 3.10.13, src/smtpd/smtpd.c, smtpd_service(): msg_info("connect from // %s", namaddr), namaddr being "name[addr]". var connectFromPattern = regexp.MustCompile(`connect from [^\[]*\[([0-9a-fA-F.:]+)\]`) diff --git a/test/e2e/main_test.go b/test/e2e/main_test.go index 845f343..d16f21c 100644 --- a/test/e2e/main_test.go +++ b/test/e2e/main_test.go @@ -274,6 +274,9 @@ func TestE2E(t *testing.T) { if err := verifyDKIM(raw, senderDomain); err != nil { t.Fatalf("DKIM did not verify against the record the panel published: %v", err) } + if err := checkReceivedHidden(raw); err != nil { + t.Fatalf("port 465 should hide the application's session: %v", err) + } if err := waitFor("send-log row to reach status=sent", 30*time.Second, 500*time.Millisecond, func() (bool, error) { rows, err := sc.panel.sendLogRows(senderDomain, "") diff --git a/test/e2e/negative_test.go b/test/e2e/negative_test.go index fce642c..b63778a 100644 --- a/test/e2e/negative_test.go +++ b/test/e2e/negative_test.go @@ -48,7 +48,7 @@ func testLevel2RateLimit(t *testing.T, sc *scenario) { from: from, to: recipient, subject: "e2e " + uniqueToken("l2-second"), body: "should be rejected", }) // The milter answers tempfail at MAIL FROM, which Postfix words as below - // (Postfix 3.7.11, src/milter/milter8.c, milter8_event(), SMFIR_TEMPFAIL: + // (Postfix 3.10.13, src/milter/milter8.c, milter8_event(), SMFIR_TEMPFAIL: // "451 4.7.1 Service unavailable - try again later"). if err := second.refusedAt("MAIL", 451, "4.7.1 Service unavailable"); err != nil { t.Fatalf("the second message over the level-2 limit: %v", err) @@ -60,7 +60,7 @@ func testLevel2RateLimit(t *testing.T, sc *scenario) { // introduce the client again, so a second MAIL FROM on the same connection // used to arrive with no client address and was let through (found by the // technical review of 2026-10-12; the same hole let an application past its - // client-address list). Postfix's half: 3.7.11, src/smtpd/smtpd.c, + // client-address list). Postfix's half: 3.10.13, src/smtpd/smtpd.c, // smtpd_proto(), calls milter_conn_event() once per connection. errs := mailTwiceOnOneConnection(login, password, from) for i, err := range errs { @@ -94,7 +94,7 @@ func testSenderLoginMismatch(t *testing.T, sc *scenario) { }) // Postfix evaluates smtpd_sender_restrictions with smtpd_delay_reject=yes // (the default): the mismatch is detected at MAIL FROM but the reject is - // only sent back at RCPT TO. (Postfix 3.7.11, src/global/mail_params.h, + // only sent back at RCPT TO. (Postfix 3.10.13, src/global/mail_params.h, // DEF_SMTPD_DELAY_REJECT 1; src/smtpd/smtpd_check.c, // reject_auth_sender_login_mismatch(): 553 5.7.1 "Sender address rejected: // not owned by user".) @@ -110,7 +110,7 @@ func testSenderLoginMismatch(t *testing.T, sc *scenario) { // The sender address used here belongs to no registered domain, so the // rejection is unambiguously about the missing AUTH and not sender ownership. // The reply: smtps's client restrictions end in "reject" (build/ -// postfix-config.sh), Postfix 3.7.11, src/smtpd/smtpd_check.c, +// postfix-config.sh), Postfix 3.10.13, src/smtpd/smtpd_check.c, // generic_checks(): var_reject_code (554) "<%s>: %s rejected: Access denied", // delayed to RCPT by smtpd_delay_reject. func testNoAuthRejected(t *testing.T, sc *scenario) { @@ -200,7 +200,7 @@ func testSessionSurvivesRestart(t *testing.T, sc *scenario) { // held to anvil's reply, it turned out the first message had always been // refused for another reason: the new login was not yet in the sender map an // smtpd from before the panel's reload held (waitForSenderMap). (An incident -// on the stand, date not recorded.) Anvil's reply: Postfix 3.7.11, +// on the stand, date not recorded.) Anvil's reply: Postfix 3.10.13, // src/smtpd/smtpd.c, mail_cmd(): "450 4.7.1 Error: too much mail from %s". func testLevel1RateLimit(t *testing.T, sc *scenario) { login, password, err := sc.panel.addApplication(sc.domainID, "l1app", "wildcard", "") diff --git a/test/e2e/port25_config_check.go b/test/e2e/port25_config_check.go index cf457bd..07841c1 100644 --- a/test/e2e/port25_config_check.go +++ b/test/e2e/port25_config_check.go @@ -34,7 +34,7 @@ type port25Case struct { // flags=, user= and argv= are arguments of pipe(8); set as -o parameters they // are dropped, and a bare "pipe" dies on every report with "fatal: missing // user= command-line attribute" - the state of 1.7.0 through 1.9.5 (date not -// recorded; Postfix 3.7.11, src/pipe/pipe.c, get_service_attr()). +// recorded; Postfix 3.10.13, src/pipe/pipe.c, get_service_attr()). const ingestService = "dmarc-ingest unix - n n - 1 pipe flags=Rq user=panel argv=/usr/local/bin/dmarc-ingest ${recipient}" var port25Cases = []port25Case{ @@ -51,8 +51,12 @@ var port25Cases = []port25Case{ // The log tailer tells this listener's refusals from the // submission ports' by the name they are logged under. {param: "syslog_name", contains: "postfix/inbound", wantExact: true}, - // The SMTP-smuggling defence, off by default on this Postfix - // (own-mta-builds, § 5); testInboundBareNewline drives it. + // The SMTP-smuggling defence. Postfix 3.10.13 has it by default + // (src/global/mail_params.h, DEF_SMTPD_FORBID_BARE_LF "normalize"); + // set explicitly since 2.6.1, when the image ran Postfix 3.7.11, + // whose default was "no" (own-mta-builds, § 5), and kept so the + // listener does not depend on the default. testInboundBareNewline + // drives it. {param: "smtpd_forbid_bare_newline", contains: "normalize", wantExact: true}, // No antispam milter configured: the inbound journal milter // alone. The submission chain (strict OpenDKIM + journal-milter) @@ -150,8 +154,9 @@ func checkPort25ConfigCombinations(s *stack) error { // cut at the space and the rest becomes a stray smtpd argument // ("fatal: unexpected command-line argument", port 25 dead while // `postfix check` stays silent). Ask Postfix itself. (Seen in 1.9.1, - // date not recorded; Postfix 3.7.11, man/man5/master.5, "-o - // name=value (short form)", NOTE 2.) + // date not recorded; Postfix 3.10.13, man/man5/master.5, "-o + // name=value (short form)": "does not support whitespace + // around the "=" or in parameter values".) seen, err := s.execIn("selfpost", "postconf", "-Ph", "smtp/inet/"+c.param) if err != nil { return fmt.Errorf("%s: postconf -P smtp/inet/%s: %v\n%s", tc.name, c.param, err, seen) diff --git a/test/e2e/port25_features_test.go b/test/e2e/port25_features_test.go index c6fb1fa..69da8c6 100644 --- a/test/e2e/port25_features_test.go +++ b/test/e2e/port25_features_test.go @@ -272,7 +272,7 @@ func testInboundRelay(t *testing.T, sc *scenario) { // Port 25's limit is the whole limit: a 12 MB message - over the // 10240000 bytes cleanup would hold it to on its own, under the 25 MiB the - // listener says - is accepted and passed on (Postfix 3.7.11, + // listener says - is accepted and passed on (Postfix 3.10.13, // src/cleanup/cleanup_init.c, cleanup_post_jail(); assumptions C-P31). big := uniqueToken("inbound-12mb") line := "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789ab\r\n" @@ -365,7 +365,7 @@ func testInboundRelay(t *testing.T, sc *scenario) { t.Fatalf("the Inbound log holds %d rows, want the two relayed messages alone:\n%s", len(rows), strings.Join(rows, "\n")) } - // Postfix 3.7.11, src/smtpd/smtpd_check.c: "User unknown in relay recipient + // Postfix 3.10.13, src/smtpd/smtpd_check.c: "User unknown in relay recipient // table" (check_rcpt_maps(); unknown_relay_recipient_reject_code, // DEF_RELAY_RCPT_CODE 550) and "Relay access denied" // (reject_unauth_destination(); DEF_RELAY_CODE 554). @@ -464,7 +464,8 @@ func testInboundRelay(t *testing.T, sc *scenario) { // smuggling sequence, with a second transaction's commands behind it - is // body, and the one message delivered carries all of it; nothing is read as // a second transaction (smtpd_forbid_bare_newline=normalize on smtp/inet, -// build/postfix-config.sh; Postfix 3.7.11, src/smtpd/smtpd.c:3649-3652). +// build/postfix-config.sh; Postfix 3.10.13, src/smtpd/smtpd.c:3713-3716, +// receive_data_message()). // Runs after testInboundRelay, whose domain, upstream and recipient it uses. func testInboundBareNewline(t *testing.T, sc *scenario) { token := uniqueToken("inbound-bare") @@ -484,6 +485,17 @@ func testInboundBareNewline(t *testing.T, sc *scenario) { if !bytes.Contains(raw, []byte(smuggled)) { t.Fatalf("the message was cut at the bare-newline \".\": what followed it is not in the delivered message\n%s", raw) } + // Port 25 keeps RFC 5321's trace: smtpd_hide_client_session is set on + // the submission ports only (build/postfix-config.sh; Postfix 3.10.13, + // proto/postconf.proto:19587-19589, "must be used for the port 25 MTA + // service"), so this Received: header opens with "from" and the HELO. + recv, err := postfixReceived(raw) + if err != nil { + t.Fatal(err) + } + if len(recv) != 1 || !strings.HasPrefix(recv[0], "from mx.stranger.e2e.test ") { + t.Fatalf("port 25's Received: header should name the sending server, got %q", recv) + } // A second transaction, had Postfix read one, would be delivered right // behind the first: give it the time and count the files that carry the // smuggled token. diff --git a/test/e2e/process_check.go b/test/e2e/process_check.go index 3f076fa..74ba8c9 100644 --- a/test/e2e/process_check.go +++ b/test/e2e/process_check.go @@ -73,7 +73,7 @@ func parseSupervisorStatus(out string) map[string]string { // port 25: smtp/inet is absent from master.cf (Debian's stock listener is // removed when INBOUND_RELAY_ENABLE is not true). For an absent service // postconf -M prints nothing on stdout and "warning: unmatched request" on -// stderr (Postfix 3.7.11, src/postconf/postconf_master.c, +// stderr (Postfix 3.10.13, src/postconf/postconf_master.c, // pcf_show_master_entries()). func checkInboundRelayOff(s *stack) error { out, err := s.execIn("selfpost", "postconf", "-M", "smtp/inet") @@ -90,7 +90,7 @@ func checkInboundRelayOff(s *stack) error { // checkCitedPackageVersions compares the Debian packages the image runs with // the versions their sources were read at (docs/development.md, the table // between the cited-package-versions markers). The image installs unpinned -// from a moving bookworm, so a point release changes them on a rebuild +// from a moving trixie, so a point release changes them on a rebuild // without anyone reading what changed (assumptions C-D1). func checkCitedPackageVersions(s *stack) error { doc, err := os.ReadFile(filepath.Join(s.repoRoot, "docs", "development.md")) diff --git a/test/e2e/restore_test.go b/test/e2e/restore_test.go index f86d56e..fb0b984 100644 --- a/test/e2e/restore_test.go +++ b/test/e2e/restore_test.go @@ -24,7 +24,7 @@ import ( // record that was published before the backup. func testFullBackupRestore(t *testing.T, sc *scenario) { // A private key the panel may not read - the state a server is in until - // its operator gives the key to group 998. The panel does not make an + // its operator gives the key to group 10998. The panel does not make an // archive without it: it refuses, names the file and says what to do. keyPath := filepath.Join(h.stageDir, "certs", "privkey.pem") if err := os.Chmod(keyPath, 0o600); err != nil { @@ -34,7 +34,7 @@ func testFullBackupRestore(t *testing.T, sc *scenario) { if err != nil { t.Fatalf("full backup with an unreadable key: %v", err) } - if resp.StatusCode != 500 || !strings.Contains(body, "not allowed to read certs/privkey.pem") || !strings.Contains(body, "chgrp 998") { + if resp.StatusCode != 500 || !strings.Contains(body, "not allowed to read certs/privkey.pem") || !strings.Contains(body, "chgrp 10998") { t.Fatalf("full backup with an unreadable key: status %d, want a refusal that names the file and the way out:\n%.800s", resp.StatusCode, body) } if err := os.Chmod(keyPath, 0o644); err != nil { diff --git a/test/e2e/sender_check_test.go b/test/e2e/sender_check_test.go index e72987a..ce12066 100644 --- a/test/e2e/sender_check_test.go +++ b/test/e2e/sender_check_test.go @@ -53,7 +53,7 @@ func testSenderCheck(t *testing.T, sc *scenario) { {"the listed address with an extension, in envelope and header", who{listLogin, listPassword, "billing+march@" + senderDomain}, "billing+march@" + senderDomain, true}, // What a mail library does with a long display name: the line is // folded, and Postfix hands the milter both of its lines (Postfix - // 3.7.11, src/milter/milter8.c, milter8_header(): the value is the + // 3.10.13, src/milter/milter8.c, milter8_header(): the value is the // whole header as cleanup's MIME state machine collected it, // continuation line included). {"a long name, folded onto a second line", wildcard, "\"The Shop, its orders and its returns desk\"\r\n ", true}, @@ -112,7 +112,7 @@ func testSenderCheck(t *testing.T, sc *scenario) { // What the header case above holds to: Postfix itself refuses the // wildcard application billing@ as envelope sender, at RCPT, because the - // address's own key names app-list only (Postfix 3.7.11, + // address's own key names app-list only (Postfix 3.10.13, // src/smtpd/smtpd_check.c, reject_auth_sender_login_mismatch(): "553 // 5.7.1 <%s>: Sender address rejected: not owned by user %s"). if res := attemptSend(sendAttempt{ @@ -250,7 +250,7 @@ func senderCheckDeferredToday(sc *scenario) (int, error) { // message is deferred, and the count on Health sees it. Past the DATA // command Postfix's cleanup, not smtpd, talks to the milters, so the warning // that names the socket is cleanup's (audit 2026-10-12, M13: the counter read -// smtpd's lines only; Postfix 3.7.11, src/smtpd/smtpd.c, milter_send() to +// smtpd's lines only; Postfix 3.10.13, src/smtpd/smtpd.c, milter_send() to // cleanup, src/cleanup/cleanup_milter.c). The panel is frozen with SIGSTOP // once DATA is answered, and thawed after the client has its reply. func testPanelHungMidMessage(t *testing.T, sc *scenario) { diff --git a/test/e2e/sink/Dockerfile b/test/e2e/sink/Dockerfile index b491aab..eef8711 100644 --- a/test/e2e/sink/Dockerfile +++ b/test/e2e/sink/Dockerfile @@ -3,7 +3,7 @@ # the raw received message (headers + body, including the DKIM-Signature # Postfix/OpenDKIM added) straight off disk. smtp-sink ships in the postfix # package itself — no new dependency. -FROM debian:bookworm-slim +FROM debian:trixie-slim ENV DEBIAN_FRONTEND=noninteractive RUN echo "postfix postfix/mailname string localhost" | debconf-set-selections \ @@ -21,7 +21,7 @@ WORKDIR /mail # smtp-sink refuses to run as root without an explicit -u to switch to after # binding — "-u root" satisfies that check while staying root throughout, # which is fine here: throwaway container, isolated e2e network, never -# exposed. (Postfix 3.7.11, src/smtpstone/smtp-sink.c: -d "Dump each mail +# exposed. (Postfix 3.10.13, src/smtpstone/smtp-sink.c: -d "Dump each mail # transaction to a single-message file" named by the template plus "a # pseudo-random hexadecimal number"; main(): "-u option is required if running # as root".) diff --git a/test/e2e/smtp_client.go b/test/e2e/smtp_client.go index daed05f..56e3125 100644 --- a/test/e2e/smtp_client.go +++ b/test/e2e/smtp_client.go @@ -237,7 +237,7 @@ func mailTwiceOnOneConnection(login, password, from string) [2]error { // reloaded, and an smtpd that was idle from before the reload can still take // a connection in the second that follows and answer 553 "not owned by user" // (seen on the stand, date not recorded; two seconds after the save it no -// longer happened, at 0, 10, 40 or 110 s of waiting). Postfix 3.7.11: an smtpd +// longer happened, at 0, 10, 40 or 110 s of waiting). Postfix 3.10.13: an smtpd // reads the texthash map once, when it starts (html/DATABASE_README.html, // "texthash"), and one from before the reload exits only when it next sees // master's status pipe closed (src/master/multi_server.c, diff --git a/test/e2e/stage.go b/test/e2e/stage.go index d43e1aa..ab93d84 100644 --- a/test/e2e/stage.go +++ b/test/e2e/stage.go @@ -117,7 +117,7 @@ func writeSelfSignedCert(certPath, keyPath string) error { } keyPEM := pem.EncodeToMemory(&pem.Block{Type: "RSA PRIVATE KEY", Bytes: x509.MarshalPKCS1PrivateKey(key)}) // Readable by everyone, which a real key must never be. On a server the - // operator gives the key to the container's selfpost group (998) so the + // operator gives the key to the container's selfpost group (10998) so the // panel can put it into a full backup; the runner of this suite is not // always root and cannot hand a file to a group it is not in. The key is // thrown away with the stage directory. diff --git a/test/e2e/supervision_test.go b/test/e2e/supervision_test.go index de78069..d7934ef 100644 --- a/test/e2e/supervision_test.go +++ b/test/e2e/supervision_test.go @@ -45,7 +45,7 @@ func waitProgramState(t *testing.T, name, want string, within time.Duration) { } // postfixStops counts the lines "postfix stop" writes to mail.log before it -// sends master TERM (Postfix 3.7.11, conf/postfix-script, "stop": $INFO +// sends master TERM (Postfix 3.10.13, conf/postfix-script, "stop": $INFO // stopping the Postfix mail system; "... with force" when master had not // gone after five seconds, which this does not count). func postfixStops(t *testing.T) int { diff --git a/test/upstream/mailparse-diff.sh b/test/upstream/mailparse-diff.sh index b72411e..b7859aa 100644 --- a/test/upstream/mailparse-diff.sh +++ b/test/upstream/mailparse-diff.sh @@ -1,17 +1,17 @@ #!/bin/sh # The From: parser port (internal/milter/mailparse.go) against OpenDKIM's own # dkim_mail_parse(): builds internal/milter/testdata/mailparse_harness.c -# against the libopendkim source the image runs, in a debian:bookworm +# against the libopendkim source the image runs, in a debian:trixie # container, and runs TestMailParseMatchesOpenDKIM with it. # # test/upstream/mailparse-diff.sh # -# The source dir is what `apt-get source opendkim` unpacks in debian:bookworm +# The source dir is what `apt-get source opendkim` unpacks in debian:trixie # (docs/development.md); it must contain libopendkim/. set -eu src=$(cd "$1" && pwd) repo=$(cd "$(dirname "$0")/../.." && pwd) -docker run --rm -v "$src":/src:ro -v "$repo/internal/milter/testdata":/h -w /h debian:bookworm sh -c ' +docker run --rm -v "$src":/src:ro -v "$repo/internal/milter/testdata":/h -w /h debian:trixie sh -c ' apt-get update -qq >/dev/null && apt-get install -y -qq gcc libc6-dev >/dev/null 2>&1 gcc -O1 -static -o /h/mailparse_harness -I /src/libopendkim /h/mailparse_harness.c' trap 'rm -f "$repo/internal/milter/testdata/mailparse_harness"' EXIT diff --git a/test/upstream/opendkim-probe.sh b/test/upstream/opendkim-probe.sh index 3950495..d55524f 100644 --- a/test/upstream/opendkim-probe.sh +++ b/test/upstream/opendkim-probe.sh @@ -16,7 +16,7 @@ set -e mkdir -p /tmp/odk && cd /tmp/odk for d in sender.e2e.test other.e2e.test mail.e2e.test; do mkdir -p keys/$d - opendkim-genkey -D keys/$d -d $d -s sel + openssl genrsa -out keys/$d/sel.private 2048 2>/dev/null echo "$d $d:sel:/tmp/odk/keys/$d/sel.private" >> KeyTable echo "*@$d $d" >> SigningTable done