postfix: long, non-repeating queue ids

The owner, 2026-10-16, of three ways to stop a reused queue id reaching an
older row (found by the technical review of inbound-upstream-result, L6):
"1" - enable_long_queue_ids=yes. A short id is the microsecond of arrival
and the queue file's inode in hex (Postfix 3.10.13, src/global/mail_queue.h,
MQID_SH_*) and repeats once the inode comes round; a later message with no
send-log row of its own (port-25 mail, a bounce Postfix sends) could then
rewrite an older Outbound log row of the same id and recipient, its page
could show the later message's stored lines (security.md's known exposure,
"enable_long_queue_ids ... ends it"), and the inbound journal's rows needed
detaching. A long id adds the time in seconds and does not repeat
(MQID_LG_*; proto/postconf.proto: existing queue files keep their names,
postqueue -p gets a wider Queue ID column). Every pattern of the panel
takes both shapes (queueLineRe's long branch, the [0-9A-Za-z]+ of the
others, queueEntryRe) - checked one by one.

Tests: e2e reads a long id on an outbound send-log row
(testExpiredMessageIsClosed) and on an inbound journal row
(testInboundUpstreamDown). go vet + go test (26 packages) and make e2e on
the test server for this tree.

Docs: security.md (the collision bullet closed for new mail, the upstream
bullet), architecture § Log tailer, delivery-log-storage's note, the
ForgetInboundQueueID comment; CHANGELOG [Unreleased] § Fixed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
mix
2026-10-16 18:59:18 +03:00
co-authored by Claude Opus 5.5
parent f4f6e0f23d
commit 22ce1bdfbb
8 changed files with 65 additions and 15 deletions
+11
View File
@@ -5,6 +5,17 @@ Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); version
## [Unreleased] ## [Unreleased]
### Fixed
- **Queue ids no longer repeat**: Postfix makes long, non-repeating queue ids
(`enable_long_queue_ids=yes`). A short id - the microsecond of arrival and
the queue file's inode - came round once the inode did, and a later message
could then rewrite an older row of the Outbound log with the same id and
recipient, put its lines on that row's page, or reach the Inbound log's
row of an earlier message. New ids read like `3Pt2mN2VXxznjll`; mail
already in the queue keeps its short id; Outbound › Queue's Queue ID
column is wider.
## [2.10.0] - 2026-10-16 ## [2.10.0] - 2026-10-16
Inbound › Log says what became of accepted mail: the upstream's answer, Inbound › Log says what became of accepted mail: the upstream's answer,
+13
View File
@@ -440,6 +440,19 @@ postconf -e \
# panel's, were exposed to it as well. # panel's, were exposed to it as well.
postconf -e "disable_mime_output_conversion=yes" postconf -e "disable_mime_output_conversion=yes"
# Long, non-repeating queue ids (owner, 2026-10-16). The panel finds a
# message's rows, its stored lines and the inbound journal's upstream result
# by the queue id of mail.log; a short id is the microsecond of arrival and
# the queue file's inode in hex and repeats once the inode comes round, so a
# later message could meet an earlier one's rows. A long id adds the time in
# seconds and does not repeat. Postfix 3.10.13, src/global/mail_params.h:1010,
# DEF_LONG_QUEUE_IDS 0; src/global/mail_queue.h, MQID_SH_* and MQID_LG_*;
# proto/postconf.proto, enable_long_queue_ids: "Enable long, non-repeating,
# queue IDs", existing queue files keep their names, postqueue -p gets a
# wider Queue ID column. Every pattern of the panel takes both shapes
# (internal/logtail, internal/postfix/queue.go).
postconf -e "enable_long_queue_ids=yes"
# What Postfix tells a milter with MAIL FROM: its defaults plus the client's # What Postfix tells a milter with MAIL FROM: its defaults plus the client's
# address. The panel's milters judge every message by that address (an # address. The panel's milters judge every message by that address (an
# application's client-address list) and record it (the inbound journal), and # application's client-address list) and record it (the inbound journal), and
+5 -1
View File
@@ -687,7 +687,11 @@ folding case). A delivery line names it in `orig_to=` when Postfix rewrote
the address, in `to=` otherwise (`log_adhoc.c` writes `orig_to=` only then), the address, in `to=` otherwise (`log_adhoc.c` writes `orig_to=` only then),
and is matched without regard to case. It updates only the newest row of its and is matched without regard to case. It updates only the newest row of its
queue id and recipient: Postfix reuses a short queue id once the message has queue id and recipient: Postfix reuses a short queue id once the message has
left the queue, and an old row must not be rewritten. `UpdateStatus`'s transaction left the queue, and an old row must not be rewritten. Since `2.10.1` the
image makes long, non-repeating queue ids (`enable_long_queue_ids=yes`,
[build/postfix-config.sh](../build/postfix-config.sh)): the time in seconds,
then the microseconds, `z` and the inode; a queue file from before keeps its
short name, and every pattern of the panel takes both. `UpdateStatus`'s transaction
starts with a write, so a commit by the `dmarc-ingest` process in between starts with a write, so a commit by the `dmarc-ingest` process in between
cannot fail it with `database is locked`. cannot fail it with `database is locked`.
Now that the log survives the container, reaching this path should mean the Now that the log survives the container, reaching this path should mean the
+1
View File
@@ -68,6 +68,7 @@ to the days the row's own message could still be in the queue; this is the
bound the roadmap item asked for. The search of the current file that the bound the roadmap item asked for. The search of the current file that the
page did before had the same exposure for a day. `enable_long_queue_ids` page did before had the same exposure for a day. `enable_long_queue_ids`
would end it and is a change to the mail path, not part of this item. would end it and is a change to the mail path, not part of this item.
*It was made in `2.10.1` (owner, 2026-10-16).*
### Retention ### Retention
+15 -9
View File
@@ -384,10 +384,11 @@ operator can act on it.
quoted RCPT TO holding `>`, `,` or `status=` cannot end the address early quoted RCPT TO holding `>`, `,` or `status=` cannot end the address early
and pass fields of its own for another recipient's row - and one holding an and pass fields of its own for another recipient's row - and one holding an
`@` is a routed address, which port 25 refuses as relaying `@` is a routed address, which port 25 refuses as relaying
(`allow_untrusted_routing=no`). Short queue ids repeat (Postfix's default (`allow_untrusted_routing=no`). Queue ids do not repeat since `2.10.1`
`enable_long_queue_ids=no`): before writing a message's rows the milter (`enable_long_queue_ids=yes`, below); for a queue file from before, which
detaches older rows of its queue id, so a message that is gone cannot take keeps its short name, the milter detaches older rows of a queue id before
a new one's results. Accepted: a row whose insert waited longer than the writing a message's rows, so a message that is gone cannot take a new
one's results. Accepted: a row whose insert waited longer than the
milter's 30 s timeout - the database held - can miss its first delivery milter's 30 s timeout - the database held - can miss its first delivery
line and stay `queued`. line and stay `queued`.
- **The inbound journal is written from what strangers send.** The inbound - **The inbound journal is written from what strangers send.** The inbound
@@ -765,8 +766,13 @@ deferred item from the roadmap.
`deploy/docker-compose.yml` sets no memory limit: what is left - many `deploy/docker-compose.yml` sets no memory limit: what is left - many
messages at once, each held to the bounds above - is the host's to messages at once, each held to the bounds above - is the host's to
size. size.
- **A message's page can show another message's log lines after a queue-id - **A message's page could show another message's log lines after a
collision.** Postfix's short queue ids repeat: the id is the microsecond of queue-id collision - until `2.10.1`.** Since then the image makes long,
non-repeating queue ids (`enable_long_queue_ids=yes`,
`build/postfix-config.sh`; owner, 2026-10-16), which ends it, and with it a
delivery line of a later message rewriting an older send-log row of the
same id and recipient. What follows is the exposure of rows from before,
whose ids are short. Postfix's short queue ids repeat: the id is the microsecond of
arrival and the queue file's inode, and inodes come round within seconds. arrival and the queue file's inode, and inodes come round within seconds.
The page takes the stored lines of the row's queue id dated within the The page takes the stored lines of the row's queue id dated within the
row's own queue lifetime (about six days), so a second message has to get row's own queue lifetime (about six days), so a second message has to get
@@ -774,9 +780,9 @@ deferred item from the roadmap.
million per reuse of the inode, and not something a sender can aim. When it million per reuse of the inode, and not something a sender can aim. When it
happens, a `domain` user could read the log lines of a message that is not happens, a `domain` user could read the log lines of a message that is not
theirs. The search of the current `mail.log` that the page did before theirs. The search of the current `mail.log` that the page did before
2.2.0 had the same exposure over a day. Trigger to revisit: a report of 2.2.0 had the same exposure over a day. It lasts for a short-id row only
foreign lines on a page, or a reason to turn on `enable_long_queue_ids`, as long as the row's queue lifetime, so it is gone from the server some
which ends it. six days after `2.10.1`.
- **A `domain` user may point an inbound domain's upstream at any host and - **A `domain` user may point an inbound domain's upstream at any host and
port.** The upstream form checks that the host is a well-formed name or IP port.** The upstream form checks that the host is a well-formed name or IP
and the port is 1–65535; loopback, private and link-local addresses are and the port is 1–65535; loopback, private and link-local addresses are
+6 -5
View File
@@ -135,11 +135,12 @@ const (
// ForgetInboundQueueID detaches the journal rows of a queue id from it: those // ForgetInboundQueueID detaches the journal rows of a queue id from it: those
// still open lose their result too, as nothing will ever report it. The // still open lose their result too, as nothing will ever report it. The
// milter calls it before it writes a new message's rows - Postfix reuses // milter calls it before it writes a new message's rows - a short queue id
// queue ids (enable_long_queue_ids=no, the default: microseconds and the // repeats (microseconds and the queue file's inode, src/global/mail_queue.h;
// queue file's inode, src/global/mail_queue.h), and only one live queue file // the image has made long, non-repeating ones since 2.10.1, but a queue file
// has an id, so older rows of it are of a message that is gone and must not // from before keeps its short name), and only one live queue file has an id,
// take the new one's results. The tailer calls it when cleanup removes a // so older rows of it are of a message that is gone and must not take the
// new one's results. The tailer calls it when cleanup removes a
// queue file it did not queue ("removed (canceled)"): the milter had written // queue file it did not queue ("removed (canceled)"): the milter had written
// the rows, but the message was never queued. // the rows, but the message was never queued.
func (s *Store) ForgetInboundQueueID(queueID string) error { func (s *Store) ForgetInboundQueueID(queueID string) error {
+3
View File
@@ -126,6 +126,9 @@ print("" if r is None else r[0]+" "+r[1])`, token)
} }
if f := strings.Fields(out); len(f) == 2 && f[1] == "deferred" { if f := strings.Fields(out); len(f) == 2 && f[1] == "deferred" {
queueID = f[0] queueID = f[0]
if !longQueueID.MatchString(queueID) {
return false, fmt.Errorf("queue id %q is not a long one (enable_long_queue_ids)", queueID)
}
return true, nil return true, nil
} }
return false, fmt.Errorf("row %q", out) return false, fmt.Errorf("row %q", out)
+11
View File
@@ -9,6 +9,12 @@ import (
"time" "time"
) )
// longQueueID is the shape of a long queue id: six or more characters of the
// time in seconds and four of the microseconds, base 52 without vowels, "z",
// the inode in base 51, which has no "z" (Postfix 3.10.13,
// src/global/mail_queue.h, MQID_LG_*).
var longQueueID = regexp.MustCompile(`^[0-9B-DF-HJ-NP-TV-Zb-df-hj-np-tv-z]{10,}z[0-9B-DF-HJ-NP-TV-Zb-df-hj-np-tv-y]+$`)
// testInboundUpstreamDown: an inbound domain whose upstream refuses // testInboundUpstreamDown: an inbound domain whose upstream refuses
// connections (the sink's address, a port nothing listens on). Port 25 takes // connections (the sink's address, a port nothing listens on). Port 25 takes
// the message; Postfix cannot hand it on and logs "relay=none ... // the message; Postfix cannot hand it on and logs "relay=none ...
@@ -65,6 +71,11 @@ print("" if r is None else "|".join(r))`, token)
return false, fmt.Errorf("deferred with host %q, reply %q", f[2], f[3]) return false, fmt.Errorf("deferred with host %q, reply %q", f[2], f[3])
} }
queueID = f[0] queueID = f[0]
// A long queue id (enable_long_queue_ids=yes,
// build/postfix-config.sh): the time, "z", the inode.
if !longQueueID.MatchString(queueID) {
return false, fmt.Errorf("queue id %q is not a long one", queueID)
}
return true, nil return true, nil
} }
return false, fmt.Errorf("row %q\n%s", out, mailLogTail(h, "postfix/")) return false, fmt.Errorf("row %q\n%s", out, mailLogTail(h, "postfix/"))