delivery-log-storage: findings of the technical and security reviews

Neither review (read-only agents, Opus and Fable, over ae06d8b..815236a) has
a Critical or High finding. What they found and what was done:

- A stranger on port 25 could grow the table: every queue-id line of inbound
  mail was stored and pruned an hour or more later, his own message-id among
  them. The tailer now leaves out the lines of a queue id that
  postfix/inbound/smtpd opened, until qmgr removes the message or another
  listener opens one under the id; 4096 such ids are remembered.
- A commit, so a disk sync, per stored line on the connection the journal
  milter shares, and a backlog written line by line. Lines are now written in
  one transaction per pass over the log, 500 at most, before the read offset
  is saved (InsertSendLogLines; follow() calls the keeper's flush).
- The prune asked the send log once per line and ran hourly. It asks once per
  queue id and runs at start and with the six-hourly sweep; with port-25 mail
  left out there is little without a row to wait for.
- The 4096-byte cut could split a character; it keeps characters whole.
- security.md did not say that a backup now carries these lines, and the plan
  said the time window excludes a reused queue id where it only limits the
  chance to the row's queue lifetime. Both are corrected; the collision is
  recorded as an accepted risk with its trigger.
- Tests that did not prove their comments: the prune is now tested with the
  sweep, a rejected row next to the prune, batching and the change from
  catching up to following, port-25 mail left out and its id reused.

Left as found and written down in the plan: lines the tailer never reads
(a file rotated away while the panel was down), and a message in the queue
across the upgrade, which shows only its later lines.

On the test server, for this tree: go vet clean, go test ./... 22 packages
ok, make e2e ok.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

docs/architecture.md

@@ -243,34 +243,48 @@ day before it is rotated; a message's page (`/outbound/log/{id}`) shows what
 Postfix wrote about it, so the tailer keeps those lines in the database as it
 reads them:
 
-- **What is kept:** every line headed by a queue id — after the timestamp, the
+- **What is kept:** a line headed by a queue id — after the timestamp, the
   host name and `program[pid]:` — with that id, the line's own time and the
   line as written, cut to 4096 bytes. The id is read from the head of the line
   only: the tail is the far end's text. `NOQUEUE` and `warning` stand in the
   same place and are not ids; the pattern accepts Postfix's short form
   (capital hexadecimal) and its long form (no vowels).
+- **Port-25 mail is left out.** A queue id opened by `postfix/inbound/smtpd`
+  is a stranger's message and never gets a send-log row; none of its lines
+  are stored, up to qmgr's `removed` or until another listener opens a
+  message under the id (4096 such ids are remembered). What a stranger can
+  make Postfix log is therefore not written to the database at his rate.
 - **No check for a row at write time.** A message's first lines (`client=`,
   `message-id=`) are logged before the journal milter writes its row at
   end-of-message, so the tailer cannot ask whether the id is in the send log.
-  Lines of messages that never get a row — inbound relay mail, bounces Postfix
-  generates — are kept too, for a while.
+  Lines that never get a row — bounces Postfix generates, port-25 mail met in
+  mid-flight after a restart — are kept for a while.
+- **Written per pass, not per line:** the lines of one pass over the log go
+  in one transaction (`InsertSendLogLines`), 500 at most, before the read
+  offset is saved. A commit per line would be a disk sync per line on the
+  connection the journal milter shares.
 - **Retention is one rule:** a line older than an hour whose queue id no
-  send-log row carries is deleted (`PruneSendLogLines`), at start, after every
-  send-log sweep and hourly between. That removes the lines of rows the sweep
-  deleted and the lines that never had a row.
+  send-log row carries is deleted (`PruneSendLogLines`), at start and after
+  every six-hourly send-log sweep. That removes the lines of rows the sweep
+  deleted and the lines that never had a row. The send log is asked once per
+  queue id; the statement still walks the table, about a second per million
+  lines, with the connection taken meanwhile.
 - **Which lines are a row's.** Postfix reuses a queue id once the message has
   left the queue, and a row outlives that by months. `SendLogLines` gives a row
   the lines of its queue id dated from one hour before the row was written to
   the queue lifetime (the larger of `maximal_queue_lifetime` and
-  `bounce_queue_lifetime`) plus a day after it. A message with several
+  `bounce_queue_lifetime`) plus a day after it. That limits a chance reuse of
+  the id to those days; it does not exclude it. A message with several
   recipients is several rows under one id, and each row's page shows all of
   the message's lines.
 - **Read twice, stored once.** The read offset is saved every few seconds, so
   after an unclean stop some lines are read again; while the tailer is
   catching up at start it writes a line only if the same line is not there.
+- **Not stored:** what the tailer never read — the tail of a file rotated away
+  while the panel was down.
 - **Size:** six to ten lines of about 200 bytes per message — more than the row
   itself. The table is in the database and therefore in a full backup, which
-  `log/` is not.
+  `log/` is not ([security.md](security.md) § Backup and domain export).
 
 **Two one-shot reads** sit beside the follow loop and are unrelated to it, both
 serving panel pages on request: `TailLines` (the last *n* lines, for

docs/plans/delivery-log-storage.md

@@ -29,20 +29,31 @@ rotation, while the row itself lives for ninety days.
 
 ### What is stored
 
-Every `mail.log` line whose head is a queue id - after the timestamp, the host
+A `mail.log` line whose head is a queue id - after the timestamp, the host
 name and `program[pid]:` - goes into a new table with that id, the line's own
 time (`lineTime`, the same reading the inbound journal uses) and the line as
-written, cut to 4096 bytes. The head is matched from the start of the line:
-the tail is the far end's words. `NOQUEUE` and `warning` are not queue ids -
-the pattern takes Postfix's short form (hexadecimal) and its long form (no
-vowels) and nothing else.
+written, cut to 4096 bytes on a character boundary. The head is matched from
+the start of the line: the tail is the far end's words. `NOQUEUE` and
+`warning` are not queue ids - the pattern takes Postfix's short form
+(hexadecimal) and its long form (no vowels) and nothing else.
 
 The tailer does not ask whether the id belongs to a send-log row before it
 writes. The first lines of a message (`client=`, `message-id=`) are logged
 before the journal milter writes the row at end-of-message, so such a check
-would drop exactly them. Lines of messages that never get a row - inbound
-relay mail, bounces Postfix generates - are stored too and removed by the
-prune below within the hour or two.
+would drop exactly them.
+
+**Mail that came in on port 25 is left out** (security review, 2026-10-12). A
+queue id that `postfix/inbound/smtpd` opened is a stranger's message; it never
+gets a send-log row, and what a stranger can make Postfix log - his own
+`message-id`, up to the cut - is not stored to be pruned later. The tailer
+remembers such ids (4096 at most) until qmgr's `removed` or until another
+listener opens a message under the id. What is still stored without a row is
+little: the bounces Postfix generates, and port-25 mail met in mid-flight
+after a restart.
+
+Lines are written in one transaction per pass over the log, 500 at most
+(technical review): a commit per line is a disk sync per line on the one
+connection the journal milter writes through.
 
 ### Which lines are a row's
 
@@ -50,19 +61,26 @@ A line is shown on a row's page when the queue id is the same **and** the
 line's time lies between one hour before the row was written and the queue
 lifetime (the larger of `maximal_queue_lifetime` and `bounce_queue_lifetime`,
 as `postconf` reports them) plus a day after it. Postfix reuses a short queue
-id once the message has left the queue; a later message under the same id
-falls outside that window. This is the bound the roadmap item asked for.
+id once the message has left the queue - the id is the microsecond of arrival
+and the inode of the queue file - so a row that lives ninety days can meet a
+later message under its id. The window does not rule that out, it limits it
+to the days the row's own message could still be in the queue; this is the
+bound the roadmap item asked for. The search of the current file that the
+page did before had the same exposure for a day. `enable_long_queue_ids`
+would end it and is a change to the mail path, not part of this item.
 
 ### Retention
 
 Lines live as long as a send-log row with their queue id does. One rule covers
 both ends: a line older than an hour whose queue id has no send-log row is
 deleted. That removes the lines of a row the retention sweep has just deleted,
-and the lines of messages that never had a row. It runs with the send-log
-sweep and once an hour between.
+and the lines of messages that never had a row. It runs at start and after
+every send-log sweep, six-hourly. It asks the send log once per queue id; at a
+million lines that is still a scan of the table, of the order of a second,
+during which the one connection is taken.
 
-There is no cap of its own: the table is bounded by what Postfix logs about
-queued messages, which its own rate limits bound.
+There is no cap of its own: with port-25 mail left out, the table is bounded
+by what authenticated applications send, which level 1 and level 2 limit.
 
 ### After an unclean stop
 
@@ -70,6 +88,16 @@ The tailer saves its place every few seconds, so after a crash it reads some
 lines twice. While it is catching up at start a line is written only if the
 same line is not there already - the rule the inbound journal follows.
 
+### What is not kept
+
+- Lines the tailer never reads are not stored: the tail of a file that was
+  rotated away while the panel was down, as with delivery statuses
+  (architecture.md § Log tailer).
+- A message in the queue across the upgrade to 2.2.0 has only its later lines
+  stored, and the page then shows those alone: the search of the file runs
+  only for a row with nothing stored. The only installation is the owner's
+  test server; no merge of the two sources is built for one moment of it.
+
 ### The page
 
 The page reads the table: the latest 200 lines of the row, oldest first. When
@@ -119,9 +147,13 @@ rotation or to the Server › Mail log page.
   today from the file through the same escaping; storing them changes where
   they are read from, not what is shown or to whom. A domain administrator
   sees the lines of messages of their own domains, as now.
-- They now enter the database and so a full backup, which `log/` does not.
+- They now enter the database and so a full backup, which `log/` does not:
+  submitting clients' addresses and SASL logins, message ids, receivers'
+  replies. [security.md](../security.md) says so.
 - A line is matched from its head, so a reply cannot make itself a line of
-  another message; Postfix does not log raw line breaks.
+  another message; Postfix does not log raw line breaks, and `postlogd` is the
+  file's only writer.
+- A stranger on port 25 does not write to the table (above).
 
 ## Done when
 
@@ -143,10 +175,10 @@ Target version cut: **`2.2.0`** (MINOR). One commit per step;
 [development.md](../development.md) § Plan checklists.
 
 - [x] Migration `0003_send_log_lines.sql`, store methods (insert, insert once, lines of a row, prune) and their tests — **Opus**
-- [x] Log tailer: store queue-id lines, write once while catching up, prune with the send-log sweep and hourly — **Opus**
+- [x] Log tailer: store queue-id lines, write once while catching up, prune with the send-log sweep — **Opus** (hourly as first built; six-hourly after the reviews)
 - [x] The two texts, mockup first: the *Outbound log* help topic and the empty note (`design:` commit) — **Opus** (the help topic is in the mockups; the empty note is not drawn there - the `out-message` mockup shows a message with lines - and changes with the page)
 - [x] Message page reads the table, falls back to the current `mail.log`; help text in `help.go` — **Opus**
 - [x] [architecture.md](../architecture.md), [guide.md](../guide.md), [schema-migrations.md](../schema-migrations.md) — **Opus** (schema-migrations.md went in with the migration)
 - [x] `go vet`, `go test`, e2e: a message's page still lists its lines after a forced rotation — **Opus** (on the test server: vet ok, 22 packages ok, e2e ok with the new step `delivery_log_survives_rotation`)
-- [ ] Technical review — **Opus**
-- [ ] Security review — **Fable**
+- [x] Technical review — **Opus** (2026-10-12; no Critical or High. Fixed: one transaction per pass instead of a commit per line, the prune asks the send log once per queue id and runs six-hourly, the cut keeps characters whole, tests for what the comments claimed. Left as found and written down under *What is not kept*: lines the tailer never read, and a message in the queue across the upgrade)
+- [x] Security review — **Fable** (2026-10-12; no Critical or High. Fixed: port-25 mail is not stored, so a stranger cannot grow the table; [security.md](../security.md) states what a backup now carries. The window against queue-id reuse is described as the limit it is)

docs/roadmap.md

@@ -36,7 +36,7 @@ in `git log` and [CHANGELOG.md](../CHANGELOG.md).
 | structured-logging | `log/slog` with levels and fields | candidate | — | — |
 | review-2026-08-followups | Remaining findings of the 2026-08-19 code review | candidate | — | — |
 | panel-notifications | E-mail notifications about important events | candidate | — | — |
-| delivery-log-storage | Keep a message's `mail.log` lines with its send-log row | in progress | 6/8 | [plans/delivery-log-storage.md](plans/delivery-log-storage.md) |
+| delivery-log-storage | Keep a message's `mail.log` lines with its send-log row | in progress | 8/8 | [plans/delivery-log-storage.md](plans/delivery-log-storage.md) |
 | password-reset | Password reset by e-mail | candidate | — | — |
 | phone-desktop-mode | A phone browser in "desktop site" mode gets the phone navigation | candidate | — | — |
 | panel-redesign | Panel redesign (accepted mockups, design contract) | shipped `2.0.0` | 27/27 | [plans/panel-redesign.md](plans/panel-redesign.md) |

docs/security.md

@@ -244,6 +244,11 @@ operator can act on it.
   private key from `certs/` when present; a domain export carries the DKIM key
   and **working** application passwords in the clear (otherwise a transfer
   without recreating credentials would be impossible).
+- Since 2.2.0 a full backup also carries, in the database, what Postfix logged
+  about each message still in the send log (`send_log_lines`): the submitting
+  client's address and SASL login, the message id, the receiving server and
+  its reply. `mail.log` itself stays out of a backup; these lines are state
+  the panel shows, kept for the send log's retention and no longer.
 - Both downloads are encrypted with a password unless the box on the form is
   unticked (it is ticked by default):
   scrypt (N=2¹⁵, r=8, p=1) → AES-256-GCM, streamed in 64 KiB chunks, each
@@ -404,7 +409,21 @@ deferred item from the roadmap.
   the file is inaccessible to others. The log holds envelope addresses and
   client IPs, but neither message bodies nor headers; it is excluded from
   backups (`log/` is skipped) so that a dump stays state rather than
-  diagnostics.
+  diagnostics. Since 2.2.0 the tailer copies the lines of queued outbound
+  messages into the database for the message's page, and those are in a
+  backup (§ Backup and domain export); mail offered on port 25 is not copied.
+- **A message's page can show another message's log lines after a queue-id
+  collision.** Postfix's short queue ids repeat: the id is the microsecond of
+  arrival and the queue file's inode, and inodes come round within seconds.
+  The page takes the stored lines of the row's queue id dated within the
+  row's own queue lifetime (about six days), so a second message has to get
+  the same microsecond on the same inode in that span - about one in a
+  million per reuse of the inode, and not something a sender can aim. When it
+  happens, a `domain` user could read the log lines of a message that is not
+  theirs. The search of the current `mail.log` that the page did before
+  2.2.0 had the same exposure over a day. Trigger to revisit: a report of
+  foreign lines on a page, or a reason to turn on `enable_long_queue_ids`,
+  which ends it.
 - **A `domain` user may point an inbound domain's upstream at any host and
   port.** The upstream form checks that the host is a well-formed name or IP
   and the port is 1–65535; loopback, private and link-local addresses are