docs: stage 2b of the audit - the log tailer, refusals, the Sender check card
test / test (push) Canceled after 0s

architecture.md
- § Log tailer: the fingerprint carries the inode; a log rotated while the
  panel was down is read from mail.log.1 to its end, or the gap is logged;
  a refusal's line (the queue id of a From: refusal, the NOQUEUE line put on
  a MAIL FROM refusal's row) and the refusal written from mail.log when the
  panel had not.
- § Sender check, what it costs: supervisord asked for half a minute, then
  the estimate and ≈; connections and messages in two columns; the count's
  saved place in the log.

guide.md
- § Outbound log: a refused message's page shows Postfix's line(s); rows
  from before, and a row written from mail.log, say so.
- § Overview and Health: the two columns and ≈.

schema-migrations row 9 was written with the migration. Audit plan line
ticked: stage 2b's lines are all done. Roadmap 30/62.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
mix
2026-10-14 17:19:05 +03:00
co-authored by Claude Opus 5.5
parent 415334c2cc
commit 5154bba61c
4 changed files with 60 additions and 26 deletions
+44 -16
View File
@@ -278,7 +278,10 @@ server itself.
starts without one). At start the time since the mark is the check's
downtime - counted from Postfix's own start if that was later, and not at
all when supervisord says Postfix is not running, since mail was then
stopped for another reason. The log tailer counts the mail: Postfix
stopped for another reason. Supervisord is asked for half a minute; when it
does not answer, the time since the mark is taken as an upper bound and the
day is marked estimated (`sender_check_days.estimated`), which the card
shows as ≈. The log tailer counts the mail: Postfix
writes a warning naming the socket and then, from the same smtpd process,
`milter-reject: … 451 4.7.1 Service unavailable` from the connection on
(the client is told `454 4.3.0` at `AUTH` and leaves), so what is counted
@@ -291,14 +294,17 @@ server itself.
unix:<socket>: can't read SMFIC_HEADER reply packet header` and refuses the
message with `4.7.1` (seen on the e2e stand with the panel frozen). One
cleanup process serves message after message, so each such warning is
counted as a message. Both go into
`sender_check_days`, a
row per UTC day kept a year, and Server › Health shows today, 30 days and
12 months. The second figure also sees a panel that is running and not
answering, which the mark does not. Downtime is cut at the UTC midnights
it crossed. A log read again after an unclean stop does not count again:
while the tailer catches up, a warning no later than the last one counted
is passed over.
counted as a message. The two are counted apart (the owner, 2026-10-14):
connections turned away in `deferred`, messages deferred mid-message in
`messages` of `sender_check_days`, a row per UTC day kept a year, and
Server › Health shows today, 30 days and 12 months in a column each. The
counts also see a panel that is running and not answering, which the mark
does not. Downtime is cut at the UTC midnights it crossed. A log read again
after an unclean stop does not count again: the count and the place in the
log it reached (the file's fingerprint and the offset past the line) are
saved in one transaction, and while the tailer catches up a warning at or
before that place in the same file is passed over - a second warning of
the same second is not lost; in another file the time decides.
- **Why it is not a process of its own.** Decided with the owner: measure
first. The role has a socket of its own so that the journal stays
fail-open, and moving it out later changes where it is started and nothing
@@ -396,13 +402,35 @@ name), and a pattern that could start there let a stranger's HELO name file a
delivery (audit 2026-10-12, M12).
**Read offset is persisted** (`logtail_state` table, in the baseline schema): the
tailer stores its position plus a fingerprint of the log's first 512 bytes, and
on start resumes from it, parsing the tail written while the panel was down. If
the fingerprint no longer matches (rotated or recreated in the meantime) it reads
the current file from the start; re-parsing lines is harmless because
`UpdateStatus` writes the same status onto the same row. Only a first-ever start,
with nothing stored, begins at end-of-file, so installing the panel does not
replay a pre-existing log.
tailer stores its position plus a fingerprint of the log - the hash of its first
512 bytes and the file's device and inode, so two logs that begin alike are two
files and a renamed file keeps its fingerprint - and on start resumes from it,
parsing the tail written while the panel was down. If the fingerprint no longer
matches, the log was rotated in the meantime: the tailer first opens
`mail.log.1` (left plain by `delaycompress`), and if that is the file the saved
place belongs to it reads it from there to its end, then the new file from its
start - what Postfix wrote between the stop and the rotation is not lost. A
`mail.log.1` that is another file (rotated twice) is logged with the time from
which nothing was read. Re-parsing lines is harmless because `UpdateStatus`
writes the same status onto the same row. Only a first-ever start, with nothing
stored, begins at end-of-file, so installing the panel does not replay a
pre-existing log.
**A refusal's line, and a refusal the panel did not write.** A message refused
for its `From:` line has a queue id (cleanup gives one before the headers are
judged); the sender check writes it on the row, and the message's page shows
the lines stored under it - only those of the minutes around the refusal, since
Postfix frees the id at once. A message the journal milter refused at
`MAIL FROM` has none: the tailer reads smtpd's `NOQUEUE: milter-reject: MAIL
from …; from=<…>` and puts it on the newest refused row of that sender (and
login, when the line names it) within two minutes (`send_log.refusal_line`),
retrying for 30 s when the row is not written yet. The sender check writes its
row through a buffer after answering 550; a panel stopped in between wrote
none. The tailer remembers each queue id's login from smtpd's `client=` line,
and 30 s after cleanup's `milter-reject: END-OF-MESSAGE … 5.7.1 From:` line it
writes the row from the line if no refused row of that id exists
(`send_log.recovered`, shown on the page). It reads from its saved offset, so
a kill is covered as well as a clean stop.
**Queue reconcile** is the backstop for what the log cannot explain at all: a
row still `queued` more than two minutes after it was accepted, whose queue id