docs: stage 1 of the audit - log tailer, reconcile, sender check, guide
test / test (push) Canceled after 0s
test / test (push) Canceled after 0s
architecture.md - § Log tailer: the rotation drain does not close the gap entirely (postlogd writes to the renamed file until the reload reaches it); the fix waits for the measurement. Delivery lines read at the head of the line (M12). - § Queue reconcile: reason "reconcile", the 4 MiB cap, the conditional close; the measurement (delivery_log_days, its four counters) and the decision rule written before the data. - § Sender check, what it costs: cleanup meets a check that hangs mid-message, with the warning seen on the stand; counted once per message. guide.md - § Outbound log: `bounced` is the panel's word when Postfix no longer holds a message and the log never said; it is counted. - § Overview and Health, Sender check: the message a hung panel deferred is counted too. Audit plan line ticked, roadmap 14/61. Stage 1's code items are done (the two deferred tailer fixes wait for the data, as planned). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
+47
-9
@@ -284,7 +284,14 @@ server itself.
|
||||
(the client is told `454 4.3.0` at `AUTH` and leaves), so what is counted
|
||||
is a connection turned away, once - a client brings a message when it
|
||||
connects. The reject line does not name the milter and the journal milter's
|
||||
level-2 refusal reads the same, hence the warning. Both go into
|
||||
level-2 refusal reads the same, hence the warning. A panel that is running
|
||||
and stops answering in the middle of a message (its socket accepts, nothing
|
||||
replies) is met by cleanup, not smtpd: past `DATA` it is cleanup that talks
|
||||
to the milters, and after the content timeout it writes `warning: milter
|
||||
unix:<socket>: can't read SMFIC_HEADER reply packet header` and refuses the
|
||||
message with `4.7.1` (seen on the e2e stand with the panel frozen). One
|
||||
cleanup process serves message after message, so each such warning is
|
||||
counted as a message. Both go into
|
||||
`sender_check_days`, a
|
||||
row per UTC day kept a year, and Server › Health shows today, 30 days and
|
||||
12 months. The second figure also sees a panel that is running and not
|
||||
@@ -375,7 +382,18 @@ the latter can drop `status=sent` lines and leave send-log rows stuck at
|
||||
recreates the file lazily on first write as mode `0600`, which the unprivileged
|
||||
panel user cannot read). `follow()` drains the old inode once more before
|
||||
switching descriptors; the panel treats a missing log file as an empty tail, not
|
||||
an error.
|
||||
an error. That drain does not close the gap entirely: postlogd keeps writing to
|
||||
the renamed file until the reload reaches it, seconds later, and a line written
|
||||
there after the drain is not read. Keeping the old file open until it has
|
||||
stopped growing is the fix; it waits for the measurement below to say how
|
||||
much it matters.
|
||||
|
||||
**Delivery lines are read at the head of the line.** The pattern that turns a
|
||||
line into a row's status (`deliveryRe`) is matched after the timestamp, the
|
||||
host name and `program[pid]:`, at the queue id - like the stored lines' and the
|
||||
inbound journal's. The tail of a line is the far end's words (a reply, a HELO
|
||||
name), and a pattern that could start there let a stranger's HELO name file a
|
||||
delivery (audit 2026-10-12, M12).
|
||||
|
||||
**Read offset is persisted** (`logtail_state` table, in the baseline schema): the
|
||||
tailer stores its position plus a fingerprint of the log's first 512 bytes, and
|
||||
@@ -388,18 +406,38 @@ replay a pre-existing log.
|
||||
|
||||
**Queue reconcile** is the backstop for what the log cannot explain at all: a
|
||||
row still `queued` more than two minutes after it was accepted, whose queue id
|
||||
`postqueue -p` no longer lists, is marked `bounced` (swept every five minutes,
|
||||
`postqueue -p` no longer lists, is marked `bounced` with the reason
|
||||
`reconcile` (swept every five minutes,
|
||||
[internal/logtail](../internal/logtail/logtail.go),
|
||||
[postfix.QueueIDs](../internal/postfix/queue.go)). Postfix having dropped the
|
||||
message means nothing more will ever be reported about it, so the row can only
|
||||
be closed on an assumption, and it is closed as a failure because a delivery the
|
||||
panel cannot evidence must not be shown as one. Three things keep the sweep from
|
||||
panel cannot evidence must not be shown as one. What keeps the sweep from
|
||||
guessing where it need not: it starts only after the tailer has read to
|
||||
end-of-file once (on a restart the log itself holds the answer), the two-minute
|
||||
grace covers messages merely in flight, and a `postqueue` that cannot be read
|
||||
leaves every row untouched rather than closing them all. Now that the log
|
||||
survives the container, reaching this path means the lines are gone for good —
|
||||
rotated past fourteen files while the panel was down, or deleted.
|
||||
end-of-file once (on a restart the log itself holds the answer); the two-minute
|
||||
grace covers messages merely in flight; a `postqueue` that cannot be read, or
|
||||
whose listing is over the 4 MiB cap, leaves every row untouched rather than
|
||||
closing them all or those past the cut; and the close is conditional on the
|
||||
row still being `queued`, so a status the log wrote in the meantime stands.
|
||||
Now that the log survives the container, reaching this path should mean the
|
||||
lines are gone for good — rotated past fourteen files while the panel was down,
|
||||
or deleted — or that the tailer was not reading.
|
||||
|
||||
Whether the sweep earns its place is measured, not argued (audit 2026-10-12,
|
||||
H3). `delivery_log_days` (migration `0007`) keeps, per UTC day for a year:
|
||||
`reconcile_closed`, the rows the sweep closed; `reconcile_contradicted`, those
|
||||
of them a delivery line later spoke of - the log's status replaces the sweep's
|
||||
and the reason is cleared, so a row counts once; `tailer_blind_seconds`, the
|
||||
time `follow()` could not read `mail.log` (the file gone, reopening it failing;
|
||||
stretches under three seconds - a rotation's tick between rename and create,
|
||||
which loses nothing - are not counted); and `queued_over_lifetime`, the rows
|
||||
still `queued` past the queue lifetime plus a maximal backoff, as the day's
|
||||
last sweep saw them. No page shows the table: it is read off a copy of the
|
||||
database. The rule, written before the data: `reconcile_closed = 0` for a month
|
||||
→ remove the sweep; closed but never contradicted → keep it; contradicted →
|
||||
each case explained by blind time means fix the tailer and keep the sweep,
|
||||
otherwise mark such rows `lost` or remove the sweep. Decided after thirty days,
|
||||
or sooner if `reconcile_contradicted` is ever non-zero.
|
||||
|
||||
**Stored lines** (`send_log_lines`, since 2.2.0;
|
||||
[internal/logtail/lines.go](../internal/logtail/lines.go)). A send-log row
|
||||
|
||||
Reference in New Issue
Block a user