ca876c16311fe438bc73fcfa0327819d3362dcf3
6
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  mixandClaude Opus 5.5
|
463d4ff3fe |
log tailer: the rotated mail.log read until postlogd lets it go
test / test (push) Canceled after 0s
postlogd opens maillog_file once per process (Postfix 3.7.11,
src/postlogd/postlogd.c, pre_jail_init()) and leaves it only when it
exits: at "postfix reload" master closes the service's status pipe
(src/master/master_service.c, master_restart_service() ->
master_status_cleanup()) and dgram_server_abort() exits on reading its end
(src/master/dgram_server.c). logrotate renames and creates, then runs the
reload (build/logrotate-mail.conf postrotate; logrotate 3.21.0,
logrotate.c, rotateLogSet(), after rotateSingleLog()), so lines go to the
renamed file in between, and the tailer, already on the new one, never
read them (assumptions A-33). follow() now keeps the renamed file open as a
rotatedTail and reads it on each tick for rotatedGrace (a minute, far
longer than the gap); its places are never saved as the resume point, so a
restart still reads the rotated file through readRotated. A truncated file
is not kept.
The RFC 3339 stamp goes (A-12): it was written for a maillog_file_format
said to arrive in Postfix 3.9; there is none in 3.7.11 nor in the 3.12
snapshot of 2026-09-25 (proto/postconf.proto; msg_logger.c,
msg_logger_print() keeps strftime("%b %d %H:%M:%S ")). Every fixture now
carries the stamp Postfix writes; logtail's TestMain fixes time.Local to
UTC, since the stamp is local time and this PC is UTC+3.
Tests: TestFollowReadsTheRotatedFileAfterTheSwitch (Linux: a line appended
to the renamed file after the switch reaches the store). Stand
(2026-10-14): go vet and go test ./... pass on the server (23 packages);
make e2e exit 0, delivery_log_survives_rotation included. The tested tree
was this commit plus the previous one; the CHANGELOG entry was written
after the run, and a comment's logrotate citation corrected
(rotateLogSet runs postrotate, not rotateSingleLog).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
mixandClaude Opus 5.5
|
0ed03b2094 |
log tailer: expiry, Postfix's time values and refusal lines as Postfix writes them
test / test (push) Canceled after 0s
Expiry (assumptions A-8). Postfix 3.7.11 logs the end of a message's queue lifetime once per message, "<id>: from=<..>, status=expired, returned to sender" (src/qmgr/qmgr_active.c, qmgr_active_done_25_generic()), with no line per recipient; the tailer waited for "to=..., status=expired", which no Postfix writes, and the sweep closed only queued rows, so such rows stayed deferred. The tailer now reads qmgr's line (ExpireMessage: each recipient's newest open row of the id becomes bounced); the sweep closes a deferred row too when Postfix no longer holds the message, and counts as missed only a final line (sent, bounced, expiry). The queue listing's '#' (forced expiry, showq_compat.c showq_message()) is read as a flag (B-PQ1). Time values (B-PQ10, B-PQ11): conv_time() ported - one number, one lower-case unit, the parameter's default unit when none (get_def_time_unit: d for the lifetimes, h for delay_warning_time), the rest ignored; postconf -h -x expands $name; bounce_queue_lifetime capped at maximal_queue_lifetime as qmgr_post_init() caps it, so LongestLifetime is the queue lifetime. Refusals (A-21, A-23, A-24, A-15): smtpd_whatsup() ends the NOQUEUE line with sasl_username=<login> on an authenticated session; the line is now matched to its row by client, login and sender, and a line without a login is no application's. The client= line's login ends at the comma. The "IPv6:" trim goes (namaddr is built from state->addr). A row recovered from cleanup's line says that line names the last recipient only. Stand (2026-10-14, make e2e, exit 0): new expired_message_is_closed - defer_transports=smtp makes the retry service defer the message, postsuper -e and postqueue -i force its expiry; mail.log has "<id>: from=<alerts@sender.e2e.test>, status=force-expired, returned to sender" and the row goes deferred -> bounced; refusals_show_postfixs_line passes with the login match. go vet and go test ./... pass on the server. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
mixandClaude Opus 5.5
|
ce363f48dd |
logtail, health, panel: cite the source of what they rely on (assumptions part A)
Comments only, no behaviour changed. Each place the log tailer, Health and cmd/panel rely on another program now says where that is written, for the version the image runs: Postfix 3.7.11 (log_adhoc(), msg_logger_print(), smtpd_whatsup(), cleanup_milter.c, qmgr_active.c, postlogd.c, showq.c, pipe.c), logrotate 3.21.0, supervisor 4.2.5, proc(5). Measurements stay measurements and say where and when; test fixtures say whether they were seen or composed. Where the source contradicts the code the comment does not bless it but names the register entry (A-8 expiry, A-12 the ISO stamp no Postfix writes, A-15 the last recipient, A-21/A-23 sasl_username, A-33 postlogd's old inode, A-38 a socket file is not a listener, A-65 the certificate served); those are fixed in their own commits. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
mixandClaude Opus 5.5
|
af130e268a |
findings of the reviews of audit stage 2b
Technical review (Opus) and security review (Fable) of 5be109b..5154bba;
reports in the session's scratchpad. Nothing above low from security; one
bug and three risks from the technical review. Migration 0009 is not
deployed, so it was extended.
- Bug: two refusals of one second swapped their NOQUEUE lines (each line
went to the newest lineless row). AttachRefusalLine now matches the
client address too (send_log.client_ip, written by the journal milter -
Postfix's line names the client and the sender, never the login: the
sasl_username part of the pattern never matched and could be fed by HELO,
security F1/F2), takes the nearest row in time and the oldest of equals,
and a line already on a row goes on no other (security F6).
- Waiting lines and recoveries lived only in memory while the saved offset
moved past them: a stop within 30 s lost them. The tracker now never saves
an offset past the oldest waiting item in its file (tracker.hold,
refusalLines.oldest) - for a recovery, the client= line that names its
login - so they are read again; the "once" guards make that harmless.
- A recovered row whose login the tailer did not see was nobody's: its
domain_id falls back to the envelope domain's.
- The fingerprint's file identity is the inode alone: the device number can
change across a host reboot and make the same log look rotated.
- A refused row carries its queue id; UpdateStatus's "newest row" now skips
refused rows, so a later message with that id does not flip it to sent.
- The alive mark: its age is read first and kept from then on, so asking
supervisord for 30 s cannot make it stale; the entrypoint removes it with
the sockets - a container restart is not the check's downtime, and a mark
from before would have counted the whole outage as "≈".
- Security F3: the sender check prints no ";" of the client's in the From:
text of its reply, and the tailer reads the envelope after the first ";"
- Postfix's own text.
- A refused row with a queue id and no stored lines says so (it was told the
mail.log text of 2.2.0); the fact's em dash is the house hyphen.
- Docs: architecture (client address, no login), guide § Health (warns on
either count), help topics log and sender-check (design
|
||
|
mixandClaude Opus 5.5
|
65d916f26a |
logtail: a refusal the panel did not get to write is written from mail.log (stage 2b)
test / test (push) Canceled after 0s
The sender check answers 550 5.7.1 and writes the refused row through a
buffer a moment later; a panel stopped in between (a rollout, a kill) left
an application refused and a log that did not know it. A refusal at MAIL
FROM needs nothing: the journal milter writes its row before it answers.
- logtail.refusalLines remembers each queue id's login from smtpd's
"<id>: client=..., sasl_username=<login>" (4096 ids at most) and, on
cleanup's "<id>: milter-reject: END-OF-MESSAGE ...: 5.7.1 From: <line> -
<why>; from=<...> to=<...>", waits 30 s and hands it to
store.RecoverRefusal - an INSERT ... WHERE NOT EXISTS a refused row of that
queue id within two minutes: the row the panel wrote, or one recovered
before, means nothing is written. It is read from the saved offset on, so
a kill is covered as well as a clean stop - which a flush of the buffer at
shutdown would not.
- Migration 0009 (not deployed yet) gains send_log.recovered; the row is
written with the line's time, the login's domain id, reason "from", the
From: line as the check printed it (to the last " - " before ";").
- The message page: a fact "Written: from mail.log after a restart: the
panel stopped before it wrote the refusal".
Tests: store (not written when the panel wrote it; written once however
often); logtail (the login from client=, the From: line with a " - " in it,
nothing before the wait is over); view (the fact). e2e
refusal_recovered_from_the_log: a From: refusal, the panel stopped, its row
deleted and the tailer's saved place moved before the message's first line
(python3 in the container, as the panel's user - a kill cannot be timed to
that moment from outside); started again, the row is back, recovered, once.
Also fixed: handlers TestMessagePageOfARefusedMessage still expected the
old sentence; it failed on the server after
|
||
|
mixandClaude Opus 5.5
|
e166a2ebd6 |
refused messages: the page shows what Postfix wrote (stage 2b, 0009)
The Delivery log box of every refused row said "This message never reached
the queue, so Postfix wrote no delivery lines for it" - untrue: Postfix
wrote a line for each. The owner, 2026-10-13: the box is required on every
message page. Pictured for him first (dev/shots, the refused variant of
out-message: wrong From with its queue id lines, rate-limited with the
NOQUEUE line); the variant has no mockup screen of its own - the guard ties
screens to pages, and its outline is out-message's.
- Wrong From: cleanup gives the message a queue id before the headers are
judged; the sender check reads {i} at end-of-headers and writes it on the
refused row. Its lines - stored under the id like any other's - are on the
page, but only those from 10 minutes before to 1 minute after the refusal
(store.SendLogLines for a rejected row): Postfix frees the id at once, and
a later message with it lends nothing. No mail.log fallback for a refused
row, for the same reason.
- Rate-limited and wrong client: refused at MAIL FROM, no queue id. Migration
0009 adds send_log.refusal_line; logtail.refusalLines matches smtpd's
"NOQUEUE: milter-reject: MAIL from ...; from=<...>" (not port 25's, not
CONNECT/EHLO) and store.AttachRefusalLine puts it on the newest refused row
of that sender (and login, when the line names it) within two minutes; a
line whose row is not written yet waits up to 30 s (retried at each pass).
- The page: the queue id fact says "given, the message was not queued" or
"none - refused at MAIL FROM, before one is given"; the box's caption is
"The line from mail.log for this refusal" for the latter; a row from
before says the line was not kept. LogLineLevel colours a refusal by its
reply: 5xx red, 4xx amber (IPv6 clients included). The id in the fact is
not muted as in the picture: the kit has no muted inline.
Tests: store (attach by sender, login, time; not a queued row, not one an
hour off; a refused row's lines are its own minutes); logtail (attach, wait
and retry, port 25 and other lines ignored); milter (the refused row carries
{i}); view (facts, caption, line levels); handlers (the line on the page,
the sentence without it). e2e refusals_show_postfixs_line: a client-address
refusal made there, and the pages of a wrong From, a rate-limited and a
wrong client refusal each show Postfix's line.
schema-migrations row 9, CHANGELOG [Unreleased], audit plan line ticked,
roadmap 25/62.
Evidence on the test server for this tree: make e2e exit 0 (the new case
in 5.4 s); go test ./... ok but for two schema tests, fixed after the run
(the schema at 0009) - store tests ok locally; the full run follows before
the push.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|