f9e75de8d16f113a45a77d611232323063fd0a68
6
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  mixandClaude Opus 5.5
|
60c7b85f84 |
findings of the reviews of audit stage 1
test / test (push) Canceled after 0s
Technical review (Opus) and security review (Fable) of 9efc9c7..a59e3d4; reports in the session's scratchpad. Fixed: - B1 (verified bug): store.UpdateStatus read before writing in its transaction; a commit by another connection in between (the dmarc-ingest process opens the same file) failed it at once with "database is locked" and the delivery line was lost. The first statement is now a write. - R3: a delivery line updated every row of its queue id and recipient; it now updates only the newest (Postfix reuses short ids). The sweep closes by row id (QueuedDelivery.ID, CloseUnreported(id)). - R4: a contradicted sweep keeps a mark, reason "reconcile-contradicted", instead of losing which row it was. - R1: contradicted cannot see a lost line (never read late). New counter reconcile_missed: at close, the sweep looks for the row's delivery line in mail.log and mail.log.1 (QueueLines). 0007 edited - it has run on no database but throwaway stands. - R2: queued_over_lifetime cannot measure "what removing the sweep would leave" while the sweep runs; its text now says rows stuck despite the sweep, and the rule uses closed - missed - contradicted. - cert-reload (R5, R6, R7, security medium/low): both files copied to .new and the copies checked as a pair before both are renamed; acts on the copy only when postconf says Postfix reads it; a failed reload is retried next pass; umask 077; says so when the mounted files cannot be read. The e2e case asserts Postfix reads the copy, the copy equals the mount, the key is 600 root. - Security low: a recipient with a quoted local part - the journal cut "a b"@x at the space, deliveryRe cut "a>b"@x at the ">" - matched no delivery line and was closed by the sweep. cleanAddress ends a bracketed address at the first ">" outside quotes; deliveryRe takes the recipient up to the ">, " before relay= (test fixtures given the relay= every real delivery line has). - Security low: postqueue -p's error carries at most 512 bytes of output. - Nits: addDeliveryLogDay refuses a column not on its list; the truncation test checks its message. - To stage 2b (page wording, mockup-first): the history step of a bounced row says the receiving server refused it, false for a row the sweep closed. To the owner when stage 2b is reached: since M13 the "deferred" figure counts whole messages too, which "Connections turned away" would not name. Found while verifying: the e2e's main positive path failed once with "553 not owned by user app1" - an smtpd idle from before the panel's reload took the connection in the reload's second. Measured on the stand (one-off probe, not committed): sending 0, 10, 40, 110 s after the save (two seconds at the soonest) always met an smtpd started after the reload. The e2e now waits for the sender map after each application it creates (waitForSenderMap: AUTH, MAIL, RCPT, RSET - no message, no row), and the level-1 case uses it instead of its own loop. Docs: architecture § Queue reconcile, schema-migrations (rows 7 and 8, chain head 8), CHANGELOG, audit plan (stage 1 note, the measured window, a new stage 2b line: 62 lines), roadmap 15/62. Evidence on the test server for this tree: go vet clean, go test ./... 23 packages ok, make e2e exit 0 (renewed certificate, hung panel, level 1 at 49/60). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
mixandClaude Opus 5.5
|
db36aa2c0e |
logtail: count the message a hung sender check defers in cleanup (audit M13)
Seen on the e2e stand, the panel frozen with SIGSTOP once DATA was
answered: the client gets 451 4.7.1 after 30 s, and the warning that names
the socket is cleanup's, not smtpd's:
postfix/cleanup[1339]: warning: milter unix:/run/selfpost/sender.sock:
can't read SMFIC_HEADER reply packet header: Connection timed out
postfix/cleanup[1339]: 7BCAF3C75DA0: milter-reject: END-OF-MESSAGE ...
4.7.1 Service unavailable - try again later; ...
The counter read smtpd lines only, so Health's "messages deferred" missed
the one case that tells a hung panel from a stopped one. milterLineRe now
takes smtpd and cleanup; cleanup's "milter unix:<socket>:" warning counts a
message each time (one cleanup process serves message after message, and the
first warning ends the message); smtpd keeps once per connection. The
composed fixture is replaced by the seen lines, with two messages through one
cleanup process and OpenDKIM hung in cleanup (not counted).
e2e panel_hung_mid_message_is_counted: freezes the panel after DATA's 354,
wants a 4xx at the end of DATA, logs what mail.log says, and waits for
Health's today count to grow. kill through sh -c: the image has no kill(1).
CHANGELOG [Unreleased], audit plan line ticked, roadmap 13/61.
Evidence on the test server for this tree (with the next commit's docs):
go vet clean, go test ./... 23 packages ok, make e2e exit 0 -
renewed_certificate_picked_up 1.2 s, panel_hung_mid_message_is_counted
31.7 s.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
mixandClaude Opus 5.5
|
96425dd515 |
e2e: a negative case passes only on its own refusal (audit stage 0)
sendResult.refusedAt(stage, code, text) and replyIs(err, code, text):
nil only for the server's reply with that code and text at that stage;
a reset, a timeout or another reply is described for the failure.
- level 1: only anvil's "450 4.7.1 Error: too much mail from", only after
a message of the loop was accepted. Held to that, it failed twice on
the stand, each time for a real reason:
1. waitForSMTPSReady took a TCP connect for readiness; Docker's port
proxy accepts before Postfix listens and then resets - the reset
once taken for the limit. It now waits for the TLS 220 greeting.
2. the first message then got "553 ... not owned by user l1app": an
smtpd idle from before the panel's postfix reload took the
connection with the old sender_login_maps. The level-1 limit had
never been what the case saw. The first message now waits (15 s at
most) for the map, failing at once on any other refusal.
Now: "level-1 limit tripped on attempt 51/60: 450 4.7.1 Error: too
much mail from 10.77.0.1".
- level 2: 451 4.7.1 at MAIL, and both MAIL FROM of one connection.
- sender mismatch: 553 at RCPT, "not owned by user".
- no AUTH and the relay case: 554 "Client host rejected: Access
denied" at RCPT; the relay case's comment claimed
reject_unauth_destination, which on 465 comes after the client
restriction - port 25 holds "554 Relay access denied".
- port 25: 550 5.1.1 for unknown recipients and report addresses.
- panel down: a 4xx reply (textproto), not just text starting with 4.
The product side (a second after an application is saved when its first
message can get 553) is written in the audit plan, not changed. Audit plan
line ticked with what was found, roadmap 7/61. No CHANGELOG line: tests only.
Evidence: make e2e on the test server for this tree, exit 0.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
mixandClaude Opus 5.5
|
c817f0833c |
from-header-binding: what the reviews had left as found, and address extensions
test / test (push) Canceled after 0s
The owner on the three things the reviews' commit left as found: can they be fixed? They can. - Downtime that crossed a UTC midnight was all booked on the day of the restart: it is cut at the midnights between (byDay). - A message with the null envelope sender - a bounce an application writes itself - left a refusal row without a domain, which no domain administrator would see: the row takes the domain the login is bound to. - A log read again after an unclean stop counted its deferrals again: while the tailer catches up, a warning no later than the last one counted is passed over. The mark is kept to the fraction of a second the log gives. And on the check being stricter than Postfix for an address with an extension (invoices+x@ passed in the envelope, was refused in the header): it makes Postfix's second try now, the address without its extension. recipient_delimiter was the distribution's default; it is set in postfix-config.sh, since the check depends on it. e2e sends one through. Test server, this tree: go vet ok, go test 23 packages ok, make e2e ok (339 s). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
mixandClaude Opus 5.5
|
89fd0a6116 |
from-header-binding: findings of the technical and security reviews
test / test (push) Canceled after 0s
Both reviews were of
|
||
|
mixandClaude Opus 5.5
|
c171fb5bc7 |
from-header-binding: the sender check, what it costs, its pages; guide, architecture, security
test / test (push) Canceled after 0s
A message's From: line is held against the application that sends it, under the rule Postfix holds the envelope sender to. OpenDKIM chooses the key by that line, so until now an application could have mail signed as another domain of the server or as the server itself. The check (internal/milter/sender.go) - A role of the panel on a socket of its own (SENDER_CHECK_SOCKET), first in smtpd_milters and strict: default_action=tempfail. The journal milter keeps its socket and stays fail-open. postfix-wrapper waits for the new socket. - It reads no database: what an application may send as comes from Postfix's own sender_login_maps, what the server signs for from OpenDKIM's SigningTable. Both are read again when they change; one that cannot be read defers mail. - One From: field, one address, allowed by the envelope's lookup; then no other name of the SigningTable may appear anywhere in the raw line. Refused: 550 5.7.1 naming the line, reduced to printable ASCII. - A refusal is a send-log row with reason "from" and the header address (migration 0006), written by another goroutine and not waited for. The journal's refusals record their reason too (limit, client), and the notifier takes only a limit's. What it costs (docs/plans/from-header-binding.md, Statistics) - The role marks itself alive every 2 s; at start the gap since the mark is downtime, counted from Postfix's start if later and not at all when supervisord says Postfix is not running. - The log tailer counts the connections Postfix turned away: a warning that names the socket, then milter-reject "Service unavailable" from the same smtpd. Seen on the stand: the client is refused from CONNECT on and leaves at AUTH (454), so no MAIL line is ever written. - sender_check_days, a row per UTC day, kept a year. Pages - Health: the card "Sender check", the socket's row, and the cards layout - boxes numbered by importance in two stacks read in rows; the left stack polls, the three readings of the right one are swapped in by id. - Outbound log: a refused row says why in its tag (wrong From, wrong client, rate-limited); the message's page shows the From: line and what the application may send as. - Help: topic "Sender check", word for word the mockup's. Found by the stand, fixed here: the names for the second look came from the sender bindings, and a domain with a key and no application yet was not among them. e2e: negative_journal_milter_fail_open is gone - it proved that mail passes with the panel down, which is no longer so. In its place sender_check_holds_the_from_line (another domain's From:, the hostname's, a stranger's, an address off the list, another sender in the display name - refused; own addresses - delivered) and panel_down_defers_mail (4xx with the panel stopped, the container alive, mail again after, downtime and one deferred message on Health, the card warning). Test server, this tree before the last cosmetic edits (a fixture row, a fact's size, docs): go vet ok, go test 23 packages ok, make e2e ok (337 s). Locally after them: internal/web/view ok. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |