findings of the reviews of audit stage 1
test / test (push) Canceled after 0s

Technical review (Opus) and security review (Fable) of 9efc9c7..a59e3d4;
reports in the session's scratchpad. Fixed:

- B1 (verified bug): store.UpdateStatus read before writing in its
  transaction; a commit by another connection in between (the dmarc-ingest
  process opens the same file) failed it at once with "database is locked"
  and the delivery line was lost. The first statement is now a write.
- R3: a delivery line updated every row of its queue id and recipient; it
  now updates only the newest (Postfix reuses short ids). The sweep closes
  by row id (QueuedDelivery.ID, CloseUnreported(id)).
- R4: a contradicted sweep keeps a mark, reason "reconcile-contradicted",
  instead of losing which row it was.
- R1: contradicted cannot see a lost line (never read late). New counter
  reconcile_missed: at close, the sweep looks for the row's delivery line
  in mail.log and mail.log.1 (QueueLines). 0007 edited - it has run on no
  database but throwaway stands.
- R2: queued_over_lifetime cannot measure "what removing the sweep would
  leave" while the sweep runs; its text now says rows stuck despite the
  sweep, and the rule uses closed - missed - contradicted.
- cert-reload (R5, R6, R7, security medium/low): both files copied to .new
  and the copies checked as a pair before both are renamed; acts on the copy
  only when postconf says Postfix reads it; a failed reload is retried next
  pass; umask 077; says so when the mounted files cannot be read. The e2e
  case asserts Postfix reads the copy, the copy equals the mount, the key is
  600 root.
- Security low: a recipient with a quoted local part - the journal cut
  "a b"@x at the space, deliveryRe cut "a>b"@x at the ">" - matched no
  delivery line and was closed by the sweep. cleanAddress ends a bracketed
  address at the first ">" outside quotes; deliveryRe takes the recipient
  up to the ">, " before relay= (test fixtures given the relay= every real
  delivery line has).
- Security low: postqueue -p's error carries at most 512 bytes of output.
- Nits: addDeliveryLogDay refuses a column not on its list; the truncation
  test checks its message.
- To stage 2b (page wording, mockup-first): the history step of a bounced
  row says the receiving server refused it, false for a row the sweep
  closed. To the owner when stage 2b is reached: since M13 the "deferred"
  figure counts whole messages too, which "Connections turned away" would
  not name.

Found while verifying: the e2e's main positive path failed once with "553
not owned by user app1" - an smtpd idle from before the panel's reload took
the connection in the reload's second. Measured on the stand (one-off
probe, not committed): sending 0, 10, 40, 110 s after the save (two seconds
at the soonest) always met an smtpd started after the reload. The e2e now
waits for the sender map after each application it creates
(waitForSenderMap: AUTH, MAIL, RCPT, RSET - no message, no row), and the
level-1 case uses it instead of its own loop.

Docs: architecture § Queue reconcile, schema-migrations (rows 7 and 8,
chain head 8), CHANGELOG, audit plan (stage 1 note, the measured window,
a new stage 2b line: 62 lines), roadmap 15/62.

Evidence on the test server for this tree: go vet clean, go test ./... 23
packages ok, make e2e exit 0 (renewed certificate, hung panel, level 1 at
49/60).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
mix
2026-10-14 15:21:52 +03:00
co-authored by Claude Opus 5.5
parent f51f5db245
commit 60c7b85f84
26 changed files with 503 additions and 143 deletions
+45 -8
View File
@@ -97,6 +97,16 @@ func TestUpdateStatusRecipientCaseInsensitive(t *testing.T) {
}
}
// rowID is the id of the newest send-log row of a queue id.
func rowID(t *testing.T, st *Store, queueID string) int64 {
t.Helper()
var id int64
if err := st.db.QueryRow(`SELECT MAX(id) FROM send_log WHERE queue_id = ?`, queueID).Scan(&id); err != nil {
t.Fatal(err)
}
return id
}
// The sweep closes only what is still queued, and says it was the sweep: a
// status the log wrote stands, whatever the sweep believed when it listed the
// row.
@@ -111,10 +121,10 @@ func TestCloseUnreportedClosesOnlyQueuedRows(t *testing.T) {
t.Fatal(err)
}
if n, err := st.CloseUnreported("Q1", "lost@example.net"); err != nil || n != 1 {
if n, err := st.CloseUnreported(rowID(t, st, "Q1")); err != nil || n != 1 {
t.Fatalf("CloseUnreported of a queued row = %d, %v; want 1", n, err)
}
if n, err := st.CloseUnreported("Q2", "done@example.net"); err != nil || n != 0 {
if n, err := st.CloseUnreported(rowID(t, st, "Q2")); err != nil || n != 0 {
t.Fatalf("CloseUnreported of a sent row = %d, %v; want 0", n, err)
}
@@ -133,14 +143,15 @@ func TestCloseUnreportedClosesOnlyQueuedRows(t *testing.T) {
}
// A delivery line about a row the sweep closed proves the sweep guessed what
// the log could have said: the log's status wins, the reason goes, and the day
// counts one contradiction - once, however often the line is read again.
// the log could have said: the log's status wins, the row is marked
// reconcile-contradicted, and the day counts one contradiction - once,
// however often the line is read again.
func TestUpdateStatusCountsAContradictedSweep(t *testing.T) {
st := openTestStore(t)
if err := st.InsertQueued(SendLogEntry{QueueID: "Q1", To: "a@example.net"}); err != nil {
t.Fatal(err)
}
if _, err := st.CloseUnreported("Q1", "a@example.net"); err != nil {
if _, err := st.CloseUnreported(rowID(t, st, "Q1")); err != nil {
t.Fatal(err)
}
for i := 0; i < 2; i++ {
@@ -152,8 +163,8 @@ func TestUpdateStatusCountsAContradictedSweep(t *testing.T) {
if err := st.db.QueryRow(`SELECT status, reason FROM send_log WHERE queue_id = 'Q1'`).Scan(&status, &reason); err != nil {
t.Fatal(err)
}
if status != StatusSent || reason != "" {
t.Fatalf("status %q reason %q, want sent and no reason", status, reason)
if status != StatusSent || reason != ReasonReconcileContradicted {
t.Fatalf("status %q reason %q, want sent and %s", status, reason, ReasonReconcileContradicted)
}
days, err := st.DeliveryLogDays()
if err != nil {
@@ -164,6 +175,32 @@ func TestUpdateStatusCountsAContradictedSweep(t *testing.T) {
}
}
// Postfix reuses a short queue id once its message has left the queue; a
// line about the message of today rewrites today's row, not the months-old
// one of the same id and recipient.
func TestUpdateStatusRewritesOnlyTheNewestRow(t *testing.T) {
st := openTestStore(t)
for i := 0; i < 2; i++ {
if err := st.InsertQueued(SendLogEntry{QueueID: "REUSED", To: "a@example.net"}); err != nil {
t.Fatal(err)
}
}
old := rowID(t, st, "REUSED") - 1
if _, err := st.db.Exec(`UPDATE send_log SET status = ? WHERE id = ?`, StatusSent, old); err != nil {
t.Fatal(err)
}
if n, err := st.UpdateStatus("REUSED", "a@example.net", StatusBounced); err != nil || n != 1 {
t.Fatalf("UpdateStatus = %d, %v; want 1 row", n, err)
}
var status string
if err := st.db.QueryRow(`SELECT status FROM send_log WHERE id = ?`, old).Scan(&status); err != nil {
t.Fatal(err)
}
if status != StatusSent {
t.Fatalf("the old row became %q, want it left sent", status)
}
}
// The reconcile sweep asks for the rows old enough that Postfix should have
// reported on them by now. A message accepted moments ago is simply in flight,
// and one the milter refused never reached the queue at all, so neither is the
@@ -199,7 +236,7 @@ func TestListQueuedOlderThan(t *testing.T) {
if err != nil {
t.Fatalf("ListQueuedOlderThan: %v", err)
}
if len(got) != 1 || got[0] != (QueuedDelivery{QueueID: "OLD1", To: "stale@example.net"}) {
if len(got) != 1 || got[0] != (QueuedDelivery{ID: got[0].ID, QueueID: "OLD1", To: "stale@example.net"}) {
t.Fatalf("got %+v, want only the stale queued row", got)
}
}