test / test (push) Canceled after 0s
Technical review (Opus) and security review (Fable) of 9efc9c7..a59e3d4; reports in the session's scratchpad. Fixed: - B1 (verified bug): store.UpdateStatus read before writing in its transaction; a commit by another connection in between (the dmarc-ingest process opens the same file) failed it at once with "database is locked" and the delivery line was lost. The first statement is now a write. - R3: a delivery line updated every row of its queue id and recipient; it now updates only the newest (Postfix reuses short ids). The sweep closes by row id (QueuedDelivery.ID, CloseUnreported(id)). - R4: a contradicted sweep keeps a mark, reason "reconcile-contradicted", instead of losing which row it was. - R1: contradicted cannot see a lost line (never read late). New counter reconcile_missed: at close, the sweep looks for the row's delivery line in mail.log and mail.log.1 (QueueLines). 0007 edited - it has run on no database but throwaway stands. - R2: queued_over_lifetime cannot measure "what removing the sweep would leave" while the sweep runs; its text now says rows stuck despite the sweep, and the rule uses closed - missed - contradicted. - cert-reload (R5, R6, R7, security medium/low): both files copied to .new and the copies checked as a pair before both are renamed; acts on the copy only when postconf says Postfix reads it; a failed reload is retried next pass; umask 077; says so when the mounted files cannot be read. The e2e case asserts Postfix reads the copy, the copy equals the mount, the key is 600 root. - Security low: a recipient with a quoted local part - the journal cut "a b"@x at the space, deliveryRe cut "a>b"@x at the ">" - matched no delivery line and was closed by the sweep. cleanAddress ends a bracketed address at the first ">" outside quotes; deliveryRe takes the recipient up to the ">, " before relay= (test fixtures given the relay= every real delivery line has). - Security low: postqueue -p's error carries at most 512 bytes of output. - Nits: addDeliveryLogDay refuses a column not on its list; the truncation test checks its message. - To stage 2b (page wording, mockup-first): the history step of a bounced row says the receiving server refused it, false for a row the sweep closed. To the owner when stage 2b is reached: since M13 the "deferred" figure counts whole messages too, which "Connections turned away" would not name. Found while verifying: the e2e's main positive path failed once with "553 not owned by user app1" - an smtpd idle from before the panel's reload took the connection in the reload's second. Measured on the stand (one-off probe, not committed): sending 0, 10, 40, 110 s after the save (two seconds at the soonest) always met an smtpd started after the reload. The e2e now waits for the sender map after each application it creates (waitForSenderMap: AUTH, MAIL, RCPT, RSET - no message, no row), and the level-1 case uses it instead of its own loop. Docs: architecture § Queue reconcile, schema-migrations (rows 7 and 8, chain head 8), CHANGELOG, audit plan (stage 1 note, the measured window, a new stage 2b line: 62 lines), roadmap 15/62. Evidence on the test server for this tree: go vet clean, go test ./... 23 packages ok, make e2e exit 0 (renewed certificate, hung panel, level 1 at 49/60). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
136 lines
4.8 KiB
Go
136 lines
4.8 KiB
Go
package postfix
|
|
|
|
import (
|
|
"bytes"
|
|
"context"
|
|
"fmt"
|
|
"os/exec"
|
|
"regexp"
|
|
"strings"
|
|
"time"
|
|
)
|
|
|
|
const postqueueTimeout = 5 * time.Second
|
|
|
|
// maxQueueOutput bounds how much of `postqueue -p` the panel keeps. A deferred
|
|
// queue with a very large backlog prints one paragraph per message, and the
|
|
// whole listing is held in memory and then rendered into a page — so the read
|
|
// is capped and the remainder reported as truncated rather than being allowed
|
|
// to size the panel's memory by the size of the queue.
|
|
const maxQueueOutput = 4 << 20
|
|
|
|
// limitWriter keeps the first remaining bytes written to it and drops the rest,
|
|
// recording that it did. Writes always report full consumption so postqueue is
|
|
// never killed by a short write; the extra output is simply discarded.
|
|
type limitWriter struct {
|
|
buf bytes.Buffer
|
|
remaining int
|
|
truncated bool
|
|
}
|
|
|
|
func (l *limitWriter) Write(p []byte) (int, error) {
|
|
keep := len(p)
|
|
if keep > l.remaining {
|
|
keep = l.remaining
|
|
l.truncated = true
|
|
}
|
|
if keep > 0 {
|
|
l.buf.Write(p[:keep])
|
|
l.remaining -= keep
|
|
}
|
|
return len(p), nil
|
|
}
|
|
|
|
// postqueueP builds the `postqueue -p` command. It is a var so tests can put
|
|
// a fake postqueue in its place, the way readQueueJSON is.
|
|
var postqueueP = func(ctx context.Context) *exec.Cmd {
|
|
// A single fixed flag and no user input, never through a shell (security.md).
|
|
return exec.CommandContext(ctx, "postqueue", "-p")
|
|
}
|
|
|
|
// listQueue runs `postqueue -p` and keeps at most maxQueueOutput bytes of it;
|
|
// truncated says whether there was more.
|
|
func listQueue() (text string, truncated bool, err error) {
|
|
ctx, cancel := context.WithTimeout(context.Background(), postqueueTimeout)
|
|
defer cancel()
|
|
cmd := postqueueP(ctx)
|
|
out := &limitWriter{remaining: maxQueueOutput}
|
|
cmd.Stdout = out
|
|
cmd.Stderr = out
|
|
err = cmd.Run()
|
|
if ctx.Err() == context.DeadlineExceeded {
|
|
return "", false, fmt.Errorf("postqueue -p timed out after %s", postqueueTimeout)
|
|
}
|
|
if err != nil {
|
|
// stdout and stderr share the buffer: the end of it says why, and up
|
|
// to 4 MiB of listing does not belong in an error.
|
|
why := strings.TrimSpace(out.buf.String())
|
|
if len(why) > 512 {
|
|
why = "..." + why[len(why)-512:]
|
|
}
|
|
return "", false, fmt.Errorf("postqueue -p: %w: %s", err, why)
|
|
}
|
|
return out.buf.String(), out.truncated, nil
|
|
}
|
|
|
|
// Queue returns Postfix's own human-readable mail-queue listing
|
|
// (architecture.md § Panel HTTP surface): active, deferred and held messages,
|
|
// exactly as an administrator would see via the CLI. The panel is responsible
|
|
// for escaping the output before display (security.md); this function returns
|
|
// it as-is. A listing over the size cap is shown cut, with a note saying so.
|
|
func Queue() (string, error) {
|
|
text, truncated, err := listQueue()
|
|
if err != nil {
|
|
return "", err
|
|
}
|
|
if truncated {
|
|
text += fmt.Sprintf("\n[listing truncated at %d bytes]\n", maxQueueOutput)
|
|
}
|
|
return text, nil
|
|
}
|
|
|
|
// QueueIDs returns the set of queue ids Postfix is still holding — everything
|
|
// in the maildrop, incoming, active, deferred and hold queues. It answers the
|
|
// one question the log-tailer's reconcile sweep asks about a send-log row stuck
|
|
// at "queued": is Postfix still working on this message, or has it left the
|
|
// queue without the panel ever seeing a delivery line for it (architecture.md §
|
|
// Log tailer)?
|
|
//
|
|
// An error means the queue could not be listed and therefore says nothing about
|
|
// any message; the caller must treat it as "no information", never as an empty
|
|
// queue. A listing cut at the size cap is such an error: the messages past the
|
|
// cut are still in the queue, and a backlog that large is the outage the sweep
|
|
// exists for (audit 2026-10-12, H3). QueueMessages treats its cap the same way.
|
|
func QueueIDs() (map[string]struct{}, error) {
|
|
out, truncated, err := listQueue()
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
if truncated {
|
|
return nil, fmt.Errorf("postqueue -p: listing is larger than %d bytes", maxQueueOutput)
|
|
}
|
|
return parseQueueIDs(out), nil
|
|
}
|
|
|
|
// queueEntryRe matches the first line of a `postqueue -p` entry, e.g.
|
|
//
|
|
// 3C5B04E6C1* 446 Thu Aug 7 10:12:31 app@example.com
|
|
//
|
|
// The id is at the start of the line, optionally flagged '*' (in the active
|
|
// queue) or '!' (on hold), and is followed by the message size. Requiring the
|
|
// size is what separates an entry from the listing's other left-margin lines:
|
|
// the '-Queue ID-' header, the '-- 5 Kbytes in 2 Requests.' trailer, a deferred
|
|
// entry's '(connect timed out)' reason, and 'Mail queue is empty'. Recipient
|
|
// lines are indented and never match.
|
|
var queueEntryRe = regexp.MustCompile(`^([0-9A-Za-z]+)[*!]?\s+\d+\s`)
|
|
|
|
func parseQueueIDs(listing string) map[string]struct{} {
|
|
ids := make(map[string]struct{})
|
|
for _, line := range strings.Split(listing, "\n") {
|
|
if m := queueEntryRe.FindStringSubmatch(line); m != nil {
|
|
ids[m[1]] = struct{}{}
|
|
}
|
|
}
|
|
return ids
|
|
}
|