inbound-upstream-result: the upstream's answer in the Inbound log
test / test (push) Waiting to run

The owner, 2026-10-16: the Inbound log lacks the upstream's answer - "Не
понятна судьба письма: принято, отклонено, хост не доступен"; the result
in the row only, a page per message later if needed. Mockups 5348330 and
46b90d2, plan 295d72e.

- Migration 0012: inbound_spam_log gains queue_id, upstream_status ('',
  queued, held, sent, deferred, bounced; CHECK), upstream_relay,
  upstream_reply, upstream_at, and a partial index on queue_id. Schema 12.
- The inbound journal milter writes an accepted row with the queue id of
  end-of-message and "queued" ("held" when the filter had it held), the
  recipient as Postfix logs it (recipientAsLogged), after detaching older
  rows of a queue id Postfix uses again; no queue id, no result.
- The tailer: a delivery line no send-log row took goes to the journal -
  status, relay= ("" for none), the reply after the first status=, cut to
  255 printable bytes, the line's time; queued/held/deferred rows only,
  sent and bounced final. qmgr's expiry makes them bounced, keeping the
  last reply. cleanup's "removed (" detaches rows of a message never
  queued. Addresses in delivery lines are read by Postfix's quoting
  (loggedAddress): a quoted RCPT TO cannot end one early.
- The page: an Upstream column - a plain tag for queued and held, else a
  tag opening the host, the reply, when and what follows; "—" for refused
  mail and older rows. Help (word for word with the mockup), guide,
  architecture, security, schema-migrations, CHANGELOG.

Reviewed before the commit (technical and security, 2026-10-16); the plan
lists each finding and what it became.

Tests: store (results, finality, expiry, held, a reused id, the queue-id
index in every plan), milter (queue id, held, quoted recipient, no {i}),
logtail (relay and reply, a forged quoted recipient, Run hands the journal
what the send log did not take, cleanup's removal), view (the column, held,
null sender), e2e (both relayed messages sent with the sink's 250; an
upstream refusing connections deferred with the reason and no host, then
expired to bounced; the refused row has none). go vet + go test (26
packages) and make e2e on the test server for this tree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
mix
2026-10-16 16:35:09 +03:00
co-authored by Claude Opus 5.5
parent 46b90d2a8d
commit d51549385b
26 changed files with 966 additions and 72 deletions
+47 -6
View File
@@ -85,8 +85,8 @@ const (
// calls it on every prune cycle so a panel change takes effect without restart.
type RetentionDays func() int
// deliveryRe matches a Postfix delivery line and captures queue-id, recipient
// and status, e.g.
// deliveryRe matches a Postfix delivery line and captures queue-id, recipient,
// orig_to, relay and status, e.g.
//
// postfix/smtp[26]: 41E862C00D9E: to=<a@example.net>, relay=…, dsn=2.0.0, status=sent (250 OK)
//
@@ -116,7 +116,19 @@ type RetentionDays func() int
// controls. A bounce whose reply quoted "status=sent" would then be filed as a
// success. The real field is always the first one after to=<…>: in
// log_adhoc() the reply is the "(%s)" after status=.
var deliveryRe = regexp.MustCompile(`^\S+ [^\s\[]+\[\d+\]: ([0-9A-Za-z]+): to=<(.*?)>, (?:orig_to=<(.*?)>, )?relay=.*?\bstatus=(\w+)`)
var deliveryRe = regexp.MustCompile(`^\S+ [^\s\[]+\[\d+\]: ([0-9A-Za-z]+): to=<(` + loggedAddress + `)>, (?:orig_to=<(` + loggedAddress + `)>, )?relay=([^,\s]*).*?\bstatus=(\w+)`)
// loggedAddress is an address as Postfix logs it, external form: a local part
// in quotes when it is not a plain dot-string - and then it may hold "<", ">",
// "," and "=", with only '"' and '\' escaped - or one with none of '"' and
// '>' (Postfix 3.10.13, src/global/quote_822_local.c,
// src/global/info_log_addr_form.c). Read by that grammar, a stranger's quoted
// RCPT TO cannot end the address early and pass "orig_to=<…>, relay=…,
// status=…" of his own as the line's fields (security review of
// inbound-upstream-result, 2026-10-16; a local part with "@" is a routed
// address, which port 25 refuses as relaying anyway: src/trivial-rewrite/
// resolve.c:467, src/smtpd/smtpd_check.c:1724, allow_untrusted_routing=no).
const loggedAddress = `"(?:[^"\\]|\\.)*"@[^>]*|[^>"]*`
// parseDelivery extracts (queue-id, recipient, status) from a mail.log line.
// ok is false for lines that are not recognised delivery results. The
@@ -133,7 +145,7 @@ func parseDelivery(line string) (queueID, recipient, status string, ok bool) {
if m[3] != "" {
recipient = m[3]
}
switch m[4] {
switch m[5] {
case "sent":
status = store.StatusSent
case "deferred":
@@ -146,6 +158,25 @@ func parseDelivery(line string) (queueID, recipient, status string, ok bool) {
return m[1], recipient, status, true
}
// deliveryReply returns the host and the reply of a delivery line (the line
// after its time stamp): relay= - which log_adhoc() writes right after to=
// and orig_to=, followed by a comma - and the "(%s)" after the first status=,
// to the end of the line (Postfix 3.10.13, src/global/log_adhoc.c). Taking the
// first status= keeps the reply whole when the far end's text holds one of
// its own. A line that is not a delivery line gives "", "".
func deliveryReply(rest string) (relay, reply string) {
loc := deliveryRe.FindStringSubmatchIndex(rest)
if loc == nil {
return "", ""
}
relay = rest[loc[8]:loc[9]]
tail := strings.TrimSpace(rest[loc[1]:])
if strings.HasPrefix(tail, "(") && strings.HasSuffix(tail, ")") {
reply = tail[1 : len(tail)-1]
}
return relay, reply
}
// expiryRe matches qmgr's line for a message it gave up on:
//
// postfix/qmgr[93]: 41E862C00D9E: from=<app@example.org>, status=expired, returned to sender
@@ -225,18 +256,28 @@ func Run(ctx context.Context, path string, st StatusStore, retention RetentionDa
}
keeper.add(line, now, catchingUp)
refusals.handle(line, now, Place{FP: tr.line.FP, End: tr.lineStart})
// A delivery or expiry line no send-log row takes may be of mail
// accepted on port 25: the inbound journal's rows take its result.
queueID, recipient, status, ok := parseDelivery(line)
if !ok {
if queueID, ok := parseExpiry(line); ok {
if _, err := st.ExpireMessage(queueID); err != nil {
n, err := st.ExpireMessage(queueID)
if err != nil {
log.Printf("log-tailer: expire %s: %v", queueID, err)
}
if n == 0 && journal != nil {
journal.expired(line, now, queueID)
}
}
return
}
if _, err := st.UpdateStatus(queueID, recipient, status); err != nil {
n, err := st.UpdateStatus(queueID, recipient, status)
if err != nil {
log.Printf("log-tailer: update %s/%s -> %s: %v", queueID, recipient, status, err)
}
if n == 0 && journal != nil {
journal.upstream(line, now, queueID, recipient, status)
}
})
}