refused messages: the page shows what Postfix wrote (stage 2b, 0009)

The Delivery log box of every refused row said "This message never reached
the queue, so Postfix wrote no delivery lines for it" - untrue: Postfix
wrote a line for each. The owner, 2026-10-13: the box is required on every
message page. Pictured for him first (dev/shots, the refused variant of
out-message: wrong From with its queue id lines, rate-limited with the
NOQUEUE line); the variant has no mockup screen of its own - the guard ties
screens to pages, and its outline is out-message's.

- Wrong From: cleanup gives the message a queue id before the headers are
  judged; the sender check reads {i} at end-of-headers and writes it on the
  refused row. Its lines - stored under the id like any other's - are on the
  page, but only those from 10 minutes before to 1 minute after the refusal
  (store.SendLogLines for a rejected row): Postfix frees the id at once, and
  a later message with it lends nothing. No mail.log fallback for a refused
  row, for the same reason.
- Rate-limited and wrong client: refused at MAIL FROM, no queue id. Migration
  0009 adds send_log.refusal_line; logtail.refusalLines matches smtpd's
  "NOQUEUE: milter-reject: MAIL from ...; from=<...>" (not port 25's, not
  CONNECT/EHLO) and store.AttachRefusalLine puts it on the newest refused row
  of that sender (and login, when the line names it) within two minutes; a
  line whose row is not written yet waits up to 30 s (retried at each pass).
- The page: the queue id fact says "given, the message was not queued" or
  "none - refused at MAIL FROM, before one is given"; the box's caption is
  "The line from mail.log for this refusal" for the latter; a row from
  before says the line was not kept. LogLineLevel colours a refusal by its
  reply: 5xx red, 4xx amber (IPv6 clients included). The id in the fact is
  not muted as in the picture: the kit has no muted inline.

Tests: store (attach by sender, login, time; not a queued row, not one an
hour off; a refused row's lines are its own minutes); logtail (attach, wait
and retry, port 25 and other lines ignored); milter (the refused row carries
{i}); view (facts, caption, line levels); handlers (the line on the page,
the sentence without it). e2e refusals_show_postfixs_line: a client-address
refusal made there, and the pages of a wrong From, a rate-limited and a
wrong client refusal each show Postfix's line.

schema-migrations row 9, CHANGELOG [Unreleased], audit plan line ticked,
roadmap 25/62.
Evidence on the test server for this tree: make e2e exit 0 (the new case
in 5.4 s); go test ./... ok but for two schema tests, fixed after the run
(the schema at 0009) - store tests ok locally; the full run follows before
the push.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
mix
2026-10-14 16:56:29 +03:00
co-authored by Claude Opus 5.5
parent c452606175
commit e166a2ebd6
22 changed files with 484 additions and 35 deletions
+8 -1
View File
@@ -39,6 +39,7 @@ type StatusStore interface {
CountQueuedOlderThan(cutoff time.Time) (int64, error)
SetQueuedOverLifetime(at time.Time, n int64) error
PruneDeliveryLogDays(now time.Time) error
RefusalStore
DeleteSendLogBefore(cutoff time.Time) (int64, error)
InsertSendLogLines(lines []store.SendLogLine, once bool) (int, error)
PruneSendLogLines(cutoff time.Time) (int64, error)
@@ -162,8 +163,13 @@ func Run(ctx context.Context, path string, st StatusStore, retention RetentionDa
}
keeper := newLineKeeper(st)
refusals := &refusalLines{st: st}
flush := func() {
keeper.flush()
refusals.retry(time.Now().UTC())
}
tr := &tracker{st: st, path: path}
return follow(ctx, path, tr, caughtUp, keeper.flush, func(line string) {
return follow(ctx, path, tr, caughtUp, flush, func(line string) {
// Until the first read reaches end-of-file the lines are those written
// while the panel was down - and, after an unclean stop, some it had
// read already.
@@ -181,6 +187,7 @@ func Run(ctx context.Context, path string, st StatusStore, retention RetentionDa
deferred.handle(line, now, catchingUp, tr.line)
}
keeper.add(line, now, catchingUp)
refusals.handle(line, now)
queueID, recipient, status, ok := parseDelivery(line)
if !ok {
return
+3
View File
@@ -283,6 +283,9 @@ func (c *captureStore) AddReconcileMissed(_ time.Time, n int64) error {
return nil
}
func (c *captureStore) AddTailerBlind(time.Time, time.Duration) error { return nil }
func (c *captureStore) AttachRefusalLine(string, string, time.Time, string) (bool, error) {
return true, nil
}
func (c *captureStore) CountQueuedOlderThan(time.Time) (int64, error) {
c.mu.Lock()
defer c.mu.Unlock()
+74
View File
@@ -0,0 +1,74 @@
package logtail
import (
"log"
"regexp"
"time"
)
// Postfix's line about a message the journal milter refused at MAIL FROM - a
// level-2 limit, a client address off the application's list - is put on the
// row the milter wrote, so the message's page shows it (audit 2026-10-12,
// stage 2b). The refusal has no queue id; the line is matched to its row by
// sender, login and time. As the stand writes it:
//
// postfix/smtpd[1149]: NOQUEUE: milter-reject: MAIL from unknown[10.77.0.1]: 451 4.7.1 Service unavailable - try again later; from=<billing@sender.e2e.test> proto=ESMTP helo=<e2e-client.e2e.test>
//
// Port 25's smtpd (postfix/inbound/smtpd) is a stranger's and has no such row.
var refusalLineRe = regexp.MustCompile(`^\S+ postfix/(?:([a-z0-9-]+)/)?smtpd\[\d+\]: NOQUEUE: milter-reject: MAIL from \S+: [45]\d\d .*?; from=<([^>]*)>(?:.*? sasl_username=<?([^>\s]*)>?)?`)
// refusalLineWait is how long a line waits for its row: the milter writes the
// row when it refuses, which can be read a moment after the line.
const refusalLineWait = 30 * time.Second
// RefusalStore is where the lines go. *store.Store satisfies it.
type RefusalStore interface {
AttachRefusalLine(login, from string, at time.Time, line string) (bool, error)
}
type pendingRefusal struct {
login, from, line string
at, until time.Time
}
// refusalLines puts each refusal line on its row, and keeps for a while the
// lines whose row is not there yet.
type refusalLines struct {
st RefusalStore
pending []pendingRefusal
}
// handle looks at one mail.log line.
func (r *refusalLines) handle(line string, now time.Time) {
at, rest := lineTime(line, now)
m := refusalLineRe.FindStringSubmatch(rest)
if m == nil || m[1] == "inbound" {
return
}
p := pendingRefusal{login: m[3], from: m[2], line: line, at: at, until: now.Add(refusalLineWait)}
if !r.attach(p) {
r.pending = append(r.pending, p)
}
}
// retry tries the waiting lines again and lets go of those that waited long
// enough.
func (r *refusalLines) retry(now time.Time) {
kept := r.pending[:0]
for _, p := range r.pending {
if r.attach(p) || now.After(p.until) {
continue
}
kept = append(kept, p)
}
r.pending = kept
}
func (r *refusalLines) attach(p pendingRefusal) bool {
ok, err := r.st.AttachRefusalLine(p.login, p.from, p.at, p.line)
if err != nil {
log.Printf("log-tailer: refusal line: %v", err)
return true // not retried: the store said no for its own reason
}
return ok
}
+58
View File
@@ -0,0 +1,58 @@
package logtail
import (
"testing"
"time"
)
type refusalProbe struct {
rows int // rows waiting for a line
lines []string
who []string
}
func (r *refusalProbe) AttachRefusalLine(login, from string, _ time.Time, line string) (bool, error) {
if r.rows == 0 {
return false, nil
}
r.rows--
r.lines = append(r.lines, line)
r.who = append(r.who, login+"|"+from)
return true, nil
}
// The NOQUEUE line of a refusal at MAIL FROM goes on its row; a line whose
// row is not written yet waits and is tried again; port 25's and other
// lines are not refusals of an application.
func TestRefusalLines(t *testing.T) {
now := time.Date(2026, 10, 14, 12, 0, 0, 0, time.UTC)
st := &refusalProbe{}
r := &refusalLines{st: st}
limit := `2026-10-14T12:00:00+00:00 mail postfix/smtpd[1149]: NOQUEUE: milter-reject: MAIL from unknown[10.77.0.1]: 451 4.7.1 Service unavailable - try again later; from=<billing@sender.e2e.test> proto=ESMTP helo=<e2e-client.e2e.test> sasl_username=<l2app>`
for _, other := range []string{
`2026-10-14T12:00:00+00:00 mail postfix/inbound/smtpd[372]: NOQUEUE: milter-reject: MAIL from unknown[203.0.113.7]: 451 4.7.1 Service unavailable - try again later; from=<a@stranger.example> proto=ESMTP helo=<x>`,
`2026-10-14T12:00:00+00:00 mail postfix/smtpd[11]: NOQUEUE: milter-reject: CONNECT from unknown[10.77.0.1]: 451 4.7.1 Service unavailable - try again later; proto=SMTP`,
`2026-10-14T12:00:00+00:00 mail postfix/smtpd[11]: NOQUEUE: reject: RCPT from unknown[10.77.0.1]: 553 5.7.1 <a@x>: Sender address rejected: not owned by user app1; from=<a@x> to=<b@y> proto=ESMTP helo=<x>`,
} {
r.handle(other, now)
}
if len(r.pending) != 0 {
t.Fatalf("lines that are not an application's refusal at MAIL wait: %+v", r.pending)
}
r.handle(limit, now) // its row is not there yet
if len(r.pending) != 1 {
t.Fatalf("pending %d, want the line waiting", len(r.pending))
}
st.rows = 1 // the milter's row is written
r.retry(now.Add(time.Second))
if len(r.pending) != 0 || len(st.lines) != 1 || st.who[0] != "l2app|billing@sender.e2e.test" {
t.Fatalf("after the retry: pending %d, attached %v", len(r.pending), st.who)
}
r.handle(limit, now) // no row ever comes
r.retry(now.Add(refusalLineWait + time.Second))
if len(r.pending) != 0 {
t.Fatal("a line kept waiting past its time")
}
}
+4
View File
@@ -98,3 +98,7 @@ func TestRetentionLoopUsesUpdatedValue(t *testing.T) {
age := time.Since(st.lastCutoff())
t.Fatalf("second prune cutoff age %v, want about 7 days", age)
}
func (s *retentionProbeStore) AttachRefusalLine(string, string, time.Time, string) (bool, error) {
return true, nil
}
+6 -2
View File
@@ -268,7 +268,10 @@ func (s *senderSession) Headers(h textproto.MIMEHeader, m *milter.Modifier) (mil
return milter.RespContinue, nil
}
log.Printf("sender-check: application %q refused: %s (From: %s)", s.login, why, shown)
s.record(shown)
// cleanup has given the message a queue id by the end of its headers
// (milter_end_of_header_macros has {i}); Postfix's lines about it - the
// refusal among them - are found by it on the message's page.
s.record(shown, macro(m, "i"))
// A reply code of its own, so that the application is told what is wrong
// and, the refusal being permanent, does not send the message again.
return milter.NewResponseStr('y', "550 5.7.1 From: "+shown+" - "+why), nil
@@ -352,7 +355,7 @@ func (s *senderSession) judge() (shown, why string, err error) {
// each recipient as an accepted message has: nothing was sent to anybody, and
// a refused message with a thousand recipients would otherwise be a thousand
// rows at the price of one.
func (s *senderSession) record(shown string) {
func (s *senderSession) record(shown, queueID string) {
if s.refused == nil {
return
}
@@ -364,6 +367,7 @@ func (s *senderSession) record(shown string) {
to += fmt.Sprintf(" and %d more", more)
}
e := store.SendLogEntry{
QueueID: queueID,
Domain: s.domain(),
AppLogin: s.login,
From: s.from,
+3 -2
View File
@@ -63,7 +63,8 @@ func offer(t *testing.T, bind *Bindings, login, envelope string, headers ...[2]s
t.Fatal(err)
}
}
resp, err := s.Headers(nil, mods(nil))
// cleanup has given the message its queue id by the end of the headers.
resp, err := s.Headers(nil, mods(map[string]string{"i": "7BCAF3C75DA0"}))
if err != nil {
t.Fatal(err)
}
@@ -153,7 +154,7 @@ func TestSenderCheck(t *testing.T) {
t.Fatalf("the reason %q names %s", reason, other)
}
}
if len(rows) != 1 || rows[0].Reason != store.ReasonFrom || rows[0].To != "someone@elsewhere.example" || rows[0].AppLogin != c.login {
if len(rows) != 1 || rows[0].Reason != store.ReasonFrom || rows[0].To != "someone@elsewhere.example" || rows[0].AppLogin != c.login || rows[0].QueueID != "7BCAF3C75DA0" {
t.Fatalf("log rows: %+v", rows)
}
})
@@ -0,0 +1,9 @@
-- What Postfix wrote about a refusal (audit 2026-10-12, stage 2b). A message
-- the journal milter refuses at MAIL FROM - a level-2 limit, a client address
-- off the application's list - has no queue id, and Postfix's one line about
-- it, "NOQUEUE: milter-reject: MAIL from ...", was nowhere the message's page
-- could find it. The log tailer puts it on the row it belongs to. A message
-- refused for its From: line has a queue id (cleanup gives one before the
-- headers are judged), and its lines are found by it like any other's.
-- DDL only; a row written earlier has '' and says so on its page.
ALTER TABLE send_log ADD COLUMN refusal_line TEXT NOT NULL DEFAULT '';
+79
View File
@@ -0,0 +1,79 @@
package store
import (
"testing"
"time"
)
// Postfix's line about a refusal at MAIL FROM goes on the row of that sender
// and login refused about then, the newest without a line; a queued row, a
// row of another sender and a row from long ago are not it (audit
// 2026-10-12, stage 2b).
func TestAttachRefusalLine(t *testing.T) {
st := openTestStore(t)
now := time.Now().UTC()
for _, e := range []SendLogEntry{
{AppLogin: "l2app", From: "billing@example.com", Reason: ReasonLimit},
{AppLogin: "l2app", From: "other@example.com", Reason: ReasonLimit},
} {
if err := st.InsertRejected(e); err != nil {
t.Fatal(err)
}
}
if err := st.InsertQueued(SendLogEntry{QueueID: "Q1", AppLogin: "l2app", From: "billing@example.com", To: "x@example.net"}); err != nil {
t.Fatal(err)
}
line := "Oct 14 12:00:00 mail postfix/smtpd[1]: NOQUEUE: milter-reject: MAIL from unknown[10.77.0.1]: 451 4.7.1 Service unavailable"
if ok, err := st.AttachRefusalLine("l2app", "Billing@example.com", now, line); err != nil || !ok {
t.Fatalf("attach = %v, %v", ok, err)
}
// The row has its line now; a second line finds no row without one.
if ok, _ := st.AttachRefusalLine("l2app", "billing@example.com", now, line); ok {
t.Fatal("a second line took a row that has one")
}
if ok, _ := st.AttachRefusalLine("l2app", "other@example.com", now.Add(-time.Hour), line); ok {
t.Fatal("a line an hour off took a row")
}
if ok, _ := st.AttachRefusalLine("someone", "other@example.com", now, line); ok {
t.Fatal("a line of another login took a row")
}
rows, _ := st.QuerySendLog(SendLogFilter{AllDomains: true}, 10, 0)
got := 0
for _, r := range rows {
full, _ := st.GetSendLog(r.ID)
if full.RefusalLine != "" {
got++
if full.From != "billing@example.com" || full.Status != StatusRejected {
t.Errorf("the line went to %+v", full)
}
}
}
if got != 1 {
t.Fatalf("%d rows have a line, want 1", got)
}
}
// A refused message's lines are those of a few minutes around the refusal:
// its queue id was freed at once, and a later message with the same id lends
// it nothing.
func TestSendLogLinesOfARefusalAreItsOwn(t *testing.T) {
st := openTestStore(t)
if err := st.InsertRejected(SendLogEntry{QueueID: "7BCAF3C75DA0", AppLogin: "shop", From: "shop@shop.example", Reason: ReasonFrom}); err != nil {
t.Fatal(err)
}
now := time.Now().UTC()
if _, err := st.InsertSendLogLines([]SendLogLine{
{QueueID: "7BCAF3C75DA0", At: now.Add(-time.Second), Line: "its own refusal"},
{QueueID: "7BCAF3C75DA0", At: now.Add(20 * time.Minute), Line: "a later message with the id"},
}, false); err != nil {
t.Fatal(err)
}
rows, _ := st.QuerySendLog(SendLogFilter{AllDomains: true}, 1, 0)
lines, err := st.SendLogLines(rows[0], 5*24*time.Hour, 10)
if err != nil {
t.Fatal(err)
}
if len(lines) != 1 || lines[0].Line != "its own refusal" {
t.Fatalf("lines = %+v, want its own alone", lines)
}
}
+5 -5
View File
@@ -47,7 +47,7 @@ func TestBaselineSchema(t *testing.T) {
"applications": "id domain_id login address_mode auth_ip_restrict auth_allowed_ips created_at",
"application_addresses": "id application_id address",
"rate_limits": "id scope ref_id max_messages window_seconds mode auto_multiplier auto_updated_at",
"send_log": "id queue_id domain app_login from_addr to_addr subject status created_at updated_at reason header_from domain_id",
"send_log": "id queue_id domain app_login from_addr to_addr subject status created_at updated_at reason header_from domain_id refusal_line",
"logtail_state": "path fingerprint read_offset updated_at",
"inbound_domains": "id name recipient_mode created_at",
"inbound_transports": "inbound_domain_id host port tls_mode",
@@ -115,8 +115,8 @@ func TestBaselineSchema(t *testing.T) {
}
var version, appID int64
if err := st.db.QueryRow("PRAGMA user_version").Scan(&version); err != nil || version != 8 {
t.Errorf("user_version = %d, %v; want 8", version, err)
if err := st.db.QueryRow("PRAGMA user_version").Scan(&version); err != nil || version != 9 {
t.Errorf("user_version = %d, %v; want 9", version, err)
}
if err := st.db.QueryRow("PRAGMA application_id").Scan(&appID); err != nil || appID != applicationID {
t.Errorf("application_id = %#x, %v; want %#x", appID, err, applicationID)
@@ -192,8 +192,8 @@ func TestOpenMigratesA20Database(t *testing.T) {
}
defer st.Close()
var version int
if err := st.db.QueryRow("PRAGMA user_version").Scan(&version); err != nil || version != 8 {
t.Errorf("user_version = %d, %v; want 8", version, err)
if err := st.db.QueryRow("PRAGMA user_version").Scan(&version); err != nil || version != 9 {
t.Errorf("user_version = %d, %v; want 9", version, err)
}
if ok, err := st.UserExists(); err != nil || !ok {
t.Errorf("the user of the 2.0 database: exists = %v, %v", ok, err)
+37 -3
View File
@@ -221,12 +221,46 @@ type SendLogRow struct {
// Reason and HeaderFrom: see SendLogEntry.
Reason string
HeaderFrom string
CreatedAt time.Time
// RefusalLine is Postfix's line about a refusal that has no queue id
// (migration 0009); "" for every other row, and for one refused before.
RefusalLine string
CreatedAt time.Time
// UpdatedAt is when the status last changed — the log-tailer's report of
// the delivery attempt. It equals CreatedAt while a row is still queued.
UpdatedAt time.Time
}
// AttachRefusalLine puts Postfix's line about a refusal at MAIL FROM on the
// row the journal milter wrote for it (audit 2026-10-12, stage 2b). Such a
// refusal has no queue id, so the row is found by what the line says: a
// refused row of that sender - and login, when the line names one - without a
// line yet, written within two minutes of the line's time; the newest when
// there are several. It reports whether a row took the line.
func (s *Store) AttachRefusalLine(login, from string, at time.Time, line string) (bool, error) {
res, err := s.db.Exec(
`UPDATE send_log SET refusal_line = ? WHERE id = (
SELECT id FROM send_log
WHERE status = ? AND queue_id = '' AND refusal_line = ''
AND reason IN ('', ?, ?) AND from_addr = ? COLLATE NOCASE
AND (? = '' OR app_login = ?)
AND created_at BETWEEN ? AND ?
ORDER BY id DESC LIMIT 1)`,
line, StatusRejected, ReasonLimit, ReasonClient, from, login, login,
at.Add(-refusalLineWindow).UTC().Format(time.RFC3339),
at.Add(refusalLineWindow).UTC().Format(time.RFC3339),
)
if err != nil {
return false, fmt.Errorf("attach refusal line: %w", err)
}
n, _ := res.RowsAffected()
return n > 0, nil
}
// refusalLineWindow is how far apart a refusal's row and its log line may be
// dated: the milter writes the row at MAIL FROM, Postfix the line a moment
// later, and the log's stamps are whole seconds.
const refusalLineWindow = 2 * time.Minute
// ErrSendLogNotFound is returned by GetSendLog for an id that no longer exists.
// Send-log rows are pruned on the retention window, so a bookmarked delivery
// disappearing is expected, not a fault.
@@ -240,10 +274,10 @@ func (s *Store) GetSendLog(id int64) (SendLogRow, error) {
createdAt, updatedAt string
)
err := s.db.QueryRow(
`SELECT id, queue_id, domain, COALESCE(domain_id, 0), app_login, from_addr, to_addr, subject, status, reason, header_from, created_at, updated_at
`SELECT id, queue_id, domain, COALESCE(domain_id, 0), app_login, from_addr, to_addr, subject, status, reason, header_from, refusal_line, created_at, updated_at
FROM send_log WHERE id = ?`, id,
).Scan(&row.ID, &row.QueueID, &row.Domain, &row.DomainID, &row.AppLogin, &row.From, &row.To,
&row.Subject, &row.Status, &row.Reason, &row.HeaderFrom, &createdAt, &updatedAt)
&row.Subject, &row.Status, &row.Reason, &row.HeaderFrom, &row.RefusalLine, &createdAt, &updatedAt)
if errors.Is(err, sql.ErrNoRows) {
return SendLogRow{}, ErrSendLogNotFound
}
+17 -1
View File
@@ -93,10 +93,19 @@ func (s *Store) InsertSendLogLines(lines []SendLogLine, once bool) (int, error)
// A line is the row's when it is dated from shortly before the row was written
// to lifetime after it; lifetime is how long Postfix may keep working on a
// message, with a margin, and the caller's to state (postfix.RetryPolicy).
//
// A refused message - one refused for its From: line has a queue id, given
// before the headers were judged - is over in a moment, and Postfix frees the
// id at once: its lines are those of a few minutes around the refusal, so
// that the next message to get the id lends it none of its own.
func (s *Store) SendLogLines(row SendLogRow, lifetime time.Duration, n int) ([]SendLogLine, error) {
if row.QueueID == "" || n <= 0 {
return nil, nil
}
before := sendLogLineBefore
if row.Status == StatusRejected {
before, lifetime = refusedLinesBefore, refusedLinesAfter
}
rows, err := s.db.Query(
`SELECT logged_at, line FROM (
SELECT id, logged_at, line FROM send_log_lines
@@ -104,7 +113,7 @@ func (s *Store) SendLogLines(row SendLogRow, lifetime time.Duration, n int) ([]S
ORDER BY id DESC LIMIT ?)
ORDER BY id`,
row.QueueID,
row.CreatedAt.Add(-sendLogLineBefore).UTC().Format(time.RFC3339),
row.CreatedAt.Add(-before).UTC().Format(time.RFC3339),
row.CreatedAt.Add(lifetime).UTC().Format(time.RFC3339),
n,
)
@@ -150,3 +159,10 @@ func (s *Store) PruneSendLogLines(cutoff time.Time) (int64, error) {
n, _ := res.RowsAffected()
return n, nil
}
// The lines of a refused message: from its connection - a slow client may
// take minutes to reach the end of its headers - to just after the refusal.
const (
refusedLinesBefore = 10 * time.Minute
refusedLinesAfter = time.Minute
)
+15 -2
View File
@@ -241,6 +241,12 @@ const deliveryLogMargin = 24 * time.Hour
// reported as a fault, and that one is logged for the operator as well.
func (h *Handlers) deliveryLog(row store.SendLogRow) ([]view.LogLine, string) {
if row.QueueID == "" {
if row.RefusalLine != "" {
return logLines([]string{row.RefusalLine}), ""
}
if row.Status == store.StatusRejected {
return nil, "Postfix's line about this refusal is not stored: the message was refused before version 2.5.1, or the line was never read."
}
return nil, "This message never reached the queue, so Postfix wrote no delivery lines for it."
}
stored, err := h.store.SendLogLines(row, h.cfg.RetryPolicy.LongestLifetime()+deliveryLogMargin, deliveryLogLines)
@@ -252,7 +258,9 @@ func (h *Handlers) deliveryLog(row store.SendLogRow) ([]view.LogLine, string) {
for i, l := range stored {
lines[i] = l.Line
}
if len(lines) == 0 {
// A refused message's id was freed at once and may be another's by now:
// what was not stored with the row is not looked for in the log.
if len(lines) == 0 && row.Status != store.StatusRejected {
lines, err = logtail.QueueLines(h.cfg.MailLogPath, row.QueueID, deliveryLogLines)
if err != nil && !errors.Is(err, fs.ErrNotExist) {
logf("panel: delivery log %s: %v", row.QueueID, err)
@@ -262,6 +270,11 @@ func (h *Handlers) deliveryLog(row store.SendLogRow) ([]view.LogLine, string) {
if len(lines) == 0 {
return nil, "Nothing is stored for this queue id and nothing is in the current mail log. Lines are kept with the row since version 2.2.0; a message accepted before that had them only in mail.log, which has since rotated."
}
return logLines(lines), ""
}
// logLines are lines of mail.log as the delivery log box shows them.
func logLines(lines []string) []view.LogLine {
out := make([]view.LogLine, len(lines))
for i, line := range lines {
// A line whose head is not a timestamp the log format recognises keeps its
@@ -269,7 +282,7 @@ func (h *Handlers) deliveryLog(row store.SendLogRow) ([]view.LogLine, string) {
stamp, rest := logtail.SplitTimestamp(line)
out[i] = view.LogLine{Time: stamp, Text: rest, Level: view.LogLineLevel(rest)}
}
return out, ""
return out
}
// deliveriesBackURL rebuilds the delivery-log URL a detail page was opened
@@ -5,6 +5,7 @@ import (
"net/url"
"strings"
"testing"
"time"
"github.com/mixeme/selfpost/internal/store"
"github.com/mixeme/selfpost/internal/web/auth"
@@ -150,3 +151,38 @@ func TestADomainAddedAgainShowsItsAssigneeNothingOld(t *testing.T) {
t.Errorf("the global log is missing a row")
}
}
// A refused message's page shows what Postfix wrote about the refusal: the
// NOQUEUE line put on the row of a refusal at MAIL FROM, or, for a row from
// before that line was kept, a sentence that says so (audit 2026-10-12,
// stage 2b).
func TestARefusedMessageShowsPostfixsLine(t *testing.T) {
h, st := inboundHandlers(t)
root := userWith(t, st, "root", store.RoleGlobal, store.Reach{})
d, _ := st.AddDomain("example.org", "mail")
st.AddApplication(d.ID, "alerts", store.AddressModeWildcard, nil)
for _, subject := range []string{"with its line", "from before"} {
if err := st.InsertRejected(store.SendLogEntry{Domain: "example.org", AppLogin: "alerts", From: "alerts@example.org", Subject: subject, Reason: store.ReasonLimit}); err != nil {
t.Fatal(err)
}
if subject == "with its line" {
line := time.Now().UTC().Format("2006-01-02T15:04:05Z") + " mail postfix/smtpd[2131]: NOQUEUE: milter-reject: MAIL from unknown[198.51.100.7]: 451 4.7.1 Service unavailable - try again later; from=<alerts@example.org> proto=ESMTP helo=<app>"
if ok, err := st.AttachRefusalLine("alerts", "alerts@example.org", time.Now(), line); err != nil || !ok {
t.Fatalf("attach = %v, %v", ok, err)
}
}
}
rows, _ := st.QuerySendLog(store.SendLogFilter{AllDomains: true}, 10, 0)
for _, r := range rows {
body := send(h.HandleDelivery, &root, "GET", "/outbound/log/"+idStr(r.ID), map[string]string{"id": idStr(r.ID)}, nil).Body.String()
if r.Subject == "with its line" {
if !strings.Contains(body, `<span class="sp-w">mail postfix/smtpd[2131]: NOQUEUE: milter-reject: MAIL from unknown[198.51.100.7]: 451 4.7.1`) {
t.Errorf("the page of a refusal with its line lacks it:\n%s", body)
}
continue
}
if !strings.Contains(body, "the message was refused before version 2.5.1, or the line was never read") {
t.Errorf("the page of a refusal without its line does not say why")
}
}
}
+24 -3
View File
@@ -12,6 +12,7 @@ package view
import (
"net/url"
"regexp"
"strconv"
"strings"
"time"
@@ -246,15 +247,23 @@ func NewOutMessage(m Meta, in MessageInput) *OutMessage {
{Label: "Application", Value: value(in.App, in.AppHref), Mono: true},
}
history := Box{No: "02", Title: "History", Help: topicLink(HelpLog, "What the statuses mean")}
delivery := Box{No: "03", Title: "Delivery log", End: Rich("Lines from ", Code("mail.log"), " with this queue id")}
if in.Status == "rejected" {
// A refused message was never accepted and never queued: its one date
// is the refusal, and it has no queue id to look anything up by.
// is the refusal. One refused for its From: line was given a queue id
// before its headers were judged; one refused at MAIL FROM was not,
// and its box holds Postfix's one line about the refusal.
facts = append(facts, Fact{Label: "Refused", Value: Plain(formatStamp(in.Accepted))})
if in.Reason == "from" {
facts = append(facts, Fact{Label: "From: line", Value: Plain(orDash(in.HeaderFrom)), Mono: true, Small: true})
history.Help = topicLink(HelpSenderCheck, "What an application may write in From:")
}
facts = append(facts, Fact{Label: "Queue id", Value: Plain("none - never queued")})
if in.QueueID != "" {
facts = append(facts, Fact{Label: "Queue id", Value: Rich(Mono(in.QueueID), " — given, the message was not queued")})
} else {
facts = append(facts, Fact{Label: "Queue id", Value: Rich("none - refused at ", Mono("MAIL FROM"), ", before one is given")})
delivery.End = Rich("The line from ", Code("mail.log"), " for this refusal")
}
} else {
facts = append(facts,
Fact{Label: "Accepted", Value: Plain(formatStamp(in.Accepted))},
@@ -276,7 +285,7 @@ func NewOutMessage(m Meta, in MessageInput) *OutMessage {
Message: Box{No: "01", Title: "Message"},
Facts: facts,
History: history,
Delivery: Box{No: "03", Title: "Delivery log", End: Rich("Lines from ", Code("mail.log"), " with this queue id")},
Delivery: delivery,
LogEmpty: EmptyState{Icon: "ti-file-text"},
}
}
@@ -330,5 +339,17 @@ func LogLineLevel(text string) string {
case strings.Contains(text, "status=deferred"), strings.Contains(text, " warning:"):
return "warn"
}
// A refusal is as permanent as its reply: "550"/"5.7.1" red, "451" amber.
if m := rejectCodeRe.FindStringSubmatch(text); m != nil {
if m[1] == "5" {
return "error"
}
return "warn"
}
return ""
}
// rejectCodeRe finds the reply of a refusal Postfix logged: the code or the
// enhanced status after "reject: <stage> from <client>[<address>]:", from
// smtpd ("reject:") and cleanup ("milter-reject:") alike.
var rejectCodeRe = regexp.MustCompile(`reject: \S+ from [^\]]*\]: ([45])(?:\d\d|\.\d+\.\d+) `)
+20 -12
View File
@@ -124,23 +124,28 @@ func TestOutMessageWithoutSubjectOrRecipient(t *testing.T) {
out := renderSignedIn(t, "out-message", p)
pageHas(t, "Message", out, `<h1 class="title is-4">(no subject)</h1>`, `<span>—</span><i class="ti ti-arrow-right sp-muted"></i><span>—</span>`,
`<span class="tag is-light">rejected</span>`, `href="/outbound/log?domain=gone.example&amp;p=2"`,
`<dd>gone.example</dd>`, `<dd class="sp-mono">old</dd>`, `<dt>Refused</dt>`, `<dd>none - never queued</dd>`,
`never reached the queue`)
`<dd>gone.example</dd>`, `<dd class="sp-mono">old</dd>`, `<dt>Refused</dt>`,
`<dd>none - refused at <span class="sp-mono">MAIL FROM</span>, before one is given</dd>`,
`The line from <code>mail.log</code> for this refusal`, `never reached the queue`)
pageLacks(t, "Message", out, `<pre`, `<a href="/outbound/domains/`, `Status reported`, `From: line`)
}
// A row refused for its From: line says so in its tag, shows what the line
// said, and its History links to the topic that explains the rule.
// said, and its History links to the topic that explains the rule. The
// message had a queue id - cleanup gives one before the headers are judged -
// and its box shows Postfix's lines by it, the refusal red.
func TestOutMessageRefusedForItsFromLine(t *testing.T) {
at := time.Date(2026, 9, 21, 13, 40, 55, 0, time.UTC)
p := NewOutMessage(admin(), MessageInput{ID: 184231, Domain: "shop.example.org", App: "shop",
p := NewOutMessage(admin(), MessageInput{ID: 184231, QueueID: "7BCAF3C75DA0", Domain: "shop.example.org", App: "shop",
From: "shop@shop.example.org", To: "k.berg@web.de", Subject: "A message from the director",
Status: "rejected", Reason: "from", HeaderFrom: "director@example.org", Accepted: at, Reported: at}).
WithDeliveryLog(nil, "This message never reached the queue, so Postfix wrote no delivery lines for it.")
WithDeliveryLog([]LogLine{{Time: "13:40:55", Text: "postfix/cleanup[2120]: 7BCAF3C75DA0: milter-reject: END-OF-MESSAGE from unknown[198.51.100.7]: 5.7.1 From: director@example.org - not an address this application sends as",
Level: LogLineLevel("postfix/cleanup[2120]: 7BCAF3C75DA0: milter-reject: END-OF-MESSAGE from unknown[198.51.100.7]: 5.7.1 From: director@example.org - not an address this application sends as")}}, "")
out := renderSignedIn(t, "out-message", p)
pageHas(t, "Message", out, `<span class="tag is-danger is-light">wrong From</span>`,
`<dt>From: line</dt><dd class="sp-mono sp-small">director@example.org</dd>`, `<dt>Refused</dt>`,
`<dd>none - never queued</dd>`, `href="/help#sender-check"`)
`<span class="sp-mono">7BCAF3C75DA0</span> — given, the message was not queued`, `href="/help#sender-check"`,
`Lines from <code>mail.log</code> with this queue id`, `<span class="sp-e">postfix/cleanup[2120]: 7BCAF3C75DA0: milter-reject`)
}
func TestSendLogTag(t *testing.T) {
@@ -184,12 +189,15 @@ func TestStepLevelAndLogLineLevel(t *testing.T) {
}
}
for text, want := range map[string]string{
"postfix/smtp[1]: Q1: to=<a@b>, status=sent (250 OK)": "",
"postfix/smtp[1]: Q1: to=<a@b>, status=deferred (try later)": "warn",
"postfix/smtp[1]: Q1: to=<a@b>, status=bounced (no such user)": "error",
"postfix/qmgr[1]: Q1: status=expired, returned to sender": "error",
"postfix/smtp[1]: warning: TLS library problem": "warn",
"opendkim[1]: Q1: DKIM-Signature field added": "",
"postfix/smtp[1]: Q1: to=<a@b>, status=sent (250 OK)": "",
"postfix/smtp[1]: Q1: to=<a@b>, status=deferred (try later)": "warn",
"postfix/smtp[1]: Q1: to=<a@b>, status=bounced (no such user)": "error",
"postfix/qmgr[1]: Q1: status=expired, returned to sender": "error",
"postfix/smtp[1]: warning: TLS library problem": "warn",
"opendkim[1]: Q1: DKIM-Signature field added": "",
"postfix/cleanup[1]: Q1: milter-reject: END-OF-MESSAGE from unknown[10.77.0.1]: 5.7.1 From: x - no": "error",
"postfix/smtpd[1]: NOQUEUE: milter-reject: MAIL from unknown[10.77.0.1]: 451 4.7.1 Service unavailable": "warn",
"postfix/smtpd[1]: NOQUEUE: reject: RCPT from unknown[2001:db8::1]: 554 5.7.1 <a@b>: Relay access denied": "error",
} {
if got := LogLineLevel(text); got != want {
t.Errorf("LogLineLevel(%q) = %q, want %q", text, got, want)