from-header-binding: the sender check, what it costs, its pages; guide, architecture, security

A message's From: line is held against the application that sends it, under
the rule Postfix holds the envelope sender to. OpenDKIM chooses the key by
that line, so until now an application could have mail signed as another
domain of the server or as the server itself.

The check (internal/milter/sender.go)
- A role of the panel on a socket of its own (SENDER_CHECK_SOCKET), first in
  smtpd_milters and strict: default_action=tempfail. The journal milter keeps
  its socket and stays fail-open. postfix-wrapper waits for the new socket.
- It reads no database: what an application may send as comes from Postfix's
  own sender_login_maps, what the server signs for from OpenDKIM's
  SigningTable. Both are read again when they change; one that cannot be
  read defers mail.
- One From: field, one address, allowed by the envelope's lookup; then no
  other name of the SigningTable may appear anywhere in the raw line.
  Refused: 550 5.7.1 naming the line, reduced to printable ASCII.
- A refusal is a send-log row with reason "from" and the header address
  (migration 0006), written by another goroutine and not waited for. The
  journal's refusals record their reason too (limit, client), and the
  notifier takes only a limit's.

What it costs (docs/plans/from-header-binding.md, Statistics)
- The role marks itself alive every 2 s; at start the gap since the mark is
  downtime, counted from Postfix's start if later and not at all when
  supervisord says Postfix is not running.
- The log tailer counts the connections Postfix turned away: a warning that
  names the socket, then milter-reject "Service unavailable" from the same
  smtpd. Seen on the stand: the client is refused from CONNECT on and leaves
  at AUTH (454), so no MAIL line is ever written.
- sender_check_days, a row per UTC day, kept a year.

Pages
- Health: the card "Sender check", the socket's row, and the cards layout -
  boxes numbered by importance in two stacks read in rows; the left stack
  polls, the three readings of the right one are swapped in by id.
- Outbound log: a refused row says why in its tag (wrong From, wrong client,
  rate-limited); the message's page shows the From: line and what the
  application may send as.
- Help: topic "Sender check", word for word the mockup's.

Found by the stand, fixed here: the names for the second look came from the
sender bindings, and a domain with a key and no application yet was not
among them.

e2e: negative_journal_milter_fail_open is gone - it proved that mail passes
with the panel down, which is no longer so. In its place
sender_check_holds_the_from_line (another domain's From:, the hostname's, a
stranger's, an address off the list, another sender in the display name -
refused; own addresses - delivered) and panel_down_defers_mail (4xx with the
panel stopped, the container alive, mail again after, downtime and one
deferred message on Health, the card warning).

Test server, this tree before the last cosmetic edits (a fixture row, a
fact's size, docs): go vet ok, go test 23 packages ok, make e2e ok (337 s).
Locally after them: internal/web/view ok.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

internal/store/migrations/0006_sender_check.sql

@@ -0,0 +1,26 @@
+-- The sender check: a message's From: line is held against the application
+-- that sends it (from-header-binding, 2.4.0). DDL only; nothing existing
+-- changes.
+
+-- Why a refused row was refused. Until now "rejected" was one status for
+-- three things, and the row did not say which:
+--   limit    a level-2 rate limit
+--   client   the application's list of client addresses
+--   from     the From: line is not the application's
+-- A row from before this migration has '' and is shown as plainly rejected.
+ALTER TABLE send_log ADD COLUMN reason TEXT NOT NULL DEFAULT '';
+
+-- For a row refused for its From: line, what the header said - cut and
+-- reduced to printable characters by the check before it is stored. from_addr
+-- stays the envelope sender, the address the application had the right to.
+ALTER TABLE send_log ADD COLUMN header_from TEXT NOT NULL DEFAULT '';
+
+-- What the strict check costs, a row per UTC day that has something to say:
+-- for how many seconds the panel did not answer while Postfix was running,
+-- and how many messages Postfix deferred because the check did not answer.
+-- Kept a year. The first day counted is in settings, "sender_check_since".
+CREATE TABLE sender_check_days (
+    day          TEXT PRIMARY KEY,
+    down_seconds INTEGER NOT NULL DEFAULT 0,
+    deferred     INTEGER NOT NULL DEFAULT 0
+);

internal/store/notify.go

@@ -176,6 +176,9 @@ type LimitRefusal struct {
 // The same status is written when an application is refused for its client
 // address (applications.auth_ip_restrict), which is no limit at all; so a
 // login counts only while a level-2 limit is set on it or on its domain.
+// Since 2.4.0 a row says why it was refused, and one refused for its client
+// address or its From: line is left out by that; a row from before has no
+// reason and is still judged the old way.
 func (s *Store) LimitRefusalsSince(since time.Time) ([]LimitRefusal, error) {
 	// The status is a literal so that the partial index of migration 0005
 	// answers the question.
@@ -184,7 +187,7 @@ func (s *Store) LimitRefusalsSince(since time.Time) ([]LimitRefusal, error) {
 		 FROM send_log l
 		 JOIN applications a ON a.login = l.app_login
 		 JOIN domains d ON d.id = a.domain_id
-		 WHERE l.status = 'rejected' AND l.created_at >= ?
+		 WHERE l.status = 'rejected' AND l.created_at >= ? AND l.reason IN ('', 'limit')
 		   AND (EXISTS (SELECT 1 FROM rate_limits r WHERE r.scope = 'application' AND r.ref_id = a.id)
 		     OR EXISTS (SELECT 1 FROM rate_limits r WHERE r.scope = 'domain' AND r.ref_id = d.id))
 		 GROUP BY d.name, a.login ORDER BY d.name, a.login`,

internal/store/schema_test.go

@@ -47,7 +47,7 @@ func TestBaselineSchema(t *testing.T) {
 		"applications":          "id domain_id login address_mode auth_ip_restrict auth_allowed_ips created_at",
 		"application_addresses": "id application_id address",
 		"rate_limits":           "id scope ref_id max_messages window_seconds mode auto_multiplier auto_updated_at",
-		"send_log":              "id queue_id domain app_login from_addr to_addr subject status created_at updated_at",
+		"send_log":              "id queue_id domain app_login from_addr to_addr subject status created_at updated_at reason header_from",
 		"logtail_state":         "path fingerprint read_offset updated_at",
 		"inbound_domains":       "id name recipient_mode created_at",
 		"inbound_transports":    "inbound_domain_id host port tls_mode",
@@ -59,6 +59,7 @@ func TestBaselineSchema(t *testing.T) {
 		"inbound_spam_list":     "id list_type entry_type value note created_at",
 		"send_log_lines":        "id queue_id logged_at line",
 		"notify_state":          "key phase count mark since sent_at",
+		"sender_check_days":     "day down_seconds deferred",
 	}
 	rows, err := st.db.Query("SELECT name FROM sqlite_master WHERE type = 'table' AND name NOT LIKE 'sqlite_%' ORDER BY name")
 	if err != nil {
@@ -113,8 +114,8 @@ func TestBaselineSchema(t *testing.T) {
 	}
 
 	var version, appID int64
-	if err := st.db.QueryRow("PRAGMA user_version").Scan(&version); err != nil || version != 5 {
-		t.Errorf("user_version = %d, %v; want 5", version, err)
+	if err := st.db.QueryRow("PRAGMA user_version").Scan(&version); err != nil || version != 6 {
+		t.Errorf("user_version = %d, %v; want 6", version, err)
 	}
 	if err := st.db.QueryRow("PRAGMA application_id").Scan(&appID); err != nil || appID != applicationID {
 		t.Errorf("application_id = %#x, %v; want %#x", appID, err, applicationID)
@@ -190,8 +191,8 @@ func TestOpenMigratesA20Database(t *testing.T) {
 	}
 	defer st.Close()
 	var version int
-	if err := st.db.QueryRow("PRAGMA user_version").Scan(&version); err != nil || version != 5 {
-		t.Errorf("user_version = %d, %v; want 5", version, err)
+	if err := st.db.QueryRow("PRAGMA user_version").Scan(&version); err != nil || version != 6 {
+		t.Errorf("user_version = %d, %v; want 6", version, err)
 	}
 	if ok, err := st.UserExists(); err != nil || !ok {
 		t.Errorf("the user of the 2.0 database: exists = %v, %v", ok, err)

internal/store/sendercheck.go

@@ -0,0 +1,146 @@
+package store
+
+import (
+	"fmt"
+	"time"
+)
+
+// Why a send-log row with status "rejected" was refused (send_log.reason).
+// A row written before the reason was recorded has "".
+const (
+	ReasonLimit  = "limit"  // a level-2 rate limit
+	ReasonClient = "client" // the application's list of client addresses
+	ReasonFrom   = "from"   // the From: line is not the application's
+)
+
+// Settings keys of the sender check's statistics.
+const (
+	// settingSenderCheckSince is the first UTC day the statistics cover.
+	settingSenderCheckSince = "sender_check_since"
+	// settingSenderCheckDeferredAt is when a message was last deferred because
+	// the check did not answer (RFC3339 UTC).
+	settingSenderCheckDeferredAt = "sender_check_deferred_at"
+)
+
+// senderCheckKeepDays is how long a day of the statistics is kept.
+const senderCheckKeepDays = 366
+
+const dayLayout = "2006-01-02"
+
+// SenderCheckTotals is what the strict sender check has cost over a period.
+type SenderCheckTotals struct {
+	Down     time.Duration // the panel not answering while Postfix was running
+	Deferred int64         // messages Postfix deferred because of it
+}
+
+// SenderCheckStats is the card of Server > Health.
+type SenderCheckStats struct {
+	Today, Month, Year SenderCheckTotals
+	// Since is the first day counted; zero before anything was.
+	Since time.Time
+	// DeferredAt is when a message was last deferred; zero if never.
+	DeferredAt time.Time
+}
+
+// StartSenderCheckStats records the first day the statistics cover, once.
+func (s *Store) StartSenderCheckStats(now time.Time) error {
+	v, err := s.GetSetting(settingSenderCheckSince)
+	if err != nil || v != "" {
+		return err
+	}
+	return s.SetSetting(settingSenderCheckSince, now.UTC().Format(dayLayout))
+}
+
+// AddSenderCheckDown adds to the day of at the time the panel did not answer
+// while Postfix was running.
+func (s *Store) AddSenderCheckDown(at time.Time, d time.Duration) error {
+	secs := int64(d / time.Second)
+	if secs <= 0 {
+		return nil
+	}
+	_, err := s.db.Exec(
+		`INSERT INTO sender_check_days (day, down_seconds) VALUES (?, ?)
+		 ON CONFLICT (day) DO UPDATE SET down_seconds = down_seconds + excluded.down_seconds`,
+		at.UTC().Format(dayLayout), secs,
+	)
+	if err != nil {
+		return fmt.Errorf("sender check downtime: %w", err)
+	}
+	return nil
+}
+
+// AddSenderCheckDeferred counts, on the day of at, n messages deferred because
+// the check did not answer, and remembers at as the last such moment.
+func (s *Store) AddSenderCheckDeferred(at time.Time, n int64) error {
+	if n <= 0 {
+		return nil
+	}
+	_, err := s.db.Exec(
+		`INSERT INTO sender_check_days (day, deferred) VALUES (?, ?)
+		 ON CONFLICT (day) DO UPDATE SET deferred = deferred + excluded.deferred`,
+		at.UTC().Format(dayLayout), n,
+	)
+	if err != nil {
+		return fmt.Errorf("sender check deferred: %w", err)
+	}
+	// Lines are read in order, but a re-read log must not move the mark back.
+	last, err := s.GetSetting(settingSenderCheckDeferredAt)
+	if err != nil {
+		return err
+	}
+	stamp := at.UTC().Format(time.RFC3339)
+	if stamp > last {
+		return s.SetSetting(settingSenderCheckDeferredAt, stamp)
+	}
+	return nil
+}
+
+// SenderCheckStats sums the days up to now: today, the last 30 days and the
+// last 12 months, each including today.
+func (s *Store) SenderCheckStats(now time.Time) (SenderCheckStats, error) {
+	now = now.UTC()
+	var out SenderCheckStats
+	sum := func(days int) (SenderCheckTotals, error) {
+		var secs, deferred int64
+		err := s.db.QueryRow(
+			`SELECT COALESCE(SUM(down_seconds), 0), COALESCE(SUM(deferred), 0)
+			 FROM sender_check_days WHERE day > ? AND day <= ?`,
+			now.AddDate(0, 0, -days).Format(dayLayout), now.Format(dayLayout),
+		).Scan(&secs, &deferred)
+		if err != nil {
+			return SenderCheckTotals{}, fmt.Errorf("sender check stats: %w", err)
+		}
+		return SenderCheckTotals{Down: time.Duration(secs) * time.Second, Deferred: deferred}, nil
+	}
+	var err error
+	if out.Today, err = sum(1); err != nil {
+		return out, err
+	}
+	if out.Month, err = sum(30); err != nil {
+		return out, err
+	}
+	if out.Year, err = sum(365); err != nil {
+		return out, err
+	}
+	since, err := s.GetSetting(settingSenderCheckSince)
+	if err != nil {
+		return out, err
+	}
+	out.Since, _ = time.Parse(dayLayout, since)
+	deferredAt, err := s.GetSetting(settingSenderCheckDeferredAt)
+	if err != nil {
+		return out, err
+	}
+	out.DeferredAt, _ = time.Parse(time.RFC3339, deferredAt)
+	return out, nil
+}
+
+// PruneSenderCheckDays deletes the days older than a year.
+func (s *Store) PruneSenderCheckDays(now time.Time) error {
+	_, err := s.db.Exec(`DELETE FROM sender_check_days WHERE day < ?`,
+		now.UTC().AddDate(0, 0, -senderCheckKeepDays).Format(dayLayout))
+	if err != nil {
+		return fmt.Errorf("prune sender check days: %w", err)
+	}
+	return nil
+}

internal/store/sendercheck_test.go

@@ -0,0 +1,114 @@
+package store
+
+import (
+	"path/filepath"
+	"testing"
+	"time"
+)
+
+func TestSenderCheckStats(t *testing.T) {
+	st, err := Open(filepath.Join(t.TempDir(), "selfpost.db"))
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer st.Close()
+
+	now := time.Date(2026, 10, 20, 15, 0, 0, 0, time.UTC)
+	day := func(n int) time.Time { return now.AddDate(0, 0, -n) }
+
+	// Nothing counted yet: zeros, and no first day.
+	got, err := st.SenderCheckStats(now)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if got != (SenderCheckStats{}) {
+		t.Fatalf("empty statistics = %+v", got)
+	}
+
+	// The first day is recorded once.
+	if err := st.StartSenderCheckStats(day(40)); err != nil {
+		t.Fatal(err)
+	}
+	if err := st.StartSenderCheckStats(now); err != nil {
+		t.Fatal(err)
+	}
+
+	must := func(err error) {
+		t.Helper()
+		if err != nil {
+			t.Fatal(err)
+		}
+	}
+	must(st.AddSenderCheckDown(now, 3*time.Second))
+	must(st.AddSenderCheckDown(now, 1500*time.Millisecond)) // whole seconds: one
+	must(st.AddSenderCheckDown(now, 400*time.Millisecond))  // nothing
+	must(st.AddSenderCheckDeferred(now.Add(-time.Hour), 2))
+	must(st.AddSenderCheckDown(day(10), 30*time.Second))
+	must(st.AddSenderCheckDeferred(day(10), 1))
+	must(st.AddSenderCheckDown(day(29), time.Minute))
+	must(st.AddSenderCheckDown(day(30), time.Hour)) // the 31st day back: the year's only
+	must(st.AddSenderCheckDeferred(day(200), 5))
+	must(st.AddSenderCheckDown(day(365), 24*time.Hour)) // a year and a day: nobody's
+
+	got, err = st.SenderCheckStats(now)
+	if err != nil {
+		t.Fatal(err)
+	}
+	want := SenderCheckStats{
+		Today:      SenderCheckTotals{Down: 4 * time.Second, Deferred: 2},
+		Month:      SenderCheckTotals{Down: 94 * time.Second, Deferred: 3},
+		Year:       SenderCheckTotals{Down: 94*time.Second + time.Hour, Deferred: 8},
+		Since:      time.Date(2026, 9, 10, 0, 0, 0, 0, time.UTC),
+		DeferredAt: now.Add(-time.Hour),
+	}
+	if got != want {
+		t.Fatalf("statistics\n got %+v\nwant %+v", got, want)
+	}
+
+	// The last deferral is the latest one seen, not the last one written: a
+	// log read again after an unclean stop does not move it back.
+	if got.DeferredAt != now.Add(-time.Hour) {
+		t.Fatalf("an older deferral moved the mark to %s", got.DeferredAt)
+	}
+
+	// A year on, the old days are gone.
+	must(st.PruneSenderCheckDays(now.AddDate(1, 0, 0)))
+	var n int
+	if err := st.db.QueryRow(`SELECT COUNT(*) FROM sender_check_days`).Scan(&n); err != nil || n != 1 {
+		t.Fatalf("%d days left after a year's prune (%v), want the newest alone", n, err)
+	}
+}
+
+// A refused row keeps why it was refused and, for its From: line, what the
+// line said; an accepted row has neither.
+func TestRefusedRowKeepsItsReason(t *testing.T) {
+	st, err := Open(filepath.Join(t.TempDir(), "selfpost.db"))
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer st.Close()
+
+	if err := st.InsertRejected(SendLogEntry{Domain: "shop.example.org", AppLogin: "shop", From: "shop@shop.example.org",
+		To: "k.berg@web.de", Subject: "s", Reason: ReasonFrom, HeaderFrom: "director@example.org"}); err != nil {
+		t.Fatal(err)
+	}
+	if err := st.InsertQueued(SendLogEntry{QueueID: "4XcB7k2Jm9z1", Domain: "shop.example.org", AppLogin: "shop",
+		From: "shop@shop.example.org", To: "k.berg@web.de"}); err != nil {
+		t.Fatal(err)
+	}
+	rows, err := st.QuerySendLog(SendLogFilter{AllDomains: true}, 10, 0)
+	if err != nil || len(rows) != 2 {
+		t.Fatalf("rows = %d, %v", len(rows), err)
+	}
+	queued, refused := rows[0], rows[1]
+	if refused.Status != StatusRejected || refused.Reason != ReasonFrom || refused.HeaderFrom != "director@example.org" || refused.To != "k.berg@web.de" {
+		t.Errorf("refused row = %+v", refused)
+	}
+	if queued.Reason != "" || queued.HeaderFrom != "" {
+		t.Errorf("queued row carries a refusal: %+v", queued)
+	}
+	one, err := st.GetSendLog(refused.ID)
+	if err != nil || one.Reason != ReasonFrom || one.HeaderFrom != "director@example.org" {
+		t.Errorf("GetSendLog = %+v, %v", one, err)
+	}
+}

internal/store/sendlog.go

@@ -34,6 +34,10 @@ type SendLogEntry struct {
 	From     string
 	To       string
 	Subject  string
+	// Reason and HeaderFrom are set on refused rows only: why the message
+	// was refused (Reason*), and for ReasonFrom what its From: line said.
+	Reason     string
+	HeaderFrom string
 }
 
 // InsertQueued records an accepted message in the send log with status
@@ -63,9 +67,9 @@ func (s *Store) InsertRejected(e SendLogEntry) error {
 	now := time.Now().UTC().Format(time.RFC3339)
 	_, err := s.db.Exec(
 		`INSERT INTO send_log
-			(queue_id, domain, app_login, from_addr, to_addr, subject, status, created_at, updated_at)
-		 VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)`,
-		e.QueueID, e.Domain, e.AppLogin, e.From, e.To, e.Subject, StatusRejected, now, now,
+			(queue_id, domain, app_login, from_addr, to_addr, subject, status, reason, header_from, created_at, updated_at)
+		 VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
+		e.QueueID, e.Domain, e.AppLogin, e.From, e.To, e.Subject, StatusRejected, e.Reason, e.HeaderFrom, now, now,
 	)
 	if err != nil {
 		return fmt.Errorf("insert rejected send_log: %w", err)
@@ -136,15 +140,18 @@ func (s *Store) ListQueuedOlderThan(cutoff time.Time) ([]QueuedDelivery, error)
 // Persistence): a SendLogEntry plus the fields that only exist once a row has
 // been written (id, current status, timestamps).
 type SendLogRow struct {
-	ID        int64
-	QueueID   string
-	Domain    string
-	AppLogin  string
-	From      string
-	To        string
-	Subject   string
-	Status    string
-	CreatedAt time.Time
+	ID       int64
+	QueueID  string
+	Domain   string
+	AppLogin string
+	From     string
+	To       string
+	Subject  string
+	Status   string
+	// Reason and HeaderFrom: see SendLogEntry.
+	Reason     string
+	HeaderFrom string
+	CreatedAt  time.Time
 	// UpdatedAt is when the status last changed — the log-tailer's report of
 	// the delivery attempt. It equals CreatedAt while a row is still queued.
 	UpdatedAt time.Time
@@ -163,10 +170,10 @@ func (s *Store) GetSendLog(id int64) (SendLogRow, error) {
 		createdAt, updatedAt string
 	)
 	err := s.db.QueryRow(
-		`SELECT id, queue_id, domain, app_login, from_addr, to_addr, subject, status, created_at, updated_at
+		`SELECT id, queue_id, domain, app_login, from_addr, to_addr, subject, status, reason, header_from, created_at, updated_at
 		 FROM send_log WHERE id = ?`, id,
 	).Scan(&row.ID, &row.QueueID, &row.Domain, &row.AppLogin, &row.From, &row.To,
-		&row.Subject, &row.Status, &createdAt, &updatedAt)
+		&row.Subject, &row.Status, &row.Reason, &row.HeaderFrom, &createdAt, &updatedAt)
 	if errors.Is(err, sql.ErrNoRows) {
 		return SendLogRow{}, ErrSendLogNotFound
 	}
@@ -205,7 +212,7 @@ func (s *Store) QuerySendLog(filter SendLogFilter, limit, offset int) ([]SendLog
 	where, args := sendLogWhere(filter)
 	args = append(args, limit, offset)
 	rows, err := s.db.Query(
-		`SELECT id, queue_id, domain, app_login, from_addr, to_addr, subject, status, created_at, updated_at
+		`SELECT id, queue_id, domain, app_login, from_addr, to_addr, subject, status, reason, header_from, created_at, updated_at
 		 FROM send_log`+where+`
 		 ORDER BY id DESC
 		 LIMIT ? OFFSET ?`,
@@ -223,7 +230,7 @@ func (s *Store) QuerySendLog(filter SendLogFilter, limit, offset int) ([]SendLog
 			createdAt, updatedAt string
 		)
 		if err := rows.Scan(&row.ID, &row.QueueID, &row.Domain, &row.AppLogin,
-			&row.From, &row.To, &row.Subject, &row.Status, &createdAt, &updatedAt); err != nil {
+			&row.From, &row.To, &row.Subject, &row.Status, &row.Reason, &row.HeaderFrom, &createdAt, &updatedAt); err != nil {
 			return nil, fmt.Errorf("scan send_log row: %w", err)
 		}
 		row.CreatedAt, _ = time.Parse(time.RFC3339, createdAt)