Files
selfpost/internal/web/view/outbound_log.go
T
mixandClaude Opus 5.5 c171fb5bc7
test / test (push) Canceled after 0s
from-header-binding: the sender check, what it costs, its pages; guide, architecture, security
A message's From: line is held against the application that sends it, under
the rule Postfix holds the envelope sender to. OpenDKIM chooses the key by
that line, so until now an application could have mail signed as another
domain of the server or as the server itself.

The check (internal/milter/sender.go)
- A role of the panel on a socket of its own (SENDER_CHECK_SOCKET), first in
  smtpd_milters and strict: default_action=tempfail. The journal milter keeps
  its socket and stays fail-open. postfix-wrapper waits for the new socket.
- It reads no database: what an application may send as comes from Postfix's
  own sender_login_maps, what the server signs for from OpenDKIM's
  SigningTable. Both are read again when they change; one that cannot be
  read defers mail.
- One From: field, one address, allowed by the envelope's lookup; then no
  other name of the SigningTable may appear anywhere in the raw line.
  Refused: 550 5.7.1 naming the line, reduced to printable ASCII.
- A refusal is a send-log row with reason "from" and the header address
  (migration 0006), written by another goroutine and not waited for. The
  journal's refusals record their reason too (limit, client), and the
  notifier takes only a limit's.

What it costs (docs/plans/from-header-binding.md, Statistics)
- The role marks itself alive every 2 s; at start the gap since the mark is
  downtime, counted from Postfix's start if later and not at all when
  supervisord says Postfix is not running.
- The log tailer counts the connections Postfix turned away: a warning that
  names the socket, then milter-reject "Service unavailable" from the same
  smtpd. Seen on the stand: the client is refused from CONNECT on and leaves
  at AUTH (454), so no MAIL line is ever written.
- sender_check_days, a row per UTC day, kept a year.

Pages
- Health: the card "Sender check", the socket's row, and the cards layout -
  boxes numbered by importance in two stacks read in rows; the left stack
  polls, the three readings of the right one are swapped in by id.
- Outbound log: a refused row says why in its tag (wrong From, wrong client,
  rate-limited); the message's page shows the From: line and what the
  application may send as.
- Help: topic "Sender check", word for word the mockup's.

Found by the stand, fixed here: the names for the second look came from the
sender bindings, and a domain with a key and no application yet was not
among them.

e2e: negative_journal_milter_fail_open is gone - it proved that mail passes
with the panel down, which is no longer so. In its place
sender_check_holds_the_from_line (another domain's From:, the hostname's, a
stranger's, an address off the list, another sender in the display name -
refused; own addresses - delivered) and panel_down_defers_mail (4xx with the
panel stopped, the container alive, mail again after, downtime and one
deferred message on Health, the card warning).

Test server, this tree before the last cosmetic edits (a fixture row, a
fact's size, docs): go vet ok, go test 23 packages ok, make e2e ok (337 s).
Locally after them: internal/web/view ok.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 19:35:43 +03:00

335 lines
11 KiB
Go

package view
// The data of the Outbound log pages: the list of messages
// (templates/out_log.html, mockup out-log.html) and one message
// (out_message.html, mockup out-message.html). The list polls: the table and
// its foot are one region (templates/out_log_rows.html) that the page and its
// fragment, GET /outbound/log/fragment, render alike.
//
// A page is a typed struct that embeds Meta, built by a constructor and filled
// by With… methods; the handler reads the journal, the view decides what is
// said and how it is worded.
import (
"net/url"
"strconv"
"strings"
"time"
)
const outboundLog = "/outbound/log"
// logQuery is the query string of a page of the log: the filters and the page,
// each left out when it says nothing, so the first unfiltered page is the bare
// path.
func logQuery(domain, app string, page int) string {
q := url.Values{}
if domain != "" {
q.Set("domain", domain)
}
if app != "" {
q.Set("app", app)
}
if page > 1 {
q.Set("p", strconv.Itoa(page))
}
if len(q) == 0 {
return ""
}
return "?" + q.Encode()
}
// LogHref is the address of a page of the Outbound log.
func LogHref(domain, app string, page int) string {
return outboundLog + logQuery(domain, app, page)
}
// FormatLogTime is the time of a row of the log: month, day and the time of day
// in UTC, as the table's "Time, UTC" column shows it.
func FormatLogTime(t time.Time) string { return t.UTC().Format("01-02 15:04:05") }
// formatStamp is a moment on the page of one message, with its zone.
func formatStamp(t time.Time) string { return t.UTC().Format("2006-01-02 15:04:05") + " UTC" }
// ---- The log
// OutLog is the data of the Outbound log: the messages of the domains the
// viewer reaches, filtered by domain and application, a page at a time.
type OutLog struct {
Meta
Head Head
Messages Box
// PollURL is the address of the fragment that answers with the table again
// for the same filters and page (the polled region is the table, not the box:
// a filter being chosen in the head must not be reset under the hand).
PollURL string
Rows []OutLogRow
Empty EmptyState
Foot Foot
domain, app string
}
// OutLogRow is one message of the table: when it was accepted, who it was from
// and to, its subject (shown whole in the cell's title when the column clips
// it), its status as a tag and the way to the message's own page.
type OutLogRow struct {
Time string
From string
To string
Subject string
Status Tag
Href string
}
// NewOutLog builds the log for a viewer. domains and apps are the choices of
// the two filters (only what the viewer reaches), domain and app the ones in
// force. retentionDays is how long rows are kept; canSettings, the global role,
// adds the way to change it.
func NewOutLog(m Meta, retentionDays int, canSettings bool, domains, apps []string, domain, app string) *OutLog {
m.Title, m.Section, m.Page = "Outbound log", "outbound", "log"
lead := Rich("Every message an application handed to the relay. Rows older than " + plural(int64(retentionDays), "day", "days") + " are deleted")
if canSettings {
lead = append(lead, Inline{Text: " — change that in "}, Link("/server/settings", "Settings"))
}
lead = append(lead, Inline{Text: "."})
options := func(any string, values []string) []Option {
out := []Option{{Value: "", Label: any}}
for _, v := range values {
out = append(out, Option{Value: v, Label: v})
}
return out
}
return &OutLog{
Meta: m,
Head: Head{Kicker: "Outbound", Title: "Outbound log", Lead: lead},
Messages: Box{No: "01", Title: "Messages", Help: topicLink(HelpLog, "What the statuses mean"), Filter: &Filter{
Action: outboundLog,
Selects: []Select{
{Name: "domain", Label: "Domain", Options: options("All domains", domains), Selected: domain},
{Name: "app", Label: "Application", Options: options("All applications", apps), Selected: app},
},
}},
Empty: EmptyState{Icon: "ti-list-details", Text: Plain("No messages logged yet.")},
PollURL: outboundLog + "/fragment" + logQuery(domain, app, 1),
Foot: Foot{ID: "out-log-foot", End: Plain("New rows appear on their own")},
domain: domain, app: app,
}
}
// DetailHref is the address of one message's page, carrying the filters and the
// page it is opened from so that its way back returns to the same list.
func (p *OutLog) DetailHref(id int64, page int) string {
return outboundLog + "/" + strconv.FormatInt(id, 10) + logQuery(p.domain, p.app, page)
}
// WithRows sets the table for page of last, and the foot under it: where the
// viewer is, and the way to the newer and older rows. The poll asks for the same
// page again.
func (p *OutLog) WithRows(rows []OutLogRow, page, last int) *OutLog {
p.Rows = rows
p.PollURL = outboundLog + "/fragment" + logQuery(p.domain, p.app, page)
if len(rows) == 0 {
return p
}
text := Rich("Page " + strconv.Itoa(page) + " of " + strconv.Itoa(last))
if page > 1 {
text = append(text, Inline{Text: " · "}, Link(LogHref(p.domain, p.app, page-1), "← Newer"))
}
p.Foot.Text = text
if page < last {
p.Foot.Link = &Anchor{Href: LogHref(p.domain, p.app, page+1), Label: "Older →"}
}
return p
}
// Fragment marks the foot to be swapped into the page by its id: the polled
// fragment is the table, and the count under it follows out of band.
func (p *OutLog) Fragment() *OutLog {
p.Foot.OOB = true
return p
}
// ---- One message
// MessageInput is what the journal holds about one message and the pages it can
// be linked to. DomainHref and AppHref are empty when the domain or the
// application no longer exists, or the viewer does not reach it: the name is
// then plain text.
type MessageInput struct {
ID int64
QueueID string
Domain string
DomainHref string
App string
AppHref string
From string
To string
Subject string
Status string
// Reason and HeaderFrom are a refused row's: why it was refused
// (store.Reason*) and, for its From: line, what that line said.
Reason string
HeaderFrom string
Accepted time.Time
Reported time.Time
// BackHref is the log the message was opened from, filters and page kept.
BackHref string
}
// OutMessage is the data of one message's page: what the journal recorded, the
// steps it went through and what Postfix wrote about it.
type OutMessage struct {
Meta
Head Head
Message Box
Facts []Fact
History Box
Steps []Step
Delivery Box
LogLines []LogLine
// LogEmpty says why there are no lines: a message that never reached the
// queue, or that predates the stored lines, is an ordinary outcome.
LogEmpty EmptyState
}
// SendLogTag is the tag of a send-log row. A refused row says why in the tag,
// so that the log answers it without the message being opened: the reasons
// are store.Reason*, and a row from before they were recorded is "rejected".
func SendLogTag(status, reason string) Tag {
if status != "rejected" {
return Tag{Status: status}
}
switch reason {
case "from":
return Tag{Status: "reject", Label: "wrong From"}
case "client":
return Tag{Status: "reject", Label: "wrong client"}
case "limit":
return Tag{Status: "rate-limited"}
}
return Tag{Status: status}
}
// dash is the value of a field a row does not have.
const dash = "—"
func orDash(s string) string {
if s == "" {
return dash
}
return s
}
// NewOutMessage builds the page of one message. Its name is its subject, and
// the line under it is who it was from and to with its status.
func NewOutMessage(m Meta, in MessageInput) *OutMessage {
title := in.Subject
if title == "" {
title = "(no subject)"
}
m.Title, m.Section, m.Page = title+" · Outbound log", "outbound", "log"
value := func(text, href string) Text {
if href == "" {
return Plain(orDash(text))
}
return Rich(Link(href, text))
}
tag := SendLogTag(in.Status, in.Reason)
facts := []Fact{
{Label: "Domain", Value: value(in.Domain, in.DomainHref)},
{Label: "Application", Value: value(in.App, in.AppHref), Mono: true},
}
history := Box{No: "02", Title: "History", Help: topicLink(HelpLog, "What the statuses mean")}
if in.Status == "rejected" {
// A refused message was never accepted and never queued: its one date
// is the refusal, and it has no queue id to look anything up by.
facts = append(facts, Fact{Label: "Refused", Value: Plain(formatStamp(in.Accepted))})
if in.Reason == "from" {
facts = append(facts, Fact{Label: "From: line", Value: Plain(orDash(in.HeaderFrom)), Mono: true, Small: true})
history.Help = topicLink(HelpSenderCheck, "What an application may write in From:")
}
facts = append(facts, Fact{Label: "Queue id", Value: Plain("none - never queued")})
} else {
facts = append(facts,
Fact{Label: "Accepted", Value: Plain(formatStamp(in.Accepted))},
Fact{Label: "Status reported", Value: Plain(formatStamp(in.Reported))},
Fact{Label: "Queue id", Value: Plain(orDash(in.QueueID)), Mono: true})
}
facts = append(facts, Fact{Label: "Journal id", Value: Plain(strconv.FormatInt(in.ID, 10)), Mono: true})
return &OutMessage{
Meta: m,
Head: Head{
Crumbs: []Crumb{{Href: in.BackHref, Label: "Outbound log"}},
Title: title,
Compact: true,
Route: Route{
{Text: orDash(in.From)}, {Arrow: true}, {Text: orDash(in.To)},
{Tag: &tag},
},
},
Message: Box{No: "01", Title: "Message"},
Facts: facts,
History: history,
Delivery: Box{No: "03", Title: "Delivery log", End: Rich("Lines from ", Code("mail.log"), " with this queue id")},
LogEmpty: EmptyState{Icon: "ti-file-text"},
}
}
// WithHistory sets the steps the message went through, oldest first.
func (p *OutMessage) WithHistory(steps []Step) *OutMessage {
p.Steps = steps
return p
}
// WithDeliveryLog sets the lines Postfix wrote about the message. When there are
// none, note says why.
func (p *OutMessage) WithDeliveryLog(lines []LogLine, note string) *OutMessage {
p.LogLines = lines
p.LogEmpty.Text = Plain(note)
return p
}
// StepTime is the time of a step of the history: the moment, or "not yet" for a
// step that has not happened.
func StepTime(t time.Time) string {
if t.IsZero() {
return "not yet"
}
return formatStamp(t)
}
// StepLevel is the level of a history step from the level of its outcome (ok,
// warn, error or anything else for "nothing to report"): a step that has not
// happened is pending whatever its outcome.
func StepLevel(outcome string, happened bool) string {
switch {
case !happened:
return LevelPending
case outcome == "warn":
return LevelWarn
case outcome == "error":
return LevelFail
}
return ""
}
// LogLineLevel is the level a line of the delivery log is drawn at: a delivery
// that was put off is amber, one that failed for good red. Postfix writes the
// outcome of an attempt as status=… in the line itself.
func LogLineLevel(text string) string {
switch {
case strings.Contains(text, "status=bounced"), strings.Contains(text, "status=expired"),
strings.Contains(text, " fatal:"), strings.Contains(text, " error:"):
return "error"
case strings.Contains(text, "status=deferred"), strings.Contains(text, " warning:"):
return "warn"
}
return ""
}