logtail, health, panel: cite the source of what they rely on (assumptions part A)
Comments only, no behaviour changed. Each place the log tailer, Health and cmd/panel rely on another program now says where that is written, for the version the image runs: Postfix 3.7.11 (log_adhoc(), msg_logger_print(), smtpd_whatsup(), cleanup_milter.c, qmgr_active.c, postlogd.c, showq.c, pipe.c), logrotate 3.21.0, supervisor 4.2.5, proc(5). Measurements stay measurements and say where and when; test fixtures say whether they were seen or composed. Where the source contradicts the code the comment does not bless it but names the register entry (A-8 expiry, A-12 the ISO stamp no Postfix writes, A-15 the last recipient, A-21/A-23 sasl_username, A-33 postlogd's old inode, A-38 a socket file is not a listener, A-65 the certificate served); those are fixed in their own commits. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
+10
-1
@@ -9,13 +9,22 @@ import (
|
||||
)
|
||||
|
||||
// certWarnDays is how close to expiry the certificate has to be before the
|
||||
// status page complains. Let's Encrypt renews at 30 days left, so two weeks is
|
||||
// status page complains. certbot renews a 90-day certificate at 30 days left
|
||||
// - a fixed 30 days before 4.0, a third of the lifetime from 4.0 (certbot
|
||||
// docs, certbot/docs/using.rst, "Renewing certificates") - so two weeks is
|
||||
// comfortably past the point where automatic renewal should have happened.
|
||||
// For a 45-day certificate a third is 15 days, one day ahead of this warning
|
||||
// (assumptions A-64).
|
||||
const certWarnDays = 14
|
||||
|
||||
// Certificate is the state of the TLS certificate Postfix serves on 465/587
|
||||
// (guide § Environment variables: TLS_CERT_FILE). The panel only reads it —
|
||||
// the file is supplied by the reverse proxy through a read-only mount.
|
||||
// Postfix itself serves the copy build/postfix-config.sh makes in
|
||||
// /etc/postfix/tls-internal, refreshed by build/postfix-cert-reload.sh up to
|
||||
// TLS_RELOAD_INTERVAL_SECONDS later, and each smtpd loads it when it starts
|
||||
// (Postfix 3.7.11, src/smtpd/smtpd.c, pre_jail_init(), TLS_SERVER_INIT) - so
|
||||
// the mounted file can be newer than what is served (assumptions A-65).
|
||||
type Certificate struct {
|
||||
Path string
|
||||
Subject string
|
||||
|
||||
@@ -33,7 +33,10 @@ func TestWorstPicksMostSevere(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestParseProcesses(t *testing.T) {
|
||||
// Real supervisorctl output: column-aligned, one line per program.
|
||||
// Real supervisorctl output: column-aligned, one line per program. Where
|
||||
// and when it was seen is not recorded; its columns are those of
|
||||
// supervisor 4.2.5 supervisorctl.py _show_statuses() and its FATAL text
|
||||
// supervisor/process.py's spawnerr.
|
||||
out := `opendkim RUNNING pid 21, uptime 0:04:10
|
||||
panel RUNNING pid 22, uptime 0:04:09
|
||||
postfix FATAL Exited too quickly (process log may have details)
|
||||
@@ -165,6 +168,7 @@ func writeCert(t *testing.T, path, cn string, validFor time.Duration) {
|
||||
}
|
||||
|
||||
func TestLivenessFromParsedProcesses(t *testing.T) {
|
||||
// Composed, in the columns of TestParseProcesses.
|
||||
allRunning := `opendkim RUNNING pid 21, uptime 0:04:10
|
||||
panel RUNNING pid 22, uptime 0:04:09
|
||||
postfix RUNNING pid 23, uptime 0:04:08
|
||||
@@ -189,6 +193,9 @@ postfix FATAL Exited too quickly
|
||||
}
|
||||
}
|
||||
|
||||
// Composed, after supervisor 4.2.5 rpcinterface.py _interpretProcessInfo()
|
||||
// (str() of a timedelta). "127:00:01" is not a form supervisor prints - it
|
||||
// only shows that the parser takes hours past 24.
|
||||
func TestUptime(t *testing.T) {
|
||||
for detail, want := range map[string]time.Duration{
|
||||
"pid 41, uptime 0:03:12": 3*time.Minute + 12*time.Second,
|
||||
|
||||
+31
-11
@@ -33,11 +33,14 @@ const (
|
||||
// status page's answer to "is the server itself under strain", next to the
|
||||
// component checks that answer "is mail flowing".
|
||||
//
|
||||
// The numbers come from the kernel's /proc filesystem, which a container shares
|
||||
// with its host unless it was started with its own namespaces — so CPU and
|
||||
// memory describe the host, while /proc/net/dev describes whatever network
|
||||
// namespace the container is in (its own veth under the default bridge, the
|
||||
// host's interfaces under network_mode: host).
|
||||
// The numbers come from the kernel's /proc filesystem. /proc/stat and
|
||||
// /proc/meminfo are system-wide and no namespace changes them (proc(5):
|
||||
// "kernel/system statistics", "statistics about memory usage on the
|
||||
// system"), so CPU and memory describe the host, while /proc/net/dev
|
||||
// describes whatever network namespace the container is in
|
||||
// (network_namespaces(7) names /proc/net among what a network namespace
|
||||
// isolates) - its own veth under the default bridge, the host's interfaces
|
||||
// under network_mode: host.
|
||||
type Machine struct {
|
||||
CPU CPU
|
||||
Memory Memory
|
||||
@@ -276,7 +279,9 @@ func (c CPU) contextText() string {
|
||||
// MemAvailable rather than MemFree: Linux spends every spare page on cache, so
|
||||
// MemFree on a healthy machine is near zero and would report a permanent
|
||||
// emergency. MemAvailable is the kernel's own estimate of what a new workload
|
||||
// could actually get.
|
||||
// could actually get (proc(5), /proc/meminfo, MemAvailable "since Linux
|
||||
// 3.14": "An estimate of how much memory is available for starting new
|
||||
// applications, without swapping").
|
||||
func readMemory(root string) Memory {
|
||||
var m Memory
|
||||
fields, err := readMeminfo(root)
|
||||
@@ -398,11 +403,15 @@ func readCPUTimes(root string) (cpuTimes, int, error) {
|
||||
continue
|
||||
}
|
||||
if fields[0] != "cpu" {
|
||||
cores++ // cpu0, cpu1, … — one line per core
|
||||
cores++ // cpu0, cpu1, … — one line per online logical CPU
|
||||
continue
|
||||
}
|
||||
// user nice system idle iowait irq softirq steal (guest fields are
|
||||
// already counted inside user/nice, so they are left out).
|
||||
// already counted inside user/nice, so they are left out). Field
|
||||
// order: proc(5), /proc/stat. That guest time is in user/nice too:
|
||||
// Linux kernel/sched/cputime.c, account_guest_time() adds it to
|
||||
// CPUTIME_USER (or CPUTIME_NICE) as well as CPUTIME_GUEST (torvalds/
|
||||
// linux master read 2026-10-14; the kernel is the host's).
|
||||
for i, f := range fields[1:] {
|
||||
if i >= 8 {
|
||||
break
|
||||
@@ -428,7 +437,11 @@ func readCPUTimes(root string) (cpuTimes, int, error) {
|
||||
|
||||
// readCPUTopology returns physical core and logical thread counts from
|
||||
// /proc/cpuinfo. Zeroes mean the file was missing or empty; the caller falls
|
||||
// back to the cpuN count from /proc/stat.
|
||||
// back to the cpuN count from /proc/stat. The keys are x86's: Linux
|
||||
// arch/x86/kernel/cpu/proc.c, show_cpuinfo() prints "processor" and, on SMP,
|
||||
// "physical id" and "core id" (show_cpuinfo_core(); torvalds/linux master
|
||||
// read 2026-10-14). proc(5) says only that the items are architecture
|
||||
// dependent; where they are missing each processor counts as a core.
|
||||
func readCPUTopology(root string) (cores, threads int) {
|
||||
data, err := os.ReadFile(filepath.Join(root, "cpuinfo"))
|
||||
if err != nil {
|
||||
@@ -522,7 +535,9 @@ func isZeroLoad(load [3]float64) bool {
|
||||
}
|
||||
|
||||
// readMeminfo returns the /proc/meminfo entries this package uses, in bytes.
|
||||
// The file reports kB (kibibytes, despite the label) for these fields.
|
||||
// The file reports kB (kibibytes, despite the label) for these fields: Linux
|
||||
// fs/proc/meminfo.c, show_val_kb() prints pages << (PAGE_SHIFT - 10)
|
||||
// (torvalds/linux master read 2026-10-14).
|
||||
func readMeminfo(root string) (map[string]uint64, error) {
|
||||
data, err := os.ReadFile(filepath.Join(root, "meminfo"))
|
||||
if err != nil {
|
||||
@@ -557,7 +572,12 @@ func readMeminfo(root string) (map[string]uint64, error) {
|
||||
// readNetDev returns each interface's byte counters from /proc/net/dev, loopback
|
||||
// excluded. Each line is " eth0: <rx bytes> <rx packets> … <tx bytes> …" — the
|
||||
// name is separated by a colon, which may or may not have a space after it, so
|
||||
// the split is on the colon and not on whitespace.
|
||||
// the split is on the colon and not on whitespace. proc(5), /proc/net/dev:
|
||||
// eight receive columns (bytes first), then the transmit ones (bytes first).
|
||||
// Linux net/core/net-procfs.c, dev_seq_printf_stats(): "%6s: %7llu ..." in
|
||||
// master read 2026-10-14; the host's kernel may be older, and the colon is
|
||||
// all this relies on (net/core/dev.c, dev_valid_name() refuses ':' in a
|
||||
// device name).
|
||||
func readNetDev(root string) (map[string]netCounters, error) {
|
||||
data, err := os.ReadFile(filepath.Join(root, "net", "dev"))
|
||||
if err != nil {
|
||||
|
||||
@@ -9,7 +9,9 @@ import (
|
||||
|
||||
// fakeProc writes a /proc-shaped directory the sampler can be pointed at, so
|
||||
// the parsing and the arithmetic are tested against known counters instead of
|
||||
// whatever the machine running the tests happens to be doing.
|
||||
// whatever the machine running the tests happens to be doing. The file
|
||||
// contents in this test are composed, in the layouts proc(5) gives for
|
||||
// /proc/stat, /proc/meminfo and /proc/net/dev.
|
||||
func fakeProc(t *testing.T, stat, meminfo, loadavg, netdev string) string {
|
||||
t.Helper()
|
||||
dir := t.TempDir()
|
||||
|
||||
@@ -16,7 +16,9 @@ import (
|
||||
// add to a poll.
|
||||
const milterDialTimeout = time.Second
|
||||
|
||||
// CheckMilter probes a milter address in the two forms Postfix takes:
|
||||
// CheckMilter probes a milter address in the two forms build/postfix-config.sh
|
||||
// lets INBOUND_ANTISPAM_MILTER take (Postfix 3.7.11, src/milter/milter8.c,
|
||||
// milter8_connect(), also accepts "local:", which that script refuses):
|
||||
// unix:/path is stat-ed like any milter socket, and inet:host:port is dialled
|
||||
// and the connection closed at once, with nothing written. required says what
|
||||
// a failure costs, as for CheckSocket: an error when mail stops without the
|
||||
|
||||
@@ -38,7 +38,9 @@ type Process struct {
|
||||
// The command takes fixed arguments and no user input, so it never goes through
|
||||
// a shell (security.md). `supervisorctl status` deliberately exits non-zero when
|
||||
// some program is not running, so the output is parsed first and the exit status
|
||||
// only matters when nothing could be parsed from it.
|
||||
// only matters when nothing could be parsed from it. supervisor 4.2.5,
|
||||
// supervisor/supervisorctl.py, do_status(): exit status 3 (NOT_RUNNING) when
|
||||
// any program is in STOPPED_STATES, 4 when supervisord does not answer.
|
||||
func Processes() ([]Process, error) {
|
||||
cmd := exec.Command("supervisorctl", "-c", supervisorConf, "status")
|
||||
out, err := cmd.CombinedOutput()
|
||||
@@ -54,7 +56,8 @@ func Processes() ([]Process, error) {
|
||||
|
||||
// supervisorStates are the state words supervisord prints. Lines whose second
|
||||
// field is not one of them are not status lines (banners, error text) and are
|
||||
// skipped, so unexpected output cannot masquerade as a process.
|
||||
// skipped, so unexpected output cannot masquerade as a process. supervisor
|
||||
// 4.2.5, supervisor/states.py, ProcessStates.
|
||||
var supervisorStates = map[string]bool{
|
||||
"STOPPED": true,
|
||||
"STARTING": true,
|
||||
@@ -68,6 +71,10 @@ var supervisorStates = map[string]bool{
|
||||
|
||||
// parseProcesses turns supervisorctl's tabular output into Process values. Each
|
||||
// status line is "<name> <STATE> <detail...>", column-aligned with spaces.
|
||||
// supervisor 4.2.5, supervisor/supervisorctl.py, _show_statuses(): the
|
||||
// template is "%(namespec)-<maxlen+3>s%(state)-10s%(desc)s" (maxlen at least
|
||||
// 30, no state word longer than 8), and an unknown name prints
|
||||
// "<name>: ERROR (no such process)".
|
||||
func parseProcesses(out string) []Process {
|
||||
var procs []Process
|
||||
for _, line := range strings.Split(out, "\n") {
|
||||
@@ -89,7 +96,9 @@ func parseProcesses(out string) []Process {
|
||||
|
||||
// Uptime reads for how long a running program has been up from the detail
|
||||
// supervisord prints for it: "pid 41, uptime 0:03:12", or with days in front,
|
||||
// "pid 41, uptime 18 days, 3:02:11".
|
||||
// "pid 41, uptime 18 days, 3:02:11". supervisor 4.2.5,
|
||||
// supervisor/rpcinterface.py, _interpretProcessInfo(): 'pid %s, uptime %s'
|
||||
// with Python's str() of a timedelta from the spawn time ("1 day, ...").
|
||||
func Uptime(detail string) (time.Duration, bool) {
|
||||
_, rest, found := strings.Cut(detail, "uptime ")
|
||||
if !found {
|
||||
|
||||
@@ -18,9 +18,17 @@ type Socket struct {
|
||||
}
|
||||
|
||||
// CheckSocket stats a milter socket. required distinguishes the two milters:
|
||||
// OpenDKIM runs with default_action=tempfail, so a missing socket stops mail
|
||||
// leaving the server, while the journal-milter fails open — mail still goes out,
|
||||
// only the send log stops being written.
|
||||
// Postfix attaches OpenDKIM with default_action=tempfail, so a missing socket
|
||||
// stops mail leaving the server, while the journal-milter is attached with
|
||||
// default_action=accept and fails open — mail still goes out, only the send
|
||||
// log stops being written (build/postfix-config.sh smtpd_milters; Postfix
|
||||
// 3.7.11, src/milter/milter8.c, milter8_comm_error(): tempfail answers "451
|
||||
// 4.7.1 Service unavailable - try again later", accept answers nothing).
|
||||
//
|
||||
// A socket file that is there is reported as listening. It is not proof of
|
||||
// it: a unix socket's file outlives the process that bound it unless that
|
||||
// process unlinks it (unix(7)), so a killed OpenDKIM or panel leaves one that
|
||||
// refuses every connection - assumptions A-38.
|
||||
func CheckSocket(name, path string, required bool) Socket {
|
||||
s := Socket{Name: name, Path: path}
|
||||
if path == "" {
|
||||
|
||||
Reference in New Issue
Block a user