Move the delivery log from the ephemeral /var/log to /data/log/mail.log so
the lines that resolve a queued send-log row survive a container recreate.
postlogd writes it as postfix, the panel reads it through the selfpost group
(dir 2750, file 0640, normalised every start); backups exclude log/.
Close the residual gap with a queue sweep: rows queued for over two minutes
whose id postqueue -p no longer lists are marked bounced. The sweep waits
until the tailer has read the log to its end and does nothing when the queue
cannot be listed, so a message in flight is never touched.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
The lines came out as one block of preformatted text, which is what the
system log page does with a tail of mail.log — right there, where the
lines are unrelated to each other and the block is the log itself. Here
they are one message's six or seven lines, and what is read off them is
the pace: the second between the connection and the banner, the ten
between DATA and the reply. Run together, every line began with a
different-width stamp and none of those numbers lined up.
So they are two columns now, when and what, the same shape the send log
itself has. logtail.SplitTimestamp takes the stamp off the head of a
line: postlogd's format, which is what this server writes, and syslog's
traditional one for a deployment that routes the log through syslogd
instead. The stamp loses its microseconds and its offset — five decimal
places are the widest part of the column and the least worth reading —
but is not converted, so the page shows the log's own wall clock rather
than a claim about which zone it was in.
A line whose head is not a stamp either parser recognises keeps its whole
text in the second column and leaves the first empty. The format is the
log's, not ours; a line we cannot split is a line we must not drop, and
the test says so.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The delivery page was a list of the fields the send-log table has no
column for, stacked one per line down the reading measure. Six values of
a few characters each — domain, application, queue id, journal id and two
timestamps — came to a page of mostly empty rows, and none of them
answered the question the log raises when a row is opened: what actually
happened to this message.
So the page states that instead. The subject heads it and the sender,
recipient and outcome are the line under it, which puts what the message
was and how it ended on the first line. Below, two columns: what the
journal recorded on the left, as a grid of tiles rather than a stack, and
on the right the two timestamps stated as the steps they stand for —
accepted and queued, then delivered, deferred, bounced, or refused before
queueing. Each step carries its status in the panel's own
ok/warn/error/unknown vocabulary, so a colour means here what it means on
the status page. A message still queued shows the report it is waiting
for as a step that has not happened, rather than dating it with the
moment the row was written.
Under both, at full width, the mail.log lines for the message's queue id.
The queue id was printed on this page as something to go and search the
system log for by hand; logtail.QueueLines does that search. It scans a
bounded tail of the current file — finding one message's lines means
reading rather than seeking — and anchors the match on the character
before the id, since queue ids are hexadecimal runs and a shorter one is
regularly the tail of a longer one. Send-log rows outlive mail.log
(retention ninety days, rotation fourteen files), so a message with
nothing left to show says so; that is the normal end state, not a fault,
and only a log that cannot be read at all is reported as one.
Two cards abreast and a block of raw log lines do not fit the reading
measure, so the page now declares itself wide — the opposite of what it
did when the column width was unified, where it was the page that stayed
prose. The mechanism is unchanged and is why the reversal costs one line:
how wide a page needs to be is the page's own property, not the
navigation entry's.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Codeberg is being retired as the project's public site, so every reference now
points at GitHub. That includes the Go module path (codeberg.org/mix/selfpost →
github.com/mixeme/selfpost): leaving an import path on a host that is going
away would break `go get` and `go install`, so this is not only a docs change.
Touches go.mod, test/e2e/go.mod, all imports, Makefile MODULE, the -ldflags
version stamp in build/Dockerfile and docs/development.md, the licence headers
in the SVG/HTML assets, and README (no more primary/mirror pair).
Comments no longer cite the archived specification. "spec 7.6.1", "spec 5.1"
and friends pointed into docs/archive/specification-v1.0.md, which is marked as
not a source of truth; each is now a reference to the live document that owns
the subject — architecture.md (with section), product.md, security.md or the
README. The review only asked for the 7.x refs (code-review.md § 4), but 4/5/6/
8/9 had the same defect, so they went too. Comments only, no behaviour change.
Also closes the remaining review items: architecture.md gained a Code layers
section with the layer diagram (A2), and TestParseDelivery gained the exotic
mail.log cases (§ 3).
Fixes a bug that last test found: the delivery-line pattern matched status=
greedily, taking the *last* occurrence on the line. Postfix appends the remote
server's reply verbatim, so a rejection whose reply quoted "status=sent" was
filed as a delivered message in the send log. It now takes the first status=
after the recipient, which is the real field.
R7 (CONTRIBUTING.md) moved to roadmap 2.x — one developer, no external PR flow,
so the file would have no audience yet. R1 (compose image tag) and the git tag
stay in roadmap § v1.x as the release-commit steps.
gofmt/go vet clean on both modules; go test ./... green except the three known
Windows-only failures (file perms, backslash paths, renaming an open file).
Not exercised on the dev server — no Docker locally.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
- logtail: persist the read position (offset + fingerprint of the log's
first 512 bytes) in a new logtail_state table (migration 0003) and
resume from it on start, so delivery lines written while the panel was
down are parsed instead of skipped and their send-log rows no longer
stay "queued" forever. Fingerprint mismatch (rotated/recreated while
down) reads the file from the start — re-parsing is idempotent; a
first-ever start with nothing stored still begins at end-of-file.
Writes are throttled to one per 5s, forced on rotation and shutdown.
- milter: count messages that passed the level-2 check but have not
reached the send log yet (internal/milter/inflight.go), so concurrent
SMTP sessions cannot each spend the same last slot. A literal
count+insert transaction, as the review suggested, is not possible:
the count happens at MAIL FROM and the insert at end-of-message.
Reservations are released after the insert, on ABORT, and after a
10-minute TTL — a client that drops mid-transaction must not be able
to hold a slot, since the limiter is fail-open by design.
Docs: architecture.md (log tailer, persistence, L2 counting),
security.md and roadmap.md (restart gap closed, container recreate
remains), CHANGELOG, progress.md, code-review.md.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Implement the structured send log (spec 7.3), the project's highest-risk
component since a milter bug can break the relay itself.
- internal/milter: go-milter v0.4.1 journal-milter. Per-connection session
collects SASL login, From, recipients and Subject across callbacks and
writes one send_log "queued" row per (queue-id, recipient) at EOM
(spec 7.3.3). Monitoring only: callbacks return Continue/Accept, recorder
errors are logged never propagated, so it can never block mail.
- internal/logtail: polling mail.log tailer with rotation handling (inode
change / truncation), parses sent/deferred/bounced/expired by queue-id +
recipient and advances rows; background retention sweep prunes rows past
SEND_LOG_RETENTION_DAYS (default 90) at startup and every 6h.
- internal/store/sendlog.go: InsertQueued, UpdateStatus (case-insensitive
recipient match), DeleteSendLogBefore + status constants.
- cmd/panel: open the store once and share it across http/milter/tailer;
replace the journal/logtail stubs with the real roles.
- build/postfix-config.sh: bounded milter timeouts (15/15/30s) so a hung
milter also fails open in seconds, not the 300s default.
Fix found in-container: SASL login (app_login) was empty because go-milter
keys macros exactly as Postfix sends them, and multi-character macro names
arrive brace-wrapped ({auth_authen}); the SASL-less Phase 0 spike could not
observe this. Added a brace-tolerant macro lookup.
Verified on selfpost.example.com: gofmt/vet/unit tests green; container e2e
records rows with correct fields and advances status via the tailer; fail-open
confirmed for both an unreachable and a hung milter; retention prunes at start.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
mixeme
mix