Move the delivery log from the ephemeral /var/log to /data/log/mail.log so
the lines that resolve a queued send-log row survive a container recreate.
postlogd writes it as postfix, the panel reads it through the selfpost group
(dir 2750, file 0640, normalised every start); backups exclude log/.
Close the residual gap with a queue sweep: rows queued for over two minutes
whose id postqueue -p no longer lists are marked bounced. The sweep waits
until the tailer has read the log to its end and does nothing when the queue
cannot be listed, so a message in flight is never touched.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
The delivery log now lists what identifies a message and nothing else —
time, sender, recipient, subject, status — and links each row to
/deliveries/{id}, which carries the rest: the sending domain, the
application it was submitted under, the Postfix queue id to search the
system log for, and when the status was last reported. Domain and
application were a column each; they were the widest thing in the table
after the addresses and repeat down every filtered page, and they remain
the log's two filters. Back returns to the page and filters the row was
opened from, rebuilt from the log's own parameters only.
Subjects are now decoded for display as well as on the way in. The milter
has decoded them since 8add005, but the rows it wrote before that still
hold the raw =?utf-8?Q?...?= header, and those are the ones an operator is
most likely to still be reading. The decoder moves to internal/mailhdr,
shared by the milter and the panel; it is idempotent, so a row decoded
once passes through unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Codeberg is being retired as the project's public site, so every reference now
points at GitHub. That includes the Go module path (codeberg.org/mix/selfpost →
github.com/mixeme/selfpost): leaving an import path on a host that is going
away would break `go get` and `go install`, so this is not only a docs change.
Touches go.mod, test/e2e/go.mod, all imports, Makefile MODULE, the -ldflags
version stamp in build/Dockerfile and docs/development.md, the licence headers
in the SVG/HTML assets, and README (no more primary/mirror pair).
Comments no longer cite the archived specification. "spec 7.6.1", "spec 5.1"
and friends pointed into docs/archive/specification-v1.0.md, which is marked as
not a source of truth; each is now a reference to the live document that owns
the subject — architecture.md (with section), product.md, security.md or the
README. The review only asked for the 7.x refs (code-review.md § 4), but 4/5/6/
8/9 had the same defect, so they went too. Comments only, no behaviour change.
Also closes the remaining review items: architecture.md gained a Code layers
section with the layer diagram (A2), and TestParseDelivery gained the exotic
mail.log cases (§ 3).
Fixes a bug that last test found: the delivery-line pattern matched status=
greedily, taking the *last* occurrence on the line. Postfix appends the remote
server's reply verbatim, so a rejection whose reply quoted "status=sent" was
filed as a delivered message in the send log. It now takes the first status=
after the recipient, which is the real field.
R7 (CONTRIBUTING.md) moved to roadmap 2.x — one developer, no external PR flow,
so the file would have no audience yet. R1 (compose image tag) and the git tag
stay in roadmap § v1.x as the release-commit steps.
gofmt/go vet clean on both modules; go test ./... green except the three known
Windows-only failures (file perms, backslash paths, renaming an open file).
Not exercised on the dev server — no Docker locally.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
- logtail: persist the read position (offset + fingerprint of the log's
first 512 bytes) in a new logtail_state table (migration 0003) and
resume from it on start, so delivery lines written while the panel was
down are parsed instead of skipped and their send-log rows no longer
stay "queued" forever. Fingerprint mismatch (rotated/recreated while
down) reads the file from the start — re-parsing is idempotent; a
first-ever start with nothing stored still begins at end-of-file.
Writes are throttled to one per 5s, forced on rotation and shutdown.
- milter: count messages that passed the level-2 check but have not
reached the send log yet (internal/milter/inflight.go), so concurrent
SMTP sessions cannot each spend the same last slot. A literal
count+insert transaction, as the review suggested, is not possible:
the count happens at MAIL FROM and the insert at end-of-message.
Reservations are released after the insert, on ABORT, and after a
10-minute TTL — a client that drops mid-transaction must not be able
to hold a slot, since the limiter is fail-open by design.
Docs: architecture.md (log tailer, persistence, L2 counting),
security.md and roadmap.md (restart gap closed, container recreate
remains), CHANGELOG, progress.md, code-review.md.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Removes ~30 stale "Phase N" / historical-staging comment references from
code and shell scripts now that v1.0 is done; fixes a stale dashboard
comment claiming applications/send-log were unimplemented; adds a CSRF ADR
to security.md documenting the Origin-check-over-tokens decision; resolves
docs/logo in roadmap.md (directory doesn't exist, criterion already met);
adds a gofmt -l check to CI so unformatted Go fails the build.
The known-limitations write-up for the log-tailer offset gap (the other
Phase 1 item) was already present in architecture.md § Log tailer, so no
change was needed there.
gofmt/go vet/go test clean on both Go modules (main + test/e2e), verified
on the dev server.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Sessions move from an in-memory map (absolute 12h TTL) to a `sessions`
table (migration 0002), storing only the SHA-256 of the token. Expiry is
now a sliding idle window (PANEL_SESSION_IDLE_DAYS, default 7, no
absolute cap), extended at most once an hour and never by the
monitoring screens' background polling (GET + HX-Request), so a
forgotten open tab doesn't keep a session alive indefinitely. A login
now survives a container restart or redeploy.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Phase 12 (UI/UX). The navigation bar now renders once from layout.html
instead of being copied into each content template, so it is present on
every authenticated page — including the domain page and its delete
confirmation, which had no links at all — and the current page is
highlighted via .Active rather than quietly dropping out of the list.
New /account page changes the administrator's username and/or password:
the current password is required and the attempt is throttled on the same
limiter as the login form, so this route cannot be used to brute-force
past that limit. A password change invalidates every other session while
keeping the one performing it; a rename carries that session over.
Backup and domain import move from a card in the middle of the domain
list to their own /backup page, one card each; the handlers themselves
are unchanged, only the page the import form renders its errors on.
The domain page gains a "Sending server settings" card (server, port,
encryption) so a client can be configured without reading the docs; 587
is listed only when SUBMISSION_ENABLE is true for this deployment, which
is a deploy-time flag the panel cannot verify at runtime.
Client-side (static/panel.js, no libraries): Copy buttons on the values
that get carried elsewhere (DKIM record, new application credentials,
server name), and the Addresses field is hidden while the address mode is
wildcard, where the server ignores it.
Verified in a container on the dev server: setup, login, every page's
nav and active item, domain and application creation, all account-form
paths including cross-session invalidation, import errors, full backup
download. gofmt/vet/test/docker build green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The journal-milter, until now a pure monitor, now refuses a message with a
4xx tempfail (RespTempFail/451) at MAIL FROM when a per-domain or per-
application limit is exceeded. Key is the client IP; the count is
COUNT(DISTINCT queue_id) over a sliding window reusing the send log; the
limit applies only when a non-empty IP binding matches the client (empty
binding => level-1 only, per spec 7.4). Enforcement is fail-open on the
milter's own errors — a limiter malfunction never blocks mail, and Postfix's
level-1 anvil limit stays the independent backstop. Refused messages are
recorded in send_log with status "rejected" for UI visibility.
- store/ratelimits.go: RateLimit type (+Active/AllowsIP), id-keyed get/set/
delete for the panel, name/login-keyed lookup + windowed distinct-message
count for the milter, DeleteRateLimitsForDomain. No migration — the
rate_limits table has existed since Phase 2.
- milter: enforce at MailFrom, fail-open helper overLimit, InsertRejected.
- web: server-side validated IP/ceiling/window forms on the domain page and
per application; routes POST /domains/{id}/ratelimit and
/applications/{aid}/ratelimit. Milter reads rows live, so no reload.
- domain/app services clear limits on deletion (rate_limits has no FK cascade).
Unit tests + container e2e (p8) green: refusal on both scopes, unregistered
IP ignored, fail-open with the panel stopped.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Three HTMX-polled monitoring screens (spec 7.2.11-13): send log with
server-side domain/application filters and pagination, Postfix queue
(postqueue -p), and a mail.log tail. Fragment endpoints return HTML
snippets, not JSON (spec 7.1); all output is auto-escaped via
html/template (spec 7.6.7).
Adds store.QuerySendLog/CountSendLog/ListApplicationLogins,
postfix.Queue(), and logtail.TailLines (a point-in-time reverse read,
independent of the background follow loop). Verified on the dev server:
gofmt/vet/test green, docker build green, container e2e (filters,
60-row pagination, <script> escaping, real postqueue/mail.log output,
existing Reload button unaffected).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Implement the structured send log (spec 7.3), the project's highest-risk
component since a milter bug can break the relay itself.
- internal/milter: go-milter v0.4.1 journal-milter. Per-connection session
collects SASL login, From, recipients and Subject across callbacks and
writes one send_log "queued" row per (queue-id, recipient) at EOM
(spec 7.3.3). Monitoring only: callbacks return Continue/Accept, recorder
errors are logged never propagated, so it can never block mail.
- internal/logtail: polling mail.log tailer with rotation handling (inode
change / truncation), parses sent/deferred/bounced/expired by queue-id +
recipient and advances rows; background retention sweep prunes rows past
SEND_LOG_RETENTION_DAYS (default 90) at startup and every 6h.
- internal/store/sendlog.go: InsertQueued, UpdateStatus (case-insensitive
recipient match), DeleteSendLogBefore + status constants.
- cmd/panel: open the store once and share it across http/milter/tailer;
replace the journal/logtail stubs with the real roles.
- build/postfix-config.sh: bounded milter timeouts (15/15/30s) so a hung
milter also fails open in seconds, not the 300s default.
Fix found in-container: SASL login (app_login) was empty because go-milter
keys macros exactly as Postfix sends them, and multi-character macro names
arrive brace-wrapped ({auth_authen}); the SASL-less Phase 0 spike could not
observe this. Added a brace-tolerant macro lookup.
Verified on selfpost.mixfed.ru: gofmt/vet/unit tests green; container e2e
records rows with correct fields and advances status via the tailer; fail-open
confirmed for both an unreachable and a hung milter; retention prunes at start.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds application accounts bound to domains: a SASL login/password in
sasldb2, a per-application address mode (wildcard @domain or an explicit
list), and matching smtpd_sender_login_maps bindings — with create,
list, edit-mode, delete and password regeneration (spec 4.1, 5.1,
7.2.5-9). Generated passwords are shown exactly once and never stored in
plaintext (7.6.1).
- internal/store/applications.go: transactional CRUD; globally unique
login; ListBindings (address->login) as the map source; logins-by-
domain for pre-cascade SASL cleanup.
- internal/app: saslpasswd2 wrapper (password via stdin, login as a
whitelisted argv element, no shell — 7.6.3); strong base64url password;
address validation that enforces domain ownership before any config
write (7.6.2); service orchestrating store + sasldb2 + map with full
rollback on partial failure.
- internal/postfix: sender_login_maps regenerated as a pure function of
the registry (many-to-one logins merged per address), atomic write,
injection backstop (7.6.4).
- Postfix reload, corrected: `postfix start-fg` forks a separate master,
so signalling the supervised process never reaches it. Reload now runs
the canonical `postfix reload` via a one-shot supervisord program the
unprivileged panel triggers over the group control socket. Verified in
mail.log.
- domain.Service.Delete purges the domain's SASL accounts, then cascades,
then rebuilds the sender map and reloads; manual reload now covers both
OpenDKIM and Postfix.
- web: application management in the domain page, one-time credential
shown inline; postfix joins the selfpost group and entrypoint normalises
/data/sasl and /data/postfix (setgid, group-readable) with self-heal.
Verified on the dev server: gofmt/vet/test green, image builds, and a
container e2e covers the full application lifecycle, domain-delete
cascade, restart persistence, and a real postfix reload.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add/list/delete of sending domains with per-domain DKIM keys and the
OpenDKIM tables that drive signing (spec 6, 7.2.2-4, 7.2.10).
internal/domain:
- Pure-Go RSA-2048 keygen; PKCS#1 PEM written atomically at 0640; the
published DNS TXT record is derived from the key on disk (single source
of truth) rather than persisted. No os/exec for key generation.
- KeyTable/SigningTable fully regenerated from the registry on every
add/delete (idempotent), written atomically; SigningTable via refile:
with *@domain, KeyTable with absolute key paths. Table writer refuses
any unsafe character as a backstop (spec 7.6.4).
- Reload without root: the unprivileged panel signals OpenDKIM through
supervisord (`supervisorctl signal USR1 opendkim`, fixed args, no
shell, no user input — spec 7.6.3). An existing key is reused, never
overwritten, so re-adding a domain keeps its published DNS valid.
- Service orchestrates registry -> key -> table rebuild -> reload, with
rollback of the row if a downstream step fails; delete cascades apps
via the DB FK and removes the key + table entries.
Infra:
- Shared `selfpost` group bridges panel (writes keys) and opendkim
(reads them); /data/opendkim is setgid so panel-created files inherit
the group, keys are 0640, RequireSafeKeys is disabled by design.
- opendkim.conf moves from verify-only (Mode v) to signing (Mode s).
- entrypoint.sh normalises the DKIM tree on every start (ownership,
setgid, perms, empty tables before opendkim starts) — self-healing
after a restore.
- supervisord control socket opened to the `selfpost` group so the panel
can request the reload.
web/store:
- Strict domain-name validation (whitelist [a-z0-9.-], DNS shape, >=2
labels), lower-case normalisation (spec 7.6.2).
- Domain queries with application counts; delete relies on ON DELETE
CASCADE. Dashboard lists domains + add form; domain page shows the
DKIM record; a dedicated confirm page warns about the app cascade
before deletion (spec 7.2.4); manual reload button (spec 7.2.12,
OpenDKIM side; Postfix reload lands in Phase 5).
- Authenticated routes moved to a sub-mux using Go 1.22 method/wildcard
patterns.
Tests: validateDomain, DKIM keygen/record roundtrip, table rendering +
injection-safety, key reuse, store cascade. Verified on the dev server:
gofmt/vet/test green, image builds, container e2e (add/delete a domain,
DKIM record shown, OpenDKIM reads panel keys and reloads, keys and
tables persist across a restart).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
mixeme
mix