selfpost/docs/development.md

SelfPost — development

What this file is. How to build, test, document, and ship changes. Open work after 1.0 (1.x+) lives in roadmap.md and linked plans/. Product boundaries: product.md. As-built layout: architecture.md.

---

Resuming work

After /clear or a fresh chat:

1. Read this file (process, docs rules, model routing).
2. Open roadmap.md for the index of open work; follow the linked plan file for the active item. Accepted risks — security.md; as-built — architecture.md.
3. Skim product.md if scope is in doubt.
4. Continue from the next unchecked step in the active plan file (not the roadmap index).

History of closed phases is in git log and CHANGELOG.md, not duplicated here.

---

Authorship and disclosure

Decided, not open for re-litigation. SelfPost is written by AI agents under a maintainer's direction, and the project says so rather than hiding it.

Concretely, this is what "says so" means, and none of it is an oversight to be tidied away later:

• Co-Authored-By: Claude <model> trailers stay in commit messages, including the ~140 commits that predate v1.0.
• The model routing table below is public, in a file the README links to.
• .cursor/rules/agent-rules.mdc ships in the repository.
• Process notes written for an agent — "after a context reset, pick an item marked agreed" in roadmap.md — stay as they are.

Why not quietly drop it. Once the trailers are in the history, removing the routing table or the rules file would not conceal authorship, it would only make the project look like it was trying to. Partial concealment reads worse than the plain statement, and the plain statement costs nothing: the code is reviewed, tested, and shipped under the same rules either way, and the security design records what was audited and what was accepted.

Revisit if: the disclosure ever conflicts with the licence or a downstream obligation — not because the convention around AI authorship shifts.

---

Model routing

Kind of work	Model	Examples
Security, infra, file permissions, Postfix/postqueue, open-relay risk	Opus	mail.log under /data, entrypoint permissions, queue reconcile
UI / JS / CSS, templates, documentation (English), README	Sonnet	adaptive polling, this file's Documentation section
Trivial mechanics: retarget links, grep, compose bump, CHANGELOG cut	Haiku	Makefile / release.yml comment fixes, deleting closed plan files
Security review (not authorship)	Fable	pre-release checklist (CHANGELOG [0.5.0] Security / security.md — done)

Default rule: risk-critical → Opus; UI / docs / boilerplate → Sonnet; trivial mechanics → Haiku. Reviewers must not be the author of the code under review.

---

Plan checklists

Open roadmap.md items with a linked plan file carry an Implementation checklist in plans/. After a context reset, work the active plan — not the roadmap prose — from the first unchecked line.

Checklist format

• - [ ] Step description — **Model** (model from the routing table above).
• Mark [x] only in the commit that actually finishes the step.
• candidate items: checklist may exist, but no code until the roadmap status is agreed.

Progress column in the roadmap index (0/N) counts every checklist line in that plan (including docs and go vet / go test steps). Update N when the checklist changes; update the numerator when steps are checked off.

Roadmap track: commit and version

For the current 1.x+ roadmap queue (roadmap.md index), stricter than the default «meaningful step» rule in § Commits and release build:

After each checklist step

1. Check off the step in the plan file; bump Progress in the roadmap index.
2. Append CHANGELOG.md under [Unreleased] for that step.
3. Commit on main (push / tag / Release only on explicit request).

After each roadmap stage (all checklist steps done; «Done when» satisfied)

1. Version cut in one commit: rename [Unreleased] to [X.Y.Z] - date, open a fresh [Unreleased], bump the image pin in deploy/docker-compose.yml.
2. Git tag vX.Y.Z and publish the GitHub Release only on explicit request (see § Release image).

Planned version cuts (from pin 1.3.0; adjust if semver changes mid-track):

Stage	ID	Cut
1	queue-retries	1.3.1 (PATCH)
2	inbound-relay	1.4.0
3	send-log-retention	1.5.0
4	domain-stats-auto-ratelimit	1.6.0
5	dmarc-reports	1.7.0
6	panel-docs	1.8.0

Docs-only prep (checklists in plans, no product code) uses the same per-step commit rule but no version cut until stage 1 ships.

---

Technology stack and tools

Component	Version / notes
Go	1.26+ (go.mod); CGO_ENABLED=0 — pure Go, static linking
SQLite	modernc.org/sqlite (pure Go, no cgo)
Build	Makefile: primary targets vet, test, build, e2e (also all, clean)
Container	Docker + Compose v2 on the dev host and in CI
Image (build stage)	golang:1.26-bookworm — build/Dockerfile
Image (runtime)	debian:bookworm-slim + Postfix, OpenDKIM, supervisord, SASL, logrotate
CI	GitHub Actions — .github/workflows/
Image registry	ghcr.io/mixeme/selfpost

Repository layout (brief; process details in architecture.md):

• cmd/panel — HTTP panel + journal-milter + log-tailer
• cmd/selfpost-backup — backup CLI (docker exec … selfpost-backup)
• internal/ — domain logic, store, web, health
• build/ — Dockerfile, supervisord, Postfix/OpenDKIM, entrypoint
• deploy/ — docker-compose.yml, proxy examples, .env.example
• test/e2e/ — separate Go module; container integration tests

---

External libraries

The project is AGPL-3.0 (LICENSE). Copyright holder and third-party notices: NOTICE. The tree does not use per-file SPDX-License-Identifier headers; AGPL-3.0 does not require them. New Go dependencies must be permissive or GPL-family (see .cursor/rules/agent-rules.mdc).

Main module (go.mod)

Package	Version	Repository	License
github.com/emersion/go-milter	v0.4.1	https://github.com/emersion/go-milter	BSD-2-Clause
golang.org/x/crypto	v0.54.0	https://github.com/golang/crypto	BSD-3-Clause
modernc.org/sqlite	v1.53.0	https://gitlab.com/cznic/sqlite (mirror: https://github.com/modernc-org/sqlite)	BSD-3-Clause

Transitive dependencies — go mod graph / go.sum; all indirect packages in the tree are AGPL-3.0-compatible.

Vendored front-end

Asset	Version	Repository	License
internal/web/view/static/htmx.min.js	2.0.4	https://github.com/bigskysoftware/htmx	0BSD
internal/web/view/static/ibm-plex-*.woff2	latin subset	https://github.com/IBM/plex	SIL OFL 1.1 (OFL.txt beside the files)

E2e module (test/e2e/go.mod)

Package	Version	Repository	License
github.com/emersion/go-msgauth	v0.6.8	https://github.com/emersion/go-msgauth	BSD-2-Clause

The test module is not part of the main go build graph and is not shipped in the image.

Debian packages in the runtime image

Postfix, OpenDKIM, supervisord, sasl2-bin, logrotate, and others come from Debian bookworm repositories; licenses are in each package's copyright file on https://packages.debian.org/bookworm/. The image also ships LICENSE, NOTICE, and the IBM Plex OFL.txt under /usr/share/doc/selfpost/. The panel serves the AGPL text at /license and the OFL text at /static/OFL.txt.

---

Building binaries and the image

Local binaries

Requires Go 1.26+ and CGO_ENABLED=0.

make build        # bin/panel, bin/selfpost-backup (VERSION=dev by default)
make build VERSION=1.2.3

Or directly:

go build -trimpath -ldflags "-X github.com/mixeme/selfpost/internal/buildinfo.Version=dev" -o bin/panel ./cmd/panel

The version is stamped into both binaries via -ldflags and must match the Docker image tag — restore checks backup version compatibility.

Docker image

From the repository root:

docker build -f build/Dockerfile -t selfpost:dev --build-arg VERSION=dev .

The Dockerfile has a build stage (go vet, go build with VERSION) and a runtime stage (Debian + mail stack). Runtime config and scripts use COPY --chmod so file modes in the image do not depend on how the build context was synced (e.g. a Windows checkout widening permissions on logrotate-mail.conf). See architecture.md § Image and processes.

---

Commits and release build

Commit on every meaningful step (a working sub-feature, a green build, end of a phase) — not every file save, and not only at phase end. Minimum: one commit per closed phase, plus intermediate commits for coherent sub-steps. Branch main unless a separate branch is requested. Push / PR only on explicit request.

Commit messages end with Co-Authored-By: Claude <model> <noreply@anthropic.com> for the model that did the step (e.g. Claude Sonnet 4.6).

Every such step also updates CHANGELOG.md under [Unreleased] (Keep a Changelog). On an explicit version cut, rename [Unreleased] to [X.Y.Z] - date and open a fresh empty [Unreleased]. Image tag / push only on explicit request (see release.yml).

Release image

The release image is published only for a SemVer version X.Y.Z: a published GitHub Release whose tag is vX.Y.Z, or a workflow_dispatch that supplies that version. Pushing a git tag alone does not publish. Ordinary commits, and a dispatch from main without a version input, do not publish. The version is the single source that drives the image tag and -ldflags in the binaries so they cannot drift apart.

Steps (on explicit request):

1. Close [Unreleased] in CHANGELOG.md and bump the pinned tag in deploy/docker-compose.yml (and any local-trial image references) in the same release commit.
2. Create and push git tag vX.Y.Z on that commit.
3. Publish the GitHub Release for vX.Y.Z (not a draft).
4. Workflow release.yml builds, e2e-gates, and publishes ghcr.io/mixeme/selfpost:X.Y.Z (checks out tag vX.Y.Z).

GitHub Release vs GHCR. The public Releases page lists only published releases. A draft is visible to maintainers only — it looks like “no releases” to everyone else. CI does not create or publish the GitHub Release; you do that in the UI. Deleting a release’s git tag on GitHub (or re-pushing tags while cleaning the registry) converts a published release back into a draft — that matches “I published three times and it keeps disappearing”. After publish, leave the tag on GitHub; clean up only unwanted GHCR package versions, not the git tag.

Push workflow and source changes to github.com/mixeme/selfpost before publishing — Actions reads that repo, not Gitea.

Gitea → GitHub tag mirror. If every tag push from Gitea is mirrored to GitHub, two things follow:

1. GitHub Release tags must not be deleted on GitHub. Many mirror setups prune remote tags that are absent on Gitea (or re-push with --force / --prune). Deleting v1.0.0 / v1.3.0 on GitHub converts a published Release back to draft. Mirror branches and new tags forward; do not delete release tags on the GitHub side. GHCR cleanup is package versions in the UI — not git push github --delete and not tag prune on the mirror.

2. Tag push runs the workflow file at that tag's commit, not main. v1.0.0 still points at a commit whose release.yml has on: push: tags and no per-arch GHCR cleanup — every mirror (re)push of that tag can republish 1.0.0-amd64 / 1.0.0-arm64. Tags from v1.3.0 onward only run release.yml on Publish release (release: published), so mirroring those tags alone does not start the image build.

   Safe mirror: push tags to GitHub without deleting existing ones; keep release tags on Gitea; publish the GitHub Release on github.com after the mirror has the tag.

Ordinary commits do not publish an image. The compose pin and the git tag must match (1.0.0 / v1.0.0 for the first published release). Intermediate CHANGELOG sections (0.2.0…0.6.0) record development history before that cut.

---

Phase closure

Before /clear at the end of a finished step:

1. Update roadmap.md (and the active plan checklist, if any): what changed, what is next.
2. Check the applicable «Done when…» criteria.
3. Append CHANGELOG.md under [Unreleased].
4. Make the final commit for the step (when the user asks for a commit).

---

Testing

Static analysis and unit tests

Main module (go test ./...); e2e is a separate module — see below.

make vet          # go vet ./...
make test         # go test ./...

Or directly:

gofmt -l .        # in CI — fails on drift
go vet ./...
go test ./...

Env documentation regression

go test ./cmd/panel -run 'TestLoadConfigKeysDocumented|TestBuildScriptKeysDocumented|TestDocumentedKeysAreRead' — every new loadConfig key must appear in the env lists in guide.md (cmd/panel/envdoc_test.go).

End-to-end (container suite)

Separate Go module test/e2e/; not included in the main module's go test ./....

make e2e
# same as: cd test/e2e && go test -v -timeout 20m ./...

Stack: deploy/docker-compose.yml + test/e2e/compose.override.yml — same cap_drop/cap_add/no-new-privileges as production. Override: high ports (20465/20587/20080), test hostname, self-signed TLS, PANEL_COOKIE_SECURE=false, isolated compose project. Mail is hermetic: CoreDNS (fake zone) + Postfix smtp-sink as sink-MX; DKIM TXT is scraped from the panel and published into the zone — the test verifies the records an operator would actually use.

Coverage (summary): bootstrap → SMTP AUTH → delivery → DKIM verify → send-log queued → sent; negatives (no AUTH, relay, sender/login mismatch, L1/L2 limits, milter fail-open, bad SELFPOST_HOSTNAME, session survives docker restart); startup checks that supervisord actually brought up OpenDKIM, the panel, and Postfix (checkSupervisorProcesses), plus logrotate config-mode and forced-rotation checks (checkLogrotateConfigMode, checkLogrotateRotation — test/e2e/logrotate_check.go). Polling with timeouts only — no fixed sleep.

Requires Docker + Compose v2 on the machine running the suite.

---

CI

Workflows in .github/workflows/. What each job runs — § Testing above.

test.yml — every push and PR to main

gofmt -l → go vet ./... → go test ./... (main module, no e2e).

release.yml — published GitHub Release, or workflow_dispatch with SemVer

Publishing a GitHub Release runs release.yml directly (release: published, same pattern as gosentry / imap-scrub). You can also run it manually via workflow_dispatch with an explicit X.Y.Z input. A bare git tag push does not run the workflow. The build always checks out vX.Y.Z, not main HEAD.

prepare takes the version from github.event.release.tag_name on a release event, or from the workflow_dispatch version input. A dispatch whose input is missing or not X.Y.Z fails in prepare.

release: published
prepare (version from release tag or workflow_dispatch input; checkout vX.Y.Z)
  → build [matrix: ubuntu-latest / ubuntu-24.04-arm]
      → docker build --load (VERSION from prepare)
      → e2e (test/e2e)
      → push ghcr.io/...:X.Y.Z-amd64 | X.Y.Z-arm64
  → merge
      → docker buildx imagetools create → unified manifest X.Y.Z
      → GitHub Packages API → drop X.Y.Z-amd64 and X.Y.Z-arm64 from GHCR

Native per-arch matrix (no QEMU): running the full Postfix/OpenDKIM stack under emulation for e2e is impractical. E2e first, then push — the registry receives the bytes that passed the gate. Only ghcr.io/mixeme/selfpost:X.Y.Z remains tagged in GHCR; per-arch names exist briefly during the merge job.

A failed e2e blocks image publication.

---

Documentation

History of deleted plans lives in git and CHANGELOG.md. There is no docs/archive/ directory.

Documentation map

Home	File
Operator install / quick start	README.md
Operator guide	guide.md
Product boundaries	product.md
As-built design	architecture.md
Development process (this file)	development.md
Security requirements and accepted risks	security.md
Roadmap (1.x+)	roadmap.md
Active design plans	plans/
Release history	CHANGELOG.md

User-facing deliverables

Artefact	Role
README.md	Overview, requirements, quick start, docs index, reference deploy, licence
guide.md	Proxy, env, DNS, IP warmup, operations, rate limiting, backup, ports, image tag
SECURITY.md	Private reporting channel, supported versions, scope
LICENSE	AGPL-3.0 full text
NOTICE	Copyright holder and third-party attributions
deploy/docker-compose.yml + proxies	Apache + nginx/Caddy/Traefik under deploy/
deploy/.env.example	Public env template; full reference in guide.md
CHANGELOG.md	Keep a Changelog

Out of scope for v1.x: CONTRIBUTING.md, man pages, a separate docs site (candidates in roadmap.md).

Maintaining documentation

1. Step rule: a new or renamed env key, panel route, or observable mail-path behaviour ships together with guide.md / .env.example and a CHANGELOG entry (see § Commits and release build).
2. Env regression: cmd/panel/envdoc_test.go fails on an undocumented loadConfig or build-script key.
3. New gaps go into roadmap.md (or the active plan file), not silent drive-by edits.

Verifying docs against code

Every claim in the docs has a source of truth in the tree; verify from code to prose.

Claim class	Source of truth
Env keys and defaults	loadConfig — cmd/panel/main.go; ${VAR:-…} in build/
Mail path	build/postfix-config.sh
Panel routes	internal/web/web.go
Backup / restore, domain export	internal/backup/, cmd/selfpost-backup/
Sessions	internal/store/sessions.go, internal/web/auth/session.go
Log rotation, reload	build/logrotate-mail.conf, build/logrotate-loop.sh, build/postfix-cert-reload.sh
Deploy	deploy/docker-compose.yml, build/Dockerfile
Operator checklist	§ User-facing deliverables; detail — guide.md
Product / out of scope	product.md
As-built	architecture.md
Mandatory security	security.md

Order: list what the code actually does → find it in guide.md / architecture.md. Before every tag, a short pass over this table — not a full prose rewrite.