selfpost/docs/development.md

# SelfPost — development

**What this file is.** How to build, test, document, and ship changes. Open
work for v1.x and 2.x lives in [roadmap.md](roadmap.md) (and, until the tag,
[v1.x-closure-plan.md](v1.x-closure-plan.md)). Product boundaries:
[product.md](product.md). As-built layout: [architecture.md](architecture.md).

---

## Resuming work

After `/clear` or a fresh chat:

1. Read this file (process, docs rules, model routing).
2. Open [roadmap.md](roadmap.md) for open work; until `v1.0.0`, also
   [v1.x-closure-plan.md](v1.x-closure-plan.md) for the remaining closure
   checklist. Accepted risks — [security.md](security.md); as-built —
   [architecture.md](architecture.md).
3. Skim [product.md](product.md) if scope is in doubt.
4. Continue from the next unchecked step in the active plan.

History of closed phases is in `git log` and [CHANGELOG.md](../CHANGELOG.md),
not duplicated here.

---

## Model routing

| Kind of work | Model | Examples |
|---|---|---|
| Security, infra, file permissions, Postfix/`postqueue`, open-relay risk | **Opus** | `mail.log` under `/data`, entrypoint permissions, queue reconcile |
| UI / JS / CSS, templates, documentation (English), README | **Sonnet** | adaptive polling, this file's Documentation section |
| Trivial mechanics: retarget links, grep, compose bump, CHANGELOG cut | **Haiku** | Makefile / release.yml comment fixes, deleting closed plan files |
| Security **review** (not authorship) | **Fable** | pre-release checklist pass ([implementation-plan.md](implementation-plan.md) § D — done) |

Default rule: risk-critical → Opus; UI / docs / boilerplate → Sonnet; trivial
mechanics → Haiku. Reviewers must not be the author of the code under review.

---

## Technology stack and tools

| Component | Version / notes |
|---|---|
| **Go** | 1.26+ (`go.mod`); `CGO_ENABLED=0` — pure Go, static linking |
| **SQLite** | `modernc.org/sqlite` (pure Go, no cgo) |
| **Build** | [Makefile](../Makefile): `vet`, `test`, `build`, `e2e` |
| **Container** | Docker + Compose v2 on the dev host and in CI |
| **Image (build stage)** | `golang:1.26-bookworm` — [build/Dockerfile](../build/Dockerfile) |
| **Image (runtime)** | `debian:bookworm-slim` + Postfix, OpenDKIM, supervisord, SASL, logrotate |
| **CI** | GitHub Actions — [.github/workflows/](../.github/workflows/) |
| **Image registry** | `ghcr.io/mixeme/selfpost` |

**Repository layout** (brief; process details in [architecture.md](architecture.md)):

- `cmd/panel` — HTTP panel + journal-milter + log-tailer
- `cmd/selfpost-backup` — backup CLI (`docker exec … selfpost-backup`)
- `internal/` — domain logic, store, web, health
- `build/` — Dockerfile, supervisord, Postfix/OpenDKIM, entrypoint
- `deploy/` — `docker-compose.yml`, proxy examples, `.env.example`
- `test/e2e/` — **separate Go module**; container integration tests

---

## External libraries

The project is **AGPL-3.0** ([LICENSE](../LICENSE)). New Go dependencies must
be permissive or GPL-family (see
[.cursor/rules/agent-rules.mdc](../.cursor/rules/agent-rules.mdc)).

### Main module (`go.mod`)

| Package | Version | Repository | License |
|---|---|---|---|
| `github.com/emersion/go-milter` | v0.4.1 | <https://github.com/emersion/go-milter> | BSD-2-Clause |
| `golang.org/x/crypto` | v0.54.0 | <https://github.com/golang/crypto> | BSD-3-Clause |
| `modernc.org/sqlite` | v1.53.0 | <https://gitlab.com/cznic/sqlite> (mirror: <https://github.com/modernc-org/sqlite>) | BSD-3-Clause |

Transitive dependencies — `go mod graph` / `go.sum`; all indirect packages in
the tree are AGPL-3.0-compatible.

### E2e module (`test/e2e/go.mod`)

| Package | Version | Repository | License |
|---|---|---|---|
| `github.com/emersion/go-msgauth` | v0.6.8 | <https://github.com/emersion/go-msgauth> | BSD-2-Clause |

The test module is not part of the main `go build` graph and is not shipped in
the image.

### Debian packages in the runtime image

Postfix, OpenDKIM, `supervisord`, `sasl2-bin`, `logrotate`, and others come
from Debian bookworm repositories; licenses are in each package's `copyright`
file on <https://packages.debian.org/bookworm/>.

---

## Building binaries and the image

### Local binaries

Requires Go 1.26+ and `CGO_ENABLED=0`.

```sh
make build        # bin/panel, bin/selfpost-backup (VERSION=dev by default)
make build VERSION=1.0.0
```

Or directly:

```sh
go build -trimpath -ldflags "-X github.com/mixeme/selfpost/internal/buildinfo.Version=dev" -o bin/panel ./cmd/panel
```

The version is stamped into both binaries via `-ldflags` and **must match the
Docker image tag** — restore checks backup version compatibility.

### Docker image

From the repository root:

```sh
docker build -f build/Dockerfile -t selfpost:dev --build-arg VERSION=dev .
```

The Dockerfile has a build stage (`go vet`, `go build` with `VERSION`) and a
runtime stage (Debian + mail stack). See [architecture.md](architecture.md) §
Image and processes.

---

## Commits and release build

Commit on every meaningful step (a working sub-feature, a green build, end of a
phase) — not every file save, and not only at phase end. Minimum: one commit per
closed phase, plus intermediate commits for coherent sub-steps. Branch `main`
unless a separate branch is requested. Push / PR only on explicit request.

Commit messages end with
`Co-Authored-By: Claude <model> <noreply@anthropic.com>` for the model that did
the step (e.g. `Claude Sonnet 4.6`).

Every such step also updates [CHANGELOG.md](../CHANGELOG.md) under
`[Unreleased]` (Keep a Changelog). On an explicit version cut, rename
`[Unreleased]` to `[X.Y.Z] - date` and open a fresh empty `[Unreleased]`. Image
tag / push only on explicit request (see `release.yml`).

### Release image

The release image is published **only on tag** `vX.Y.Z` (not on every push to
`main`). The tag is the single source of version: it drives the image tag and
`-ldflags` in the binaries so they cannot drift apart.

**Steps (on explicit request):**

1. Close `[Unreleased]` in [CHANGELOG.md](../CHANGELOG.md).
2. Create and push git tag `vX.Y.Z`.
3. Workflow [release.yml](../.github/workflows/release.yml) builds, e2e-gates,
   and publishes `ghcr.io/mixeme/selfpost:X.Y.Z`.
4. Update the pinned tag in
   [deploy/docker-compose.yml](../deploy/docker-compose.yml) in the **same**
   commit as the tag (see [roadmap.md](roadmap.md) § «v1.x — documentation and
   deploy tail»).

Ordinary commits **do not** publish an image.

---

## Phase closure

Before `/clear` at the end of a finished step:

1. Update [roadmap.md](roadmap.md) (and the active plan checklist, if any): what
   changed, what is next.
2. Check the applicable «Done when…» criteria.
3. Append [CHANGELOG.md](../CHANGELOG.md) under `[Unreleased]`.
4. Make the final commit for the step (when the user asks for a commit).

---

## Testing

### Static analysis and unit tests

Main module (`go test ./...`); e2e is a separate module — see below.

```sh
make vet          # go vet ./...
make test         # go test ./...
```

Or directly:

```sh
gofmt -l .        # in CI — fails on drift
go vet ./...
go test ./...
```

### Env documentation regression

`go test ./cmd/panel -run TestLoadConfig` — every new `loadConfig` key must
appear in the env lists in [guide.md](guide.md)
([cmd/panel/envdoc_test.go](../cmd/panel/envdoc_test.go)).

### End-to-end (container suite)

Separate Go module `test/e2e/`; **not** included in the main module's
`go test ./...`.

```sh
make e2e
# same as: cd test/e2e && go test -v -timeout 20m ./...
```

**Stack:** [deploy/docker-compose.yml](../deploy/docker-compose.yml) +
[test/e2e/compose.override.yml](../test/e2e/compose.override.yml) — same
`cap_drop`/`cap_add`/`no-new-privileges` as production. Override: high ports
(`20465`/`20587`/`20080`), test hostname, self-signed TLS,
`PANEL_COOKIE_SECURE=false`, isolated compose project. Mail is hermetic:
CoreDNS (fake zone) + Postfix `smtp-sink` as sink-MX; DKIM TXT is scraped from
the panel and published into the zone — the test verifies the records an
operator would actually use.

**Coverage (summary):** bootstrap → SMTP AUTH → delivery → DKIM verify →
send-log `queued → sent`; negatives (no AUTH, relay, sender/login mismatch,
L1/L2 limits, milter fail-open, bad `SELFPOST_HOSTNAME`, session survives
`docker restart`). Polling with timeouts only — no fixed `sleep`.

Requires **Docker + Compose v2** on the machine running the suite.

---

## CI

Workflows in [.github/workflows/](../.github/workflows/). What each job runs —
[§ Testing](#testing) above.

### `test.yml` — every push and PR to `main`

`gofmt -l` → `go vet ./...` → `go test ./...` (main module, no e2e).

### `release.yml` — push of tag `vX.Y.Z` or `workflow_dispatch`

```
prepare (version from tag)
  → build [matrix: ubuntu-latest / ubuntu-24.04-arm]
      → docker build --load (VERSION from tag)
      → e2e (test/e2e)
      → push ghcr.io/...:X.Y.Z-amd64 | X.Y.Z-arm64
  → merge
      → docker buildx imagetools create → unified manifest X.Y.Z
```

Native per-arch matrix (no QEMU): running the full Postfix/OpenDKIM stack under
emulation for e2e is impractical. E2e first, then push — the registry receives
the bytes that passed the gate.

A failed e2e **blocks** image publication.

---

## Documentation

History of deleted plans lives in git and [CHANGELOG.md](../CHANGELOG.md).
There is no `docs/archive/` directory.

### Documentation map

| Home | File |
|---|---|
| Operator install / quick start | [README.md](../README.md) |
| Operator guide | [guide.md](guide.md) |
| Product boundaries | [product.md](product.md) |
| As-built design | [architecture.md](architecture.md) |
| Development process (this file) | [development.md](development.md) |
| Security requirements and accepted risks | [security.md](security.md) |
| Internal roadmap (v1.x tail, 2.x) | [roadmap.md](roadmap.md) |
| Release history | [CHANGELOG.md](../CHANGELOG.md) |

`implementation-plan.md` remains only until the release cut (describes the
closed release gate); delete it in the release commit. Temporary
[v1.x-closure-plan.md](v1.x-closure-plan.md) goes away with that cut too.

### User-facing deliverables

| Artefact | Role |
|---|---|
| [README.md](../README.md) | Overview, requirements, quick start, docs index, reference deploy, licence |
| [guide.md](guide.md) | Proxy, env, DNS, IP warmup, operations, rate limiting, backup, ports, image tag |
| [LICENSE](../LICENSE) | AGPL-3.0 full text |
| [deploy/docker-compose.yml](../deploy/docker-compose.yml) + proxies | Apache + nginx/Caddy/Traefik under [deploy/](../deploy/) |
| [deploy/.env.example](../deploy/.env.example) | Public env template; full reference in [guide.md](guide.md) |
| [CHANGELOG.md](../CHANGELOG.md) | Keep a Changelog |

Out of scope for v1.x: `CONTRIBUTING.md`, man pages, a separate docs site
(candidates in [roadmap.md](roadmap.md)).

### Maintaining documentation

1. **Step rule:** a new or renamed env key, panel route, or observable mail-path
   behaviour ships together with [guide.md](guide.md) / `.env.example` and a
   CHANGELOG entry (see [§ Commits and release build](#commits-and-release-build)).
2. **Env regression:** [cmd/panel/envdoc_test.go](../cmd/panel/envdoc_test.go)
   fails on an undocumented `loadConfig` or build-script key.
3. **New gaps** go into [roadmap.md](roadmap.md) (or the active plan file), not
   silent drive-by edits.

### Verifying docs against code

Every claim in the docs has a **source of truth in the tree**; verify from code
to prose.

| Claim class | Source of truth |
|---|---|
| Env keys and defaults | `loadConfig` — [cmd/panel/main.go](../cmd/panel/main.go); `${VAR:-…}` in [build/](../build/) |
| Mail path | [build/postfix-config.sh](../build/postfix-config.sh) |
| Panel routes | [internal/web/web.go](../internal/web/web.go) |
| Backup / restore, domain export | [internal/backup/](../internal/backup/), [cmd/selfpost-backup/](../cmd/selfpost-backup/) |
| Sessions | [internal/store/sessions.go](../internal/store/sessions.go), [internal/web/session.go](../internal/web/session.go) |
| Log rotation, reload | [build/logrotate-mail.conf](../build/logrotate-mail.conf), [build/logrotate-loop.sh](../build/logrotate-loop.sh), [build/postfix-cert-reload.sh](../build/postfix-cert-reload.sh) |
| Deploy | [deploy/docker-compose.yml](../deploy/docker-compose.yml), [build/Dockerfile](../build/Dockerfile) |
| Operator checklist | [§ User-facing deliverables](#user-facing-deliverables); detail — [guide.md](guide.md) |
| Product / out of scope | [product.md](product.md) |
| As-built | [architecture.md](architecture.md) |
| Mandatory security | [security.md](security.md) |

Order: list what the code actually does → find it in [guide.md](guide.md) /
`architecture.md`. Before every tag, a short pass over this table — not a full
prose rewrite.