From 75207f0d1e442a0e2298ae5c4dce9493a2dbefb8 Mon Sep 17 00:00:00 2001 From: Mix Date: Tue, 22 Sep 2026 15:24:51 +0300 Subject: [PATCH] release: 1.0.0: Public 1.0 image; delivery log; docs and SECURITY.md (incl. 1.0.1) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit [1.0.0] Added: - A **Delivery log** on each delivery's page (`/deliveries/{id}`): the - A **History** block on the same page: the journal's two timestamps stated as Fixed: - Postfix config: copy TLS cert/key from the (often `:ro`, host-owned) mount - Postfix config: allow `maillog_file` under `/data` via - E2e: read `/data/setup-token` via `docker compose exec` (file is `0600` - Entrypoint: check `SELFPOST_HOSTNAME` before `/data` setup (so a bad identity - E2e gate: wait for host-published `/healthz` before panel setup, and stop … (3 more in CHANGELOG) Changed: - Document what `.spbk` and `.spde` stand for (SelfPost backup / SelfPost domain - Docs aligned with the code: setup URL is `/setup/` (README and - Full backups no longer carry `/data/log`. It is Postfix's raw log plus its - Monitoring screens (status, mail queue, system log, deliveries) use adaptive - Documentation package consolidated into `docs/development.md`: Documentation … (4 more in CHANGELOG) [1.0.1] A documentation and packaging release: no change to the mail path, the database, or the on-disk layout. Upgrading is a tag bump. Added: - `SECURITY.md` — how to report a vulnerability privately (GitHub private - [docs/plans/](docs/plans/) — one document per agreed extension: the optional Changed: - The panel's **Account** entry is now called **Settings** — nav link, page - The signed-in name in the panel's nav is now labelled `User:`, so it reads as - [docs/product.md](docs/product.md) reframes the future line: the inbound - [docs/security.md](docs/security.md) is now in English, matching the rest of - The README documentation table now points at `SECURITY.md` for reporting a … (2 more in CHANGELOG) Fixed: - The panel's static assets are served with a content ETag and Squashed CHANGELOG sections: [1.0.0], [1.0.1]. Full detail: CHANGELOG.md (1.0.0, 1.0.1). --- .cursor/rules/agent-rules.mdc | 24 + .github/workflows/release.yml | 66 ++- .gitignore | 2 +- CHANGELOG.md | 196 ++++++- Makefile | 2 +- README.md | 569 ++++++++------------ SECURITY.md | 81 +++ build/Dockerfile | 2 +- build/entrypoint.sh | 151 +++--- build/logrotate-loop.sh | 10 +- build/logrotate-mail.conf | 10 +- build/postfix-config.sh | 51 +- build/supervisord.conf | 2 +- cmd/panel/envdoc_test.go | 11 +- cmd/panel/main.go | 5 +- deploy/.env.example | 4 +- deploy/docker-compose.yml | 11 +- deploy/traefik/extract-cert.sh | 8 +- docs/architecture.md | 98 ++-- docs/archive/specification-v1.0.md | 448 --------------- docs/development.md | 417 ++++++++++---- docs/documentation-plan.md | 86 --- docs/guide.md | 385 +++++++++++++ docs/implementation-plan.md | 30 -- docs/plans/domain-admin.md | 72 +++ docs/plans/inbound-relay.md | 168 ++++++ docs/plans/web-split.md | 56 ++ docs/product.md | 12 +- docs/progress.md | 63 --- docs/roadmap.md | 245 +++------ docs/security.md | 284 +++++----- internal/app/sasl.go | 3 +- internal/app/service.go | 6 +- internal/backup/backup.go | 11 +- internal/backup/backup_test.go | 9 +- internal/domain/service.go | 8 +- internal/health/cert.go | 2 +- internal/logtail/logtail.go | 250 ++++++++- internal/logtail/logtail_test.go | 269 +++++++++ internal/milter/milter.go | 4 +- internal/milter/ratelimit.go | 4 +- internal/postfix/queue.go | 41 ++ internal/postfix/queue_test.go | 43 ++ internal/store/ratelimits.go | 10 +- internal/store/sendlog.go | 43 +- internal/store/sendlog_test.go | 52 ++ internal/web/handlers_account.go | 2 +- internal/web/handlers_apps.go | 2 +- internal/web/handlers_monitor.go | 166 +++++- internal/web/handlers_monitor_test.go | 119 ++++ internal/web/handlers_ratelimit.go | 12 +- internal/web/middleware.go | 2 +- internal/web/static.go | 59 ++ internal/web/static/panel.css | 106 +++- internal/web/static/panel.js | 100 +++- internal/web/static_test.go | 74 +++ internal/web/templates/account.html | 2 +- internal/web/templates/backup.html | 10 +- internal/web/templates/deliveries.html | 5 +- internal/web/templates/deliveries_rows.html | 4 +- internal/web/templates/delivery.html | 153 ++++-- internal/web/templates/domain_detail.html | 2 +- internal/web/templates/encrypt_fields.html | 7 +- internal/web/templates/layout.html | 4 +- internal/web/templates/mail_queue_body.html | 2 +- internal/web/templates/status.html | 2 +- internal/web/templates/status_body.html | 2 +- internal/web/templates/system_log_body.html | 2 +- internal/web/templates_test.go | 4 +- internal/web/web.go | 9 +- test/e2e/compose.override.yml | 4 + test/e2e/hostname_gate_test.go | 16 +- test/e2e/mail_helpers.go | 2 +- test/e2e/main_test.go | 55 +- test/e2e/negative_test.go | 2 +- test/e2e/panel_client.go | 48 +- test/e2e/process_check.go | 5 +- test/e2e/smtp_client.go | 4 +- test/e2e/stack.go | 8 + test/e2e/stage.go | 14 +- 80 files changed, 3670 insertions(+), 1662 deletions(-) create mode 100644 .cursor/rules/agent-rules.mdc create mode 100644 SECURITY.md delete mode 100644 docs/archive/specification-v1.0.md delete mode 100644 docs/documentation-plan.md create mode 100644 docs/guide.md delete mode 100644 docs/implementation-plan.md create mode 100644 docs/plans/domain-admin.md create mode 100644 docs/plans/inbound-relay.md create mode 100644 docs/plans/web-split.md delete mode 100644 docs/progress.md create mode 100644 internal/postfix/queue_test.go create mode 100644 internal/web/static.go create mode 100644 internal/web/static_test.go diff --git a/.cursor/rules/agent-rules.mdc b/.cursor/rules/agent-rules.mdc new file mode 100644 index 0000000..ce4ab34 --- /dev/null +++ b/.cursor/rules/agent-rules.mdc @@ -0,0 +1,24 @@ +--- +description: Agent rules for the SelfPost repository (formerly spec §12) +alwaysApply: true +--- + +# Agent rules + +1. **No git commits** without an explicit instruction in the prompt. +2. After Go changes: `go build`, `go vet`; fix all issues. Run `go test` when + tests exist. +3. Before calling a container task done: the image builds and the container starts. +4. Iterate: minimal skeleton first, then features. +5. Security requirements — `docs/security.md`; implement with the feature, not deferred. +6. Do not implement out-of-scope items (`docs/product.md`); do not change fixed + assumptions without agreement. +7. For large tasks: propose a plan before coding unless the user already approved one. +8. Check licence compatibility of new Go dependencies (permissive or GPL-family for + AGPL-3.0). + +Model routing — `docs/development.md` § «Model routing». Pre-release security +**review** (not authorship) — Fable. + +Commit protocol, CHANGELOG, and phase closure — `docs/development.md` § +«Commits and release build» and «Phase closure». diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index b64c5c9..0e2bb58 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -7,7 +7,7 @@ name: release # can never drift apart (the invariant restore's version check in spec 7.5.A # depends on). # -# Native per-architecture builds (plan implementation-plan.md C.4), not qemu: +# Native per-architecture builds (see docs/development.md), not qemu: # running the full Postfix/OpenDKIM stack under emulation for the e2e gate # below is impractically slow. Each arch builds, e2e-gates and pushes its own # tag on its own native runner; a merge job then combines them into the one @@ -68,7 +68,7 @@ jobs: go-version: "1.26" cache-dependency-path: test/e2e/go.sum - - name: e2e (gates publishing — see docs/implementation-plan.md C.4) + - name: e2e (gates publishing — see docs/development.md) run: cd test/e2e && go test -v -timeout 20m ./... - name: Log in to ghcr.io @@ -79,9 +79,35 @@ jobs: password: ${{ secrets.GITHUB_TOKEN }} - name: Push per-arch tag + # GHCR sometimes rejects the final manifest with "unknown blob" after + # every layer reports Pushed (registry race / overwrite of the same + # version tag). Retries are idempotent for identical local bytes. run: | - docker tag selfpost:e2e "ghcr.io/${{ github.repository }}:${{ needs.prepare.outputs.version }}-${{ matrix.arch }}" - docker push "ghcr.io/${{ github.repository }}:${{ needs.prepare.outputs.version }}-${{ matrix.arch }}" + set -euo pipefail + image="ghcr.io/${{ github.repository }}:${{ needs.prepare.outputs.version }}-${{ matrix.arch }}" + docker tag selfpost:e2e "$image" + attempt=1 + max=4 + backoff=15 + while true; do + set +e + out=$(docker push "$image" 2>&1) + rc=$? + set -e + if [ "$rc" -eq 0 ]; then + printf '%s\n' "$out" + exit 0 + fi + printf '%s\n' "$out" >&2 + if [ "$attempt" -ge "$max" ] || ! grep -qiE 'unknown blob|blob unknown|blob upload invalid|manifest unknown|received unexpected HTTP status: 5[0-9]{2}|429 Too Many Requests|temporarily unavailable' <<<"$out"; then + echo "::error::docker push failed for ${image} (attempt ${attempt}/${max}, exit ${rc})" >&2 + exit "$rc" + fi + echo "::warning::transient GHCR error pushing ${image} (attempt ${attempt}/${max}); retry in ${backoff}s" >&2 + sleep "$backoff" + attempt=$((attempt + 1)) + backoff=$((backoff * 2)) + done merge: needs: [prepare, build] @@ -97,8 +123,32 @@ jobs: password: ${{ secrets.GITHUB_TOKEN }} - name: Merge per-arch tags into the version manifest + # Same GHCR "unknown blob" class of failure can hit imagetools create + # when assembling the multi-arch version tag. run: | - docker buildx imagetools create \ - -t "ghcr.io/${{ github.repository }}:${{ needs.prepare.outputs.version }}" \ - "ghcr.io/${{ github.repository }}:${{ needs.prepare.outputs.version }}-amd64" \ - "ghcr.io/${{ github.repository }}:${{ needs.prepare.outputs.version }}-arm64" + set -euo pipefail + version_tag="ghcr.io/${{ github.repository }}:${{ needs.prepare.outputs.version }}" + amd64="ghcr.io/${{ github.repository }}:${{ needs.prepare.outputs.version }}-amd64" + arm64="ghcr.io/${{ github.repository }}:${{ needs.prepare.outputs.version }}-arm64" + attempt=1 + max=4 + backoff=15 + while true; do + set +e + out=$(docker buildx imagetools create -t "$version_tag" "$amd64" "$arm64" 2>&1) + rc=$? + set -e + if [ "$rc" -eq 0 ]; then + printf '%s\n' "$out" + exit 0 + fi + printf '%s\n' "$out" >&2 + if [ "$attempt" -ge "$max" ] || ! grep -qiE 'unknown blob|blob unknown|blob upload invalid|manifest unknown|received unexpected HTTP status: 5[0-9]{2}|429 Too Many Requests|temporarily unavailable' <<<"$out"; then + echo "::error::imagetools create failed for ${version_tag} (attempt ${attempt}/${max}, exit ${rc})" >&2 + exit "$rc" + fi + echo "::warning::transient GHCR error merging ${version_tag} (attempt ${attempt}/${max}); retry in ${backoff}s" >&2 + sleep "$backoff" + attempt=$((attempt + 1)) + backoff=$((backoff * 2)) + done diff --git a/.gitignore b/.gitignore index b204197..e8ba910 100644 --- a/.gitignore +++ b/.gitignore @@ -1,7 +1,7 @@ # Development-only material (secrets, server access notes) — never published. dev/ -# Build artifacts (binaries are built on the dev server, not committed). +# Build artifacts (not committed). /bin/ *.exe diff --git a/CHANGELOG.md b/CHANGELOG.md index 85713f1..280644d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,196 @@ Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); version ## [Unreleased] +## [1.0.1] - 2026-08-09 + +A documentation and packaging release: no change to the mail path, the +database, or the on-disk layout. Upgrading is a tag bump. + +### Added + +- `SECURITY.md` — how to report a vulnerability privately (GitHub private + vulnerability reporting, `public@mixeme.ru` as fallback), which releases get + fixes, and what is in and out of scope for a relay. No response time is + promised. Without it a finder's default move is a public issue, which + discloses a relay flaw to everyone the moment it is filed. +- [docs/plans/](docs/plans/) — one document per agreed extension: the optional + inbound relay, the domain-admin role, and splitting the oversized `web` + package. Each states scope, open questions, and what has to be true before + coding starts. [docs/roadmap.md](docs/roadmap.md) is restructured around them + as a 1.x+ tracker instead of a 2.x wishlist, and now says how to read it from + outside the project: nothing in it is a commitment, there are no dates, and + the stated order is a recommendation. + +### Changed + +- The panel's **Account** entry is now called **Settings** — nav link, page + heading, browser title, and the operator guide. The route stays `/account`, + so existing links and bookmarks are unaffected. +- The signed-in name in the panel's nav is now labelled `User:`, so it reads as + the current account rather than as a stray word above the Settings link. +- [docs/product.md](docs/product.md) reframes the future line: the inbound + relay and the domain-admin role are agreed **1.x+** extensions tracked in the + roadmap and the plans, with the inbound relay targeting a MINOR bump by + default and a 2.x major still possible pending implementation. Only items the + roadmap still marks *candidate* need explicit approval before coding. It + previously put the whole line behind a 2.x.x that nothing had committed to. +- [docs/security.md](docs/security.md) is now in English, matching the rest of + the published docs — it is linked from the README table and from + `SECURITY.md`, so a reader following either landed in Russian. Content is + unchanged: same requirements, same accepted risks, same ADR. The reviewing + model is no longer named in the text; the fact that a pre-release review ran, + and its date, stay. The roadmap and the plans are in English for the same + reason, and neither records the model assigned to an item any more. +- The README documentation table now points at `SECURITY.md` for reporting a + vulnerability, and the `docs/security.md` row is renamed *Security design* — + with two files a reader could reasonably call "security", the table said + which is which only by accident. The roadmap row no longer calls the file + internal and Russian, because it is neither. `development.md` lists + `SECURITY.md` among the user-facing deliverables. +- `docs/development.md` records the decision on authorship: SelfPost is written + by AI agents under a maintainer's direction and the project discloses that, + so the `Co-Authored-By` trailers, the model routing table, and the agent + rules file all stay. Written down to settle the question rather than have it + reopened at each release. +- The two remaining Russian source comments are in English: + `deploy/traefik/extract-cert.sh` (quote from spec 10.3) and + `internal/app/sasl.go`, where the quotation from the closed plan is dropped + rather than translated — rendered in English it restated the sentence it was + attached to. The Cyrillic that remains is test data, where it is the point. + +### Fixed + +- The panel's static assets are served with a content ETag and + `Cache-Control: no-cache`. They are embedded in the binary, so their + modification times are the zero value and no `Last-Modified` was sent; with + no validator at all the browser was free to guess how long to keep them, + which is why a tab kept showing the previous favicon after the new mark + shipped. Each asset is now hashed once at startup, so an unchanged one costs + a bodyless 304 and a changed one is picked up on the next load. A browser + that cached an asset *before* this release still has nothing to revalidate + against, so that one copy has to be cleared by hand. + +## [1.0.0] - 2026-08-09 + +### Added + +- A **Delivery log** on each delivery's page (`/deliveries/{id}`): the + `mail.log` lines Postfix wrote about that message, oldest first — the + connection to the receiving server, its reply, and the status that reply was + filed as. It is a table of two columns, when and what, so the seconds between + the connection and the reply line up down one edge; the timestamp is the + log's own wall clock without its microseconds and offset, and a line whose + head is not a timestamp keeps its whole text under *Message* + (`logtail.SplitTimestamp`, which reads both postlogd's format and syslog's). + The queue id was printed on this page as something to go and search + the system log for by hand; the search is done for the operator instead + (`logtail.QueueLines`). The read is a bounded tail of the current log and + matches only lines carrying this message's queue id, anchored so a shorter id + is not found inside a longer one. Send-log rows outlive `mail.log` — retention + is ninety days, rotation keeps fourteen files — so a message with no lines + left says so rather than reporting a failure. +- A **History** block on the same page: the journal's two timestamps stated as + the steps they stand for — accepted and queued, then delivered, deferred, + bounced, or refused before queueing — each with the status it reached in the + panel's own ok/warn/error/unknown badge vocabulary. A message still queued + shows the delivery report it is waiting for as a step that has not happened. + +### Fixed + +- Postfix config: copy TLS cert/key from the (often `:ro`, host-owned) mount + into `/etc/postfix/tls-internal` as `root:root` before `postconf` / + `postfix check`. Bind-mounted keys owned by the CI/host UID made + `postfix check` fail and the container exit before supervisord started. +- Postfix config: allow `maillog_file` under `/data` via + `maillog_file_prefixes=/var,/dev/stdout,/data`, and pin the `postlog` + master.cf service. After mail.log moved to `/data/log`, `postfix check` + fatally rejected the path (default prefixes are only `/var` and + `/dev/stdout`) and often left stderr/mail.log empty. +- E2e: read `/data/setup-token` via `docker compose exec` (file is `0600` + panel-owned; host `ReadFile` got permission denied on CI). Reclaim `/data` + ownership before TempDir/stage cleanup so panel/postfix UIDs do not fail + Go's `RemoveAll`. +- Entrypoint: check `SELFPOST_HOSTNAME` before `/data` setup (so a bad identity + fails with the FATAL text, not an earlier `set -e` abort), and `chmod 755 + /data` after chown so OpenDKIM/Postfix can traverse bind mounts that arrive + as mode `0700` (Go `TempDir`, some host umasks) — otherwise KeyTable is + unreachable and the container crash-loops. E2e stand uses `restart: "no"` and + surfaces selfpost logs from the supervisor readiness check. +- E2e gate: wait for host-published `/healthz` before panel setup, and stop + ordered `TestE2E` subtests after a failure so a nil panel client cannot panic + and mask the real error (release CI on both amd64 and arm64). +- Release CI: retry `docker push` / `imagetools create` on transient GHCR + `unknown blob` (and similar) errors after layers already uploaded. +- A send-log row could stay `queued` forever after the container was recreated. + `mail.log` moved from the ephemeral `/var/log` into the data volume + (`/data/log/mail.log`, `./data/log/` on the host), so the delivery lines that + resolve a queued row now outlive the container the same way the journal does. + `postlogd` writes the file as user `postfix` and the unprivileged panel reads + it through the shared `selfpost` group (directory `2750`, file `0640`, + re-normalised on every start); logrotate creates each new file the same way. + Existing deployments need no action beyond the upgrade — the directory is + created on first start — but the log written by the previous image is gone + with its container, and the tailer starts the new file from its end. +- A row whose delivery lines are gone for good is no longer left `queued` + indefinitely: every five minutes the tailer compares rows still queued from + more than two minutes ago against `postqueue -p`, and marks `bounced` those + whose message Postfix no longer holds — it will never report on them again. + The sweep waits until the tailer has read the log to its end (on a restart the + log itself holds the answer) and does nothing at all if the queue cannot be + listed, so a message merely in flight, or a `postqueue` that fails, never + closes a row. + +### Changed + +- Document what `.spbk` and `.spde` stand for (SelfPost backup / SelfPost domain + export) in the operator guide, security notes, architecture, and the Backup / + Export panel copy. +- Docs aligned with the code: setup URL is `/setup/` (README and + guide; local trial rewrites the printed `https:///…` link to + `http://127.0.0.1:8080/…`); domain import uses the file extension / magic + bytes for the password field, not an "encrypted" checkbox; architecture + layering and route table match `web`→`store` and `POST /domains/import`; + OpenDKIM drops to `opendkim` via `UserID`; guide drops the archived + "spec 7.5" pointer, clarifies `POSTFIX_SENDER_LOGIN_MAPS` vs panel writes, + and states logrotate keeps 14 daily files. Intermediate CHANGELOG cuts from + before the published `1.0.0` image are called out in the guide and + `development.md`. Roadmap points at CHANGELOG `[0.5.0]` Security and + refreshed `internal/web` size / symbol links. +- Full backups no longer carry `/data/log`. It is Postfix's raw log plus its + fourteen rotated copies — diagnostic output rather than state to restore, and + otherwise by far the largest thing in the archive. +- Monitoring screens (status, mail queue, system log, deliveries) use adaptive + HTMX polling: 5 s while the operator is active on the page, 30 s when the tab + is visible but idle, and no requests while the tab is hidden. Scheduling + lives in `panel.js` (`data-poll` markers) instead of `hx-trigger="every …"`, + which would need `unsafe-eval` under the panel's CSP. +- Documentation package consolidated into `docs/development.md`: Documentation + map, user-facing deliverables, maintenance rules, and code-to-prose + verification table (from closed `documentation-plan.md`); resuming work, + model routing, commits, and phase closure (from closed `progress.md`). + `docs/archive/` removed — history is git + CHANGELOG. README Documentation + index lists operator docs plus the internal roadmap. Agent rules point at + `development.md`. +- The delivery page is laid out in two columns: what the journal recorded on + the left, what happened to the message on the right, and the delivery log at + full width under both. The facts the page used to stack one per line — domain, + application, queue id, journal id and the two timestamps — are a grid of tiles + instead, since a page of mostly empty rows was what the full-width stack came + to for six short values. The subject heads the page and the sender, recipient + and outcome are the line under it, so what the message was and how it ended + are both on the first line. The page takes the whole column rather than the + reading measure, as the other three monitoring pages already did. +- `docs/development.md` restructured into stack, dependencies, build, release, + testing, and CI; agent rules moved to `.cursor/rules/agent-rules.mdc`; + dev-host-specific workflow and `example.com` references removed from docs. +- `docs/development.md` and `.cursor/rules/agent-rules.mdc` translated to + English; `roadmap.md` remains Russian (internal tracker). +- Deploy pin and local-trial image tag set to `ghcr.io/mixeme/selfpost:1.0.0` + (compose and git tag `v1.0.0` cut together). Retired + `docs/implementation-plan.md` and `docs/v1.x-closure-plan.md`; Makefile, + `release.yml`, and e2e comments point at `docs/development.md`. Roadmap + v1.x documentation/deploy tail closed. + ## [0.6.0] - 2026-08-08 ### Added @@ -244,7 +434,7 @@ Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); version null recipient of a double bounce, `orig_to=` alongside `to=`, an unrecognised status word, a capitalised one, and a cleanup line. - docs: README *Encrypting a backup or export*; `docs/security.md` § - *Резервная копия и экспорт домена* + accepted risk (encryption is opt-in); + *Backup and domain export* + accepted risk (encryption is opt-in); `docs/architecture.md` persistence § envelope summary. - docs: `docs/roadmap.md` v1.x tail — retire `implementation-plan.md` in the release commit (move to `docs/archive/`, retarget its references in README, @@ -582,5 +772,5 @@ of `docs/implementation-plan.md`). - Security pass against spec 7.6 (exec safety, config-write sanitization, server-side validation, rate limiting, session/cookie hardening, output escaping, non-root panel) — full compliance, no code changes required. -- Live production deployment on `selfpost.example.com` with a real Let's - Encrypt certificate; end-to-end delivery confirmed (DKIM pass, SPF pass). +- Live production deployment with a real Let's Encrypt certificate; + end-to-end delivery confirmed (DKIM pass, SPF pass). diff --git a/Makefile b/Makefile index 491605c..f8ab028 100644 --- a/Makefile +++ b/Makefile @@ -30,7 +30,7 @@ test: clean: rm -rf bin -# Hermetic container e2e (plan implementation-plan.md C.4): separate Go module +# Hermetic container e2e (see docs/development.md): separate Go module # under test/e2e so its test-only dependencies (DKIM verification) never enter # this module's build graph. Builds the image fresh from this checkout, brings # up deploy/docker-compose.yml plus a test-only override on high ports and an diff --git a/README.md b/README.md index cf2c50b..6b987df 100644 --- a/README.md +++ b/README.md @@ -5,379 +5,260 @@ # SelfPost Self-hosted outbound SMTP relay with a web control panel, shipped as a single -Docker image. Postfix + OpenDKIM + a small Go panel run together under -`supervisord`; the panel manages multiple sending domains, per-domain DKIM keys -and SASL-authenticated applications bound to their domain. +Docker image. Postfix, OpenDKIM, and a small Go panel run together under +`supervisord`; you configure domains, DKIM keys, and SASL applications once, +then point your apps at the SMTP endpoint. -SelfPost sends mail straight to the internet from your own IP, with DKIM -signing, and is configured once through the panel. It is **outbound only** — it -does not receive mail, provide mailboxes, or offer webmail. +SelfPost sends mail straight to the internet from **your own IP**, with per-domain +DKIM signing. It is **outbound only** — no inbound mail, mailboxes, or webmail. -> **Status: v1.0 implemented; pre-release polish in progress.** Open v1.x -> questions: [docs/implementation-plan.md](docs/implementation-plan.md). -> Documentation pass before the next tag: -> [docs/documentation-plan.md](docs/documentation-plan.md). Accepted security -> trade-offs: [docs/security.md](docs/security.md). +**For:** operators who run their own VPS or home server and want a simple relay +they control, without a third-party SMTP provider. -## Requirements (site checklist) +**Key properties:** one container, multi-domain DKIM, SASL per application, +send log and DNS checks in the panel, encrypted backups. -Providing these is the operator's job, not a feature of SelfPost — the panel -can't fix a blocked port or a missing PTR record for you. +## Features -- [ ] A static IP address. +- Outbound SMTP (465/smtps; optional 587 submission) with per-domain DKIM signing +- Web panel — domains, applications, deliveries, mail queue, system log, backup +- Multi-domain relay — each SASL application is bound to one sending domain +- DNS status checks (PTR, SPF, DKIM, DMARC) with in-panel re-check +- Two-level rate limiting — IP backstop (Postfix) and per-domain/per-app limits +- Full-server backup and single-domain export/import (optional password encryption) +- Single Docker image; data in a `./data` bind mount + +## Documentation + +| Document | Contents | +|---|---| +| [**Operator guide**](docs/guide.md) | Reverse proxy, environment variables, DNS, IP warmup, panel operations, rate limiting, backup/restore, ports, image tag | +| [Product boundaries](docs/product.md) | Purpose, deployment assumptions, out-of-scope items, multi-domain model | +| [Architecture](docs/architecture.md) | As-built technical design | +| [Security design](docs/security.md) | Mandatory requirements, accepted risks, the CSRF ADR | +| [Development](docs/development.md) | Building, testing, docs rules, model routing, commits | +| [Roadmap](docs/roadmap.md) | Open work (1.x+) — direction, not commitments | +| [CHANGELOG](CHANGELOG.md) | Release history | + +Found a vulnerability? Do not open an issue — [SECURITY.md](SECURITY.md) has +the private reporting channel and the scope. + +Repository: — source, issues, releases, and +the `ghcr.io/mixeme/selfpost` image. + +## Requirements + +Providing these is the operator's job — SelfPost cannot fix a blocked port or a +missing PTR record for you. Details: [Operator guide](docs/guide.md). + +### Platform + +- Docker + Compose v2 on the host +- A reverse proxy in front of the panel (SelfPost never terminates HTTPS itself) +- Rough sizing: **1 vCPU**, **512 MB–1 GB RAM**, **8–10 GB disk** (send log and + rotated `mail.log` are the main growth drivers; both sit in the `./data` + volume, and both are capped — 90 days and 14 files by default) + +### Network and IP + +- [ ] Static IP address - [ ] Outbound TCP port 25 unblocked (many consumer/cloud hosts block it by - default — check with your provider before anything else). -- [ ] PTR/rDNS for that IP set to your mail hostname (see [DNS setup](#dns-setup)). -- [ ] Reasonable starting IP reputation — a fresh IP still needs [warmup](#ip-warmup). -- [ ] A reverse proxy in front of the panel (see [Reverse proxy](#reverse-proxy-mandatory)) — SelfPost never terminates HTTPS itself. -- [ ] Docker + Compose v2 on the host. + default) +- [ ] PTR/rDNS for that IP pointing at your mail hostname (`SELFPOST_HOSTNAME`) +- [ ] Reasonable starting IP reputation — a fresh IP still needs + [warmup](docs/guide.md#ip-warmup) + +### Per sending domain + +For every domain you add in the panel: + +- [ ] SPF TXT record authorizing this server +- [ ] DKIM TXT record (value shown on the domain page) +- [ ] DMARC `_dmarc` TXT record + +See [DNS setup](docs/guide.md#dns-setup) in the operator guide. ## Quick start +> **First boot — create the admin account.** On a fresh container SelfPost prints +> a **one-time setup URL** (valid ten minutes). Open it in a browser to choose +> the administrator username and password. Until you do, the panel has no login. +> Production deploy: [step 3](#3-start-selfpost). + +One container, panel at `http://127.0.0.1:8080` — no reverse proxy, no TLS +files, no compose files. Good for clicking through the UI on your machine; +outbound mail will not reach the real internet without DNS, PTR, and port 25. + ```sh -mkdir -p selfpost && cd selfpost +docker run --rm -d --name selfpost-try \ + -p 127.0.0.1:8080:8080 \ + -e SELFPOST_HOSTNAME=mail.local.test \ + -e PANEL_COOKIE_SECURE=false \ + -v selfpost-try-data:/data \ + ghcr.io/mixeme/selfpost:1.0.1 +``` + +**Get the setup URL** (pick one): + +```sh +docker logs selfpost-try 2>&1 | grep -m1 'http' +``` + +```sh +docker exec selfpost-try cat /data/setup-token +``` + +The printed URL is `https://mail.local.test/setup/`. For this local +trial rewrite it to `http://127.0.0.1:8080/setup/` (same path token; +`PANEL_COOKIE_SECURE=false` so the cookie works over plain HTTP). Open it +before it expires. + +When finished: + +```sh +docker rm -f selfpost-try && docker volume rm selfpost-try-data +``` + +More detail (limitations, optional throwaway TLS for local SMTP): [Local +trial](docs/guide.md#local-trial) in the operator guide. + +## Reference deploy + +Production layout: one `docker-compose.yml`, a `.env`, persistent `./data`, and +TLS PEM files at `./certs` (read by Postfix on 465/587). The panel is reached +only through a reverse proxy on 443 — port 8080 is bound to localhost in the +default compose file. + +| Artefact | Path | +|---|---| +| Compose file (fixed image tag) | [deploy/docker-compose.yml](deploy/docker-compose.yml) | +| Environment template | [deploy/.env.example](deploy/.env.example) | +| Apache vhost (recommended) | [deploy/apache/selfpost-vhost.conf](deploy/apache/selfpost-vhost.conf) | +| nginx | [deploy/nginx/](deploy/nginx/) | +| Caddy | [deploy/caddy/](deploy/caddy/) | +| Traefik | [deploy/traefik/](deploy/traefik/) | + +### 1. Fetch the base files + +```sh +mkdir -p selfpost/data selfpost/certs && cd selfpost curl -O https://raw.githubusercontent.com/mixeme/selfpost/main/deploy/docker-compose.yml curl -O https://raw.githubusercontent.com/mixeme/selfpost/main/deploy/.env.example -mv .env.example .env # then edit SELFPOST_HOSTNAME etc. +cp .env.example .env +``` + +Edit `.env` — at minimum set `SELFPOST_HOSTNAME` to your mail hostname (bare +FQDN, e.g. `mail.example.com`). It must match the PTR record you request from +your provider and the certificate your proxy will obtain. + +### 2. Reverse proxy and TLS + +Pick one proxy. In every case the proxy terminates HTTPS for the panel; the +same certificate must end up under `./certs` as `fullchain.pem` and +`privkey.pem` so Postfix can serve it on 465 (and 587 if enabled). The proxy +must **pass the original `Host` header** — details and rationale: +[Reverse proxy](docs/guide.md#reverse-proxy-mandatory). + +**Apache (recommended, on the host).** Install Apache with `ssl`, `proxy`, and +`proxy_http` enabled. Copy +[deploy/apache/selfpost-vhost.conf](deploy/apache/selfpost-vhost.conf) into your +vhost directory, replace `mail.example.com` with your hostname, enable the site, +then issue a certificate: + +```sh +sudo certbot --apache -d mail.example.com +``` + +Point `./certs` at the PEM files certbot wrote (symlink is fine): + +```sh +ln -s /etc/letsencrypt/live/mail.example.com certs +``` + +**nginx (containerised).** From the `deploy/` directory, merge the nginx +fragment and issue the first certificate before nginx can serve HTTPS: + +```sh +docker compose -f docker-compose.yml -f nginx/docker-compose.nginx.yml \ + run --rm certbot certonly --webroot -w /var/www/certbot \ + -d mail.example.com --email you@example.com --agree-tos --no-eff-email + +docker compose -f docker-compose.yml -f nginx/docker-compose.nginx.yml up -d +``` + +Edit [deploy/nginx/nginx.conf.example](deploy/nginx/nginx.conf.example) and +replace `mail.example.com` first. The fragment bind-mounts certbot's output into +both nginx and SelfPost. + +**Caddy (containerised, automatic ACME).** Edit +[deploy/caddy/Caddyfile](deploy/caddy/Caddyfile) and the `` placeholders +in [deploy/caddy/docker-compose.caddy.yml](deploy/caddy/docker-compose.caddy.yml), +then: + +```sh +docker compose -f docker-compose.yml -f caddy/docker-compose.caddy.yml up -d +``` + +Verify Caddy's on-disk cert path for your version before relying on the +default mount — see the comment at the top of the Caddy compose fragment. + +**Traefik (containerised).** Edit the `Host(...)` label and ACME email in +[deploy/traefik/docker-compose.traefik.yml](deploy/traefik/docker-compose.traefik.yml), +start the stack, then extract PEM files for Postfix whenever Traefik issues or +renews a certificate: + +```sh +docker compose -f docker-compose.yml -f traefik/docker-compose.traefik.yml up -d +./traefik/extract-cert.sh ./traefik/letsencrypt/acme.json mail.example.com ./traefik/extracted-certs +``` + +Schedule `extract-cert.sh` (cron or a timer) alongside Traefik's renewals. + +### 3. Start SelfPost + +If you used Apache on the host (step 2, first option), start only the base +compose file from your `selfpost/` directory: + +```sh docker compose up -d ``` -`SELFPOST_HOSTNAME` is required — the container exits immediately with an -explanatory error if it's unset, since it doubles as the Postfix HELO name, -the SASL realm, and must match both the PTR record and the certificate -CN/SAN. +The nginx/Caddy/Traefik fragments from step 2 already include `docker compose up +-d` — skip this if you ran one of those. -This starts SelfPost alone; it assumes Apache is already installed on the host -as the reverse proxy (see below) and expects certificates at `./certs`. The -first log line (`docker compose logs -f`) prints the one-time setup link — -open it to create the admin account. That username and password can be changed -later from the panel's *Account* page (changing the password signs out every -other session). - -The same link is also written to `/data/setup-token` inside the container — -`./data/setup-token` on the host, mode `0600` — and deleted the moment setup -completes. If this host ships its container logs to a central aggregator, -prefer the file: the link is a bearer token valid for ten minutes, and reading -it this way keeps it out of the log pipeline (and out of whatever retains it -afterwards) entirely. +**Get the setup URL** — open it in a browser to create the admin account +([first boot](#quick-start)): ```sh -docker compose exec selfpost cat /data/setup-token +docker compose logs selfpost 2>&1 | grep -m1 'http' ``` -## Reverse proxy (mandatory) - -SelfPost's panel speaks plain HTTP and never terminates TLS itself — a reverse -proxy in front of it is not optional. The proxy is also the project's only -source of TLS certificates: whatever it obtains via ACME/Let's Encrypt gets -bind-mounted **read-only** into the SelfPost container, and Postfix uses those -same PEM files for TLS on 465 (and 587, if enabled). If the panel and the mail -service share one hostname — the common case — it's genuinely one certificate -serving both. - -SelfPost isn't tied to a specific proxy; pick whichever fits your host: - -| Proxy | Where certs live | Fragment | -|---|---|---| -| **Apache** (default/recommended) | Host disk, via the certbot Apache plugin — PEM files ready to bind-mount, no extraction step. | [deploy/apache/selfpost-vhost.conf](deploy/apache/selfpost-vhost.conf) | -| nginx | Host disk, via a certbot sidecar container — same PEM-ready shape as Apache. | [deploy/nginx/](deploy/nginx/) | -| Caddy | Automatic ACME, zero extra containers — simplest, but its on-disk cert path is versioned internal layout, not a stable API; verify it for the Caddy version you run. | [deploy/caddy/](deploy/caddy/) | -| Traefik | Bundled inside `acme.json` — needs a small extraction script to produce standalone PEM files. | [deploy/traefik/](deploy/traefik/) | - -Apache is the recommended default because the certbot Apache plugin already -writes plain `fullchain.pem`/`privkey.pem` files to a predictable path with no -extra moving parts between "certificate issued" and "Postfix can read it." - -**The proxy needs no security configuration of its own.** The panel emits its -own `Content-Security-Policy`, `Strict-Transport-Security`, `X-Frame-Options`, -`X-Content-Type-Options` and `Referrer-Policy` — deliberately, so the part -that's easy to get wrong lives in the service rather than in a config file -somebody edits under pressure. There is exactly one thing the proxy must do: -**pass the original `Host` header through**. All four fragments above already -do (Apache `ProxyPreserveHost On`, nginx `proxy_set_header Host $host`, Caddy -and Traefik by default). A proxy that rewrites `Host` instead makes the panel -reject every form submission as cross-origin — the log says so explicitly, -printing the `Origin` and `Host` it compared. - -## Environment variables - -Copy [deploy/.env.example](deploy/.env.example) to `.env` next to your -`docker-compose.yml`. The table below lists every variable an operator is -expected to set; defaults match the code exactly. - -| Variable | Purpose | Default | Set in | -|---|---|---|---| -| `SELFPOST_HOSTNAME` | Mail-server identity: Postfix HELO/EHLO, SASL realm, certificate CN/SAN, and the hostname the PTR check expects. Bare FQDN only — no scheme or port. | *(required)* | `.env` | -| `SUBMISSION_ENABLE` | When `true`, also listen on port 587 with STARTTLS (RFC 6409 submission) alongside the primary 465/smtps listener. | `false` | `.env` | -| `RATE_LIMIT_MESSAGES_PER_IP` | Level-1 backstop: maximum messages one client IP may submit per window (Postfix `smtpd_client_message_rate_limit`). See [Rate limiting](#rate-limiting). | `100` | `.env` | -| `RATE_LIMIT_WINDOW_SECONDS` | Level-1 window length in seconds (Postfix `anvil_rate_time_unit`). | `3600` | `.env` | -| `SEND_LOG_RETENTION_DAYS` | Days of send-log history kept before the background sweep deletes rows — the main driver of `/data` growth over time. | `90` | `.env` | -| `PANEL_SESSION_IDLE_DAYS` | Sliding idle timeout for the panel login session, in days. There is no absolute cap: an admin who keeps coming back stays signed in indefinitely. | `7` | `.env` | -| `SELFPOST_DNS_RESOLVERS` | Comma-separated recursive resolvers the panel's PTR/SPF/DKIM/DMARC checks query directly (so they report what the internet sees, not what this host's stub resolver synthesises). | `1.1.1.1:53`, `8.8.8.8:53`, `9.9.9.9:53` when unset | `.env` | -| `TRUSTED_PROXY_CIDR` | Comma-separated CIDRs (bare IPs allowed) of reverse proxies allowed to supply `X-Forwarded-For` for login, setup, and account-change rate-limiting. **Leave unset unless you know the exact address of your reverse proxy.** A wrong value lets a client spoof its rate-limit key by sending a forged `X-Forwarded-For` header — the panel trusts the last hop only when the TCP peer matches one of these CIDRs. Behind the default Apache host-network setup this is typically the Docker bridge gateway, e.g. `172.18.0.1`. | *(empty — XFF ignored)* | `.env` | - -TLS certificate paths (`TLS_CERT_FILE`, `TLS_KEY_FILE`) are fixed in -[deploy/docker-compose.yml](deploy/docker-compose.yml) to match the `./certs` -bind mount — configure the mount, not these variables. - -**Internal variables (not part of the operator interface).** The following are -read by the panel or startup scripts but are not meant to be changed in a -normal deployment; documenting them here avoids treating accidental overrides as -supported configuration: - -- **Panel paths and tuning:** `SELFPOST_DATA_DIR` (`/data`), `SELFPOST_DB_PATH` - (`/data/selfpost.db`), `SELFPOST_SETUP_TOKEN_FILE` - (`/data/setup-token`), `PANEL_HTTP_ADDR` (`:8080`), - `JOURNAL_MILTER_SOCKET` (`/run/selfpost/journal.sock`), `MAIL_LOG` - (`/var/log/mail.log`), `PANEL_COOKIE_SECURE` (`true`), `OPENDKIM_SOCKET` - (`/run/opendkim/opendkim.sock`), `OPENDKIM_DIR` (`/data/opendkim`), - `DKIM_SELECTOR_DEFAULT` (`selfpost`), `SASL_DB_PATH` - (`/data/sasl/sasldb2`), `SASL_REALM` (defaults to `SELFPOST_HOSTNAME`), - `POSTFIX_DIR` (`/data/postfix`), `POSTFIX_SENDER_LOGIN_MAPS` - (`/data/postfix/sender_login_maps`). -- **Milter and Postfix startup:** `MILTER_CONNECT_TIMEOUT` (`15s`), - `MILTER_COMMAND_TIMEOUT` (`15s`), `MILTER_CONTENT_TIMEOUT` (`30s`), - `MILTER_WAIT_TIMEOUT` (`30` seconds). -- **Background maintenance:** `TLS_RELOAD_INTERVAL_SECONDS` (`86400` — daily - `postfix reload` to pick up renewed certificates), - `LOGROTATE_INTERVAL_SECONDS` (`21600` — check `mail.log` rotation every six - hours; rotated logs are kept 14 days and each rotation triggers - `postfix reload`). - -## DNS setup - -Two different scopes — don't confuse them: - -**Server level (once, for the machine itself):** -- **PTR/rDNS** for the server's IP, pointing at its mail hostname. Most - receiving mail servers weigh this heavily; get it from whoever assigns the IP - (hosting provider's panel/support), not from your own DNS zone. - -**Domain level (for *every* sending domain you add in the panel):** -- **SPF** — a TXT record on the domain authorizing this server to send on its - behalf (e.g. `v=spf1 a mx ip4: -all`, adjusted to your setup). -- **DKIM** — a TXT record with the exact value the panel shows on that - domain's page (`domain page → DKIM TXT record`), one selector per domain. -- **DMARC** — a `_dmarc` TXT record (even a conservative `p=none` starts - building reporting/reputation history). - -Skipping any of the three per-domain records is the single most common reason -mail lands in spam even though SelfPost delivered it correctly — DKIM passing -doesn't help if SPF/DMARC are absent. **Whenever you add a new domain in the -panel, add its DNS records at the same time**, not later. - -The panel checks both scopes for you and tells you what is actually published: -the *Status* page verifies the server's hostname and its reverse record -(forward-confirmed reverse DNS), and each domain's page shows a *DNS status* -card comparing the published DKIM record against the key this server signs with, -plus the domain's SPF and DMARC records. Results are cached for a few minutes; -use *Re-check* right after publishing a record. The SPF check is deliberately -shallow — it looks for a mechanism that literally covers this server's address -and does not follow `include:` or `redirect=`, so a record that authorizes the -server through an include is reported as "cannot tell" rather than as a failure. - -## IP warmup - -A brand-new IP has no sending history, so receiving servers are cautious with -it regardless of how correct your DKIM/SPF/DMARC are. Start with low volume to -a domain, increase gradually over days/weeks rather than sending everything on -day one, and check the IP against major blocklists (Spamhaus and similar) -before and during warmup. This is inherent to how mail reputation works on the -public internet, not something SelfPost's configuration can shortcut. - -## Operations - -After sign-in the panel opens on **Status** — the place to answer "is the -service healthy and will mail be accepted?" - -- **Status** (`/status`) — supervised processes (Postfix, OpenDKIM, panel), - TLS certificate validity and expiry, milter socket presence, and a short - Postfix queue summary. The **Machine** card adds the resource usage of the - host underneath — processor (with the load average), memory and swap, and - per-interface network throughput and totals — read from the kernel's - counters; CPU and throughput are measured between refreshes, so they appear - one refresh after the page opens. A fully busy processor or a machine out of - memory is a warning here, because both delay or kill the mail path; - throughput is only reported. The hostname block compares `SELFPOST_HOSTNAME` - against the PTR record the internet publishes for this server's IP - (forward-confirmed reverse DNS); use *Re-check* after changing DNS. The - **Reload configuration** button re-applies OpenDKIM tables and the Postfix - sender map from the database — use it if daemons drifted from what the panel - shows after manual edits under `/data`. -- **Domains** (`/domains`) — add sending domains, inspect each domain's DKIM - TXT value, SPF/DMARC checks, and SASL applications. Per-domain rate limits - (level 2) are configured here. *Export domain* writes a single-domain archive; - *Import a domain* on the Backup page reads one back in. -- **Deliveries** (`/deliveries`) — searchable send log with server-side filters - by domain and application. A row identifies its message and nothing more — - time, sender, recipient, subject and status `queued` (accepted, not yet - delivered), `sent` (handed off successfully), or `rejected` (refused — for - example by a level-2 rate limit); *Details* opens that row's own page - (`/deliveries/{id}`) with the sending domain, the application it was - submitted under, the Postfix queue id to search the system log for, and when - the status was last reported. Retention is controlled by - `SEND_LOG_RETENTION_DAYS`. -- **Mail queue** (`/mail-queue`) — live view of messages Postfix is still - trying to deliver or deferring. -- **System log** (`/system-log`) — tail of `/var/log/mail.log` (Postfix and - related daemon lines). The log rotates daily (14 files kept) with a - `postfix reload` after each rotation; a background loop checks every six - hours. -- **Backup** (`/backup`) — download a full-server backup or import a - single-domain export. See [Backup, restore, and moving a single domain](#backup-restore-and-moving-a-single-domain). -- **Account** (`/account`) — change the administrator username and/or password. - Application SASL logins are separate and are not changed here. - -**Sessions.** A login survives a container restart: sessions live in SQLite, not -in memory. Expiry is a sliding idle window (`PANEL_SESSION_IDLE_DAYS`, default -seven days) with no absolute lifetime cap — an admin who keeps using the panel -stays signed in indefinitely. HTMX polling on the monitoring screens -(Deliveries, Mail queue, System log, and the Status health fragment) does -**not** count as activity, so an auto-refreshing tab left open will not keep a -session alive forever. Changing the password signs out every other session but -leaves the current browser signed in. - -**Upgrading.** Bump the pinned image tag in `docker-compose.yml` to the target -release, then `docker compose up -d`. The backup version check requires the -running image to match the version that created a full backup — see [Fixed image -tag](#fixed-image-tag). - -**Container health.** The image declares a Docker `HEALTHCHECK` that probes -`GET /healthz` on port 8080 (unauthenticated). It returns `200 ok` when -OpenDKIM, the panel, and Postfix are all `RUNNING` under supervisord; -otherwise `503 unhealthy`. This catches a dead mail path that would still leave -the HTTP server up, but it does **not** verify TLS certificates, DNS records, -or end-to-end delivery — use the authenticated **Status** page for that. External -monitoring can use the same endpoint through the reverse proxy if you expose it, -or poll `docker inspect` health state on the host. - -## Rate limiting - -SelfPost applies two independent limits; both can refuse a submission, but only -level 2 writes a `rejected` row in the send log. - -**Level 1 (IP backstop)** — always on, configured via `.env`: - -- `RATE_LIMIT_MESSAGES_PER_IP` → Postfix `smtpd_client_message_rate_limit` -- `RATE_LIMIT_WINDOW_SECONDS` → Postfix `anvil_rate_time_unit` - -This is an anvil limit per connecting client IP. It keeps working even if the -journal-milter (level 2) is down. - -**Level 2 (per domain / per application)** — optional, configured in the panel -on each domain's page or on an individual application. You set a message -ceiling, a time window, and optionally restrict the limit to specific client -IPs; an empty IP list means the differentiated limit does not apply. When -exceeded, Postfix returns a 4xx and the refusal is recorded in Deliveries as -`rejected`. - -## Backup, restore, and moving a single domain - -Two related but distinct operations — spec 7.5: - -- **Full backup** (whole `/data`: SQLite, all domains' DKIM keys, all - applications' SASL credentials, `manifest.json` with the version that - created it): panel button (*Backup* → *Full backup*), or from the - host: - ```sh - docker exec selfpost-backup > selfpost-backup.tar.gz - ``` - **Restore** means unpacking that archive into a fresh `/data` bind mount and - starting a container of the **exact same image version** that created it — - SelfPost refuses to start otherwise and tells you which tag to use. On the - first successful start after restore, `manifest.json` from the archive is - **deleted** — it guards only that one boot, so a later in-place upgrade is - not blocked. This is why the compose file below pins a fixed tag rather than - `:latest`: without a known version, there'd be no way to tell which image - restoring a given backup actually requires. - - **Alternative: archive `./data` while stopped.** If the service can be taken - offline, `docker compose down` then `tar czf selfpost-data.tar.gz ./data` on - the host is safe — nothing is writing to SQLite. Do **not** tar `./data` while - the container is running: the database uses WAL mode and a naive copy can - capture an inconsistent snapshot. The panel/CLI backup remains preferable when - you cannot afford downtime because it takes a consistent SQLite snapshot via - the Backup API on a live container. - -- **Export/import a single domain** (domain page → *Export domain* to write the - file, *Backup* → *Import a domain* to read it back in): moves one domain — its DKIM key and its applications' **working** - SASL passwords — to a different SelfPost instance without regenerating - anything, so DNS (the DKIM TXT record) doesn't need to change. Unlike a full - restore, this works across different hostnames/instances. - -Both files are **secrets** — they contain the admin password hash (full -backup) or working application credentials (domain export) in the clear or in -directly reversible form. Treat them like any other credential material: -restrict who can read them, don't email them around — and encrypt them, which -SelfPost can do for you. - -### Encrypting a backup or export - -Both download forms carry an **Encrypt with a password** checkbox. Ticked, the -file that comes down is an encrypted envelope instead of the plain archive: - -| Artefact | Plain | Encrypted | -|----------|-------|-----------| -| Full backup | `.tar.gz` | `.spbk` | -| Domain export | `.json` | `.spde` | - -The key is derived from the password with scrypt and the contents are sealed -with AES-256-GCM, in chunks, so a truncated or altered file fails to open rather -than restoring quietly. **SelfPost does not store the password** — lose it and -the file is unrecoverable, which is the entire point. - -*Import a domain* takes an encrypted export directly: tick **The file is -encrypted** and give the password. - -A full backup has to be turned back into a plain archive before it can be -unpacked into `/data`, which the CLI does with the same password: - ```sh -docker exec -i selfpost-backup -decrypt < backup.spbk > backup.tar.gz +cat ./data/setup-token ``` -The CLI also *writes* encrypted backups for scripted/cron use. The password -comes from `SELFPOST_BACKUP_PASSWORD` or `-password-file ` (first line), -never from a command-line argument, which would be visible in the process list: +The file is deleted as soon as setup completes. If logs are shipped to a +central aggregator, prefer `cat ./data/setup-token` so the bearer token does +not enter the log pipeline. -```sh -docker exec -e SELFPOST_BACKUP_PASSWORD="$PW" selfpost-backup > backup.spbk -``` +### 4. DNS and sending -With no password set, the CLI keeps writing the plain `.tar.gz` it always has. +Before sending real mail: -## Published ports +1. Confirm PTR/rDNS for the server IP points at `SELFPOST_HOSTNAME` (Status + page → *Re-check*). +2. For each domain you add in the panel, publish SPF, DKIM, and DMARC at the + same time ([DNS setup](docs/guide.md#dns-setup)). +3. Warm up a new IP gradually ([IP warmup](docs/guide.md#ip-warmup)). -`deploy/docker-compose.yml` maps **465** and **587** to the host. Port 465 -(smtps) is always active. Port **587** is published even when -`SUBMISSION_ENABLE=false`; nothing listens until you set it to `true` — harmless, -but it can look like an open port in external scans. +### Ports and upgrades -## Fixed image tag +The compose file maps **465** (always) and **587** (when +`SUBMISSION_ENABLE=true`). Bump the pinned image tag deliberately when +upgrading — never use `:latest` ([why](docs/guide.md#fixed-image-tag)). -`deploy/docker-compose.yml` pins an explicit version (`ghcr.io/mixeme/selfpost:X.Y.Z`), -deliberately never `:latest`. This is a direct consequence of the backup -version check above: the panel binary's embedded version and the image tag -that produced it are the same value by construction (the release CI stamps -both from one git tag — see `.github/workflows/release.yml`), so pinning the -tag is what makes "restore into the same version" a checkable fact rather than -a guess. Upgrade by bumping the tag deliberately, not by riding a moving -target. - -## Machine requirements - -Rough guide, not a hard floor: **1 vCPU**, **512MB–1GB RAM** (the stack — three -processes plus SQLite — idles around 100–150MB; the rest is headroom for -backups, log-tailer/retention sweeps and concurrent TLS handshakes coinciding), -**8–10GB disk**. Disk usage grows mainly from the send log (bounded by -`SEND_LOG_RETENTION_DAYS`, default 90) and the rotated `mail.log` (kept 14 days -in-image), not from the application itself. On boxes with little RAM, a small -swap file is cheap insurance against those occasional coincident spikes. - -## Repository - - — the single home of the project: source, -issues, releases and the `ghcr.io/mixeme/selfpost` image all live there. +Optional variables (`TRUSTED_PROXY_CIDR`, rate limits, retention): see +[Environment variables](docs/guide.md#environment-variables). ## License diff --git a/SECURITY.md b/SECURITY.md new file mode 100644 index 0000000..e5dd6aa --- /dev/null +++ b/SECURITY.md @@ -0,0 +1,81 @@ +# Security policy + +## Supported versions + +SelfPost follows SemVer. Fixes are issued for the **latest minor release of the +1.x line** only; there is no backporting to earlier minors. Upgrade before +reporting if you are behind — the image tag is `ghcr.io/mixeme/selfpost:X.Y.Z`. + +| Version | Supported | +|---|---| +| latest 1.x | yes | +| earlier 1.x | no — upgrade first | +| 0.x | no (pre-release) | + +## Reporting a vulnerability + +**Do not open a public issue.** Use GitHub's private vulnerability reporting: +the *Report a vulnerability* button under the repository's +[Security tab](https://github.com/mixeme/selfpost/security). If you cannot use +it, mail `public@mixeme.ru` instead. + +Useful in a report: the image tag, the reverse proxy in front of the panel, the +steps to reproduce, and what an attacker gains. A relevant excerpt of +`mail.log` or the panel's system log helps; strip recipient addresses first. + +**No response time is promised.** SelfPost is maintained by one person, and a +deadline that cannot be honoured is worse than none. Reports are read and +answered as soon as the maintainer is able; a fix ships in a patch release, +with the timeline agreed in the thread. + +Disclosure is coordinated by request, not by demand: please hold public details +until a patch is out. If you get no reply, that is not a request for a +continued embargo — disclose at your own discretion. Reporters are credited in +the CHANGELOG unless they ask not to be. + +## In scope + +The relay's job is to accept authenticated mail from an application and hand it +to the internet as the operator's domain, and nothing else. Breaking that is in +scope: + +- **Open relay** — mail accepted from an unauthenticated sender, or relayed for + a domain the sending application is not bound to +- **SASL bypass** — sending without valid credentials, or credential recovery + from anything the container exposes +- **Cross-domain access** — an application or a panel session reaching a domain + it was not granted +- **Secret disclosure** — DKIM private keys, the admin password hash, session + tokens, or backup encryption material leaking to an unauthorised party +- **Panel authentication and session flaws** — login bypass, session fixation, + CSRF on state-changing routes, privilege escalation +- **Rate-limit bypass** — evading either the Postfix-level backstop or the + per-domain and per-application limits +- **Container escape** or privilege escalation from the panel's unprivileged + user to root + +## Out of scope + +These are the operator's responsibility or accepted trade-offs, documented in +[docs/security.md](docs/security.md) and the +[operator guide](docs/guide.md): + +- Host configuration the operator controls: a blocked port 25, a missing or + wrong PTR record, DNS records not published, a self-signed or expired + certificate on the reverse proxy +- Anything requiring the attacker to already have root on the host or write + access to the `./data` bind mount +- Missing hardening headers or TLS options on the reverse proxy — SelfPost + never terminates HTTPS itself +- Deliverability outcomes: mail rejected or filtered by a receiving provider is + a policy decision of that provider, not a defect +- Denial of service through sheer volume against a single-tenant relay +- Vulnerabilities in upstream Postfix, OpenDKIM, or the base image — report + those upstream; if SelfPost's configuration makes an upstream issue + exploitable when it otherwise would not be, that *is* in scope + +## Reports we cannot act on + +Automated scanner output with no demonstrated impact, and reports whose only +content is a version number compared against a CVE list, are closed without +investigation. diff --git a/build/Dockerfile b/build/Dockerfile index 7aad6f9..90fabda 100644 --- a/build/Dockerfile +++ b/build/Dockerfile @@ -105,7 +105,7 @@ RUN chmod +x /usr/local/bin/postfix-wrapper.sh /usr/local/bin/postfix-config.sh EXPOSE 8080 465 587 # Liveness probe: panel HTTP plus mail-path processes (opendkim, panel, postfix). -# Does not verify TLS, DNS, or end-to-end delivery — see README Operations. +# Does not verify TLS, DNS, or end-to-end delivery — see docs/guide.md Operations. HEALTHCHECK --interval=30s --timeout=5s --start-period=90s --retries=3 \ CMD curl -fsS http://127.0.0.1:8080/healthz || exit 1 diff --git a/build/entrypoint.sh b/build/entrypoint.sh index 779f92e..c24179f 100644 --- a/build/entrypoint.sh +++ b/build/entrypoint.sh @@ -1,69 +1,12 @@ #!/bin/sh # Container entrypoint (runs as root, PID 1 until it execs supervisord). # -# The persistent root /data is a host bind mount (spec 9), so it arrives owned -# by the host user (typically root), not by the unprivileged panel user that -# actually writes the SQLite database, setup token and DKIM keys (spec 7.6.8). -# Fix its ownership here — the one place still running as root — before handing -# off to supervisord, which starts the panel as the panel user. +# SELFPOST_HOSTNAME is checked first (plan B.3): it must fail fast with a clear +# message before /data normalisation or Postfix config, so a bad identity never +# looks like a permissions or packaging problem (and so the e2e hostname-gate +# tests see the FATAL text rather than an earlier set -e abort). set -e -chown panel:panel /data -# Restored backups or previously-created state may contain panel-owned files -# under /data; make sure they stay writable without disturbing anything that a -# later phase deliberately hands to another service. -find /data -mindepth 1 -maxdepth 1 ! -user panel -exec chown -R panel:panel {} + - -# DKIM key tree (spec 6, 9). The panel (user `panel`) generates keys and writes -# the OpenDKIM tables; OpenDKIM (user `opendkim`) must read them. Normalise the -# tree on every start so it is correct whether /data is fresh, restarted, or -# just restored from a backup: -# - group `selfpost` + setgid on directories so anything the panel creates -# inherits the shared group OpenDKIM traverses; -# - private keys and tables group-readable (0640); -# - both table files present (empty is fine) BEFORE OpenDKIM starts, so the -# daemon comes up cleanly with no domains yet. -mkdir -p /data/opendkim/keys -for t in /data/opendkim/KeyTable /data/opendkim/SigningTable; do - [ -e "$t" ] || : > "$t" -done -chown -R panel:selfpost /data/opendkim -find /data/opendkim -type d -exec chmod 2750 {} + -chmod 0640 /data/opendkim/KeyTable /data/opendkim/SigningTable -find /data/opendkim/keys -type f -name '*.private' -exec chmod 0640 {} + - -# Application SASL accounts (spec 5.1, 9). The panel (user `panel`) writes the -# sasldb2 via saslpasswd2; Postfix (user `postfix`) reads it to authenticate SMTP -# clients. Share it through the `selfpost` group the same way as the DKIM tree: -# setgid directory so new files inherit the group, and the database itself -# group-readable (0640). -mkdir -p /data/sasl -chown -R panel:selfpost /data/sasl -chmod 2750 /data/sasl -[ -e /data/sasl/sasldb2 ] && chmod 0640 /data/sasl/sasldb2 - -# Postfix sender_login_maps (spec 5.1). The panel writes it; Postfix reads it. -# Ensure the file exists (empty is fine) before Postfix starts so a reload that -# references it never fails on a missing file, and keep it group-readable. -mkdir -p /data/postfix -[ -e /data/postfix/sender_login_maps ] || : > /data/postfix/sender_login_maps -chown -R panel:selfpost /data/postfix -chmod 2750 /data/postfix -chmod 0640 /data/postfix/sender_login_maps - -# Milter socket directories (spec 5 p.3, 7.3). Postfix (user `postfix`) must -# actually CONNECT to both milter sockets — OpenDKIM's and the panel's -# journal-milter — not just probe them at start-up. The sockets are -# created by the opendkim and panel users respectively, so bridge them to -# `postfix` through the shared `selfpost` group: group-owned + setgid dirs mean -# each socket created inside inherits group `selfpost`, and group-traversable -# (2750) lets postfix reach it. Without this, smtpd cannot talk to OpenDKIM and, -# because signing is strict (default_action=tempfail), rejects all mail. -mkdir -p /run/opendkim /run/selfpost -chown opendkim:selfpost /run/opendkim -chown panel:selfpost /run/selfpost -chmod 2750 /run/opendkim /run/selfpost - # SELFPOST_HOSTNAME is an identity, not a setting with a safe default: it must # simultaneously match the PTR/rDNS record, the certificate CN/SAN, and the # Cyrus SASL realm (spec 5.2 p.3, 8). The panel (main.go saslRealm()) and @@ -109,6 +52,92 @@ case "$SELFPOST_HOSTNAME" in ;; esac +# The persistent root /data is a host bind mount (spec 9), so it arrives owned +# by the host user (typically root), not by the unprivileged panel user that +# actually writes the SQLite database, setup token and DKIM keys (spec 7.6.8). +# Fix its ownership here — the one place still running as root — before handing +# off to supervisord, which starts the panel as the panel user. +# +# Mode must stay world-traversable (0755): OpenDKIM and Postfix reach their +# trees under /data as other users. Go's testing.TempDir is 0700, and a bare +# chown would leave that mode in place — opendkim then cannot read KeyTable and +# the container crash-loops (e2e TestHostnameGate/valid_hostname_starts). +chown panel:panel /data +chmod 755 /data +# Restored backups or previously-created state may contain panel-owned files +# under /data; make sure they stay writable without disturbing anything that a +# later phase deliberately hands to another service. /data/log is exempt: it is +# deliberately owned by postfix (postlogd writes the delivery log there) and is +# normalised on its own below. +find /data -mindepth 1 -maxdepth 1 ! -user panel ! -name log -exec chown -R panel:panel {} + + +# DKIM key tree (spec 6, 9). The panel (user `panel`) generates keys and writes +# the OpenDKIM tables; OpenDKIM (user `opendkim`) must read them. Normalise the +# tree on every start so it is correct whether /data is fresh, restarted, or +# just restored from a backup: +# - group `selfpost` + setgid on directories so anything the panel creates +# inherits the shared group OpenDKIM traverses; +# - private keys and tables group-readable (0640); +# - both table files present (empty is fine) BEFORE OpenDKIM starts, so the +# daemon comes up cleanly with no domains yet. +mkdir -p /data/opendkim/keys +for t in /data/opendkim/KeyTable /data/opendkim/SigningTable; do + [ -e "$t" ] || : > "$t" +done +chown -R panel:selfpost /data/opendkim +find /data/opendkim -type d -exec chmod 2750 {} + +chmod 0640 /data/opendkim/KeyTable /data/opendkim/SigningTable +find /data/opendkim/keys -type f -name '*.private' -exec chmod 0640 {} + + +# Application SASL accounts (spec 5.1, 9). The panel (user `panel`) writes the +# sasldb2 via saslpasswd2; Postfix (user `postfix`) reads it to authenticate SMTP +# clients. Share it through the `selfpost` group the same way as the DKIM tree: +# setgid directory so new files inherit the group, and the database itself +# group-readable (0640). +mkdir -p /data/sasl +chown -R panel:selfpost /data/sasl +chmod 2750 /data/sasl +[ -e /data/sasl/sasldb2 ] && chmod 0640 /data/sasl/sasldb2 + +# Postfix sender_login_maps (spec 5.1). The panel writes it; Postfix reads it. +# Ensure the file exists (empty is fine) before Postfix starts so a reload that +# references it never fails on a missing file, and keep it group-readable. +mkdir -p /data/postfix +[ -e /data/postfix/sender_login_maps ] || : > /data/postfix/sender_login_maps +chown -R panel:selfpost /data/postfix +chmod 2750 /data/postfix +chmod 0640 /data/postfix/sender_login_maps + +# Delivery log (architecture.md § Log tailer). postlogd writes it as user +# `postfix`; the panel reads it for the log-tailer and the System log page. It +# lives under /data — not the ephemeral /var/log — so the delivery lines that +# resolve a "queued" send-log row survive a container recreate. +# +# postlogd creates a missing log itself, but at 0600, which the unprivileged +# panel cannot read; so create it here (and re-normalise an existing one, plus +# whatever logrotate left behind) at 0640 owned postfix:selfpost. The setgid +# directory keeps the shared group on anything created inside it later, and +# 2750 keeps it group-traversable but not group-writable — logrotate refuses to +# rotate a log whose directory is writable by a non-root group. +mkdir -p /data/log +[ -e /data/log/mail.log ] || : > /data/log/mail.log +chown -R postfix:selfpost /data/log +chmod 2750 /data/log +find /data/log -type f -exec chmod 0640 {} + + +# Milter socket directories (spec 5 p.3, 7.3). Postfix (user `postfix`) must +# actually CONNECT to both milter sockets — OpenDKIM's and the panel's +# journal-milter — not just probe them at start-up. The sockets are +# created by the opendkim and panel users respectively, so bridge them to +# `postfix` through the shared `selfpost` group: group-owned + setgid dirs mean +# each socket created inside inherits group `selfpost`, and group-traversable +# (2750) lets postfix reach it. Without this, smtpd cannot talk to OpenDKIM and, +# because signing is strict (default_action=tempfail), rejects all mail. +mkdir -p /run/opendkim /run/selfpost +chown opendkim:selfpost /run/opendkim +chown panel:selfpost /run/selfpost +chmod 2750 /run/opendkim /run/selfpost + # Generate the outbound-relay Postfix configuration from the environment (spec # 5). Kept out of the image build so cert paths, rate limits, hostname and the # optional 587 service are all driven by env at run time, and re-derived on every diff --git a/build/logrotate-loop.sh b/build/logrotate-loop.sh index e016a6c..c934d21 100644 --- a/build/logrotate-loop.sh +++ b/build/logrotate-loop.sh @@ -1,13 +1,13 @@ #!/bin/sh -# Periodic logrotate for /var/log/mail.log (spec 9, 10). Rotation renames the -# file, recreates it (`create 0644 root root`, matching a cold container +# Periodic logrotate for /data/log/mail.log (spec 9, 10). Rotation renames the +# file, recreates it (`create 0640 postfix selfpost`, matching a cold container # start), then runs `postfix reload` (the same mechanism `postfix logrotate` # uses): postlogd keeps writing to the renamed inode until reload, and the # panel's log-tailer holds its own descriptor on that inode, so nothing # written before the reload is lost. `create` (rather than `nocreate`) matters -# here beyond timing: a reload-triggered recreate lands the file at 0600, -# which the unprivileged panel process cannot read — confirmed on a live -# container — so logrotate must be the one to create it at 0644. +# here beyond timing: a postlogd-triggered recreate lands the file at 0600 +# owned by postfix, which the unprivileged panel process cannot read — +# confirmed on a live container — so logrotate must be the one to create it. # # logrotate itself only rotates once the configured "daily" period has elapsed # (tracked in /var/lib/logrotate/status), so it is safe to invoke this more diff --git a/build/logrotate-mail.conf b/build/logrotate-mail.conf index 5f4e402..9912d40 100644 --- a/build/logrotate-mail.conf +++ b/build/logrotate-mail.conf @@ -1,11 +1,17 @@ -/var/log/mail.log { +# The delivery log lives under the persistent /data, not the ephemeral +# /var/log, so the lines that resolve a "queued" send-log row outlive the +# container (architecture.md § Log tailer). Path and ownership match +# build/postfix-config.sh (maillog_file) and build/entrypoint.sh: postlogd +# writes it as user postfix, the unprivileged panel reads it through the shared +# selfpost group, hence create 0640 postfix selfpost rather than 0644 root root. +/data/log/mail.log { daily rotate 14 missingok notifempty compress delaycompress - create 0644 root root + create 0640 postfix selfpost postrotate /usr/sbin/postfix reload endscript diff --git a/build/postfix-config.sh b/build/postfix-config.sh index 61578f4..a595c96 100644 --- a/build/postfix-config.sh +++ b/build/postfix-config.sh @@ -23,8 +23,26 @@ HOSTNAME_VALUE="${SELFPOST_HOSTNAME:-$(hostname -f 2>/dev/null || hostname)}" # TLS material supplied by the reverse-proxy through a read-only bind mount # (spec 5.2). The relay requires TLS on 465; if these files are absent the # master still starts but TLS handshakes on 465 fail until they appear. -TLS_CERT="${TLS_CERT_FILE:-/etc/postfix/tls/fullchain.pem}" -TLS_KEY="${TLS_KEY_FILE:-/etc/postfix/tls/privkey.pem}" +# +# Postfix insists the private key is root-owned and mode 0600. The bind mount +# is often :ro and owned by the host user (e2e TempDir / CI runner UID), so +# copy into a writable internal dir and normalise ownership before postconf +# and `postfix check`. +TLS_CERT_SRC="${TLS_CERT_FILE:-/etc/postfix/tls/fullchain.pem}" +TLS_KEY_SRC="${TLS_KEY_FILE:-/etc/postfix/tls/privkey.pem}" +TLS_INTERNAL_DIR=/etc/postfix/tls-internal +TLS_CERT="$TLS_CERT_SRC" +TLS_KEY="$TLS_KEY_SRC" +if [ -f "$TLS_CERT_SRC" ] && [ -f "$TLS_KEY_SRC" ]; then + mkdir -p "$TLS_INTERNAL_DIR" + cp -f "$TLS_CERT_SRC" "$TLS_INTERNAL_DIR/fullchain.pem" + cp -f "$TLS_KEY_SRC" "$TLS_INTERNAL_DIR/privkey.pem" + chown root:root "$TLS_INTERNAL_DIR/fullchain.pem" "$TLS_INTERNAL_DIR/privkey.pem" + chmod 0644 "$TLS_INTERNAL_DIR/fullchain.pem" + chmod 0600 "$TLS_INTERNAL_DIR/privkey.pem" + TLS_CERT="$TLS_INTERNAL_DIR/fullchain.pem" + TLS_KEY="$TLS_INTERNAL_DIR/privkey.pem" +fi # Level-1 rate limit (native Postfix anvil, spec 5 p.5 / 7.4). Conservative # defaults, sensible during IP warm-up (spec 10). @@ -45,15 +63,32 @@ SASLDB_PATH="${SASL_DB_PATH:-/data/sasl/sasldb2}" # when a client library needs STARTTLS on 587 instead of implicit TLS on 465). SUBMISSION_ENABLE="${SUBMISSION_ENABLE:-false}" +# Delivery log, written by postlogd and read by the panel's log-tailer. It lives +# under the persistent /data (not the ephemeral /var/log) so the delivery lines +# for messages still marked "queued" survive a container recreate — without +# them those rows could never be resolved (architecture.md § Log tailer). The +# default must match cmd/panel/main.go's MAIL_LOG; entrypoint.sh creates the +# directory and the file with the ownership postlogd writes and the panel reads. +MAIL_LOG_PATH="${MAIL_LOG:-/data/log/mail.log}" + # --- main.cf ----------------------------------------------------------------- +# maillog_file lives under the persistent /data bind mount (architecture.md § +# Log tailer). Postfix's default maillog_file_prefixes are only /var and +# /dev/stdout — without /data, `postfix check` rejects the path. postconf -e \ "myhostname=${HOSTNAME_VALUE}" \ - "maillog_file=/var/log/mail.log" \ + "maillog_file=${MAIL_LOG_PATH}" \ + "maillog_file_prefixes=/var,/dev/stdout,/data" \ "mydestination=" \ "relayhost=" \ "inet_interfaces=all" \ "inet_protocols=all" +# postlogd is mandatory whenever maillog_file is set (MAILLOG_README). Debian's +# stock master.cf usually has it; pin it explicitly so a stripped/upgraded +# image cannot lose the service. +postconf -M "postlog/unix-dgram=postlog unix-dgram n - n - 1 postlogd" + # This is an outbound relay: no local delivery, no per-user aliases. Empty # these so a misfiled recipient never gets delivered locally. postconf -e \ @@ -181,4 +216,12 @@ EOF # Validate the generated configuration; fail loudly if postconf produced # anything Postfix rejects, before the wrapper tries to start it. -postfix check +set +e +check_out=$(postfix check 2>&1) +ec=$? +set -e +if [ "$ec" -ne 0 ]; then + echo "postfix-config: postfix check failed exit $ec" >&2 + printf '%s\n' "$check_out" >&2 + exit "$ec" +fi diff --git a/build/supervisord.conf b/build/supervisord.conf index 9b15635..99b3234 100644 --- a/build/supervisord.conf +++ b/build/supervisord.conf @@ -103,7 +103,7 @@ stdout_logfile_maxbytes=0 stderr_logfile=/dev/stderr stderr_logfile_maxbytes=0 -; Periodic logrotate for /var/log/mail.log (spec 9, 10: daily, 7-14 files kept +; Periodic logrotate for /data/log/mail.log (spec 9, 10: daily, 7-14 files kept ; in the image). Runs as root so logrotate can read/rotate the log; never exits ; non-zero, so it neither trips the crashexit listener nor needs restarting. [program:logrotate] diff --git a/cmd/panel/envdoc_test.go b/cmd/panel/envdoc_test.go index 3bb8675..2cb1b6a 100644 --- a/cmd/panel/envdoc_test.go +++ b/cmd/panel/envdoc_test.go @@ -5,7 +5,7 @@ import ( "testing" ) -// documentedPublic matches the README "Environment variables" table. +// documentedPublic matches docs/guide.md "Environment variables" table. var documentedPublic = []string{ "SELFPOST_HOSTNAME", "SUBMISSION_ENABLE", @@ -17,7 +17,7 @@ var documentedPublic = []string{ "TRUSTED_PROXY_CIDR", } -// documentedInternal matches README "Internal variables (not part of the operator interface)". +// documentedInternal matches guide.md "Internal variables (not part of the operator interface)". var documentedInternal = []string{ "SELFPOST_DATA_DIR", "SELFPOST_DB_PATH", @@ -82,6 +82,7 @@ var buildScriptKeys = []string{ "RATE_LIMIT_WINDOW_SECONDS", "OPENDKIM_SOCKET", "JOURNAL_MILTER_SOCKET", + "MAIL_LOG", "POSTFIX_SENDER_LOGIN_MAPS", "SASL_DB_PATH", "SUBMISSION_ENABLE", @@ -105,7 +106,7 @@ func TestLoadConfigKeysDocumented(t *testing.T) { doc := documentedKeys() for _, key := range loadConfigKeys { if !slices.Contains(doc, key) { - t.Errorf("loadConfig reads %s but it is not listed in README public, internal, or compose-fixed env docs", key) + t.Errorf("loadConfig reads %s but it is not listed in guide.md public, internal, or compose-fixed env docs", key) } } } @@ -114,7 +115,7 @@ func TestBuildScriptKeysDocumented(t *testing.T) { doc := documentedKeys() for _, key := range buildScriptKeys { if !slices.Contains(doc, key) { - t.Errorf("build scripts read %s but it is not listed in README env documentation", key) + t.Errorf("build scripts read %s but it is not listed in guide.md env documentation", key) } } } @@ -127,7 +128,7 @@ func TestDocumentedKeysAreRead(t *testing.T) { for _, key := range documentedKeys() { if !slices.Contains(read, key) { - t.Errorf("README documents %s but no code in loadConfig or build scripts reads it", key) + t.Errorf("guide.md documents %s but no code in loadConfig or build scripts reads it", key) } } } diff --git a/cmd/panel/main.go b/cmd/panel/main.go index a2ea695..dfae2f1 100644 --- a/cmd/panel/main.go +++ b/cmd/panel/main.go @@ -81,7 +81,10 @@ func loadConfig() config { return config{ httpAddr: envDefault("PANEL_HTTP_ADDR", ":8080"), journalSocket: envDefault("JOURNAL_MILTER_SOCKET", "/run/selfpost/journal.sock"), - mailLog: envDefault("MAIL_LOG", "/var/log/mail.log"), + // Postfix's delivery log, under /data so the lines that resolve a + // "queued" send-log row outlive the container. The default must match + // maillog_file in build/postfix-config.sh. + mailLog: envDefault("MAIL_LOG", "/data/log/mail.log"), // Send-log retention window (architecture.md § Persistence). // Non-positive/invalid falls back to the 90-day default inside the // log-tailer. diff --git a/deploy/.env.example b/deploy/.env.example index b12d881..48a9c1c 100644 --- a/deploy/.env.example +++ b/deploy/.env.example @@ -1,5 +1,5 @@ # Copy to .env next to docker-compose.yml and fill in. -# Full reference: README "Environment variables". +# Full reference: docs/guide.md "Environment variables". # Required — the container refuses to start without it. Hostname used both for # the panel's TLS vhost and for Postfix's SASL realm (SASL_REALM defaults to @@ -11,7 +11,7 @@ SELFPOST_HOSTNAME=mail.example.com # primary 465/smtps listener. SUBMISSION_ENABLE=false -# Level-1 backstop rate limit (anvil) — see README "Rate limiting". +# Level-1 backstop rate limit (anvil) — see docs/guide.md "Rate limiting". RATE_LIMIT_MESSAGES_PER_IP=100 RATE_LIMIT_WINDOW_SECONDS=3600 diff --git a/deploy/docker-compose.yml b/deploy/docker-compose.yml index d673e0b..6f803df 100644 --- a/deploy/docker-compose.yml +++ b/deploy/docker-compose.yml @@ -5,7 +5,7 @@ # installed on the HOST (the target audience for this project typically runs # Apache there already — spec 10.5) and reverse-proxies HTTPS for the panel; # it is not containerised here. See ../apache/selfpost-vhost.conf for a ready -# vhost fragment, and the "Reverse proxy" section of ../../README.md for the +# vhost fragment, and docs/guide.md "Reverse proxy" for the # certbot steps that produce the PEM files this compose file mounts. # # Usage: @@ -22,7 +22,7 @@ services: selfpost: - image: ghcr.io/mixeme/selfpost:0.1.0 + image: ghcr.io/mixeme/selfpost:1.0.1 restart: unless-stopped environment: SELFPOST_HOSTNAME: "${SELFPOST_HOSTNAME:?set the mail/panel hostname, e.g. mail.example.com}" @@ -47,13 +47,16 @@ services: # resolver synthesises. Empty uses public defaults. SELFPOST_DNS_RESOLVERS: "${SELFPOST_DNS_RESOLVERS:-}" # Reverse-proxy CIDRs allowed to supply X-Forwarded-For for login/setup - # rate-limiting. See README "Environment variables" — wrong value lets a + # rate-limiting. See docs/guide.md "Environment variables" — wrong value lets a # client spoof its rate-limit key. TRUSTED_PROXY_CIDR: "${TRUSTED_PROXY_CIDR:-}" volumes: # All persistent state lives under /data (spec 9): SQLite DB, DKIM keys, # sasldb2, sender map, setup token. Back this up (panel button or the # selfpost-backup CLI) before you touch it directly. + # Postfix's mail.log is here too (./data/log, rotated daily, 14 files + # kept) so the delivery lines that resolve a queued send-log row survive + # a container recreate; it is diagnostics and is left out of backups. - ./data:/data # Read-only: SelfPost only ever reads certificates, never manages them # (spec 10 p.2). Point this at wherever your reverse-proxy/certbot @@ -68,7 +71,7 @@ services: # panel is never directly reachable from the internet without TLS. - "465:465" # 587 is mapped even when SUBMISSION_ENABLE=false; Postfix listens only - # when the variable is true — see README "Published ports". + # when the variable is true — see docs/guide.md "Published ports". - "587:587" - "127.0.0.1:8080:8080" # Hardening (spec 10 p.6). SelfPost's entrypoint still needs to run as diff --git a/deploy/traefik/extract-cert.sh b/deploy/traefik/extract-cert.sh index b2c04fa..4eb1bf4 100644 --- a/deploy/traefik/extract-cert.sh +++ b/deploy/traefik/extract-cert.sh @@ -1,9 +1,9 @@ #!/bin/sh # Extracts a PEM cert/key pair for one domain out of Traefik's acme.json -# (spec 10.3: "Traefik — сертификаты в acme.json, потребуется шаг извлечения -# PEM"). Run this on the host, after Traefik has issued or renewed the -# certificate, and again on a schedule (cron/systemd timer) since acme.json -# is not itself watched by SelfPost/Postfix. +# (spec 10.3: "Traefik keeps certificates in acme.json, so a PEM extraction +# step is required"). Run this on the host, after Traefik has issued or +# renewed the certificate, and again on a schedule (cron/systemd timer) since +# acme.json is not itself watched by SelfPost/Postfix. # # Requires jq. Usage: ./extract-cert.sh set -eu diff --git a/docs/architecture.md b/docs/architecture.md index 5e2bfce..afa414f 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -2,9 +2,9 @@ **Source of truth:** the code tree, not historical specs. Synchronise this file when env keys, routes, or mail-path behaviour change. Verification method: -[documentation-plan.md](documentation-plan.md) §2. +[development.md](development.md) § «Verifying docs against code». -User install/operations: [README.md](../README.md). Product boundaries: +User install/operations: [README.md](../README.md), [guide.md](guide.md). Product boundaries: [product.md](product.md). --- @@ -26,7 +26,7 @@ Managed programs ([build/supervisord.conf](../build/supervisord.conf)): | Program | User | Priority | Role | |---|---|---|---| -| `opendkim` | opendkim | 100 | DKIM signing milter | +| `opendkim` | root → `opendkim` (`UserID` in opendkim.conf) | 100 | DKIM signing milter | | `panel` | panel | 200 | HTTP UI + journal-milter + log-tailer goroutine | | `postfix` | root (wrapper) | 300 | MTA — started only after both milter sockets exist | | `postfix-reload` | root | — | On-demand `postfix reload` (autostart off) | @@ -94,10 +94,20 @@ Milter chain in Postfix: OpenDKIM (tempfail) then journal (accept on failure). ### Log tailer and `mail.log` rotation -`mail.log` lives under `/var/log` (not in `/data`). Rotation uses rename + -`postfix reload` ([build/logrotate-mail.conf](../build/logrotate-mail.conf)), not -`copytruncate` — the latter can drop `status=sent` lines and leave send-log rows -stuck at `queued`. After rename, logrotate runs `create 0644 root root` (Postfix +`mail.log` lives at `/data/log/mail.log` — inside the persistent bind mount, so +the delivery lines that resolve a `queued` send-log row are not lost when the +container is recreated. `postlogd` writes it as user `postfix`; the panel reads +it through the shared `selfpost` group (directory `2750 postfix:selfpost`, file +`0640`, both normalised on every start by +[build/entrypoint.sh](../build/entrypoint.sh)). The path is one default in two +places, `maillog_file` in [build/postfix-config.sh](../build/postfix-config.sh) +and `MAIL_LOG` in [cmd/panel/main.go](../cmd/panel/main.go). Backups exclude +`log/`: it is diagnostic output, not state to restore. + +Rotation uses rename + `postfix reload` +([build/logrotate-mail.conf](../build/logrotate-mail.conf)), not `copytruncate` — +the latter can drop `status=sent` lines and leave send-log rows stuck at +`queued`. After rename, logrotate runs `create 0640 postfix selfpost` (postlogd recreates the file lazily on first write as mode `0600`, which the unprivileged panel user cannot read). `follow()` drains the old inode once more before switching descriptors; the panel treats a missing log file as an empty tail, not @@ -112,21 +122,39 @@ the current file from the start; re-parsing lines is harmless because with nothing stored, begins at end-of-file, so installing the panel does not replay a pre-existing log. -**Remaining gap:** +**Queue reconcile** is the backstop for what the log cannot explain at all: a +row still `queued` more than two minutes after it was accepted, whose queue id +`postqueue -p` no longer lists, is marked `bounced` (swept every five minutes, +[internal/logtail](../internal/logtail/logtail.go), +[postfix.QueueIDs](../internal/postfix/queue.go)). Postfix having dropped the +message means nothing more will ever be reported about it, so the row can only +be closed on an assumption, and it is closed as a failure because a delivery the +panel cannot evidence must not be shown as one. Three things keep the sweep from +guessing where it need not: it starts only after the tailer has read to +end-of-file once (on a restart the log itself holds the answer), the two-minute +grace covers messages merely in flight, and a `postqueue` that cannot be read +leaves every row untouched rather than closing them all. Now that the log +survives the container, reaching this path means the lines are gone for good — +rotated past fourteen files while the panel was down, or deleted. -- **Container recreate** — `/var/log` is ephemeral; the log is lost with the - container, so the delivery lines for rows still `queued` are gone with it and - those rows stay `queued` forever. - -Possible follow-ups if this becomes painful: mount the mail log under `/data`, or -reconcile stuck rows via `postqueue`. +**Two one-shot reads** sit beside the follow loop and are unrelated to it, both +serving panel pages on request: `TailLines` (the last *n* lines, for +`/system-log`) and `QueueLines` (the lines carrying one queue-id, for +`/deliveries/{id}`). `QueueLines` scans a bounded tail of the current file — +finding a message's lines means reading rather than seeking — and matches the id +anchored on the character before it, since queue ids are hexadecimal runs and a +shorter one is regularly the tail of a longer one. Send-log rows outlive the log +(retention 90 days, rotation 14 files), so an empty result is the expected end +state for an older message and the page reports it as such, not as a failure. --- ## Panel HTTP surface -Route table: [internal/web/web.go](../internal/web/web.go). Authenticated -unless noted. +Canonical routes: [internal/web/web.go](../internal/web/web.go). Authenticated +unless noted. The table below is a summary — HTMX fragment endpoints +(`/status/fragment`, `/deliveries/rows`, `/mail-queue/body`, +`/system-log/body`, …) and every POST variant live in `web.go`. | Route | Purpose | |---|---| @@ -135,16 +163,20 @@ unless noted. | `/login`, `/logout` | Session auth | | `/status` | Process, cert, socket, PTR checks; machine CPU/memory/network | | `/domains`, `/domains/*` | Domain and application CRUD, DKIM, L2 limits | +| `/domains/import` | Domain import (`POST`; form on the Backup page) | | `/deliveries` | Send log with filters | -| `/deliveries/{id}` | One send-log row in full | +| `/deliveries/{id}` | One send-log row in full, with its `mail.log` lines | | `/mail-queue` | Postfix queue view | | `/system-log` | `mail.log` tail | | `/reload` | Reload OpenDKIM + Postfix maps | -| `/backup` | Full backup download, domain import | +| `/backup` | Full backup download (page also hosts the import form) | | `/account` | Admin username/password | -HTMX polling refreshes monitoring fragments; polling does not extend session -idle timeout (only non-`HX-Request` GET and mutating requests count as activity). +HTMX polling refreshes monitoring fragments (5 s while the operator is active on +the page, 30 s when the tab is visible but idle, none when hidden — scheduled in +`panel.js` via `data-poll`, not `hx-trigger="every …"`); polling does not extend +session idle timeout (only non-`HX-Request` GET and mutating requests count as +activity). ### Sessions @@ -169,10 +201,12 @@ cookie and idle timeout has not expired. ## Code layers -Handlers never touch SQLite or the filesystem directly; every write that has to -land in more than one place (SQLite row, `sasldb2` entry, Postfix map, OpenDKIM -table) goes through a service, which is also where the rollback of a partial -failure lives. The adapters below the services are the only code that knows +Multi-store writes that must land in more than one place (SQLite row, +`sasldb2` entry, Postfix map, OpenDKIM table) go through a service, which is +also where the rollback of a partial failure lives. Handlers may call +`store` directly for single-table reads and simple writes (sessions, admin, +send-log queries); the first-run setup-token file is read and written in +`web` itself. The adapters below the services are the only code that knows about Postfix, OpenDKIM, DNS or the log file, which is what makes them substitutable in tests — `milter.Store`, `app.SenderMaps` and `logtail.StatusStore` are the seams the unit tests replace with fakes. @@ -207,6 +241,7 @@ flowchart TB panel --> logtail backupcli --> backupPkg backupcli --> secretfile + web --> store web --> domainSvc web --> appSvc web --> backupPkg @@ -237,6 +272,7 @@ single-connection trade-off that follows from it. | `opendkim/` | DKIM keys + tables | | `sasl/sasldb2` | Application SASL credentials | | `postfix/sender_login_maps` | Login → From binding | +| `log/mail.log` | Postfix delivery log + rotated copies (excluded from backups) | | `manifest.json` | Backup version stamp (consumed on restore) | Not in `/data`: TLS certificates (reverse-proxy mount), Postfix queue @@ -247,17 +283,17 @@ Not in `/data`: TLS certificates (reverse-proxy mount), Postfix queue `postfix reload` in `postrotate` — see § Log tailer above). **Backup:** panel button or `selfpost-backup` CLI — SQLite snapshot + tar of -`/data` tree; version check on restore. Stopped-container `tar` of `./data` is -safe (see README). +`/data` tree, minus `log/`, the setup token and any `tls/`; version check on +restore. Stopped-container `tar` of `./data` is safe (see guide). **Optional encryption** of the two secret-bearing downloads ([internal/secretfile](../internal/secretfile/secretfile.go)): password → scrypt → AES-256-GCM over 64 KiB chunks, each authenticated with the header, its counter and an end-of-stream flag (so truncation and reordering fail to -open). Full backup `.tar.gz` → `.spbk`, domain export `.json` → `.spde`; the -plain forms remain the default. Domain import detects the envelope by magic -bytes; an encrypted full backup is converted back with `selfpost-backup --decrypt` before restore. +open). Full backup `.tar.gz` → `.spbk` (SelfPost backup), domain export +`.json` → `.spde` (SelfPost domain export); the plain forms remain the +default. Domain import detects the envelope by magic bytes; an encrypted full +backup is converted back with `selfpost-backup -decrypt` before restore. --- @@ -270,5 +306,5 @@ origin check, no CSRF tokens) are documented there separately. ## Configuration -Public and internal env vars: [README § Environment variables](../README.md#environment-variables). +Public and internal env vars: [guide § Environment variables](guide.md#environment-variables). Regression test: [cmd/panel/envdoc_test.go](../cmd/panel/envdoc_test.go). diff --git a/docs/archive/specification-v1.0.md b/docs/archive/specification-v1.0.md deleted file mode 100644 index a9ebcae..0000000 --- a/docs/archive/specification-v1.0.md +++ /dev/null @@ -1,448 +0,0 @@ -> **Исторический снимок v1.0, не источник истины.** Актуальные документы: -> [product.md](../product.md), [architecture.md](../architecture.md), -> [development.md](../development.md), [security.md](../security.md), -> [README.md](../../README.md). - -# Техническое задание: SelfPost - -**Версия:** 1.0 -**Тип проекта:** self-hosted SMTP relay с веб-панелью управления -**Формат поставки:** один Docker-образ - ---- - -## 1. Цель и назначение - -SelfPost — это self-hosted SMTP-релей для личного использования, предназначенный для **исходящей отправки** почты из скриптов, приложений и систем уведомлений. Релей отправляет почту напрямую в интернет с собственного IP, с подписью DKIM, и управляется через веб-панель. - -**Ключевой сценарий использования:** пользователь настраивает релей один раз (домен, DKIM, разрешённые отправители), после чего его приложения/скрипты отправляют почту через SMTP-endpoint релея, а релей доставляет её получателям от имени настроенного домена. - ---- - -## 2. Контекст и ограничения - -Эти ограничения — результат предварительного анализа и **не подлежат пересмотру** без явного согласования: - -1. **Развёртывание — VPS или домашний сервер** (одноплатники вроде Raspberry Pi пока вне рассмотрения, но не исключены в будущем). Оба варианта равноправны — образ один и тот же. Инфраструктурные предпосылки для отправки с собственного IP считаются обеспеченными оператором (см. заметку ниже). -2. **Отправка с собственного IP** (DIY), без промежуточного relay-провайдера. Требует, чтобы VPS-провайдер разблокировал порт 25 (обычно по тикету) и позволял настроить PTR/rDNS. -3. **Один контейнер.** Postfix, OpenDKIM и панель управления работают внутри одного образа под управлением process-supervisor. Это осознанное решение в пользу простоты развёртывания. -4. **Панель управления публично доступна** из интернета. Это накладывает обязательные требования к безопасности (раздел 7). - -> **Предпосылки инфраструктуры (вне зоны ответственности проекта).** SelfPost исходит из того, что площадка (VPS или домашний сервер) уже обеспечивает условия для отправки с собственного IP: разблокированный исходящий порт 25, статический IP, настраиваемый PTR/rDNS и приемлемая репутация IP. Обеспечение этих условий — задача оператора при развёртывании, а не функция SelfPost. Проект не пытается их детектировать, обходить или компенсировать; при их отсутствии почта просто не будет доставляться, и это ожидаемо. - ---- - -## 3. Что НЕ входит в объём (Out of Scope) - -Явно исключено, чтобы не было scope creep: - -- ❌ Приём входящей почты (IMAP/POP3, mailbox'ы, доставка в ящики) -- ❌ Веб-почта (webmail) -- ❌ Мультитенантность (организации, несколько **пользователей панели**, роли) — панель рассчитана на **одного** администратора. Управление несколькими отправляющими доменами — это НЕ мультитенантность и входит в объём (см. раздел 4.1). -- ❌ Антиспам/антивирус для входящей почты (rspamd, ClamAV) -- ❌ Реализация собственного SMTP-сервера или MTA — используется готовый Postfix -- ❌ Dovecot и любой полноценный mail-стек ради SASL — аутентификация делается на лёгком Cyrus SASL (`sasldb2`), см. раздел 5.1 - ---- - -## 4. Архитектура - -Единый Docker-образ на базе **Debian slim** (например, `debian:bookworm-slim`). Выбор зафиксирован в пользу простоты поддержки: glibc даёт более предсказуемое поведение DNS-резолвера при постоянных MX-lookup'ах (в отличие от musl в Alpine), а подавляющее большинство документации и примеров конфигов Postfix/OpenDKIM ориентировано на Debian/apt — что снижает риск ошибок, особенно при генерации кода агентом. Экономия размера на Alpine здесь непринципиальна (образ всё равно тянет Python ради supervisord, а деплой идёт на VPS/сервер, не на ресурсно-ограниченный одноплатник). Внутри — три процесса под управлением `supervisord`: - -``` -┌───────────────────────────────────────────────┐ -│ Docker-контейнер SelfPost │ -│ │ -│ supervisord (PID 1) │ -│ ├── postfix (start-fg) │ -│ ├── opendkim (foreground) │ -│ └── panel (Go-бинарник, один процесс, │ -│ совмещает несколько ролей: │ -│ • HTTP-сервер панели :8080 │ -│ • journal-milter (приём :internal │ -│ From/To/Subject/SASL-user на этапе │ -│ DATA, см. 7.3) │ -│ • log-tailer — горутина, следит за │ -│ mail.log и обновляет статус доставки) │ -│ │ -│ Общая файловая система: │ -│ /etc/postfix/... ← панель пишет │ -│ /etc/opendkim/keys ← DKIM-ключи │ -│ /data/selfpost.db ← SQLite: домены, │ -│ приложения, администратор, журнал │ -│ отправки (см. 7.3, 9) │ -│ │ -│ Порты: │ -│ 465 (smtps, implicit TLS — основной, только │ -│ SASL-аутентификация) │ -│ 587 (submission, STARTTLS — опционально, │ -│ добавляется по необходимости, раздел 5) │ -│ 25 (исходящий в интернет) │ -│ 8080 (панель, за reverse-proxy) │ -└───────────────────────────────────────────────┘ -``` - -**Взаимодействие панели и Postfix:** панель пишет конфиг-файлы напрямую в локальную ФС (`/etc/postfix/...`) и вызывает `postfix reload` как sibling-процесс. Поскольку всё в одном контейнере, inotify-вотчер и shared volumes между контейнерами **не нужны**. - -**Milter-цепочка Postfix теперь состоит из двух шагов:** OpenDKIM (подпись) и journal-milter (логирование для журнала отправки, раздел 7.3) — оба сконфигурированы в `smtpd_milters`/`non_smtpd_milters`. Это остаётся в рамках «одного контейнера, трёх процессов под supervisord»: journal-milter — не новый процесс, а дополнительная роль внутри уже существующего panel-бинарника. - -**Порядок старта — обязателен, чтобы исключить ошибки соединения при холодном старте контейнера.** `priority=` в supervisord задаёт только порядок отправки команд на запуск, а не готовность сокетов — сама по себе она не гарантирует, что milter-сокеты уже слушают к моменту, когда Postfix попытается к ним подключиться. Поэтому: -1. `priority`: OpenDKIM стартует первым, затем panel (включая journal-milter listener). -2. **Postfix оборачивается стартовым скриптом**, который блокируется в цикле опроса (например, `test -S ` для unix-сокетов OpenDKIM/journal-milter, с интервалом и таймаутом — например, до 30 секунд), и запускает `postfix start-fg` только после того, как **оба** milter-сокета подтверждённо готовы. -3. **Если готовность не наступает в пределах таймаута** — скрипт завершается с ошибкой (не запускает Postfix «на всякий случай»), чтобы supervisord/Docker увидели явный сбой старта в логах, а не тихо продолжили с сервисом, который будет отклонять почту из-за недоступного milter'а. -4. Это решает только **проблему холодного старта** (первые секунды жизни контейнера) и не отменяет уже принятое поведение для **runtime**-сбоев после успешного старта: если OpenDKIM или journal-milter упадёт во время работы (не при старте), продолжает действовать fail-open, описанный в разделах 7.3/7.4 — это два разных сценария с разной правильной реакцией: на старте лучше подождать и не запускать Postfix «в дырявом» состоянии, а во время работы — не ронять уже идущий приём почты из-за вспомогательного компонента. - -**Требование к supervisord:** если любой из трёх процессов падает и не может быть перезапущен, контейнер должен завершаться (а не оставаться «живым» с мёртвым Postfix), чтобы Docker с `restart: unless-stopped` корректно его перезапустил. - -### 4.1. Доменная модель (ядро продукта) - -SelfPost — **мультидоменный** релей. Две связанные сущности: - -- **Отправляющий домен** — например, `example.com`. Имеет свой DKIM-ключ/селектор. -- **Приложение (учётная запись)** — SASL-пара логин/пароль, привязанная к конкретному домену. **У одного домена может быть несколько приложений** (например, `newsletter@example.com` и `alerts@example.com`, или просто «прод-сервер» и «staging-сервер» под один и тот же домен) — каждое со своим отдельным логином/паролем, но все они имеют право слать **только от имени того домена, к которому привязаны**, никогда от имени другого домена. - -**Режим адресов отправки — настройка на уровне приложения**, не домена (разные приложения одного домена могут выбирать разный режим независимо). При создании или редактировании приложения выбирается один из двух вариантов: - -1. **Любой адрес домена** — приложению разрешено указывать в From любой адрес в пределах своего домена (`*@example.com`). Удобно, когда одно приложение шлёт от разных адресов (например, `noreply@`, `alerts@`, `billing@`) без необходимости заводить под каждый отдельное приложение. -2. **Список конкретных адресов** — приложению разрешены только явно перечисленные From-адреса в пределах своего домена (например, только `alerts@example.com`). Более строгий вариант; попытка отправить с адреса не из списка отклоняется, даже если адрес принадлежит тому же домену. - -В обоих случаях адрес обязан принадлежать домену приложения — выйти за пределы своего домена нельзя ни в одном режиме. - -Для каждого добавленного домена система управляет: - -1. **DKIM-ключ и селектор** — свои для каждого домена (см. раздел 6). Общие для всех приложений этого домена. -2. **Одно или несколько приложений** — у каждого своя SASL-пара логин/пароль и свой режим адресов (см. раздел 5.1). Приложения одного домена независимы: удаление/перевыпуск/смена режима одного не затрагивает другие. -3. **Разрешённые From-адреса** — определяются режимом каждого приложения (см. выше). В любом случае приложение не может отправлять от имени чужого домена — попытка залогиниться кредами приложения из домена A и отправить с адреса домена B отклоняется (см. привязку в разделе 5.1). - -**Жизненный цикл домена (что делает панель при добавлении):** -- создаёт запись домена; -- генерирует DKIM-ключ + селектор для домена; -- показывает пользователю всё, что нужно внести в DNS для этого домена (DKIM TXT, а также напоминание про SPF/DMARC — см. раздел 10); -- **не создаёт приложение автоматически** — добавление первого (и последующих) приложений для домена делается отдельным действием, см. ниже. - -**Жизненный цикл приложения (что делает панель при добавлении приложения к домену):** -- генерирует SASL-пару логин/пароль (пароль показывается **один раз**, см. 7.2); -- принимает выбор режима адресов: «любой адрес домена» либо «список конкретных адресов» (если выбран список — принимает сами адреса, с валидацией, что каждый принадлежит домену приложения); -- регистрирует привязку в `smtpd_sender_login_maps` соответственно режиму — либо wildcard-запись `@example.com → логин`, либо отдельная запись на каждый разрешённый адрес `конкретный-адрес@example.com → логин` (несколько логинов и/или несколько записей могут вести к одному домену — это штатный случай, а не исключение); -- применяет изменения (`postfix reload`). - -**Изменение режима существующего приложения** — доступно как отдельное действие (переключение «любой адрес» ↔ «список», редактирование списка адресов), с пересборкой соответствующих записей в `smtpd_sender_login_maps` и `postfix reload`. - -**При удалении домена** — удаляются его DKIM-ключ **и все приложения**, привязанные к этому домену. -**При удалении приложения** — удаляются только его SASL-креды и все его записи в карте привязки; домен и остальные приложения не затрагиваются. - -Эта модель — не мультитенантность (панелью по-прежнему управляет один администратор); это управление несколькими отправляющими доменами одного владельца, каждый из которых может обслуживать несколько приложений с независимо настроенными правами на адреса отправки. - ---- - -## 5. Компонент: Postfix - -Конфигурация Postfix как исходящего релея: - -1. **Приём почты — порт 465 (`smtps`, implicit TLS) как основной.** TLS устанавливается сразу при подключении (wrapper mode, `smtpd_tls_wrappermode = yes` в `master.cf` для сервиса `smtps`), в отличие от STARTTLS — клиент не отправляет ни байта в открытом виде до установления TLS-сессии. **Только для аутентифицированных клиентов (SASL, см. 5.1).** `mynetworks` не используется как способ авторизации отправки — пускают креды, а не сеть. **Категорически недопустима конфигурация open relay.** - - **Порт 587 (submission, STARTTLS) — не входит в базовую поставку, добавляется по необходимости.** Многие SMTP-библиотеки в приложениях по умолчанию ожидают именно 587 — при необходимости совместимости сервис `submission` в `master.cf` включается тем же способом, что и `smtps` (аналогичная SASL/milter/rate-limit конфигурация, единственное отличие — механизм установления TLS, см. 5.1 п.2). Это осознанно не включено в основной сценарий, чтобы не открывать лишнюю поверхность без необходимости. -2. **Отправка** — напрямую в интернет (без `relayhost`), с MX-lookup и TLS (`smtp_tls_security_level = may` минимум). -3. **DKIM** — подключён как milter к OpenDKIM (через unix-socket или inet-socket внутри контейнера). Подпись — per-domain (раздел 6). -4. **Ограничения** — `smtpd_recipient_restrictions` запрещает релей для неаутентифицированных (`permit_sasl_authenticated`, `reject_unauth_destination`). Обязательна привязка отправителя к логину (раздел 5.1). -5. **Базовый rate-limiting по client IP** — встроенный механизм Postfix (демон `anvil`): `smtpd_client_message_rate_limit` + `anvil_rate_time_unit` ограничивают число сообщений с одного client IP за окно времени. Работает на уровне приёма соединения, **независимо от journal-milter** — это базовый backstop, который продолжает действовать даже при сбое milter'а (см. раздел 7.4). Значения — конфигурируемые (раздел 8), по умолчанию консервативные, особенно уместные на этапе прогрева IP (раздел 10). Применяется одинаково к обоим портам приёма (465 и, если включён, 587). -6. Домены, приложения (креды) и привязки — управляются панелью (файлы конфигурации, которые панель редактирует, с последующим `postfix reload`). - -Базовый образ-ориентир для понимания подхода: `boky/postfix` (env-driven relay), но здесь конфигурация собирается под DIY-отправку с SASL-аутентификацией и мультидоменом, а не под relay через провайдера. - -### 5.1. SASL-аутентификация и привязка приложений к домену - -1. **Бэкенд SASL — Cyrus SASL с локальной базой `sasldb2`** (лёгкий, без Dovecot и без полноценного mail-стека). Панель создаёт/удаляет учётные записи приложений (эквивалент `saslpasswd2`), Postfix проверяет их через `smtpd_sasl_auth_enable = yes`. -2. **TLS обязателен для аутентификации на обоих портах, но механизм различается:** - - На **465 (`smtps`)** — TLS уже установлен к моменту аутентификации по определению wrapper mode, отдельная настройка не требуется сверх `smtpd_tls_wrappermode = yes`. - - На **587 (`submission`, если включён)** — `smtpd_tls_auth_only = yes`, чтобы креды не могли быть отправлены до STARTTLS по незашифрованному каналу. - В обоих случаях результат одинаков: креды никогда не передаются в открытом виде. -3. **Привязка логина к разрешённым адресам (критично).** Настраивается `smtpd_sender_login_maps` + `reject_sender_login_mismatch` в `smtpd_sender_restrictions`. Без этого любой валидный аккаунт сможет слать от имени любого домена — это дыра, а не опция. Формат записи в карте зависит от режима адресов, выбранного для приложения (см. раздел 4.1): - - **режим «любой адрес домена»** — wildcard-запись `@example.com логин` (Postfix поддерживает доменные wildcard-записи в `sender_login_maps`); - - **режим «список конкретных адресов»** — отдельная запись на каждый разрешённый адрес: `alerts@example.com логин`, `noreply@example.com логин` и т.д. - Обе формы одновременно допустимы в карте для разных приложений/доменов. Панель поддерживает карту в актуальном состоянии при добавлении/удалении/изменении доменов и приложений, генерируя нужный тип записи в зависимости от режима. -4. **Многие-ко-одному.** У одного домена может быть несколько приложений (логинов), каждое со своим независимым режимом адресов; ни одно приложение не может получить право слать от имени домена, к которому оно не привязано. - -### 5.2. TLS-сертификаты для Postfix - -**SelfPost всегда работает за reverse-proxy, и именно reverse-proxy отвечает за выпуск, хранение и автообновление TLS-сертификатов** (ACME/Let's Encrypt). Проект **не** содержит собственного ACME-клиента и не выпускает сертификаты сам — он только потребляет готовые. - -Модель простая: - -1. **Источник** — reverse-proxy (Caddy/Traefik) кладёт PEM-файлы (цепочка + приватный ключ) для почтового hostname в директорию на хосте, смонтированную в контейнер SelfPost **bind mount'ом, только на чтение** (та же схема, что и для остального персистентного состояния — раздел 9). Пути к файлам задаются переменными окружения (см. раздел 8). -2. **Потребление** — Postfix настроен читать эти файлы (`smtpd_tls_cert_file` / `smtpd_tls_key_file`) для TLS на 465 (`smtps`, основной), 587 (`submission`, если включён) и opportunistic TLS на порту 25. Один и тот же сертификат обслуживает оба порта приёма. -3. **Соответствие hostname** — CN/SAN сертификата должен совпадать с HELO-hostname (`SELFPOST_HOSTNAME`) и PTR/rDNS. Проще всего использовать общий hostname для панели и почты, тогда это один и тот же сертификат от reverse-proxy. -4. **Обновление** — когда reverse-proxy обновляет сертификат, файлы в смонтированной директории меняются, и Postfix нужно перечитать их через `postfix reload`. Чтобы деплой оставался простым, применяется **периодический `postfix reload` (например, раз в сутки)** — этого достаточно, т.к. сертификаты обновляются раз в ~2-3 месяца, а суточная задержка применения некритична. Реализовать как отдельную периодическую задачу под supervisord (или cron внутри контейнера). Inotify-вотчер на файл сертификата допустим как альтернатива, но периодический reload проще и надёжнее. - -Персистентность сертификатов (хранение, ACME-account key) — **ответственность reverse-proxy**, не SelfPost (см. раздел 9). Контейнер SelfPost хранит сертификаты только как read-only mount и не заботится об их выживании при рестарте. - ---- - -## 6. Компонент: OpenDKIM - -DKIM-подпись — **строго per-domain**. Каждый отправляющий домен имеет собственную пару ключей и селектор. - -1. **Генерация при добавлении домена** — панель создаёт для нового домена пару DKIM-ключей и селектор. Все ключи должны **переживать перезапуск контейнера** (см. раздел 9). -2. **Подпись** — исходящая почта каждого домена подписывается его собственным ключом. OpenDKIM настраивается с `KeyTable` + `SigningTable`, которые панель поддерживает в актуальном состоянии (запись на каждый домен), с последующим reload OpenDKIM. -3. **Показ DNS-записи per-domain** — панель показывает публичную часть ключа в формате DNS TXT-записи **для каждого домена отдельно**, чтобы пользователь внёс её в DNS соответствующего домена. -4. **Селектор** — на домен; может быть общим значением по умолчанию (например, `selfpost`) для всех доменов, т.к. селекторы живут в пространстве имён каждого домена и не конфликтуют. Значение по умолчанию — конфигурируемое. -5. **При удалении домена** — его ключ и записи в `KeyTable`/`SigningTable` удаляются, OpenDKIM перечитывается. - ---- - -## 7. Компонент: Панель управления - -### 7.1. Технологический стек - -- **Язык: Go.** Обоснование: статический бинарник (минимум рантайм-зависимостей в образе), компилятор ловит ошибки до развёртывания (важно, т.к. код пишет ИИ-агент), простая упаковка. -- **Зависимости — минимальные.** Приоритет стандартной библиотеки: `net/http`, `html/template`, `os/exec`. Внешние зависимости допускаются только для того, что не покрыто stdlib (например, bcrypt из `golang.org/x/crypto`). Каждая внешняя зависимость должна быть обоснована. -- HTTP-сервер слушает `:8080` внутри контейнера (HTTPS терминируется reverse-proxy, см. раздел 10). - -**Front-end:** - -- **Server-rendered** через `html/template`. Никакого SPA, JS-фреймворка (React/Vue/Svelte) или build-шага (webpack/vite/npm). Весь UI отдаётся тем же Go-бинарником — единая точка деплоя, ноль `node_modules`. Это требование, а не рекомендация: оно прямо вытекает из приоритета простоты поддержки. -- **HTMX** — единственная фронтенд-зависимость, подключается одним статическим `