From 61f525e2d73f07a17598829471e7464b3611b2ca Mon Sep 17 00:00:00 2001 From: Mikhail Yenuchenko Date: Wed, 15 Jul 2026 21:23:19 +0300 Subject: [PATCH] Phase 10: deployment (Apache compose + proxy fragments, CI release) + docs - deploy/docker-compose.yml: pinned-tag ghcr image, hardened (cap_drop ALL + minimal cap_add, no-new-privileges, panel bound to 127.0.0.1 only). Apache itself runs on the host (spec 10.5), fragment at deploy/apache/. - Alternative reverse-proxy fragments: nginx (+certbot sidecar), Caddy (automatic ACME), Traefik (+acme.json PEM extraction script). - .github/workflows/release.yml: tag-triggered ghcr.io publish, version piped from the git tag into both the binary ldflags and the image tag (spec 10.1). - Closed a gap from Phase 1: logrotate was installed but never invoked; wired up build/logrotate-mail.conf + logrotate-loop.sh + a supervisor program (copytruncate, since postlogd holds mail.log open with nothing to signal on rotation). - README rewritten: site requirements checklist, reverse-proxy comparison, DNS setup (server- vs domain-level), IP warmup, backup/restore vs domain export/import, fixed-tag rationale, machine requirements. Co-Authored-By: Claude Sonnet 5 --- .github/workflows/release.yml | 47 ++++++++ README.md | 135 +++++++++++++++++++++- build/Dockerfile | 6 +- build/logrotate-loop.sh | 23 ++++ build/logrotate-mail.conf | 9 ++ build/supervisord.conf | 14 +++ deploy/.env.example | 17 +++ deploy/apache/selfpost-vhost.conf | 41 +++++++ deploy/caddy/Caddyfile | 3 + deploy/caddy/docker-compose.caddy.yml | 49 ++++++++ deploy/docker-compose.yml | 80 +++++++++++++ deploy/nginx/docker-compose.nginx.yml | 55 +++++++++ deploy/nginx/nginx.conf.example | 32 +++++ deploy/traefik/docker-compose.traefik.yml | 52 +++++++++ deploy/traefik/extract-cert.sh | 28 +++++ docs/progress.md | 11 +- 16 files changed, 594 insertions(+), 8 deletions(-) create mode 100644 .github/workflows/release.yml create mode 100644 build/logrotate-loop.sh create mode 100644 build/logrotate-mail.conf create mode 100644 deploy/.env.example create mode 100644 deploy/apache/selfpost-vhost.conf create mode 100644 deploy/caddy/Caddyfile create mode 100644 deploy/caddy/docker-compose.caddy.yml create mode 100644 deploy/docker-compose.yml create mode 100644 deploy/nginx/docker-compose.nginx.yml create mode 100644 deploy/nginx/nginx.conf.example create mode 100644 deploy/traefik/docker-compose.traefik.yml create mode 100644 deploy/traefik/extract-cert.sh diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml new file mode 100644 index 0000000..556d10c --- /dev/null +++ b/.github/workflows/release.yml @@ -0,0 +1,47 @@ +name: release + +# Publishes an immutable, version-tagged image on ghcr.io (spec 10.1). +# Ordinary commits do not publish anything — only a pushed tag matching +# vX.Y.Z does. That tag is the single source the version comes from: it goes +# into both the image tag and the panel binary's -ldflags version, so the two +# can never drift apart (the invariant restore's version check in spec 7.5.A +# depends on). +on: + push: + tags: + - "v[0-9]+.[0-9]+.[0-9]+" + +permissions: + contents: read + packages: write + +jobs: + build-and-push: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - name: Derive version from tag + id: version + run: echo "version=${GITHUB_REF_NAME#v}" >> "$GITHUB_OUTPUT" + + - uses: docker/setup-qemu-action@v3 + - uses: docker/setup-buildx-action@v3 + + - name: Log in to ghcr.io + uses: docker/login-action@v3 + with: + registry: ghcr.io + username: ${{ github.actor }} + password: ${{ secrets.GITHUB_TOKEN }} + + - name: Build and push + uses: docker/build-push-action@v6 + with: + context: . + file: build/Dockerfile + platforms: linux/amd64,linux/arm64 + build-args: | + VERSION=${{ steps.version.outputs.version }} + push: true + tags: ghcr.io/${{ github.repository }}:${{ steps.version.outputs.version }} diff --git a/README.md b/README.md index 52bfd4f..16cc38c 100644 --- a/README.md +++ b/README.md @@ -13,12 +13,137 @@ does not receive mail, provide mailboxes, or offer webmail. > for the full requirements and [docs/implementation-plan.md](docs/implementation-plan.md) > for the phased build plan. -## Requirements (site prerequisites) +## Requirements (site checklist) -SelfPost assumes the host already provides the conditions for sending from your -own IP — an unblocked outbound port 25, a static IP, configurable PTR/rDNS and a -reasonable IP reputation. Providing these is the operator's job, not a feature of -SelfPost. Detailed deployment docs land in a later phase. +Providing these is the operator's job, not a feature of SelfPost — the panel +can't fix a blocked port or a missing PTR record for you. + +- [ ] A static IP address. +- [ ] Outbound TCP port 25 unblocked (many consumer/cloud hosts block it by + default — check with your provider before anything else). +- [ ] PTR/rDNS for that IP set to your mail hostname (see [DNS setup](#dns-setup)). +- [ ] Reasonable starting IP reputation — a fresh IP still needs [warmup](#ip-warmup). +- [ ] A reverse proxy in front of the panel (see [Reverse proxy](#reverse-proxy-mandatory)) — SelfPost never terminates HTTPS itself. +- [ ] Docker + Compose v2 on the host. + +## Quick start + +```sh +mkdir -p selfpost && cd selfpost +curl -O https://raw.githubusercontent.com/mixeme/selfpost/main/deploy/docker-compose.yml +curl -O https://raw.githubusercontent.com/mixeme/selfpost/main/deploy/.env.example +mv .env.example .env # then edit SELFPOST_HOSTNAME etc. +docker compose up -d +``` + +This starts SelfPost alone; it assumes Apache is already installed on the host +as the reverse proxy (see below) and expects certificates at `./certs`. The +first log line (`docker compose logs -f`) prints the one-time setup link — +open it to create the admin account. + +## Reverse proxy (mandatory) + +SelfPost's panel speaks plain HTTP and never terminates TLS itself — a reverse +proxy in front of it is not optional. The proxy is also the project's only +source of TLS certificates: whatever it obtains via ACME/Let's Encrypt gets +bind-mounted **read-only** into the SelfPost container, and Postfix uses those +same PEM files for TLS on 465 (and 587, if enabled). If the panel and the mail +service share one hostname — the common case — it's genuinely one certificate +serving both. + +SelfPost isn't tied to a specific proxy; pick whichever fits your host: + +| Proxy | Where certs live | Fragment | +|---|---|---| +| **Apache** (default/recommended) | Host disk, via the certbot Apache plugin — PEM files ready to bind-mount, no extraction step. | [deploy/apache/selfpost-vhost.conf](deploy/apache/selfpost-vhost.conf) | +| nginx | Host disk, via a certbot sidecar container — same PEM-ready shape as Apache. | [deploy/nginx/](deploy/nginx/) | +| Caddy | Automatic ACME, zero extra containers — simplest, but its on-disk cert path is versioned internal layout, not a stable API; verify it for the Caddy version you run. | [deploy/caddy/](deploy/caddy/) | +| Traefik | Bundled inside `acme.json` — needs a small extraction script to produce standalone PEM files. | [deploy/traefik/](deploy/traefik/) | + +Apache is the recommended default because the certbot Apache plugin already +writes plain `fullchain.pem`/`privkey.pem` files to a predictable path with no +extra moving parts between "certificate issued" and "Postfix can read it." + +## DNS setup + +Two different scopes — don't confuse them: + +**Server level (once, for the machine itself):** +- **PTR/rDNS** for the server's IP, pointing at its mail hostname. Most + receiving mail servers weigh this heavily; get it from whoever assigns the IP + (hosting provider's panel/support), not from your own DNS zone. + +**Domain level (for *every* sending domain you add in the panel):** +- **SPF** — a TXT record on the domain authorizing this server to send on its + behalf (e.g. `v=spf1 a mx ip4: -all`, adjusted to your setup). +- **DKIM** — a TXT record with the exact value the panel shows on that + domain's page (`domain page → DKIM TXT record`), one selector per domain. +- **DMARC** — a `_dmarc` TXT record (even a conservative `p=none` starts + building reporting/reputation history). + +Skipping any of the three per-domain records is the single most common reason +mail lands in spam even though SelfPost delivered it correctly — DKIM passing +doesn't help if SPF/DMARC are absent. **Whenever you add a new domain in the +panel, add its DNS records at the same time**, not later. + +## IP warmup + +A brand-new IP has no sending history, so receiving servers are cautious with +it regardless of how correct your DKIM/SPF/DMARC are. Start with low volume to +a domain, increase gradually over days/weeks rather than sending everything on +day one, and check the IP against major blocklists (Spamhaus and similar) +before and during warmup. This is inherent to how mail reputation works on the +public internet, not something SelfPost's configuration can shortcut. + +## Backup, restore, and moving a single domain + +Two related but distinct operations — spec 7.5: + +- **Full backup** (whole `/data`: SQLite, all domains' DKIM keys, all + applications' SASL credentials, `manifest.json` with the version that + created it): panel button (dashboard → *Backup & migration*), or from the + host: + ```sh + docker exec selfpost-backup > selfpost-backup.tar.gz + ``` + **Restore** means unpacking that archive into a fresh `/data` bind mount and + starting a container of the **exact same image version** that created it — + SelfPost refuses to start otherwise and tells you which tag to use. This is + why the compose file below pins a fixed tag rather than `:latest`: without a + known version, there'd be no way to tell which image restoring a given + backup actually requires. + +- **Export/import a single domain** (dashboard → domain page → *Export + domain*): moves one domain — its DKIM key and its applications' **working** + SASL passwords — to a different SelfPost instance without regenerating + anything, so DNS (the DKIM TXT record) doesn't need to change. Unlike a full + restore, this works across different hostnames/instances. + +Both files are **secrets** — they contain the admin password hash (full +backup) or working application credentials (domain export) in the clear or in +directly reversible form. Treat them like any other credential material: +encrypt at rest, restrict who can read them, don't email them around. + +## Fixed image tag + +`deploy/docker-compose.yml` pins an explicit version (`ghcr.io/mixeme/selfpost:X.Y.Z`), +deliberately never `:latest`. This is a direct consequence of the backup +version check above: the panel binary's embedded version and the image tag +that produced it are the same value by construction (the release CI stamps +both from one git tag — see `.github/workflows/release.yml`), so pinning the +tag is what makes "restore into the same version" a checkable fact rather than +a guess. Upgrade by bumping the tag deliberately, not by riding a moving +target. + +## Machine requirements + +Rough guide, not a hard floor: **1 vCPU**, **512MB–1GB RAM** (the stack — three +processes plus SQLite — idles around 100–150MB; the rest is headroom for +backups, log-tailer/retention sweeps and concurrent TLS handshakes coinciding), +**8–10GB disk**. Disk usage grows mainly from the send log (bounded by +`SEND_LOG_RETENTION_DAYS`, default 90) and the rotated `mail.log` (kept 14 days +in-image), not from the application itself. On boxes with little RAM, a small +swap file is cheap insurance against those occasional coincident spikes. ## Repository diff --git a/build/Dockerfile b/build/Dockerfile index 0f41e96..3810b8e 100644 --- a/build/Dockerfile +++ b/build/Dockerfile @@ -76,15 +76,17 @@ COPY --from=build /out/panel /usr/local/bin/panel COPY --from=build /out/selfpost-backup /usr/local/bin/selfpost-backup COPY build/opendkim.conf /etc/opendkim.conf +COPY build/logrotate-mail.conf /etc/logrotate.d/mail COPY build/postfix-wrapper.sh /usr/local/bin/postfix-wrapper.sh COPY build/postfix-config.sh /usr/local/bin/postfix-config.sh COPY build/postfix-cert-reload.sh /usr/local/bin/postfix-cert-reload.sh +COPY build/logrotate-loop.sh /usr/local/bin/logrotate-loop.sh COPY build/crashexit.py /usr/local/bin/crashexit.py COPY build/entrypoint.sh /usr/local/bin/entrypoint.sh COPY build/supervisord.conf /etc/supervisor/supervisord.conf RUN chmod +x /usr/local/bin/postfix-wrapper.sh /usr/local/bin/postfix-config.sh \ - /usr/local/bin/postfix-cert-reload.sh /usr/local/bin/crashexit.py \ - /usr/local/bin/entrypoint.sh + /usr/local/bin/postfix-cert-reload.sh /usr/local/bin/logrotate-loop.sh \ + /usr/local/bin/crashexit.py /usr/local/bin/entrypoint.sh # Published submission ports: 465 (smtps, primary) and 587 (submission, optional) # plus the panel on 8080. Outbound delivery dials remote MXs on 25 as a client, diff --git a/build/logrotate-loop.sh b/build/logrotate-loop.sh new file mode 100644 index 0000000..74f4af0 --- /dev/null +++ b/build/logrotate-loop.sh @@ -0,0 +1,23 @@ +#!/bin/sh +# Periodic logrotate for /var/log/mail.log (spec 9, 10). Postfix's maillog_file +# is written by postlogd, which keeps the file open for the life of the +# process — there is no daemon to signal on rotation, so the logrotate.d config +# uses copytruncate (a brief truncation race can drop the last few in-flight +# lines, which is an acceptable trade for not having to reload Postfix on every +# rotation). +# +# logrotate itself only rotates once the configured "daily" period has elapsed +# (tracked in /var/lib/logrotate/status), so it is safe to invoke this more +# often than daily — polling merely bounds how late a legitimate rotation runs. +set -eu + +INTERVAL="${LOGROTATE_INTERVAL_SECONDS:-21600}" + +while true; do + if logrotate /etc/logrotate.d/mail; then + : + else + echo "logrotate-loop: logrotate failed, will retry after ${INTERVAL}s" >&2 + fi + sleep "${INTERVAL}" +done diff --git a/build/logrotate-mail.conf b/build/logrotate-mail.conf new file mode 100644 index 0000000..60eb1c7 --- /dev/null +++ b/build/logrotate-mail.conf @@ -0,0 +1,9 @@ +/var/log/mail.log { + daily + rotate 14 + missingok + notifempty + compress + delaycompress + copytruncate +} diff --git a/build/supervisord.conf b/build/supervisord.conf index 18e13bd..9b15635 100644 --- a/build/supervisord.conf +++ b/build/supervisord.conf @@ -103,6 +103,20 @@ stdout_logfile_maxbytes=0 stderr_logfile=/dev/stderr stderr_logfile_maxbytes=0 +; Periodic logrotate for /var/log/mail.log (spec 9, 10: daily, 7-14 files kept +; in the image). Runs as root so logrotate can read/rotate the log; never exits +; non-zero, so it neither trips the crashexit listener nor needs restarting. +[program:logrotate] +command=/usr/local/bin/logrotate-loop.sh +priority=400 +autostart=true +autorestart=true +startsecs=0 +stdout_logfile=/dev/stdout +stdout_logfile_maxbytes=0 +stderr_logfile=/dev/stderr +stderr_logfile_maxbytes=0 + [eventlistener:crashexit] command=/usr/local/bin/crashexit.py events=PROCESS_STATE_FATAL diff --git a/deploy/.env.example b/deploy/.env.example new file mode 100644 index 0000000..1c5f942 --- /dev/null +++ b/deploy/.env.example @@ -0,0 +1,17 @@ +# Copy to .env next to docker-compose.yml and fill in. + +# Hostname used both for the panel's TLS vhost and for Postfix's SASL realm +# (SASL_REALM defaults to this) and myhostname. Must match the certificate +# your reverse proxy obtains. +SELFPOST_HOSTNAME=mail.example.com + +# Set to true to also enable RFC 6409 submission (587/STARTTLS) alongside the +# primary 465/smtps listener. +SUBMISSION_ENABLE=false + +# Level-1 backstop rate limit (anvil) — see README "Rate limiting". +RATE_LIMIT_MESSAGES_PER_IP=100 +RATE_LIMIT_WINDOW_SECONDS=3600 + +# Days of send-log history kept before the background sweep deletes rows. +SEND_LOG_RETENTION_DAYS=90 diff --git a/deploy/apache/selfpost-vhost.conf b/deploy/apache/selfpost-vhost.conf new file mode 100644 index 0000000..556d6a1 --- /dev/null +++ b/deploy/apache/selfpost-vhost.conf @@ -0,0 +1,41 @@ +# Example host Apache vhost for the SelfPost panel (spec 10.3, default +# scenario). Install on the host running Apache — NOT inside the SelfPost +# container. Requires mod_ssl and mod_proxy/mod_proxy_http enabled +# (`a2enmod ssl proxy proxy_http`). +# +# Certificates: obtain with the Apache certbot plugin against THIS vhost +# (`certbot --apache -d mail.example.com`). Certbot edits this file in place +# to add the SSLCertificateFile/SSLCertificateKeyFile directives and a +# :80 -> :443 redirect vhost, and renews in the background via its own timer. +# The resulting PEM files land at +# /etc/letsencrypt/live/mail.example.com/{fullchain,privkey}.pem on the host +# — bind-mount that directory read-only into the SelfPost container (see +# ../docker-compose.yml, the `./certs:/etc/postfix/tls:ro` volume) so Postfix +# can present the very same certificate on 465/587. One certificate, two +# consumers (spec 10 p.4) — no separate cert just for the panel. + + + ServerName mail.example.com + + # Filled in by `certbot --apache`; shown here for clarity. + # SSLEngine on + # SSLCertificateFile /etc/letsencrypt/live/mail.example.com/fullchain.pem + # SSLCertificateKeyFile /etc/letsencrypt/live/mail.example.com/privkey.pem + + ProxyPreserveHost On + ProxyPass "/" "http://127.0.0.1:8080/" + ProxyPassReverse "/" "http://127.0.0.1:8080/" + + # The panel is plain HTTP behind the proxy; it never terminates TLS + # itself (spec 10 p.2). Cookies are still marked Secure by the panel + # (spec 7.6.6) because the browser only ever sees this HTTPS vhost. + + + + ServerName mail.example.com + # certbot's http-01 challenge needs this on plain :80; everything else + # redirects to HTTPS once certbot adds the RewriteRule/Redirect block. + RewriteEngine On + RewriteCond %{REQUEST_URI} !^/\.well-known/acme-challenge/ + RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [R=301,L] + diff --git a/deploy/caddy/Caddyfile b/deploy/caddy/Caddyfile new file mode 100644 index 0000000..99757ce --- /dev/null +++ b/deploy/caddy/Caddyfile @@ -0,0 +1,3 @@ +mail.example.com { + reverse_proxy selfpost:8080 +} diff --git a/deploy/caddy/docker-compose.caddy.yml b/deploy/caddy/docker-compose.caddy.yml new file mode 100644 index 0000000..50c8088 --- /dev/null +++ b/deploy/caddy/docker-compose.caddy.yml @@ -0,0 +1,49 @@ + +# Alternative reverse-proxy: Caddy (spec 10.3) — the simplest option, fully +# automatic ACME with no separate certbot container. Caddy writes certificates +# as plain PEM under its own data directory, which this fragment bind-mounts +# to the host so SelfPost can read the same files. +# +# NOTE (spec 10.3 explicitly flags this): Caddy's on-disk cert path includes +# the ACME CA's name as a path segment, e.g. +# /caddy/certificates/acme-v02.api.letsencrypt.org-directory//.crt +# This has been stable across recent Caddy releases but is an internal +# storage detail, not a documented public API — VERIFY the exact path against +# the Caddy version you deploy (`docker compose exec caddy find /data/caddy/certificates -name '*.crt'`) +# before wiring it into the volume mount below. +# +# Merge with the base file — run this from the deploy/ directory (Compose +# resolves every relative path in both files against the directory of the +# FIRST -f file, i.e. deploy/, which is why paths below are ./caddy/...): +# +# docker compose -f docker-compose.yml -f caddy/docker-compose.caddy.yml up -d + +services: + selfpost: + ports: !override + - "465:465" + - "587:587" + environment: + # Caddy names certificate files after the hostname, not + # fullchain.pem/privkey.pem like certbot — override the base file's + # paths to match. + TLS_CERT_FILE: /etc/postfix/tls/.crt + TLS_KEY_FILE: /etc/postfix/tls/.key + volumes: !override + - ./data:/data + # Replace with your actual mail/panel hostname, matching + # the Caddyfile below and SELFPOST_HOSTNAME. + - ./caddy/caddy-data/caddy/certificates/acme-v02.api.letsencrypt.org-directory/:/etc/postfix/tls:ro + + caddy: + image: caddy:2 + restart: unless-stopped + depends_on: + - selfpost + ports: + - "80:80" + - "443:443" + volumes: + - ./caddy/Caddyfile:/etc/caddy/Caddyfile:ro + - ./caddy/caddy-data:/data + - ./caddy/caddy-config:/config diff --git a/deploy/docker-compose.yml b/deploy/docker-compose.yml new file mode 100644 index 0000000..4bc1842 --- /dev/null +++ b/deploy/docker-compose.yml @@ -0,0 +1,80 @@ + +# SelfPost — default deployment, reverse-proxy = Apache (spec 10, 10.5). +# +# This file only runs SelfPost itself. Apache is assumed to already be +# installed on the HOST (the target audience for this project typically runs +# Apache there already — spec 10.5) and reverse-proxies HTTPS for the panel; +# it is not containerised here. See ../apache/selfpost-vhost.conf for a ready +# vhost fragment, and the "Reverse proxy" section of ../../README.md for the +# certbot steps that produce the PEM files this compose file mounts. +# +# Usage: +# 1. Copy this file (and .env.example as .env) next to your own ./data and +# ./certs directories, or adjust the paths below. +# 2. Fill in .env (hostname, at least one strong TLS_CERT/KEY path). +# 3. docker compose up -d +# +# The image tag below is FIXED on purpose (spec 10 p.10, 7.5.A): backup +# restore compares the manifest version against the running binary's version, +# so ":latest" would make that check meaningless. Bump the tag deliberately +# when you want to upgrade. + +services: + selfpost: + image: ghcr.io/mixeme/selfpost:1.0.0 + restart: unless-stopped + environment: + SELFPOST_HOSTNAME: "${SELFPOST_HOSTNAME:?set the mail/panel hostname, e.g. mail.example.com}" + # Path Postfix reads inside the container — matches the certs bind mount + # below. Point these at your reverse-proxy's PEM output (spec 10 p.2). + TLS_CERT_FILE: /etc/postfix/tls/fullchain.pem + TLS_KEY_FILE: /etc/postfix/tls/privkey.pem + # Set to true to also publish RFC 6409 submission (587/STARTTLS) + # alongside the primary 465/smtps listener (spec 5). + SUBMISSION_ENABLE: "${SUBMISSION_ENABLE:-false}" + # Level-1 backstop rate limit (anvil, spec 5.5, 7.4); per-domain/app + # limits (level 2) are configured later from the panel itself. + RATE_LIMIT_MESSAGES_PER_IP: "${RATE_LIMIT_MESSAGES_PER_IP:-100}" + RATE_LIMIT_WINDOW_SECONDS: "${RATE_LIMIT_WINDOW_SECONDS:-3600}" + # How long the send log keeps rows before the background sweep deletes + # them (spec 7.3, 9) — the main driver of /data growth over time. + SEND_LOG_RETENTION_DAYS: "${SEND_LOG_RETENTION_DAYS:-90}" + volumes: + # All persistent state lives under /data (spec 9): SQLite DB, DKIM keys, + # sasldb2, sender map, setup token. Back this up (panel button or the + # selfpost-backup CLI) before you touch it directly. + - ./data:/data + # Read-only: SelfPost only ever reads certificates, never manages them + # (spec 10 p.2). Point this at wherever your reverse-proxy/certbot + # writes PEM files, e.g. /etc/letsencrypt on the host. + - ./certs:/etc/postfix/tls:ro + ports: + # 465 (smtps, primary) and optionally 587 (submission/STARTTLS) are + # published directly — mail traffic bypasses Apache entirely, it only + # ever proxies the panel's HTTP(S) (spec 10 p.2-3). The panel itself + # (8080) is intentionally NOT published here: Apache reaches it over + # the host network at 127.0.0.1:8080 (see the vhost fragment), so the + # panel is never directly reachable from the internet without TLS. + - "465:465" + - "587:587" + - "127.0.0.1:8080:8080" + # Hardening (spec 10 p.6). SelfPost's entrypoint still needs to run as + # root very briefly to fix /data ownership and normalise permissions + # under the shared `selfpost` group (see build/entrypoint.sh) before + # supervisord drops the panel to an unprivileged user — so this cannot be + # `user: panel` or a fully read-only rootfs without breaking that startup + # self-healing. What IS applied: no privilege escalation past what the + # image already grants, and every Linux capability dropped except the + # small set Postfix/OpenDKIM genuinely need (binding <1024, chown/setuid + # during startup, and DAC overrides for cross-user file access within the + # shared group). + security_opt: + - no-new-privileges:true + cap_drop: + - ALL + cap_add: + - NET_BIND_SERVICE + - CHOWN + - SETUID + - SETGID + - DAC_OVERRIDE diff --git a/deploy/nginx/docker-compose.nginx.yml b/deploy/nginx/docker-compose.nginx.yml new file mode 100644 index 0000000..83e7892 --- /dev/null +++ b/deploy/nginx/docker-compose.nginx.yml @@ -0,0 +1,55 @@ + +# Alternative reverse-proxy: nginx (spec 10.3). PEM files land on the host +# exactly like the Apache+certbot scenario — nginx and certbot are close +# cousins here, both writing/reading plain PEM under /etc/letsencrypt. +# +# This is a FRAGMENT, not a full replacement for ../docker-compose.yml: it +# adds an nginx + certbot pair and removes the panel's host port publish (nginx +# takes over 80/443 and proxies to the panel over the compose network +# instead). Merge it with the base file — run this from the deploy/ directory +# (Compose resolves every relative path in both files against the directory of +# the FIRST -f file, i.e. deploy/, which is why paths below are ./nginx/...): +# +# docker compose -f docker-compose.yml -f nginx/docker-compose.nginx.yml up -d +# +# First-run certificate issuance (webroot method, before nginx has a cert to +# serve — run once): +# docker compose -f docker-compose.yml -f nginx/docker-compose.nginx.yml \ +# run --rm certbot certonly --webroot -w /var/www/certbot \ +# -d mail.example.com --email you@example.com --agree-tos --no-eff-email + +services: + selfpost: + ports: !override + - "465:465" + - "587:587" + # No host publish for 8080 here: nginx reaches it over the compose + # network at selfpost:8080 instead (see nginx.conf.example). + volumes: !override + - ./data:/data + # Same host directory certbot below writes into — plain bind mount, + # no named volume, so the PEM files are as directly inspectable as in + # the Apache scenario (spec 10.3). + - ./nginx/certbot-etc/live/mail.example.com:/etc/postfix/tls:ro + + nginx: + image: nginx:1.27 + restart: unless-stopped + depends_on: + - selfpost + ports: + - "80:80" + - "443:443" + volumes: + - ./nginx/nginx.conf.example:/etc/nginx/conf.d/selfpost.conf:ro + - ./nginx/certbot-etc:/etc/letsencrypt:ro + - ./nginx/certbot-www:/var/www/certbot:ro + + certbot: + image: certbot/certbot:latest + volumes: + - ./nginx/certbot-etc:/etc/letsencrypt + - ./nginx/certbot-www:/var/www/certbot + # Renewal twice a day is certbot's own recommended cadence; it no-ops + # until a certificate is within its renewal window. + entrypoint: sh -c 'trap exit TERM; while :; do certbot renew; sleep 12h & wait $${!}; done' diff --git a/deploy/nginx/nginx.conf.example b/deploy/nginx/nginx.conf.example new file mode 100644 index 0000000..a5e9ee3 --- /dev/null +++ b/deploy/nginx/nginx.conf.example @@ -0,0 +1,32 @@ +# nginx vhost for the SelfPost panel (spec 10.3). Proxies HTTPS to the panel +# over the compose network; mail (465/587) is published directly by the +# selfpost container and never touches nginx. + +server { + listen 80; + server_name mail.example.com; + + location /.well-known/acme-challenge/ { + root /var/www/certbot; + } + + location / { + return 301 https://$host$request_uri; + } +} + +server { + listen 443 ssl; + http2 on; + server_name mail.example.com; + + ssl_certificate /etc/letsencrypt/live/mail.example.com/fullchain.pem; + ssl_certificate_key /etc/letsencrypt/live/mail.example.com/privkey.pem; + + location / { + proxy_pass http://selfpost:8080; + proxy_set_header Host $host; + proxy_set_header X-Forwarded-Proto $scheme; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + } +} diff --git a/deploy/traefik/docker-compose.traefik.yml b/deploy/traefik/docker-compose.traefik.yml new file mode 100644 index 0000000..79edc9c --- /dev/null +++ b/deploy/traefik/docker-compose.traefik.yml @@ -0,0 +1,52 @@ + +# Alternative reverse-proxy: Traefik (spec 10.3). Traefik does NOT write plain +# PEM files — everything (cert + key) lives bundled inside acme.json. SelfPost +# needs separate PEM files for Postfix, so an extraction step is required; see +# extract-cert.sh in this directory and the cron/systemd-timer note below. +# +# Merge with the base file — run this from the deploy/ directory (Compose +# resolves every relative path in both files against the directory of the +# FIRST -f file, i.e. deploy/, which is why paths below are ./traefik/...): +# +# docker compose -f docker-compose.yml -f traefik/docker-compose.traefik.yml up -d + +services: + selfpost: + ports: !override + - "465:465" + - "587:587" + volumes: !override + - ./data:/data + # Populated by extract-cert.sh from traefik's acme.json — see below. + - ./traefik/extracted-certs:/etc/postfix/tls:ro + labels: + # Traefik only proxies the panel; it never sees the mail ports. + # These live on the selfpost service (not traefik's) because the + # docker provider reads routing labels off the container being routed + # to, not off traefik itself. + - traefik.enable=true + - traefik.http.routers.selfpost.rule=Host(`mail.example.com`) + - traefik.http.routers.selfpost.entrypoints=websecure + - traefik.http.routers.selfpost.tls.certresolver=le + - traefik.http.services.selfpost.loadbalancer.server.port=8080 + + traefik: + image: traefik:v3.1 + restart: unless-stopped + depends_on: + - selfpost + command: + - --providers.docker=true + - --providers.docker.exposedbydefault=false + - --entrypoints.web.address=:80 + - --entrypoints.websecure.address=:443 + - --entrypoints.web.http.redirections.entrypoint.to=websecure + - --certificatesresolvers.le.acme.email=you@example.com + - --certificatesresolvers.le.acme.storage=/letsencrypt/acme.json + - --certificatesresolvers.le.acme.httpchallenge.entrypoint=web + ports: + - "80:80" + - "443:443" + volumes: + - /var/run/docker.sock:/var/run/docker.sock:ro + - ./traefik/letsencrypt:/letsencrypt diff --git a/deploy/traefik/extract-cert.sh b/deploy/traefik/extract-cert.sh new file mode 100644 index 0000000..b2c04fa --- /dev/null +++ b/deploy/traefik/extract-cert.sh @@ -0,0 +1,28 @@ +#!/bin/sh +# Extracts a PEM cert/key pair for one domain out of Traefik's acme.json +# (spec 10.3: "Traefik — сертификаты в acme.json, потребуется шаг извлечения +# PEM"). Run this on the host, after Traefik has issued or renewed the +# certificate, and again on a schedule (cron/systemd timer) since acme.json +# is not itself watched by SelfPost/Postfix. +# +# Requires jq. Usage: ./extract-cert.sh +set -eu + +ACME_JSON="${1:?path to acme.json}" +DOMAIN="${2:?domain name, e.g. mail.example.com}" +OUT_DIR="${3:?output directory, e.g. ./extracted-certs}" + +mkdir -p "$OUT_DIR" + +jq -r --arg domain "$DOMAIN" ' + .le.Certificates[] + | select(.domain.main == $domain) + | .certificate' "$ACME_JSON" | base64 -d > "$OUT_DIR/fullchain.pem" + +jq -r --arg domain "$DOMAIN" ' + .le.Certificates[] + | select(.domain.main == $domain) + | .key' "$ACME_JSON" | base64 -d > "$OUT_DIR/privkey.pem" + +chmod 0640 "$OUT_DIR/fullchain.pem" "$OUT_DIR/privkey.pem" +echo "extracted $DOMAIN to $OUT_DIR/{fullchain,privkey}.pem" diff --git a/docs/progress.md b/docs/progress.md index 2d455cf..be7eba7 100644 --- a/docs/progress.md +++ b/docs/progress.md @@ -46,7 +46,7 @@ ## Текущее состояние -- **Текущая фаза:** 9 ✅ **закрыта** → следующая **Фаза 10** (деплой + документация) на **Sonnet** (compose-файлы и документация). +- **Текущая фаза:** 10 ✅ **закрыта** → следующая **Фаза 11** (финальный проход по безопасности) на **Opus**. - **Ключевая находка Фазы 9 (SASL-секреты обратимы, как и предвидело ТЗ 7.5.Б):** `sasldb2` (Berkeley DB, db5.3) хранит пароль приложения как **плейнтекст** в свойстве `userPassword` — подтверждено на сервере. Значит экспорт домена читает его через `db_dump` и на импорте **перезаписывает под локальный realm** через `saslpasswd2` (плейнтекст realm-независим) → креды работают на другом хосте с ДРУГИМ hostname/realm без перевыпуска. Полный бэкап копирует `sasldb2` **побитово** (keyed по исходному realm), поэтому restore обязан идти на **тот же hostname** (миграция всей машины). `db-util` (даёт `db_dump`) добавлен явной зависимостью в Dockerfile. Файл `/data/setup-token` содержит **полный URL**, а не голый токен (для e2e: `TOKEN=${FULL##*/}`). - **Ключевая находка Фазы 6 (исправлена):** go-milter хранит имена макросов **как их шлёт Postfix** — многосимвольные имена приходят в фигурных скобках (`{auth_authen}`, `{client_addr}`), односимвольные — голыми (`i`). Спайк Фазы 0 без SASL этого не увидел (`auth_authen` был пуст «и так»). Первый прогон в контейнере дал пустой `app_login`; фикс — brace-толерантный `macro(m,name)` (пробует голый ключ, затем `{name}`). Зафиксировано в памяти [[milter-implementation-facts]]. - **Прежняя фаза:** 5 ✅ закрыта (код `b2692e4`, доки `ec4d4b9`/`2dbd8d0`). @@ -54,6 +54,14 @@ - **Тупик, который обошли (важно для будущих тестов доставки):** нельзя тестировать доставку, отправляя с домена, который хостит сам приёмник. `mx.example.net` хостит `mixeme.ru`, поэтому письма `mixeme.ru → mixeme.ru` он жёстко отбивал `554 does not meet our delivery requirements` (own-domain anti-spoofing) при валидном DKIM+SPF. Решение: отдельный домен-отправитель `test.example.org` (не на `mc`), которому приёмник доверяет как обычной входящей почте. Первый контакт был `451 Greylisted` (норма) → принят после авто-ретраев Postfix. - **Артефакты теста на сервере:** контейнер `p5` (домены mixeme.ru id1 / test.example.org id2), скрипт/лог `/tmp/p5retry.sh`+`/root/p5retry.log`, IMAP-читалка `/tmp/imapread5.py`. DNS `test.example.org` (A/SPF/DKIM) можно снять после Фазы 6-тестов; в ящике `selfpost@mixeme.ru` остались bounce-письма от ранних mixeme.ru→mixeme.ru попыток (шум, можно удалить). +### Сделано в Фазе 10 +- **Найден и закрыт пробел с Фазы 1:** `logrotate` был установлен пакетом в образ, но никогда не запускался (ни cron, ни supervisor-программы) — `mail.log` рос бы неограниченно. Добавлены `build/logrotate-mail.conf` (`/var/log/mail.log`, `daily`/`rotate 14`/`compress`/`delaycompress`/`copytruncate`) и `build/logrotate-loop.sh` (poll-цикл, дефолт 6ч; сам `logrotate` решает, пора ли ротировать, по `/var/lib/logrotate/status`) + supervisor-программа `[program:logrotate]` (по образцу `cert-reload`). **`copytruncate`, а не сигнал Postfix** — `maillog_file` пишет `postlogd`, который держит файл открытым весь свой жизненный цикл, и никакой демон ротацию не подхватывает; `copytruncate` избавляет от необходимости `postfix reload` на каждую ротацию ценой маленького окна потери нескольких строк лога при truncate — приемлемо для мониторингового лога. Проверено на сервере: `logrotate -f` реально ротирует (`mail.log` truncated, `mail.log.1` с прежним содержимым), процесс `logrotate` в `supervisorctl status` — RUNNING. +- **`deploy/docker-compose.yml`** (ТЗ 10, 10.5): единственный сервис `selfpost`, **Apache не контейнеризован** — предполагается, что он уже стоит на хосте (целевая аудитория — ТЗ 10.5), с фрагментом `deploy/apache/selfpost-vhost.conf` (host-vhost + certbot Apache-плагин, готовые PEM без шага извлечения). Образ — **фиксированный тег** `ghcr.io/mixeme/selfpost:X.Y.Z` (не `:latest`, ТЗ 10 п.10, обязательное условие для проверки версии при restore из Фазы 9). Порты 465/587 публикуются напрямую (почта мимо Apache), 8080 — только на `127.0.0.1` (Apache достаёт панель по localhost, наружу панель без TLS не видна). Hardening (ТЗ 10 п.6): `no-new-privileges`, `cap_drop: ALL` + точечный `cap_add` (`NET_BIND_SERVICE`/`CHOWN`/`SETUID`/`SETGID`/`DAC_OVERRIDE` — entrypoint по-прежнему стартует под root на доли секунды, чтобы починить владельца `/data` и права `selfpost`-группы, см. Фазы 3-4, поэтому `user: panel` и read-only rootfs целиком не подходят). +- **Альтернативные фрагменты reverse-proxy** (ТЗ 10.3, каждый — отдельный `docker compose -f docker-compose.yml -f /docker-compose..yml`): `deploy/nginx/` (контейнеризованный nginx + certbot-сайдкар, PEM тем же bind-mount способом, что и Apache), `deploy/caddy/` (автоматический ACME без доп. контейнеров; путь хранения PEM у Caddy включает имя ACME CA как сегмент пути — **явно помечено как нестабильная внутренняя деталь**, требует проверки под конкретную версию, как и просило ТЗ), `deploy/traefik/` (сертификаты внутри `acme.json`, `deploy/traefik/extract-cert.sh` — скрипт извлечения PEM через `jq`). **Важная находка при проверке:** Docker Compose резолвит все относительные пути в объединяемых файлах относительно каталога **первого** `-f`-файла (т.е. `deploy/`), а не каталога самого фрагмента — путь во фрагментах поэтому не `./nginx.conf.example`, а `./nginx/nginx.conf.example` и т.п.; проверено `docker compose config` для всех трёх фрагментов на сервере (пути резолвились верно только после этого фикса). Списки `ports`/`volumes` в фрагментах используют YAML-тег `!override` (Docker Compose merge-тег) — без него списки **дополняются**, а не заменяются, и тогда порт 8080 из базового файла остался бы опубликован рядом с портами прокси; подтверждено на сервере (`docker compose config` показывает итоговый `selfpost` только с портами/томами фрагмента, не суммой). +- **CI-публикация образа** (`.github/workflows/release.yml`, ТЗ 10.1): триггер — push тега `vX.Y.Z` (обычные коммиты ничего не публикуют); версия выводится из тега один раз (`GITHUB_REF_NAME` без `v`) и идёт **и** в `-ldflags` бинарника, **и** в тег образа — структурно исключает рассинхрон, на который опирается проверка версии при restore (Фаза 9). Публикация в `ghcr.io/mixeme/selfpost` (обоснование ТЗ 10.1: `ghcr` бесплатен и без анонимного rate-limit, в отличие от Docker Hub). Мультиплатформенная сборка (`linux/amd64`+`linux/arm64`) через `buildx`/`qemu`. +- **README.md переписан** (ТЗ 10 п.7-11): чеклист «Requirements», Quick start со ссылкой на `deploy/`, раздел «Reverse proxy» (таблица всех 4 вариантов + обоснование дефолта Apache), «DNS setup» (явное разделение уровня сервера/уровня домена — PTR один раз vs SPF/DKIM/DMARC на каждый добавленный домен), «IP warmup», «Backup, restore, and moving a single domain» (полный бэкап vs экспорт/импорт домена — разница в сценарии переноса, оба файла — секреты), «Fixed image tag» (связь с проверкой версии restore), «Machine requirements» (1 vCPU, 512МБ-1ГБ, 8-10ГБ диска, рекомендация swap). +- **Проверено на сервере** (selfpost.example.com): `gofmt`/`vet`/`test` зелёные (Go-код фазы не менялся); `docker build` образа с новым logrotate-путём — ок (`selfpost:p10`); контейнер стартует, все процессы (включая новый `logrotate`) RUNNING; ручная принудительная ротация мониторингового лога отработала; `docker compose config` — валиден для базового файла и всех трёх альтернативных фрагментов (nginx/caddy/traefik), после чего смёрженный `selfpost`-сервис содержит ожидаемые порты/тома/env, а не сумму базового+фрагмента; сборка `docker compose up` базового файла реально создаёт контейнер и сеть (сама попытка упёрлась только в конфликт порта 465 с посторонним контейнером `p6`, оставшимся от более ранней фазы, — не дефект текущей конфигурации). CI-workflow не запускался (нет прав пушить теги в этой сессии) — синтаксис YAML проверен парсером. + ### Сделано в Фазе 9 - **Полный бэкап** (`internal/backup/backup.go`, ТЗ 7.5.А): `Create(w, Params)` пишет `tar.gz` всего `/data` — **консистентный снимок SQLite через `VACUUM INTO`** во временный файл (не побайтовое копирование живого WAL-файла), DKIM-ключи, `sasldb2`, карта Postfix, + `manifest.json` (`format`/`version`/`created_at`). Имена в архиве — относительно `/data`, так что распаковка в bind-mount восстанавливает состояние на месте. **Исключаются**: живой `selfpost.db`(+`-wal`/`-shm`/`-journal`, заменён снимком под тем же именем), `setup-token`, стейл-`manifest.json`, и каталог **`tls/`** — сертификаты это зона reverse-proxy (ТЗ 7.5.А); исключение держит гарантию даже если оператор положил серты в `/data/tls`. Каталоги-записи сохраняются (моды/пустые). Очередь Postfix не входит (ТЗ). - **Гварда версии при restore** (`CheckRestore`, вызывается в `run()` **до** `store.Open` в `cmd/panel/main.go`): если в `/data` лежит `manifest.json` (значит бэкап распакован), его версия обязана совпасть с версией бинарника, иначе панель **отказывается стартовать** с сообщением, каким тегом образа восстанавливать (`selfpost:`); при совпадении манифест **потребляется** (удаляется) — гвардит только первый старт после restore и не блокирует обычный in-place апгрейд образа; отсутствие манифеста = обычный старт. Restore — не отдельная ветка кода: состояние (Postfix/OpenDKIM) регенерируется из восстановленного SQLite тем же путём, что при любом старте. @@ -179,3 +187,4 @@ - **Фаза 8** (2026-07-14, Opus) — дифференцированные лимиты (rate limit уровень 2, ТЗ 7.4): journal-milter из чистого монитора стал отклонять письмо `4xx` (`RespTempFail` 451) на стадии MAIL FROM при превышении лимита домена/приложения; ключ — client IP, счёт — `COUNT(DISTINCT queue_id)` в скользящем окне по `send_log`, применяется только при непустой IP-привязке (иначе только уровень 1). Строго **fail-open** на собственных ошибках (сбой лимитера не блокирует почту, уровень-1 anvil независим). Отклонения пишутся `send_log` статусом `rejected` для UI. Store `internal/store/ratelimits.go` (таблица `rate_limits` уже была с Фазы 2 — миграции нет), панель-формы на домене и приложении с серверной валидацией IP/чисел (ТЗ 7.6.2), очистка лимитов при каскадном удалении. Юниты + контейнерный e2e (`p8`): реджект на обоих уровнях, unregistered-IP игнор, fail-open при остановке панели — зелёные. Все критерии «Готово когда» Фазы 8 выполнены. - **Фаза 9** (2026-07-14, Opus) — бэкап/restore + экспорт/импорт домена (ТЗ 7.5, 11.6): полный `tar.gz`-бэкап всего `/data` (консистентный снимок SQLite через `VACUUM INTO`, DKIM-ключи, `sasldb2`, `manifest.json` с версией; без TLS-сертов/`tls/` и очереди Postfix) двумя путями — кнопка `POST /backup` и CLI `selfpost-backup` через `docker exec`; гварда версии `CheckRestore` до `store.Open` (несовпадение → отказ старта с указанием тега; совпадение → манифест потребляется, restore идёт обычным стартом без отдельной ветки). Экспорт/импорт домена: `DomainExport` (DKIM-ключ PKCS#1 PEM + приложения с **рабочими паролями**), секреты SASL читаются из `sasldb2` через `db_dump` (`userPassword` — плейнтекст), на импорте **ре-кеятся под локальный realm** через `saslpasswd2` → креды работают на инстансе с другим hostname без перевыпуска, DKIM DNS-запись не меняется. `db-util` добавлен в образ. Юниты + контейнерный e2e (экспорт↔импорт кросс-realm с проверкой SMTP-auth 235; CLI+кнопка бэкап; restore той же версии; отказ при несовпадении версии) зелёные. - **Фаза 7** (2026-07-13, Sonnet) — UI мониторинга: три экрана (журнал отправки с серверными фильтрами домен/приложение + пагинацией, очередь Postfix `postqueue -p`, хвост `mail.log`), все с HTMX-polling каждые 5с; fragment-эндпоинты отдают HTML (ТЗ 7.1), вывод экранирован `html/template` (ТЗ 7.6.7, проверено на `