diff --git a/README.md b/README.md index bd43894..3afcce6 100644 --- a/README.md +++ b/README.md @@ -5,50 +5,73 @@ # SelfPost Self-hosted outbound SMTP relay with a web control panel, shipped as a single -Docker image. Postfix + OpenDKIM + a small Go panel run together under -`supervisord`; the panel manages multiple sending domains, per-domain DKIM keys -and SASL-authenticated applications bound to their domain. +Docker image. Postfix, OpenDKIM, and a small Go panel run together under +`supervisord`; you configure domains, DKIM keys, and SASL applications once, +then point your apps at the SMTP endpoint. -SelfPost sends mail straight to the internet from your own IP, with DKIM -signing, and is configured once through the panel. It is **outbound only** — it -does not receive mail, provide mailboxes, or offer webmail. +SelfPost sends mail straight to the internet from **your own IP**, with per-domain +DKIM signing. It is **outbound only** — no inbound mail, mailboxes, or webmail. -> **Status: v1.0 implemented; pre-release polish in progress.** Open v1.x -> questions: [docs/implementation-plan.md](docs/implementation-plan.md). -> Documentation pass before the next tag: -> [docs/documentation-plan.md](docs/documentation-plan.md). Accepted security -> trade-offs: [docs/security.md](docs/security.md). +**For:** operators who run their own VPS or home server and want a simple relay +they control, without a third-party SMTP provider. -## Table of contents +**Key properties:** one container, multi-domain DKIM, SASL per application, +send log and DNS checks in the panel, encrypted backups. -- [Requirements (site checklist)](#requirements-site-checklist) -- [Quick start](#quick-start) -- [Reverse proxy (mandatory)](#reverse-proxy-mandatory) -- [Environment variables](#environment-variables) -- [DNS setup](#dns-setup) -- [IP warmup](#ip-warmup) -- [Operations](#operations) -- [Rate limiting](#rate-limiting) -- [Backup, restore, and moving a single domain](#backup-restore-and-moving-a-single-domain) - - [Encrypting a backup or export](#encrypting-a-backup-or-export) -- [Published ports](#published-ports) -- [Fixed image tag](#fixed-image-tag) -- [Machine requirements](#machine-requirements) -- [Repository](#repository) -- [License](#license) +## Features -## Requirements (site checklist) +- Outbound SMTP (465/smtps; optional 587 submission) with per-domain DKIM signing +- Web panel — domains, applications, deliveries, mail queue, system log, backup +- Multi-domain relay — each SASL application is bound to one sending domain +- DNS status checks (PTR, SPF, DKIM, DMARC) with in-panel re-check +- Two-level rate limiting — IP backstop (Postfix) and per-domain/per-app limits +- Full-server backup and single-domain export/import (optional password encryption) +- Single Docker image; data in a `./data` bind mount -Providing these is the operator's job, not a feature of SelfPost — the panel -can't fix a blocked port or a missing PTR record for you. +## Documentation -- [ ] A static IP address. +| Document | Contents | +|---|---| +| [**Operator guide**](docs/guide.md) | Reverse proxy, environment variables, DNS, IP warmup, panel operations, rate limiting, backup/restore, ports, image tag | +| [Product boundaries](docs/product.md) | Purpose, deployment assumptions, out-of-scope items, multi-domain model | +| [Architecture](docs/architecture.md) | As-built technical design | +| [Security](docs/security.md) | Accepted security trade-offs and requirements | +| [Development](docs/development.md) | Building, testing, and contributing | +| [CHANGELOG](CHANGELOG.md) | Release history | + +Repository: — source, issues, releases, and +the `ghcr.io/mixeme/selfpost` image. + +## Requirements + +Providing these is the operator's job — SelfPost cannot fix a blocked port or a +missing PTR record for you. Details: [Operator guide](docs/guide.md). + +### Platform + +- Docker + Compose v2 on the host +- A reverse proxy in front of the panel (SelfPost never terminates HTTPS itself) +- Rough sizing: **1 vCPU**, **512 MB–1 GB RAM**, **8–10 GB disk** (send log and + rotated `mail.log` are the main growth drivers) + +### Network and IP + +- [ ] Static IP address - [ ] Outbound TCP port 25 unblocked (many consumer/cloud hosts block it by - default — check with your provider before anything else). -- [ ] PTR/rDNS for that IP set to your mail hostname (see [DNS setup](#dns-setup)). -- [ ] Reasonable starting IP reputation — a fresh IP still needs [warmup](#ip-warmup). -- [ ] A reverse proxy in front of the panel (see [Reverse proxy](#reverse-proxy-mandatory)) — SelfPost never terminates HTTPS itself. -- [ ] Docker + Compose v2 on the host. + default) +- [ ] PTR/rDNS for that IP pointing at your mail hostname (`SELFPOST_HOSTNAME`) +- [ ] Reasonable starting IP reputation — a fresh IP still needs + [warmup](docs/guide.md#ip-warmup) + +### Per sending domain + +For every domain you add in the panel: + +- [ ] SPF TXT record authorizing this server +- [ ] DKIM TXT record (value shown on the domain page) +- [ ] DMARC `_dmarc` TXT record + +See [DNS setup](docs/guide.md#dns-setup) in the operator guide. ## Quick start @@ -60,346 +83,34 @@ mv .env.example .env # then edit SELFPOST_HOSTNAME etc. docker compose up -d ``` -`SELFPOST_HOSTNAME` is required — the container exits immediately with an -explanatory error if it's unset, since it doubles as the Postfix HELO name, -the SASL realm, and must match both the PTR record and the certificate -CN/SAN. +`SELFPOST_HOSTNAME` is required — bare FQDN only (e.g. `mail.example.com`). It +is the Postfix HELO name, the SASL realm, and must match the PTR record and +certificate CN/SAN. -This starts SelfPost alone; it assumes Apache is already installed on the host -as the reverse proxy (see below) and expects certificates at `./certs`. The -first log line (`docker compose logs -f`) prints the one-time setup link — -open it to create the admin account. That username and password can be changed -later from the panel's *Account* page (changing the password signs out every -other session). +This starts SelfPost only. You still need a reverse proxy with TLS certificates +bind-mounted at `./certs` — see [Reference deploy](#reference-deploy) and the +[reverse-proxy section](docs/guide.md#reverse-proxy-mandatory) in the operator +guide. -The same link is also written to `/data/setup-token` inside the container — -`./data/setup-token` on the host, mode `0600` — and deleted the moment setup -completes. If this host ships its container logs to a central aggregator, -prefer the file: the link is a bearer token valid for ten minutes, and reading -it this way keeps it out of the log pipeline (and out of whatever retains it -afterwards) entirely. +On first start, `docker compose logs -f` prints a one-time setup link to create +the admin account. The same link is also in `./data/setup-token` on the host — +see [Operations → First-time setup link](docs/guide.md#operations). -```sh -docker compose exec selfpost cat /data/setup-token -``` +## Reference deploy -## Reverse proxy (mandatory) +| Artefact | Path | +|---|---| +| Compose file (fixed image tag) | [deploy/docker-compose.yml](deploy/docker-compose.yml) | +| Environment template | [deploy/.env.example](deploy/.env.example) | +| Apache vhost (recommended) | [deploy/apache/selfpost-vhost.conf](deploy/apache/selfpost-vhost.conf) | +| nginx | [deploy/nginx/](deploy/nginx/) | +| Caddy | [deploy/caddy/](deploy/caddy/) | +| Traefik | [deploy/traefik/](deploy/traefik/) | -SelfPost's panel speaks plain HTTP and never terminates TLS itself — a reverse -proxy in front of it is not optional. The proxy is also the project's only -source of TLS certificates: whatever it obtains via ACME/Let's Encrypt gets -bind-mounted **read-only** into the SelfPost container, and Postfix uses those -same PEM files for TLS on 465 (and 587, if enabled). If the panel and the mail -service share one hostname — the common case — it's genuinely one certificate -serving both. - -SelfPost isn't tied to a specific proxy; pick whichever fits your host: - -| Proxy | Where certs live | Fragment | -|---|---|---| -| **Apache** (default/recommended) | Host disk, via the certbot Apache plugin — PEM files ready to bind-mount, no extraction step. | [deploy/apache/selfpost-vhost.conf](deploy/apache/selfpost-vhost.conf) | -| nginx | Host disk, via a certbot sidecar container — same PEM-ready shape as Apache. | [deploy/nginx/](deploy/nginx/) | -| Caddy | Automatic ACME, zero extra containers — simplest, but its on-disk cert path is versioned internal layout, not a stable API; verify it for the Caddy version you run. | [deploy/caddy/](deploy/caddy/) | -| Traefik | Bundled inside `acme.json` — needs a small extraction script to produce standalone PEM files. | [deploy/traefik/](deploy/traefik/) | - -Apache is the recommended default because the certbot Apache plugin already -writes plain `fullchain.pem`/`privkey.pem` files to a predictable path with no -extra moving parts between "certificate issued" and "Postfix can read it." - -**The proxy needs no security configuration of its own.** The panel emits its -own `Content-Security-Policy`, `Strict-Transport-Security`, `X-Frame-Options`, -`X-Content-Type-Options` and `Referrer-Policy` — deliberately, so the part -that's easy to get wrong lives in the service rather than in a config file -somebody edits under pressure. There is exactly one thing the proxy must do: -**pass the original `Host` header through**. All four fragments above already -do (Apache `ProxyPreserveHost On`, nginx `proxy_set_header Host $host`, Caddy -and Traefik by default). A proxy that rewrites `Host` instead makes the panel -reject every form submission as cross-origin — the log says so explicitly, -printing the `Origin` and `Host` it compared. - -## Environment variables - -Copy [deploy/.env.example](deploy/.env.example) to `.env` next to your -`docker-compose.yml`. The table below lists every variable an operator is -expected to set; defaults match the code exactly. - -| Variable | Purpose | Default | Set in | -|---|---|---|---| -| `SELFPOST_HOSTNAME` | Mail-server identity: Postfix HELO/EHLO, SASL realm, certificate CN/SAN, and the hostname the PTR check expects. Bare FQDN only — no scheme or port. | *(required)* | `.env` | -| `SUBMISSION_ENABLE` | When `true`, also listen on port 587 with STARTTLS (RFC 6409 submission) alongside the primary 465/smtps listener. | `false` | `.env` | -| `RATE_LIMIT_MESSAGES_PER_IP` | Level-1 backstop: maximum messages one client IP may submit per window (Postfix `smtpd_client_message_rate_limit`). See [Rate limiting](#rate-limiting). | `100` | `.env` | -| `RATE_LIMIT_WINDOW_SECONDS` | Level-1 window length in seconds (Postfix `anvil_rate_time_unit`). | `3600` | `.env` | -| `SEND_LOG_RETENTION_DAYS` | Days of send-log history kept before the background sweep deletes rows — the main driver of `/data` growth over time. | `90` | `.env` | -| `PANEL_SESSION_IDLE_DAYS` | Sliding idle timeout for the panel login session, in days. There is no absolute cap: an admin who keeps coming back stays signed in indefinitely. | `7` | `.env` | -| `SELFPOST_DNS_RESOLVERS` | Comma-separated recursive resolvers the panel's PTR/SPF/DKIM/DMARC checks query directly (so they report what the internet sees, not what this host's stub resolver synthesises). | `1.1.1.1:53`, `8.8.8.8:53`, `9.9.9.9:53` when unset | `.env` | -| `TRUSTED_PROXY_CIDR` | Comma-separated CIDRs (bare IPs allowed) of reverse proxies allowed to supply `X-Forwarded-For` for login, setup, and account-change rate-limiting. **Leave unset unless you know the exact address of your reverse proxy.** A wrong value lets a client spoof its rate-limit key by sending a forged `X-Forwarded-For` header — the panel trusts the last hop only when the TCP peer matches one of these CIDRs. Behind the default Apache host-network setup this is typically the Docker bridge gateway, e.g. `172.18.0.1`. | *(empty — XFF ignored)* | `.env` | - -TLS certificate paths (`TLS_CERT_FILE`, `TLS_KEY_FILE`) are fixed in -[deploy/docker-compose.yml](deploy/docker-compose.yml) to match the `./certs` -bind mount — configure the mount, not these variables. - -**Internal variables (not part of the operator interface).** The following are -read by the panel or startup scripts but are not meant to be changed in a -normal deployment; documenting them here avoids treating accidental overrides as -supported configuration: - -- **Panel paths and tuning:** `SELFPOST_DATA_DIR` (`/data`), `SELFPOST_DB_PATH` - (`/data/selfpost.db`), `SELFPOST_SETUP_TOKEN_FILE` - (`/data/setup-token`), `PANEL_HTTP_ADDR` (`:8080`), - `JOURNAL_MILTER_SOCKET` (`/run/selfpost/journal.sock`), `MAIL_LOG` - (`/var/log/mail.log`), `PANEL_COOKIE_SECURE` (`true`), `OPENDKIM_SOCKET` - (`/run/opendkim/opendkim.sock`), `OPENDKIM_DIR` (`/data/opendkim`), - `DKIM_SELECTOR_DEFAULT` (`selfpost`), `SASL_DB_PATH` - (`/data/sasl/sasldb2`), `SASL_REALM` (defaults to `SELFPOST_HOSTNAME`), - `POSTFIX_DIR` (`/data/postfix`), `POSTFIX_SENDER_LOGIN_MAPS` - (`/data/postfix/sender_login_maps`). -- **Milter and Postfix startup:** `MILTER_CONNECT_TIMEOUT` (`15s`), - `MILTER_COMMAND_TIMEOUT` (`15s`), `MILTER_CONTENT_TIMEOUT` (`30s`), - `MILTER_WAIT_TIMEOUT` (`30` seconds). -- **Background maintenance:** `TLS_RELOAD_INTERVAL_SECONDS` (`86400` — daily - `postfix reload` to pick up renewed certificates), - `LOGROTATE_INTERVAL_SECONDS` (`21600` — check `mail.log` rotation every six - hours; rotated logs are kept 14 days and each rotation triggers - `postfix reload`). - -## DNS setup - -Two different scopes — don't confuse them: - -**Server level (once, for the machine itself):** -- **PTR/rDNS** for the server's IP, pointing at its mail hostname. Most - receiving mail servers weigh this heavily; get it from whoever assigns the IP - (hosting provider's panel/support), not from your own DNS zone. - -**Domain level (for *every* sending domain you add in the panel):** -- **SPF** — a TXT record on the domain authorizing this server to send on its - behalf (e.g. `v=spf1 a mx ip4: -all`, adjusted to your setup). -- **DKIM** — a TXT record with the exact value the panel shows on that - domain's page (`domain page → DKIM TXT record`), one selector per domain. -- **DMARC** — a `_dmarc` TXT record (even a conservative `p=none` starts - building reporting/reputation history). - -Skipping any of the three per-domain records is the single most common reason -mail lands in spam even though SelfPost delivered it correctly — DKIM passing -doesn't help if SPF/DMARC are absent. **Whenever you add a new domain in the -panel, add its DNS records at the same time**, not later. - -The panel checks both scopes for you and tells you what is actually published: -the *Status* page verifies the server's hostname and its reverse record -(forward-confirmed reverse DNS), and each domain's page shows a *DNS status* -card comparing the published DKIM record against the key this server signs with, -plus the domain's SPF and DMARC records. Results are cached for a few minutes; -use *Re-check* right after publishing a record. The SPF check is deliberately -shallow — it looks for a mechanism that literally covers this server's address -and does not follow `include:` or `redirect=`, so a record that authorizes the -server through an include is reported as "cannot tell" rather than as a failure. - -## IP warmup - -A brand-new IP has no sending history, so receiving servers are cautious with -it regardless of how correct your DKIM/SPF/DMARC are. Start with low volume to -a domain, increase gradually over days/weeks rather than sending everything on -day one, and check the IP against major blocklists (Spamhaus and similar) -before and during warmup. This is inherent to how mail reputation works on the -public internet, not something SelfPost's configuration can shortcut. - -## Operations - -After sign-in the panel opens on **Status** — the place to answer "is the -service healthy and will mail be accepted?" - -- **Status** (`/status`) — supervised processes (Postfix, OpenDKIM, panel), - TLS certificate validity and expiry, milter socket presence, and a short - Postfix queue summary. The **Machine** card adds the resource usage of the - host underneath — processor (with the load average), memory and swap, and - per-interface network throughput and totals — read from the kernel's - counters; CPU and throughput are measured between refreshes, so they appear - one refresh after the page opens. A fully busy processor or a machine out of - memory is a warning here, because both delay or kill the mail path; - throughput is only reported. The hostname block compares `SELFPOST_HOSTNAME` - against the PTR record the internet publishes for this server's IP - (forward-confirmed reverse DNS); use *Re-check* after changing DNS. The - **Reload configuration** button re-applies OpenDKIM tables and the Postfix - sender map from the database — use it if daemons drifted from what the panel - shows after manual edits under `/data`. -- **Domains** (`/domains`) — add sending domains, inspect each domain's DKIM - TXT value, SPF/DMARC checks, and SASL applications. Per-domain rate limits - (level 2) are configured here. *Export domain* writes a single-domain archive; - *Import a domain* on the Backup page reads one back in. -- **Deliveries** (`/deliveries`) — searchable send log with server-side filters - by domain and application. A row identifies its message and nothing more — - time, sender, recipient, subject and status `queued` (accepted, not yet - delivered), `sent` (handed off successfully), or `rejected` (refused — for - example by a level-2 rate limit); *Details* opens that row's own page - (`/deliveries/{id}`). That page carries the sending domain, the application it - was submitted under, the Postfix queue id and the journal id, beside the - message's history — when it was accepted and what Postfix later reported for - the recipient — and, under both, the `mail.log` lines for its queue id: the - connection to the receiving server, the server's reply, and the status that - reply was filed as. Rows outlive `mail.log`, so an older message's lines may - have rotated away; the page says so. Retention is controlled by - `SEND_LOG_RETENTION_DAYS`. -- **Mail queue** (`/mail-queue`) — live view of messages Postfix is still - trying to deliver or deferring. -- **System log** (`/system-log`) — tail of `/var/log/mail.log` (Postfix and - related daemon lines). The log rotates daily (14 files kept) with a - `postfix reload` after each rotation; a background loop checks every six - hours. -- **Backup** (`/backup`) — download a full-server backup or import a - single-domain export. See [Backup, restore, and moving a single domain](#backup-restore-and-moving-a-single-domain). -- **Account** (`/account`) — change the administrator username and/or password. - Application SASL logins are separate and are not changed here. - -**Sessions.** A login survives a container restart: sessions live in SQLite, not -in memory. Expiry is a sliding idle window (`PANEL_SESSION_IDLE_DAYS`, default -seven days) with no absolute lifetime cap — an admin who keeps using the panel -stays signed in indefinitely. HTMX polling on the monitoring screens -(Deliveries, Mail queue, System log, and the Status health fragment) does -**not** count as activity, so an auto-refreshing tab left open will not keep a -session alive forever. Changing the password signs out every other session but -leaves the current browser signed in. - -**Upgrading.** Bump the pinned image tag in `docker-compose.yml` to the target -release, then `docker compose up -d`. The backup version check requires the -running image to match the version that created a full backup — see [Fixed image -tag](#fixed-image-tag). - -**Container health.** The image declares a Docker `HEALTHCHECK` that probes -`GET /healthz` on port 8080 (unauthenticated). It returns `200 ok` when -OpenDKIM, the panel, and Postfix are all `RUNNING` under supervisord; -otherwise `503 unhealthy`. This catches a dead mail path that would still leave -the HTTP server up, but it does **not** verify TLS certificates, DNS records, -or end-to-end delivery — use the authenticated **Status** page for that. External -monitoring can use the same endpoint through the reverse proxy if you expose it, -or poll `docker inspect` health state on the host. - -## Rate limiting - -SelfPost applies two independent limits; both can refuse a submission, but only -level 2 writes a `rejected` row in the send log. - -**Level 1 (IP backstop)** — always on, configured via `.env`: - -- `RATE_LIMIT_MESSAGES_PER_IP` → Postfix `smtpd_client_message_rate_limit` -- `RATE_LIMIT_WINDOW_SECONDS` → Postfix `anvil_rate_time_unit` - -This is an anvil limit per connecting client IP. It keeps working even if the -journal-milter (level 2) is down. - -**Level 2 (per domain / per application)** — optional, configured in the panel -on each domain's page or on an individual application. You set a message -ceiling, a time window, and optionally restrict the limit to specific client -IPs; an empty IP list means the differentiated limit does not apply. When -exceeded, Postfix returns a 4xx and the refusal is recorded in Deliveries as -`rejected`. - -## Backup, restore, and moving a single domain - -Two related but distinct operations — spec 7.5: - -- **Full backup** (whole `/data`: SQLite, all domains' DKIM keys, all - applications' SASL credentials, `manifest.json` with the version that - created it): panel button (*Backup* → *Full backup*), or from the - host: - ```sh - docker exec selfpost-backup > selfpost-backup.tar.gz - ``` - **Restore** means unpacking that archive into a fresh `/data` bind mount and - starting a container of the **exact same image version** that created it — - SelfPost refuses to start otherwise and tells you which tag to use. On the - first successful start after restore, `manifest.json` from the archive is - **deleted** — it guards only that one boot, so a later in-place upgrade is - not blocked. This is why the compose file below pins a fixed tag rather than - `:latest`: without a known version, there'd be no way to tell which image - restoring a given backup actually requires. - - **Alternative: archive `./data` while stopped.** If the service can be taken - offline, `docker compose down` then `tar czf selfpost-data.tar.gz ./data` on - the host is safe — nothing is writing to SQLite. Do **not** tar `./data` while - the container is running: the database uses WAL mode and a naive copy can - capture an inconsistent snapshot. The panel/CLI backup remains preferable when - you cannot afford downtime because it takes a consistent SQLite snapshot via - the Backup API on a live container. - -- **Export/import a single domain** (domain page → *Export domain* to write the - file, *Backup* → *Import a domain* to read it back in): moves one domain — its DKIM key and its applications' **working** - SASL passwords — to a different SelfPost instance without regenerating - anything, so DNS (the DKIM TXT record) doesn't need to change. Unlike a full - restore, this works across different hostnames/instances. - -Both files are **secrets** — they contain the admin password hash (full -backup) or working application credentials (domain export) in the clear or in -directly reversible form. Treat them like any other credential material: -restrict who can read them, don't email them around — and encrypt them, which -SelfPost can do for you. - -### Encrypting a backup or export - -Both download forms carry an **Encrypt with a password** checkbox. Ticked, the -file that comes down is an encrypted envelope instead of the plain archive: - -| Artefact | Plain | Encrypted | -|----------|-------|-----------| -| Full backup | `.tar.gz` | `.spbk` | -| Domain export | `.json` | `.spde` | - -The key is derived from the password with scrypt and the contents are sealed -with AES-256-GCM, in chunks, so a truncated or altered file fails to open rather -than restoring quietly. **SelfPost does not store the password** — lose it and -the file is unrecoverable, which is the entire point. - -*Import a domain* takes an encrypted export directly: tick **The file is -encrypted** and give the password. - -A full backup has to be turned back into a plain archive before it can be -unpacked into `/data`, which the CLI does with the same password: - -```sh -docker exec -i selfpost-backup -decrypt < backup.spbk > backup.tar.gz -``` - -The CLI also *writes* encrypted backups for scripted/cron use. The password -comes from `SELFPOST_BACKUP_PASSWORD` or `-password-file ` (first line), -never from a command-line argument, which would be visible in the process list: - -```sh -docker exec -e SELFPOST_BACKUP_PASSWORD="$PW" selfpost-backup > backup.spbk -``` - -With no password set, the CLI keeps writing the plain `.tar.gz` it always has. - -## Published ports - -`deploy/docker-compose.yml` maps **465** and **587** to the host. Port 465 -(smtps) is always active. Port **587** is published even when -`SUBMISSION_ENABLE=false`; nothing listens until you set it to `true` — harmless, -but it can look like an open port in external scans. - -## Fixed image tag - -`deploy/docker-compose.yml` pins an explicit version (`ghcr.io/mixeme/selfpost:X.Y.Z`), -deliberately never `:latest`. This is a direct consequence of the backup -version check above: the panel binary's embedded version and the image tag -that produced it are the same value by construction (the release CI stamps -both from one git tag — see `.github/workflows/release.yml`), so pinning the -tag is what makes "restore into the same version" a checkable fact rather than -a guess. Upgrade by bumping the tag deliberately, not by riding a moving -target. - -## Machine requirements - -Rough guide, not a hard floor: **1 vCPU**, **512MB–1GB RAM** (the stack — three -processes plus SQLite — idles around 100–150MB; the rest is headroom for -backups, log-tailer/retention sweeps and concurrent TLS handshakes coinciding), -**8–10GB disk**. Disk usage grows mainly from the send log (bounded by -`SEND_LOG_RETENTION_DAYS`, default 90) and the rotated `mail.log` (kept 14 days -in-image), not from the application itself. On boxes with little RAM, a small -swap file is cheap insurance against those occasional coincident spikes. - -## Repository - - — the single home of the project: source, -issues, releases and the `ghcr.io/mixeme/selfpost` image all live there. +The compose file maps ports **465** (always) and **587** (when +`SUBMISSION_ENABLE=true`). TLS certificate paths inside the container are fixed +to match the `./certs` bind mount. Bump the pinned image tag deliberately when +upgrading — never use `:latest` ([why](docs/guide.md#fixed-image-tag)). ## License diff --git a/build/Dockerfile b/build/Dockerfile index 7aad6f9..90fabda 100644 --- a/build/Dockerfile +++ b/build/Dockerfile @@ -105,7 +105,7 @@ RUN chmod +x /usr/local/bin/postfix-wrapper.sh /usr/local/bin/postfix-config.sh EXPOSE 8080 465 587 # Liveness probe: panel HTTP plus mail-path processes (opendkim, panel, postfix). -# Does not verify TLS, DNS, or end-to-end delivery — see README Operations. +# Does not verify TLS, DNS, or end-to-end delivery — see docs/guide.md Operations. HEALTHCHECK --interval=30s --timeout=5s --start-period=90s --retries=3 \ CMD curl -fsS http://127.0.0.1:8080/healthz || exit 1 diff --git a/cmd/panel/envdoc_test.go b/cmd/panel/envdoc_test.go index 3bb8675..1e580c4 100644 --- a/cmd/panel/envdoc_test.go +++ b/cmd/panel/envdoc_test.go @@ -5,7 +5,7 @@ import ( "testing" ) -// documentedPublic matches the README "Environment variables" table. +// documentedPublic matches docs/guide.md "Environment variables" table. var documentedPublic = []string{ "SELFPOST_HOSTNAME", "SUBMISSION_ENABLE", @@ -17,7 +17,7 @@ var documentedPublic = []string{ "TRUSTED_PROXY_CIDR", } -// documentedInternal matches README "Internal variables (not part of the operator interface)". +// documentedInternal matches guide.md "Internal variables (not part of the operator interface)". var documentedInternal = []string{ "SELFPOST_DATA_DIR", "SELFPOST_DB_PATH", @@ -105,7 +105,7 @@ func TestLoadConfigKeysDocumented(t *testing.T) { doc := documentedKeys() for _, key := range loadConfigKeys { if !slices.Contains(doc, key) { - t.Errorf("loadConfig reads %s but it is not listed in README public, internal, or compose-fixed env docs", key) + t.Errorf("loadConfig reads %s but it is not listed in guide.md public, internal, or compose-fixed env docs", key) } } } @@ -114,7 +114,7 @@ func TestBuildScriptKeysDocumented(t *testing.T) { doc := documentedKeys() for _, key := range buildScriptKeys { if !slices.Contains(doc, key) { - t.Errorf("build scripts read %s but it is not listed in README env documentation", key) + t.Errorf("build scripts read %s but it is not listed in guide.md env documentation", key) } } } @@ -127,7 +127,7 @@ func TestDocumentedKeysAreRead(t *testing.T) { for _, key := range documentedKeys() { if !slices.Contains(read, key) { - t.Errorf("README documents %s but no code in loadConfig or build scripts reads it", key) + t.Errorf("guide.md documents %s but no code in loadConfig or build scripts reads it", key) } } } diff --git a/deploy/.env.example b/deploy/.env.example index b12d881..48a9c1c 100644 --- a/deploy/.env.example +++ b/deploy/.env.example @@ -1,5 +1,5 @@ # Copy to .env next to docker-compose.yml and fill in. -# Full reference: README "Environment variables". +# Full reference: docs/guide.md "Environment variables". # Required — the container refuses to start without it. Hostname used both for # the panel's TLS vhost and for Postfix's SASL realm (SASL_REALM defaults to @@ -11,7 +11,7 @@ SELFPOST_HOSTNAME=mail.example.com # primary 465/smtps listener. SUBMISSION_ENABLE=false -# Level-1 backstop rate limit (anvil) — see README "Rate limiting". +# Level-1 backstop rate limit (anvil) — see docs/guide.md "Rate limiting". RATE_LIMIT_MESSAGES_PER_IP=100 RATE_LIMIT_WINDOW_SECONDS=3600 diff --git a/deploy/docker-compose.yml b/deploy/docker-compose.yml index d673e0b..3b8e68a 100644 --- a/deploy/docker-compose.yml +++ b/deploy/docker-compose.yml @@ -5,7 +5,7 @@ # installed on the HOST (the target audience for this project typically runs # Apache there already — spec 10.5) and reverse-proxies HTTPS for the panel; # it is not containerised here. See ../apache/selfpost-vhost.conf for a ready -# vhost fragment, and the "Reverse proxy" section of ../../README.md for the +# vhost fragment, and docs/guide.md "Reverse proxy" for the # certbot steps that produce the PEM files this compose file mounts. # # Usage: @@ -47,7 +47,7 @@ services: # resolver synthesises. Empty uses public defaults. SELFPOST_DNS_RESOLVERS: "${SELFPOST_DNS_RESOLVERS:-}" # Reverse-proxy CIDRs allowed to supply X-Forwarded-For for login/setup - # rate-limiting. See README "Environment variables" — wrong value lets a + # rate-limiting. See docs/guide.md "Environment variables" — wrong value lets a # client spoof its rate-limit key. TRUSTED_PROXY_CIDR: "${TRUSTED_PROXY_CIDR:-}" volumes: @@ -68,7 +68,7 @@ services: # panel is never directly reachable from the internet without TLS. - "465:465" # 587 is mapped even when SUBMISSION_ENABLE=false; Postfix listens only - # when the variable is true — see README "Published ports". + # when the variable is true — see docs/guide.md "Published ports". - "587:587" - "127.0.0.1:8080:8080" # Hardening (spec 10 p.6). SelfPost's entrypoint still needs to run as diff --git a/docs/architecture.md b/docs/architecture.md index 96f88c7..273cb27 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -4,7 +4,7 @@ when env keys, routes, or mail-path behaviour change. Verification method: [documentation-plan.md](documentation-plan.md) §2. -User install/operations: [README.md](../README.md). Product boundaries: +User install/operations: [README.md](../README.md), [guide.md](guide.md). Product boundaries: [product.md](product.md). --- @@ -258,7 +258,7 @@ Not in `/data`: TLS certificates (reverse-proxy mount), Postfix queue **Backup:** panel button or `selfpost-backup` CLI — SQLite snapshot + tar of `/data` tree; version check on restore. Stopped-container `tar` of `./data` is -safe (see README). +safe (see guide). **Optional encryption** of the two secret-bearing downloads ([internal/secretfile](../internal/secretfile/secretfile.go)): password → @@ -280,5 +280,5 @@ origin check, no CSRF tokens) are documented there separately. ## Configuration -Public and internal env vars: [README § Environment variables](../README.md#environment-variables). +Public and internal env vars: [guide § Environment variables](guide.md#environment-variables). Regression test: [cmd/panel/envdoc_test.go](../cmd/panel/envdoc_test.go). diff --git a/docs/development.md b/docs/development.md index daa57aa..7dfe632 100644 --- a/docs/development.md +++ b/docs/development.md @@ -39,7 +39,7 @@ go build -trimpath -ldflags "-X github.com/mixeme/selfpost/internal/buildinfo.Ve ``` **Env documentation regression:** `go test ./cmd/panel -run TestLoadConfig` — -new `loadConfig` keys must appear in README env lists +new `loadConfig` keys must appear in guide.md env lists ([cmd/panel/envdoc_test.go](../cmd/panel/envdoc_test.go)). --- diff --git a/docs/documentation-plan.md b/docs/documentation-plan.md index 37b848e..dabb77c 100644 --- a/docs/documentation-plan.md +++ b/docs/documentation-plan.md @@ -9,7 +9,7 @@ | Дом | Файл | |---|---| -| Пользовательская поставка | [README.md](../README.md) | +| Пользовательская поставка | [README.md](../README.md) + [guide.md](guide.md) | | Границы продукта | [product.md](product.md) | | As-built устройство | [architecture.md](architecture.md) | | Процесс разработки | [development.md](development.md) | @@ -28,10 +28,11 @@ | Артефакт | Состояние | |---|---| -| [README.md](../README.md) | Установка, площадка, DNS, прогрев IP, эксплуатация, rate limiting, env, бэкап, репозиторий, образ, лицензия | +| [README.md](../README.md) | Краткий обзор, требования, quick start, ссылки на документацию, reference deploy, лицензия | +| [guide.md](guide.md) | Прокси, env, DNS, прогрев IP, эксплуатация, rate limiting, бэкап, порты, тег образа | | [LICENSE](../LICENSE) | AGPL-3.0, полный текст | | [deploy/docker-compose.yml](../deploy/docker-compose.yml) + прокси | Apache + nginx/Caddy/Traefik в [deploy/](../deploy/) | -| [deploy/.env.example](../deploy/.env.example) | Публичные переменные; полный справочник в README | +| [deploy/.env.example](../deploy/.env.example) | Публичные переменные; полный справочник в [guide.md](guide.md) | | [CHANGELOG.md](../CHANGELOG.md) | Keep a Changelog | ### Рабочие документы @@ -57,12 +58,12 @@ | Сессии | [internal/store/sessions.go](../internal/store/sessions.go), [internal/web/session.go](../internal/web/session.go) | | Ротация лога, reload | [build/logrotate-mail.conf](../build/logrotate-mail.conf), [build/logrotate-loop.sh](../build/logrotate-loop.sh), [build/postfix-cert-reload.sh](../build/postfix-cert-reload.sh) | | Деплой | [deploy/docker-compose.yml](../deploy/docker-compose.yml), [build/Dockerfile](../build/Dockerfile) | -| Чеклист README | таблица «Поставляемое пользователю» выше | +| Чеклист README | таблица «Поставляемое пользователю» выше; детали — [guide.md](guide.md) | | Продукт, out of scope | [product.md](product.md) | | As-built | [architecture.md](architecture.md) | | Обязательная безопасность | [security.md](security.md) | -Порядок: перечислить фактическое в коде → найти в README / `architecture.md`. +Порядок: перечислить фактическое в коде → найти в [guide.md](guide.md) / `architecture.md`. Перед каждым тегом — короткий проход по этой таблице, не полная ревизия текста. --- @@ -70,7 +71,7 @@ ## 3. Правила поддержки 1. **Правило шага:** новая/переименованная env-переменная, маршрут панели или - наблюдаемое поведение почтового тракта закрываются вместе с README / + наблюдаемое поведение почтового тракта закрываются вместе с [guide.md](guide.md) / `.env.example` и записью в CHANGELOG (протокол — [progress.md](progress.md)). 2. **Регресс env (D7):** [cmd/panel/envdoc_test.go](../cmd/panel/envdoc_test.go) — падает на недокументированном ключе `loadConfig` или build-скриптов. diff --git a/docs/guide.md b/docs/guide.md new file mode 100644 index 0000000..55521f6 --- /dev/null +++ b/docs/guide.md @@ -0,0 +1,331 @@ +# SelfPost operator guide + +Detailed install, configuration, and day-to-day operations. For a short +overview and quick start, see [README.md](../README.md). + +## Table of contents + +- [Reverse proxy (mandatory)](#reverse-proxy-mandatory) +- [Environment variables](#environment-variables) +- [DNS setup](#dns-setup) +- [IP warmup](#ip-warmup) +- [Operations](#operations) +- [Rate limiting](#rate-limiting) +- [Backup, restore, and moving a single domain](#backup-restore-and-moving-a-single-domain) + - [Encrypting a backup or export](#encrypting-a-backup-or-export) +- [Published ports](#published-ports) +- [Fixed image tag](#fixed-image-tag) + +## Reverse proxy (mandatory) + +SelfPost's panel speaks plain HTTP and never terminates TLS itself — a reverse +proxy in front of it is not optional. The proxy is also the project's only +source of TLS certificates: whatever it obtains via ACME/Let's Encrypt gets +bind-mounted **read-only** into the SelfPost container, and Postfix uses those +same PEM files for TLS on 465 (and 587, if enabled). If the panel and the mail +service share one hostname — the common case — it's genuinely one certificate +serving both. + +SelfPost isn't tied to a specific proxy; pick whichever fits your host: + +| Proxy | Where certs live | Fragment | +|---|---|---| +| **Apache** (default/recommended) | Host disk, via the certbot Apache plugin — PEM files ready to bind-mount, no extraction step. | [deploy/apache/selfpost-vhost.conf](../deploy/apache/selfpost-vhost.conf) | +| nginx | Host disk, via a certbot sidecar container — same PEM-ready shape as Apache. | [deploy/nginx/](../deploy/nginx/) | +| Caddy | Automatic ACME, zero extra containers — simplest, but its on-disk cert path is versioned internal layout, not a stable API; verify it for the Caddy version you run. | [deploy/caddy/](../deploy/caddy/) | +| Traefik | Bundled inside `acme.json` — needs a small extraction script to produce standalone PEM files. | [deploy/traefik/](../deploy/traefik/) | + +Apache is the recommended default because the certbot Apache plugin already +writes plain `fullchain.pem`/`privkey.pem` files to a predictable path with no +extra moving parts between "certificate issued" and "Postfix can read it." + +**The proxy needs no security configuration of its own.** The panel emits its +own `Content-Security-Policy`, `Strict-Transport-Security`, `X-Frame-Options`, +`X-Content-Type-Options` and `Referrer-Policy` — deliberately, so the part +that's easy to get wrong lives in the service rather than in a config file +somebody edits under pressure. There is exactly one thing the proxy must do: +**pass the original `Host` header through**. All four fragments above already +do (Apache `ProxyPreserveHost On`, nginx `proxy_set_header Host $host`, Caddy +and Traefik by default). A proxy that rewrites `Host` instead makes the panel +reject every form submission as cross-origin — the log says so explicitly, +printing the `Origin` and `Host` it compared. + +## Environment variables + +Copy [deploy/.env.example](../deploy/.env.example) to `.env` next to your +`docker-compose.yml`. The table below lists every variable an operator is +expected to set; defaults match the code exactly. + +| Variable | Purpose | Default | Set in | +|---|---|---|---| +| `SELFPOST_HOSTNAME` | Mail-server identity: Postfix HELO/EHLO, SASL realm, certificate CN/SAN, and the hostname the PTR check expects. Bare FQDN only — no scheme or port. | *(required)* | `.env` | +| `SUBMISSION_ENABLE` | When `true`, also listen on port 587 with STARTTLS (RFC 6409 submission) alongside the primary 465/smtps listener. | `false` | `.env` | +| `RATE_LIMIT_MESSAGES_PER_IP` | Level-1 backstop: maximum messages one client IP may submit per window (Postfix `smtpd_client_message_rate_limit`). See [Rate limiting](#rate-limiting). | `100` | `.env` | +| `RATE_LIMIT_WINDOW_SECONDS` | Level-1 window length in seconds (Postfix `anvil_rate_time_unit`). | `3600` | `.env` | +| `SEND_LOG_RETENTION_DAYS` | Days of send-log history kept before the background sweep deletes rows — the main driver of `/data` growth over time. | `90` | `.env` | +| `PANEL_SESSION_IDLE_DAYS` | Sliding idle timeout for the panel login session, in days. There is no absolute cap: an admin who keeps coming back stays signed in indefinitely. | `7` | `.env` | +| `SELFPOST_DNS_RESOLVERS` | Comma-separated recursive resolvers the panel's PTR/SPF/DKIM/DMARC checks query directly (so they report what the internet sees, not what this host's stub resolver synthesises). | `1.1.1.1:53`, `8.8.8.8:53`, `9.9.9.9:53` when unset | `.env` | +| `TRUSTED_PROXY_CIDR` | Comma-separated CIDRs (bare IPs allowed) of reverse proxies allowed to supply `X-Forwarded-For` for login, setup, and account-change rate-limiting. **Leave unset unless you know the exact address of your reverse proxy.** A wrong value lets a client spoof its rate-limit key by sending a forged `X-Forwarded-For` header — the panel trusts the last hop only when the TCP peer matches one of these CIDRs. Behind the default Apache host-network setup this is typically the Docker bridge gateway, e.g. `172.18.0.1`. | *(empty — XFF ignored)* | `.env` | + +TLS certificate paths (`TLS_CERT_FILE`, `TLS_KEY_FILE`) are fixed in +[deploy/docker-compose.yml](../deploy/docker-compose.yml) to match the `./certs` +bind mount — configure the mount, not these variables. + +**Internal variables (not part of the operator interface).** The following are +read by the panel or startup scripts but are not meant to be changed in a +normal deployment; documenting them here avoids treating accidental overrides as +supported configuration: + +- **Panel paths and tuning:** `SELFPOST_DATA_DIR` (`/data`), `SELFPOST_DB_PATH` + (`/data/selfpost.db`), `SELFPOST_SETUP_TOKEN_FILE` + (`/data/setup-token`), `PANEL_HTTP_ADDR` (`:8080`), + `JOURNAL_MILTER_SOCKET` (`/run/selfpost/journal.sock`), `MAIL_LOG` + (`/var/log/mail.log`), `PANEL_COOKIE_SECURE` (`true`), `OPENDKIM_SOCKET` + (`/run/opendkim/opendkim.sock`), `OPENDKIM_DIR` (`/data/opendkim`), + `DKIM_SELECTOR_DEFAULT` (`selfpost`), `SASL_DB_PATH` + (`/data/sasl/sasldb2`), `SASL_REALM` (defaults to `SELFPOST_HOSTNAME`), + `POSTFIX_DIR` (`/data/postfix`), `POSTFIX_SENDER_LOGIN_MAPS` + (`/data/postfix/sender_login_maps`). +- **Milter and Postfix startup:** `MILTER_CONNECT_TIMEOUT` (`15s`), + `MILTER_COMMAND_TIMEOUT` (`15s`), `MILTER_CONTENT_TIMEOUT` (`30s`), + `MILTER_WAIT_TIMEOUT` (`30` seconds). +- **Background maintenance:** `TLS_RELOAD_INTERVAL_SECONDS` (`86400` — daily + `postfix reload` to pick up renewed certificates), + `LOGROTATE_INTERVAL_SECONDS` (`21600` — check `mail.log` rotation every six + hours; rotated logs are kept 14 days and each rotation triggers + `postfix reload`). + +## DNS setup + +Two different scopes — don't confuse them: + +**Server level (once, for the machine itself):** +- **PTR/rDNS** for the server's IP, pointing at its mail hostname. Most + receiving mail servers weigh this heavily; get it from whoever assigns the IP + (hosting provider's panel/support), not from your own DNS zone. + +**Domain level (for *every* sending domain you add in the panel):** +- **SPF** — a TXT record on the domain authorizing this server to send on its + behalf (e.g. `v=spf1 a mx ip4: -all`, adjusted to your setup). +- **DKIM** — a TXT record with the exact value the panel shows on that + domain's page (`domain page → DKIM TXT record`), one selector per domain. +- **DMARC** — a `_dmarc` TXT record (even a conservative `p=none` starts + building reporting/reputation history). + +Skipping any of the three per-domain records is the single most common reason +mail lands in spam even though SelfPost delivered it correctly — DKIM passing +doesn't help if SPF/DMARC are absent. **Whenever you add a new domain in the +panel, add its DNS records at the same time**, not later. + +The panel checks both scopes for you and tells you what is actually published: +the *Status* page verifies the server's hostname and its reverse record +(forward-confirmed reverse DNS), and each domain's page shows a *DNS status* +card comparing the published DKIM record against the key this server signs with, +plus the domain's SPF and DMARC records. Results are cached for a few minutes; +use *Re-check* right after publishing a record. The SPF check is deliberately +shallow — it looks for a mechanism that literally covers this server's address +and does not follow `include:` or `redirect=`, so a record that authorizes the +server through an include is reported as "cannot tell" rather than as a failure. + +## IP warmup + +A brand-new IP has no sending history, so receiving servers are cautious with +it regardless of how correct your DKIM/SPF/DMARC are. Start with low volume to +a domain, increase gradually over days/weeks rather than sending everything on +day one, and check the IP against major blocklists (Spamhaus and similar) +before and during warmup. This is inherent to how mail reputation works on the +public internet, not something SelfPost's configuration can shortcut. + +## Operations + +After sign-in the panel opens on **Status** — the place to answer "is the +service healthy and will mail be accepted?" + +- **Status** (`/status`) — supervised processes (Postfix, OpenDKIM, panel), + TLS certificate validity and expiry, milter socket presence, and a short + Postfix queue summary. The **Machine** card adds the resource usage of the + host underneath — processor (with the load average), memory and swap, and + per-interface network throughput and totals — read from the kernel's + counters; CPU and throughput are measured between refreshes, so they appear + one refresh after the page opens. A fully busy processor or a machine out of + memory is a warning here, because both delay or kill the mail path; + throughput is only reported. The hostname block compares `SELFPOST_HOSTNAME` + against the PTR record the internet publishes for this server's IP + (forward-confirmed reverse DNS); use *Re-check* after changing DNS. The + **Reload configuration** button re-applies OpenDKIM tables and the Postfix + sender map from the database — use it if daemons drifted from what the panel + shows after manual edits under `/data`. +- **Domains** (`/domains`) — add sending domains, inspect each domain's DKIM + TXT value, SPF/DMARC checks, and SASL applications. Per-domain rate limits + (level 2) are configured here. *Export domain* writes a single-domain archive; + *Import a domain* on the Backup page reads one back in. +- **Deliveries** (`/deliveries`) — searchable send log with server-side filters + by domain and application. A row identifies its message and nothing more — + time, sender, recipient, subject and status `queued` (accepted, not yet + delivered), `sent` (handed off successfully), or `rejected` (refused — for + example by a level-2 rate limit); *Details* opens that row's own page + (`/deliveries/{id}`). That page carries the sending domain, the application it + was submitted under, the Postfix queue id and the journal id, beside the + message's history — when it was accepted and what Postfix later reported for + the recipient — and, under both, the `mail.log` lines for its queue id: the + connection to the receiving server, the server's reply, and the status that + reply was filed as. Rows outlive `mail.log`, so an older message's lines may + have rotated away; the page says so. Retention is controlled by + `SEND_LOG_RETENTION_DAYS`. +- **Mail queue** (`/mail-queue`) — live view of messages Postfix is still + trying to deliver or deferring. +- **System log** (`/system-log`) — tail of `/var/log/mail.log` (Postfix and + related daemon lines). The log rotates daily (14 files kept) with a + `postfix reload` after each rotation; a background loop checks every six + hours. +- **Backup** (`/backup`) — download a full-server backup or import a + single-domain export. See [Backup, restore, and moving a single domain](#backup-restore-and-moving-a-single-domain). +- **Account** (`/account`) — change the administrator username and/or password. + Application SASL logins are separate and are not changed here. + +**Sessions.** A login survives a container restart: sessions live in SQLite, not +in memory. Expiry is a sliding idle window (`PANEL_SESSION_IDLE_DAYS`, default +seven days) with no absolute lifetime cap — an admin who keeps using the panel +stays signed in indefinitely. HTMX polling on the monitoring screens +(Deliveries, Mail queue, System log, and the Status health fragment) does +**not** count as activity, so an auto-refreshing tab left open will not keep a +session alive forever. Changing the password signs out every other session but +leaves the current browser signed in. + +**Upgrading.** Bump the pinned image tag in `docker-compose.yml` to the target +release, then `docker compose up -d`. The backup version check requires the +running image to match the version that created a full backup — see [Fixed image +tag](#fixed-image-tag). + +**Container health.** The image declares a Docker `HEALTHCHECK` that probes +`GET /healthz` on port 8080 (unauthenticated). It returns `200 ok` when +OpenDKIM, the panel, and Postfix are all `RUNNING` under supervisord; +otherwise `503 unhealthy`. This catches a dead mail path that would still leave +the HTTP server up, but it does **not** verify TLS certificates, DNS records, +or end-to-end delivery — use the authenticated **Status** page for that. External +monitoring can use the same endpoint through the reverse proxy if you expose it, +or poll `docker inspect` health state on the host. + +**First-time setup link.** On first start the one-time setup URL is printed in +the container log (`docker compose logs -f`) and written to `/data/setup-token` +inside the container — `./data/setup-token` on the host, mode `0600` — then +deleted when setup completes. The link is a bearer token valid for ten minutes. +If this host ships container logs to a central aggregator, prefer reading the +file: + +```sh +docker compose exec selfpost cat /data/setup-token +``` + +## Rate limiting + +SelfPost applies two independent limits; both can refuse a submission, but only +level 2 writes a `rejected` row in the send log. + +**Level 1 (IP backstop)** — always on, configured via `.env`: + +- `RATE_LIMIT_MESSAGES_PER_IP` → Postfix `smtpd_client_message_rate_limit` +- `RATE_LIMIT_WINDOW_SECONDS` → Postfix `anvil_rate_time_unit` + +This is an anvil limit per connecting client IP. It keeps working even if the +journal-milter (level 2) is down. + +**Level 2 (per domain / per application)** — optional, configured in the panel +on each domain's page or on an individual application. You set a message +ceiling, a time window, and optionally restrict the limit to specific client +IPs; an empty IP list means the differentiated limit does not apply. When +exceeded, Postfix returns a 4xx and the refusal is recorded in Deliveries as +`rejected`. + +## Backup, restore, and moving a single domain + +Two related but distinct operations — spec 7.5: + +- **Full backup** (whole `/data`: SQLite, all domains' DKIM keys, all + applications' SASL credentials, `manifest.json` with the version that + created it): panel button (*Backup* → *Full backup*), or from the + host: + ```sh + docker exec selfpost-backup > selfpost-backup.tar.gz + ``` + **Restore** means unpacking that archive into a fresh `/data` bind mount and + starting a container of the **exact same image version** that created it — + SelfPost refuses to start otherwise and tells you which tag to use. On the + first successful start after restore, `manifest.json` from the archive is + **deleted** — it guards only that one boot, so a later in-place upgrade is + not blocked. This is why the compose file pins a fixed tag rather than + `:latest`: without a known version, there'd be no way to tell which image + restoring a given backup actually requires. + + **Alternative: archive `./data` while stopped.** If the service can be taken + offline, `docker compose down` then `tar czf selfpost-data.tar.gz ./data` on + the host is safe — nothing is writing to SQLite. Do **not** tar `./data` while + the container is running: the database uses WAL mode and a naive copy can + capture an inconsistent snapshot. The panel/CLI backup remains preferable when + you cannot afford downtime because it takes a consistent SQLite snapshot via + the Backup API on a live container. + +- **Export/import a single domain** (domain page → *Export domain* to write the + file, *Backup* → *Import a domain* to read it back in): moves one domain — its DKIM key and its applications' **working** + SASL passwords — to a different SelfPost instance without regenerating + anything, so DNS (the DKIM TXT record) doesn't need to change. Unlike a full + restore, this works across different hostnames/instances. + +Both files are **secrets** — they contain the admin password hash (full +backup) or working application credentials (domain export) in the clear or in +directly reversible form. Treat them like any other credential material: +restrict who can read them, don't email them around — and encrypt them, which +SelfPost can do for you. + +### Encrypting a backup or export + +Both download forms carry an **Encrypt with a password** checkbox. Ticked, the +file that comes down is an encrypted envelope instead of the plain archive: + +| Artefact | Plain | Encrypted | +|----------|-------|-----------| +| Full backup | `.tar.gz` | `.spbk` | +| Domain export | `.json` | `.spde` | + +The key is derived from the password with scrypt and the contents are sealed +with AES-256-GCM, in chunks, so a truncated or altered file fails to open rather +than restoring quietly. **SelfPost does not store the password** — lose it and +the file is unrecoverable, which is the entire point. + +*Import a domain* takes an encrypted export directly: tick **The file is +encrypted** and give the password. + +A full backup has to be turned back into a plain archive before it can be +unpacked into `/data`, which the CLI does with the same password: + +```sh +docker exec -i selfpost-backup -decrypt < backup.spbk > backup.tar.gz +``` + +The CLI also *writes* encrypted backups for scripted/cron use. The password +comes from `SELFPOST_BACKUP_PASSWORD` or `-password-file ` (first line), +never from a command-line argument, which would be visible in the process list: + +```sh +docker exec -e SELFPOST_BACKUP_PASSWORD="$PW" selfpost-backup > backup.spbk +``` + +With no password set, the CLI keeps writing the plain `.tar.gz` it always has. + +## Published ports + +`deploy/docker-compose.yml` maps **465** and **587** to the host. Port 465 +(smtps) is always active. Port **587** is published even when +`SUBMISSION_ENABLE=false`; nothing listens until you set it to `true` — harmless, +but it can look like an open port in external scans. + +## Fixed image tag + +`deploy/docker-compose.yml` pins an explicit version (`ghcr.io/mixeme/selfpost:X.Y.Z`), +deliberately never `:latest`. This is a direct consequence of the backup +version check above: the panel binary's embedded version and the image tag +that produced it are the same value by construction (the release CI stamps +both from one git tag — see `.github/workflows/release.yml`), so pinning the +tag is what makes "restore into the same version" a checkable fact rather than +a guess. Upgrade by bumping the tag deliberately, not by riding a moving +target. diff --git a/docs/product.md b/docs/product.md index f889d43..1ef0893 100644 --- a/docs/product.md +++ b/docs/product.md @@ -2,8 +2,9 @@ **What this file is.** Stable product definition for SelfPost v1.0: purpose, deployment assumptions, explicit out-of-scope items, and the multi-domain -model. User-facing install and operations live in [README.md](../README.md); -as-built technical detail in [architecture.md](architecture.md). +model. User-facing overview in [README.md](../README.md); install and operations +in [guide.md](guide.md); as-built technical detail in +[architecture.md](architecture.md). --- diff --git a/internal/app/service.go b/internal/app/service.go index 43beb3e..d9bb886 100644 --- a/internal/app/service.go +++ b/internal/app/service.go @@ -174,7 +174,7 @@ func (s *Service) Delete(id int64) error { if err := s.sasl.Delete(a.Login); err != nil { return err } - // Drop the application's level-2 limit, if any (README § Rate limiting); + // Drop the application's level-2 limit, if any (guide § Rate limiting); // rate_limits has no cascade of its own. if err := s.store.DeleteRateLimit(store.RateLimitScopeApp, id); err != nil { return err @@ -182,7 +182,7 @@ func (s *Service) Delete(id int64) error { return s.Resync() } -// RateLimit returns the application-level differentiated rate limit (README § +// RateLimit returns the application-level differentiated rate limit (guide § // Rate limiting), and whether one is configured, for the application's edit // form. func (s *Service) RateLimit(appID int64) (store.RateLimit, bool, error) { @@ -202,7 +202,7 @@ func (s *Service) SaveRateLimit(appID int64, ips []string, maxMessages, windowSe }) } -// ClearRateLimit removes the application-level rate limit (README § Rate +// ClearRateLimit removes the application-level rate limit (guide § Rate // limiting). func (s *Service) ClearRateLimit(appID int64) error { return s.store.DeleteRateLimit(store.RateLimitScopeApp, appID) diff --git a/internal/domain/service.go b/internal/domain/service.go index 44f77f9..f0bc64c 100644 --- a/internal/domain/service.go +++ b/internal/domain/service.go @@ -41,7 +41,7 @@ type Service struct { } // NewService builds the domain service. selectorDefault is the DKIM selector -// assigned to new domains (README § Environment variables: +// assigned to new domains (guide § Environment variables: // DKIM_SELECTOR_DEFAULT); it is operator-configured, not user input. apps is // used only on deletion, to clear the SASL accounts and sender-map bindings of // the domain's applications. @@ -111,7 +111,7 @@ func (s *Service) Delete(id int64) error { } // Drop the domain's own level-2 limit and those of its applications while the // application rows still exist (the cleanup query joins them). rate_limits - // has no cascade of its own (ref_id is a plain integer, README § Rate + // has no cascade of its own (ref_id is a plain integer, guide § Rate // limiting; architecture.md § Persistence). if err := s.store.DeleteRateLimitsForDomain(id); err != nil { return fmt.Errorf("clear rate limits for %s: %w", d.Name, err) @@ -138,7 +138,7 @@ func (s *Service) DKIMRecord(d store.Domain) (DKIMRecord, error) { return s.odk.Record(d.Name, d.DKIMSelector) } -// RateLimit returns the domain-level differentiated rate limit (README § Rate +// RateLimit returns the domain-level differentiated rate limit (guide § Rate // limiting), and whether one is configured, for the domain's edit form. func (s *Service) RateLimit(domainID int64) (store.RateLimit, bool, error) { return s.store.GetRateLimit(store.RateLimitScopeDomain, domainID) @@ -158,7 +158,7 @@ func (s *Service) SaveRateLimit(domainID int64, ips []string, maxMessages, windo } // ClearRateLimit removes the domain-level rate limit, falling back to level 1 -// only (README § Rate limiting). +// only (guide § Rate limiting). func (s *Service) ClearRateLimit(domainID int64) error { return s.store.DeleteRateLimit(store.RateLimitScopeDomain, domainID) } diff --git a/internal/health/cert.go b/internal/health/cert.go index 221253a..8831396 100644 --- a/internal/health/cert.go +++ b/internal/health/cert.go @@ -14,7 +14,7 @@ import ( const certWarnDays = 14 // Certificate is the state of the TLS certificate Postfix serves on 465/587 -// (README § Environment variables: TLS_CERT_FILE). The panel only reads it — +// (guide § Environment variables: TLS_CERT_FILE). The panel only reads it — // the file is supplied by the reverse proxy through a read-only mount. type Certificate struct { Path string diff --git a/internal/logtail/logtail.go b/internal/logtail/logtail.go index 314efc3..aa322c6 100644 --- a/internal/logtail/logtail.go +++ b/internal/logtail/logtail.go @@ -40,7 +40,7 @@ const ( // startup). The window itself is configurable; the cadence need not be. retentionInterval = 6 * time.Hour // defaultRetentionDays applies when the configured value is unset/invalid - // (README § Environment variables: SEND_LOG_RETENTION_DAYS). + // (guide § Environment variables: SEND_LOG_RETENTION_DAYS). defaultRetentionDays = 90 ) @@ -280,12 +280,19 @@ func isQueueIDByte(b byte) bool { return b >= '0' && b <= '9' || b >= 'A' && b <= 'Z' || b >= 'a' && b <= 'z' } -// Timestamps at the head of a mail.log line. The first is what Postfix's own -// postlogd writes, which is what this server runs (maillog_file in -// build/postfix-config.sh) — RFC 3339 down to microseconds and with an offset. -// The second is syslog's traditional format, for a deployment that routes the -// log through syslogd instead; it carries no year and no zone, which is why it -// is not the one being matched first. +// Timestamps at the head of a mail.log line, in the two formats postlogd +// writes (maillog_file in build/postfix-config.sh). +// +// syslogStampRe is the one that matches in practice today: the format is +// controlled by maillog_file_format, which arrived in Postfix 3.9, and the +// image is built on Debian's 3.7 — where the parameter does not exist and the +// only format is syslog's traditional one. It carries no year and no zone, so +// the stamp shown is a wall clock and nothing more, which is all this column +// claims to be. +// +// isoStampRe is for the RFC 3339 format that same parameter selects once the +// base image carries a Postfix new enough to offer it. Matching it first costs +// one failed anchor per line and means the upgrade needs no change here. var ( isoStampRe = regexp.MustCompile(`^(\d{4}-\d{2}-\d{2})T(\d{2}:\d{2}:\d{2})(?:\.\d+)?(?:Z|[+-]\d{2}:?\d{2})?\s`) syslogStampRe = regexp.MustCompile(`^([A-Z][a-z]{2}\s+\d{1,2} \d{2}:\d{2}:\d{2})\s`) diff --git a/internal/logtail/logtail_test.go b/internal/logtail/logtail_test.go index 69eac1d..def7e61 100644 --- a/internal/logtail/logtail_test.go +++ b/internal/logtail/logtail_test.go @@ -465,7 +465,7 @@ func TestSplitTimestamp(t *testing.T) { stamp, rest string }{ { - name: "postlogd, which is what this server writes", + name: "RFC 3339, which maillog_file_format selects on Postfix 3.9 and up", line: "2026-08-03T05:15:52.219218+00:00 mail postfix/smtp[26]: 4A1B2C3D: to=, status=sent (250 OK)", stamp: "2026-08-03 05:15:52", rest: "mail postfix/smtp[26]: 4A1B2C3D: to=, status=sent (250 OK)", @@ -483,11 +483,20 @@ func TestSplitTimestamp(t *testing.T) { rest: "mail opendkim[30]: 4A1B2C3D: DKIM-Signature field added", }, { - name: "syslog's traditional format, padded day", + name: "syslog's traditional format, space-padded day", line: "Aug 3 05:15:52 mail postfix/smtpd[20]: 4A1B2C3D: client=app.example.ru[203.0.113.4]", stamp: "Aug 3 05:15:52", rest: "mail postfix/smtpd[20]: 4A1B2C3D: client=app.example.ru[203.0.113.4]", }, + { + // Copied off the live relay (Postfix 3.7, which has no + // maillog_file_format), so this is the shape the panel actually + // meets: zero-padded day, and the host is the container's name. + name: "syslog's traditional format as the live relay writes it", + line: "Aug 08 07:26:41 selfpost postfix/master[231]: daemon started -- version 3.7.11, configuration /etc/postfix", + stamp: "Aug 08 07:26:41", + rest: "selfpost postfix/master[231]: daemon started -- version 3.7.11, configuration /etc/postfix", + }, { name: "unrecognised head keeps the whole line", line: "mail postfix/smtp[26]: 4A1B2C3D: to=, status=sent (250 OK)", diff --git a/internal/milter/milter.go b/internal/milter/milter.go index d15a062..cb6808e 100644 --- a/internal/milter/milter.go +++ b/internal/milter/milter.go @@ -27,7 +27,7 @@ import ( // Store is the persistence the milter needs on the receive path: recording // accepted messages (architecture.md § Mail path) and, for level-2 rate -// limiting (README § Rate limiting), looking up the configured limits and +// limiting (guide § Rate limiting), looking up the configured limits and // counting recent messages. *store.Store satisfies it; tests substitute a // fake. type Store interface { @@ -76,7 +76,7 @@ func (s *session) Connect(host, family string, port uint16, addr net.IP, m *milt // macros). This is also the earliest stage where both the sending domain (from // the sender) and the application (the login) are known, so the level-2 rate // limit is enforced here: over the limit, the message is refused with a 4xx -// tempfail before recipients are even offered (README § Rate limiting). +// tempfail before recipients are even offered (guide § Rate limiting). // Enforcement is fail-open — see overLimit. func (s *session) MailFrom(from string, m *milter.Modifier) (milter.Response, error) { s.releaseReservations() // a previous transaction that ended without EOM/ABORT diff --git a/internal/milter/ratelimit.go b/internal/milter/ratelimit.go index 418d4b8..d7a5645 100644 --- a/internal/milter/ratelimit.go +++ b/internal/milter/ratelimit.go @@ -8,7 +8,7 @@ import ( ) // overLimit reports whether the message currently being received should be -// refused under a level-2 differentiated limit (README § Rate limiting). It +// refused under a level-2 differentiated limit (guide § Rate limiting). It // checks the domain-level and application-level limits in turn; either being // exceeded is enough to refuse. // @@ -80,7 +80,7 @@ func (s *session) releaseReservations() { } // recordRejected writes a send-log row for a message refused by a level-2 -// limit (README § Rate limiting — refusals are recorded too), so the rejection +// limit (guide § Rate limiting — refusals are recorded too), so the rejection // shows up in the monitoring screen. Only MAIL-stage fields are known; the // write is best-effort and never affects the response. func (s *session) recordRejected() { diff --git a/internal/store/ratelimits.go b/internal/store/ratelimits.go index a2d6010..0b7c186 100644 --- a/internal/store/ratelimits.go +++ b/internal/store/ratelimits.go @@ -9,7 +9,7 @@ import ( "time" ) -// Rate-limit scopes (README § Rate limiting). A level-2 limit is attached +// Rate-limit scopes (guide § Rate limiting). A level-2 limit is attached // either to a domain (counted across all its applications and IPs) or to a // single application. const ( @@ -17,7 +17,7 @@ const ( RateLimitScopeApp = "application" ) -// RateLimit is a differentiated level-2 rate limit (README § Rate limiting): +// RateLimit is a differentiated level-2 rate limit (guide § Rate limiting): // an optional set of expected client IPs plus a message ceiling over a sliding // window, attached to a domain or an application. It is enforced in the // journal-milter; level 1 (Postfix anvil, architecture.md § Mail path) is the @@ -27,7 +27,7 @@ const ( // Both the IP binding and the ceiling are optional in the schema, but a limit // is only enforced when it is Active(): the design deliberately allows an // admin to leave the IP binding empty for apps that send from changing IPs, in -// which case only level 1 protects them (README § Rate limiting). +// which case only level 1 protects them (guide § Rate limiting). type RateLimit struct { Scope string RefID int64 @@ -38,7 +38,7 @@ type RateLimit struct { // Active reports whether the limit is fully configured and should be enforced. // A missing IP binding, ceiling or window leaves the differentiated limit -// inert (README § Rate limiting): the IP binding is what scopes the limit to a +// inert (guide § Rate limiting): the IP binding is what scopes the limit to a // known sender. func (r RateLimit) Active() bool { return len(r.AllowedIPs) > 0 && r.MaxMessages > 0 && r.WindowSeconds > 0 @@ -153,7 +153,7 @@ func (s *Store) RateLimit(scope, ref string) (RateLimit, bool, error) { // CountMessages returns how many distinct messages the reference (a domain // name or an application login) has queued since t, for the level-2 sliding -// window (README § Rate limiting). It counts distinct queue-ids — one message +// window (guide § Rate limiting). It counts distinct queue-ids — one message // with many recipients is one message, matching level 1's per-message // semantics — and excludes rows that were themselves rejected by a limit (they // were never sent). It reuses the send log the journal already writes (README diff --git a/internal/store/sendlog.go b/internal/store/sendlog.go index 1d9952e..66f0054 100644 --- a/internal/store/sendlog.go +++ b/internal/store/sendlog.go @@ -17,7 +17,7 @@ const ( StatusDeferred = "deferred" StatusBounced = "bounced" // StatusRejected marks a message the journal-milter refused with a 4xx under - // a level-2 rate limit (README § Rate limiting). Such a row never gets a + // a level-2 rate limit (guide § Rate limiting). Such a row never gets a // queue-id and is excluded from the level-2 message count (it was never // sent). StatusRejected = "rejected" @@ -55,7 +55,7 @@ func (s *Store) InsertQueued(e SendLogEntry) error { } // InsertRejected records a message the journal-milter refused under a level-2 -// rate limit (README § Rate limiting), so the rejection is visible in the +// rate limit (guide § Rate limiting), so the rejection is visible in the // send-log UI. Only the fields known at MAIL FROM are set (domain, sender, app // login); there is no queue-id or recipient because the message was rejected // before it was queued. diff --git a/internal/web/handlers_apps.go b/internal/web/handlers_apps.go index 700280d..0da9824 100644 --- a/internal/web/handlers_apps.go +++ b/internal/web/handlers_apps.go @@ -30,7 +30,7 @@ type detailView struct { FormAddrs string NewCred *newCred // RateLimitErr surfaces a validation error from a domain- or - // application-level rate-limit form (README § Rate limiting) as a page + // application-level rate-limit form (guide § Rate limiting) as a page // banner. RateLimitErr string // ExportErr surfaces a rejected encryption password from the export card. diff --git a/internal/web/handlers_ratelimit.go b/internal/web/handlers_ratelimit.go index 021d875..410b7ec 100644 --- a/internal/web/handlers_ratelimit.go +++ b/internal/web/handlers_ratelimit.go @@ -11,13 +11,13 @@ import ( ) // defaultRateLimitWindowSeconds is the sliding-window length used when an -// admin sets a message ceiling but leaves the window blank (README § Rate -// limiting, matching the level-1 default hour; README § Environment variables: +// admin sets a message ceiling but leaves the window blank (guide § Rate +// limiting, matching the level-1 default hour; guide § Environment variables: // RATE_LIMIT_WINDOW_SECONDS). const defaultRateLimitWindowSeconds = 3600 // rateLimitInput is the validated result of a rate-limit form submission. -// clear means "remove the differentiated limit" (README § Rate limiting: an +// clear means "remove the differentiated limit" (guide § Rate limiting: an // empty IP binding leaves only level 1). type rateLimitInput struct { clear bool @@ -42,7 +42,7 @@ func parseRateLimitForm(r *http.Request) (rateLimitInput, error) { return rateLimitInput{}, err } if len(ips) == 0 { - // No IP binding: the differentiated limit does not apply (README § Rate + // No IP binding: the differentiated limit does not apply (guide § Rate // limiting). return rateLimitInput{clear: true}, nil } @@ -92,7 +92,7 @@ func parsePositiveInt(raw string, def int) (int, error) { } // handleDomainRateLimit saves or clears a domain-level differentiated rate -// limit (README § Rate limiting). No reload is needed — the milter reads the +// limit (guide § Rate limiting). No reload is needed — the milter reads the // row live. func (s *Server) handleDomainRateLimit(w http.ResponseWriter, r *http.Request) { d, ok := s.lookupDomain(w, r) @@ -116,7 +116,7 @@ func (s *Server) handleDomainRateLimit(w http.ResponseWriter, r *http.Request) { } // handleAppRateLimit saves or clears an application-level differentiated rate -// limit (README § Rate limiting). +// limit (guide § Rate limiting). func (s *Server) handleAppRateLimit(w http.ResponseWriter, r *http.Request) { a, ok := s.lookupApplication(w, r) if !ok { diff --git a/internal/web/web.go b/internal/web/web.go index aa3a206..2b5f83a 100644 --- a/internal/web/web.go +++ b/internal/web/web.go @@ -24,7 +24,7 @@ var assetsFS embed.FS // Config holds the panel's HTTP-facing configuration. type Config struct { // Hostname is the server's external hostname, used to build the absolute - // setup link shown in the logs (security.md; README § Environment + // setup link shown in the logs (security.md; guide § Environment // variables for SELFPOST_HOSTNAME). Hostname string // CookieSecure sets the Secure attribute on the session cookie. It defaults @@ -53,7 +53,7 @@ type Config struct { // honoured, so the header can't be spoofed by anyone but a trusted proxy. // Empty (the default) keeps rate-limiting keyed on RemoteAddr only. TrustedProxyCIDRs []*net.IPNet - // TLSCertFile is the certificate Postfix serves on 465/587 (README § + // TLSCertFile is the certificate Postfix serves on 465/587 (guide § // Environment variables), read read-only by the status page to report how // much validity is left. TLSCertFile string diff --git a/test/e2e/negative_test.go b/test/e2e/negative_test.go index f8e09b9..4f9be0a 100644 --- a/test/e2e/negative_test.go +++ b/test/e2e/negative_test.go @@ -195,7 +195,7 @@ func testSessionSurvivesRestart(t *testing.T, sc *scenario) { } // testLevel1RateLimit is plan C.4 negative check 4: the native Postfix anvil -// backstop (smtpd_client_message_rate_limit, README § Rate limiting), set by +// backstop (smtpd_client_message_rate_limit, guide § Rate limiting), set by // the override to RATE_LIMIT_MESSAGES_PER_IP=50, rejects once exceeded. It // uses a dedicated application with no level-2 limit of its own, and retries // well past that count, so the result is unambiguous regardless of how much of diff --git a/test/e2e/panel_client.go b/test/e2e/panel_client.go index edf6efd..a42ab54 100644 --- a/test/e2e/panel_client.go +++ b/test/e2e/panel_client.go @@ -170,7 +170,7 @@ func (c *panelClient) addApplication(domainID, login, mode, addresses string) (a return appLogin, password, nil } -// setRateLimit saves a level-2 differentiated limit (README § Rate limiting) +// setRateLimit saves a level-2 differentiated limit (guide § Rate limiting) // on either a domain (/domains/{id}/ratelimit) or an application // (/applications/{id}/ratelimit). func (c *panelClient) setRateLimit(path, allowedIP string, maxMessages, windowSeconds int) error {