Files
selfpost/docs/plans/send-log-retention.md
T
mix 0730b2090e release: 1.3.0: 1.3.0-1.3.1: security pass, queue persist, full backup layout
[1.3.0]
Security and quality after 1.2.5: domain-admin send-log authorization, fail-closed sign-in and application delete, level-2 rate-limit race fix, restore Resync, expanded tests, operator docs, release CI, and OFL for IBM Plex. Upgrading from 1.2.x is a tag bump; no migration.
Added:
  - licence: the SIL Open Font License 1.1 text now travels with the IBM Plex
  - docs: agreed roadmap item **queue-retries** — show this Postfix's retry
Security:
  - The independent security review of the send-log authorization and
Fixed:
  - test (e2e): send-log status scrapers follow the badge markup in
  - security (panel): the Deliveries list is scoped to a domain administrator's
  - mail (level-2 rate limit): the ceiling is no longer overshot by messages that
  - panel (sign-in): a session that cannot be written to the database no longer
  - panel (applications): deleting an application removes its SASL credentials
  … (2 more in CHANGELOG)
Changed:
  - docs: operator and as-built docs aligned with the code after a full
  - docs: [guide.md](docs/guide.md) reorganised into **Installation**, **Instance
  - docs: the 2026-08-13 full-tree review plan is complete — every phase (P0–P7)
  - licence: [NOTICE](NOTICE) tells modifiers to update `SourceURL` in
  - ci: the release image is published only for a **published** GitHub Release
  … (9 more in CHANGELOG)
[1.3.1]
Retry policy in the panel, persistent Postfix queue, and self-contained full backups. Mail queue and delivery history show this Postfix's first retry, backoff cap, and queue lifetime. Deferred mail survives container recreate. Full backups now archive `data/`, compose, `.env`, and `certs/`; the archive layout breaks 1.3.0 flat backups (unpack those with `tar xzf backup.tar.gz -C ./data` as before). Upgrading from 1.3.0 is a tag bump; no schema migration.
Added:
  - Mail queue and a delivery's history show this Postfix's retry policy
  - docs: **Plan checklists** in [development.md](docs/development.md) — format,
  - docs: **Implementation checklists** with model routing in
  - docs: roadmap candidates **send-log-retention** (panel Settings for delivery
  - Postfix queue under `/data/postfix/queue` — deferred and active mail survive
  … (1 more in CHANGELOG)
Changed:
  - **Breaking:** full-backup archive layout — paths are prefixed with `data/`;
  - docs: operator guide, architecture, security, and backup UI updated for
  - docs: HTML mockups for a full panel UI refresh
Changed:
  - docs: panel UI mockups after review — Status restored to live readings
  - docs: panel UI design system and mockups rebuilt as separate pages
  - docs: panel UI mockups — add-domain sits in the list card (Domains and
Fixed:
  - ci (GHCR): per-arch package tags (`X.Y.Z-amd64`, `X.Y.Z-arm64`) are dropped

Squashed CHANGELOG sections: [1.3.0], [1.3.1].

Full detail: CHANGELOG.md (1.3.0, 1.3.1).
2026-09-22 15:24:52 +03:00

5.4 KiB
Raw Blame History

Plan: send-log-retention

Status: candidate
Date: 2026-08-17
Version: 1.x MINOR; no schema migration required (uses existing settings table).


Goal

Let the global administrator change how long delivery journal rows (send_log, /deliveries) are kept, from the panel — without editing .env.

Context (as-built)

Retention already exists, but only via environment:

This plan moves the effective retention into SQLite settings, with env as bootstrap only.

Scope

In:

  • Settings card on /settings (global administrator only): Send log retention (days).
  • Key send_log_retention_days in settings.
  • Validation: integer range 7–365 (exact bounds fixed at implementation).
  • On first use: if setting missing, seed from env (SEND_LOG_RETENTION_DAYS, default 90) at panel start or first save.
  • Log-tailer reads the setting each prune cycle (no container restart).
  • Delivery pages and guide copy show the current retention, not a hardcoded 90.
  • Tests; guide.md; CHANGELOG.md.

Out:

  • Per-domain retention (instance-wide only).
  • mail.log rotation (logrotate, 14 daily files — unchanged).
  • Immediate prune on save when lowering retention (next 6 h cycle is enough; optional «Prune now» not in v1).
  • Domain-admin access to this setting.

Architecture

flowchart LR
  settingsPage["/settings form"] --> sqlite["settings.send_log_retention_days"]
  env["SEND_LOG_RETENTION_DAYS bootstrap"] --> sqlite
  sqlite --> retentionLoop["logtail retentionLoop"]
  retentionLoop --> prune["DeleteSendLogBefore"]
  1. Read path — GetSendLogRetentionDays(): settings value if valid, else env default.
  2. Write path — POST /settings (global admin): validate, SetSetting, flash confirmation.
  3. Prune path — change logtail.retentionLoop to accept func() int or RetentionReader that queries settings each cycle (same 6 h ticker).
  4. Copy — inject retention days into delivery list/detail templates and remove hardcoded «ninety days».

SEND_LOG_RETENTION_DAYS remains documented in guide.md as the initial default until changed in Settings.

Relation to domain-stats-auto-ratelimit

domain-stats-auto-ratelimit uses a 30-day stats window. Requires effective retention ≥ 30 for full accuracy. When retention < 30:

  • Stats UI shows a warning and uses min(30, retention) as the window, or
  • Settings validation warns when saving a value below 30 while stats/auto are enabled (pick one at implementation; document in guide).

Recommended roadmap order: send-log-retention before or parallel with domain-stats-auto-ratelimit.

Panel UI

New card on settings.html (global admin block, near rate limits or under a «Deliveries» heading):

  • Number input: retention days (7–365).
  • Muted copy: rows older than this are deleted from /deliveries; main driver of /data growth; does not affect mail.log rotation.

Domain administrators keep the narrow credentials-only settings page.

Tests

  • Save/load setting; reject out-of-range values.
  • retentionLoop uses updated value without process restart (mock reader).
  • Bootstrap: empty settings → env default used for prune.
  • Template/delivery copy reflects configured days.

go test / go vet on touched packages.

Done when

  • Global admin can set retention on /settings; value persists in SQLite.
  • Prune uses the panel value on the next cycle; env remains bootstrap default.
  • Guide documents panel vs env; CHANGELOG entry added.
  • Hardcoded «ninety days» removed from delivery UI.

Risks

  • Operator lowers retention while bookmarking old delivery URLs — existing behaviour; copy already notes pruned rows are gone.
  • Settings change without restart — must be tested so log-tailer never keeps a stale int from panel start only.

Version: 1.x MINOR.

Implementation checklist

Target version cut: 1.5.0 (MINOR). One commit per step; code only after roadmap status is agreed. See development.md § Plan checklists.

  • GetSendLogRetentionDays / SetSetting key send_log_retention_days (7–365) — Opus
  • Bootstrap from SEND_LOG_RETENTION_DAYS when settings empty — Opus
  • logtail.retentionLoop: read setting each prune cycle — Opus
  • Settings card on /settings (settings.html) — Sonnet
  • Remove hardcoded «ninety days» in handlers and templates — Sonnet
  • Tests: save/load, range, loop without restart — Sonnet
  • guide.md — Sonnet
  • go vet, go test on touched packages — Haiku