delivery log: measure the sweep before deciding its fate (audit H3, 0007)

Migration 0007 (DDL only): delivery_log_days, a row per UTC day, kept a
year, shown nowhere - read off a copy of the database after a month by the
decision rule written in the audit plan.

- reconcile_closed: the sweep adds the rows it actually closed.
- reconcile_contradicted: store.UpdateStatus, in one transaction, counts the
  rows with reason "reconcile" a delivery line now speaks of, writes the
  log's status and clears the reason - so a re-read line counts once.
- tailer_blind_seconds: follow() tracks stretches in which the file is gone
  or reopening fails (logtail.blindness); stored when a stretch ends and
  every minute while it lasts; under 3 s not counted (the rotation's tick
  between rename and create loses nothing).
- queued_over_lifetime: every sweep overwrites the day's level with the rows
  queued past LongestLifetime + MaximalBackoff (a message Postfix still
  works on stays queued until its first retry after the lifetime).
- retentionLoop prunes days older than a year.

Tests: store (sums per day, the level overwritten, under a second nothing,
prune; a contradicted sweep counted once and its reason cleared; count of
queued rows); logtail (the sweep counts what it closed; countStuck records
the level; blindness - a one-tick gap not counted, 75 s stored as 60 + 15
from where the last part ended). Schema tests at user_version 7.

schema-migrations.md: chain head 7 (it still said 5 since 2.4.0), row 7.
CHANGELOG [Unreleased], audit plan line ticked, roadmap 10/61.
Evidence: go vet ./... clean; go test ./internal/store ok, ./internal/logtail
ok but for TestFollowTailsAndRotates (Windows); the server run of the stage
follows before the push.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
mix
2026-10-14 14:20:44 +03:00
co-authored by Claude Opus 5.5
parent 9adb44845c
commit ffefa9d6df
13 changed files with 459 additions and 14 deletions
+5 -3
View File
@@ -17,8 +17,8 @@ When you add or review a migration, update this file in the same change.
| Field | Value |
|---|---|
| Chain head | `user_version = 5` |
| Files | `0001_init.sql` (the 2.0 baseline), `0002_inbound_spam_log.sql`, `0003_send_log_lines.sql`, `0004_notifications.sql`, `0005_send_log_rejected_index.sql`, `0006_sender_check.sql` |
| Chain head | `user_version = 7` |
| Files | `0001_init.sql` (the 2.0 baseline), `0002_inbound_spam_log.sql`, `0003_send_log_lines.sql`, `0004_notifications.sql`, `0005_send_log_rejected_index.sql`, `0006_sender_check.sql`, `0007_delivery_log_days.sql` |
| File mark | `PRAGMA application_id = 0x53503230` ("SP20") |
| Database file | `/data/selfpost.db` (bind mount) |
| Compatibility | 2.x MINOR releases must boot a `2.0.0` data directory. **Nothing is carried over from 1.x.** |
@@ -65,7 +65,8 @@ features after them add files, the first being `0002_inbound_spam_log.sql` of
inbound-antispam-panel (`2.1.0`), then `0003_send_log_lines.sql` of
delivery-log-storage (`2.2.0`), `0004_notifications.sql` and
`0005_send_log_rejected_index.sql` of panel-notifications (`2.3.0`), and
`0006_sender_check.sql` of from-header-binding (`2.4.0`).
`0006_sender_check.sql` of from-header-binding (`2.4.0`), and
`0007_delivery_log_days.sql` of the 2026-10-12 audit (`2.5.0`).
---
@@ -79,6 +80,7 @@ delivery-log-storage (`2.2.0`), `0004_notifications.sql` and
| 4 | `0004_notifications.sql` | 2.3.0 | DDL | `users.notify` (default 1: every user starts with notifications on) and `notify_state` |
| 5 | `0005_send_log_rejected_index.sql` | 2.3.0 | DDL | `idx_send_log_rejected`, a partial index of the refused rows of `send_log` by time; a file of its own because the test installation had run 0004 before the review asked for it |
| 6 | `0006_sender_check.sql` | 2.4.0 | DDL | `send_log.reason` and `send_log.header_from` (both default `''`: a row refused earlier keeps no reason), and `sender_check_days` |
| 7 | `0007_delivery_log_days.sql` | 2.5.0 | DDL | `delivery_log_days`: what the reconcile sweep and the log tailer did, a row per UTC day, kept a year, shown nowhere - the measurement that decides the sweep's fate (audit 2026-10-12, stage 1). `send_log.reason` gains the value `reconcile` without a schema change |
**Kind:** *DDL* — schema only; *data* — `INSERT`/`UPDATE` that must stay correct
for operators upgrading from older 2.x images.