mixandClaude Opus 5.5
e166a2ebd6
The Delivery log box of every refused row said "This message never reached
the queue, so Postfix wrote no delivery lines for it" - untrue: Postfix
wrote a line for each. The owner, 2026-10-13: the box is required on every
message page. Pictured for him first (dev/shots, the refused variant of
out-message: wrong From with its queue id lines, rate-limited with the
NOQUEUE line); the variant has no mockup screen of its own - the guard ties
screens to pages, and its outline is out-message's.
- Wrong From: cleanup gives the message a queue id before the headers are
judged; the sender check reads {i} at end-of-headers and writes it on the
refused row. Its lines - stored under the id like any other's - are on the
page, but only those from 10 minutes before to 1 minute after the refusal
(store.SendLogLines for a rejected row): Postfix frees the id at once, and
a later message with it lends nothing. No mail.log fallback for a refused
row, for the same reason.
- Rate-limited and wrong client: refused at MAIL FROM, no queue id. Migration
0009 adds send_log.refusal_line; logtail.refusalLines matches smtpd's
"NOQUEUE: milter-reject: MAIL from ...; from=<...>" (not port 25's, not
CONNECT/EHLO) and store.AttachRefusalLine puts it on the newest refused row
of that sender (and login, when the line names it) within two minutes; a
line whose row is not written yet waits up to 30 s (retried at each pass).
- The page: the queue id fact says "given, the message was not queued" or
"none - refused at MAIL FROM, before one is given"; the box's caption is
"The line from mail.log for this refusal" for the latter; a row from
before says the line was not kept. LogLineLevel colours a refusal by its
reply: 5xx red, 4xx amber (IPv6 clients included). The id in the fact is
not muted as in the picture: the kit has no muted inline.
Tests: store (attach by sender, login, time; not a queued row, not one an
hour off; a refused row's lines are its own minutes); logtail (attach, wait
and retry, port 25 and other lines ignored); milter (the refused row carries
{i}); view (facts, caption, line levels); handlers (the line on the page,
the sentence without it). e2e refusals_show_postfixs_line: a client-address
refusal made there, and the pages of a wrong From, a rate-limited and a
wrong client refusal each show Postfix's line.
schema-migrations row 9, CHANGELOG [Unreleased], audit plan line ticked,
roadmap 25/62.
Evidence on the test server for this tree: make e2e exit 0 (the new case
in 5.4 s); go test ./... ok but for two schema tests, fixed after the run
(the schema at 0009) - store tests ok locally; the full run follows before
the push.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
10 KiB
10 KiB
SQLite schema and migrations
What this file is. A living reference for the embedded SQLite schema: the 2.0 baseline, the migrations added after it, and how a database is opened. When you add or review a migration, update this file in the same change.
Source of truth in code: internal/store/migrations/*.sql, applied by
internal/store/store.go (migrate()).
Related: architecture.md § Persistence; plans/panel-redesign.md § No compatibility; development.md § Documentation.
Current snapshot
| Field | Value |
|---|---|
| Chain head | user_version = 9 |
| Files | 0001_init.sql (the 2.0 baseline), 0002_inbound_spam_log.sql, 0003_send_log_lines.sql, 0004_notifications.sql, 0005_send_log_rejected_index.sql, 0006_sender_check.sql, 0007_delivery_log_days.sql, 0008_tenant_boundary.sql, 0009_refusal_line.sql |
| File mark | PRAGMA application_id = 0x53503230 ("SP20") |
| Database file | /data/selfpost.db (bind mount) |
| Compatibility | 2.x MINOR releases must boot a 2.0.0 data directory. Nothing is carried over from 1.x. |
2.0 starts from an empty data directory
Decided by the owner on 2026-09-21: the redesign is a breaking release, the
1.x migration chain (0001 … 0010, with its admin table created in the
first file and dropped in the fifth) is deleted from the binary, and there is
no upgrade path. The only live 1.x installation is the owner's and is set up
again on 2.0. The 1.x chain stays in git history (last present at 1.9.5).
What that means in practice:
- A fresh
/datagets the baseline and nothing else. - A 1.x database is refused, not migrated.
store.OpenreturnsErrForeignSchemafor a file that has a schema version but not the 2.xapplication_id. Without that check a 1.x file atuser_version = 10would read as "already migrated" and the panel would start on tables it does not know. The file is not modified. The same applies to restoring a 1.x full backup into a 2.0 container. - A database whose version is higher than the build knows is refused too.
- Domains can still be carried by hand: a 1.x domain export (JSON, with the DKIM key and applications) imports into 2.0 — the transfer format is not the database.
How migrations run
- Migrations are embedded at build time (
//go:embed migrations/*.sql). - Filenames are sorted lexicographically; file order = version number.
PRAGMA user_versionrecords progress: after file i (0-based), version is i + 1.- Each pending migration runs in its own transaction, then bumps
user_version. - There is no down-migration; fixes ship as a new
00NN_*.sqlfile.
Implication: once 2.0.0 is released, 0001_init.sql is never edited
again and no file is deleted, renamed or reordered — every 2.x database has
already run it. 2.0.0 and 2.0.1 shipped with the baseline alone; the
features after them add files, the first being 0002_inbound_spam_log.sql of
inbound-antispam-panel (2.1.0), then 0003_send_log_lines.sql of
delivery-log-storage (2.2.0), 0004_notifications.sql and
0005_send_log_rejected_index.sql of panel-notifications (2.3.0), and
0006_sender_check.sql of from-header-binding (2.4.0), and
0007_delivery_log_days.sql and 0008_tenant_boundary.sql of the 2026-10-12
audit (2.5.0).
Migration chain
| Ver | File | Shipped | Kind | Summary |
|---|---|---|---|---|
| 1 | 0001_init.sql |
2.0.0 | DDL | The whole 2.0 schema, below |
| 2 | 0002_inbound_spam_log.sql |
2.1.0 | DDL | inbound_spam_log, inbound_spam_list and two indexes; nothing existing changes |
| 3 | 0003_send_log_lines.sql |
2.2.0 | DDL | send_log_lines and one index; nothing existing changes |
| 4 | 0004_notifications.sql |
2.3.0 | DDL | users.notify (default 1: every user starts with notifications on) and notify_state |
| 5 | 0005_send_log_rejected_index.sql |
2.3.0 | DDL | idx_send_log_rejected, a partial index of the refused rows of send_log by time; a file of its own because the test installation had run 0004 before the review asked for it |
| 6 | 0006_sender_check.sql |
2.4.0 | DDL | send_log.reason and send_log.header_from (both default '': a row refused earlier keeps no reason), and sender_check_days |
| 7 | 0007_delivery_log_days.sql |
2.5.0 | DDL | delivery_log_days: what the reconcile sweep and the log tailer did, a row per UTC day, kept a year, shown nowhere - the measurement that decides the sweep's fate (audit 2026-10-12, stage 1). send_log.reason gains the values reconcile and reconcile-contradicted without a schema change |
| 8 | 0008_tenant_boundary.sql |
2.5.0 | DDL + data | idx_applications_login_nocase: an application login is unique without regard to case (audit 2026-10-12, H4). send_log.domain_id and inbound_spam_log.inbound_domain_id (ON DELETE SET NULL), each with an index, filled once by name from the registry as it was then: a row belongs to a domain by id, and a domain deleted and added again starts with an empty log (L12) |
| 9 | 0009_refusal_line.sql |
2.5.1 | DDL | send_log.refusal_line: Postfix's line about a refusal at MAIL FROM (a level-2 limit, a client address off the list), which has no queue id; the log tailer puts it on the row (audit 2026-10-12, stage 2b) |
Kind: DDL — schema only; data — INSERT/UPDATE that must stay correct
for operators upgrading from older 2.x images.
Schema at head (v2)
| Table | Role |
|---|---|
settings |
Key/value settings of the instance — what is true whoever is signed in (send-log retention). Nothing that belongs to a user |
users |
Panel logins. role is global or domain; email is the user's own address; notify is the user's own switch for notification e-mails (v4); all_domains / all_inbound_domains widen a domain user to every domain of that direction |
sessions |
Panel login sessions (token hash, owning user by id and deleted with them, sliding idle expiry) |
domains |
Sending (outbound) domains. dmarc_rua is where aggregate reports go ('' = no reports) |
user_domains |
Outbound domains assigned to a domain user |
applications |
SASL applications per domain, with the client-IP restriction (auth_ip_restrict, auth_allowed_ips) |
application_addresses |
Explicit From addresses (list mode) |
rate_limits |
Level-2 limits per domain / application, manual or auto |
send_log |
Delivery journal |
logtail_state |
Log-tailer read position |
inbound_domains |
Inbound relay domains |
inbound_transports |
Upstream host / port / TLS per inbound domain |
inbound_recipients |
Allow-list when recipient_mode = list |
user_inbound_domains |
Inbound domains assigned to a domain user — granted separately from the outbound ones |
dmarc_reports |
Parsed aggregate report summaries |
dmarc_report_records |
Per-source rows inside a report |
inbound_spam_log |
Inbound antispam journal: one row per recipient of a message offered on port 25, with the decision (accept, reject, tempfail, quarantine) and who made it (rspamd, postfix, selfpost). Metadata only (v2) |
inbound_spam_list |
Allow / deny entries for the external filter (address, domain, ip), instance-wide; unique per list (v2) |
notify_state |
What the panel has already written to its users about, one row per subject (dns:<domain>, cert, queue, limit:<domain>), so that each thing is said once and a restart says nothing again (v4) |
send_log_lines |
The mail.log lines Postfix wrote under a queue id, with the line's own time; what a message's page shows once the log has rotated. No reference to send_log: a line belongs to a row by queue id and time, and goes when no row carries its id (v3) |
TestBaselineSchema (internal/store/schema_test.go) pins every table, column
and index of this list; TestOpenMigratesA20Database opens a database that
holds the baseline alone.
What changed against the last 1.x schema
| 1.x | 2.0 | Why |
|---|---|---|
admin created in 0001, dropped in 0005 |
never exists | the reason for the squash |
users.role = global / domain_admin |
global / domain |
the role is a reach, not a rank — everyone in the panel is an administrator |
users.dmarc_report_email |
users.email |
a user's e-mail is theirs; it is what the panel writes to, and a domain's report address can be filled with it |
instance setting dmarc_report_email (mirror of the global user's field); domains.dmarc_rua NULL = inherit the instance default |
domains.dmarc_rua alone: the address, '' = no reports |
a domain's report address is one of three explicit choices — no reports, hosted, a typed address; nothing is inherited and no user's default exists (2026-10-10) |
| — | user_inbound_domains |
inbound is delegated per user, separately from outbound |
| — | users.all_domains, users.all_inbound_domains |
"All" includes domains added later |
rate_limits.allowed_ips (unused since 1.9.0) |
gone | client IPs belong to the application (auth_allowed_ips) |
Where a domain's DMARC reports go
domains.dmarc_rua is the address itself and the only thing stored; '' means
no reports, and it is what a new domain has. The choice shown in Domain
settings is derived from it: '' is no reports, the address SelfPost derives
for the domain and the server's hostname (dmarc.HostedReportAddress, offered
while hosted reports are enabled) is hosted, anything else is an address typed
for the domain. Nothing refers to a user: deleting one changes no domain.
Adding a migration
- Add
internal/store/migrations/00NN_short_name.sql(next number). - Update the table in Migration chain, the snapshot at the top, and the affected rows of Schema at head.
- Update
TestBaselineSchemaif tables, columns or indexes change, and add a test that opens a database at the previous version and migrates it. - A data migration states in a comment which 2.x versions it is written for.