audit 4: cards ordered by number, a notification owed to each recipient
test / test (push) Canceled after 0s

The cards layout (owner, 2026-10-13; the mockup first, b90a2f0): box_open
writes data-order for a box with InCards and a number, and panel.css orders
by it up to twelve - Health's eight and the kit's three.
TestCardsBoxesCarryTheirNumberAsTheirOrder reads the rendered pages.

Notifications (owner, 2026-10-14): migration 0011, notify_addressees. open
sends a subject's first message to every recipient and records who was
handed it; followUp, on every check while the subject is open and still
wrong, hands it to the owed and untold who are still recipients; closed
sends the end to the told (all recipients for a subject from before the
table). Tests: a failed copy goes out at the next check, once; the hourly
ceiling holds one back; a user added mid-subject hears nothing of it.

Also in the roadmap, from the owner today: dns-record-collapse,
notification-mail-recheck, settings-save-row (agreed, design first).

Tested on the test server: go vet + go test of 25 packages, make e2e (all
pass, 644 s) on this tree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
mix
2026-10-15 11:49:50 +03:00
co-authored by Claude Opus 5.5
parent b90a2f0321
commit eee089ad4b
17 changed files with 538 additions and 74 deletions
+8
View File
@@ -5,6 +5,8 @@ Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); version
## [Unreleased]
Migration `0011` (a table, nothing existing changes).
### Added
- **The Outbound log filters by status and by day**, beside domain and
@@ -65,6 +67,12 @@ Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); version
logins, and the page said "internal error". Every rebuild now runs, and
the page says the domain is deleted and that Reload configuration
finishes the rest (audit 2026-10-12, L17).
- **A notification whose copy failed for one user was never sent to them**
once another user's copy had gone out, and a copy held back by the hourly
ceiling was dropped. A notification is now owed to each recipient: the
next check sends it to those who did not get it, once; a user added while
the problem is open is not mailed about it; "it is over" goes to those
who were told.
## [2.5.3] - 2026-10-15
+9 -2
View File
@@ -393,8 +393,15 @@ is sent; the operator's view is guide § Notifications.
deleted domain's row goes with it.
- **Said means sent.** A problem that arises while no user can be told, or
while Postfix does not take the message, stays unsaid and is told at a
later check. Its ending, with nobody left to tell, is simply over. If one
recipient's copy fails and another's goes out, the subject is said.
later check. Its ending, with nobody left to tell, is simply over.
- **Owed to each recipient** (since `2.5.4`, `notify_addressees`): the
subject's first message records who it was addressed to and who was
handed a copy; while the subject is open, every later check hands it to
those not told yet - a copy that failed, or that the ceiling held back -
who are still recipients. A user who comes to hear of a subject only while
it is open is not mailed about it, and its ending goes to those who were
told. A subject opened before `2.5.4` has no such record, and its ending
goes to every recipient, as it did.
- **The notifier's own mail is not a subject.** A notification waiting for a
mailbox that does not answer, or its bounce coming back to
`postmaster@<hostname>` - which receives no mail - is left out of the
+21 -2
View File
@@ -1283,7 +1283,7 @@ command whose leftover is the owner's alone.
`mi_rd_cmd`, go-milter v0.4.1 `HandleMilterCommands` on `io.EOF` - the
comment on CheckSocket cites them). The socket nobody listens on is
`milter_test.go`.
- [ ] **The `cards` layout takes its phone order from the box's number**
- [x] **The `cards` layout takes its phone order from the box's number**
(owner, 2026-10-13, from the Not done list of from-header-binding). The
order on a phone is written in `panel.css` for four boxes to a stack
(`.sp-cards > .column:first-child > .box:nth-child(1..4)`); a fifth box
@@ -1293,7 +1293,15 @@ command whose leftover is the owner's alone.
checks that the numbers of a rendered `cards` page run in order. A change
to a component: the mockup's components page and `theme.css` first
(`design:` commit). - **Sonnet**
- [ ] **A notification is owed to each recipient, not to the subject**
*Done 2026-10-15:* an attribute, not a class: the mockup's check
(`check.py`) refuses a class `theme.css` defines and no page uses, and a
class per number up to twelve would be eight such. `data-order="N"` is
not a style either, so the CSP holds; `theme.css` and `panel.css` have
`.sp-cards [data-order="1..12"]`. `box_open` writes it for a box with
`InCards` (Health's eight, the kit's three) and a number.
`TestCardsBoxesCarryTheirNumberAsTheirOrder` reads the rendered pages:
each box's data-order is its number, and the stylesheet has its rule.
- [x] **A notification is owed to each recipient, not to the subject**
(owner, 2026-10-14; from the reviews of panel-notifications, not found by
the audit). `Notifier.send` ([notify.go](../internal/notify/notify.go))
counts a subject as said when one recipient's copy was handed to Postfix;
@@ -1307,6 +1315,17 @@ command whose leftover is the owner's alone.
open is not mailed about it. "Cleared" goes to those who were told of the
problem. Tests: one copy fails and goes out at the next check, once; the
ceiling; a user added mid-subject. - **Opus**
*Done 2026-10-15:* migration `0011`, `notify_addressees (key, address,
told_at)`, gone with the subject's row. `open` sends the first message
to every recipient and records who was handed it; `followUp`, on every
check while the subject is open and still wrong (DNS bad, the
certificate at its level, the queue still stuck, a limit within its
day), hands the current message to the owed and untold who are still
recipients; `closed` sends the end to the told (all recipients for a
subject opened before the table). A cleared copy that fails is not
retried - the subject is gone with it, as before. Tests: a failed copy
goes out at the next check, once; the ceiling holds one back; a user
added mid-subject hears nothing of it, its end included.
- [ ] **A DNS outage is "unknown", not "wrong"** (M14, M15; owner,
2026-10-14 - the two findings had no step). Each resolver gets a time
budget of its own, so a silent first resolver no longer leaves the second
+1 -1
View File
@@ -115,5 +115,5 @@ to the owner before any code ([panel-redesign.md](panel-redesign.md)
- [x] Panel: Account, Settings, Health card, help
- [x] guide.md, architecture.md, security.md, schema-migrations.md
- [x] `go vet`, `go test`, e2e: a level-2 refusal reaches a mailbox as a signed notification (on the test server: vet ok, 23 packages ok, e2e ok; "once" and "clears" are held by the unit tests of `internal/notify`)
- [x] Technical review — **Opus** (2026-10-12; no Critical. Fixed: refusals counted by the registered domain and only under a limit, with an index of their own; account e-mails may be at public mail hosts; "said" means sent; DNS checks must be 45 minutes apart; the notifier's own mail is not a queue subject and a cut or unreadable listing is no listing; the tables must name the selector the panel shows; an unanswered `a`/`mx` lookup is not an SPF mismatch; a pass stops between domains at shutdown; days are rounded as a person counts. Left as found: a recipient whose copy failed while another's went out is not retried; a certificate that lives under 14 days is reported at each renewal)
- [x] Technical review — **Opus** (2026-10-12; no Critical. Fixed: refusals counted by the registered domain and only under a limit, with an index of their own; account e-mails may be at public mail hosts; "said" means sent; DNS checks must be 45 minutes apart; the notifier's own mail is not a queue subject and a cut or unreadable listing is no listing; the tables must name the selector the panel shows; an unanswered `a`/`mx` lookup is not an SPF mismatch; a pass stops between domains at shutdown; days are rounded as a person counts. Left as found: a recipient whose copy failed while another's went out is not retried (since 2.5.4 it is: audit 2026-10-12, stage 4); a certificate that lives under 14 days is reported at each renewal)
- [x] Security review — **Fable** (2026-10-12; no Critical. Fixed: the limit notification was built on the domain a client writes in `MAIL FROM` - a mail cannon and a way to write to another tenant's users; a ceiling of 60 messages an hour; [security.md](../security.md) corrected on who can submit locally and on what the account e-mail now is. Recorded as accepted, with a roadmap candidate: an application can have a message signed as the server by writing its name into `From:` - the header is not held against the login, for sending domains either)
+59 -1
View File
@@ -38,11 +38,14 @@ in `git log` and [CHANGELOG.md](../CHANGELOG.md).
| delivery-log-storage | Keep a message's `mail.log` lines with its send-log row | shipped `2.2.0` | 8/8 | [plans/delivery-log-storage.md](plans/delivery-log-storage.md) |
| password-reset | Password reset by e-mail | agreed (2026-10-13) | — | — |
| local-time | The panel shows time, and counts its days, in a time zone other than UTC | agreed (2026-10-13; design questions open) | — | — |
| notification-mail-recheck | The Notification mail records on Server › Settings can be checked again from where they are shown | agreed (2026-10-15) | — | — |
| settings-save-row | Server › Settings: the Save button no longer stands alone under a long right column | agreed (2026-10-15) | — | — |
| dns-record-collapse | A DNS record that is right is shown folded: its name and its tag, the values on request | agreed (2026-10-15) | — | — |
| phone-desktop-mode | A phone browser in "desktop site" mode gets the phone navigation | agreed (2026-10-13: after stages 1 and 2 of audit-2026-10-12) | — | — |
| from-header-binding | Hold a message's `From:` header against the application that sends it | shipped `2.4.0` | 9/9 | [plans/from-header-binding.md](plans/from-header-binding.md) |
| panel-redesign | Panel redesign (accepted mockups, design contract) | shipped `2.0.0` | 27/27 | [plans/panel-redesign.md](plans/panel-redesign.md) |
| schema-squash | Squash SQLite migrations into a 2.x baseline | built in panel-redesign (stage 1) | — | [plans/panel-redesign.md](plans/panel-redesign.md) |
| audit-2026-10-12 | Follow-ups of the 2026-10-12 full-tree code audit, seven stages | in progress (stage 0 shipped `2.4.2`, stages 1+2 `2.5.0`, 2b `2.5.1`, 2c `2.5.2`, 3 `2.5.3`; next stage 4 → `2.5.4`) | 62/81 | [audit-2026-10-12.md](audit-2026-10-12.md) |
| audit-2026-10-12 | Follow-ups of the 2026-10-12 full-tree code audit, seven stages | in progress (stage 0 shipped `2.4.2`, stages 1+2 `2.5.0`, 2b `2.5.1`, 2c `2.5.2`, 3 `2.5.3`; next stage 4 → `2.5.4`) | 64/81 | [audit-2026-10-12.md](audit-2026-10-12.md) |
**Recommended order** (not binding; owner, 2026-10-09: feature order follows
development convenience): **panel-redesign** shipped as `2.0.0` (2026-10-11) and
@@ -697,3 +700,58 @@ joins the index only when the owner says so.
[product.md](product.md). Notes and open questions:
[plans/inbound-quarantine.md](plans/inbound-quarantine.md); the screens
`in-quarantine` and `in-quarantine-message` are drawn and stay unbuilt.
## dns-record-collapse
**Problem (owner, 2026-10-15):** a DNS record block - the DKIM, SPF and
DMARC records on an outbound domain's page, the server's records on
Server › Settings, the MX of an inbound domain - shows the host, the type
and the value to publish whether or not the record is already right. When
DNS holds the record as it should be, the values are of no use on the page:
they only push the records that need work further down.
**Goal:** a record whose check is *ok* is drawn folded - its name and its
tag, one line - and opens on request to show the host, type and value (to
copy them again, or to compare). A record that is missing, wrong or not
checked yet stays open, as it is now, with *In DNS now* where it differs.
**Notes:** the block is the one `dns_record` partial (components.html), so
the change is made once; it is a design change, mockup first (the kit's
dns_record section and the four pages that draw records), wording
included. No JavaScript needed for the folding itself: `<details>` opens
and closes without it under the panel's CSP. Whether a fold remembers being
opened across the polls of a page, and whether *Re-check DNS* folds a record
that has just turned right, are for the mockup.
## notification-mail-recheck
**Problem (owner, 2026-10-15):** Server › Settings shows the DNS records of
the server's own name in its *Notification mail* box (DKIM and SPF, with
their tags), and nothing on that page says how to check them again after
publishing. The way exists elsewhere: *Re-check DNS* on Server › Health
re-runs the server's checks, these records' included
(`HandleHealthRecheck` → `serverMail(true)`), and Health has a Notification
mail card of its own - but from Settings it cannot be found.
**Goal:** the records can be checked again from where they are shown - a
*Re-check DNS* action in the box's head, or at least the box saying where
the check is and how long a result is kept. Design change, mockup first;
decide together with [dns-record-collapse](#dns-record-collapse) (a record
that turns right on the re-check folds).
## settings-save-row
**Problem (owner, 2026-10-15, a screenshot of `selfpost.mixfed.ru`):** on
Server › Settings the *Save settings* button sits under the two columns, at
the left, after a long gap: the left column (retention, sending rate limits)
is short, the right one (the notifications switch, then *Notification mail*
with its records open) is long. The button saves the retention fields and
the switch; the records under the switch are not fields, and they are what
pushes the button down.
**Goal:** the button reads as the save of the fields it saves. Either the
gap goes by itself when [dns-record-collapse](#dns-record-collapse) folds
the records that are right, or the page changes - the records out of the
form's columns, or the button nearer its fields. Decide with the mockup,
after dns-record-collapse, on a page whose records are missing as well as
on one whose records are right.
+5 -3
View File
@@ -17,8 +17,8 @@ When you add or review a migration, update this file in the same change.
| Field | Value |
|---|---|
| Chain head | `user_version = 10` |
| Files | `0001_init.sql` (the 2.0 baseline), `0002_inbound_spam_log.sql`, `0003_send_log_lines.sql`, `0004_notifications.sql`, `0005_send_log_rejected_index.sql`, `0006_sender_check.sql`, `0007_delivery_log_days.sql`, `0008_tenant_boundary.sql`, `0009_refusal_line.sql`, `0010_send_log_indexes.sql` |
| Chain head | `user_version = 11` |
| Files | `0001_init.sql` (the 2.0 baseline), `0002_inbound_spam_log.sql`, `0003_send_log_lines.sql`, `0004_notifications.sql`, `0005_send_log_rejected_index.sql`, `0006_sender_check.sql`, `0007_delivery_log_days.sql`, `0008_tenant_boundary.sql`, `0009_refusal_line.sql`, `0010_send_log_indexes.sql`, `0011_notify_addressees.sql` |
| File mark | `PRAGMA application_id = 0x53503230` ("SP20") |
| Database file | `/data/selfpost.db` (bind mount) |
| Compatibility | 2.x MINOR releases must boot a `2.0.0` data directory. **Nothing is carried over from 1.x.** |
@@ -68,7 +68,8 @@ delivery-log-storage (`2.2.0`), `0004_notifications.sql` and
`0006_sender_check.sql` of from-header-binding (`2.4.0`), and
`0007_delivery_log_days.sql` and `0008_tenant_boundary.sql` of the 2026-10-12
audit (`2.5.0`), `0009_refusal_line.sql` (`2.5.1`) and
`0010_send_log_indexes.sql` (`2.5.3`).
`0010_send_log_indexes.sql` (`2.5.3`) and `0011_notify_addressees.sql`
(`2.5.4`).
---
@@ -86,6 +87,7 @@ audit (`2.5.0`), `0009_refusal_line.sql` (`2.5.1`) and
| 8 | `0008_tenant_boundary.sql` | 2.5.0 | DDL + data | `idx_applications_login_nocase`: an application login is unique without regard to case (audit 2026-10-12, H4). `send_log.domain_id` and `inbound_spam_log.inbound_domain_id` (`ON DELETE SET NULL`), each with an index, filled once by name from the registry as it was then: a row belongs to a domain by id, and a domain deleted and added again starts with an empty log (L12) |
| 9 | `0009_refusal_line.sql` | 2.5.1 | DDL | `send_log.refusal_line`: Postfix's line about a refusal at MAIL FROM (a level-2 limit, a client address off the list), which has no queue id; the log tailer puts it on the row; `send_log.recovered`, a refusal row the tailer wrote from `mail.log` because the panel stopped before writing it; `sender_check_days.messages` (messages deferred mid-message, counted apart from the connections turned away) and `sender_check_days.estimated` (a day whose downtime is an upper bound) (audit 2026-10-12, stage 2b) |
| 10 | `0010_send_log_indexes.sql` | 2.5.3 | DDL | Four indexes of `send_log`, each for a question that walked the table or sorted a domain's log (audit 2026-10-12, M2, stage 3): `idx_send_log_open`, partial, the queued and deferred rows by time (the tailer's sweep every 5 min); `idx_send_log_domain_id` and `idx_send_log_app_login`, a page of one domain's or one application's log in id order; `idx_send_log_refusal_line`, partial, whether a refusal line is already on a row. `TestSendLogQueriesUseTheirIndexes` checks the plans of these and of the other by-time and by-scope statements the store runs |
| 11 | `0011_notify_addressees.sql` | 2.5.4 | DDL | `notify_addressees (key, address, told_at)`: who a subject's notification is owed to and who was handed it, so a copy that failed or was held back by the hourly ceiling goes out at a later check and a subject's end goes to those told (audit 2026-10-12, stage 4); deleted with its `notify_state` row |
**Kind:** *DDL* — schema only; *data* — `INSERT`/`UPDATE` that must stay correct
for operators upgrading from older 2.x images.
+156 -28
View File
@@ -73,6 +73,8 @@ type Store interface {
SetNotifyState(st store.NotifyState) error
DeleteNotifyState(key string) error
ListNotifyStateKeys(prefix string) ([]string, error)
NotifyAddressees(key string) (map[string]bool, error)
SetNotifyAddressees(key string, owed map[string]bool, at time.Time) error
LimitRefusalsSince(since time.Time) ([]store.LimitRefusal, error)
ListDomains() ([]store.Domain, error)
}
@@ -175,24 +177,20 @@ const (
failed
)
// send writes one message to every recipient of a subject, each in a message
// of their own - users are not shown each other's addresses.
//
// What the caller makes of the outcome is the rule that keeps a thing from
// being said never: a problem counts as said only when it was sent, so one
// that arose while nobody could be told, or while the mail system was down, is
// told at a later check. A recipient whose copy failed while another's went
// out is not tried again: the subject is said.
func (n *Notifier) send(domainName string, m message) outcome {
to, err := n.st.NotifyRecipients(domainName)
if err != nil {
log.Printf("notify: recipients: %v", err)
return failed
}
if len(to) == 0 {
return nobody
}
handed := 0
// A message is owed to each recipient, not to the subject (owner,
// 2026-10-14). The first message of a subject goes to everyone who is to hear
// of it then (open), and who was handed a copy is recorded; while the subject
// is open, each later check hands it to those owed it who have not had it -
// a copy that failed, or that the hourly ceiling held back - and who are
// still to hear of it (followUp); a user who comes to hear of the subject
// only while it is open is not mailed about it. That it is over goes to
// those who were told (closed).
// deliver hands m to each address, each in a message of its own - users are
// not shown each other's addresses - under the hourly ceiling, and returns
// those that were handed a copy.
func (n *Notifier) deliver(to []string, m message) []string {
var handed []string
for _, addr := range to {
if !n.underCeiling() {
log.Printf("notify: more than %d messages in an hour; %q to %s is held back", hourlyCeiling, m.subject, addr)
@@ -203,15 +201,137 @@ func (n *Notifier) send(domainName string, m message) outcome {
continue
}
n.sentAt = append(n.sentAt, n.now())
handed++
handed = append(handed, addr)
}
if handed == 0 {
if len(to) > 0 {
log.Printf("notify: %q sent to %d of %d", m.subject, len(handed), len(to))
}
return handed
}
// recipients are who is to hear of a subject of domainName now ("" for the
// server's own subjects).
func (n *Notifier) recipients(domainName string) ([]string, bool) {
to, err := n.st.NotifyRecipients(domainName)
if err != nil {
log.Printf("notify: recipients: %v", err)
return nil, false
}
return to, true
}
// open says m, the first message of the subject key, to everyone who is to
// hear of it, and records who was handed a copy.
//
// What the caller makes of the outcome is the rule that keeps a thing from
// being said never: a problem counts as said only when it was sent, so one
// that arose while nobody could be told, or while the mail system was down, is
// told at a later check.
func (n *Notifier) open(key, domainName string, m message) outcome {
to, ok := n.recipients(domainName)
if !ok {
return failed
}
log.Printf("notify: %q sent to %d of %d", m.subject, handed, len(to))
if len(to) == 0 {
return nobody
}
handed := n.deliver(to, m)
if len(handed) == 0 {
return failed
}
owed := make(map[string]bool, len(to))
for _, addr := range to {
owed[addr] = false
}
for _, addr := range handed {
owed[addr] = true
}
if err := n.st.SetNotifyAddressees(key, owed, n.now()); err != nil {
log.Printf("notify: %v", err)
}
return sent
}
// followUp hands m, the message of an open subject, to those it is owed to
// who have not had it and are still to hear of it.
func (n *Notifier) followUp(key, domainName string, m message) {
owed, err := n.st.NotifyAddressees(key)
if err != nil {
log.Printf("notify: %v", err)
return
}
var due []string
for addr, told := range owed {
if !told {
due = append(due, addr)
}
}
if len(due) == 0 {
return
}
now, ok := n.recipients(domainName)
if !ok {
return
}
due = intersect(due, now)
handed := n.deliver(due, m)
if len(handed) == 0 {
return
}
for _, addr := range handed {
owed[addr] = true
}
if err := n.st.SetNotifyAddressees(key, owed, n.now()); err != nil {
log.Printf("notify: %v", err)
}
}
// closed says m - the subject is over - to those who were told of it and are
// still to hear of it. A subject said before 2.5.4 has no record of who was
// told, and its end goes to everyone who is to hear of it, as it did then.
func (n *Notifier) closed(key, domainName string, m message) outcome {
owed, err := n.st.NotifyAddressees(key)
if err != nil {
log.Printf("notify: %v", err)
return failed
}
to, ok := n.recipients(domainName)
if !ok {
return failed
}
if len(owed) > 0 {
var told []string
for addr, t := range owed {
if t {
told = append(told, addr)
}
}
to = intersect(told, to)
}
if len(to) == 0 {
return nobody
}
if len(n.deliver(to, m)) == 0 {
return failed
}
return sent
}
// intersect is the addresses of a that are in b, in b's order.
func intersect(a, b []string) []string {
in := make(map[string]bool, len(a))
for _, x := range a {
in[x] = true
}
var out []string
for _, x := range b {
if in[x] {
out = append(out, x)
}
}
return out
}
// underCeiling reports whether one more message may go out this hour.
func (n *Notifier) underCeiling() bool {
cutoff := n.now().Add(-time.Hour)
@@ -335,7 +455,7 @@ func (n *Notifier) dnsStep(name string, res dnscheck.Domain) {
case st.Phase == phasePending && bad:
st.Count++
if st.Count >= dnsChecks && now.Sub(st.Since) >= dnsApart {
if n.send(name, dnsProblem(n.hostname, name, faults, st.Since)) == sent {
if n.open(key, name, dnsProblem(n.hostname, name, faults, st.Since)) == sent {
st.Phase, st.Count, st.Mark, st.SentAt = phaseOpen, 0, "", now
}
}
@@ -346,6 +466,7 @@ func (n *Notifier) dnsStep(name string, res dnscheck.Domain) {
st.Count, st.Mark = 0, ""
n.save(st)
}
n.followUp(key, name, dnsProblem(n.hostname, name, faults, st.Since))
case st.Phase == phaseOpen && !bad:
st.Count++
if st.Count == 1 {
@@ -354,7 +475,7 @@ func (n *Notifier) dnsStep(name string, res dnscheck.Domain) {
firstGood, _ := time.Parse(time.RFC3339, st.Mark)
if st.Count >= dnsChecks && now.Sub(firstGood) >= dnsApart {
// With nobody to tell any more, the trouble is over all the same.
if n.send(name, dnsCleared(n.hostname, name, st.Since)) != failed {
if n.closed(key, name, dnsCleared(n.hostname, name, st.Since)) != failed {
n.forget(key)
return
}
@@ -394,10 +515,14 @@ func (n *Notifier) checkCertificate() {
case left <= certFirstDays*24*time.Hour:
level = "first"
}
if level == "" || (have && (st.Phase == level || st.Phase == "last")) {
if level == "" {
return
}
if n.send("", certExpiring(n.hostname, c.NotAfter, left)) != sent {
if have && (st.Phase == level || st.Phase == "last") {
n.followUp(key, "", certExpiring(n.hostname, c.NotAfter, left))
return
}
if n.open(key, "", certExpiring(n.hostname, c.NotAfter, left)) != sent {
return
}
n.save(store.NotifyState{Key: key, Phase: level, Mark: mark, Since: n.now(), SentAt: n.now()})
@@ -446,11 +571,13 @@ func (n *Notifier) checkQueue() {
}
switch {
case len(old) > 0 && !have:
if n.send("", queueStuck(n.hostname, old, now)) == sent {
if n.open(key, "", queueStuck(n.hostname, old, now)) == sent {
n.save(store.NotifyState{Key: key, Phase: phaseOpen, Since: now, SentAt: now})
}
case len(old) > 0 && have:
n.followUp(key, "", queueStuck(n.hostname, old, now))
case len(old) == 0 && have:
if n.send("", queueCleared(n.hostname, st.Since)) != failed {
if n.closed(key, "", queueCleared(n.hostname, st.Since)) != failed {
n.forget(key)
}
}
@@ -482,9 +609,10 @@ func (n *Notifier) CheckLimits() {
// told about, or more of the same. Told longer ago: everything in the
// window is new since then.
if have && now.Sub(st.SentAt) < limitRepeat {
n.followUp(key, r.Domain, limitRefused(n.hostname, r))
continue
}
if n.send(r.Domain, limitRefused(n.hostname, r)) == sent {
if n.open(key, r.Domain, limitRefused(n.hostname, r)) == sent {
n.save(store.NotifyState{Key: key, Phase: phaseOpen, Since: r.First, SentAt: now})
}
}
+78 -7
View File
@@ -2,6 +2,7 @@ package notify
import (
"errors"
"fmt"
"path/filepath"
"strings"
"testing"
@@ -15,13 +16,14 @@ import (
// outbox collects what would have been handed to Postfix.
type outbox struct {
sent []string // "to|subject"
body []string
fail bool
sent []string // "to|subject"
body []string
fail bool
failTo map[string]bool // addresses whose copy fails
}
func (o *outbox) Send(to, subject, body string) error {
if o.fail {
if o.fail || o.failTo[to] {
return errors.New("sendmail: queue file write error")
}
o.sent = append(o.sent, to+"|"+subject)
@@ -474,16 +476,16 @@ func TestQueueLeavesOutTheNotifiersOwnMail(t *testing.T) {
func TestHourlyCeiling(t *testing.T) {
b := newBench(t)
for i := 0; i < hourlyCeiling+5; i++ {
b.n.send("", message{subject: "s", body: "b"})
b.n.open(fmt.Sprintf("test:%d", i), "", message{subject: "s", body: "b"})
}
if got := len(b.out.take()); got != hourlyCeiling {
t.Errorf("%d messages left in one hour, ceiling is %d", got, hourlyCeiling)
}
if b.n.send("", message{subject: "s", body: "b"}) != failed {
if b.n.open("test:x", "", message{subject: "s", body: "b"}) != failed {
t.Error("a message over the ceiling was counted as said")
}
b.now = b.now.Add(61 * time.Minute)
if b.n.send("", message{subject: "s", body: "b"}) != sent {
if b.n.open("test:y", "", message{subject: "s", body: "b"}) != sent {
t.Error("the ceiling did not lift after an hour")
}
}
@@ -548,3 +550,72 @@ func TestMailerComposesAndRefuses(t *testing.T) {
}
}
}
// A notification is owed to each recipient (owner, 2026-10-14): a copy that
// failed goes out at the next check, once, and the subject's end goes to
// those who were told.
func TestACopyThatFailedGoesOutAtTheNextCheck(t *testing.T) {
b := newBench(t)
b.dns["example.com"] = badDKIM()
problem := "[SelfPost] example.com: DKIM does not match"
b.out.failTo = map[string]bool{"mine@example.org": true}
want(t, "first bad check", b.hour())
want(t, "said, one copy failing", b.hour(), "root@example.org|"+problem)
b.out.failTo = nil
want(t, "the next check", b.hour(), "mine@example.org|"+problem)
want(t, "and not again", b.hour())
b.dns["example.com"] = good()
want(t, "first good check", b.hour())
cleared := "[SelfPost] example.com: DNS records match again"
want(t, "over", b.hour(), "mine@example.org|"+cleared, "root@example.org|"+cleared)
}
// The hourly ceiling holds a copy back instead of dropping it: it goes out at
// a check after the hour.
func TestTheCeilingHoldsACopyBack(t *testing.T) {
b := newBench(t)
b.dns["example.com"] = badDKIM()
want(t, "first bad check", b.hour())
// All but one of the hour's messages are spent.
for i := 0; i < hourlyCeiling-1; i++ {
b.n.sentAt = append(b.n.sentAt, b.now)
}
problem := "[SelfPost] example.com: DKIM does not match"
got := b.hour()
if len(got) != 1 {
t.Fatalf("under the ceiling with room for one: sent %v", got)
}
want(t, "the next check, the hour over", b.hour(), map[string]string{
"mine@example.org|" + problem: "root@example.org|" + problem,
"root@example.org|" + problem: "mine@example.org|" + problem,
}[got[0]])
want(t, "and not again", b.hour())
}
// A user who comes to hear of a subject while it is open is not mailed about
// it, nor about its end: the message was owed to those there when it was
// said.
func TestAUserAddedWhileASubjectIsOpenIsNotMailed(t *testing.T) {
b := newBench(t)
b.dns["example.com"] = badDKIM()
want(t, "first bad check", b.hour())
problem := "[SelfPost] example.com: DKIM does not match"
want(t, "said", b.hour(), "mine@example.org|"+problem, "root@example.org|"+problem)
id, err := b.st.CreateUser("late", "hash", store.RoleGlobal, store.Reach{})
if err != nil {
t.Fatal(err)
}
if err := b.st.UpdateUser(id, "late", "hash", "late@example.org"); err != nil {
t.Fatal(err)
}
want(t, "still wrong, a user added", b.hour())
b.dns["example.com"] = good()
want(t, "first good check", b.hour())
cleared := "[SelfPost] example.com: DNS records match again"
want(t, "over", b.hour(), "mine@example.org|"+cleared, "root@example.org|"+cleared)
if owed, _ := b.st.NotifyAddressees("dns:example.com"); len(owed) != 0 {
t.Errorf("who the subject was owed to outlived it: %v", owed)
}
}
@@ -0,0 +1,15 @@
-- Who a notification is owed to, per subject (audit 2026-10-12, stage 4;
-- owner, 2026-10-14). A subject used to count as said once one recipient's
-- copy was handed to Postfix: a recipient whose copy failed, or was held back
-- by the hourly ceiling, was never told. The recipients a subject's first
-- message was addressed to are kept here with whether each was handed it;
-- a later check sends to those not told yet, and "cleared" goes to those who
-- were told. Rows of a subject go with its notify_state row
-- (Store.DeleteNotifyState). A subject opened before this file has none, and
-- is told cleared to every recipient, as before.
CREATE TABLE notify_addressees (
key TEXT NOT NULL,
address TEXT NOT NULL,
told_at TEXT NOT NULL DEFAULT '',
PRIMARY KEY (key, address)
);
+60 -2
View File
@@ -127,11 +127,69 @@ func (s *Store) SetNotifyState(st NotifyState) error {
}
// DeleteNotifyState forgets a subject: nothing is wrong with it any more, or
// it no longer exists.
// it no longer exists. Who it was owed to goes with it.
func (s *Store) DeleteNotifyState(key string) error {
if _, err := s.db.Exec("DELETE FROM notify_state WHERE key = ?", key); err != nil {
tx, err := s.db.Begin()
if err != nil {
return fmt.Errorf("delete notify state %s: %w", key, err)
}
defer tx.Rollback()
for _, stmt := range []string{"DELETE FROM notify_addressees WHERE key = ?", "DELETE FROM notify_state WHERE key = ?"} {
if _, err := tx.Exec(stmt, key); err != nil {
return fmt.Errorf("delete notify state %s: %w", key, err)
}
}
if err := tx.Commit(); err != nil {
return fmt.Errorf("delete notify state %s: %w", key, err)
}
return nil
}
// NotifyAddressees returns who the subject's message is owed to, each with
// whether they were handed a copy (migration 0011). None for a subject not
// said yet, or said before 2.5.4.
func (s *Store) NotifyAddressees(key string) (map[string]bool, error) {
rows, err := s.db.Query("SELECT address, told_at FROM notify_addressees WHERE key = ?", key)
if err != nil {
return nil, fmt.Errorf("notify addressees %s: %w", key, err)
}
defer rows.Close()
out := map[string]bool{}
for rows.Next() {
var addr, toldAt string
if err := rows.Scan(&addr, &toldAt); err != nil {
return nil, fmt.Errorf("notify addressees %s: %w", key, err)
}
out[addr] = toldAt != ""
}
return out, rows.Err()
}
// SetNotifyAddressees records who the subject's message is owed to: owed
// maps an address to whether it was handed a copy, and replaces what was
// recorded. The told are dated at, the time of this record: the date is
// for a person reading the table, the code asks only whether.
func (s *Store) SetNotifyAddressees(key string, owed map[string]bool, at time.Time) error {
tx, err := s.db.Begin()
if err != nil {
return fmt.Errorf("set notify addressees %s: %w", key, err)
}
defer tx.Rollback()
if _, err := tx.Exec("DELETE FROM notify_addressees WHERE key = ?", key); err != nil {
return fmt.Errorf("set notify addressees %s: %w", key, err)
}
for addr, told := range owed {
toldAt := ""
if told {
toldAt = at.UTC().Format(time.RFC3339)
}
if _, err := tx.Exec("INSERT INTO notify_addressees (key, address, told_at) VALUES (?, ?, ?)", key, addr, toldAt); err != nil {
return fmt.Errorf("set notify addressees %s: %w", key, err)
}
}
if err := tx.Commit(); err != nil {
return fmt.Errorf("set notify addressees %s: %w", key, err)
}
return nil
}
+5 -4
View File
@@ -60,6 +60,7 @@ func TestBaselineSchema(t *testing.T) {
"inbound_spam_list": "id list_type entry_type value note created_at",
"send_log_lines": "id queue_id logged_at line",
"notify_state": "key phase count mark since sent_at",
"notify_addressees": "key address told_at",
"sender_check_days": "day down_seconds deferred messages estimated",
"delivery_log_days": "day reconcile_closed reconcile_missed reconcile_contradicted tailer_blind_seconds queued_over_lifetime",
}
@@ -117,8 +118,8 @@ func TestBaselineSchema(t *testing.T) {
}
var version, appID int64
if err := st.db.QueryRow("PRAGMA user_version").Scan(&version); err != nil || version != 10 {
t.Errorf("user_version = %d, %v; want 10", version, err)
if err := st.db.QueryRow("PRAGMA user_version").Scan(&version); err != nil || version != 11 {
t.Errorf("user_version = %d, %v; want 11", version, err)
}
if err := st.db.QueryRow("PRAGMA application_id").Scan(&appID); err != nil || appID != applicationID {
t.Errorf("application_id = %#x, %v; want %#x", appID, err, applicationID)
@@ -194,8 +195,8 @@ func TestOpenMigratesA20Database(t *testing.T) {
}
defer st.Close()
var version int
if err := st.db.QueryRow("PRAGMA user_version").Scan(&version); err != nil || version != 10 {
t.Errorf("user_version = %d, %v; want 10", version, err)
if err := st.db.QueryRow("PRAGMA user_version").Scan(&version); err != nil || version != 11 {
t.Errorf("user_version = %d, %v; want 11", version, err)
}
if ok, err := st.UserExists(); err != nil || !ok {
t.Errorf("the user of the 2.0 database: exists = %v, %v", ok, err)
+71
View File
@@ -0,0 +1,71 @@
package view
import (
"net/http"
"net/http/httptest"
"os"
"regexp"
"strings"
"testing"
)
// On a phone the cards layout shows its boxes in the order of their numbers,
// whichever stack they stand in: each box of the layout carries its number as
// data-order, and the stylesheet has a rule for it. Before, the stylesheet
// ordered by a box's place in its stack and knew four to a stack; a fifth
// landed out of place with nothing failing (owner, 2026-10-13).
func TestCardsBoxesCarryTheirNumberAsTheirOrder(t *testing.T) {
css, err := os.ReadFile("static/panel.css")
if err != nil {
t.Fatal(err)
}
box := regexp.MustCompile(`<div class="box[^"]*"[^>]*?(?: data-order="(\d+)")?>\s*<div class="sp-box-head"><span class="sp-no">(\d+)</span>`)
rec := httptest.NewRecorder()
kitEngine(t).Render(rec, http.StatusOK, "components", KitPage())
for name, page := range map[string]string{
"health": renderSignedIn(t, "health", healthFixture(false)),
"kit": rec.Body.String(),
} {
start := strings.Index(page, `<div class="columns sp-cards">`)
if start < 0 {
t.Fatalf("%s has no cards layout", name)
}
// The layout runs to the first page part after it that is not a box
// of it; its boxes are the numbered ones up to there.
region := page[start:]
if end := strings.Index(region, `<footer`); end > 0 {
region = region[:end]
}
found := 0
for _, m := range box.FindAllStringSubmatch(region, -1) {
order, no := m[1], strings.TrimLeft(m[2], "0")
if name == "kit" && found == 3 {
break // the kit's cards hold three numbered boxes; the rest of the page is other sections
}
found++
if order != no {
t.Errorf("%s: box %s carries data-order %q", name, m[2], order)
continue
}
if !strings.Contains(string(css), `.sp-cards [data-order="`+order+`"] { order: `+order+`; }`) {
t.Errorf("%s: panel.css has no rule for data-order %s", name, order)
}
}
if found == 0 {
t.Errorf("%s: no numbered boxes in the cards layout", name)
}
}
}
// Outside the cards layout a box has no order: the attribute is the layout's.
func TestABoxOutsideCardsHasNoOrder(t *testing.T) {
if got := (Box{No: "03"}).Order(); got != "" {
t.Errorf("a box outside cards has order %q", got)
}
if got := (Box{No: "03", InCards: true}).Order(); got != "3" {
t.Errorf("a cards box 03 has order %q", got)
}
if got := (Box{Icon: "ti-x", InCards: true}).Order(); got != "" {
t.Errorf("a cards box without a number has order %q", got)
}
}
+21
View File
@@ -12,6 +12,7 @@ package view
import (
"fmt"
"html/template"
"strconv"
"strings"
)
@@ -301,6 +302,26 @@ type Box struct {
OOB bool
// DomainPick: see Box.
DomainPick bool
// InCards is a box of the cards layout, which orders its boxes on a
// phone by their numbers (Order).
InCards bool
}
// Order is a cards box's number as the layout orders it on a phone
// (data-order, panel.css: the boxes come in the order of their numbers
// whichever stack they stand in): "3" for "03", empty for a box with no
// number or outside the layout. Before, the stylesheet ordered by a box's place in its stack and
// knew four to a stack; a fifth landed out of place with nothing failing
// (owner, 2026-10-13).
func (b Box) Order() string {
if !b.InCards {
return ""
}
n, err := strconv.Atoi(b.No)
if err != nil || n < 1 {
return ""
}
return strconv.Itoa(n)
}
// HelpLink is a Box's link to its Help topic.
+8 -8
View File
@@ -139,16 +139,16 @@ func NewHealth(m Meta, flash string) *Health {
Actions: []Action{{Label: "Re-check DNS", Icon: "ti-refresh", Post: "/server/health/recheck"}},
},
// Numbered by importance; the odd ones are the left stack.
Processes: Box{No: "01", Title: "Processes", ID: "processes"},
Sockets: Box{No: "02", Title: "Milter sockets", ID: "sockets"},
SenderCheck: Box{No: "03", Title: "Sender check", ID: "sender-check",
Processes: Box{No: "01", InCards: true, Title: "Processes", ID: "processes"},
Sockets: Box{No: "02", InCards: true, Title: "Milter sockets", ID: "sockets"},
SenderCheck: Box{No: "03", InCards: true, Title: "Sender check", ID: "sender-check",
Help: topicLink(HelpSenderCheck, "What is checked and what these numbers are for")},
Certificate: Box{No: "04", Title: "TLS certificate", ID: "certificate", Help: topicLink(HelpCertificate, "When it warns and what to do")},
Hostname: Box{No: "05", Title: "Hostname and reverse DNS", ID: "hostname", Help: topicLink(HelpRDNS, "Forward and reverse DNS")},
NotifyMail: Box{No: "06", Title: "Notification mail", ID: "notification-mail",
Certificate: Box{No: "04", InCards: true, Title: "TLS certificate", ID: "certificate", Help: topicLink(HelpCertificate, "When it warns and what to do")},
Hostname: Box{No: "05", InCards: true, Title: "Hostname and reverse DNS", ID: "hostname", Help: topicLink(HelpRDNS, "Forward and reverse DNS")},
NotifyMail: Box{No: "06", InCards: true, Title: "Notification mail", ID: "notification-mail",
Help: topicLink(HelpNotifications, "What the panel sends and what makes it arrive")},
Machine: Box{No: "07", Title: "Machine", ID: "machine", Help: topicLink(HelpChecks, "What these readings mean")},
Configuration: Box{No: "08", Title: "Configuration", ID: "configuration",
Machine: Box{No: "07", InCards: true, Title: "Machine", ID: "machine", Help: topicLink(HelpChecks, "What these readings mean")},
Configuration: Box{No: "08", InCards: true, Title: "Configuration", ID: "configuration",
End: Plain("Safe to run at any time")},
Reload: Action{Label: "Reload configuration", Icon: "ti-reload", Post: "/server/health/reload"},
}
+3 -3
View File
@@ -86,14 +86,14 @@ func KitPage() *Kit {
{Label: "signed out", Value: Rich("The split brand / form screen, no navbar. ", Link("/login", "Example"))},
},
Postmark: Box{No: "01", Title: "postmark", End: Plain("Worst status of the page's checks")},
Postmark: Box{No: "01", InCards: true, Title: "postmark", End: Plain("Worst status of the page's checks")},
Postmarks: []Postmark{
{Top: "SELFPOST", Word: "OK", Bottom: "6 OF 6 CHECKS", Level: LevelOK},
{Top: "SELFPOST", Word: "WARN", Bottom: "1 OF 6 CHECKS", Level: LevelWarn},
{Top: "DNS", Word: "FAIL", Bottom: "1 OF 3 RECORDS", Level: LevelFail},
},
StatusTag: Box{No: "03", Title: "status_tag", End: Plain("One mapping, one function")},
StatusTag: Box{No: "03", InCards: true, Title: "status_tag", End: Plain("One mapping, one function")},
Sheet: Sheet{ID: "kit-sheet", Tag: Tag{Status: "reject"}, Title: "sheet", Rows: []SheetRow{
{Label: "What it holds", Lines: []string{"What does not fit the row, as name and value"}},
{Label: "How it closes", Lines: []string{"Esc", "a click outside", "the cross"}, Mono: true},
@@ -106,7 +106,7 @@ func KitPage() *Kit {
{"held: quarantine · role: global", Tag{Status: "quarantine"}},
},
BoxHead: Box{No: "02", Title: "box · box_head",
BoxHead: Box{No: "02", InCards: true, Title: "box · box_head",
End: Plain("The end slot: a note, a tag, a link or a filter form"),
Help: &HelpLink{Href: "/help#checks", Title: "Optional: the section's Help topic"}},
BoxFoot: Foot{Text: Plain("box_foot — a legend, paging"), End: Plain("end slot")},
+14 -9
View File
@@ -329,20 +329,25 @@ code { background: var(--sp-well); color: var(--sp-ink); border-radius: 3px; pad
.sp-cards { display: flex; flex-direction: column; padding-inline: .75rem; }
.sp-cards > .column { display: contents; }
.sp-cards .box { margin-bottom: 1.5rem; }
.sp-cards > .column:first-child > .box:nth-child(1) { order: 1; }
.sp-cards > .column:first-child > .box:nth-child(2) { order: 3; }
.sp-cards > .column:first-child > .box:nth-child(3) { order: 5; }
.sp-cards > .column:first-child > .box:nth-child(4) { order: 7; }
.sp-cards > .column:last-child > .box:nth-child(1) { order: 2; }
.sp-cards > .column:last-child > .box:nth-child(2) { order: 4; }
.sp-cards > .column:last-child > .box:nth-child(3) { order: 6; }
.sp-cards > .column:last-child > .box:nth-child(4) { order: 8; }
.sp-cards [data-order="1"] { order: 1; }
.sp-cards [data-order="2"] { order: 2; }
.sp-cards [data-order="3"] { order: 3; }
.sp-cards [data-order="4"] { order: 4; }
.sp-cards [data-order="5"] { order: 5; }
.sp-cards [data-order="6"] { order: 6; }
.sp-cards [data-order="7"] { order: 7; }
.sp-cards [data-order="8"] { order: 8; }
.sp-cards [data-order="9"] { order: 9; }
.sp-cards [data-order="10"] { order: 10; }
.sp-cards [data-order="11"] { order: 11; }
.sp-cards [data-order="12"] { order: 12; }
}
/* cards: two stacks of boxes that are read in rows - the odd ones on the left,
the even ones on the right, each stack closed up so a short box leaves no
gap beside a tall one. On a phone the stacks dissolve and the boxes come in
the order of their numbers (the rules above). */
the order of their numbers: each box carries data-order, its own number (the
rules above, up to twelve), wherever it stands in its stack. */
/* sheet: what does not fit a table row, opened over the page from the row's
tag. It is the browser's own popover (the tag is a button with
+4 -4
View File
@@ -84,7 +84,7 @@
sp-credential (a secret shown once). A box foot, a table in its
table-container, a run of records, a log or an empty state sit directly
inside; anything else goes in body_open … body_close. */}}
{{define "box_open"}}<div class="box{{if eq .Variant "danger"}} sp-danger-zone{{else if eq .Variant "credential"}} sp-credential{{end}}"{{if .ID}} id="{{.ID}}"{{end}}{{if .OOB}} hx-swap-oob="true"{{end}}{{if .Poll}} data-poll aria-live="polite" hx-get="{{.Poll}}" hx-trigger="load" hx-swap="outerHTML"{{end}}{{if .DomainPick}} data-domain-pick{{end}}>
{{define "box_open"}}<div class="box{{if eq .Variant "danger"}} sp-danger-zone{{else if eq .Variant "credential"}} sp-credential{{end}}"{{if .ID}} id="{{.ID}}"{{end}}{{if .OOB}} hx-swap-oob="true"{{end}}{{if .Poll}} data-poll aria-live="polite" hx-get="{{.Poll}}" hx-trigger="load" hx-swap="outerHTML"{{end}}{{if .DomainPick}} data-domain-pick{{end}}{{with .Order}} data-order="{{.}}"{{end}}>
{{template "box_head" .}}
{{end}}
{{define "box_close"}}</div>{{end}}
@@ -93,9 +93,9 @@
`column is-6` stacks of numbered boxes that are read in rows, the odd ones
on the left and the even ones on the right, each stack closed up so that a
short box leaves no gap beside a tall one. On a phone the stacks dissolve
and the boxes come in the order of their numbers (panel.css) - which
knows four boxes to a stack and takes every child of a stack for a box:
a fifth, or a flash put between them, needs the rules extended first. */}}
and the boxes come in the order of their numbers: box_open writes each
numbered box's number as data-order, which panel.css orders by, up to
twelve. */}}
{{define "cards_open"}}<div class="columns sp-cards">{{end}}
{{define "cards_close"}}</div>{{end}}