Compare commits
3
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
cb5a4a22e1 | ||
|
|
5747f0b25c | ||
|
|
a618ffb0e4 |
@@ -78,6 +78,21 @@ Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); version
|
|||||||
the form's own message, naming the field and the application (audit
|
the form's own message, naming the field and the application (audit
|
||||||
2026-10-12, M10).
|
2026-10-12, M10).
|
||||||
|
|
||||||
|
- **A domain added again starts with an empty log.** Send-log rows belonged
|
||||||
|
to a domain by its name, so a domain deleted and added again showed its
|
||||||
|
earlier mail to whoever it was assigned to next, and its statistics and
|
||||||
|
level-2 count went on counting it. A row now belongs to the domain of the
|
||||||
|
application that sent it, by id (migration `0008`, filled once from the
|
||||||
|
registry); the rows of a deleted domain are the global role's alone. The
|
||||||
|
inbound log follows the same rule (audit 2026-10-12, L12).
|
||||||
|
|
||||||
|
- **A refusal no longer says which other domain sends from this server.**
|
||||||
|
A `From:` line that named another sender of the server was refused with
|
||||||
|
that sender's domain in the reply, which confirmed to whoever tried names
|
||||||
|
that it is hosted here. The reply now stops at "it names another sender of
|
||||||
|
this server"; the line as written stays in the reply and on the refused
|
||||||
|
message's page (audit 2026-10-12, L8).
|
||||||
|
|
||||||
### Added
|
### Added
|
||||||
|
|
||||||
- **The delivery log measures its own guesses** (migration `0007`). A table
|
- **The delivery log measures its own guesses** (migration `0007`). A table
|
||||||
|
|||||||
@@ -672,7 +672,7 @@ has already run is never edited — the next file is added; the published
|
|||||||
- [x] **Import validates like the form**: route `DMARCRua`, IPs, limit mode,
|
- [x] **Import validates like the form**: route `DMARCRua`, IPs, limit mode,
|
||||||
window and count through the same parsers; refuse the file with the
|
window and count through the same parsers; refuse the file with the
|
||||||
validator's text. Tests per field. — **Opus**
|
validator's text. Tests per field. — **Opus**
|
||||||
- [ ] **Send-log rows belong to a domain by id** (L12; owner, 2026-10-12).
|
- [x] **Send-log rows belong to a domain by id** (L12; owner, 2026-10-12).
|
||||||
`send_log.domain_id INTEGER REFERENCES domains(id) ON DELETE SET NULL` in
|
`send_log.domain_id INTEGER REFERENCES domains(id) ON DELETE SET NULL` in
|
||||||
migration `0008`, backfilled once by name against the registry as it is at
|
migration `0008`, backfilled once by name against the registry as it is at
|
||||||
migration time (rows of a domain already gone stay NULL); written by the
|
migration time (rows of a domain already gone stay NULL); written by the
|
||||||
@@ -686,9 +686,11 @@ has already run is never edited — the next file is added; the published
|
|||||||
migration — my proposal, so that the two logs do not follow two rules.
|
migration — my proposal, so that the two logs do not follow two rules.
|
||||||
Tests: delete-and-re-add shows nothing to the new assignee; the old rows
|
Tests: delete-and-re-add shows nothing to the new assignee; the old rows
|
||||||
are still in the global log. — **Opus**
|
are still in the global log. — **Opus**
|
||||||
- [ ] Docs: security.md § Roles (the guard) and the struck accepted risk, guide
|
*Done 2026-10-14.* DMARC reports stay by name on purpose: a report is
|
||||||
|
about the name in DNS, and whoever holds the name now is its reader.
|
||||||
|
- [x] Docs: security.md § Roles (the guard) and the struck accepted risk, guide
|
||||||
§ Domain export, schema-migrations.md. — **Sonnet**
|
§ Domain export, schema-migrations.md. — **Sonnet**
|
||||||
- [ ] **Which key OpenDKIM signs with, seen and not reasoned** (owner,
|
- [x] **Which key OpenDKIM signs with, seen and not reasoned** (owner,
|
||||||
2026-10-13, from the Not verified list of from-header-binding). The check
|
2026-10-13, from the Not verified list of from-header-binding). The check
|
||||||
and OpenDKIM read `From:` with two parsers; that they agree was argued
|
and OpenDKIM read `From:` with two parsers; that they agree was argued
|
||||||
from the check's two rules, never run. An e2e case sends, as one
|
from the check's two rules, never run. An e2e case sends, as one
|
||||||
@@ -699,7 +701,12 @@ has already run is never edited — the next file is added; the published
|
|||||||
shared with the check's unit test, so a line added to one is run by the
|
shared with the check's unit test, so a line added to one is run by the
|
||||||
other. A disagreement found is fixed in the check before the cut, and
|
other. A disagreement found is fixed in the check before the cut, and
|
||||||
security.md § Mail path says the claim is tested. - **Opus**
|
security.md § Mail path says the claim is tested. - **Opus**
|
||||||
- [ ] **The 550 reply names no other domain** (L8): `sender.go:334` stops at
|
*Done 2026-10-14:* fifteen lines (`internal/milter/testdata/from-lines-passed.txt`),
|
||||||
|
each signed for the application's domain alone, the signature valid. No
|
||||||
|
disagreement. Groups are refused by the check, so none is in the file;
|
||||||
|
a comment before the address - `(the shop) a@domain`, valid RFC 5322 - is
|
||||||
|
refused too ("it must be one address"): strict, not a hole, left as it is.
|
||||||
|
- [x] **The 550 reply names no other domain** (L8): `sender.go:334` stops at
|
||||||
"it names another sender of this server"; the refused message's page
|
"it names another sender of this server"; the refused message's page
|
||||||
still shows the `From:` line as written (`header_from`), so the operator
|
still shows the `From:` line as written (`header_from`), so the operator
|
||||||
loses nothing. Test pins the reply text; e2e's sender-check negatives
|
loses nothing. Test pins the reply text; e2e's sender-check negatives
|
||||||
|
|||||||
+10
-2
@@ -1098,7 +1098,10 @@ to a domain:
|
|||||||
who the application may send as, its client IP allow-list and its rate limit
|
who the application may send as, its client IP allow-list and its rate limit
|
||||||
are validated and saved together, or not at all, also when the application
|
are validated and saved together, or not at all, also when the application
|
||||||
is created. A password is shown once, on the page that answers the creation
|
is created. A password is shown once, on the page that answers the creation
|
||||||
or *New password*, and is never shown again.
|
or *New password*, and is never shown again. A new login is stored in lower
|
||||||
|
case - Postfix does not tell `Ops` from `ops`, so the panel does not let two
|
||||||
|
applications be told apart by case - and the page that shows the password
|
||||||
|
shows the login as it was stored.
|
||||||
- **Domain settings** (`…/settings`) — the *DMARC report address* (see [DMARC
|
- **Domain settings** (`…/settings`) — the *DMARC report address* (see [DMARC
|
||||||
reports](#dmarc-reports)), the domain's *Rate limit* (see [Rate limiting —
|
reports](#dmarc-reports)), the domain's *Rate limit* (see [Rate limiting —
|
||||||
level 2](#rate-limiting--level-2-domain-and-application)), *Export domain*
|
level 2](#rate-limiting--level-2-domain-and-application)), *Export domain*
|
||||||
@@ -1294,7 +1297,12 @@ applications' **working** SASL passwords, and configured **rate limits**
|
|||||||
instance without regenerating anything, so DNS (the DKIM TXT record)
|
instance without regenerating anything, so DNS (the DKIM TXT record)
|
||||||
doesn't need to change. Unlike a full restore (see [Full backup and
|
doesn't need to change. Unlike a full restore (see [Full backup and
|
||||||
restore](#full-backup-and-restore)), this works across different
|
restore](#full-backup-and-restore)), this works across different
|
||||||
hostnames/instances. *Import* is for the global role only; *export* is
|
hostnames/instances. The file is checked like the forms before anything is
|
||||||
|
created - the report address, every rate limit, every client address - and
|
||||||
|
refused with the form's own message, naming the field and the application.
|
||||||
|
An application login keeps the case it had (new logins are created in lower
|
||||||
|
case), and one that differs from a login of this server only in case is
|
||||||
|
refused. *Import* is for the global role only; *export* is
|
||||||
available to any user who can access the domain, **including a user with the
|
available to any user who can access the domain, **including a user with the
|
||||||
*domain* role** for a domain assigned to them — so that user can walk away with
|
*domain* role** for a domain assigned to them — so that user can walk away with
|
||||||
the domain's working SASL passwords, in the clear if they untick the encryption
|
the domain's working SASL passwords, in the clear if they untick the encryption
|
||||||
|
|||||||
+1
-1
@@ -42,7 +42,7 @@ in `git log` and [CHANGELOG.md](../CHANGELOG.md).
|
|||||||
| from-header-binding | Hold a message's `From:` header against the application that sends it | shipped `2.4.0` | 9/9 | [plans/from-header-binding.md](plans/from-header-binding.md) |
|
| from-header-binding | Hold a message's `From:` header against the application that sends it | shipped `2.4.0` | 9/9 | [plans/from-header-binding.md](plans/from-header-binding.md) |
|
||||||
| panel-redesign | Panel redesign (accepted mockups, design contract) | shipped `2.0.0` | 27/27 | [plans/panel-redesign.md](plans/panel-redesign.md) |
|
| panel-redesign | Panel redesign (accepted mockups, design contract) | shipped `2.0.0` | 27/27 | [plans/panel-redesign.md](plans/panel-redesign.md) |
|
||||||
| schema-squash | Squash SQLite migrations into a 2.x baseline | built in panel-redesign (stage 1) | — | [plans/panel-redesign.md](plans/panel-redesign.md) |
|
| schema-squash | Squash SQLite migrations into a 2.x baseline | built in panel-redesign (stage 1) | — | [plans/panel-redesign.md](plans/panel-redesign.md) |
|
||||||
| audit-2026-10-12 | Follow-ups of the 2026-10-12 full-tree code audit, seven stages | in progress (stage 0 shipped `2.4.2`; next stages 1+2 → `2.5.0`) | 18/62 | [audit-2026-10-12.md](audit-2026-10-12.md) |
|
| audit-2026-10-12 | Follow-ups of the 2026-10-12 full-tree code audit, seven stages | in progress (stage 0 shipped `2.4.2`; next stages 1+2 → `2.5.0`) | 22/62 | [audit-2026-10-12.md](audit-2026-10-12.md) |
|
||||||
|
|
||||||
**Recommended order** (not binding; owner, 2026-10-09: feature order follows
|
**Recommended order** (not binding; owner, 2026-10-09: feature order follows
|
||||||
development convenience): **panel-redesign** shipped as `2.0.0` (2026-10-11) and
|
development convenience): **panel-redesign** shipped as `2.0.0` (2026-10-11) and
|
||||||
|
|||||||
@@ -82,7 +82,7 @@ audit (`2.5.0`).
|
|||||||
| 5 | `0005_send_log_rejected_index.sql` | 2.3.0 | DDL | `idx_send_log_rejected`, a partial index of the refused rows of `send_log` by time; a file of its own because the test installation had run 0004 before the review asked for it |
|
| 5 | `0005_send_log_rejected_index.sql` | 2.3.0 | DDL | `idx_send_log_rejected`, a partial index of the refused rows of `send_log` by time; a file of its own because the test installation had run 0004 before the review asked for it |
|
||||||
| 6 | `0006_sender_check.sql` | 2.4.0 | DDL | `send_log.reason` and `send_log.header_from` (both default `''`: a row refused earlier keeps no reason), and `sender_check_days` |
|
| 6 | `0006_sender_check.sql` | 2.4.0 | DDL | `send_log.reason` and `send_log.header_from` (both default `''`: a row refused earlier keeps no reason), and `sender_check_days` |
|
||||||
| 7 | `0007_delivery_log_days.sql` | 2.5.0 | DDL | `delivery_log_days`: what the reconcile sweep and the log tailer did, a row per UTC day, kept a year, shown nowhere - the measurement that decides the sweep's fate (audit 2026-10-12, stage 1). `send_log.reason` gains the values `reconcile` and `reconcile-contradicted` without a schema change |
|
| 7 | `0007_delivery_log_days.sql` | 2.5.0 | DDL | `delivery_log_days`: what the reconcile sweep and the log tailer did, a row per UTC day, kept a year, shown nowhere - the measurement that decides the sweep's fate (audit 2026-10-12, stage 1). `send_log.reason` gains the values `reconcile` and `reconcile-contradicted` without a schema change |
|
||||||
| 8 | `0008_tenant_boundary.sql` | 2.5.0 | DDL | `idx_applications_login_nocase`: an application login is unique without regard to case (audit 2026-10-12, H4) |
|
| 8 | `0008_tenant_boundary.sql` | 2.5.0 | DDL + data | `idx_applications_login_nocase`: an application login is unique without regard to case (audit 2026-10-12, H4). `send_log.domain_id` and `inbound_spam_log.inbound_domain_id` (`ON DELETE SET NULL`), each with an index, filled once by name from the registry as it was then: a row belongs to a domain by id, and a domain deleted and added again starts with an empty log (L12) |
|
||||||
|
|
||||||
**Kind:** *DDL* — schema only; *data* — `INSERT`/`UPDATE` that must stay correct
|
**Kind:** *DDL* — schema only; *data* — `INSERT`/`UPDATE` that must stay correct
|
||||||
for operators upgrading from older 2.x images.
|
for operators upgrading from older 2.x images.
|
||||||
|
|||||||
+22
-8
@@ -137,7 +137,7 @@ assigned outbound domains and inbound domains **separately**, each list with
|
|||||||
| Outbound: domain page, applications, Domain settings (report address, rate limit, export), log | every domain | assigned outbound domains |
|
| Outbound: domain page, applications, Domain settings (report address, rate limit, export), log | every domain | assigned outbound domains |
|
||||||
| Outbound › DMARC reports | every report, with the server's ingest statistics | reports of assigned domains, without the ingest box |
|
| Outbound › DMARC reports | every report, with the server's ingest statistics | reports of assigned domains, without the ingest box |
|
||||||
| Inbound: domain page, upstream, recipients | every inbound domain | assigned inbound domains |
|
| Inbound: domain page, upstream, recipients | every inbound domain | assigned inbound domains |
|
||||||
| Inbound › Log | every row, including those of a domain deleted since | rows of assigned inbound domains (by name), without the links into the filter lists |
|
| Inbound › Log | every row, including those of a domain deleted since | rows of assigned inbound domains (by id: not those of an earlier domain of the same name), without the links into the filter lists |
|
||||||
| Inbound › Filter lists; the inbound log's retention | yes | no — 404 |
|
| Inbound › Filter lists; the inbound log's retention | yes | no — 404 |
|
||||||
| Add or delete a domain, either direction; import a domain | yes | no |
|
| Add or delete a domain, either direction; import a domain | yes | no |
|
||||||
| Account, Help | yes | yes |
|
| Account, Help | yes | yes |
|
||||||
@@ -152,7 +152,17 @@ assigned outbound domains and inbound domains **separately**, each list with
|
|||||||
(`lookupDomain`, `requireInboundDomain`) and lists are filtered by assignment
|
(`lookupDomain`, `requireInboundDomain`) and lists are filtered by assignment
|
||||||
in the query. Each denied path has a test for a `domain` user, including
|
in the query. Each denied path has a test for a `domain` user, including
|
||||||
another tenant's id.
|
another tenant's id.
|
||||||
- The panel will not remove or demote the last `global` user.
|
- The panel will not remove or demote the last `global` user. The guard is
|
||||||
|
inside the transaction that deletes or demotes: the write comes first and
|
||||||
|
the count after it, so two administrators deleting or demoting each other
|
||||||
|
at the same moment cannot both succeed; first-user setup is one statement
|
||||||
|
that inserts only into an empty table. A user's name, password, e-mail,
|
||||||
|
role and assignments are saved in one transaction, so a failure part way
|
||||||
|
leaves no domain user with nothing assigned.
|
||||||
|
- A log row - outbound or inbound - belongs to a domain by its id, written
|
||||||
|
when the row is: the outbound row's from the application that sent it. A
|
||||||
|
domain deleted and added again starts with an empty log for whoever it is
|
||||||
|
assigned to; the rows of the deleted one are the global role's alone.
|
||||||
- A `domain` user can export an assigned domain, and an export carries the
|
- A `domain` user can export an assigned domain, and an export carries the
|
||||||
working application passwords (encrypted unless the box is unticked): assign
|
working application passwords (encrypted unless the box is unticked): assign
|
||||||
domains with that in mind.
|
domains with that in mind.
|
||||||
@@ -203,7 +213,13 @@ operator can act on it.
|
|||||||
looks at the raw value once more for any other name of OpenDKIM's
|
looks at the raw value once more for any other name of OpenDKIM's
|
||||||
`SigningTable`, so that a line its parser and OpenDKIM's could read
|
`SigningTable`, so that a line its parser and OpenDKIM's could read
|
||||||
differently is refused rather than argued about; outside quoted strings
|
differently is refused rather than argued about; outside quoted strings
|
||||||
the line may hold one `@`. What this does not promise is what a display
|
the line may hold one `@`. That the two agree is tested, not only argued:
|
||||||
|
the e2e sends every `From:` line the check passes - a list it shares with
|
||||||
|
the check's unit test, `internal/milter/testdata/from-lines-passed.txt` -
|
||||||
|
and wants the message signed for the application's domain and no other.
|
||||||
|
The reply to a refusal says that the line names another sender of the
|
||||||
|
server, not which: it would confirm to whoever tries names that a domain
|
||||||
|
is hosted here. What this does not promise is what a display
|
||||||
name looks like to a person: a name of another sender written without an
|
name looks like to a person: a name of another sender written without an
|
||||||
`@` the scan can find (a full-width dot, a space inside the name) passes,
|
`@` the scan can find (a full-width dot, a space inside the name) passes,
|
||||||
signed with the application's own key - a forged display name is as old
|
signed with the application's own key - a forged display name is as old
|
||||||
@@ -420,11 +436,9 @@ deferred item from the roadmap.
|
|||||||
through - but no message data. The inbound journal itself is in the
|
through - but no message data. The inbound journal itself is in the
|
||||||
database, 0640. Trigger to revisit: a shared host, where the sidecar should
|
database, 0640. Trigger to revisit: a shared host, where the sidecar should
|
||||||
instead be given the files through a group it shares with the panel.
|
instead be given the files through a group it shares with the panel.
|
||||||
- **A row of the inbound log belongs to a domain by name.** A `domain` user
|
- ~~**A row of the inbound log belongs to a domain by name.**~~ Struck in
|
||||||
sees the rows whose inbound domain has the name of one assigned to them, so
|
`2.5.0` (audit 2026-10-12, L12): both logs now hold a domain by id, see
|
||||||
an inbound domain that is deleted and added again shows its earlier rows to
|
§ Roles.
|
||||||
whoever it is assigned to next. Accepted: the journal is of a mail domain,
|
|
||||||
and the rows age out with the retention.
|
|
||||||
- **Level-2 in-flight reservations can outlive a dropped SMTP session until TTL.**
|
- **Level-2 in-flight reservations can outlive a dropped SMTP session until TTL.**
|
||||||
The journal milter cannot observe connection-close directly through
|
The journal milter cannot observe connection-close directly through
|
||||||
go-milter, so if a client disconnects without an explicit `Abort`, its
|
go-milter, so if a client disconnects without an explicit `Abort`, its
|
||||||
|
|||||||
@@ -331,7 +331,11 @@ func (s *senderSession) judge() (shown, why string, err error) {
|
|||||||
return shown, "", fmt.Errorf("signing table: %w", err)
|
return shown, "", fmt.Errorf("signing table: %w", err)
|
||||||
}
|
}
|
||||||
if signs {
|
if signs {
|
||||||
return printable(raw), "it names another sender of this server, " + d, nil
|
// Which one is not said: the reply would confirm to whoever
|
||||||
|
// tries names that another tenant sends from this server (audit
|
||||||
|
// 2026-10-12, L8). The line as written is in the reply and on
|
||||||
|
// the refused message's page.
|
||||||
|
return printable(raw), "it names another sender of this server", nil
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
// And one that needs to know nothing of OpenDKIM's parser: outside quoted
|
// And one that needs to know nothing of OpenDKIM's parser: outside quoted
|
||||||
|
|||||||
@@ -3,6 +3,7 @@ package milter
|
|||||||
import (
|
import (
|
||||||
"os"
|
"os"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
|
"strconv"
|
||||||
"strings"
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
@@ -144,6 +145,14 @@ func TestSenderCheck(t *testing.T) {
|
|||||||
if !strings.HasPrefix(text, "y 550 5.7.1 From: ") || !strings.Contains(text, c.wantText) {
|
if !strings.HasPrefix(text, "y 550 5.7.1 From: ") || !strings.Contains(text, c.wantText) {
|
||||||
t.Fatalf("reply %q, want a 550 5.7.1 saying %q", text, c.wantText)
|
t.Fatalf("reply %q, want a 550 5.7.1 saying %q", text, c.wantText)
|
||||||
}
|
}
|
||||||
|
// The reason names no other domain of the server (audit L8); the
|
||||||
|
// From: line before it is what the client wrote itself.
|
||||||
|
reason := text[strings.LastIndex(text, " - ")+3:]
|
||||||
|
for _, other := range []string{"example.com", "mail.example.org", "example.org"} {
|
||||||
|
if strings.Contains(reason, other) {
|
||||||
|
t.Fatalf("the reason %q names %s", reason, other)
|
||||||
|
}
|
||||||
|
}
|
||||||
if len(rows) != 1 || rows[0].Reason != store.ReasonFrom || rows[0].To != "someone@elsewhere.example" || rows[0].AppLogin != c.login {
|
if len(rows) != 1 || rows[0].Reason != store.ReasonFrom || rows[0].To != "someone@elsewhere.example" || rows[0].AppLogin != c.login {
|
||||||
t.Fatalf("log rows: %+v", rows)
|
t.Fatalf("log rows: %+v", rows)
|
||||||
}
|
}
|
||||||
@@ -271,3 +280,37 @@ func TestSenderCheckRefusalWithNullSenderHasItsDomain(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// passedFromLines reads testdata/from-lines-passed.txt with {domain} put in.
|
||||||
|
func passedFromLines(t *testing.T, domain string) []string {
|
||||||
|
t.Helper()
|
||||||
|
raw, err := os.ReadFile(filepath.Join("testdata", "from-lines-passed.txt"))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
var out []string
|
||||||
|
for _, line := range strings.Split(string(raw), "\n") {
|
||||||
|
line = strings.TrimSpace(line)
|
||||||
|
if line == "" || strings.HasPrefix(line, "#") {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
s, err := strconv.Unquote(line)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("%s: %v", line, err)
|
||||||
|
}
|
||||||
|
out = append(out, strings.ReplaceAll(s, "{domain}", domain))
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
// Every line of the shared file is passed for the domain's application; the
|
||||||
|
// e2e then checks OpenDKIM signs each for that domain alone.
|
||||||
|
func TestSenderCheckPassesTheSharedLines(t *testing.T) {
|
||||||
|
bind := NewBindings(writeMaps(t, senderMaps))
|
||||||
|
for _, line := range passedFromLines(t, "shop.example.org") {
|
||||||
|
resp, rows := offer(t, bind, "shop", "shop@shop.example.org", [2]string{"From", line})
|
||||||
|
if resp != milter.RespContinue || len(rows) != 0 {
|
||||||
|
t.Errorf("From: %q refused: %s", line, replyText(resp))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
+22
@@ -0,0 +1,22 @@
|
|||||||
|
# From: lines the sender check lets through for an application of {domain}
|
||||||
|
# (audit 2026-10-12, stage 2: which key OpenDKIM signs with, seen and not
|
||||||
|
# reasoned). One Go-quoted string per line; {domain} is the application's
|
||||||
|
# domain. Two tests read this file: the check's unit test
|
||||||
|
# (internal/milter/sender_test.go) wants each line passed, and the e2e
|
||||||
|
# (test/e2e/dkim_from_test.go) sends each one and wants the message signed
|
||||||
|
# for {domain} and nothing else. A line added here is run by both.
|
||||||
|
"a@{domain}"
|
||||||
|
"<a@{domain}>"
|
||||||
|
"\"The Shop\" <Orders@{domain}>"
|
||||||
|
"orders+48213@{domain}"
|
||||||
|
"\"shop@{domain}\" <a@{domain}>"
|
||||||
|
"\"Orders, returns\" <a@{domain}>"
|
||||||
|
"\"The \\\"Shop\\\"\" <a@{domain}>"
|
||||||
|
"\"someone@gmail.example\" <a@{domain}>"
|
||||||
|
"=?utf-8?b?0JzQsNCz0LDQt9C40L0=?= <a@{domain}>"
|
||||||
|
"=?windows-1251?B?zODj4Ofo7Q==?= <a@{domain}>"
|
||||||
|
"=?koi8-r?Q?=ED=C1=C7=C1=DA=C9=CE?= <a@{domain}>"
|
||||||
|
"\"A very long display name of the shop\"\r\n <a@{domain}>"
|
||||||
|
"=?utf-8?b?0JzQsNCz0LDQt9C40L0=?=\r\n\t<a@{domain}>"
|
||||||
|
"a@{domain} (the shop)"
|
||||||
|
"a@{domain} (the (nested) shop)"
|
||||||
@@ -42,6 +42,10 @@ type InboundSpamEntry struct {
|
|||||||
At time.Time
|
At time.Time
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// inboundDomainID is the inbound_domain_id a journal row is written with: the
|
||||||
|
// inbound domain registered under the row's name now (audit 2026-10-12, L12).
|
||||||
|
const inboundDomainID = `(SELECT d.id FROM inbound_domains d WHERE d.name = ?)`
|
||||||
|
|
||||||
// InsertInboundSpam records one journal row. It is called from the inbound
|
// InsertInboundSpam records one journal row. It is called from the inbound
|
||||||
// journal milter's hot path and from the log tailer, so it returns any error
|
// journal milter's hot path and from the log tailer, so it returns any error
|
||||||
// for the caller to log; neither of them may hold up mail over it.
|
// for the caller to log; neither of them may hold up mail over it.
|
||||||
@@ -52,10 +56,10 @@ func (s *Store) InsertInboundSpam(e InboundSpamEntry) error {
|
|||||||
}
|
}
|
||||||
_, err := s.db.Exec(
|
_, err := s.db.Exec(
|
||||||
`INSERT INTO inbound_spam_log
|
`INSERT INTO inbound_spam_log
|
||||||
(inbound_domain, client_ip, from_addr, to_addr, subject, decision, engine, detail, created_at)
|
(inbound_domain, client_ip, from_addr, to_addr, subject, decision, engine, detail, created_at, inbound_domain_id)
|
||||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)`,
|
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, `+inboundDomainID+`)`,
|
||||||
e.InboundDomain, e.ClientIP, e.From, e.To, e.Subject, e.Decision, e.Engine, e.Detail,
|
e.InboundDomain, e.ClientIP, e.From, e.To, e.Subject, e.Decision, e.Engine, e.Detail,
|
||||||
at.UTC().Format(time.RFC3339),
|
at.UTC().Format(time.RFC3339), e.InboundDomain,
|
||||||
)
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("insert inbound_spam_log: %w", err)
|
return fmt.Errorf("insert inbound_spam_log: %w", err)
|
||||||
@@ -73,13 +77,13 @@ func (s *Store) InsertInboundSpamOnce(e InboundSpamEntry) (bool, error) {
|
|||||||
at := e.At.UTC().Format(time.RFC3339)
|
at := e.At.UTC().Format(time.RFC3339)
|
||||||
res, err := s.db.Exec(
|
res, err := s.db.Exec(
|
||||||
`INSERT INTO inbound_spam_log
|
`INSERT INTO inbound_spam_log
|
||||||
(inbound_domain, client_ip, from_addr, to_addr, subject, decision, engine, detail, created_at)
|
(inbound_domain, client_ip, from_addr, to_addr, subject, decision, engine, detail, created_at, inbound_domain_id)
|
||||||
SELECT ?, ?, ?, ?, ?, ?, ?, ?, ?
|
SELECT ?, ?, ?, ?, ?, ?, ?, ?, ?, `+inboundDomainID+`
|
||||||
WHERE NOT EXISTS (
|
WHERE NOT EXISTS (
|
||||||
SELECT 1 FROM inbound_spam_log
|
SELECT 1 FROM inbound_spam_log
|
||||||
WHERE created_at = ? AND client_ip = ? AND from_addr = ? AND to_addr = ?
|
WHERE created_at = ? AND client_ip = ? AND from_addr = ? AND to_addr = ?
|
||||||
AND decision = ? AND engine = ? AND detail = ?)`,
|
AND decision = ? AND engine = ? AND detail = ?)`,
|
||||||
e.InboundDomain, e.ClientIP, e.From, e.To, e.Subject, e.Decision, e.Engine, e.Detail, at,
|
e.InboundDomain, e.ClientIP, e.From, e.To, e.Subject, e.Decision, e.Engine, e.Detail, at, e.InboundDomain,
|
||||||
at, e.ClientIP, e.From, e.To, e.Decision, e.Engine, e.Detail,
|
at, e.ClientIP, e.From, e.To, e.Decision, e.Engine, e.Detail,
|
||||||
)
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -133,9 +137,10 @@ type InboundSpamRow struct {
|
|||||||
// the UI: an empty field matches everything. Since and Until are days, both
|
// the UI: an empty field matches everything. Since and Until are days, both
|
||||||
// inclusive and read as UTC days whatever the time of day they carry.
|
// inclusive and read as UTC days whatever the time of day they carry.
|
||||||
//
|
//
|
||||||
// Domains is the authorization scope, which no query parameter may widen: the
|
// DomainIDs is the authorization scope, which no query parameter may widen:
|
||||||
// exhaustive set of inbound domain names the caller may read, applied as an IN
|
// the exhaustive set of inbound domains the caller may read, by id - a row of
|
||||||
// constraint. nil lifts it (the global role reads the whole journal, including
|
// a domain deleted since is nobody's (audit 2026-10-12, L12) - applied as an
|
||||||
|
// IN constraint. nil lifts it (the global role reads the whole journal, including
|
||||||
// rows of a domain deleted since); an empty set is a caller who reaches
|
// rows of a domain deleted since); an empty set is a caller who reaches
|
||||||
// nothing and gets nothing. Unlike SendLogFilter, forgetting the scope is
|
// nothing and gets nothing. Unlike SendLogFilter, forgetting the scope is
|
||||||
// therefore the permissive case - the caller states "all" by leaving it out -
|
// therefore the permissive case - the caller states "all" by leaving it out -
|
||||||
@@ -145,7 +150,9 @@ type InboundSpamFilter struct {
|
|||||||
Decision string
|
Decision string
|
||||||
Since time.Time
|
Since time.Time
|
||||||
Until time.Time
|
Until time.Time
|
||||||
Domains []string
|
// DomainIDs: the inbound domains the caller may read, by id; nil reads
|
||||||
|
// everything (the global role).
|
||||||
|
DomainIDs []int64
|
||||||
}
|
}
|
||||||
|
|
||||||
// QueryInboundSpam returns journal rows matching filter, newest first, for
|
// QueryInboundSpam returns journal rows matching filter, newest first, for
|
||||||
@@ -197,16 +204,16 @@ func (s *Store) CountInboundSpam(filter InboundSpamFilter) (int64, error) {
|
|||||||
func inboundSpamWhere(f InboundSpamFilter) (string, []any) {
|
func inboundSpamWhere(f InboundSpamFilter) (string, []any) {
|
||||||
var clauses []string
|
var clauses []string
|
||||||
var args []any
|
var args []any
|
||||||
if f.Domains != nil {
|
if f.DomainIDs != nil {
|
||||||
if len(f.Domains) == 0 {
|
if len(f.DomainIDs) == 0 {
|
||||||
return " WHERE 1 = 0", nil
|
return " WHERE 1 = 0", nil
|
||||||
}
|
}
|
||||||
marks := make([]string, len(f.Domains))
|
marks := make([]string, len(f.DomainIDs))
|
||||||
for i, name := range f.Domains {
|
for i, id := range f.DomainIDs {
|
||||||
marks[i] = "?"
|
marks[i] = "?"
|
||||||
args = append(args, name)
|
args = append(args, id)
|
||||||
}
|
}
|
||||||
clauses = append(clauses, "inbound_domain IN ("+strings.Join(marks, ", ")+")")
|
clauses = append(clauses, "inbound_domain_id IN ("+strings.Join(marks, ", ")+")")
|
||||||
}
|
}
|
||||||
if f.Domain != "" {
|
if f.Domain != "" {
|
||||||
clauses = append(clauses, "inbound_domain = ?")
|
clauses = append(clauses, "inbound_domain = ?")
|
||||||
|
|||||||
@@ -172,6 +172,14 @@ func TestQueryInboundSpamFilters(t *testing.T) {
|
|||||||
t.Fatalf("InsertInboundSpam: %v", err)
|
t.Fatalf("InsertInboundSpam: %v", err)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
ids := map[string]int64{}
|
||||||
|
for _, name := range []string{"a.example", "b.example"} {
|
||||||
|
d, err := st.AddInboundDomain(name)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
ids[name] = d.ID
|
||||||
|
}
|
||||||
add("a.example", "1@a.example", SpamDecisionAccept, day(20, 23))
|
add("a.example", "1@a.example", SpamDecisionAccept, day(20, 23))
|
||||||
add("a.example", "2@a.example", SpamDecisionReject, day(21, 0))
|
add("a.example", "2@a.example", SpamDecisionReject, day(21, 0))
|
||||||
add("b.example", "3@b.example", SpamDecisionAccept, day(21, 12))
|
add("b.example", "3@b.example", SpamDecisionAccept, day(21, 12))
|
||||||
@@ -227,13 +235,13 @@ func TestQueryInboundSpamFilters(t *testing.T) {
|
|||||||
|
|
||||||
// The scope: nil is everything, a set is its domains, an empty set nothing,
|
// The scope: nil is everything, a set is its domains, an empty set nothing,
|
||||||
// and a chosen domain outside the set is nothing too.
|
// and a chosen domain outside the set is nothing too.
|
||||||
if got := to(InboundSpamFilter{Domains: []string{"b.example"}}); got != "43" {
|
if got := to(InboundSpamFilter{DomainIDs: []int64{ids["b.example"]}}); got != "43" {
|
||||||
t.Errorf("scope = %q, want 43", got)
|
t.Errorf("scope = %q, want 43", got)
|
||||||
}
|
}
|
||||||
if got := to(InboundSpamFilter{Domains: []string{}}); got != "" {
|
if got := to(InboundSpamFilter{DomainIDs: []int64{}}); got != "" {
|
||||||
t.Errorf("empty scope = %q, want nothing", got)
|
t.Errorf("empty scope = %q, want nothing", got)
|
||||||
}
|
}
|
||||||
if got := to(InboundSpamFilter{Domains: []string{"b.example"}, Domain: "a.example"}); got != "" {
|
if got := to(InboundSpamFilter{DomainIDs: []int64{ids["b.example"]}, Domain: "a.example"}); got != "" {
|
||||||
t.Errorf("a chosen domain outside the scope = %q, want nothing", got)
|
t.Errorf("a chosen domain outside the scope = %q, want nothing", got)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -9,3 +9,20 @@
|
|||||||
-- (case-sensitive) stays: SQLite cannot change a column's collation without
|
-- (case-sensitive) stays: SQLite cannot change a column's collation without
|
||||||
-- rebuilding the table, and this index is the stricter rule.
|
-- rebuilding the table, and this index is the stricter rule.
|
||||||
CREATE UNIQUE INDEX idx_applications_login_nocase ON applications (login COLLATE NOCASE);
|
CREATE UNIQUE INDEX idx_applications_login_nocase ON applications (login COLLATE NOCASE);
|
||||||
|
|
||||||
|
-- A send-log row belongs to a domain by its id, not its name (audit L12; the
|
||||||
|
-- owner, 2026-10-12). A domain deleted and added again under its old name
|
||||||
|
-- used to show the earlier rows to whoever it was assigned to next. The id is
|
||||||
|
-- the domain of the login the row was written for; a row of a domain deleted
|
||||||
|
-- since - and every row older than this migration whose domain is gone - is
|
||||||
|
-- NULL and nobody's but the global role's. The name stays for display and for
|
||||||
|
-- the global view. Filled once from the registry as it is now.
|
||||||
|
ALTER TABLE send_log ADD COLUMN domain_id INTEGER REFERENCES domains(id) ON DELETE SET NULL;
|
||||||
|
UPDATE send_log SET domain_id = (SELECT d.id FROM domains d WHERE d.name = send_log.domain);
|
||||||
|
CREATE INDEX idx_send_log_domain_id_created_at ON send_log (domain_id, created_at);
|
||||||
|
|
||||||
|
-- The inbound journal follows the same rule, so the two logs do not follow two
|
||||||
|
-- (security.md's accepted risk of the inbound log, struck).
|
||||||
|
ALTER TABLE inbound_spam_log ADD COLUMN inbound_domain_id INTEGER REFERENCES inbound_domains(id) ON DELETE SET NULL;
|
||||||
|
UPDATE inbound_spam_log SET inbound_domain_id = (SELECT d.id FROM inbound_domains d WHERE d.name = inbound_spam_log.inbound_domain);
|
||||||
|
CREATE INDEX idx_inbound_spam_log_domain_id_created_at ON inbound_spam_log (inbound_domain_id, created_at);
|
||||||
|
|||||||
@@ -185,8 +185,8 @@ func (s *Store) LimitRefusalsSince(since time.Time) ([]LimitRefusal, error) {
|
|||||||
rows, err := s.db.Query(
|
rows, err := s.db.Query(
|
||||||
`SELECT d.name, a.login, COUNT(*), MIN(l.created_at)
|
`SELECT d.name, a.login, COUNT(*), MIN(l.created_at)
|
||||||
FROM send_log l
|
FROM send_log l
|
||||||
JOIN applications a ON a.login = l.app_login
|
JOIN domains d ON d.id = l.domain_id
|
||||||
JOIN domains d ON d.id = a.domain_id
|
JOIN applications a ON a.login = l.app_login AND a.domain_id = d.id
|
||||||
WHERE l.status = 'rejected' AND l.created_at >= ? AND l.reason IN ('', 'limit')
|
WHERE l.status = 'rejected' AND l.created_at >= ? AND l.reason IN ('', 'limit')
|
||||||
AND (EXISTS (SELECT 1 FROM rate_limits r WHERE r.scope = 'application' AND r.ref_id = a.id)
|
AND (EXISTS (SELECT 1 FROM rate_limits r WHERE r.scope = 'application' AND r.ref_id = a.id)
|
||||||
OR EXISTS (SELECT 1 FROM rate_limits r WHERE r.scope = 'domain' AND r.ref_id = d.id))
|
OR EXISTS (SELECT 1 FROM rate_limits r WHERE r.scope = 'domain' AND r.ref_id = d.id))
|
||||||
|
|||||||
@@ -170,13 +170,17 @@ func (s *Store) RateLimit(scope, ref string) (RateLimit, bool, error) {
|
|||||||
// semantics — and excludes rows that were themselves rejected by a limit (they
|
// semantics — and excludes rows that were themselves rejected by a limit (they
|
||||||
// were never sent). It reuses the send log the journal already writes (README
|
// were never sent). It reuses the send log the journal already writes (README
|
||||||
// § Rate limiting — the limiter reuses the send log).
|
// § Rate limiting — the limiter reuses the send log).
|
||||||
|
//
|
||||||
|
// A domain's messages are the rows of the domain registered under ref now, by
|
||||||
|
// id: rows a deleted domain of the same name left behind are not its (audit
|
||||||
|
// 2026-10-12, L12).
|
||||||
func (s *Store) CountMessages(scope, ref string, since time.Time) (int64, error) {
|
func (s *Store) CountMessages(scope, ref string, since time.Time) (int64, error) {
|
||||||
var column string
|
var match string
|
||||||
switch scope {
|
switch scope {
|
||||||
case RateLimitScopeDomain:
|
case RateLimitScopeDomain:
|
||||||
column = "domain"
|
match = domainByName
|
||||||
case RateLimitScopeApp:
|
case RateLimitScopeApp:
|
||||||
column = "app_login"
|
match = "app_login = ?"
|
||||||
default:
|
default:
|
||||||
return 0, fmt.Errorf("unknown rate-limit scope %q", scope)
|
return 0, fmt.Errorf("unknown rate-limit scope %q", scope)
|
||||||
}
|
}
|
||||||
@@ -185,7 +189,7 @@ func (s *Store) CountMessages(scope, ref string, since time.Time) (int64, error)
|
|||||||
// same format is chronologically correct (as in DeleteSendLogBefore).
|
// same format is chronologically correct (as in DeleteSendLogBefore).
|
||||||
err := s.db.QueryRow(
|
err := s.db.QueryRow(
|
||||||
`SELECT COUNT(DISTINCT queue_id) FROM send_log
|
`SELECT COUNT(DISTINCT queue_id) FROM send_log
|
||||||
WHERE `+column+` = ? AND status != ? AND created_at >= ?`,
|
WHERE `+match+` AND status != ? AND created_at >= ?`,
|
||||||
ref, StatusRejected, since.UTC().Format(time.RFC3339),
|
ref, StatusRejected, since.UTC().Format(time.RFC3339),
|
||||||
).Scan(&n)
|
).Scan(&n)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
|||||||
@@ -84,6 +84,11 @@ func TestRateLimitByNameAndLogin(t *testing.T) {
|
|||||||
|
|
||||||
func TestCountMessagesDistinctAndWindowed(t *testing.T) {
|
func TestCountMessagesDistinctAndWindowed(t *testing.T) {
|
||||||
st := openTestStore(t)
|
st := openTestStore(t)
|
||||||
|
// A domain's messages are those of its applications (send_log.domain_id).
|
||||||
|
d, _ := st.AddDomain("example.com", "selfpost")
|
||||||
|
if _, err := st.AddApplication(d.ID, "app1", AddressModeWildcard, nil); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
// Two recipients share a queue-id → one message. A second message → two.
|
// Two recipients share a queue-id → one message. A second message → two.
|
||||||
for _, to := range []string{"a@x.net", "b@x.net"} {
|
for _, to := range []string{"a@x.net", "b@x.net"} {
|
||||||
|
|||||||
@@ -47,7 +47,7 @@ func TestBaselineSchema(t *testing.T) {
|
|||||||
"applications": "id domain_id login address_mode auth_ip_restrict auth_allowed_ips created_at",
|
"applications": "id domain_id login address_mode auth_ip_restrict auth_allowed_ips created_at",
|
||||||
"application_addresses": "id application_id address",
|
"application_addresses": "id application_id address",
|
||||||
"rate_limits": "id scope ref_id max_messages window_seconds mode auto_multiplier auto_updated_at",
|
"rate_limits": "id scope ref_id max_messages window_seconds mode auto_multiplier auto_updated_at",
|
||||||
"send_log": "id queue_id domain app_login from_addr to_addr subject status created_at updated_at reason header_from",
|
"send_log": "id queue_id domain app_login from_addr to_addr subject status created_at updated_at reason header_from domain_id",
|
||||||
"logtail_state": "path fingerprint read_offset updated_at",
|
"logtail_state": "path fingerprint read_offset updated_at",
|
||||||
"inbound_domains": "id name recipient_mode created_at",
|
"inbound_domains": "id name recipient_mode created_at",
|
||||||
"inbound_transports": "inbound_domain_id host port tls_mode",
|
"inbound_transports": "inbound_domain_id host port tls_mode",
|
||||||
@@ -55,7 +55,7 @@ func TestBaselineSchema(t *testing.T) {
|
|||||||
"user_inbound_domains": "user_id inbound_domain_id",
|
"user_inbound_domains": "user_id inbound_domain_id",
|
||||||
"dmarc_reports": "id domain reporter report_id period_begin period_end received_at contact_email policy_p policy_sp policy_pct policy_adkim policy_aspf pass_count fail_count recipient",
|
"dmarc_reports": "id domain reporter report_id period_begin period_end received_at contact_email policy_p policy_sp policy_pct policy_adkim policy_aspf pass_count fail_count recipient",
|
||||||
"dmarc_report_records": "id report_row_id source_ip count disposition spf_result dkim_result header_from",
|
"dmarc_report_records": "id report_row_id source_ip count disposition spf_result dkim_result header_from",
|
||||||
"inbound_spam_log": "id inbound_domain client_ip from_addr to_addr subject decision engine detail created_at",
|
"inbound_spam_log": "id inbound_domain client_ip from_addr to_addr subject decision engine detail created_at inbound_domain_id",
|
||||||
"inbound_spam_list": "id list_type entry_type value note created_at",
|
"inbound_spam_list": "id list_type entry_type value note created_at",
|
||||||
"send_log_lines": "id queue_id logged_at line",
|
"send_log_lines": "id queue_id logged_at line",
|
||||||
"notify_state": "key phase count mark since sent_at",
|
"notify_state": "key phase count mark since sent_at",
|
||||||
@@ -105,8 +105,8 @@ func TestBaselineSchema(t *testing.T) {
|
|||||||
rows.Close()
|
rows.Close()
|
||||||
wantIdx := []string{
|
wantIdx := []string{
|
||||||
"idx_applications_login_nocase", "idx_dmarc_report_records_report", "idx_dmarc_reports_domain", "idx_dmarc_reports_received",
|
"idx_applications_login_nocase", "idx_dmarc_report_records_report", "idx_dmarc_reports_domain", "idx_dmarc_reports_received",
|
||||||
"idx_inbound_spam_log_created_at", "idx_inbound_spam_log_domain_created_at",
|
"idx_inbound_spam_log_created_at", "idx_inbound_spam_log_domain_created_at", "idx_inbound_spam_log_domain_id_created_at",
|
||||||
"idx_send_log_app_login_created_at", "idx_send_log_created_at", "idx_send_log_domain", "idx_send_log_lines_queue_id", "idx_send_log_queue_id", "idx_send_log_rejected",
|
"idx_send_log_app_login_created_at", "idx_send_log_created_at", "idx_send_log_domain", "idx_send_log_domain_id_created_at", "idx_send_log_lines_queue_id", "idx_send_log_queue_id", "idx_send_log_rejected",
|
||||||
"idx_sessions_expires_at",
|
"idx_sessions_expires_at",
|
||||||
}
|
}
|
||||||
sort.Strings(wantIdx)
|
sort.Strings(wantIdx)
|
||||||
|
|||||||
+29
-19
@@ -40,6 +40,12 @@ type SendLogEntry struct {
|
|||||||
HeaderFrom string
|
HeaderFrom string
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// loginDomainID is the domain_id a row is written with: the domain of the
|
||||||
|
// application whose login it is, NULL for a login that is no application's
|
||||||
|
// (audit 2026-10-12, L12). The domain name of the row stays what the
|
||||||
|
// envelope said.
|
||||||
|
const loginDomainID = `(SELECT a.domain_id FROM applications a WHERE a.login = ?)`
|
||||||
|
|
||||||
// InsertQueued records an accepted message in the send log with status
|
// InsertQueued records an accepted message in the send log with status
|
||||||
// "queued". It is called from the journal-milter hot path, so it returns any
|
// "queued". It is called from the journal-milter hot path, so it returns any
|
||||||
// error for the caller to log rather than deciding policy here; the milter
|
// error for the caller to log rather than deciding policy here; the milter
|
||||||
@@ -48,9 +54,9 @@ func (s *Store) InsertQueued(e SendLogEntry) error {
|
|||||||
now := time.Now().UTC().Format(time.RFC3339)
|
now := time.Now().UTC().Format(time.RFC3339)
|
||||||
_, err := s.db.Exec(
|
_, err := s.db.Exec(
|
||||||
`INSERT INTO send_log
|
`INSERT INTO send_log
|
||||||
(queue_id, domain, app_login, from_addr, to_addr, subject, status, created_at, updated_at)
|
(queue_id, domain, app_login, from_addr, to_addr, subject, status, created_at, updated_at, domain_id)
|
||||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)`,
|
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, `+loginDomainID+`)`,
|
||||||
e.QueueID, e.Domain, e.AppLogin, e.From, e.To, e.Subject, StatusQueued, now, now,
|
e.QueueID, e.Domain, e.AppLogin, e.From, e.To, e.Subject, StatusQueued, now, now, e.AppLogin,
|
||||||
)
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("insert send_log: %w", err)
|
return fmt.Errorf("insert send_log: %w", err)
|
||||||
@@ -67,9 +73,9 @@ func (s *Store) InsertRejected(e SendLogEntry) error {
|
|||||||
now := time.Now().UTC().Format(time.RFC3339)
|
now := time.Now().UTC().Format(time.RFC3339)
|
||||||
_, err := s.db.Exec(
|
_, err := s.db.Exec(
|
||||||
`INSERT INTO send_log
|
`INSERT INTO send_log
|
||||||
(queue_id, domain, app_login, from_addr, to_addr, subject, status, reason, header_from, created_at, updated_at)
|
(queue_id, domain, app_login, from_addr, to_addr, subject, status, reason, header_from, created_at, updated_at, domain_id)
|
||||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
|
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, `+loginDomainID+`)`,
|
||||||
e.QueueID, e.Domain, e.AppLogin, e.From, e.To, e.Subject, StatusRejected, e.Reason, e.HeaderFrom, now, now,
|
e.QueueID, e.Domain, e.AppLogin, e.From, e.To, e.Subject, StatusRejected, e.Reason, e.HeaderFrom, now, now, e.AppLogin,
|
||||||
)
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("insert rejected send_log: %w", err)
|
return fmt.Errorf("insert rejected send_log: %w", err)
|
||||||
@@ -204,6 +210,9 @@ type SendLogRow struct {
|
|||||||
ID int64
|
ID int64
|
||||||
QueueID string
|
QueueID string
|
||||||
Domain string
|
Domain string
|
||||||
|
// DomainID is the domain the row belongs to; 0 when it is nobody's but
|
||||||
|
// the global role's (its domain was deleted, or its login was none).
|
||||||
|
DomainID int64
|
||||||
AppLogin string
|
AppLogin string
|
||||||
From string
|
From string
|
||||||
To string
|
To string
|
||||||
@@ -231,9 +240,9 @@ func (s *Store) GetSendLog(id int64) (SendLogRow, error) {
|
|||||||
createdAt, updatedAt string
|
createdAt, updatedAt string
|
||||||
)
|
)
|
||||||
err := s.db.QueryRow(
|
err := s.db.QueryRow(
|
||||||
`SELECT id, queue_id, domain, app_login, from_addr, to_addr, subject, status, reason, header_from, created_at, updated_at
|
`SELECT id, queue_id, domain, COALESCE(domain_id, 0), app_login, from_addr, to_addr, subject, status, reason, header_from, created_at, updated_at
|
||||||
FROM send_log WHERE id = ?`, id,
|
FROM send_log WHERE id = ?`, id,
|
||||||
).Scan(&row.ID, &row.QueueID, &row.Domain, &row.AppLogin, &row.From, &row.To,
|
).Scan(&row.ID, &row.QueueID, &row.Domain, &row.DomainID, &row.AppLogin, &row.From, &row.To,
|
||||||
&row.Subject, &row.Status, &row.Reason, &row.HeaderFrom, &createdAt, &updatedAt)
|
&row.Subject, &row.Status, &row.Reason, &row.HeaderFrom, &createdAt, &updatedAt)
|
||||||
if errors.Is(err, sql.ErrNoRows) {
|
if errors.Is(err, sql.ErrNoRows) {
|
||||||
return SendLogRow{}, ErrSendLogNotFound
|
return SendLogRow{}, ErrSendLogNotFound
|
||||||
@@ -252,9 +261,10 @@ func (s *Store) GetSendLog(id int64) (SendLogRow, error) {
|
|||||||
// Domain and AppLogin are the operator's own filters, chosen in the UI: an
|
// Domain and AppLogin are the operator's own filters, chosen in the UI: an
|
||||||
// empty field matches everything.
|
// empty field matches everything.
|
||||||
//
|
//
|
||||||
// Domains and AllDomains are the authorization scope, which no query parameter
|
// DomainIDs and AllDomains are the authorization scope, which no query
|
||||||
// may widen. Domains is the exhaustive set of domain names the caller is
|
// parameter may widen. DomainIDs is the exhaustive set of domains the caller
|
||||||
// entitled to read, applied as an IN constraint; AllDomains lifts that
|
// is entitled to read, by id - a row whose domain was deleted is nobody's
|
||||||
|
// (audit 2026-10-12, L12) - applied as an IN constraint; AllDomains lifts that
|
||||||
// restriction and is the only way to read the whole journal. A zero-valued
|
// restriction and is the only way to read the whole journal. A zero-valued
|
||||||
// filter therefore matches *no* rows: a caller that forgets to state a scope
|
// filter therefore matches *no* rows: a caller that forgets to state a scope
|
||||||
// gets an empty log rather than every tenant's mail, which is the failure mode
|
// gets an empty log rather than every tenant's mail, which is the failure mode
|
||||||
@@ -262,7 +272,7 @@ func (s *Store) GetSendLog(id int64) (SendLogRow, error) {
|
|||||||
type SendLogFilter struct {
|
type SendLogFilter struct {
|
||||||
Domain string
|
Domain string
|
||||||
AppLogin string
|
AppLogin string
|
||||||
Domains []string
|
DomainIDs []int64
|
||||||
AllDomains bool
|
AllDomains bool
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -273,7 +283,7 @@ func (s *Store) QuerySendLog(filter SendLogFilter, limit, offset int) ([]SendLog
|
|||||||
where, args := sendLogWhere(filter)
|
where, args := sendLogWhere(filter)
|
||||||
args = append(args, limit, offset)
|
args = append(args, limit, offset)
|
||||||
rows, err := s.db.Query(
|
rows, err := s.db.Query(
|
||||||
`SELECT id, queue_id, domain, app_login, from_addr, to_addr, subject, status, reason, header_from, created_at, updated_at
|
`SELECT id, queue_id, domain, COALESCE(domain_id, 0), app_login, from_addr, to_addr, subject, status, reason, header_from, created_at, updated_at
|
||||||
FROM send_log`+where+`
|
FROM send_log`+where+`
|
||||||
ORDER BY id DESC
|
ORDER BY id DESC
|
||||||
LIMIT ? OFFSET ?`,
|
LIMIT ? OFFSET ?`,
|
||||||
@@ -290,7 +300,7 @@ func (s *Store) QuerySendLog(filter SendLogFilter, limit, offset int) ([]SendLog
|
|||||||
row SendLogRow
|
row SendLogRow
|
||||||
createdAt, updatedAt string
|
createdAt, updatedAt string
|
||||||
)
|
)
|
||||||
if err := rows.Scan(&row.ID, &row.QueueID, &row.Domain, &row.AppLogin,
|
if err := rows.Scan(&row.ID, &row.QueueID, &row.Domain, &row.DomainID, &row.AppLogin,
|
||||||
&row.From, &row.To, &row.Subject, &row.Status, &row.Reason, &row.HeaderFrom, &createdAt, &updatedAt); err != nil {
|
&row.From, &row.To, &row.Subject, &row.Status, &row.Reason, &row.HeaderFrom, &createdAt, &updatedAt); err != nil {
|
||||||
return nil, fmt.Errorf("scan send_log row: %w", err)
|
return nil, fmt.Errorf("scan send_log row: %w", err)
|
||||||
}
|
}
|
||||||
@@ -319,15 +329,15 @@ func sendLogWhere(f SendLogFilter) (string, []any) {
|
|||||||
// No scope is not "no restriction": a domain administrator whose last
|
// No scope is not "no restriction": a domain administrator whose last
|
||||||
// assignment was deleted owns nothing in the journal and must see
|
// assignment was deleted owns nothing in the journal and must see
|
||||||
// nothing, and the same clause catches a caller that never set a scope.
|
// nothing, and the same clause catches a caller that never set a scope.
|
||||||
if len(f.Domains) == 0 {
|
if len(f.DomainIDs) == 0 {
|
||||||
return " WHERE 1 = 0", nil
|
return " WHERE 1 = 0", nil
|
||||||
}
|
}
|
||||||
marks := make([]string, len(f.Domains))
|
marks := make([]string, len(f.DomainIDs))
|
||||||
for i, name := range f.Domains {
|
for i, id := range f.DomainIDs {
|
||||||
marks[i] = "?"
|
marks[i] = "?"
|
||||||
args = append(args, name)
|
args = append(args, id)
|
||||||
}
|
}
|
||||||
clauses = append(clauses, "domain IN ("+strings.Join(marks, ", ")+")")
|
clauses = append(clauses, "domain_id IN ("+strings.Join(marks, ", ")+")")
|
||||||
}
|
}
|
||||||
if f.Domain != "" {
|
if f.Domain != "" {
|
||||||
clauses = append(clauses, "domain = ?")
|
clauses = append(clauses, "domain = ?")
|
||||||
|
|||||||
@@ -274,7 +274,13 @@ func TestUpdateStatusNoMatch(t *testing.T) {
|
|||||||
// decided who is asking, and the safe answer to that is nothing.
|
// decided who is asking, and the safe answer to that is nothing.
|
||||||
func TestSendLogScopeIsMandatory(t *testing.T) {
|
func TestSendLogScopeIsMandatory(t *testing.T) {
|
||||||
st := openTestStore(t)
|
st := openTestStore(t)
|
||||||
|
ids := map[string]int64{}
|
||||||
for _, domain := range []string{"first.example.ru", "second.example.ru"} {
|
for _, domain := range []string{"first.example.ru", "second.example.ru"} {
|
||||||
|
d := addTestDomain(t, st, domain)
|
||||||
|
ids[domain] = d.ID
|
||||||
|
if _, err := st.AddApplication(d.ID, "app-"+domain, AddressModeWildcard, nil); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
if err := st.InsertQueued(SendLogEntry{
|
if err := st.InsertQueued(SendLogEntry{
|
||||||
QueueID: "Q-" + domain, Domain: domain, AppLogin: "app-" + domain,
|
QueueID: "Q-" + domain, Domain: domain, AppLogin: "app-" + domain,
|
||||||
From: "noreply@" + domain, To: "public@example.net", Subject: domain,
|
From: "noreply@" + domain, To: "public@example.net", Subject: domain,
|
||||||
@@ -288,14 +294,14 @@ func TestSendLogScopeIsMandatory(t *testing.T) {
|
|||||||
want []string
|
want []string
|
||||||
}{
|
}{
|
||||||
"no scope": {SendLogFilter{}, nil},
|
"no scope": {SendLogFilter{}, nil},
|
||||||
"empty scope": {SendLogFilter{Domains: []string{}}, nil},
|
"empty scope": {SendLogFilter{DomainIDs: []int64{}}, nil},
|
||||||
"all domains": {SendLogFilter{AllDomains: true}, []string{"first.example.ru", "second.example.ru"}},
|
"all domains": {SendLogFilter{AllDomains: true}, []string{"first.example.ru", "second.example.ru"}},
|
||||||
"one domain": {SendLogFilter{Domains: []string{"first.example.ru"}}, []string{"first.example.ru"}},
|
"one domain": {SendLogFilter{DomainIDs: []int64{ids["first.example.ru"]}}, []string{"first.example.ru"}},
|
||||||
"two domains": {SendLogFilter{Domains: []string{"first.example.ru", "second.example.ru"}}, []string{"first.example.ru", "second.example.ru"}},
|
"two domains": {SendLogFilter{DomainIDs: []int64{ids["first.example.ru"], ids["second.example.ru"]}}, []string{"first.example.ru", "second.example.ru"}},
|
||||||
"unknown domain": {SendLogFilter{Domains: []string{"third.example.ru"}}, nil},
|
"unknown domain": {SendLogFilter{DomainIDs: []int64{9999}}, nil},
|
||||||
"filter within": {SendLogFilter{Domain: "first.example.ru", Domains: []string{"first.example.ru", "second.example.ru"}}, []string{"first.example.ru"}},
|
"filter within": {SendLogFilter{Domain: "first.example.ru", DomainIDs: []int64{ids["first.example.ru"], ids["second.example.ru"]}}, []string{"first.example.ru"}},
|
||||||
"filter outside": {SendLogFilter{Domain: "second.example.ru", Domains: []string{"first.example.ru"}}, nil},
|
"filter outside": {SendLogFilter{Domain: "second.example.ru", DomainIDs: []int64{ids["first.example.ru"]}}, nil},
|
||||||
"app filter outside": {SendLogFilter{AppLogin: "app-second.example.ru", Domains: []string{"first.example.ru"}}, nil},
|
"app filter outside": {SendLogFilter{AppLogin: "app-second.example.ru", DomainIDs: []int64{ids["first.example.ru"]}}, nil},
|
||||||
} {
|
} {
|
||||||
rows, err := st.QuerySendLog(tc.filter, 50, 0)
|
rows, err := st.QuerySendLog(tc.filter, 50, 0)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -352,3 +358,46 @@ func TestDeleteSendLogBefore(t *testing.T) {
|
|||||||
t.Fatalf("retention kept wrong rows: %+v", rows)
|
t.Fatalf("retention kept wrong rows: %+v", rows)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// A domain deleted and added again under its old name starts with an empty
|
||||||
|
// log for whoever it is assigned to: its earlier rows belong to no domain and
|
||||||
|
// only the global role reads them (audit 2026-10-12, L12). Statistics and the
|
||||||
|
// level-2 count follow the same rule.
|
||||||
|
func TestADomainAddedAgainStartsWithAnEmptyLog(t *testing.T) {
|
||||||
|
st := openTestStore(t)
|
||||||
|
old := addTestDomain(t, st, "shop.example")
|
||||||
|
if _, err := st.AddApplication(old.ID, "shop", AddressModeWildcard, nil); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := st.InsertQueued(SendLogEntry{QueueID: "OLD", Domain: "shop.example", AppLogin: "shop", To: "a@example.net"}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := st.DeleteDomain(old.ID); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
again := addTestDomain(t, st, "shop.example")
|
||||||
|
if _, err := st.AddApplication(again.ID, "shop", AddressModeWildcard, nil); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := st.InsertQueued(SendLogEntry{QueueID: "NEW", Domain: "shop.example", AppLogin: "shop", To: "a@example.net"}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
rows, err := st.QuerySendLog(SendLogFilter{DomainIDs: []int64{again.ID}}, 10, 0)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(rows) != 1 || rows[0].QueueID != "NEW" || rows[0].DomainID != again.ID {
|
||||||
|
t.Fatalf("the new domain's log = %+v, want its own row alone", rows)
|
||||||
|
}
|
||||||
|
all, _ := st.QuerySendLog(SendLogFilter{AllDomains: true}, 10, 0)
|
||||||
|
if len(all) != 2 {
|
||||||
|
t.Fatalf("the global log has %d rows, want both", len(all))
|
||||||
|
}
|
||||||
|
if n, err := st.CountMessages(RateLimitScopeDomain, "shop.example", time.Now().Add(-time.Hour)); err != nil || n != 1 {
|
||||||
|
t.Fatalf("level-2 count = %d, %v; want the new domain's one message", n, err)
|
||||||
|
}
|
||||||
|
if stats, err := st.DomainSendStats("shop.example", 90, time.Now().Add(-time.Hour)); err != nil || stats.Total != 1 {
|
||||||
|
t.Fatalf("statistics = %+v, %v; want one message", stats, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
+15
-8
@@ -46,10 +46,11 @@ func StatsWindow(retentionDays int, entityCreated time.Time) (since time.Time, h
|
|||||||
}
|
}
|
||||||
|
|
||||||
// DomainSendStats returns 30-day (or shorter when retention is lower) send
|
// DomainSendStats returns 30-day (or shorter when retention is lower) send
|
||||||
// statistics for a domain name.
|
// statistics for a domain, named by its name: the rows that belong to the
|
||||||
|
// domain registered under it now (audit 2026-10-12, L12).
|
||||||
func (s *Store) DomainSendStats(name string, retentionDays int, createdAt time.Time) (SendStats, error) {
|
func (s *Store) DomainSendStats(name string, retentionDays int, createdAt time.Time) (SendStats, error) {
|
||||||
since, hours, windowDays := StatsWindow(retentionDays, createdAt)
|
since, hours, windowDays := StatsWindow(retentionDays, createdAt)
|
||||||
stats, err := s.sendStats("domain", name, since, hours)
|
stats, err := s.sendStats(domainByName, name, since, hours)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return SendStats{}, err
|
return SendStats{}, err
|
||||||
}
|
}
|
||||||
@@ -61,7 +62,7 @@ func (s *Store) DomainSendStats(name string, retentionDays int, createdAt time.T
|
|||||||
// AppSendStats returns send statistics for an application login.
|
// AppSendStats returns send statistics for an application login.
|
||||||
func (s *Store) AppSendStats(login string, retentionDays int, createdAt time.Time) (SendStats, error) {
|
func (s *Store) AppSendStats(login string, retentionDays int, createdAt time.Time) (SendStats, error) {
|
||||||
since, hours, windowDays := StatsWindow(retentionDays, createdAt)
|
since, hours, windowDays := StatsWindow(retentionDays, createdAt)
|
||||||
stats, err := s.sendStats("app_login", login, since, hours)
|
stats, err := s.sendStats("app_login = ?", login, since, hours)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return SendStats{}, err
|
return SendStats{}, err
|
||||||
}
|
}
|
||||||
@@ -70,30 +71,36 @@ func (s *Store) AppSendStats(login string, retentionDays int, createdAt time.Tim
|
|||||||
return stats, nil
|
return stats, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func (s *Store) sendStats(column, ref string, since time.Time, hours float64) (SendStats, error) {
|
// domainByName selects a domain's send-log rows by the id of the domain
|
||||||
|
// registered under the name it is given.
|
||||||
|
const domainByName = "domain_id = (SELECT d.id FROM domains d WHERE d.name = ?)"
|
||||||
|
|
||||||
|
// sendStats counts the rows that match, a condition with one placeholder
|
||||||
|
// taking ref.
|
||||||
|
func (s *Store) sendStats(match, ref string, since time.Time, hours float64) (SendStats, error) {
|
||||||
sinceStr := since.UTC().Format(time.RFC3339)
|
sinceStr := since.UTC().Format(time.RFC3339)
|
||||||
|
|
||||||
var total int64
|
var total int64
|
||||||
err := s.db.QueryRow(
|
err := s.db.QueryRow(
|
||||||
`SELECT COUNT(DISTINCT queue_id) FROM send_log
|
`SELECT COUNT(DISTINCT queue_id) FROM send_log
|
||||||
WHERE `+column+` = ? AND status != ? AND created_at >= ?`,
|
WHERE `+match+` AND status != ? AND created_at >= ?`,
|
||||||
ref, StatusRejected, sinceStr,
|
ref, StatusRejected, sinceStr,
|
||||||
).Scan(&total)
|
).Scan(&total)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return SendStats{}, fmt.Errorf("send stats total %s %q: %w", column, ref, err)
|
return SendStats{}, fmt.Errorf("send stats total %q: %w", ref, err)
|
||||||
}
|
}
|
||||||
|
|
||||||
var peak int64
|
var peak int64
|
||||||
err = s.db.QueryRow(
|
err = s.db.QueryRow(
|
||||||
`SELECT COALESCE(MAX(bucket_count), 0) FROM (
|
`SELECT COALESCE(MAX(bucket_count), 0) FROM (
|
||||||
SELECT COUNT(DISTINCT queue_id) AS bucket_count FROM send_log
|
SELECT COUNT(DISTINCT queue_id) AS bucket_count FROM send_log
|
||||||
WHERE `+column+` = ? AND status != ? AND created_at >= ?
|
WHERE `+match+` AND status != ? AND created_at >= ?
|
||||||
GROUP BY substr(created_at, 1, 13)
|
GROUP BY substr(created_at, 1, 13)
|
||||||
)`,
|
)`,
|
||||||
ref, StatusRejected, sinceStr,
|
ref, StatusRejected, sinceStr,
|
||||||
).Scan(&peak)
|
).Scan(&peak)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return SendStats{}, fmt.Errorf("send stats peak %s %q: %w", column, ref, err)
|
return SendStats{}, fmt.Errorf("send stats peak %q: %w", ref, err)
|
||||||
}
|
}
|
||||||
|
|
||||||
avg := float64(total) / hours
|
avg := float64(total) / hours
|
||||||
|
|||||||
@@ -57,3 +57,13 @@ func domainNameSet(domains []store.Domain) map[string]bool {
|
|||||||
}
|
}
|
||||||
return m
|
return m
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// domainIDSet is the set of the ids of domains; a send-log row's DomainID is
|
||||||
|
// in it when the row belongs to one of them (0, a row of no domain, never is).
|
||||||
|
func domainIDSet(domains []store.Domain) map[int64]bool {
|
||||||
|
m := make(map[int64]bool, len(domains))
|
||||||
|
for _, d := range domains {
|
||||||
|
m[d.ID] = true
|
||||||
|
}
|
||||||
|
return m
|
||||||
|
}
|
||||||
|
|||||||
@@ -189,6 +189,10 @@ func TestOverviewPageShowsVerdictsAndDomains(t *testing.T) {
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("AddDomain: %v", err)
|
t.Fatalf("AddDomain: %v", err)
|
||||||
}
|
}
|
||||||
|
// A row is the domain's through its application (send_log.domain_id).
|
||||||
|
if _, err := h.store.AddApplication(d.ID, "app", store.AddressModeWildcard, nil); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
if err := h.store.InsertQueued(store.SendLogEntry{QueueID: "Q1", Domain: d.Name, AppLogin: "app", From: "noreply@" + d.Name, To: "public@example.net", Subject: "Hello"}); err != nil {
|
if err := h.store.InsertQueued(store.SendLogEntry{QueueID: "Q1", Domain: d.Name, AppLogin: "app", From: "noreply@" + d.Name, To: "public@example.net", Subject: "Hello"}); err != nil {
|
||||||
t.Fatalf("InsertQueued: %v", err)
|
t.Fatalf("InsertQueued: %v", err)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -39,9 +39,11 @@ func (h *Handlers) HandleInboundLog(w http.ResponseWriter, r *http.Request) {
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
names := make([]string, 0, len(assigned))
|
names := make([]string, 0, len(assigned))
|
||||||
|
ids := make([]int64, 0, len(assigned))
|
||||||
offered := make(map[string]bool, len(assigned))
|
offered := make(map[string]bool, len(assigned))
|
||||||
for _, d := range assigned {
|
for _, d := range assigned {
|
||||||
names = append(names, d.Name)
|
names = append(names, d.Name)
|
||||||
|
ids = append(ids, d.ID)
|
||||||
offered[d.Name] = true
|
offered[d.Name] = true
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -65,7 +67,7 @@ func (h *Handlers) HandleInboundLog(w http.ResponseWriter, r *http.Request) {
|
|||||||
// behind by a domain that has since been deleted; anyone else only what is
|
// behind by a domain that has since been deleted; anyone else only what is
|
||||||
// assigned to them, however empty that is.
|
// assigned to them, however empty that is.
|
||||||
if !p.IsGlobal() {
|
if !p.IsGlobal() {
|
||||||
query.Domains = names
|
query.DomainIDs = ids
|
||||||
}
|
}
|
||||||
|
|
||||||
total, err := h.store.CountInboundSpam(query)
|
total, err := h.store.CountInboundSpam(query)
|
||||||
|
|||||||
@@ -94,7 +94,7 @@ func (h *Handlers) HandleDelivery(w http.ResponseWriter, r *http.Request) {
|
|||||||
http.Error(w, "internal error", http.StatusInternalServerError)
|
http.Error(w, "internal error", http.StatusInternalServerError)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
if !p.IsGlobal() && !domainNameSet(allowed)[row.Domain] {
|
if !p.IsGlobal() && !domainIDSet(allowed)[row.DomainID] {
|
||||||
http.NotFound(w, r)
|
http.NotFound(w, r)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
@@ -113,7 +113,7 @@ func (h *Handlers) HandleDelivery(w http.ResponseWriter, r *http.Request) {
|
|||||||
// The names link to their pages when there is a page to go to: a domain
|
// The names link to their pages when there is a page to go to: a domain
|
||||||
// that was deleted since, or an application of another domain, is text.
|
// that was deleted since, or an application of another domain, is text.
|
||||||
for _, d := range allowed {
|
for _, d := range allowed {
|
||||||
if d.Name != row.Domain {
|
if d.ID != row.DomainID {
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
in.DomainHref = view.DomainHref(d.ID)
|
in.DomainHref = view.DomainHref(d.ID)
|
||||||
@@ -317,8 +317,10 @@ func (h *Handlers) outLogPage(r *http.Request) (*view.OutLog, error) {
|
|||||||
allowedNames := domainNameSet(assigned)
|
allowedNames := domainNameSet(assigned)
|
||||||
|
|
||||||
domainNames := make([]string, 0, len(assigned))
|
domainNames := make([]string, 0, len(assigned))
|
||||||
|
domainIDs := make([]int64, 0, len(assigned))
|
||||||
for _, d := range assigned {
|
for _, d := range assigned {
|
||||||
domainNames = append(domainNames, d.Name)
|
domainNames = append(domainNames, d.Name)
|
||||||
|
domainIDs = append(domainIDs, d.ID)
|
||||||
}
|
}
|
||||||
|
|
||||||
loginSet := make(map[string]bool)
|
loginSet := make(map[string]bool)
|
||||||
@@ -341,8 +343,10 @@ func (h *Handlers) outLogPage(r *http.Request) (*view.OutLog, error) {
|
|||||||
Domain: q.Get("domain"),
|
Domain: q.Get("domain"),
|
||||||
AppLogin: q.Get("app"),
|
AppLogin: q.Get("app"),
|
||||||
// A global administrator reads the whole journal, including rows left
|
// A global administrator reads the whole journal, including rows left
|
||||||
// behind by a domain that has since been deleted.
|
// behind by a domain that has since been deleted; anyone else the rows
|
||||||
Domains: domainNames,
|
// of the domains assigned to them, by id - not those an earlier domain
|
||||||
|
// of the same name left (audit 2026-10-12, L12).
|
||||||
|
DomainIDs: domainIDs,
|
||||||
AllDomains: p.IsGlobal(),
|
AllDomains: p.IsGlobal(),
|
||||||
}
|
}
|
||||||
if !p.IsGlobal() {
|
if !p.IsGlobal() {
|
||||||
|
|||||||
@@ -7,6 +7,7 @@ import (
|
|||||||
"testing"
|
"testing"
|
||||||
|
|
||||||
"github.com/mixeme/selfpost/internal/store"
|
"github.com/mixeme/selfpost/internal/store"
|
||||||
|
"github.com/mixeme/selfpost/internal/web/auth"
|
||||||
)
|
)
|
||||||
|
|
||||||
// The users form's writes, driven through the handlers (audit 2026-10-12, M7
|
// The users form's writes, driven through the handlers (audit 2026-10-12, M7
|
||||||
@@ -97,3 +98,55 @@ func TestUserFormCreatesWithEmailAndReach(t *testing.T) {
|
|||||||
t.Fatalf("created %+v", u)
|
t.Fatalf("created %+v", u)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// A domain deleted and added again under its old name starts with an empty
|
||||||
|
// log for whoever it is assigned to; its earlier rows are the global role's
|
||||||
|
// alone (audit 2026-10-12, L12).
|
||||||
|
func TestADomainAddedAgainShowsItsAssigneeNothingOld(t *testing.T) {
|
||||||
|
h, st := inboundHandlers(t)
|
||||||
|
root := userWith(t, st, "root", store.RoleGlobal, store.Reach{})
|
||||||
|
old, _ := st.AddDomain("shop.example", "mail")
|
||||||
|
st.AddApplication(old.ID, "shop", store.AddressModeWildcard, nil)
|
||||||
|
if err := st.InsertQueued(store.SendLogEntry{QueueID: "OLDQ", Domain: "shop.example", AppLogin: "shop", To: "a@example.net", Subject: "old mail"}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := st.DeleteDomain(old.ID); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
again, _ := st.AddDomain("shop.example", "mail")
|
||||||
|
st.AddApplication(again.ID, "shop", store.AddressModeWildcard, nil)
|
||||||
|
if err := st.InsertQueued(store.SendLogEntry{QueueID: "NEWQ", Domain: "shop.example", AppLogin: "shop", To: "a@example.net", Subject: "new mail"}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
owner := userWith(t, st, "owner", store.RoleDomain, store.Reach{DomainIDs: []int64{again.ID}})
|
||||||
|
rows, _ := st.QuerySendLog(store.SendLogFilter{AllDomains: true}, 10, 0)
|
||||||
|
var oldID, newID int64
|
||||||
|
for _, r := range rows {
|
||||||
|
if r.QueueID == "OLDQ" {
|
||||||
|
oldID = r.ID
|
||||||
|
} else {
|
||||||
|
newID = r.ID
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
page := func(p *auth.Principal, id int64) int {
|
||||||
|
return send(h.HandleDelivery, p, "GET", "/outbound/log/"+idStr(id), map[string]string{"id": idStr(id)}, nil).Code
|
||||||
|
}
|
||||||
|
if code := page(&owner, oldID); code != http.StatusNotFound {
|
||||||
|
t.Errorf("the old row's page for the new assignee = %d, want 404", code)
|
||||||
|
}
|
||||||
|
if code := page(&owner, newID); code != http.StatusOK {
|
||||||
|
t.Errorf("the new row's page for its assignee = %d, want 200", code)
|
||||||
|
}
|
||||||
|
if code := page(&root, oldID); code != http.StatusOK {
|
||||||
|
t.Errorf("the old row's page for the global role = %d, want 200", code)
|
||||||
|
}
|
||||||
|
body := send(h.HandleDeliveries, &owner, "GET", "/outbound/log", nil, nil).Body.String()
|
||||||
|
if strings.Contains(body, "old mail") || !strings.Contains(body, "new mail") {
|
||||||
|
t.Errorf("the new assignee's log shows the old row or misses the new one")
|
||||||
|
}
|
||||||
|
body = send(h.HandleDeliveries, &root, "GET", "/outbound/log", nil, nil).Body.String()
|
||||||
|
if !strings.Contains(body, "old mail") || !strings.Contains(body, "new mail") {
|
||||||
|
t.Errorf("the global log is missing a row")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -0,0 +1,84 @@
|
|||||||
|
package e2e
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/emersion/go-msgauth/dkim"
|
||||||
|
)
|
||||||
|
|
||||||
|
// passedFromLines reads the From: lines the sender check passes, the file its
|
||||||
|
// unit test reads too, with the application's domain put in.
|
||||||
|
func passedFromLines(repoRoot, domain string) ([]string, error) {
|
||||||
|
raw, err := os.ReadFile(filepath.Join(repoRoot, "internal", "milter", "testdata", "from-lines-passed.txt"))
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
var out []string
|
||||||
|
for _, line := range strings.Split(string(raw), "\n") {
|
||||||
|
line = strings.TrimSpace(line)
|
||||||
|
if line == "" || strings.HasPrefix(line, "#") {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
s, err := strconv.Unquote(line)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("%s: %w", line, err)
|
||||||
|
}
|
||||||
|
out = append(out, strings.ReplaceAll(s, "{domain}", domain))
|
||||||
|
}
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// testSignedForItsDomainOnly: the sender check and OpenDKIM read From: with
|
||||||
|
// two parsers, and that they agree was argued, never run (audit 2026-10-12,
|
||||||
|
// stage 2). Every From: line the check passes for an application is sent as
|
||||||
|
// that application, and what arrives must be signed for its domain - with a
|
||||||
|
// signature that validates - and for no other: a line the check reads as the
|
||||||
|
// application's own and OpenDKIM as another sender's would be signed with
|
||||||
|
// that sender's key.
|
||||||
|
func testSignedForItsDomainOnly(t *testing.T, sc *scenario) {
|
||||||
|
lines, err := passedFromLines(h.repoRoot, senderDomain)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
for _, line := range lines {
|
||||||
|
token := uniqueToken("dkimfrom")
|
||||||
|
res := attemptSend(sendAttempt{
|
||||||
|
authLogin: sc.appLogin, authPassword: sc.appPassword,
|
||||||
|
from: "alerts@" + senderDomain, headerFrom: line, to: recipient,
|
||||||
|
subject: "e2e " + token, body: "which key signs this",
|
||||||
|
})
|
||||||
|
if !res.ok() {
|
||||||
|
t.Errorf("From: %q was refused: %v", line, res.firstErr())
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
raw, err := findSinkMessage(h.stageDir, token, 30*time.Second)
|
||||||
|
if err != nil {
|
||||||
|
t.Errorf("From: %q never reached the sink: %v", line, err)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
verifications, err := dkim.VerifyWithOptions(bytes.NewReader(raw), &dkim.VerifyOptions{LookupTXT: lookupTXTFunc()})
|
||||||
|
if err != nil {
|
||||||
|
t.Errorf("From: %q: dkim verify: %v", line, err)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
valid := false
|
||||||
|
for _, v := range verifications {
|
||||||
|
if v.Domain != senderDomain {
|
||||||
|
t.Errorf("From: %q was signed for %s", line, v.Domain)
|
||||||
|
}
|
||||||
|
if v.Domain == senderDomain && v.Err == nil {
|
||||||
|
valid = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !valid {
|
||||||
|
t.Errorf("From: %q has no valid signature for %s (%d signature(s))", line, senderDomain, len(verifications))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -310,6 +310,9 @@ func TestE2E(t *testing.T) {
|
|||||||
run("sender_check_holds_the_from_line", func(t *testing.T) {
|
run("sender_check_holds_the_from_line", func(t *testing.T) {
|
||||||
testSenderCheck(t, sc)
|
testSenderCheck(t, sc)
|
||||||
})
|
})
|
||||||
|
run("passed_from_lines_signed_for_their_domain_only", func(t *testing.T) {
|
||||||
|
testSignedForItsDomainOnly(t, sc)
|
||||||
|
})
|
||||||
run("panel_down_defers_mail", func(t *testing.T) {
|
run("panel_down_defers_mail", func(t *testing.T) {
|
||||||
testPanelDownDefersMail(t, sc)
|
testPanelDownDefersMail(t, sc)
|
||||||
})
|
})
|
||||||
|
|||||||
@@ -83,6 +83,10 @@ func testSenderCheck(t *testing.T, sc *scenario) {
|
|||||||
// DATA for the client; it is permanent and says what is wrong.
|
// DATA for the client; it is permanent and says what is wrong.
|
||||||
if e := fmt.Sprint(res.dataErr); res.dataErr == nil || !strings.HasPrefix(e, "550 ") || !strings.Contains(e, "5.7.1 From:") {
|
if e := fmt.Sprint(res.dataErr); res.dataErr == nil || !strings.HasPrefix(e, "550 ") || !strings.Contains(e, "5.7.1 From:") {
|
||||||
t.Errorf("%s: want 550 5.7.1 naming the From: line at the end of DATA, got %v", c.name, res.firstErr())
|
t.Errorf("%s: want 550 5.7.1 naming the From: line at the end of DATA, got %v", c.name, res.firstErr())
|
||||||
|
} else if reason := e[strings.LastIndex(e, " - ")+3:]; strings.Contains(reason, otherDomain) {
|
||||||
|
// What the client wrote is said back; that the other domain is
|
||||||
|
// a sender of this server is not (audit 2026-10-12, L8).
|
||||||
|
t.Errorf("%s: the reason %q names %s", c.name, reason, otherDomain)
|
||||||
}
|
}
|
||||||
if _, err := findSinkMessage(h.stageDir, token, 2*time.Second); err == nil {
|
if _, err := findSinkMessage(h.stageDir, token, 2*time.Second); err == nil {
|
||||||
t.Errorf("%s: the refused message reached the sink", c.name)
|
t.Errorf("%s: the refused message reached the sink", c.name)
|
||||||
|
|||||||
Reference in New Issue
Block a user