inbound-antispam-panel: Filter lists, kind column, matches, note on a sheet
Follows the design commit before it (owner's words quoted there). The lists' tables are Value, Kind, Matched and Remove. The kind tag opens the entry's sheet: note, day added, how often it decided a message in the days the log keeps, and the last match. Matches are read off the journal on each view, nothing is stored (inbound.Lists.Usage, store.ListSpamListHits). A row counts for an entry when the list decided the message - a rejected row whose reply is the deny rule's message, an accepted row whose verdict carries the allow rule's symbol for that kind - and the entry fits the row's sender or client address the way rspamd matches it: an address whole, a domain as the sender's domain, a network containing the client (an IPv4-mapped address as IPv4). A failure to count shows the lists with "never" and logs why. For that the journal milter keeps the lists' own symbols ahead of its six-symbol cap: rspamd writes symbols heaviest first, so an allow entry's -100 came last and was cut behind any six others. The plan gains "Filter list usage": what counting from the journal implies, and - the owner asked for it in writing - the conditions under which a counter kept with each entry becomes worth a migration (the journal cut by its row cap and not by age; a short retention; counts that must outlive the journal; a slow page). multimap.conf says which of its texts SelfPost reads back. Tests: Usage over every kind on both lists and the rows that are nobody's; the milter's symbol order; the store query; the page and its sheets; a handler test from journal rows to the numbers on the page. Seen: the fixture page with a sheet open, headless Edge at 1360 px - the same as the accepted drawing. Checked on the test server, uncommitted tree: go vet ./... ok, go test ./... ok (22 packages), make e2e ok. Not seen in a real run: a count above zero. It needs an entry added in the panel, which I cannot sign in to, and then mail that the list decides; in particular the deny rule's message arriving in mail.log as the reply of a milter-reject line is what the trial with rspamc showed, not yet what Postfix logged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
+5
-1
@@ -52,7 +52,11 @@ Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); version
|
|||||||
*Deny sender* on accepted mail, *Allow sender* on mail the filter refused.
|
*Deny sender* on accepted mail, *Allow sender* on mail the filter refused.
|
||||||
- **Inbound › Filter lists** (global role): add and remove allow and deny
|
- **Inbound › Filter lists** (global role): add and remove allow and deny
|
||||||
entries; the page says where the lists go and whether the filter is on to
|
entries; the page says where the lists go and whether the filter is on to
|
||||||
read them.
|
read them. Each entry shows how often it decided a message - *Matched*, or
|
||||||
|
*never* - and its kind opens a sheet with the note, the day it was added and
|
||||||
|
the last match, so that entries nothing matches any more can be found and
|
||||||
|
removed. The count is read from the inbound log and reaches back as far as
|
||||||
|
the log does.
|
||||||
- **Server › Settings**: *Keep inbound log rows, days*, shown with the inbound
|
- **Server › Settings**: *Keep inbound log rows, days*, shown with the inbound
|
||||||
relay on.
|
relay on.
|
||||||
- Help: *Inbound log* and *Filter lists*.
|
- Help: *Inbound log* and *Filter lists*.
|
||||||
|
|||||||
@@ -11,6 +11,10 @@
|
|||||||
# by the deny list still overrides.
|
# by the deny list still overrides.
|
||||||
#
|
#
|
||||||
# A domain entry is that domain, not its subdomains.
|
# A domain entry is that domain, not its subdomains.
|
||||||
|
#
|
||||||
|
# SelfPost counts how often an entry decided a message by what these rules
|
||||||
|
# leave behind: the deny message below and the names of the allow symbols.
|
||||||
|
# Change either and Inbound > Filter lists shows "never" for every entry.
|
||||||
|
|
||||||
SELFPOST_DENY_IP {
|
SELFPOST_DENY_IP {
|
||||||
type = "ip";
|
type = "ip";
|
||||||
|
|||||||
@@ -114,6 +114,13 @@ smtp/inet` removes Debian's stock smtpd). Outbound delivery still uses the
|
|||||||
prefilters rspamd lets the one that ran first decide, and deny then won or
|
prefilters rspamd lets the one that ran first decide, and deny then won or
|
||||||
lost by rule order (seen on rspamd 3.11). A refusal by the deny list
|
lost by rule order (seen on rspamd 3.11). A refusal by the deny list
|
||||||
overrides any score, so deny wins whichever entries match.
|
overrides any score, so deny wins whichever entries match.
|
||||||
|
How often an entry decided a message is not stored: `Lists.Usage` reads it
|
||||||
|
off the journal on each view of Filter lists - the rows a list decided (the
|
||||||
|
deny rule's reply, the allow rule's symbol, which the journal milter keeps
|
||||||
|
ahead of its six-symbol cap) matched against the entries as rspamd matches
|
||||||
|
them. When that stops being good enough is written in
|
||||||
|
[plans/inbound-antispam-panel.md](plans/inbound-antispam-panel.md) § Filter
|
||||||
|
list usage.
|
||||||
**Inbound relay** (`INBOUND_RELAY_ENABLE=true`): accepts only
|
**Inbound relay** (`INBOUND_RELAY_ENABLE=true`): accepts only
|
||||||
`relay_domains` + `relay_recipient_maps` (`reject_unauth_destination`,
|
`relay_domains` + `relay_recipient_maps` (`reject_unauth_destination`,
|
||||||
`reject_unlisted_recipient`). Maps under `/data/postfix/` (`relay_domains`,
|
`reject_unlisted_recipient`). Maps under `/data/postfix/` (`relay_domains`,
|
||||||
|
|||||||
+7
-1
@@ -876,7 +876,13 @@ anyone on the host; they hold addresses, not secrets.
|
|||||||
In the panel the lists are **Inbound › Filter lists** (global role), and what
|
In the panel the lists are **Inbound › Filter lists** (global role), and what
|
||||||
port 25 decided about each message is **Inbound › Log**: accepted mail with the
|
port 25 decided about each message is **Inbound › Log**: accepted mail with the
|
||||||
filter's verdict, refused mail with the reply it was given. A row of the log
|
filter's verdict, refused mail with the reply it was given. A row of the log
|
||||||
leads to the lists with its sender filled in. A user with the *domain* role
|
leads to the lists with its sender filled in. Beside each entry of a list is
|
||||||
|
how often it decided a message, counted from the log - so over the days the
|
||||||
|
log keeps - and *never* marks an entry that may have outlived its use. The
|
||||||
|
count depends on two things in
|
||||||
|
[multimap.conf](../deploy/antispam/rspamd/multimap.conf): the message of the
|
||||||
|
deny rules and the names of the allow symbols; change them and every entry
|
||||||
|
reads *never*. A user with the *domain* role
|
||||||
sees the log of the inbound domains assigned to them, and not the lists. How
|
sees the log of the inbound domains assigned to them, and not the lists. How
|
||||||
long the log keeps a row is set under **Server › Settings**.
|
long the log keeps a row is set under **Server › Settings**.
|
||||||
|
|
||||||
|
|||||||
@@ -97,8 +97,9 @@ changes to it: the sender column is headed *From*; *Why* is not a column but
|
|||||||
a sheet over the page, opened by the decision tag, holding a centred table of
|
a sheet over the page, opened by the decision tag, holding a centred table of
|
||||||
who decided, the verdict and symbols or the reply, and the subject whole
|
who decided, the verdict and symbols or the reply, and the subject whole
|
||||||
(kept on the sheet at his word: the table clips it); a message Postfix itself
|
(kept on the sheet at his word: the table clips it); a message Postfix itself
|
||||||
refused offers no *Deny sender*. Inbound › Filter lists is built and not yet
|
refused offers no *Deny sender*. Inbound › Filter lists was redrawn at his word
|
||||||
accepted.
|
the same day - kind in a column of its own, the note on the entry's sheet,
|
||||||
|
matches counted (§ Filter list usage) - and awaits his look at the result.
|
||||||
|
|
||||||
### Retention
|
### Retention
|
||||||
|
|
||||||
@@ -126,6 +127,46 @@ Mirror send-log retention:
|
|||||||
List change → atomic map rewrite (same pattern as Postfix maps). No reload:
|
List change → atomic map rewrite (same pattern as Postfix maps). No reload:
|
||||||
rspamd rereads a changed map file by itself (about ten seconds on 3.11).
|
rspamd rereads a changed map file by itself (about ten seconds on 3.11).
|
||||||
|
|
||||||
|
### Filter list usage (owner, 2026-10-12)
|
||||||
|
|
||||||
|
Each entry shows how often it decided a message (*Matched*, or *never*) and,
|
||||||
|
on its sheet, when it last did - so that entries nothing matches any more can
|
||||||
|
be removed. **The count is read off the journal on every view of the page;
|
||||||
|
nothing is stored for it.** A journal row counts for an entry when the list
|
||||||
|
decided the message - a rejected row whose reply is the deny rule's message,
|
||||||
|
an accepted row whose verdict carries the allow rule's symbol
|
||||||
|
(`SELFPOST_ALLOW_IP` / `_FROM` / `_DOMAIN`, which the journal milter always
|
||||||
|
keeps) - and the entry fits the row's sender or client address the way rspamd
|
||||||
|
matches it (`inbound.Lists.Usage`).
|
||||||
|
|
||||||
|
What follows from counting this way, and is accepted:
|
||||||
|
|
||||||
|
- the count reaches back as far as the journal does: its retention (90 days by
|
||||||
|
default), which is what the sheet says;
|
||||||
|
- a row counts for every fitting entry of the deciding list, since the journal
|
||||||
|
does not say which of two the filter met first;
|
||||||
|
- a message refused at end-of-message is one row whatever its recipient count,
|
||||||
|
so a count is of messages, roughly.
|
||||||
|
|
||||||
|
**When to move to a counter kept with each entry** (a migration adding hits
|
||||||
|
and last-hit to `inbound_spam_list`, bumped where the journal row is written;
|
||||||
|
the owner asked for the condition to be written down). Any one of:
|
||||||
|
|
||||||
|
1. **The journal is cut by its 10 000-row cap and not by age.** Then it holds
|
||||||
|
fewer days than its retention, the sheet's "in the last 90 days" is no
|
||||||
|
longer true, and *never* stops being evidence that an entry is stale. The
|
||||||
|
sign: the log tailer's prune line keeps reporting rows removed while the
|
||||||
|
oldest row of Inbound › Log is younger than the retention.
|
||||||
|
2. **The retention is set short** (towards its minimum of 7 days) for reasons
|
||||||
|
of its own, privacy or disk. A week without a match does not make an entry
|
||||||
|
stale.
|
||||||
|
3. **Counts are wanted that outlive the journal** - since the entry was added.
|
||||||
|
4. **The page is slow.** A view reads every journal row a list decided and
|
||||||
|
tries it against every entry. With the cap that is at most 10 000 rows
|
||||||
|
times the number of entries; past a few hundred entries, measure it.
|
||||||
|
|
||||||
|
Until then no migration is spent on it.
|
||||||
|
|
||||||
### Activation
|
### Activation
|
||||||
|
|
||||||
| Condition | Behaviour |
|
| Condition | Behaviour |
|
||||||
|
|||||||
@@ -21,6 +21,7 @@ type ListStore interface {
|
|||||||
AddSpamListEntry(list, kind, value, note string) (store.SpamListEntry, error)
|
AddSpamListEntry(list, kind, value, note string) (store.SpamListEntry, error)
|
||||||
ListSpamListEntries() ([]store.SpamListEntry, error)
|
ListSpamListEntries() ([]store.SpamListEntry, error)
|
||||||
DeleteSpamListEntry(id int64) error
|
DeleteSpamListEntry(id int64) error
|
||||||
|
ListSpamListHits(marks ...string) ([]store.SpamListHit, error)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Lists owns the allow and deny lists of the external spam filter: the rows in
|
// Lists owns the allow and deny lists of the external spam filter: the rows in
|
||||||
@@ -236,3 +237,115 @@ func normalizeNote(raw string) (string, error) {
|
|||||||
}
|
}
|
||||||
return note, nil
|
return note, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// What a list leaves in the journal when it decides a message - the two ends
|
||||||
|
// of deploy/antispam/rspamd/multimap.conf this package depends on. A deny rule
|
||||||
|
// refuses with a message, which the log tailer files as the reply of a
|
||||||
|
// rejected row. An allow rule adds a symbol named after the kind of entry,
|
||||||
|
// which the journal milter keeps in the verdict of an accepted row.
|
||||||
|
const (
|
||||||
|
denyReplyMark = "this server's deny list"
|
||||||
|
allowMark = "SELFPOST_ALLOW_"
|
||||||
|
)
|
||||||
|
|
||||||
|
var allowSymbol = map[string]string{
|
||||||
|
store.SpamEntryIP: "SELFPOST_ALLOW_IP(",
|
||||||
|
store.SpamEntryAddress: "SELFPOST_ALLOW_FROM(",
|
||||||
|
store.SpamEntryDomain: "SELFPOST_ALLOW_DOMAIN(",
|
||||||
|
}
|
||||||
|
|
||||||
|
// Usage is how often an entry decided a message, as far back as the journal
|
||||||
|
// reaches, and when it last did. The zero value is an entry that never did.
|
||||||
|
type Usage struct {
|
||||||
|
Hits int
|
||||||
|
Last time.Time
|
||||||
|
}
|
||||||
|
|
||||||
|
// Usage counts, for every entry, the journal rows it decided. Nothing is
|
||||||
|
// stored for this: the journal already says which list decided a message and
|
||||||
|
// who the message was from, and an entry is matched against that the way the
|
||||||
|
// filter matches it - an address whole, a domain as the sender's domain, an
|
||||||
|
// address or network against the client's. So the count reaches back as far
|
||||||
|
// as the journal does (its retention, or less when it is full), and a row
|
||||||
|
// counts for every entry of the deciding list that fits it: the journal does
|
||||||
|
// not say which of two fitting entries the filter met first.
|
||||||
|
//
|
||||||
|
// A counter kept with each entry would not depend on the journal; the plan
|
||||||
|
// (docs/plans/inbound-antispam-panel.md, Filter list usage) says when that is
|
||||||
|
// worth a migration.
|
||||||
|
func (l *Lists) Usage() (map[int64]Usage, error) {
|
||||||
|
entries, err := l.store.ListSpamListEntries()
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
out := make(map[int64]Usage, len(entries))
|
||||||
|
if len(entries) == 0 {
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
hits, err := l.store.ListSpamListHits(denyReplyMark, allowMark)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
type matcher struct {
|
||||||
|
id int64
|
||||||
|
list string
|
||||||
|
kind string
|
||||||
|
value string
|
||||||
|
prefix netip.Prefix
|
||||||
|
}
|
||||||
|
matchers := make([]matcher, 0, len(entries))
|
||||||
|
for _, e := range entries {
|
||||||
|
m := matcher{id: e.ID, list: e.List, kind: e.Kind, value: e.Value}
|
||||||
|
if e.Kind == store.SpamEntryIP {
|
||||||
|
if p, err := netip.ParsePrefix(e.Value); err == nil {
|
||||||
|
m.prefix = p
|
||||||
|
} else if a, err := netip.ParseAddr(e.Value); err == nil {
|
||||||
|
m.prefix = netip.PrefixFrom(a, a.BitLen())
|
||||||
|
} else {
|
||||||
|
continue // not an entry this package wrote
|
||||||
|
}
|
||||||
|
}
|
||||||
|
matchers = append(matchers, m)
|
||||||
|
}
|
||||||
|
for _, h := range hits {
|
||||||
|
from := strings.ToLower(h.From)
|
||||||
|
domain := ""
|
||||||
|
if i := strings.LastIndexByte(from, '@'); i >= 0 {
|
||||||
|
domain = from[i+1:]
|
||||||
|
}
|
||||||
|
client, clientErr := netip.ParseAddr(h.ClientIP)
|
||||||
|
client = client.Unmap()
|
||||||
|
denied := h.Decision != store.SpamDecisionAccept && strings.Contains(h.Detail, denyReplyMark)
|
||||||
|
for _, m := range matchers {
|
||||||
|
switch m.list {
|
||||||
|
case store.SpamListDeny:
|
||||||
|
if !denied {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
case store.SpamListAllow:
|
||||||
|
if h.Decision != store.SpamDecisionAccept || !strings.Contains(h.Detail, allowSymbol[m.kind]) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
}
|
||||||
|
fits := false
|
||||||
|
switch m.kind {
|
||||||
|
case store.SpamEntryAddress:
|
||||||
|
fits = from == m.value
|
||||||
|
case store.SpamEntryDomain:
|
||||||
|
fits = domain == m.value
|
||||||
|
case store.SpamEntryIP:
|
||||||
|
fits = clientErr == nil && m.prefix.Contains(client)
|
||||||
|
}
|
||||||
|
if !fits {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
u := out[m.id]
|
||||||
|
u.Hits++
|
||||||
|
if h.CreatedAt.After(u.Last) {
|
||||||
|
u.Last = h.CreatedAt
|
||||||
|
}
|
||||||
|
out[m.id] = u
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|||||||
@@ -276,3 +276,95 @@ func TestListsLastWritten(t *testing.T) {
|
|||||||
t.Errorf("LastWritten counts a stray file: %v", got)
|
t.Errorf("LastWritten counts a stray file: %v", got)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// How often an entry decided a message is read off the journal: the list that
|
||||||
|
// decided, and whether the entry fits who the message was from.
|
||||||
|
func TestListsUsage(t *testing.T) {
|
||||||
|
l, st, _ := testLists(t)
|
||||||
|
add := func(list, kind, value string) int64 {
|
||||||
|
t.Helper()
|
||||||
|
e, err := l.Add(list, kind, value, "")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Add(%s, %s, %s): %v", list, kind, value, err)
|
||||||
|
}
|
||||||
|
return e.ID
|
||||||
|
}
|
||||||
|
denyDomain := add(store.SpamListDeny, store.SpamEntryDomain, "evil.example")
|
||||||
|
denyAddr := add(store.SpamListDeny, store.SpamEntryAddress, "spammer@evil.example")
|
||||||
|
denyNet := add(store.SpamListDeny, store.SpamEntryIP, "198.51.100.0/24")
|
||||||
|
denyV6 := add(store.SpamListDeny, store.SpamEntryIP, "2001:db8::/32")
|
||||||
|
allowAddr := add(store.SpamListAllow, store.SpamEntryAddress, "billing@vendor.example")
|
||||||
|
allowDomain := add(store.SpamListAllow, store.SpamEntryDomain, "vendor.example")
|
||||||
|
allowIP := add(store.SpamListAllow, store.SpamEntryIP, "203.0.113.5")
|
||||||
|
unused := add(store.SpamListAllow, store.SpamEntryDomain, "quiet.example")
|
||||||
|
|
||||||
|
const denied = "5.7.1 Refused by this server's deny list"
|
||||||
|
at := time.Date(2026, 10, 1, 0, 0, 0, 0, time.UTC)
|
||||||
|
hour := 0
|
||||||
|
row := func(from, ip, decision, engine, detail string) {
|
||||||
|
t.Helper()
|
||||||
|
hour++
|
||||||
|
err := st.InsertInboundSpam(store.InboundSpamEntry{
|
||||||
|
InboundDomain: "in.example.com", To: "a@in.example.com", From: from, ClientIP: ip,
|
||||||
|
Decision: decision, Engine: engine, Detail: detail, At: at.Add(time.Duration(hour) * time.Hour),
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// hour 1: the address and its domain are both on the deny list, and both fit.
|
||||||
|
row("Spammer@Evil.Example", "192.0.2.9", store.SpamDecisionReject, store.SpamEngineRspamd, denied)
|
||||||
|
// hour 2: another address of the domain.
|
||||||
|
row("other@evil.example", "192.0.2.9", store.SpamDecisionReject, store.SpamEngineRspamd, denied)
|
||||||
|
// hour 3, 4: the networks, v4 (as Postfix may log it, mapped) and v6.
|
||||||
|
row("x@elsewhere.example", "::ffff:198.51.100.77", store.SpamDecisionReject, store.SpamEngineRspamd, denied)
|
||||||
|
row("x@elsewhere.example", "2001:db8::1", store.SpamDecisionReject, store.SpamEngineRspamd, denied)
|
||||||
|
// hour 5: a subdomain is not the domain.
|
||||||
|
row("x@sub.evil.example", "192.0.2.9", store.SpamDecisionReject, store.SpamEngineRspamd, denied)
|
||||||
|
// hour 6: allowed by its address and its domain - two symbols, two entries.
|
||||||
|
row("billing@vendor.example", "192.0.2.50", store.SpamDecisionAccept, store.SpamEngineRspamd,
|
||||||
|
"no action, score -199.00 / 15.00; SELFPOST_ALLOW_DOMAIN(-100.00) SELFPOST_ALLOW_FROM(-100.00)")
|
||||||
|
// hour 7: allowed by its domain alone.
|
||||||
|
row("accounts@vendor.example", "192.0.2.50", store.SpamDecisionAccept, store.SpamEngineRspamd,
|
||||||
|
"no action, score -99.00 / 15.00; SELFPOST_ALLOW_DOMAIN(-100.00)")
|
||||||
|
// hour 8: allowed by the client address; the sender's domain is not why.
|
||||||
|
row("x@quiet.example", "203.0.113.5", store.SpamDecisionAccept, store.SpamEngineRspamd,
|
||||||
|
"no action, score -99.00 / 15.00; SELFPOST_ALLOW_IP(-100.00)")
|
||||||
|
// Nobody's: the filter's own refusal, an accept with no list symbol, a
|
||||||
|
// refusal by Postfix, and a sender who spells the mark in his own address.
|
||||||
|
row("spammer@evil.example", "192.0.2.9", store.SpamDecisionReject, store.SpamEngineRspamd, "5.7.1 Spam message rejected")
|
||||||
|
row("billing@vendor.example", "192.0.2.50", store.SpamDecisionAccept, store.SpamEngineRspamd, "no action, score 0.10 / 15.00")
|
||||||
|
row("spammer@evil.example", "198.51.100.1", store.SpamDecisionReject, store.SpamEnginePostfix, "550 5.1.1 <nobody@in.example.com>: this server's deny list")
|
||||||
|
row("billing@vendor.example", "203.0.113.5", store.SpamDecisionAccept, store.SpamEngineSelfPost, "")
|
||||||
|
|
||||||
|
usage, err := l.Usage()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Usage: %v", err)
|
||||||
|
}
|
||||||
|
for name, c := range map[string]struct {
|
||||||
|
id int64
|
||||||
|
hits int
|
||||||
|
last int // hour
|
||||||
|
}{
|
||||||
|
"deny domain": {denyDomain, 2, 2}, "deny address": {denyAddr, 1, 1},
|
||||||
|
"deny network": {denyNet, 1, 3}, "deny v6 network": {denyV6, 1, 4},
|
||||||
|
"allow address": {allowAddr, 1, 6}, "allow domain": {allowDomain, 2, 7}, "allow ip": {allowIP, 1, 8},
|
||||||
|
"unused": {unused, 0, 0},
|
||||||
|
} {
|
||||||
|
got := usage[c.id]
|
||||||
|
wantLast := time.Time{}
|
||||||
|
if c.hits > 0 {
|
||||||
|
wantLast = at.Add(time.Duration(c.last) * time.Hour)
|
||||||
|
}
|
||||||
|
if got.Hits != c.hits || !got.Last.Equal(wantLast) {
|
||||||
|
t.Errorf("%s: %d hits, last %v; want %d, %v", name, got.Hits, got.Last, c.hits, wantLast)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestListsUsageWithNoEntries(t *testing.T) {
|
||||||
|
l, _, _ := testLists(t)
|
||||||
|
if usage, err := l.Usage(); err != nil || len(usage) != 0 {
|
||||||
|
t.Errorf("Usage = %v, %v", usage, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -142,12 +142,16 @@ func (s *inboundSession) record() {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// What the journal keeps of a verdict: enough to see why, on one table row.
|
// What the journal keeps of a verdict: enough to see why, on one line.
|
||||||
const (
|
const (
|
||||||
verdictMaxSymbols = 6
|
verdictMaxSymbols = 6
|
||||||
verdictMaxBytes = 255
|
verdictMaxBytes = 255
|
||||||
)
|
)
|
||||||
|
|
||||||
|
// ListSymbolPrefix starts the names of the symbols the filter lists' rules add
|
||||||
|
// (deploy/antispam/rspamd/multimap.conf): SELFPOST_ALLOW_IP, _FROM, _DOMAIN.
|
||||||
|
const ListSymbolPrefix = "SELFPOST_"
|
||||||
|
|
||||||
var (
|
var (
|
||||||
// "default: False [7.20 / 15.00]" - the score and the reject threshold.
|
// "default: False [7.20 / 15.00]" - the score and the reject threshold.
|
||||||
rspamdScore = regexp.MustCompile(`\[(-?[0-9]+(?:\.[0-9]+)?) / (-?[0-9]+(?:\.[0-9]+)?)\]`)
|
rspamdScore = regexp.MustCompile(`\[(-?[0-9]+(?:\.[0-9]+)?) / (-?[0-9]+(?:\.[0-9]+)?)\]`)
|
||||||
@@ -182,15 +186,28 @@ func rspamdVerdict(actionHeader, resultHeader string) (action, detail string) {
|
|||||||
}
|
}
|
||||||
detail = strings.Join(parts, ", ")
|
detail = strings.Join(parts, ", ")
|
||||||
|
|
||||||
var symbols []string
|
// The filter lists' own symbols come first and are kept whatever else there
|
||||||
|
// is: Filter lists counts how often an entry decided by finding them here,
|
||||||
|
// and rspamd writes symbols heaviest first, which puts an allow entry's
|
||||||
|
// -100 last, behind any six others.
|
||||||
|
var symbols, rest []string
|
||||||
for _, m := range rspamdSymbol.FindAllStringSubmatch(resultHeader, -1) {
|
for _, m := range rspamdSymbol.FindAllStringSubmatch(resultHeader, -1) {
|
||||||
if strings.Trim(m[2], "-0.") == "" { // weight zero: informational
|
if strings.Trim(m[2], "-0.") == "" { // weight zero: informational
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
symbols = append(symbols, m[1]+"("+m[2]+")")
|
if strings.HasPrefix(m[1], ListSymbolPrefix) {
|
||||||
|
if len(symbols) < verdictMaxSymbols {
|
||||||
|
symbols = append(symbols, m[1]+"("+m[2]+")")
|
||||||
|
}
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
rest = append(rest, m[1]+"("+m[2]+")")
|
||||||
|
}
|
||||||
|
for _, sym := range rest {
|
||||||
if len(symbols) == verdictMaxSymbols {
|
if len(symbols) == verdictMaxSymbols {
|
||||||
break
|
break
|
||||||
}
|
}
|
||||||
|
symbols = append(symbols, sym)
|
||||||
}
|
}
|
||||||
if len(symbols) > 0 {
|
if len(symbols) > 0 {
|
||||||
if detail != "" {
|
if detail != "" {
|
||||||
|
|||||||
@@ -222,3 +222,15 @@ func TestInboundAcceptsWhenTheStoreFails(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// rspamd writes symbols heaviest first, so an allow entry's -100 comes last.
|
||||||
|
// Filter lists counts an entry's matches by finding that symbol in the journal:
|
||||||
|
// it is kept whatever else there is.
|
||||||
|
func TestRspamdVerdictKeepsTheListSymbols(t *testing.T) {
|
||||||
|
result := "default: False [-91.00 / 15.00]; SYM_A(3.00)[]; SYM_B(2.00)[]; SYM_C(1.50)[]; SYM_D(1.00)[]; SYM_E(0.80)[]; SYM_F(0.50)[]; SYM_G(0.20)[]; ARC_NA(0.00)[]; SELFPOST_ALLOW_FROM(-100.00)[billing@vendor.example]"
|
||||||
|
_, detail := rspamdVerdict("no action", result)
|
||||||
|
want := "no action, score -91.00 / 15.00; SELFPOST_ALLOW_FROM(-100.00) SYM_A(3.00) SYM_B(2.00) SYM_C(1.50) SYM_D(1.00) SYM_E(0.80)"
|
||||||
|
if detail != want {
|
||||||
|
t.Errorf("detail = %q\nwant %q", detail, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -304,3 +304,48 @@ func (s *Store) DeleteSpamListEntry(id int64) error {
|
|||||||
}
|
}
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// SpamListHit is what the journal holds about one message a filter list
|
||||||
|
// decided: who it was from, which server offered it, when, and the detail that
|
||||||
|
// says which list it was.
|
||||||
|
type SpamListHit struct {
|
||||||
|
ClientIP string
|
||||||
|
From string
|
||||||
|
Decision string
|
||||||
|
Detail string
|
||||||
|
CreatedAt time.Time
|
||||||
|
}
|
||||||
|
|
||||||
|
// ListSpamListHits returns the journal rows of the filter in whose detail one
|
||||||
|
// of the marks occurs - the marks the filter lists leave (internal/inbound).
|
||||||
|
// Which entry a row belongs to is for the caller to work out; here the rows
|
||||||
|
// are only narrowed, so that a page view does not read the whole journal.
|
||||||
|
func (s *Store) ListSpamListHits(marks ...string) ([]SpamListHit, error) {
|
||||||
|
if len(marks) == 0 {
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
likes := make([]string, len(marks))
|
||||||
|
args := []any{SpamEngineRspamd}
|
||||||
|
for i, m := range marks {
|
||||||
|
likes[i] = "instr(detail, ?) > 0"
|
||||||
|
args = append(args, m)
|
||||||
|
}
|
||||||
|
rows, err := s.db.Query(
|
||||||
|
`SELECT client_ip, from_addr, decision, detail, created_at FROM inbound_spam_log
|
||||||
|
WHERE engine = ? AND (`+strings.Join(likes, " OR ")+`)`, args...)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("query filter list hits: %w", err)
|
||||||
|
}
|
||||||
|
defer rows.Close()
|
||||||
|
var out []SpamListHit
|
||||||
|
for rows.Next() {
|
||||||
|
var h SpamListHit
|
||||||
|
var created string
|
||||||
|
if err := rows.Scan(&h.ClientIP, &h.From, &h.Decision, &h.Detail, &created); err != nil {
|
||||||
|
return nil, fmt.Errorf("scan filter list hit: %w", err)
|
||||||
|
}
|
||||||
|
h.CreatedAt, _ = time.Parse(time.RFC3339, created)
|
||||||
|
out = append(out, h)
|
||||||
|
}
|
||||||
|
return out, rows.Err()
|
||||||
|
}
|
||||||
|
|||||||
@@ -252,3 +252,35 @@ func TestQueryInboundSpamFilters(t *testing.T) {
|
|||||||
t.Errorf("injected domain = %q", got)
|
t.Errorf("injected domain = %q", got)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestListSpamListHits(t *testing.T) {
|
||||||
|
st := openTestStore(t)
|
||||||
|
add := func(engine, decision, detail string) {
|
||||||
|
t.Helper()
|
||||||
|
err := st.InsertInboundSpam(InboundSpamEntry{
|
||||||
|
InboundDomain: "in.example.com", ClientIP: "203.0.113.7", From: "a@example.net", To: "a@in.example.com",
|
||||||
|
Decision: decision, Engine: engine, Detail: detail, At: time.Date(2026, 10, 1, 9, 0, 0, 0, time.UTC),
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
add(SpamEngineRspamd, SpamDecisionReject, "5.7.1 Refused by this server's deny list")
|
||||||
|
add(SpamEngineRspamd, SpamDecisionAccept, "no action, score -99.00 / 15.00; SELFPOST_ALLOW_IP(-100.00)")
|
||||||
|
add(SpamEngineRspamd, SpamDecisionReject, "5.7.1 Spam message rejected")
|
||||||
|
add(SpamEnginePostfix, SpamDecisionReject, "550 5.1.1 this server's deny list")
|
||||||
|
// a mark with the characters LIKE would treat as wildcards is text
|
||||||
|
add(SpamEngineRspamd, SpamDecisionAccept, "no action; SELFPOSTXALLOWX(1.00)")
|
||||||
|
|
||||||
|
hits, err := st.ListSpamListHits("this server's deny list", "SELFPOST_ALLOW_")
|
||||||
|
if err != nil || len(hits) != 2 {
|
||||||
|
t.Fatalf("hits = %+v, %v; want the two rows a list decided", hits, err)
|
||||||
|
}
|
||||||
|
h := hits[0]
|
||||||
|
if h.From != "a@example.net" || h.ClientIP != "203.0.113.7" || h.Decision != SpamDecisionReject || h.CreatedAt.Hour() != 9 {
|
||||||
|
t.Errorf("hit = %+v", h)
|
||||||
|
}
|
||||||
|
if hits, err := st.ListSpamListHits(); err != nil || hits != nil {
|
||||||
|
t.Errorf("no marks: %v, %v", hits, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -167,12 +167,21 @@ func (h *Handlers) renderFilterLists(w http.ResponseWriter, r *http.Request, sta
|
|||||||
http.Error(w, "internal error", http.StatusInternalServerError)
|
http.Error(w, "internal error", http.StatusInternalServerError)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
// How often each entry decided a message is counted from the journal. A
|
||||||
|
// failure there is not a reason to hide the lists: they are shown without
|
||||||
|
// the counts' worth - every entry "never" - and the log says why.
|
||||||
|
usage, err := h.lists.Usage()
|
||||||
|
if err != nil {
|
||||||
|
logf("panel: filter lists: usage: %v", err)
|
||||||
|
}
|
||||||
on, _, _ := h.inboundFilter()
|
on, _, _ := h.inboundFilter()
|
||||||
in := make([]view.FilterEntryInput, len(entries))
|
in := make([]view.FilterEntryInput, len(entries))
|
||||||
for i, e := range entries {
|
for i, e := range entries {
|
||||||
in[i] = view.FilterEntryInput{ID: e.ID, List: e.List, Kind: e.Kind, Value: e.Value, Note: e.Note, Added: e.CreatedAt}
|
in[i] = view.FilterEntryInput{ID: e.ID, List: e.List, Kind: e.Kind, Value: e.Value, Note: e.Note, Added: e.CreatedAt,
|
||||||
|
Hits: usage[e.ID].Hits, LastHit: usage[e.ID].Last}
|
||||||
}
|
}
|
||||||
page := view.NewFilterLists(h.shellMeta(r), view.FilterListsInput{FilterOn: on, LastWritten: h.lists.LastWritten()}).
|
page := view.NewFilterLists(h.shellMeta(r), view.FilterListsInput{
|
||||||
|
FilterOn: on, LastWritten: h.lists.LastWritten(), RetentionDays: h.inboundSpamLogRetentionDays()}).
|
||||||
WithEntries(in).
|
WithEntries(in).
|
||||||
WithResult(flash, formErr)
|
WithResult(flash, formErr)
|
||||||
// A blank list or kind leaves the form on its first choice.
|
// A blank list or kind leaves the form on its first choice.
|
||||||
|
|||||||
@@ -364,9 +364,10 @@ func TestFilterListsAddAndRemove(t *testing.T) {
|
|||||||
has(t, "lists", page,
|
has(t, "lists", page,
|
||||||
`<div class="notification is-success is-light">`, `Entry added.`,
|
`<div class="notification is-success is-light">`, `Entry added.`,
|
||||||
`<h2>Deny</h2><div class="sp-end">1 entry · wins over Allow</div>`, `<h2>Allow</h2><div class="sp-end">1 entry</div>`,
|
`<h2>Deny</h2><div class="sp-end">1 entry · wins over Allow</div>`, `<h2>Allow</h2><div class="sp-end">1 entry</div>`,
|
||||||
`bulk.example <span class="tag is-light">domain</span></td><td class="sp-muted">newsletter</td>`,
|
`<td class="sp-mono sp-small">bulk.example</td><td><button type="button" class="tag is-light" popovertarget="entry-`,
|
||||||
`198.51.100.0/24 <span class="tag is-light">ip</span></td><td class="sp-muted">—</td>`,
|
`<tr><th>Note</th><td>newsletter</td></tr><tr><th>Added</th><td class="sp-mono sp-small">`+time.Now().UTC().Format("2006-01-02")+`</td></tr><tr><th>Matched</th><td>never in the last 90 days</td></tr>`,
|
||||||
`<td class="sp-muted sp-nowrap sp-small">`+time.Now().UTC().Format("2006-01-02")+`</td>`,
|
`<td class="sp-mono sp-small">198.51.100.0/24</td>`, `<tr><th>Note</th><td>—</td></tr>`,
|
||||||
|
`<td class="sp-muted sp-nowrap">never</td>`,
|
||||||
)
|
)
|
||||||
lacks(t, "lists", page, `No sender is turned away`)
|
lacks(t, "lists", page, `No sender is turned away`)
|
||||||
|
|
||||||
@@ -635,3 +636,39 @@ func TestHelpHasTheInboundLogAndFilterListsTopics(t *testing.T) {
|
|||||||
has(t, "log", s.get(h.HandleInboundLog, "/inbound/log", globalPrincipal).Body.String(), `href="/help#inbound-log"`)
|
has(t, "log", s.get(h.HandleInboundLog, "/inbound/log", globalPrincipal).Body.String(), `href="/help#inbound-log"`)
|
||||||
has(t, "lists", s.get(h.HandleFilterLists, "/inbound/filter-lists", globalPrincipal).Body.String(), `href="/help#filter-lists"`)
|
has(t, "lists", s.get(h.HandleFilterLists, "/inbound/filter-lists", globalPrincipal).Body.String(), `href="/help#filter-lists"`)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// An entry's count comes from the journal: a row counts for the entry that
|
||||||
|
// fits it on the list that decided it.
|
||||||
|
func TestFilterListsCountMatchesFromTheJournal(t *testing.T) {
|
||||||
|
s := newInboundLogStand(t)
|
||||||
|
for _, e := range [][3]string{{"deny", "domain", "bulk.example"}, {"allow", "address", "billing@vendor.example"}, {"deny", "ip", "203.0.113.0/24"}} {
|
||||||
|
if _, err := s.h.lists.Add(e[0], e[1], e[2], ""); err != nil {
|
||||||
|
t.Fatalf("Add %v: %v", e, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
at := time.Date(2026, 10, 1, 9, 30, 0, 0, time.UTC)
|
||||||
|
for i, row := range []store.InboundSpamEntry{
|
||||||
|
{From: "promo@bulk.example", ClientIP: "198.51.100.9", Decision: store.SpamDecisionReject, Engine: store.SpamEngineRspamd, Detail: "5.7.1 Refused by this server's deny list"},
|
||||||
|
{From: "other@BULK.example", ClientIP: "198.51.100.9", Decision: store.SpamDecisionReject, Engine: store.SpamEngineRspamd, Detail: "5.7.1 Refused by this server's deny list"},
|
||||||
|
{From: "billing@vendor.example", ClientIP: "192.0.2.5", Decision: store.SpamDecisionAccept, Engine: store.SpamEngineRspamd, Detail: "no action, score -99.00 / 15.00; SELFPOST_ALLOW_FROM(-100.00)"},
|
||||||
|
// refused by the filter for its own reasons: nobody's match
|
||||||
|
{From: "promo@bulk.example", ClientIP: "198.51.100.9", Decision: store.SpamDecisionReject, Engine: store.SpamEngineRspamd, Detail: "5.7.1 Spam message rejected"},
|
||||||
|
} {
|
||||||
|
row.InboundDomain, row.To, row.At = "a.example.com", "alice@a.example.com", at.Add(time.Duration(i)*time.Hour)
|
||||||
|
if err := s.st.InsertInboundSpam(row); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
page := s.get(s.h.HandleFilterLists, "/inbound/filter-lists", globalPrincipal).Body.String()
|
||||||
|
has(t, "lists", page,
|
||||||
|
`<tr><th>Matched</th><td>2 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-10-01 10:30</td></tr>`,
|
||||||
|
`<td class="sp-nowrap">2</td>`,
|
||||||
|
`<tr><th>Matched</th><td>1 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-10-01 11:30</td></tr>`,
|
||||||
|
`<td class="sp-nowrap">1</td>`,
|
||||||
|
// the network nobody came from
|
||||||
|
`<td class="sp-muted sp-nowrap">never</td>`,
|
||||||
|
)
|
||||||
|
if got := strings.Count(page, `<td class="sp-muted sp-nowrap">never</td>`); got != 1 {
|
||||||
|
t.Errorf("%d entries say never, want the one network", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -450,12 +450,13 @@ func inLogFixture() *InLog {
|
|||||||
// allowed, and a filter that reads them.
|
// allowed, and a filter that reads them.
|
||||||
func inFilterListsFixture() *FilterLists {
|
func inFilterListsFixture() *FilterLists {
|
||||||
day := func(m time.Month, d int) time.Time { return time.Date(2026, m, d, 0, 0, 0, 0, time.UTC) }
|
day := func(m time.Month, d int) time.Time { return time.Date(2026, m, d, 0, 0, 0, 0, time.UTC) }
|
||||||
return NewFilterLists(admin(), FilterListsInput{FilterOn: true, LastWritten: time.Date(2026, 9, 11, 8, 40, 0, 0, time.UTC)}).
|
at := func(d, h, m int) time.Time { return time.Date(2026, 9, d, h, m, 0, 0, time.UTC) }
|
||||||
|
return NewFilterLists(admin(), FilterListsInput{FilterOn: true, LastWritten: time.Date(2026, 9, 11, 8, 40, 0, 0, time.UTC), RetentionDays: 90}).
|
||||||
WithEntries([]FilterEntryInput{
|
WithEntries([]FilterEntryInput{
|
||||||
{ID: 1, List: "deny", Kind: "domain", Value: "bulk.example", Note: "newsletter nobody subscribed to", Added: day(9, 2)},
|
{ID: 1, List: "deny", Kind: "domain", Value: "bulk.example", Note: "newsletter nobody subscribed to", Added: day(9, 2), Hits: 41, LastHit: at(21, 13, 57)},
|
||||||
{ID: 2, List: "deny", Kind: "ip", Value: "198.51.100.0/24", Note: "dictionary attack on lists.example.org", Added: day(8, 19)},
|
{ID: 2, List: "deny", Kind: "ip", Value: "198.51.100.0/24", Note: "dictionary attack on lists.example.org", Added: day(8, 19), Hits: 1203, LastHit: at(21, 9, 12)},
|
||||||
{ID: 3, List: "deny", Kind: "address", Value: "win@lottery.example", Added: day(7, 30)},
|
{ID: 3, List: "deny", Kind: "address", Value: "win@lottery.example", Added: day(7, 30)},
|
||||||
{ID: 4, List: "allow", Kind: "domain", Value: "partner.example", Note: "invoices were scored as spam", Added: day(9, 11)},
|
{ID: 4, List: "allow", Kind: "domain", Value: "partner.example", Note: "invoices were scored as spam", Added: day(9, 11), Hits: 7, LastHit: at(19, 16, 40)},
|
||||||
{ID: 5, List: "allow", Kind: "address", Value: "noreply@github.example", Note: "security advisories", Added: day(6, 4)},
|
{ID: 5, List: "allow", Kind: "address", Value: "noreply@github.example", Note: "security advisories", Added: day(6, 4), Hits: 18, LastHit: at(21, 13, 30)},
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -205,7 +205,7 @@ var inboundTopics = []HelpTopic{
|
|||||||
Strong("Deny wins"), ".",
|
Strong("Deny wins"), ".",
|
||||||
Br(), Br(),
|
Br(), Br(),
|
||||||
"SelfPost does not apply the lists itself: it writes them as files under ", Code("/data/antispam/"),
|
"SelfPost does not apply the lists itself: it writes them as files under ", Code("/data/antispam/"),
|
||||||
", and rspamd rereads a file within seconds of a change. With the spam filter off the lists are kept and can be edited, but nothing reads them. The sender matched is the envelope sender — the one in the log — not the From line a mail program shows."),
|
", and rspamd rereads a file within seconds of a change. With the spam filter off the lists are kept and can be edited, but nothing reads them. The sender matched is the envelope sender — the one in the log — not the From line a mail program shows. Matched counts the messages in the Inbound log that the entry decided, so it reaches back as far as the log does; never means not once in that time, and such an entry is one to consider removing."),
|
||||||
}},
|
}},
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -46,6 +46,8 @@ type FilterLists struct {
|
|||||||
|
|
||||||
Where Box
|
Where Box
|
||||||
WhereFacts []Fact
|
WhereFacts []Fact
|
||||||
|
|
||||||
|
retentionDays int
|
||||||
}
|
}
|
||||||
|
|
||||||
// FilterList is one of the two boxes of entries.
|
// FilterList is one of the two boxes of entries.
|
||||||
@@ -55,24 +57,29 @@ type FilterList struct {
|
|||||||
Empty EmptyState
|
Empty EmptyState
|
||||||
}
|
}
|
||||||
|
|
||||||
// FilterRow is one entry of a list: the value with the kind of entry as a tag,
|
// FilterRow is one entry of a list: the value, the kind of entry as a tag that
|
||||||
// the operator's note, the day it was added and the way to remove it.
|
// opens the entry's sheet (the note, the day it was added, how often it decided
|
||||||
|
// a message and when last), how often it decided, and the way to remove it.
|
||||||
|
// Matched is empty for an entry that never did; the table then says "never".
|
||||||
type FilterRow struct {
|
type FilterRow struct {
|
||||||
Value string
|
Value string
|
||||||
Kind Tag
|
Entry Sheet
|
||||||
Note string
|
Matched string
|
||||||
Added string
|
Remove RowAction
|
||||||
Remove RowAction
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// FilterEntryInput is what the page knows of one entry of a list.
|
// FilterEntryInput is what the page knows of one entry of a list. Hits and
|
||||||
|
// LastHit are counted from the Inbound log, so they reach back as far as it
|
||||||
|
// does.
|
||||||
type FilterEntryInput struct {
|
type FilterEntryInput struct {
|
||||||
ID int64
|
ID int64
|
||||||
List string
|
List string
|
||||||
Kind string
|
Kind string
|
||||||
Value string
|
Value string
|
||||||
Note string
|
Note string
|
||||||
Added time.Time
|
Added time.Time
|
||||||
|
Hits int
|
||||||
|
LastHit time.Time
|
||||||
}
|
}
|
||||||
|
|
||||||
// FilterListsInput is the state of the filter the lists are written for: whether
|
// FilterListsInput is the state of the filter the lists are written for: whether
|
||||||
@@ -80,6 +87,9 @@ type FilterEntryInput struct {
|
|||||||
type FilterListsInput struct {
|
type FilterListsInput struct {
|
||||||
FilterOn bool
|
FilterOn bool
|
||||||
LastWritten time.Time
|
LastWritten time.Time
|
||||||
|
// RetentionDays is how long the Inbound log keeps a row: how far back an
|
||||||
|
// entry's matches are counted.
|
||||||
|
RetentionDays int
|
||||||
}
|
}
|
||||||
|
|
||||||
// NewFilterLists builds the page. The form starts on the Deny list for a sender
|
// NewFilterLists builds the page. The form starts on the Deny list for a sender
|
||||||
@@ -113,6 +123,7 @@ func NewFilterLists(m Meta, in FilterListsInput) *FilterLists {
|
|||||||
}
|
}
|
||||||
p.Deny.Box.End = Plain(plural(0, "entry", "entries") + " · wins over Allow")
|
p.Deny.Box.End = Plain(plural(0, "entry", "entries") + " · wins over Allow")
|
||||||
p.Allow.Box.End = Plain(plural(0, "entry", "entries"))
|
p.Allow.Box.End = Plain(plural(0, "entry", "entries"))
|
||||||
|
p.retentionDays = in.RetentionDays
|
||||||
p.setWhere(in)
|
p.setWhere(in)
|
||||||
return p
|
return p
|
||||||
}
|
}
|
||||||
@@ -144,10 +155,25 @@ func (p *FilterLists) setWhere(in FilterListsInput) {
|
|||||||
func (p *FilterLists) WithEntries(entries []FilterEntryInput) *FilterLists {
|
func (p *FilterLists) WithEntries(entries []FilterEntryInput) *FilterLists {
|
||||||
p.Deny.Rows, p.Allow.Rows = nil, nil
|
p.Deny.Rows, p.Allow.Rows = nil, nil
|
||||||
for _, e := range entries {
|
for _, e := range entries {
|
||||||
row := FilterRow{
|
id := strconv.FormatInt(e.ID, 10)
|
||||||
Value: e.Value, Kind: Tag{Label: e.Kind}, Note: e.Note, Added: e.Added.UTC().Format("2006-01-02"),
|
window := "in the last " + plural(int64(p.retentionDays), "day", "days")
|
||||||
Remove: RowAction{Label: "Remove", Post: inboundFilterLists + "/" + strconv.FormatInt(e.ID, 10) + "/delete", Danger: true},
|
sheet := []SheetRow{
|
||||||
|
{Label: "Note", Lines: []string{orDash(e.Note)}},
|
||||||
|
{Label: "Added", Lines: []string{e.Added.UTC().Format("2006-01-02")}, Mono: true},
|
||||||
}
|
}
|
||||||
|
row := FilterRow{
|
||||||
|
Value: e.Value,
|
||||||
|
Remove: RowAction{Label: "Remove", Post: inboundFilterLists + "/" + id + "/delete", Danger: true},
|
||||||
|
}
|
||||||
|
if e.Hits > 0 {
|
||||||
|
row.Matched = strconv.Itoa(e.Hits)
|
||||||
|
sheet = append(sheet,
|
||||||
|
SheetRow{Label: "Matched", Lines: []string{row.Matched + " " + window}},
|
||||||
|
SheetRow{Label: "Last match", Lines: []string{e.LastHit.UTC().Format("2006-01-02 15:04")}, Mono: true})
|
||||||
|
} else {
|
||||||
|
sheet = append(sheet, SheetRow{Label: "Matched", Lines: []string{"never " + window}})
|
||||||
|
}
|
||||||
|
row.Entry = Sheet{ID: "entry-" + id, Tag: Tag{Label: e.Kind}, Title: e.Value, Rows: sheet}
|
||||||
switch e.List {
|
switch e.List {
|
||||||
case "deny":
|
case "deny":
|
||||||
p.Deny.Rows = append(p.Deny.Rows, row)
|
p.Deny.Rows = append(p.Deny.Rows, row)
|
||||||
|
|||||||
@@ -183,18 +183,27 @@ func TestFilterListsShowsTheFormTheListsAndWhereTheyGo(t *testing.T) {
|
|||||||
`href="/help#filter-lists" title="How the lists are applied"`,
|
`href="/help#filter-lists" title="How the lists are applied"`,
|
||||||
`<h2>Deny</h2><div class="sp-end">3 entries · wins over Allow</div>`,
|
`<h2>Deny</h2><div class="sp-end">3 entries · wins over Allow</div>`,
|
||||||
`<h2>Allow</h2><div class="sp-end">2 entries</div>`,
|
`<h2>Allow</h2><div class="sp-end">2 entries</div>`,
|
||||||
`<td class="sp-mono sp-small">bulk.example <span class="tag is-light">domain</span></td><td class="sp-muted">newsletter nobody subscribed to</td><td class="sp-muted sp-nowrap sp-small">2026-09-02</td>`,
|
`<thead><tr><th>Value</th><th>Kind</th><th>Matched</th><th></th></tr></thead>`,
|
||||||
`198.51.100.0/24 <span class="tag is-light">ip</span>`,
|
// the kind is a tag that opens the entry's sheet; the count is beside it
|
||||||
`win@lottery.example <span class="tag is-light">address</span></td><td class="sp-muted">—</td>`,
|
`<td class="sp-mono sp-small">bulk.example</td><td><button type="button" class="tag is-light" popovertarget="entry-1">domain</button><div class="sp-sheet" id="entry-1" popover>`,
|
||||||
|
`<div class="sp-sheet-head"><span class="tag is-light">domain</span><h2>bulk.example</h2>`,
|
||||||
|
`<tr><th>Note</th><td>newsletter nobody subscribed to</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-09-02</td></tr><tr><th>Matched</th><td>41 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-09-21 13:57</td></tr></tbody>`,
|
||||||
|
`</div></td><td class="sp-nowrap">41</td><td class="sp-actions">`,
|
||||||
|
`<td class="sp-nowrap">1203</td>`,
|
||||||
|
// an entry that never decided anything says so, in the row and on its sheet
|
||||||
|
`<td class="sp-mono sp-small">win@lottery.example</td><td><button type="button" class="tag is-light" popovertarget="entry-3">address</button>`,
|
||||||
|
`<tr><th>Note</th><td>—</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-07-30</td></tr><tr><th>Matched</th><td>never in the last 90 days</td></tr></tbody>`,
|
||||||
|
`</div></td><td class="sp-muted sp-nowrap">never</td><td class="sp-actions">`,
|
||||||
`<form method="post" action="/inbound/filter-lists/3/delete"><button type="submit" class="has-text-danger">Remove</button></form>`,
|
`<form method="post" action="/inbound/filter-lists/3/delete"><button type="submit" class="has-text-danger">Remove</button></form>`,
|
||||||
`<h2>Where the lists go</h2><div class="sp-end"><span class="tag is-success is-light">applied</span></div>`,
|
`<h2>Where the lists go</h2><div class="sp-end"><span class="tag is-success is-light">applied</span></div>`,
|
||||||
`rspamd, from map files it rereads within seconds of a change`,
|
`rspamd, from map files it rereads within seconds of a change`,
|
||||||
`<span class="sp-mono">/data/antispam/</span> — six, one for each list and kind of entry`,
|
`<span class="sp-mono">/data/antispam/</span> — six, one for each list and kind of entry`,
|
||||||
`2026-09-11 08:40 UTC`, `Deny wins, whichever entries they are.`,
|
`2026-09-11 08:40 UTC`, `Deny wins, whichever entries they are.`,
|
||||||
)
|
)
|
||||||
if got := strings.Count(out, `<table`); got != 2 {
|
if got := strings.Count(out, `<table class="table is-fullwidth is-hoverable">`); got != 2 {
|
||||||
t.Errorf("%d tables, want the two lists", got)
|
t.Errorf("%d list tables, want the two lists", got)
|
||||||
}
|
}
|
||||||
|
pageLacks(t, "Filter lists", out, `<th>Note</th><th>`, `<th>Added</th><th>`)
|
||||||
}
|
}
|
||||||
|
|
||||||
// With no filter attached the lists are still the operator's to edit, and the
|
// With no filter attached the lists are still the operator's to edit, and the
|
||||||
@@ -213,8 +222,8 @@ func TestFilterListsOffAndEmpty(t *testing.T) {
|
|||||||
`No sender is turned away by hand.`,
|
`No sender is turned away by hand.`,
|
||||||
)
|
)
|
||||||
pageLacks(t, "Filter lists", out, `rspamd, from map files`, `is-success is-light">applied`, `No sender is let through by hand.`)
|
pageLacks(t, "Filter lists", out, `rspamd, from map files`, `is-success is-light">applied`, `No sender is let through by hand.`)
|
||||||
if got := strings.Count(out, `<table`); got != 1 {
|
if got := strings.Count(out, `<table class="table is-fullwidth is-hoverable">`); got != 1 {
|
||||||
t.Errorf("%d tables, want the one list that has entries", got)
|
t.Errorf("%d list tables, want the one list that has entries", got)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -1,7 +1,8 @@
|
|||||||
{{/* Inbound > Filter lists (mockup in-filter-lists.html): the allow and deny
|
{{/* Inbound > Filter lists (mockup in-filter-lists.html): the allow and deny
|
||||||
lists that correct the spam filter by hand, for the whole instance, and
|
lists that correct the spam filter by hand, for the whole instance, and
|
||||||
where they go. Global role only. The form adds an entry; each row of a list
|
where they go. Global role only. The form adds an entry; each row of a list
|
||||||
removes its own. */}}
|
removes its own, says how often the entry decided a message, and opens a
|
||||||
|
sheet with the entry's note from the tag of its kind. */}}
|
||||||
{{define "content"}}
|
{{define "content"}}
|
||||||
{{template "page_head" .Head}}
|
{{template "page_head" .Head}}
|
||||||
{{if .Flash}}{{template "flash" .Flash}}{{end}}
|
{{if .Flash}}{{template "flash" .Flash}}{{end}}
|
||||||
@@ -54,9 +55,9 @@
|
|||||||
table, or what is absent when the list is empty. */}}
|
table, or what is absent when the list is empty. */}}
|
||||||
{{define "filter_list"}}{{template "box_open" .Box}}
|
{{define "filter_list"}}{{template "box_open" .Box}}
|
||||||
{{if .Rows}}<div class="table-container"><table class="table is-fullwidth is-hoverable">
|
{{if .Rows}}<div class="table-container"><table class="table is-fullwidth is-hoverable">
|
||||||
<thead><tr><th>Value</th><th>Note</th><th>Added</th><th></th></tr></thead>
|
<thead><tr><th>Value</th><th>Kind</th><th>Matched</th><th></th></tr></thead>
|
||||||
<tbody>{{range .Rows}}
|
<tbody>{{range .Rows}}
|
||||||
<tr><td class="sp-mono sp-small">{{.Value}} {{template "tag" .Kind}}</td>{{if .Note}}<td class="sp-muted">{{.Note}}</td>{{else}}<td class="sp-muted">—</td>{{end}}<td class="sp-muted sp-nowrap sp-small">{{.Added}}</td><td class="sp-actions">{{template "row_action" .Remove}}</td></tr>{{end}}
|
<tr><td class="sp-mono sp-small">{{.Value}}</td><td>{{template "sheet_tag" .Entry}}</td>{{if .Matched}}<td class="sp-nowrap">{{.Matched}}</td>{{else}}<td class="sp-muted sp-nowrap">never</td>{{end}}<td class="sp-actions">{{template "row_action" .Remove}}</td></tr>{{end}}
|
||||||
</tbody>
|
</tbody>
|
||||||
</table></div>
|
</table></div>
|
||||||
{{else}}{{template "empty_state" .Empty}}
|
{{else}}{{template "empty_state" .Empty}}
|
||||||
|
|||||||
Reference in New Issue
Block a user