inbound-antispam-panel: Filter lists, kind column, matches, note on a sheet

Follows the design commit before it (owner's words quoted there).

The lists' tables are Value, Kind, Matched and Remove. The kind tag
opens the entry's sheet: note, day added, how often it decided a
message in the days the log keeps, and the last match.

Matches are read off the journal on each view, nothing is stored
(inbound.Lists.Usage, store.ListSpamListHits). A row counts for an
entry when the list decided the message - a rejected row whose reply is
the deny rule's message, an accepted row whose verdict carries the
allow rule's symbol for that kind - and the entry fits the row's sender
or client address the way rspamd matches it: an address whole, a domain
as the sender's domain, a network containing the client (an
IPv4-mapped address as IPv4). A failure to count shows the lists with
"never" and logs why.

For that the journal milter keeps the lists' own symbols ahead of its
six-symbol cap: rspamd writes symbols heaviest first, so an allow
entry's -100 came last and was cut behind any six others.

The plan gains "Filter list usage": what counting from the journal
implies, and - the owner asked for it in writing - the conditions under
which a counter kept with each entry becomes worth a migration (the
journal cut by its row cap and not by age; a short retention; counts
that must outlive the journal; a slow page). multimap.conf says which
of its texts SelfPost reads back.

Tests: Usage over every kind on both lists and the rows that are
nobody's; the milter's symbol order; the store query; the page and its
sheets; a handler test from journal rows to the numbers on the page.

Seen: the fixture page with a sheet open, headless Edge at 1360 px -
the same as the accepted drawing.

Checked on the test server, uncommitted tree: go vet ./... ok,
go test ./... ok (22 packages), make e2e ok.

Not seen in a real run: a count above zero. It needs an entry added in
the panel, which I cannot sign in to, and then mail that the list
decides; in particular the deny rule's message arriving in mail.log as
the reply of a milter-reject line is what the trial with rspamc showed,
not yet what Postfix logged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
mix
2026-10-12 01:08:26 +03:00
co-authored by Claude Opus 5.5
parent e0ccff257d
commit 7192c3861e
18 changed files with 501 additions and 45 deletions
+113
View File
@@ -21,6 +21,7 @@ type ListStore interface {
AddSpamListEntry(list, kind, value, note string) (store.SpamListEntry, error)
ListSpamListEntries() ([]store.SpamListEntry, error)
DeleteSpamListEntry(id int64) error
ListSpamListHits(marks ...string) ([]store.SpamListHit, error)
}
// Lists owns the allow and deny lists of the external spam filter: the rows in
@@ -236,3 +237,115 @@ func normalizeNote(raw string) (string, error) {
}
return note, nil
}
// What a list leaves in the journal when it decides a message - the two ends
// of deploy/antispam/rspamd/multimap.conf this package depends on. A deny rule
// refuses with a message, which the log tailer files as the reply of a
// rejected row. An allow rule adds a symbol named after the kind of entry,
// which the journal milter keeps in the verdict of an accepted row.
const (
denyReplyMark = "this server's deny list"
allowMark = "SELFPOST_ALLOW_"
)
var allowSymbol = map[string]string{
store.SpamEntryIP: "SELFPOST_ALLOW_IP(",
store.SpamEntryAddress: "SELFPOST_ALLOW_FROM(",
store.SpamEntryDomain: "SELFPOST_ALLOW_DOMAIN(",
}
// Usage is how often an entry decided a message, as far back as the journal
// reaches, and when it last did. The zero value is an entry that never did.
type Usage struct {
Hits int
Last time.Time
}
// Usage counts, for every entry, the journal rows it decided. Nothing is
// stored for this: the journal already says which list decided a message and
// who the message was from, and an entry is matched against that the way the
// filter matches it - an address whole, a domain as the sender's domain, an
// address or network against the client's. So the count reaches back as far
// as the journal does (its retention, or less when it is full), and a row
// counts for every entry of the deciding list that fits it: the journal does
// not say which of two fitting entries the filter met first.
//
// A counter kept with each entry would not depend on the journal; the plan
// (docs/plans/inbound-antispam-panel.md, Filter list usage) says when that is
// worth a migration.
func (l *Lists) Usage() (map[int64]Usage, error) {
entries, err := l.store.ListSpamListEntries()
if err != nil {
return nil, err
}
out := make(map[int64]Usage, len(entries))
if len(entries) == 0 {
return out, nil
}
hits, err := l.store.ListSpamListHits(denyReplyMark, allowMark)
if err != nil {
return nil, err
}
type matcher struct {
id int64
list string
kind string
value string
prefix netip.Prefix
}
matchers := make([]matcher, 0, len(entries))
for _, e := range entries {
m := matcher{id: e.ID, list: e.List, kind: e.Kind, value: e.Value}
if e.Kind == store.SpamEntryIP {
if p, err := netip.ParsePrefix(e.Value); err == nil {
m.prefix = p
} else if a, err := netip.ParseAddr(e.Value); err == nil {
m.prefix = netip.PrefixFrom(a, a.BitLen())
} else {
continue // not an entry this package wrote
}
}
matchers = append(matchers, m)
}
for _, h := range hits {
from := strings.ToLower(h.From)
domain := ""
if i := strings.LastIndexByte(from, '@'); i >= 0 {
domain = from[i+1:]
}
client, clientErr := netip.ParseAddr(h.ClientIP)
client = client.Unmap()
denied := h.Decision != store.SpamDecisionAccept && strings.Contains(h.Detail, denyReplyMark)
for _, m := range matchers {
switch m.list {
case store.SpamListDeny:
if !denied {
continue
}
case store.SpamListAllow:
if h.Decision != store.SpamDecisionAccept || !strings.Contains(h.Detail, allowSymbol[m.kind]) {
continue
}
}
fits := false
switch m.kind {
case store.SpamEntryAddress:
fits = from == m.value
case store.SpamEntryDomain:
fits = domain == m.value
case store.SpamEntryIP:
fits = clientErr == nil && m.prefix.Contains(client)
}
if !fits {
continue
}
u := out[m.id]
u.Hits++
if h.CreatedAt.After(u.Last) {
u.Last = h.CreatedAt
}
out[m.id] = u
}
}
return out, nil
}
+92
View File
@@ -276,3 +276,95 @@ func TestListsLastWritten(t *testing.T) {
t.Errorf("LastWritten counts a stray file: %v", got)
}
}
// How often an entry decided a message is read off the journal: the list that
// decided, and whether the entry fits who the message was from.
func TestListsUsage(t *testing.T) {
l, st, _ := testLists(t)
add := func(list, kind, value string) int64 {
t.Helper()
e, err := l.Add(list, kind, value, "")
if err != nil {
t.Fatalf("Add(%s, %s, %s): %v", list, kind, value, err)
}
return e.ID
}
denyDomain := add(store.SpamListDeny, store.SpamEntryDomain, "evil.example")
denyAddr := add(store.SpamListDeny, store.SpamEntryAddress, "spammer@evil.example")
denyNet := add(store.SpamListDeny, store.SpamEntryIP, "198.51.100.0/24")
denyV6 := add(store.SpamListDeny, store.SpamEntryIP, "2001:db8::/32")
allowAddr := add(store.SpamListAllow, store.SpamEntryAddress, "billing@vendor.example")
allowDomain := add(store.SpamListAllow, store.SpamEntryDomain, "vendor.example")
allowIP := add(store.SpamListAllow, store.SpamEntryIP, "203.0.113.5")
unused := add(store.SpamListAllow, store.SpamEntryDomain, "quiet.example")
const denied = "5.7.1 Refused by this server's deny list"
at := time.Date(2026, 10, 1, 0, 0, 0, 0, time.UTC)
hour := 0
row := func(from, ip, decision, engine, detail string) {
t.Helper()
hour++
err := st.InsertInboundSpam(store.InboundSpamEntry{
InboundDomain: "in.example.com", To: "a@in.example.com", From: from, ClientIP: ip,
Decision: decision, Engine: engine, Detail: detail, At: at.Add(time.Duration(hour) * time.Hour),
})
if err != nil {
t.Fatal(err)
}
}
// hour 1: the address and its domain are both on the deny list, and both fit.
row("Spammer@Evil.Example", "192.0.2.9", store.SpamDecisionReject, store.SpamEngineRspamd, denied)
// hour 2: another address of the domain.
row("other@evil.example", "192.0.2.9", store.SpamDecisionReject, store.SpamEngineRspamd, denied)
// hour 3, 4: the networks, v4 (as Postfix may log it, mapped) and v6.
row("x@elsewhere.example", "::ffff:198.51.100.77", store.SpamDecisionReject, store.SpamEngineRspamd, denied)
row("x@elsewhere.example", "2001:db8::1", store.SpamDecisionReject, store.SpamEngineRspamd, denied)
// hour 5: a subdomain is not the domain.
row("x@sub.evil.example", "192.0.2.9", store.SpamDecisionReject, store.SpamEngineRspamd, denied)
// hour 6: allowed by its address and its domain - two symbols, two entries.
row("billing@vendor.example", "192.0.2.50", store.SpamDecisionAccept, store.SpamEngineRspamd,
"no action, score -199.00 / 15.00; SELFPOST_ALLOW_DOMAIN(-100.00) SELFPOST_ALLOW_FROM(-100.00)")
// hour 7: allowed by its domain alone.
row("accounts@vendor.example", "192.0.2.50", store.SpamDecisionAccept, store.SpamEngineRspamd,
"no action, score -99.00 / 15.00; SELFPOST_ALLOW_DOMAIN(-100.00)")
// hour 8: allowed by the client address; the sender's domain is not why.
row("x@quiet.example", "203.0.113.5", store.SpamDecisionAccept, store.SpamEngineRspamd,
"no action, score -99.00 / 15.00; SELFPOST_ALLOW_IP(-100.00)")
// Nobody's: the filter's own refusal, an accept with no list symbol, a
// refusal by Postfix, and a sender who spells the mark in his own address.
row("spammer@evil.example", "192.0.2.9", store.SpamDecisionReject, store.SpamEngineRspamd, "5.7.1 Spam message rejected")
row("billing@vendor.example", "192.0.2.50", store.SpamDecisionAccept, store.SpamEngineRspamd, "no action, score 0.10 / 15.00")
row("spammer@evil.example", "198.51.100.1", store.SpamDecisionReject, store.SpamEnginePostfix, "550 5.1.1 <nobody@in.example.com>: this server's deny list")
row("billing@vendor.example", "203.0.113.5", store.SpamDecisionAccept, store.SpamEngineSelfPost, "")
usage, err := l.Usage()
if err != nil {
t.Fatalf("Usage: %v", err)
}
for name, c := range map[string]struct {
id int64
hits int
last int // hour
}{
"deny domain": {denyDomain, 2, 2}, "deny address": {denyAddr, 1, 1},
"deny network": {denyNet, 1, 3}, "deny v6 network": {denyV6, 1, 4},
"allow address": {allowAddr, 1, 6}, "allow domain": {allowDomain, 2, 7}, "allow ip": {allowIP, 1, 8},
"unused": {unused, 0, 0},
} {
got := usage[c.id]
wantLast := time.Time{}
if c.hits > 0 {
wantLast = at.Add(time.Duration(c.last) * time.Hour)
}
if got.Hits != c.hits || !got.Last.Equal(wantLast) {
t.Errorf("%s: %d hits, last %v; want %d, %v", name, got.Hits, got.Last, c.hits, wantLast)
}
}
}
func TestListsUsageWithNoEntries(t *testing.T) {
l, _, _ := testLists(t)
if usage, err := l.Usage(); err != nil || len(usage) != 0 {
t.Errorf("Usage = %v, %v", usage, err)
}
}
+20 -3
View File
@@ -142,12 +142,16 @@ func (s *inboundSession) record() {
}
}
// What the journal keeps of a verdict: enough to see why, on one table row.
// What the journal keeps of a verdict: enough to see why, on one line.
const (
verdictMaxSymbols = 6
verdictMaxBytes = 255
)
// ListSymbolPrefix starts the names of the symbols the filter lists' rules add
// (deploy/antispam/rspamd/multimap.conf): SELFPOST_ALLOW_IP, _FROM, _DOMAIN.
const ListSymbolPrefix = "SELFPOST_"
var (
// "default: False [7.20 / 15.00]" - the score and the reject threshold.
rspamdScore = regexp.MustCompile(`\[(-?[0-9]+(?:\.[0-9]+)?) / (-?[0-9]+(?:\.[0-9]+)?)\]`)
@@ -182,15 +186,28 @@ func rspamdVerdict(actionHeader, resultHeader string) (action, detail string) {
}
detail = strings.Join(parts, ", ")
var symbols []string
// The filter lists' own symbols come first and are kept whatever else there
// is: Filter lists counts how often an entry decided by finding them here,
// and rspamd writes symbols heaviest first, which puts an allow entry's
// -100 last, behind any six others.
var symbols, rest []string
for _, m := range rspamdSymbol.FindAllStringSubmatch(resultHeader, -1) {
if strings.Trim(m[2], "-0.") == "" { // weight zero: informational
continue
}
symbols = append(symbols, m[1]+"("+m[2]+")")
if strings.HasPrefix(m[1], ListSymbolPrefix) {
if len(symbols) < verdictMaxSymbols {
symbols = append(symbols, m[1]+"("+m[2]+")")
}
continue
}
rest = append(rest, m[1]+"("+m[2]+")")
}
for _, sym := range rest {
if len(symbols) == verdictMaxSymbols {
break
}
symbols = append(symbols, sym)
}
if len(symbols) > 0 {
if detail != "" {
+12
View File
@@ -222,3 +222,15 @@ func TestInboundAcceptsWhenTheStoreFails(t *testing.T) {
}
}
}
// rspamd writes symbols heaviest first, so an allow entry's -100 comes last.
// Filter lists counts an entry's matches by finding that symbol in the journal:
// it is kept whatever else there is.
func TestRspamdVerdictKeepsTheListSymbols(t *testing.T) {
result := "default: False [-91.00 / 15.00]; SYM_A(3.00)[]; SYM_B(2.00)[]; SYM_C(1.50)[]; SYM_D(1.00)[]; SYM_E(0.80)[]; SYM_F(0.50)[]; SYM_G(0.20)[]; ARC_NA(0.00)[]; SELFPOST_ALLOW_FROM(-100.00)[billing@vendor.example]"
_, detail := rspamdVerdict("no action", result)
want := "no action, score -91.00 / 15.00; SELFPOST_ALLOW_FROM(-100.00) SYM_A(3.00) SYM_B(2.00) SYM_C(1.50) SYM_D(1.00) SYM_E(0.80)"
if detail != want {
t.Errorf("detail = %q\nwant %q", detail, want)
}
}
+45
View File
@@ -304,3 +304,48 @@ func (s *Store) DeleteSpamListEntry(id int64) error {
}
return nil
}
// SpamListHit is what the journal holds about one message a filter list
// decided: who it was from, which server offered it, when, and the detail that
// says which list it was.
type SpamListHit struct {
ClientIP string
From string
Decision string
Detail string
CreatedAt time.Time
}
// ListSpamListHits returns the journal rows of the filter in whose detail one
// of the marks occurs - the marks the filter lists leave (internal/inbound).
// Which entry a row belongs to is for the caller to work out; here the rows
// are only narrowed, so that a page view does not read the whole journal.
func (s *Store) ListSpamListHits(marks ...string) ([]SpamListHit, error) {
if len(marks) == 0 {
return nil, nil
}
likes := make([]string, len(marks))
args := []any{SpamEngineRspamd}
for i, m := range marks {
likes[i] = "instr(detail, ?) > 0"
args = append(args, m)
}
rows, err := s.db.Query(
`SELECT client_ip, from_addr, decision, detail, created_at FROM inbound_spam_log
WHERE engine = ? AND (`+strings.Join(likes, " OR ")+`)`, args...)
if err != nil {
return nil, fmt.Errorf("query filter list hits: %w", err)
}
defer rows.Close()
var out []SpamListHit
for rows.Next() {
var h SpamListHit
var created string
if err := rows.Scan(&h.ClientIP, &h.From, &h.Decision, &h.Detail, &created); err != nil {
return nil, fmt.Errorf("scan filter list hit: %w", err)
}
h.CreatedAt, _ = time.Parse(time.RFC3339, created)
out = append(out, h)
}
return out, rows.Err()
}
+32
View File
@@ -252,3 +252,35 @@ func TestQueryInboundSpamFilters(t *testing.T) {
t.Errorf("injected domain = %q", got)
}
}
func TestListSpamListHits(t *testing.T) {
st := openTestStore(t)
add := func(engine, decision, detail string) {
t.Helper()
err := st.InsertInboundSpam(InboundSpamEntry{
InboundDomain: "in.example.com", ClientIP: "203.0.113.7", From: "a@example.net", To: "a@in.example.com",
Decision: decision, Engine: engine, Detail: detail, At: time.Date(2026, 10, 1, 9, 0, 0, 0, time.UTC),
})
if err != nil {
t.Fatal(err)
}
}
add(SpamEngineRspamd, SpamDecisionReject, "5.7.1 Refused by this server's deny list")
add(SpamEngineRspamd, SpamDecisionAccept, "no action, score -99.00 / 15.00; SELFPOST_ALLOW_IP(-100.00)")
add(SpamEngineRspamd, SpamDecisionReject, "5.7.1 Spam message rejected")
add(SpamEnginePostfix, SpamDecisionReject, "550 5.1.1 this server's deny list")
// a mark with the characters LIKE would treat as wildcards is text
add(SpamEngineRspamd, SpamDecisionAccept, "no action; SELFPOSTXALLOWX(1.00)")
hits, err := st.ListSpamListHits("this server's deny list", "SELFPOST_ALLOW_")
if err != nil || len(hits) != 2 {
t.Fatalf("hits = %+v, %v; want the two rows a list decided", hits, err)
}
h := hits[0]
if h.From != "a@example.net" || h.ClientIP != "203.0.113.7" || h.Decision != SpamDecisionReject || h.CreatedAt.Hour() != 9 {
t.Errorf("hit = %+v", h)
}
if hits, err := st.ListSpamListHits(); err != nil || hits != nil {
t.Errorf("no marks: %v, %v", hits, err)
}
}
+11 -2
View File
@@ -167,12 +167,21 @@ func (h *Handlers) renderFilterLists(w http.ResponseWriter, r *http.Request, sta
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
// How often each entry decided a message is counted from the journal. A
// failure there is not a reason to hide the lists: they are shown without
// the counts' worth - every entry "never" - and the log says why.
usage, err := h.lists.Usage()
if err != nil {
logf("panel: filter lists: usage: %v", err)
}
on, _, _ := h.inboundFilter()
in := make([]view.FilterEntryInput, len(entries))
for i, e := range entries {
in[i] = view.FilterEntryInput{ID: e.ID, List: e.List, Kind: e.Kind, Value: e.Value, Note: e.Note, Added: e.CreatedAt}
in[i] = view.FilterEntryInput{ID: e.ID, List: e.List, Kind: e.Kind, Value: e.Value, Note: e.Note, Added: e.CreatedAt,
Hits: usage[e.ID].Hits, LastHit: usage[e.ID].Last}
}
page := view.NewFilterLists(h.shellMeta(r), view.FilterListsInput{FilterOn: on, LastWritten: h.lists.LastWritten()}).
page := view.NewFilterLists(h.shellMeta(r), view.FilterListsInput{
FilterOn: on, LastWritten: h.lists.LastWritten(), RetentionDays: h.inboundSpamLogRetentionDays()}).
WithEntries(in).
WithResult(flash, formErr)
// A blank list or kind leaves the form on its first choice.
@@ -364,9 +364,10 @@ func TestFilterListsAddAndRemove(t *testing.T) {
has(t, "lists", page,
`<div class="notification is-success is-light">`, `Entry added.`,
`<h2>Deny</h2><div class="sp-end">1 entry · wins over Allow</div>`, `<h2>Allow</h2><div class="sp-end">1 entry</div>`,
`bulk.example <span class="tag is-light">domain</span></td><td class="sp-muted">newsletter</td>`,
`198.51.100.0/24 <span class="tag is-light">ip</span></td><td class="sp-muted">—</td>`,
`<td class="sp-muted sp-nowrap sp-small">`+time.Now().UTC().Format("2006-01-02")+`</td>`,
`<td class="sp-mono sp-small">bulk.example</td><td><button type="button" class="tag is-light" popovertarget="entry-`,
`<tr><th>Note</th><td>newsletter</td></tr><tr><th>Added</th><td class="sp-mono sp-small">`+time.Now().UTC().Format("2006-01-02")+`</td></tr><tr><th>Matched</th><td>never in the last 90 days</td></tr>`,
`<td class="sp-mono sp-small">198.51.100.0/24</td>`, `<tr><th>Note</th><td>—</td></tr>`,
`<td class="sp-muted sp-nowrap">never</td>`,
)
lacks(t, "lists", page, `No sender is turned away`)
@@ -635,3 +636,39 @@ func TestHelpHasTheInboundLogAndFilterListsTopics(t *testing.T) {
has(t, "log", s.get(h.HandleInboundLog, "/inbound/log", globalPrincipal).Body.String(), `href="/help#inbound-log"`)
has(t, "lists", s.get(h.HandleFilterLists, "/inbound/filter-lists", globalPrincipal).Body.String(), `href="/help#filter-lists"`)
}
// An entry's count comes from the journal: a row counts for the entry that
// fits it on the list that decided it.
func TestFilterListsCountMatchesFromTheJournal(t *testing.T) {
s := newInboundLogStand(t)
for _, e := range [][3]string{{"deny", "domain", "bulk.example"}, {"allow", "address", "billing@vendor.example"}, {"deny", "ip", "203.0.113.0/24"}} {
if _, err := s.h.lists.Add(e[0], e[1], e[2], ""); err != nil {
t.Fatalf("Add %v: %v", e, err)
}
}
at := time.Date(2026, 10, 1, 9, 30, 0, 0, time.UTC)
for i, row := range []store.InboundSpamEntry{
{From: "promo@bulk.example", ClientIP: "198.51.100.9", Decision: store.SpamDecisionReject, Engine: store.SpamEngineRspamd, Detail: "5.7.1 Refused by this server's deny list"},
{From: "other@BULK.example", ClientIP: "198.51.100.9", Decision: store.SpamDecisionReject, Engine: store.SpamEngineRspamd, Detail: "5.7.1 Refused by this server's deny list"},
{From: "billing@vendor.example", ClientIP: "192.0.2.5", Decision: store.SpamDecisionAccept, Engine: store.SpamEngineRspamd, Detail: "no action, score -99.00 / 15.00; SELFPOST_ALLOW_FROM(-100.00)"},
// refused by the filter for its own reasons: nobody's match
{From: "promo@bulk.example", ClientIP: "198.51.100.9", Decision: store.SpamDecisionReject, Engine: store.SpamEngineRspamd, Detail: "5.7.1 Spam message rejected"},
} {
row.InboundDomain, row.To, row.At = "a.example.com", "alice@a.example.com", at.Add(time.Duration(i)*time.Hour)
if err := s.st.InsertInboundSpam(row); err != nil {
t.Fatal(err)
}
}
page := s.get(s.h.HandleFilterLists, "/inbound/filter-lists", globalPrincipal).Body.String()
has(t, "lists", page,
`<tr><th>Matched</th><td>2 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-10-01 10:30</td></tr>`,
`<td class="sp-nowrap">2</td>`,
`<tr><th>Matched</th><td>1 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-10-01 11:30</td></tr>`,
`<td class="sp-nowrap">1</td>`,
// the network nobody came from
`<td class="sp-muted sp-nowrap">never</td>`,
)
if got := strings.Count(page, `<td class="sp-muted sp-nowrap">never</td>`); got != 1 {
t.Errorf("%d entries say never, want the one network", got)
}
}
+6 -5
View File
@@ -450,12 +450,13 @@ func inLogFixture() *InLog {
// allowed, and a filter that reads them.
func inFilterListsFixture() *FilterLists {
day := func(m time.Month, d int) time.Time { return time.Date(2026, m, d, 0, 0, 0, 0, time.UTC) }
return NewFilterLists(admin(), FilterListsInput{FilterOn: true, LastWritten: time.Date(2026, 9, 11, 8, 40, 0, 0, time.UTC)}).
at := func(d, h, m int) time.Time { return time.Date(2026, 9, d, h, m, 0, 0, time.UTC) }
return NewFilterLists(admin(), FilterListsInput{FilterOn: true, LastWritten: time.Date(2026, 9, 11, 8, 40, 0, 0, time.UTC), RetentionDays: 90}).
WithEntries([]FilterEntryInput{
{ID: 1, List: "deny", Kind: "domain", Value: "bulk.example", Note: "newsletter nobody subscribed to", Added: day(9, 2)},
{ID: 2, List: "deny", Kind: "ip", Value: "198.51.100.0/24", Note: "dictionary attack on lists.example.org", Added: day(8, 19)},
{ID: 1, List: "deny", Kind: "domain", Value: "bulk.example", Note: "newsletter nobody subscribed to", Added: day(9, 2), Hits: 41, LastHit: at(21, 13, 57)},
{ID: 2, List: "deny", Kind: "ip", Value: "198.51.100.0/24", Note: "dictionary attack on lists.example.org", Added: day(8, 19), Hits: 1203, LastHit: at(21, 9, 12)},
{ID: 3, List: "deny", Kind: "address", Value: "win@lottery.example", Added: day(7, 30)},
{ID: 4, List: "allow", Kind: "domain", Value: "partner.example", Note: "invoices were scored as spam", Added: day(9, 11)},
{ID: 5, List: "allow", Kind: "address", Value: "noreply@github.example", Note: "security advisories", Added: day(6, 4)},
{ID: 4, List: "allow", Kind: "domain", Value: "partner.example", Note: "invoices were scored as spam", Added: day(9, 11), Hits: 7, LastHit: at(19, 16, 40)},
{ID: 5, List: "allow", Kind: "address", Value: "noreply@github.example", Note: "security advisories", Added: day(6, 4), Hits: 18, LastHit: at(21, 13, 30)},
})
}
+1 -1
View File
@@ -205,7 +205,7 @@ var inboundTopics = []HelpTopic{
Strong("Deny wins"), ".",
Br(), Br(),
"SelfPost does not apply the lists itself: it writes them as files under ", Code("/data/antispam/"),
", and rspamd rereads a file within seconds of a change. With the spam filter off the lists are kept and can be edited, but nothing reads them. The sender matched is the envelope sender — the one in the log — not the From line a mail program shows."),
", and rspamd rereads a file within seconds of a change. With the spam filter off the lists are kept and can be edited, but nothing reads them. The sender matched is the envelope sender — the one in the log — not the From line a mail program shows. Matched counts the messages in the Inbound log that the entry decided, so it reaches back as far as the log does; never means not once in that time, and such an entry is one to consider removing."),
}},
}
+43 -17
View File
@@ -46,6 +46,8 @@ type FilterLists struct {
Where Box
WhereFacts []Fact
retentionDays int
}
// FilterList is one of the two boxes of entries.
@@ -55,24 +57,29 @@ type FilterList struct {
Empty EmptyState
}
// FilterRow is one entry of a list: the value with the kind of entry as a tag,
// the operator's note, the day it was added and the way to remove it.
// FilterRow is one entry of a list: the value, the kind of entry as a tag that
// opens the entry's sheet (the note, the day it was added, how often it decided
// a message and when last), how often it decided, and the way to remove it.
// Matched is empty for an entry that never did; the table then says "never".
type FilterRow struct {
Value string
Kind Tag
Note string
Added string
Remove RowAction
Value string
Entry Sheet
Matched string
Remove RowAction
}
// FilterEntryInput is what the page knows of one entry of a list.
// FilterEntryInput is what the page knows of one entry of a list. Hits and
// LastHit are counted from the Inbound log, so they reach back as far as it
// does.
type FilterEntryInput struct {
ID int64
List string
Kind string
Value string
Note string
Added time.Time
ID int64
List string
Kind string
Value string
Note string
Added time.Time
Hits int
LastHit time.Time
}
// FilterListsInput is the state of the filter the lists are written for: whether
@@ -80,6 +87,9 @@ type FilterEntryInput struct {
type FilterListsInput struct {
FilterOn bool
LastWritten time.Time
// RetentionDays is how long the Inbound log keeps a row: how far back an
// entry's matches are counted.
RetentionDays int
}
// NewFilterLists builds the page. The form starts on the Deny list for a sender
@@ -113,6 +123,7 @@ func NewFilterLists(m Meta, in FilterListsInput) *FilterLists {
}
p.Deny.Box.End = Plain(plural(0, "entry", "entries") + " · wins over Allow")
p.Allow.Box.End = Plain(plural(0, "entry", "entries"))
p.retentionDays = in.RetentionDays
p.setWhere(in)
return p
}
@@ -144,10 +155,25 @@ func (p *FilterLists) setWhere(in FilterListsInput) {
func (p *FilterLists) WithEntries(entries []FilterEntryInput) *FilterLists {
p.Deny.Rows, p.Allow.Rows = nil, nil
for _, e := range entries {
row := FilterRow{
Value: e.Value, Kind: Tag{Label: e.Kind}, Note: e.Note, Added: e.Added.UTC().Format("2006-01-02"),
Remove: RowAction{Label: "Remove", Post: inboundFilterLists + "/" + strconv.FormatInt(e.ID, 10) + "/delete", Danger: true},
id := strconv.FormatInt(e.ID, 10)
window := "in the last " + plural(int64(p.retentionDays), "day", "days")
sheet := []SheetRow{
{Label: "Note", Lines: []string{orDash(e.Note)}},
{Label: "Added", Lines: []string{e.Added.UTC().Format("2006-01-02")}, Mono: true},
}
row := FilterRow{
Value: e.Value,
Remove: RowAction{Label: "Remove", Post: inboundFilterLists + "/" + id + "/delete", Danger: true},
}
if e.Hits > 0 {
row.Matched = strconv.Itoa(e.Hits)
sheet = append(sheet,
SheetRow{Label: "Matched", Lines: []string{row.Matched + " " + window}},
SheetRow{Label: "Last match", Lines: []string{e.LastHit.UTC().Format("2006-01-02 15:04")}, Mono: true})
} else {
sheet = append(sheet, SheetRow{Label: "Matched", Lines: []string{"never " + window}})
}
row.Entry = Sheet{ID: "entry-" + id, Tag: Tag{Label: e.Kind}, Title: e.Value, Rows: sheet}
switch e.List {
case "deny":
p.Deny.Rows = append(p.Deny.Rows, row)
+16 -7
View File
@@ -183,18 +183,27 @@ func TestFilterListsShowsTheFormTheListsAndWhereTheyGo(t *testing.T) {
`href="/help#filter-lists" title="How the lists are applied"`,
`<h2>Deny</h2><div class="sp-end">3 entries · wins over Allow</div>`,
`<h2>Allow</h2><div class="sp-end">2 entries</div>`,
`<td class="sp-mono sp-small">bulk.example <span class="tag is-light">domain</span></td><td class="sp-muted">newsletter nobody subscribed to</td><td class="sp-muted sp-nowrap sp-small">2026-09-02</td>`,
`198.51.100.0/24 <span class="tag is-light">ip</span>`,
`win@lottery.example <span class="tag is-light">address</span></td><td class="sp-muted">—</td>`,
`<thead><tr><th>Value</th><th>Kind</th><th>Matched</th><th></th></tr></thead>`,
// the kind is a tag that opens the entry's sheet; the count is beside it
`<td class="sp-mono sp-small">bulk.example</td><td><button type="button" class="tag is-light" popovertarget="entry-1">domain</button><div class="sp-sheet" id="entry-1" popover>`,
`<div class="sp-sheet-head"><span class="tag is-light">domain</span><h2>bulk.example</h2>`,
`<tr><th>Note</th><td>newsletter nobody subscribed to</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-09-02</td></tr><tr><th>Matched</th><td>41 in the last 90 days</td></tr><tr><th>Last match</th><td class="sp-mono sp-small">2026-09-21 13:57</td></tr></tbody>`,
`</div></td><td class="sp-nowrap">41</td><td class="sp-actions">`,
`<td class="sp-nowrap">1203</td>`,
// an entry that never decided anything says so, in the row and on its sheet
`<td class="sp-mono sp-small">win@lottery.example</td><td><button type="button" class="tag is-light" popovertarget="entry-3">address</button>`,
`<tr><th>Note</th><td>—</td></tr><tr><th>Added</th><td class="sp-mono sp-small">2026-07-30</td></tr><tr><th>Matched</th><td>never in the last 90 days</td></tr></tbody>`,
`</div></td><td class="sp-muted sp-nowrap">never</td><td class="sp-actions">`,
`<form method="post" action="/inbound/filter-lists/3/delete"><button type="submit" class="has-text-danger">Remove</button></form>`,
`<h2>Where the lists go</h2><div class="sp-end"><span class="tag is-success is-light">applied</span></div>`,
`rspamd, from map files it rereads within seconds of a change`,
`<span class="sp-mono">/data/antispam/</span> — six, one for each list and kind of entry`,
`2026-09-11 08:40 UTC`, `Deny wins, whichever entries they are.`,
)
if got := strings.Count(out, `<table`); got != 2 {
t.Errorf("%d tables, want the two lists", got)
if got := strings.Count(out, `<table class="table is-fullwidth is-hoverable">`); got != 2 {
t.Errorf("%d list tables, want the two lists", got)
}
pageLacks(t, "Filter lists", out, `<th>Note</th><th>`, `<th>Added</th><th>`)
}
// With no filter attached the lists are still the operator's to edit, and the
@@ -213,8 +222,8 @@ func TestFilterListsOffAndEmpty(t *testing.T) {
`No sender is turned away by hand.`,
)
pageLacks(t, "Filter lists", out, `rspamd, from map files`, `is-success is-light">applied`, `No sender is let through by hand.`)
if got := strings.Count(out, `<table`); got != 1 {
t.Errorf("%d tables, want the one list that has entries", got)
if got := strings.Count(out, `<table class="table is-fullwidth is-hoverable">`); got != 1 {
t.Errorf("%d list tables, want the one list that has entries", got)
}
}
@@ -1,7 +1,8 @@
{{/* Inbound > Filter lists (mockup in-filter-lists.html): the allow and deny
lists that correct the spam filter by hand, for the whole instance, and
where they go. Global role only. The form adds an entry; each row of a list
removes its own. */}}
removes its own, says how often the entry decided a message, and opens a
sheet with the entry's note from the tag of its kind. */}}
{{define "content"}}
{{template "page_head" .Head}}
{{if .Flash}}{{template "flash" .Flash}}{{end}}
@@ -54,9 +55,9 @@
table, or what is absent when the list is empty. */}}
{{define "filter_list"}}{{template "box_open" .Box}}
{{if .Rows}}<div class="table-container"><table class="table is-fullwidth is-hoverable">
<thead><tr><th>Value</th><th>Note</th><th>Added</th><th></th></tr></thead>
<thead><tr><th>Value</th><th>Kind</th><th>Matched</th><th></th></tr></thead>
<tbody>{{range .Rows}}
<tr><td class="sp-mono sp-small">{{.Value}} {{template "tag" .Kind}}</td>{{if .Note}}<td class="sp-muted">{{.Note}}</td>{{else}}<td class="sp-muted">—</td>{{end}}<td class="sp-muted sp-nowrap sp-small">{{.Added}}</td><td class="sp-actions">{{template "row_action" .Remove}}</td></tr>{{end}}
<tr><td class="sp-mono sp-small">{{.Value}}</td><td>{{template "sheet_tag" .Entry}}</td>{{if .Matched}}<td class="sp-nowrap">{{.Matched}}</td>{{else}}<td class="sp-muted sp-nowrap">never</td>{{end}}<td class="sp-actions">{{template "row_action" .Remove}}</td></tr>{{end}}
</tbody>
</table></div>
{{else}}{{template "empty_state" .Empty}}