audit 3: every per-message question of the send log has its index
test / test (push) Canceled after 0s

Migration 0010 (DDL only; 0009 went to stage 2b). EXPLAIN QUERY PLAN of
every send_log statement showed, beyond M2 as written (stage 2 had already
moved the by-domain counts onto (domain_id, created_at)):

- the tailer's sweep of open rows (every 5 min) and its count scanned the
  table: partial idx_send_log_open (created_at) WHERE status IN ('queued',
  'deferred'), the statuses literal in the statements, ORDER BY +id (else
  SQLite walks the table in id order to skip a sort);
- AttachRefusalLine's NOT EXISTS (... refusal_line = ?1) scanned the table
  for every refusal line: partial idx_send_log_refusal_line, the query
  carrying the index's WHERE; its row found by idx_send_log_rejected
  (status literal, +queue_id) instead of every refusal without a queue id;
- an application's level-2 count went through its domain's index:
  +domain_id in appByLogin;
- a page of one domain's or one application's log sorted all of it:
  idx_send_log_domain_id, idx_send_log_app_login (id order; several domains
  still sort, top-N). idx_send_log_domain stays for the global filter.

The statements are constants and builders (countMessagesQuery,
peakPerHourQuery, listOpenQuery, countOpenQuery, sendLogPageQuery,
attachRefusalLineQuery, limitRefusalsQuery) and
TestSendLogQueriesUseTheirIndexes explains exactly those: the index each
must use, no SCAN of send_log, no sort where none is wanted; the literal
statuses checked against the constants. It replaces the hand-written plan
check of TestLimitRefusalsSince.

Tested on the test server: go vet + go test of 24 packages, make e2e (all
pass) on this tree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
mix
2026-10-15 10:10:52 +03:00
co-authored by Claude Opus 5.5
parent 8e3241ef98
commit 508ac96c79
13 changed files with 268 additions and 89 deletions
+15 -13
View File
@@ -168,6 +168,20 @@ type LimitRefusal struct {
First time.Time
}
// limitRefusalsQuery is LimitRefusalsSince's. The status is a literal so
// that the partial index of migration 0005 answers the question: the
// query's term matches the index's WHERE exactly
// (https://sqlite.org/partialindex.html § 3) when the statement is first
// prepared.
const limitRefusalsQuery = `SELECT d.name, a.login, COUNT(*), MIN(l.created_at)
FROM send_log l
JOIN domains d ON d.id = l.domain_id
JOIN applications a ON a.login = l.app_login AND a.domain_id = d.id
WHERE l.status = 'rejected' AND l.created_at >= ? AND l.reason IN ('', 'limit')
AND (EXISTS (SELECT 1 FROM rate_limits r WHERE r.scope = 'application' AND r.ref_id = a.id)
OR EXISTS (SELECT 1 FROM rate_limits r WHERE r.scope = 'domain' AND r.ref_id = d.id))
GROUP BY d.name, a.login ORDER BY d.name, a.login`
// LimitRefusalsSince returns, per sending domain, the messages the journal
// milter refused under a level-2 limit at or after since.
//
@@ -183,19 +197,7 @@ type LimitRefusal struct {
// address or its From: line is left out by that; a row from before has no
// reason and is still judged the old way.
func (s *Store) LimitRefusalsSince(since time.Time) ([]LimitRefusal, error) {
// The status is a literal so that the partial index of migration 0005
// answers the question: the query's term matches the index's WHERE
// exactly (https://sqlite.org/partialindex.html § 3) when the statement is
// first prepared.
rows, err := s.db.Query(
`SELECT d.name, a.login, COUNT(*), MIN(l.created_at)
FROM send_log l
JOIN domains d ON d.id = l.domain_id
JOIN applications a ON a.login = l.app_login AND a.domain_id = d.id
WHERE l.status = 'rejected' AND l.created_at >= ? AND l.reason IN ('', 'limit')
AND (EXISTS (SELECT 1 FROM rate_limits r WHERE r.scope = 'application' AND r.ref_id = a.id)
OR EXISTS (SELECT 1 FROM rate_limits r WHERE r.scope = 'domain' AND r.ref_id = d.id))
GROUP BY d.name, a.login ORDER BY d.name, a.login`,
rows, err := s.db.Query(limitRefusalsQuery,
since.UTC().Format(time.RFC3339),
)
if err != nil {