audit 3: every per-message question of the send log has its index
test / test (push) Canceled after 0s

Migration 0010 (DDL only; 0009 went to stage 2b). EXPLAIN QUERY PLAN of
every send_log statement showed, beyond M2 as written (stage 2 had already
moved the by-domain counts onto (domain_id, created_at)):

- the tailer's sweep of open rows (every 5 min) and its count scanned the
  table: partial idx_send_log_open (created_at) WHERE status IN ('queued',
  'deferred'), the statuses literal in the statements, ORDER BY +id (else
  SQLite walks the table in id order to skip a sort);
- AttachRefusalLine's NOT EXISTS (... refusal_line = ?1) scanned the table
  for every refusal line: partial idx_send_log_refusal_line, the query
  carrying the index's WHERE; its row found by idx_send_log_rejected
  (status literal, +queue_id) instead of every refusal without a queue id;
- an application's level-2 count went through its domain's index:
  +domain_id in appByLogin;
- a page of one domain's or one application's log sorted all of it:
  idx_send_log_domain_id, idx_send_log_app_login (id order; several domains
  still sort, top-N). idx_send_log_domain stays for the global filter.

The statements are constants and builders (countMessagesQuery,
peakPerHourQuery, listOpenQuery, countOpenQuery, sendLogPageQuery,
attachRefusalLineQuery, limitRefusalsQuery) and
TestSendLogQueriesUseTheirIndexes explains exactly those: the index each
must use, no SCAN of send_log, no sort where none is wanted; the literal
statuses checked against the constants. It replaces the hand-written plan
check of TestLimitRefusalsSince.

Tested on the test server: go vet + go test of 24 packages, make e2e (all
pass) on this tree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
mix
2026-10-15 10:10:52 +03:00
co-authored by Claude Opus 5.5
parent 8e3241ef98
commit 508ac96c79
13 changed files with 268 additions and 89 deletions
+27 -17
View File
@@ -89,7 +89,31 @@ const domainByName = "domain_id = (SELECT d.id FROM domains d WHERE d.name = ?)"
// one more than the largest number already assigned; modernc.org/sqlite
// v1.60.1, conn.go bind() and argFinder.find(): ?NNN takes argument NNN, a
// plain ? the argument of its number.
const appByLogin = "app_login = ?1 AND domain_id = (SELECT a.domain_id FROM applications a WHERE a.login = ?1)"
//
// The unary + keeps domain_id out of the choice of index: the rows are found
// by the login and the time (idx_send_log_app_login_created_at), not among
// all of the domain's (https://sqlite.org/optoverview.html § 18,
// "Disqualifying WHERE clause terms using unary-+"; audit 2026-10-12, stage 3).
const appByLogin = "app_login = ?1 AND +domain_id = (SELECT a.domain_id FROM applications a WHERE a.login = ?1)"
// countMessagesQuery counts the distinct messages of the rows match selects
// (domainByName or appByLogin, then the status to leave out and the start of
// the window): the level-2 count and the statistics' total ask it alike.
func countMessagesQuery(match string) string {
return `SELECT COUNT(DISTINCT queue_id) FROM send_log
WHERE ` + match + ` AND status != ? AND created_at >= ?`
}
// peakPerHourQuery is the busiest UTC hour of the same rows: substr counts
// characters from 1 (https://sqlite.org/lang_corefunc.html#substr), so 1-13
// of an RFC3339 UTC stamp is YYYY-MM-DDTHH.
func peakPerHourQuery(match string) string {
return `SELECT COALESCE(MAX(bucket_count), 0) FROM (
SELECT COUNT(DISTINCT queue_id) AS bucket_count FROM send_log
WHERE ` + match + ` AND status != ? AND created_at >= ?
GROUP BY substr(created_at, 1, 13)
)`
}
// sendStats counts the rows that match, a condition with one placeholder
// taking ref.
@@ -97,27 +121,13 @@ func (s *Store) sendStats(match, ref string, since time.Time, hours float64) (Se
sinceStr := since.UTC().Format(time.RFC3339)
var total int64
err := s.db.QueryRow(
`SELECT COUNT(DISTINCT queue_id) FROM send_log
WHERE `+match+` AND status != ? AND created_at >= ?`,
ref, StatusRejected, sinceStr,
).Scan(&total)
err := s.db.QueryRow(countMessagesQuery(match), ref, StatusRejected, sinceStr).Scan(&total)
if err != nil {
return SendStats{}, fmt.Errorf("send stats total %q: %w", ref, err)
}
// The busiest UTC hour: substr counts characters from 1
// (https://sqlite.org/lang_corefunc.html#substr), so 1-13 of an RFC3339
// UTC stamp is YYYY-MM-DDTHH.
var peak int64
err = s.db.QueryRow(
`SELECT COALESCE(MAX(bucket_count), 0) FROM (
SELECT COUNT(DISTINCT queue_id) AS bucket_count FROM send_log
WHERE `+match+` AND status != ? AND created_at >= ?
GROUP BY substr(created_at, 1, 13)
)`,
ref, StatusRejected, sinceStr,
).Scan(&peak)
err = s.db.QueryRow(peakPerHourQuery(match), ref, StatusRejected, sinceStr).Scan(&peak)
if err != nil {
return SendStats{}, fmt.Errorf("send stats peak %q: %w", ref, err)
}