mix/selfpost
1
0
Fork 0
Code Issues Pull Requests Actions 21 Packages Projects Releases Wiki Activity

logtail: count the message a hung sender check defers in cleanup (audit M13)

Browse Source 
Seen on the e2e stand, the panel frozen with SIGSTOP once DATA was
answered: the client gets 451 4.7.1 after 30 s, and the warning that names
the socket is cleanup's, not smtpd's:

  postfix/cleanup[1339]: warning: milter unix:/run/selfpost/sender.sock:
    can't read SMFIC_HEADER reply packet header: Connection timed out
  postfix/cleanup[1339]: 7BCAF3C75DA0: milter-reject: END-OF-MESSAGE ...
    4.7.1 Service unavailable - try again later; ...

The counter read smtpd lines only, so Health's "messages deferred" missed
the one case that tells a hung panel from a stopped one. milterLineRe now
takes smtpd and cleanup; cleanup's "milter unix:<socket>:" warning counts a
message each time (one cleanup process serves message after message, and the
first warning ends the message); smtpd keeps once per connection. The
composed fixture is replaced by the seen lines, with two messages through one
cleanup process and OpenDKIM hung in cleanup (not counted).

e2e panel_hung_mid_message_is_counted: freezes the panel after DATA's 354,
wants a 4xx at the end of DATA, logs what mail.log says, and waits for
Health's today count to grow. kill through sh -c: the image has no kill(1).

CHANGELOG [Unreleased], audit plan line ticked, roadmap 13/61.
Evidence on the test server for this tree (with the next commit's docs):
go vet clean, go test ./... 23 packages ok, make e2e exit 0 -
renewed_certificate_picked_up 1.2 s, panel_hung_mid_message_is_counted
31.7 s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
mix
2026-10-14 14:42:01 +03:00
co-authored by Claude Opus 5.5
parent 7af37654a1
commit db36aa2c0e
7 changed files with 185 additions and 30 deletions
Download Patch File Download Diff File Expand all files Collapse all files
internal/logtail/sendercheck_test.go
+17 -11
View File
@@ -24,11 +24,11 @@ func (d *deferredStore) AddSenderCheckDeferred(at time.Time, n int64) error {
}
// The lines of process 1183 and of cleanup are as the e2e stand wrote them
// with the panel stopped and for a refused From: (2026-10-12; the stamps are
// put in ISO form). Around them are the lines of two other things that use
// the same words - a level-2 refusal of the journal milter, OpenDKIM not
// answering - and, composed from Postfix's source rather than seen, a check
// that stops answering in the middle of a message.
// with the panel stopped and for a refused From: (2026-10-12), and with the
// panel frozen in the middle of a message (cleanup 1339, 2026-10-14); the
// stamps are put in ISO form. Around them are the lines of two other things
// that use the same words - a level-2 refusal of the journal milter, OpenDKIM
// not answering.
func TestDeferredCounter(t *testing.T) {
st := &deferredStore{}
c := newDeferredCounter(&SenderCheck{Store: st, Socket: "/run/selfpost/sender.sock"})
@@ -61,11 +61,17 @@ func TestDeferredCounter(t *testing.T) {
`2026-10-12T16:24:00.100000+00:00 mail postfix/smtpd[1201]: warning: connect to Milter service unix:/run/opendkim/opendkim.sock: Connection refused`,
`2026-10-12T16:24:00.200000+00:00 mail postfix/smtpd[1201]: NOQUEUE: milter-reject: CONNECT from unknown[10.77.0.1]: 451 4.7.1 Service unavailable - try again later; proto=SMTP`,
// The check stops answering in the middle of a message, the next day.
// The check stops answering in the middle of a message, the next day:
// cleanup meets it. Two messages through the same cleanup process are
// two messages.
`2026-10-13T00:00:05.000000+00:00 mail postfix/smtpd[1300]: connect from unknown[10.77.0.1]`,
`2026-10-13T00:00:21.000000+00:00 mail postfix/smtpd[1300]: warning: milter unix:/run/selfpost/sender.sock: can't read SMFIC_EOH reply packet header: Connection timed out`,
`2026-10-13T00:00:21.050000+00:00 mail postfix/smtpd[1300]: warning: milter unix:/run/selfpost/sender.sock: can't read SMFIC_BODYEOB reply packet header: Connection timed out`,
`2026-10-13T00:00:21.100000+00:00 mail postfix/cleanup[1195]: 4XcB7k2Jm9z1: milter-reject: END-OF-MESSAGE from unknown[10.77.0.1]: 4.7.1 Service unavailable - try again later; from=<a@sender.e2e.test> to=<b@sink.e2e.test> proto=ESMTP helo=<x>`,
`2026-10-13T00:00:35.000000+00:00 mail postfix/cleanup[1339]: warning: milter unix:/run/selfpost/sender.sock: can't read SMFIC_HEADER reply packet header: Connection timed out`,
`2026-10-13T00:00:35.100000+00:00 mail postfix/cleanup[1339]: 7BCAF3C75DA0: milter-reject: END-OF-MESSAGE from unknown[10.77.0.1]: 4.7.1 Service unavailable - try again later; from=<alerts@sender.e2e.test> to=<rcpt@sink.e2e.test> proto=ESMTP helo=<e2e-client.e2e.test>`,
`2026-10-13T00:01:20.000000+00:00 mail postfix/cleanup[1339]: warning: milter unix:/run/selfpost/sender.sock: can't read SMFIC_HEADER reply packet header: Connection timed out`,
`2026-10-13T00:01:20.100000+00:00 mail postfix/cleanup[1339]: 8C1D43C75DA0: milter-reject: END-OF-MESSAGE from unknown[10.77.0.1]: 4.7.1 Service unavailable - try again later; from=<alerts@sender.e2e.test> to=<rcpt@sink.e2e.test> proto=ESMTP helo=<e2e-client.e2e.test>`,
// OpenDKIM hung in the middle of a message: another socket.
`2026-10-13T00:01:30.000000+00:00 mail postfix/cleanup[1339]: warning: milter unix:/run/opendkim/opendkim.sock: can't read SMFIC_HEADER reply packet header: Connection timed out`,
// The check's own verdict is a refusal, not a deferral, and it is
// cleanup that writes it.
@@ -73,8 +79,8 @@ func TestDeferredCounter(t *testing.T) {
} {
c.handle(line, now, false)
}
if len(st.days) != 2 || st.days["2026-10-12"] != 1 || st.days["2026-10-13"] != 1 {
t.Fatalf("deferred per day = %v, want one on each of the two days", st.days)
if len(st.days) != 2 || st.days["2026-10-12"] != 1 || st.days["2026-10-13"] != 2 {
t.Fatalf("deferred per day = %v, want one on the first day and two on the second", st.days)
}
}