mix/selfpost
1
0
Fork 0
Code Issues Pull Requests Actions 21 Packages Projects Releases Wiki Activity
136 Commits 1 Branch 3 Tags
c817f0833cd834ddaaab89161b0c61b620d1136e
Commit Graph
13 Commits
Author SHA1 Message Date
mixandClaude Opus 5.5 c817f0833c from-header-binding: what the reviews had left as found, and address extensions
test / test (push) Canceled after 0s
Details 
The owner on the three things the reviews' commit left as found: can they be
fixed? They can.

- Downtime that crossed a UTC midnight was all booked on the day of the
  restart: it is cut at the midnights between (byDay).
- A message with the null envelope sender - a bounce an application writes
  itself - left a refusal row without a domain, which no domain
  administrator would see: the row takes the domain the login is bound to.
- A log read again after an unclean stop counted its deferrals again: while
  the tailer catches up, a warning no later than the last one counted is
  passed over. The mark is kept to the fraction of a second the log gives.

And on the check being stricter than Postfix for an address with an
extension (invoices+x@ passed in the envelope, was refused in the header):
it makes Postfix's second try now, the address without its extension.
recipient_delimiter was the distribution's default; it is set in
postfix-config.sh, since the check depends on it. e2e sends one through.

Test server, this tree: go vet ok, go test 23 packages ok, make e2e ok (339 s).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 21:29:17 +03:00
mixandClaude Opus 5.5 89fd0a6116 from-header-binding: findings of the technical and security reviews
test / test (push) Canceled after 0s
Details 
Both reviews were of c171fb5, read-only, given the code and questions.

Found by both, and the one that would have refused ordinary mail from the
first day: a folded From: line - what a mail library makes of a long display
name - did not parse and was refused with 550. The line is unfolded before it
is read. e2e sends one through Postfix now.

Technical (Opus)
- A display name in a charset Go does not convert (windows-1251, koi8-r) was
  refused: such a name is passed through unread, only the address is judged.
- The statistics ran between binding the socket and serving it, with a
  supervisorctl that has no timeout: they are kept beside the check now and
  nothing of them is waited for. A supervisorctl that fails is logged.
- The count of deferred mail followed smtpd's reject lines and so missed a
  check that stops answering in the middle of a message, where cleanup
  writes them: it is taken at Postfix's warning, the line that names the
  socket. The test line for that case had been invented with the wrong
  process; it is marked as composed from Postfix's source.
- A card whose figures could not be read showed zeros and "ok": dashes and
  "unknown".
- entrypoint removes the sockets a killed container left, which satisfied
  the wrapper's wait before the check was listening.
- Outside quoted strings the line may hold one "@" only - a rule that needs
  no knowledge of OpenDKIM's parser.
- Comments that still spoke of two sockets, or of a session replaced after a
  refusal; the limits of the cards layout on a phone, said where it is defined.

Security (Fable; no Critical or High)
- Another sender's name in an encoded display name passed the second look:
  the decoded name is looked at as well.
- A client could arm the deferred counter by sending the socket's name where
  Postfix quotes it in a warning: the two warnings are matched from their
  first word.
- A refused message wrote a row per recipient: one row, naming the first
  recipient and how many more.
- Docs: the check's lookup is stricter than Postfix's for address extensions
  and the case of logins, rather than "cannot disagree"; what a display name
  looks like to a person is not promised; the timeouts and the 454 at AUTH
  stated as they are; "messages" are connections turned away.

Left as found, in the plan: downtime that crosses midnight is booked on the
day of the restart; a message with the null envelope sender leaves a row
without a domain; a log read again after an unclean stop counts again.

Test server, this tree: go vet ok, go test 23 packages ok, make e2e ok (330 s).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 20:06:02 +03:00
mixandClaude Opus 5.5 c171fb5bc7 from-header-binding: the sender check, what it costs, its pages; guide, architecture, security
test / test (push) Canceled after 0s
Details 
A message's From: line is held against the application that sends it, under
the rule Postfix holds the envelope sender to. OpenDKIM chooses the key by
that line, so until now an application could have mail signed as another
domain of the server or as the server itself.

The check (internal/milter/sender.go)
- A role of the panel on a socket of its own (SENDER_CHECK_SOCKET), first in
  smtpd_milters and strict: default_action=tempfail. The journal milter keeps
  its socket and stays fail-open. postfix-wrapper waits for the new socket.
- It reads no database: what an application may send as comes from Postfix's
  own sender_login_maps, what the server signs for from OpenDKIM's
  SigningTable. Both are read again when they change; one that cannot be
  read defers mail.
- One From: field, one address, allowed by the envelope's lookup; then no
  other name of the SigningTable may appear anywhere in the raw line.
  Refused: 550 5.7.1 naming the line, reduced to printable ASCII.
- A refusal is a send-log row with reason "from" and the header address
  (migration 0006), written by another goroutine and not waited for. The
  journal's refusals record their reason too (limit, client), and the
  notifier takes only a limit's.

What it costs (docs/plans/from-header-binding.md, Statistics)
- The role marks itself alive every 2 s; at start the gap since the mark is
  downtime, counted from Postfix's start if later and not at all when
  supervisord says Postfix is not running.
- The log tailer counts the connections Postfix turned away: a warning that
  names the socket, then milter-reject "Service unavailable" from the same
  smtpd. Seen on the stand: the client is refused from CONNECT on and leaves
  at AUTH (454), so no MAIL line is ever written.
- sender_check_days, a row per UTC day, kept a year.

Pages
- Health: the card "Sender check", the socket's row, and the cards layout -
  boxes numbered by importance in two stacks read in rows; the left stack
  polls, the three readings of the right one are swapped in by id.
- Outbound log: a refused row says why in its tag (wrong From, wrong client,
  rate-limited); the message's page shows the From: line and what the
  application may send as.
- Help: topic "Sender check", word for word the mockup's.

Found by the stand, fixed here: the names for the second look came from the
sender bindings, and a domain with a key and no application yet was not
among them.

e2e: negative_journal_milter_fail_open is gone - it proved that mail passes
with the panel down, which is no longer so. In its place
sender_check_holds_the_from_line (another domain's From:, the hostname's, a
stranger's, an address off the list, another sender in the display name -
refused; own addresses - delivered) and panel_down_defers_mail (4xx with the
panel stopped, the container alive, mail again after, downtime and one
deferred message on Health, the card warning).

Test server, this tree before the last cosmetic edits (a fixture row, a
fact's size, docs): go vet ok, go test 23 packages ok, make e2e ok (337 s).
Locally after them: internal/web/view ok.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 19:35:43 +03:00
mixandClaude Opus 5.5 a990e7cc23 delivery-log-storage: findings of the technical and security reviews
test / test (push) Canceled after 0s
Details 
Neither review (read-only agents, Opus and Fable, over ae06d8b..815236a) has
a Critical or High finding. What they found and what was done:

- A stranger on port 25 could grow the table: every queue-id line of inbound
  mail was stored and pruned an hour or more later, his own message-id among
  them. The tailer now leaves out the lines of a queue id that
  postfix/inbound/smtpd opened, until qmgr removes the message or another
  listener opens one under the id; 4096 such ids are remembered.
- A commit, so a disk sync, per stored line on the connection the journal
  milter shares, and a backlog written line by line. Lines are now written in
  one transaction per pass over the log, 500 at most, before the read offset
  is saved (InsertSendLogLines; follow() calls the keeper's flush).
- The prune asked the send log once per line and ran hourly. It asks once per
  queue id and runs at start and with the six-hourly sweep; with port-25 mail
  left out there is little without a row to wait for.
- The 4096-byte cut could split a character; it keeps characters whole.
- security.md did not say that a backup now carries these lines, and the plan
  said the time window excludes a reused queue id where it only limits the
  chance to the row's queue lifetime. Both are corrected; the collision is
  recorded as an accepted risk with its trigger.
- Tests that did not prove their comments: the prune is now tested with the
  sweep, a rejected row next to the prune, batching and the change from
  catching up to following, port-25 mail left out and its id reused.

Left as found and written down in the plan: lines the tailer never reads
(a file rotated away while the panel was down), and a message in the queue
across the upgrade, which shows only its later lines.

On the test server, for this tree: go vet clean, go test ./... 22 packages
ok, make e2e ok.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 10:41:04 +03:00
mixandClaude Opus 5.5 81df131217 delivery-log-storage: the tailer keeps the lines of queued messages 
Every mail.log line headed by a queue id is written to send_log_lines as the
tailer reads it, dated by the line's own stamp. The id is taken from the head
of the line only - the tail is the far end's text - and has to look like one:
"NOQUEUE:" and "warning:" stand in the same place.

The tailer does not check for a send-log row first: a message's first lines
come before the journal milter writes it. Lines that never get a row are
removed by the prune, which deletes lines older than an hour whose queue id
no row carries. It runs at start, after each send-log sweep and hourly.

While the tailer catches up at start a line is written only if it is not
there already, as the inbound journal does: its place is saved every few
seconds, so an unclean stop means reading some lines twice.

Checked on Windows: go vet clean; go test ./internal/logtail passes except
TestFollowTailsAndRotates, which fails there on renaming an open file and
always did. New tests: which line heads are queue ids, a line dated by its
stamp, no second copy after a rewound offset, the prune cadence and cutoff.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 10:11:28 +03:00
mixandClaude Opus 5.5 89774ab847 inbound-antispam-panel: three things the reviews left as found
test / test (push) Canceled after 0s
Details 
The owner asked for them to be put right (2026-10-12).

  - A refusal is not logged twice after an unclean stop. The tailer
    saves its place every five seconds, so on start it reads some
    lines again; for the lines it reads on start a row is written only
    if the same row is not there already (store.InsertInboundSpamOnce:
    same moment, client, sender, recipient, decision, engine, detail).
    Lines read as they are written are always new, so two identical
    refusals in one second stay two rows.
  - The Inbound log offers Deny / Allow sender only for a sender the
    lists would take (inbound.Listable): an envelope sender may be
    spelt in ways the form refuses, and the link led to a refusal.
  - A network of one address is stored as the address: 192.0.2.7/32
    and 192.0.2.7 were two entries of one list.

Checked on the test server, uncommitted tree: go vet ./... ok,
go test ./... ok (22 packages), make e2e ok.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 08:17:23 +03:00
mixandClaude Opus 5.5 c731a98a0f inbound-antispam-panel: findings of the technical and security reviews
test / test (push) Canceled after 0s
Details 
Steps 11 and 12 of docs/plans/inbound-antispam-panel.md: 12/12. Two
read-only reviews of d65d632..HEAD by different models (Opus,
technical; Fable, security), each given the code and questions, not
conclusions. Nothing Critical or High in the feature; the High the
technical review found outside it is the commit before this one.

Log tailer (internal/logtail/inbound.go):
  - the reply of a refusal ends at the first "; from=<", Postfix's own,
    and the HELO name after it is whatever is left. Taking the longest
    reply let a HELO name that spelt "; from=<..> to=<..> proto=.."
    supply the row's sender, recipient and reply - and so make a
    refusal read as the deny list's;
  - an address with a quote, a bracket or a space is not kept: a quoted
    sender could otherwise supply the recipient;
  - the journal is cut to its row cap every 500 rows written, not only
    every six hours;
  - a queue id is forgotten when the message is queued or the id is
    opened by another listener, and a stale place in the queue of
    arrivals no longer drops a live id;
  - milter-hold lines are not read: the held message reaches the
    journal milter, which writes its row - it was written twice.
Lists (internal/inbound/lists.go): a change of the table and the
rewrite of the files are one step under a mutex; a failed rewrite puts
a removed entry back and rewrites after taking a new one out.
Store: the row cap removes the oldest by time, as the page orders them.
Page: on a sheet a refusal's text is a Reply whatever it starts with.

Help and the guide say that an allow entry for a sender address trusts
an address anyone can claim (design commit before this one).

docs/security.md: the two reviews; the roles table gains Inbound > Log
and Filter lists; Mail path describes the milters, the tailer and the
lists; four accepted risks - the filter's verdict while it is down, an
allow entry by sender address, the list files' mode, log rows held by
domain name. The plan's stale lines (size and rate refusals, who reads
the journal) are corrected.

Left as found, told to the owner: the journal's rows after an unclean
stop may be written twice (the tailer saves its place every five
seconds); a "Deny sender" link is offered for an address the form then
refuses; 1.2.3.4 and 1.2.3.4/32 are two entries.

Checked on the test server, uncommitted tree: go vet ./... ok,
go test ./... ok (22 packages), make e2e ok.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 07:51:50 +03:00
mixandClaude Opus 5.5 27fd355e53 inbound-antispam-panel: refused inbound mail from mail.log; retention 
Step 4/12 of docs/plans/inbound-antispam-panel.md.

The log tailer writes an inbound_spam_log row for mail port 25 refused
(internal/logtail/inbound.go): a "reject:" line of the port-25 smtpd is
Postfix's own policy; a milter-reject / milter-hold / milter-discard
line of cleanup is the filter's verdict. Only a recipient at an inbound
domain is a row, so "Relay access denied" for a stranger's domain is
not. The row is dated by its log line; the reply text is cut to 255
bytes of printable ASCII; a reply starting with 4 is a tempfail.

To tell port 25 from 465 / 587 the listener now logs as
postfix/inbound (syslog_name). cleanup logs under one name for every
listener, so its verdict counts only for a queue id the port-25 smtpd
was seen to open (4096 remembered). Lines are matched from the head,
after timestamp and host: their tail is the sender's own text.

Retention: PruneInboundSpam every six hours - older than the
inbound_spam_log_retention_days setting (seeded from
INBOUND_SPAM_LOG_RETENTION_DAYS, 90, range 7-365), then down to 10 000
rows. The setting has no field in the panel yet.

Known limits, in architecture.md: a log line names one recipient, so a
message refused at end-of-message is one row; a line re-read after an
unclean stop would be a second row.

Checked on the test server, uncommitted tree: go vet ./... ok,
go test ./... ok (22 packages), make e2e ok - the unknown recipient at
the inbound domain refused on port 25 has its row (reject by postfix),
the stranger's domain has none.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-11 22:42:25 +03:00
mix 60b743e6c2 release: 1.6.0: 30-day send stats and optional auto level-2 rate limits 
30-day sending statistics per domain and application, plus optional auto level-2 rate limits derived from average send rate.
Added:
  - Domain page **Sending statistics** card: total messages, peak and average
  - Level-2 rate limit **Manual** / **Auto** mode on domain and application
  - Domain export JSON now includes rate limits (`mode`, `auto_multiplier`,
Changed:
  - Auto mode uses the level-1 window (`RATE_LIMIT_WINDOW_SECONDS`); manual

Full detail: CHANGELOG.md (1.6.0).
 2026-09-22 15:24:52 +03:00
mix dca7d99c1b release: 1.5.0: Send-log retention configurable in panel Settings 
Send-log retention is configurable from panel **Settings** (global administrator); `SEND_LOG_RETENTION_DAYS` seeds the initial default only.
Added:
  - Panel **Settings** (global administrator): **Send log retention (days)** —
Changed:
  - Deliveries list and delivery detail pages show the configured retention

Full detail: CHANGELOG.md (1.5.0).
 2026-09-22 15:24:52 +03:00
mix 75207f0d1e release: 1.0.0: Public 1.0 image; delivery log; docs and SECURITY.md (incl. 1.0.1) 
[1.0.0]
Added:
  - A **Delivery log** on each delivery's page (`/deliveries/{id}`): the
  - A **History** block on the same page: the journal's two timestamps stated as
Fixed:
  - Postfix config: copy TLS cert/key from the (often `:ro`, host-owned) mount
  - Postfix config: allow `maillog_file` under `/data` via
  - E2e: read `/data/setup-token` via `docker compose exec` (file is `0600`
  - Entrypoint: check `SELFPOST_HOSTNAME` before `/data` setup (so a bad identity
  - E2e gate: wait for host-published `/healthz` before panel setup, and stop
  … (3 more in CHANGELOG)
Changed:
  - Document what `.spbk` and `.spde` stand for (SelfPost backup / SelfPost domain
  - Docs aligned with the code: setup URL is `/setup/<token>` (README and
  - Full backups no longer carry `/data/log`. It is Postfix's raw log plus its
  - Monitoring screens (status, mail queue, system log, deliveries) use adaptive
  - Documentation package consolidated into `docs/development.md`: Documentation
  … (4 more in CHANGELOG)
[1.0.1]
A documentation and packaging release: no change to the mail path, the database, or the on-disk layout. Upgrading is a tag bump.
Added:
  - `SECURITY.md` — how to report a vulnerability privately (GitHub private
  - [docs/plans/](docs/plans/) — one document per agreed extension: the optional
Changed:
  - The panel's **Account** entry is now called **Settings** — nav link, page
  - The signed-in name in the panel's nav is now labelled `User:`, so it reads as
  - [docs/product.md](docs/product.md) reframes the future line: the inbound
  - [docs/security.md](docs/security.md) is now in English, matching the rest of
  - The README documentation table now points at `SECURITY.md` for reporting a
  … (2 more in CHANGELOG)
Fixed:
  - The panel's static assets are served with a content ETag and

Squashed CHANGELOG sections: [1.0.0], [1.0.1].

Full detail: CHANGELOG.md (1.0.0, 1.0.1).
 2026-09-22 15:24:51 +03:00
mix 5e5d157496 release: 0.5.0: Pre-1.0 security review, domain-admin role, doc consolidation 
A bounce could be recorded as a successful delivery. The log-tailer's
Fixed:
  - A bounce could be recorded as a successful delivery. The log-tailer's
  - Log-tailer resumes where it stopped instead of jumping to end-of-file on
  - Level-2 rate limit no longer overshoots under concurrency: messages that
Changed:
  - The project has a single public home: `github.com/mixeme/selfpost`. Codeberg
  - Code comments no longer cite the archived specification. References like
  - `docs/code-review.md` is gone. Its plan is finished — phases 0 (bar the
  - `docs/architecture.md` gained a *Code layers* section: a diagram of
  - Phase 1 doc/code hygiene (`docs/code-review.md`): removed ~30 stale
  … (1 more in CHANGELOG)
Security:
  - Pre-release security review (plan § D, model Fable, 2026-08-06): full pass
  - `saslpasswd2` argv: the application login is now passed after a `--`
Added:
  - Optional password encryption for the two secret-bearing downloads (plan
  - Domain import accepts an encrypted export: the envelope is detected by its
  - `selfpost-backup` writes encrypted archives and reads them back:
  - `TestParseDelivery` covers the exotic mail.log shapes the review asked for
  - docs: README *Encrypting a backup or export*; `docs/security.md` §
  … (14 more in CHANGELOG)
Removed:
  - docs: `security.md` — accepted risk «restore old backup revives session rows»
Changed:
  - docs: `implementation-plan.md` trimmed to the sole open v1.x gate — pre-release
  - docs: `architecture.md` — sessions (SQLite, idle renew, password change),
  - docs: `development.md` — expanded e2e stack and `release.yml` matrix workflow.
  - docs: `security.md` — accepted risk for send-log rows stuck at `queued` after
  - docs: `roadmap.md` — optional send-log / `mail.log` follow-ups under v1.x tail.
  … (10 more in CHANGELOG)

Full detail: CHANGELOG.md (0.5.0).
 2026-09-22 15:24:51 +03:00
mix 8ce050ef7f release: 0.1.0: Phases 0-11: panel, Postfix/DKIM, milter, backup, Docker image 
Initial feature-complete implementation of the v1.0 specification (phases 0-11 of `docs/implementation-plan.md`).
Added:
  - Panel (Go, single static binary) with SQLite persistence, one-time
  - Domain management with per-domain DKIM (RSA-2048, generated in pure Go) and
  - Application (sender identity) management: SASL credentials via `sasldb2`,
  - Full Postfix relay config generated from env at container start: SMTPS 465,
  - Journal milter (pure Go, `go-milter`) recording every send to `send_log`;
  … (6 more in CHANGELOG)

Full detail: CHANGELOG.md (0.1.0).
 2026-09-22 15:24:51 +03:00