docs: retire the test review plan, keep its durable findings

Every item in TEST_REVIEW_PLAN.md is done or decided, so the working document goes as its own header instructed. What outlives it moves to where a later reviewer will actually look: - TESTS.md gains the -coverpkg command and the 84.4% baseline (per-package figures understate the suite), design principle 9 (redundancy is judged by comparing coverage profiles, and identical coverage alone is not grounds for deletion), a table of the look-alike tests that are kept with the reason each survives, and the list of functions deliberately at 0%. - STANDARDS.md's "Intentional behavior" section points at both lists, so the mechanism REVIEW.md describes still reaches them. Dropped as spent: the per-item checklists, the suggested order, and the model-selection table. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-04 22:52:43 +03:00
parent 28f0a0d8e2
commit a735bfd116
3 changed files with 42 additions and 205 deletions
@@ -66,6 +66,10 @@ change to their shape has to stay compatible on its own.
 - **History tab is session-only.** `JobRuntime.Logs` exists only in memory for the
  current process. Log files on disk feed aggregate statistics via `SeedStats`
  only. See [ARCHITECTURE.md](ARCHITECTURE.md).
 - Several tests share a coverage profile with another test on purpose, and a few
  functions sit at 0% on purpose. Both lists live in
  [TESTS.md](TESTS.md) — check them before reporting a test as redundant or a
  coverage gap as an oversight.
 ## Out of scope
@@ -55,6 +55,17 @@ go test -coverprofile=coverage.out ./src/runner
 go tool cover -html=coverage.out
 ```
 Per-package coverage understates the suite, because several packages are
 exercised from another one's tests — `domain.NewRuntime`, for instance, is
 covered by the `app` tests. Measure the engine packages together instead:
 ```bash
 go test -coverpkg=./src/domain,./src/storage,./src/runner,./src/scheduler,./src/app ./src/domain ./src/storage ./src/runner ./src/scheduler ./src/app
 ```
 That figure was 84.4% at the 2026-08-04 review, which is the number to compare
 against before concluding that coverage has slipped.
 ---
 ## Test Files Overview
@@ -545,6 +556,23 @@ Tests main view construction with an injected `*app.Service`.
 8. **Geometry is measured, not eyeballed** — The `ui` tests that build widgets under `test.NewApp()` assert sizes and offsets, and several re-run under a scaled theme. That is what keeps [STANDARDS.md](STANDARDS.md)'s "measure at build time, never a pixel constant" rule enforceable rather than aspirational.
 9. **Redundancy is measured, not read** — Before deleting a test as a duplicate, run both in isolation with `-coverprofile` and compare the profiles. Identical coverage alone is *not* grounds for deletion: several kept tests hit the same statements while asserting genuinely different properties (see the table below). Deletion requires identical coverage **and** assertions that are a subset of the survivor's.
 ---
 ## Look-alike tests that are kept
 Every pair here has an identical coverage profile, so a redundancy pass will
 flag them again. They were measured under principle 9 and kept because the
 assertions differ — not because nobody looked.
 | Tests | Why both stay |
 |-------|---------------|
 | `TestSetGlobalPausePersistsToConfigFile` / `TestSetGlobalPauseUpdatesRuntimesAndEmits` | The first asserts the flag reaches `gosentry.json`, which is what makes the pause survive a restart; the second asserts the in-memory runtimes and the emitted event. |
 | `TestRunDueQueueDrainsMultipleOverlaps` / `TestRunDueQueueRerunsAfterFinish` | The first drains three queued occurrences rather than one, so it is the test that would catch a drain loop that fires only once. |
 | `TestCreateStartupShortcutHandlesCyrillicPath` / `TestCreateStartupShortcutHandlesSpaces` | Non-ASCII paths and paths with spaces are different real-world failure modes for the WScript.Shell COM call. |
 | `TestRunJobLogFileAllHeaders` / `TestRunJobRecordFields` / `TestRunJobWritesLogFile` | Three different subjects: the log file's headers, the returned `RunRecord`'s fields, and the log file's name and directory. The fixtures differ too — only `TestRunJobWritesLogFile` runs the `Manual` trigger. Merging them into one `RunJob` call was measured and declined: it saves ~90 ms (the three cost 0.14 s combined; the `runner` package's seconds are `TestRunJobTimesOut` and `TestRunJobZeroTimeoutMeansNoTimeout`, which wait on purpose) and would drop the `Manual` path from the header assertions. |
 ---
 ## Remaining Test Coverage Gaps
@@ -552,3 +580,13 @@ Tests main view construction with an injected `*app.Service`.
 - Full GUI E2E — tab navigation, dialog flows, and native file pickers are not exercised end-to-end; the `ui` tests assemble views and measure them, but nothing drives a real window.
 - History is session-only by design — `.log` files seed aggregate stats only, not the History table (see [STANDARDS.md](STANDARDS.md))
 - Fyne's headless driver cannot report a maximized window, which is why window-size persistence stays frozen in [ROADMAP.md](ROADMAP.md)
 ### Functions deliberately at 0%
 A coverage run over the non-UI packages reports these as uncovered. All are
 intentional; none is an oversight to be "fixed" with a test.
 - The real `Clock` — a fake is injected everywhere it is used.
 - `storage.OpenStore`, `storage.ResolvePaths`, `app.Service.Start`, `app.Service.Open` — process entry points, exercised by running the app.
 - The autostart and desktop-icon wrappers — OS integration, driven only on a real desktop.
 - `app.Service.ShouldNotifyOnFailure` — a getter under the mutex.
@@ -1,205 +0,0 @@
 # Test-suite review — action plan
 Working document for the findings of the 2026-08-04 review of the test suite.
 It is not part of the permanent doc set: delete it once every item below is
 either done or moved to [ROADMAP.md](ROADMAP.md).
 The rules the findings were judged against live in [STANDARDS.md](STANDARDS.md);
 the suite itself is described in [TESTS.md](TESTS.md).
 ## Baseline the review started from
 - 172 tests, 4693 lines of test code.
 - 84.4% statement coverage across `domain`, `storage`, `runner`, `scheduler`,
  and `app` measured together with `-coverpkg` (per-package figures understate
  it, because e.g. `domain.NewRuntime` is exercised from the `app` tests).
 - [TESTS.md](TESTS.md) documents 171 of the 172 tests.
 Overall finding: the suite is **not** padded. Every test but one carries a real
 assertion, and most record the property they pin. The items below are the
 exceptions.
 Method note: redundancy was not judged by reading. Each suspected pair was run
 in isolation with `-coverprofile` and the profiles compared. "Identical
 coverage" below means the two profiles were byte-identical after sorting.
 Identical coverage alone is *not* grounds for deletion — several kept tests hit
 the same statements while asserting genuinely different properties. Deletion
 requires identical coverage **and** assertions that are a subset.
 ## 1. Delete the measured duplicates
 Each of these has a byte-identical coverage profile with an existing test whose
 assertions are a superset. Roughly 30 lines total.
 - [x] `TestCleanupLogsKeepsFilesWithinAgeLimit`
      ([cleanup_test.go:53](../src/runner/cleanup_test.go)) — delete.
      `TestCleanupLogsRemovesFilesPastMaxAge` already asserts that the file
      inside the age limit survives.
 - [x] `TestRunDueEmptyOverlapInheritsGlobal`
      ([run_test.go:457](../src/app/run_test.go)) — delete. It builds the same
      service as `TestRunDueQueueRerunsAfterFinish` (parallel mode, global
      `queue`, a job with an empty `OverlapPolicy`) and asserts strictly less.
      Before deleting, move its one unique line — the setup guard
      `svc.jobs[0].OverlapPolicy != ""` — into `TestRunDueQueueRerunsAfterFinish`,
      so that test still states out loud that it is exercising the inherited
      policy rather than an explicit one.
 - [x] `TestSameWindowsPathHandlesSpaces`
      ([autostart_windows_test.go:20](../src/platform/autostart/autostart_windows_test.go))
      — delete. It is the same case as `TestSameWindowsPathIgnoresCaseAndQuotes`
      (quoted path, mixed case); `sameWindowsPath` does not split on spaces, so
      the space in the fixture reaches no new code. If the spaces case is worth
      naming, fold the path into the surviving test's fixture instead.
 After the deletions, re-run the affected packages and confirm coverage is
 unchanged:
 ```bash
 go test -coverpkg=./src/domain,./src/storage,./src/runner,./src/scheduler,./src/app ./src/domain ./src/storage ./src/runner ./src/scheduler ./src/app
 ```
 ## 2. Fix the documentation drift
 - [x] [TESTS.md](TESTS.md) claims `TestLoadOrCreateConfigCreatesDefaultsOnFirstRun`
      verifies that a missing config file is created "with sane defaults **and a
      sample job**". The test never touches jobs, and `storage.defaultJobs` sits
      at 0% coverage. Decide which half is wrong: either drop the claim from the
      table, or add the assertion that the seeded `jobs.json` contains the
      sample jobs. Adding the assertion is the better outcome — `defaultJobs` is
      the only accidental coverage gap the review found.
 - [x] [TESTS.md](TESTS.md) does not list
      `TestCancelRowOverlapAddsBackOneInnerPadding`
      ([layout_test.go:35](../src/ui/layout_test.go)). Add it to the
      `src/ui/layout_test.go` table.
 ## 3. Replace the hand-rolled helper in test code
 - [x] [seed_test.go:34](../src/runner/seed_test.go) defines `itoa`: 18 lines of
      digit-by-digit conversion with a fresh allocation per digit, in a file
      that already imports `strconv`. Replace the calls with
      `strconv.FormatInt` and delete the helper. Untested logic inside a test
      file is exactly what produces a test result nobody can trust.
 ## 4. Thin tests — decide, then act
 None of these is wrong; each is close enough to worthless that it should be
 either justified or removed. Grouped because they want one decision, not four.
 - [x] `TestEmitWithNoObserversIsNoop`
      ([events_test.go:36](../src/app/events_test.go)) — the only test in the
      suite with no assertion at all. Ranging over a nil slice cannot panic in
      Go, so it pins nothing. Delete.
 - [x] `TestStoreReturnsWiredStore`
      ([service_test.go:51](../src/app/service_test.go)) — asserts that a
      one-line getter returns its own field. Delete.
 - [x] `TestMainViewBuilds`
      ([mainwindow_test.go:72](../src/ui/mainwindow_test.go)) — a smoke test;
      `TestMainViewFitsTheDefaultWindowSize` builds the same view. Its only
      unique coverage is `w.SetContent(content)` and `recordStartup(0, true)`.
      Either fold those two calls into the sizing test and delete this one, or
      keep it and say in its comment that `recordStartup` is what it is for.
      Done as neither: folding an assertionless `recordStartup` call into the
      sizing test would have put unrelated work inside an F1/F3 regression
      guard. It became `TestMainViewRecordStartupAddsHistoryRow`, which calls
      the closure for both wordings `run.go` selects between and asserts the
      two rows arrive in the History table, read back through the table's own
      cell callbacks so the refresh is proved too. Same unique coverage, plus
      the previously uncovered `!windowShown` branch.
 - [x] `TestFilteredJobIndexesAll` / `ByNamedFolder` / `NoFolder` / `EmptySlice`
      ([jobs_view_test.go:59-101](../src/ui/jobs_view_test.go)) — four tests
      over one small pure function. Collapse into one table-driven test in the
      style of `TestFilterValue` directly above them; the `EmptySlice` case
      becomes one row rather than a function.
 ## 5. Runtime cost of the runner tests — declined, with measurements
 `TestRunJobLogFileAllHeaders`, `TestRunJobRecordFields`, and
 `TestRunJobWritesLogFile` ([runner_test.go](../src/runner/runner_test.go)) have
 identical coverage profiles but assert three genuinely different things — log
 headers, `RunRecord` field values, and the log file's name and directory. They
 are **not** duplicates and should not be deleted on that basis.
 - [x] Decided: **do not merge them.** The premise was wrong. Per-test timings
      from `go test -count=1 -v ./src/runner`:
      | Test | Time |
      |---|---|
      | `TestRunJobTimesOut` | 2.10 s |
      | `TestRunJobZeroTimeoutMeansNoTimeout` | 1.05 s |
      | `TestRunJobWritesLogFile` | 0.05 s |
      | `TestRunJobLogFileAllHeaders` | 0.05 s |
      | `TestRunJobRecordFields` | 0.04 s |
      The three candidates cost 0.14 s combined, so the merge buys back about
      90 ms. The package's runtime is the two deliberate waits in the timeout
      tests plus build time — subprocess spawn is not what makes `runner` slow.
      Against that, the three fixtures differ in ways the assertions read:
      `TestRunJobWritesLogFile` runs the `Manual` trigger, the other two run
      `Schedule`, and each uses its own job ID and name. Merging forces one
      fixture and drops the `Manual` path from the log-header assertions — the
      exact silent loss this item warned about, for 90 ms.
      If `runner` wall time ever does become a problem, the two timeout tests
      are where the seconds are.
 ## Explicitly not changing
 Recorded here so a later pass does not re-report them:
 - `TestSetGlobalPausePersistsToConfigFile` has the same coverage as
  `TestSetGlobalPauseUpdatesRuntimesAndEmits` but asserts a different property —
  that the flag reaches `gosentry.json`, which is what makes the pause survive a
  restart.
 - `TestRunDueQueueDrainsMultipleOverlaps` has the same coverage as
  `TestRunDueQueueRerunsAfterFinish`, but drains three queued occurrences rather
  than one. It is the test that would catch a drain loop that fires once.
 - `TestCreateStartupShortcutHandlesCyrillicPath` and `...HandlesSpaces` cover the
  same statements, but non-ASCII paths and paths with spaces are different
  real-world failure modes for the WScript.Shell COM call.
 - The 0% functions in the non-UI packages are deliberate: the real `Clock` (a
  fake is injected everywhere), `OpenStore` / `ResolvePaths` / `Service.Start` /
  `Service.Open` and the autostart and desktop-icon wrappers (process entry
  points and OS integration), and `ShouldNotifyOnFailure` (a getter under the
  mutex). `storage.defaultJobs` is the exception — see item 2.
 ## Suggested order
 1. Item 2 (docs) — smallest, and the `defaultJobs` assertion is the only one
   that adds coverage.
 2. Item 3 (`itoa`) — independent of everything else.
 3. Item 1 (deletions) — one commit, with the coverage re-run as evidence.
 4. Item 4 (thin tests) — needs a judgment call per test.
 5. Item 5 — measured and declined; see the item.
 Items 1 and 4 change the test inventory, so [TESTS.md](TESTS.md) has to be
 updated in the same commit. No [CHANGELOG.md](CHANGELOG.md) entry is needed:
 none of this changes shipped behavior.
 ## Which model to use
 For running these items in Claude Code. The deciding factor here is not task
 size — it is that **the feedback loop is slow**: the `ui` package needs the
 MSYS2 UCRT64 toolchain with CGO on, and a cold `go test ./src/ui/...` took
 **258 s** during the review. A model that gets an edit right on the first pass
 is worth more than a faster one that needs a second build to find out.
 | Item | Model | Why |
 |---|---|---|
 | 3 — `itoa` → `strconv.FormatInt` | **Haiku 4.5** (`claude-haiku-4-5`) | A mechanical substitution in one file, in the `runner` package, which needs no CGO and runs in ~5 s. Nothing to weigh. |
 | 2 — docs, and the `defaultJobs` assertion | **Sonnet 5** (`claude-sonnet-5`) | Two doc edits plus one new assertion in `storage`. Reading `loadOrCreateJobs` to write the assertion is real work, but the answer is not in doubt. No CGO. |
 | 1 — the three deletions | **Sonnet 5** | Deleting is easy; the judgment is narrow and already made in this document (which line to carry over from `TestRunDueEmptyOverlapInheritsGlobal`, and that identical coverage must be re-verified afterwards). One of the three is in `platform/autostart`, which is Windows-gated but CGO-free. |
 | 4 — the four thin tests | **Opus 5** (`claude-opus-5`) | This is the only item that is genuinely a judgment call rather than an execution task: whether each test should exist at all, and — for `TestMainViewBuilds` — whether to fold two calls into the sizing test or keep it with a better comment. Two of the four are in `ui`, so a wrong call costs a 4-minute rebuild to discover. |
 | 5 — merging the runner tests | **Opus 5**, if attempted | It requires holding three distinct sets of assertions and confirming none is silently dropped in the merge. It is also the item most likely to be *not worth doing* — a model that will say so is the point. |
 Two notes on this table:
 - **Sonnet 5 is the reasonable single choice** if you would rather not switch
  models per item. It is near-Opus on coding and agentic work, and only item 4
  really rewards the step up. The introductory pricing through **2026-08-31**
  ($2/$10 per MTok vs $3/$15) makes it cheaper than usual relative to Opus 5's
  $5/$25.
 - **Fast mode is available on Opus 5** (toggle with `/fast`). It is the same
  model with higher output throughput, not a downgrade — but it bills at
  $10/$50, so it only pays for itself when you are waiting on the output. Given
  that the actual wait here is the Fyne build rather than token generation, it
  is unlikely to help on this plan.