diff --git a/SESSION-0014.1-TRANSCRIPT-2026-05-28T07-08--2026-05-28T07-40.md b/SESSION-0014.1-TRANSCRIPT-2026-05-28T07-08--2026-05-28T07-40.md new file mode 100644 index 0000000..f8f0ea1 --- /dev/null +++ b/SESSION-0014.1-TRANSCRIPT-2026-05-28T07-08--2026-05-28T07-40.md @@ -0,0 +1,351 @@ +# Session N.1 — Transcript + +> Parent: SESSION-N-TRANSCRIPT-2026-05-28T07-01--.md +> Date: 2026-05-28 (PST) +> Start: 2026-05-28T07-08 PST +> End: 2026-05-28T07-40 PST +> Goal: Ship rfc-app v0.18.0 — email + webhook hygiene proposal +> (Wave 7 candidate #3; OHM roadmap items #18 + #20). +> Outcome: **All 5 slices shipped + merged + tagged + pushed to +> both remotes; ohm-rfc pin bumped to 0.18.0; 295 tests +> passing (was 252 pre-release).** + +## Pre-session state + +- **rfc-app**: `main` at `1456c8b` (v0.17.0), working tree clean, + no uncommitted changes. Remotes: `origin` = + `https://git.wiggleverse.org/ben.stull/rfc-app.git` (canonical), + `benstull` = `git@git.benstull.org:benstull/rfc-app.git` (mirror). + Test baseline (run before any work): + `PYTHONPATH=. .venv/bin/pytest -q` → 252 passed in 53.29s. +- **ohm-rfc**: `main` at `8cce148`, `.rfc-app-version = 0.17.0`, + remote = `ssh://git@git.wiggleverse.org:2222/ben/ohm-rfc.git` + (canonical only — no mirror configured, matches the prompt's + expectation). +- **Backend dev env**: Python venv at + `/Users/benstull/git/rfc-app/backend/.venv`. Tests need + `PYTHONPATH=.` (the prompt's example `pytest backend/tests/` was + one cwd-level off; backend tests live under `backend/tests/` + and import `from app import ...` so `cd backend && PYTHONPATH=. pytest` + is the working invocation). + +## Finding 1 — remote name mismatch with the prompt + +The prompt said push to `canonical` and `mirror`. Actual remote +names in this repo are `origin` (canonical) and `benstull` +(mirror). Proceeded with the actual names; final pushes hit both +remotes successfully. Surfacing for the driver's transcript — no +action needed beyond the rename if the prompt's convention is +intended. + +## Turn-by-turn arc + +### Arc 1 — Required reading + repo survey + +Read in order: +- `RFC-APP-EMAIL-HYGIENE-PROPOSAL.md` — the binding spec, three + surfaces (template hygiene, webhook tightening, observability), + 5-slice plan, four open questions. +- `SESSION-PROTOCOL.md` — §5 (subagent transcripts) is the + binding section for this subsession. +- `~/git/rfc-app/CLAUDE.md` — VERSION + frontend/package.json are + the canonical version pins; SemVer pre-1.0 minor can break. +- Looked at `email_otc.py`, `email_invite.py`, `email.py`, + `webhooks.py`, `config.py`, `invites.py`, `api_notifications.py`, + `db.py`, the migrations directory (next slot was 020), test + fixture `tmp_env` in `test_propose_vertical.py`, and a sample + v0.17.0 CHANGELOG entry for shape. + +**Open question #4 settled:** `invites.py` (per-RFC invite, +v0.16.0) does NOT have its own send path — it calls into +`email_invite.py`'s `send_invite_email` helper. So Slice 2's +single migration of `email_invite.py` covers both admin-create +and per-RFC invite paths automatically. + +**Open question #3 settled:** the dev-fallback path DOES write to +`outbound_emails` with `status='deferred'` per the proposal's +recommendation. Tests rely on this (no SMTP_HOST in CI/tests). + +**Open questions #1, #2** — operator-territory, not framework. +Left for the operator's next move. + +### Arc 2 — Slice 1: build_envelope helper + unit tests + +Created `backend/app/email_envelope.py` with a single +`build_envelope(...)` function. Key decisions: + +- `msgid_domain` defaults to the @-domain of `from_address` (so + Message-IDs align with the sending domain by default); falls + back to `"localhost"` if from_address has no `@` (defensive). +- `Auto-Submitted: auto-generated` set by default + (`is_transactional=True`) — every kind of mail rfc-app sends is + transactional in the RFC 3834 sense. +- `List-Unsubscribe-Post: List-Unsubscribe=One-Click` only emitted + when an `unsubscribe_url` is present (mailto-only callers don't + get it). +- `body_html=None` → single-part text/plain; html-present → + multipart/alternative with text/plain FIRST per RFC 2046. + +15 unit tests in `test_email_envelope.py` covering: always-present +headers; Auto-Submitted toggle; the three List-Unsubscribe shapes +(none / mailto-only / full one-click); plain vs multipart bodies; +Message-ID domain default + override + fallback. + +After Slice 1: 267 passed (252 + 15). Committed as `92059f3`. + +### Arc 3 — Slice 2: migrate send paths + POST unsubscribe + +Migrated `email_otc.py`, `email_invite.py`, `email._deliver`, +`email._send_bundle`, and (out of scope per proposal but a +natural sibling) `digest.py` to call `build_envelope`. + +Per-kind unsubscribe matrix: +- OTC: no `List-Unsubscribe` (per proposal tradeoff discussion — + the recipient explicitly requested the code). +- Invite (admin or per-RFC): `List-Unsubscribe: ` only. + No URL form because the invitee isn't a user yet, so no + per-user opt-out row exists. +- Notification (`_send_one`): full one-click — mailto + signed + URL + `List-Unsubscribe-Post`. +- Bundle (`_send_bundle`) + Digest: same as notification, BUT + uses a new synthetic `all` category instead of any single + per-category flag. The bundle/digest spans multiple categories, + so a per-category opt-out wouldn't honor user intent. Added + `all` → `email_opt_out_all = 1` to the `_CATEGORY_COLUMN` map + in `api_notifications.py`. + +`EmailConfig` gained `unsubscribe_mailto` (env: +`EMAIL_UNSUBSCRIBE_MAILTO`, default = `EMAIL_FROM`). Tested via +`test_invite_envelope_respects_email_unsubscribe_mailto_override`. + +Added `POST /api/email/unsubscribe` as the RFC 8058 receiver. Its +body is hint-only — the token signature carries the authority. +Returns `{ok, category}`. + +The `_SENT` test buffer now also carries `envelope["message"]` +(the constructed `EmailMessage`) so new tests can assert on +headers directly while preserving the legacy `to`/`from`/ +`subject`/`body` keys for backward compatibility. + +Added 10 integration tests: +- `test_otc_vertical.py`: 2 (no List-Unsubscribe; headers present). +- `test_admin_create_user_invite_vertical.py`: 3 (headers; mailto- + only; respects override). +- `test_notifications_vertical.py`: 5 (full one-click on + notification; POST endpoint flips per-category; `all` via POST; + `all` via GET; POST refuses invalid token). + +**Friction in Arc 3:** my first notification-header test failed +with "watcher notification did not fire" — I'd forgotten to call +`provision_user_row(user_id=1, login="ben", role="owner")` and to +pass `email="alice@test"` to `sign_in_as`. Once both were added +the test passed. No code change needed — just test setup. + +After Slice 2: 277 passed (267 + 10). Committed as `e9fdc47`. + +### Arc 4 — Slice 3: webhook tightening + +`config.py`: read `RFC_APP_INSECURE_WEBHOOKS` first; if `1`, call +`_optional("GITEA_WEBHOOK_SECRET")` (empty allowed); otherwise +call `_required("GITEA_WEBHOOK_SECRET")` (raises if empty). This +is the startup-loud-failure shape the proposal targets. + +`webhooks.py`: defense-in-depth at request time — if +`config.webhook_secret` is empty AND the bypass isn't set, 500 +with "Webhook receiver misconfigured." If the bypass IS set, log +a loud per-request warning. Unknown-repo branch gains an INFO +log line ("hook may be on a fork or stale"). + +`tmp_env` fixture in `test_propose_vertical.py`: was setting +`GITEA_WEBHOOK_SECRET=""`. Updated to bind a non-empty fake +secret so all pre-v0.18.0 tests boot under the new mandatory- +secret regime. This was the right call — the alternative (set +`RFC_APP_INSECURE_WEBHOOKS=1` in tests) would have left the +signature-verification path untested by default. + +New `test_webhooks_vertical.py` (7 tests): +- 3 startup-time `load_config()` tests (empty + no bypass raises; + empty + bypass loads; bound secret loads). +- 3 request-time signature tests (valid sig accepted; invalid sig + 401; missing sig 401). +- 1 unknown-repo logging test using `caplog`. + +After Slice 3: 284 passed (277 + 7). Committed as `d3daa97`. + +### Arc 5 — Slice 4: outbound_emails table + admin endpoint + +Migration `020_outbound_emails.sql` with the table per the +proposal's spec (status enum includes `'bounced'` for Slice 5). + +`email.record_outbound(...)` is a sibling of `_deliver`. It +swallows `RuntimeError` (so pure-helper unit tests that don't +call `db.init()` don't crash) and any other exception (so an +audit write never breaks a send). + +`_deliver` writes one row per send: status='deferred' when +`SMTP_HOST` is unset; status='sent' on SMTP success; status='failed' +with `error=ExceptionClassName: message` on exception. Threaded +`kind` and `notification_id` kwargs through `_send_one` → +`_deliver` so the row carries the FK back to `notifications`. + +`email_otc.py` and `email_invite.py` were trickier — they don't +go through `_deliver`, they have their own SMTP send. Added +`record_outbound` calls at each return point (disabled / dev- +fallback / success / exception). + +`digest.py`: gained `kind='digest'` on its `_deliver` call. + +`GET /api/admin/outbound-emails` admin endpoint with three filters +(kind, status, to_address — case-insensitive). Returns +`{items, has_more}` matching the rest of the admin surface. + +7 new tests in `test_outbound_emails_vertical.py` covering write +paths and admin endpoint. + +After Slice 4: 291 passed (284 + 7). Committed as `281a844`. + +### Arc 6 — Slice 5: bounce correlation + +Extended `BounceBody` with optional `message_id` (max_length=1000). +When supplied, the handler looks up the matching `outbound_emails` +row and stamps `status='bounced'` + appends `"bounce ()"` +to `error`. Response shape gained `correlated_id` (nullable). + +**Intentional response-shape change:** the response body now +includes `correlated_id`. One pre-existing test asserted on the +exact shape `{ok: True, matched: False}` — +`test_bounce_webhook_refuses_unsigned_when_secret_configured` in +`test_e2e_smoke.py`. Updated the assertion to include +`correlated_id: None`. Documented in the CHANGELOG entry. + +4 new tests in `test_outbound_emails_vertical.py`: +- bounce with matching message_id stamps the row + flips + global opt-out. +- bounce with unknown message_id logs but 200s (provider replay + case). +- bounce without message_id falls back to legacy v1 behavior. +- bounced rows show up in admin endpoint with `?status=bounced`. + +After Slice 5: 295 passed (291 + 4 new + 1 updated). Committed as +`4666c4a`. + +### Arc 7 — Release gesture + +1. `VERSION` 0.17.0 → 0.18.0. +2. `frontend/package.json#version` 0.17.0 → 0.18.0. +3. CHANGELOG entry inserted above v0.17.0. Five-slice summary + + Added / Changed / Tests / Migration / Upgrade-steps blocks. + Upgrade-steps block per SPEC §20.4 covers: MUST + GITEA_WEBHOOK_SECRET non-empty; MAY RFC_APP_INSECURE_WEBHOOKS=1 + for local dev; MAY EMAIL_UNSUBSCRIBE_MAILTO override; MUST + apply migration 020 (auto); MUST rebuild + restart; SHOULD + run mail-tester.com probe. +4. Release commit `0562d53`. +5. Final test run: 295 passed in 59.11s. Green. +6. Merged feature branch to main with `--no-ff` → + merge commit `31913b1`. +7. Tagged `v0.18.0` on `31913b1`. +8. Pushed `main`, `v0.18.0` tag, and feature branch to BOTH + `origin` (canonical) and `benstull` (mirror). All four pushes + reported successful by the remote. +9. Bumped `/Users/benstull/projects/wiggleverse/ohm-rfc/.rfc-app-version` + 0.17.0 → 0.18.0. Committed as `c527aca`, pushed to ohm-rfc + `origin/main`. + +### Arc 8 — Transcript (this file) + +Wrote the transcript before returning to driver N, per Session-M +lesson + §5 of the protocol. + +## Cut state + +| Surface | State | +|---|---| +| rfc-app `main` | `31913b1` (merge commit), tagged **v0.18.0** at `31913b1`. Pushed to `origin` + `benstull`. | +| rfc-app feature branch | `feature/v0.18.0-email-webhook-hygiene` at `0562d53`. Pushed to both remotes. | +| rfc-app `VERSION` + `frontend/package.json` | `0.18.0`. | +| ohm-rfc `main` | `c527aca`, `.rfc-app-version = 0.18.0`. Pushed to `origin` only (no mirror; matches the project convention). | +| Tests | **295 passed, 0 failed** (was 252 pre-release). Run time ~59s. | +| OHM live deploy | UNCHANGED — driver N drives the deploy gesture after the operator-led SMTP fix lands. | + +| Slice | Commit | Status | +|---|---|---| +| 1: build_envelope helper + unit tests | `92059f3` | ✅ shipped | +| 2: migrate send paths + POST unsubscribe | `e9fdc47` | ✅ shipped | +| 3: webhook tightening | `d3daa97` | ✅ shipped | +| 4: outbound_emails + admin endpoint | `281a844` | ✅ shipped | +| 5: bounce correlation | `4666c4a` | ✅ shipped | +| Release: VERSION + CHANGELOG + frontend/package.json | `0562d53` | ✅ shipped | +| Merge to main | `31913b1` | ✅ shipped | +| Tag `v0.18.0` | at `31913b1` | ✅ pushed both remotes | +| ohm-rfc pin bump | `c527aca` | ✅ pushed | + +## §19.2 candidates surfaced + +1. **Per-RFC invite path consolidation.** Per the proposal's open + question #4: confirmed `invites.py` (v0.16.0 per-RFC invite) + uses `email_invite.py`'s helper, NOT its own send path. So + Slice 2 covered it automatically. But the per-RFC invite and + admin-create invite share the same template + From identity + today; if a future minor needs them to diverge (e.g., + per-RFC carries the RFC slug in its Reply-To), the split goes + on `email_invite.py` and the per-RFC caller passes its own + override kwargs. Track as a §19.2 candidate but only worth a + session when an actual deployment asks for the divergence. +2. **HTML alternative body templates.** The proposal called out a + minimal HTML variant for OTC and a button-style HTML variant + for invites. v0.18.0 did NOT ship these — every send path + still passes only `body_plain` to `build_envelope`. The helper + supports `body_html`, so adding HTML variants is a follow-up + minor with zero infrastructure work; just the template authoring. + §19.2 candidate. +3. **Sender Policy Framework + DKIM rotation rhythm.** The + `mail-tester.com` SHOULD step in the upgrade-steps block + surfaces this — once OHM runs the probe with real SMTP, any + DKIM or SPF score gap drops onto the operator's plate, NOT + the framework's. But if the framework wants to surface + suspected DNS misalignment proactively (e.g., parse the + bounce body for SPF-fail signals and log a hint), that's a + future minor. §19.2 candidate. +4. **Admin UI for `outbound_emails`.** Proposal explicitly + deferred this; operator queries the endpoint with curl + jq + for now. When a need arises (e.g., the admin user-management + page wants to show "last email sent" inline per user), the + table is already populated and indexed for it. +5. **Remote-name mismatch with the prompt** (already named in the + Finding above). The prompt's "canonical / mirror" convention + doesn't match this repo's `origin` / `benstull` reality. Worth + either (a) renaming the remotes to match, or (b) updating + future driver/subagent prompts to say "the local origin / + mirror config, whatever that is." Procedural §19.2 candidate. + +## What driver N needs to know + +- **All 5 slices shipped clean.** v0.18.0 = full proposal scope. + Tag + main + feature branch are all pushed to both remotes. +- **295 tests passing**, 0 failed. The one test that needed an + intentional update (`test_e2e_smoke.test_bounce_webhook_refuses_unsigned_when_secret_configured`) + was updated in the Slice 5 commit and documented in the + CHANGELOG entry under "Changed → api_notifications.py." +- **ohm-rfc pin bumped to 0.18.0 at `c527aca` on `main`, + pushed.** Ready for `flotilla deploy ohm-rfc-app` once the + operator-led SMTP fix lands. +- **OHM-specific upgrade impact: zero overlay change.** + `GITEA_WEBHOOK_SECRET` has been bound on OHM since v0.14.0; + the new mandatory-at-startup check is a no-op for OHM + specifically. The new `EMAIL_UNSUBSCRIBE_MAILTO` is optional + (falls back to `EMAIL_FROM`), so OHM picks up the default + behavior cleanly without operator action. The + `outbound_emails` migration auto-applies on first startup + after the new image rolls. +- **Mail-tester probe is a SHOULD step.** Operator gesture, not + driver gesture. Worth surfacing on the operator's plate after + the deploy lands. +- **Remote names note:** the prompt said `canonical` + `mirror`; + actual config is `origin` + `benstull`. Pushed to both, no + failures. Driver N may want to either rename the remotes or + update the prompt template. +- **Subsession transcript:** this file, at + `~/git/ohm-infra/SESSION-N.1-TRANSCRIPT-2026-05-28T07-08--2026-05-28T07-40.md`. + Driver N publishes this via `publish-transcript.sh` alongside + the parent session transcript.