Files
session-history/0008/SESSION-0008.0-TRANSCRIPT-2026-05-27T20-00--2026-05-27T22-40.md
T
Ben Stull d5125190ae Session 0018.0 (Wave 9): rewrite legacy session refs in transcript bodies + README scoping
- Rewrite every legacy `Session L` / `Session M.1` etc. reference inside
  transcript bodies to the numeric form (`Session 0012.0` / `Session 0013.1`),
  via a mechanical Python sed pass scoped to canonical-filename transcripts
  only (502 replacements across 39 files; 21 files in this published repo +
  18 sibling files in `~/git/ohm-infra/`). The §23 rename was originally
  filename-only — session 0018.0 reverses that and rewrites the bodies too,
  so cross-references inside a transcript match the filename you'd browse
  to. The mapping table (`A → 0001` … `N → 0014`) lives in
  `SESSION-PROTOCOL.md` §1 (also amended this session) for anyone hitting
  an old external link.

- Amend README.md:
  - New "Where the record begins" section noting OHM started with two
    vibe-coded prototypes (no transcripts; the discipline that produced this
    record didn't exist yet). The transcripts here begin at the
    spec-driven-development phase — not at the literal beginning of OHM.
  - Update the existing "Sessions 0001 through 0013 originally shipped
    under a letter-form naming" paragraph to reflect that body refs are
    now also in numeric form.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 09:40:06 -07:00

914 lines
44 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Session 0008.0 — Transcript
> Date: 2026-05-27
> Goal: Execute Slice 5 of the `ohm-rfc-app-flotilla` SPEC §20 slicing
> plan — hardening + first real upgrade + operator guide. Per §20.5:
> drive a real rfc-app version bump through `flotilla pin check` →
> `flotilla deploy`; tighten failure paths discovered in Slice 4; lock
> down `--json` on every read verb; ship a single-page operator guide;
> add a bring-up replay CI test. Exit criterion: a real rfc-app upgrade
> lands on OHM via flotilla with no manual SSH steps; the operator
> guide is sufficient for laptop re-provisioning. Ships as v1.0.0.
>
> Outcome: **flotilla v1.0.0 shipped end-to-end in-session.** The
> §20.5 exit criterion was met live: `flotilla deploy ohm-rfc-app`
> drove the rfc-app v0.2.3 → v0.3.0 upgrade against the live OHM VM,
> all 9 phases green, `/api/health` verified at v0.3.0 in 2.5 seconds.
> 156 tests passing (132 baseline → 156, +24 across hardening,
> `secret set`, bring-up replay). Three repos updated in lockstep:
> rfc-app v0.3.0 tagged + pushed; ohm-rfc pin bumped to 0.3.0; flotilla
> v1.0.0 committed + tagged + pushed.
>
> Significant additions beyond the §20.5 brief:
>
> 1. **`flotilla secret set` verb** (§19.3 rule-2 correction to SPEC
> §7.5 / §7.6). The Slice-5 live bootstrap of GCP Secret Manager
> surfaced enough manual-gcloud friction (six `secrets create`
> calls, ADC re-auth, the `${!var}` zsh-history quirk, CRLF source
> breakage, `OAUTH_CLIENT_SECRET` lost-from-shell, IAM removal
> with wrong email) that the v1 read-only-flotilla stance from §7.6
> became untenable for ongoing iteration. `secret set` reads bytes
> from stdin, creates the SM entry (or adds a version), and binds
> at `@latest` in one command. Bytes never enter argv or shell
> history. IAM widens from `secretmanager.secretAccessor` to
> `secretmanager.admin` (or `secretCreator + secretVersionAdder`)
> only when using the verb; the bind-only path still only needs
> accessor.
>
> 2. **§19.2 candidate: framework-side required-env manifest.** The
> user's "how do new features just work?" question surfaced the gap
> between rfc-app's `Upgrade steps:` CHANGELOG prose and what
> flotilla can pre-validate. A new §19.2 entry proposes a fifth
> versioned contract between flotilla and rfc-app — a machine-
> readable manifest of required env vars per release — so the
> deploy gesture can refuse to start with missing keys, naming them.
> Not built in v1.0.0; documented so the option is ready when a
> forgotten-secret incident surfaces it.
>
> 3. **rfc-app v0.3.0 released** (private-beta gate + anonymous read
> mode + iOS-Safari scroll fix). The 0.3.0 WIP was already authored
> in the working tree; the scroll fix (`.app: 100vh → 100dvh`) was
> added to the same release. Diagnosis: iOS Safari measures `100vh`
> as the URL-bar-hidden viewport, so the `.app` overflows what's
> visible; combined with `body { overflow: hidden }`, single-finger
> touches were consumed by the (blocked) page-level scroll attempt
> and never reached the nested `.chrome-pane` scroll. Two-finger
> touches bypassed the page-level layer. Switching to `100dvh`
> (with `100vh` fallback) fixes the trap on iOS 15.4+ / Chrome 108+.
>
> 4. **OHM roadmap committed** to `ohm-rfc/ROADMAP.md`. Eleven feature
> items from the user spanning auth migration, collaboration model,
> compliance, and instrumentation; one item per session/version
> discipline. Plus a twelfth: publicly post the session transcripts.
>
> Five findings worth flagging permanently:
>
> 1. **`100vh` on iOS Safari is a viewport-trap when combined with
> `body { overflow: hidden }`.** The pattern produces a "two-finger
> scroll to start, one-finger thereafter" symptom that's hard to
> diagnose without knowing the URL-bar interaction. Fix is
> `100dvh` with `100vh` fallback. Any future flex-column app-shell
> in the rfc-app family should default to `100dvh`.
>
> 2. **iOS scrollable containers need the dynamic viewport.** Same
> finding generalized — every `.chrome-pane`-style nested scroll
> container in the rfc-app family inherits its constrained height
> from the `.app` root. If that root uses `100vh`, every nested
> scroll on iOS gets the trap. The framework's other chrome views
> (`/admin/*`, `/settings/notifications`) were also affected and
> fixed by the same change.
>
> 3. **`gcloud auth login` is not `gcloud auth application-default
> login`.** Operators with working `gcloud compute ssh` may still
> have no ADC, causing flotilla's Secret Manager reads to fail with
> "no Application Default Credentials available." The browser
> consent screen for ADC must also have **every scope checkbox
> selected** — ADC issued without the required scopes will
> authenticate but be rejected by the Secret Manager API.
> Documented in `docs/secret-bootstrap.md` §1 and §6 pre-flight.
>
4. **GCP IAM principal `ben.stull@wiggleverse.org` differs from
> Wiggleverse email `ben@wiggleverse.org`.** A `remove-iam-policy-
> binding` with the latter silently fails with "Policy binding with
> the specified principal, role, and condition not found." The
> `gcloud config get-value account` value is canonical for IAM
> commands. `docs/secret-bootstrap.md` now uses
> `$OPERATOR_GCP_EMAIL` derived from that, not a hard-coded
> address.
>
> 5. **zsh's history-substitution clashes with bash indirect
> expansion.** `${!var}` triggers `event not found: var` in zsh
> rather than expanding indirectly. Bash idioms in the bootstrap
> doc are now wrapped in `bash <<'BASH' … BASH` to sidestep this.
> Also surfaced: `source` on a CRLF-line-ending `.env` produces
> spurious `command not found: ^M` errors; the doc now leads with
> a `tr -d '\r'` normalization step.
>
> Two §19.3 rule-2 spec corrections in this session:
> - §7.5: `secret set` added as the fourth verb.
> - §7.6: narrowed (rotation ceremonies + multi-version policy still
> deferred; bytes-level creation now in v1).
> - §11.2: IAM widening noted for `secret set`.
> - §12.1: verb inventory updated.
> - §19.2: new candidate "framework-side required-env manifest."
---
## Pre-session state
- flotilla repo on `main`, last commit `491e72e Release 0.3.0:
Slice 3 — pin reading + dry-run deploy`.
- Slice 4 code lived in the working tree, uncommitted: `deploy.py`,
`ssh.py`, `health.py`, `deploy_log.py`, migration 004, plus the
CLI extensions and 58 new tests. `VERSION` was bumped to 0.4.0;
`CHANGELOG.md` had the 0.4.0 entry written. Per Session 0007.0's
outcome, the v0.4.0 cut was gated on the first real live deploy —
which had not yet happened.
- rfc-app at v0.2.3 (latest tag); `/Users/benstull/git/rfc-app` had a
large in-progress 0.3.0 release authored but uncommitted — private-
beta gate, allowlist table, `BetaPending.jsx`, nginx config rename,
admin allowlist UI, anonymous read mode.
- ohm-rfc `.rfc-app-version` committed at `0.3.0` — but rfc-app had
no `v0.3.0` tag. Pin was ahead of reality (Session 0007.0 had not done a
live deploy, so the spec assertion in §13.3 that current pin is
0.3.0 was aspirational).
- OHM VM `ohm-app` in `wiggleverse-ohm` running rfc-app **v0.2.2**
(the last hand-deployed version). `/api/health` returned 404 —
expected, because v0.2.2 predates the route's introduction in
v0.2.3.
- GCP Secret Manager API on `wiggleverse-ohm`: not enabled. Zero
`ohm-rfc-app-*` secrets in the project. The Slice 2 bootstrap had
never been performed; the v0.2.0 CHANGELOG entry was still "pending
operator gesture."
## Turn-by-turn arc
The session moves through four arcs, in order:
### Arc 1 — Slice 5 hardening (code-only, in-flotilla)
Started by reading every Slice 4 file end-to-end (`deploy.py`,
`ssh.py`, `health.py`, `deploy_log.py`, `cli.py`, the migration, all
of `test_*.py`). Surfaced these hardening targets:
- **§3 invariant 1 hole** in `deploy.py`: `all_secret_bytes =
resolved.all_bytes()` produced a `list[bytes]` snapshot held on the
stack across all phases. The `bytes` objects were immutable copies,
so `resolved.scrub()` zeroing the underlying bytearrays did not
zero them. Bytes survived on the stack until function return. Fix:
replace the snapshot with `_ResolvedSecrets.redact_live(text)` that
builds the needle list lazily from the live bytearrays at the point
of failure. After the call returns and `scrub()` runs, no copies
remain.
- **`deploy status` silently exits 0 on unhealthy responses.** A 404
from `/api/health` (the exact OHM-current state) produced
`HTTP 404 version=None status=None` and a zero exit. Fix: classify
by HTTP status — 4xx is `endpoint not reachable (framework
pre-0.2.3? proxy mis-routed?)` and exits 1; 5xx is server-side and
exits 1; 2xx without a parseable body is malformed and exits 1.
Same exit-code discipline under `--json` so scripts can pipe to
`jq` and still branch on `$?`.
- **`pin check` used `list_deploys(limit=50)` + Python filter.** If
50 failures intervened, the actual last-succeeded would silently
drop out of the window. Fix: new `deploy_log.last_succeeded()` —
a direct `WHERE outcome='succeeded' ORDER BY id DESC LIMIT 1`
query. Test pins this behavior past 60 intervening failures.
- **`resolve_operator_account` ignored returncode.** Used `proc.
stdout` even when gcloud returned non-zero or when gcloud emitted
the literal `(unset)`. Fix: bail to `"unknown"` on non-zero or
`(unset)`.
- **`--json` gaps**: `pin check` lacked it; `deploy watch` lacked it.
Added: `pin check --json` emits a structured comparison (pin,
status: matches/ahead/no_successful_deploys, last_succeeded_deploy
row); `deploy watch --json` emits NDJSON, one JSON object per
reading, suitable for `… | jq -c`.
- **`deploy log --limit=0/negative`** wasn't guarded. Tiny — added an
explicit refuse-with-message.
- Minor: removed unused `field` import in `deploy.py`.
Tests for each fix landed in `test_deploy.py`, `test_deploy_log.py`,
`test_cli_deploy.py`. 132 tests → 146 tests after hardening.
### Arc 2 — Operator guide + bring-up replay CI test
Two §20.5 deliverables that complete the v1.0.0 documentation surface:
- **`docs/operator-guide.md`** — single-page bring-up + day-to-day
reference. Sections: prerequisites; clone + install; gcloud
authentication (with the ADC distinction called out); deployment
registration (pointing at `scripts/register-ohm.sh`); secret
binding; first-deploy dry-run; the real deploy gesture; verify;
watch; recovery patterns; where state lives; JSON output inventory.
Sufficient to bring flotilla up on a new laptop without reading
the SPEC. The §20.5 exit criterion's "operator-guide-sufficient"
bar.
- **`tests/test_bringup_replay.py`** — one large test that walks the
operator guide's exact verb sequence against stubbed GCP/SSH/
network. Every SPEC §12.1 verb is exercised; a coverage-assertion
set at the end catches new verbs that land without being added to
the guide. Doc/code drift → CI failure.
146 tests → 147 (the replay test is a single test that exercises
many verbs).
### Arc 3 — Live Slice 2 bootstrap + first deploy (where the friction was)
Initial plan: cut v1.0.0 immediately and deploy. Reality: every step
of the path had latent friction that had to be surfaced and fixed.
This arc is the §19.3 rule-2 fuel for the whole session.
#### 3a. Pin bump attempt fails because of repository-side state
The user attempted the gesture from `ohm-rfc`:
```
echo 0.4.0 > .rfc-app-version && git commit && git push
```
which failed with `nothing to commit` — `git commit` without `-a`
or prior `git add` doesn't stage modifications. Fixed in the
operator guide §2.1 to show `git commit -am "..."`.
But also: `0.4.0` is not a real rfc-app tag. Latest rfc-app tag is
`v0.2.3`. Pinning to `0.4.0` would have hit phase 3 (`git checkout
v0.4.0`) failure on the VM. The user picked "pin to 0.2.3 (existing
tag)" — first real deploy is the upgrade from VM's current v0.2.2 to
the framework's actual latest v0.2.3.
#### 3b. Flotilla state was empty on the user's laptop
`ohm-rfc-app-flotilla deployment list` showed `(no deployments
registered)`. The SQLite at `~/.ohm-rfc-app-flotilla/ohm-rfc-app-flotilla.db`
existed but had no rows — Slice 1's `scripts/register-ohm.sh` had not
been run on this laptop. Ran it, deployment registered.
Then `register-ohm.sh` failed with `ohm-rfc-app-flotilla: command not
found` — the script calls the bare CLI name but the user hadn't
activated the venv. Two fixes documented: `source .venv/bin/activate`
first, OR `PATH=.venv/bin:$PATH ./scripts/register-ohm.sh`. The user
chose activate.
#### 3c. /api/health was 404 — diagnosis
The OHM VM serves `{"detail":"Not Found"}` at `/api/health`. That's
FastAPI's default 404 body — the framework IS running, the route just
doesn't exist. Strong hypothesis: deployed code is pre-0.2.3 (which
is when the route was introduced).
Confirmed via `gcloud compute ssh` + `git describe`: VM was on
`v0.2.2` (`018e323 Release 0.2.2: Philosophy.jsx now renders mermaid`).
One version below the `/api/health` introduction. The deploy itself
will fix this.
The user also tried `gcloud config set project ohm-rfc-app` and got
a permission warning — `ohm-rfc-app` is the flotilla **deployment
name**, not a GCP project. Three different names share the prefix:
the GCP project is `wiggleverse-ohm`, the VM is `ohm-app`, the
deployment-record name is `ohm-rfc-app`. The operator guide table now
makes this distinction explicit.
#### 3d. Slice 2 bootstrap (Secret Manager) — the friction tour
Pre-flight check `gcloud secrets list --project=wiggleverse-ohm
--filter='name:ohm-rfc-app'` returned `API [secretmanager.googleapis.com]
not enabled`. The Slice 2 bootstrap had never been done. Updated the
runbook in-session as each friction point surfaced:
1. **`gcloud services enable secretmanager.googleapis.com`** —
pre-requisite, done.
2. **IAM grant** — `gcloud projects add-iam-policy-binding
wiggleverse-ohm --member='user:ben@wiggleverse.org'
--role='roles/secretmanager.admin'`. The user's wiggleverse email
`ben@wiggleverse.org` was used. Later turned out the GCP-side
principal is actually `ben.stull@wiggleverse.org` — but `add` is
idempotent and tolerated the typo by creating a new binding for
the (nonexistent-as-IAM-principal) `ben@wiggleverse.org`. The
`remove` at the end is where it bit us; see 3i.
3. **Pull the live `.env`** — `gcloud compute ssh ohm-app …
'sudo cat /opt/ohm-app/backend/.env' > /tmp/ohm-rfc-app.env`. 1135
bytes, 39 lines, chmod 600.
4. **`source` errors with CRLF**. First attempt at `source /tmp/ohm-
rfc-app.env` produced `command not found: ^M` and `command not
found: gto_oge…^M`. The `.env` on the VM had Windows line endings.
`sudo cat` preserved them. zsh's `source` parsed each `\r` as a
syntax error on otherwise-blank lines; on lines where a value
wrapped to the next visual row, the wrapped portion was
interpreted as a standalone command.
Fix: `tr -d '\r' < /tmp/ohm-rfc-app.env > /tmp/…unix && mv …`.
Verified with `file /tmp/ohm-rfc-app.env` → ASCII text (no "with
CRLF").
5. **Heads-up to the user**: the `^M`-broken `source` had echoed the
start of `GITEA_BOT_TOKEN` into the terminal scrollback. Worth a
`clear` + scrollback-clear at end-of-bootstrap.
6. **Spot-check after re-source**: `GITEA_BOT_TOKEN` length 40,
`SECRET_KEY` length 65 — both populated, both confidence-passable
without printing the actual bytes.
7. **zsh history-substitution on `${!key}`**. The overlay-reconcile
loop uses bash indirect expansion `${!key:-}`. zsh interprets the
`!` as history substitution and fails with `event not found: key`.
Fix: wrap the loops in `bash <<'BASH' … BASH` to run them under
bash regardless of the operator's interactive shell. Doc updated.
8. **OAUTH_CLIENT_SECRET missing in shell.** After re-sourcing,
`length=0` for OAUTH_CLIENT_SECRET despite being present in the
file. Likely cause: the value's shape conflicted with bash's
`source` parser (special characters or multi-line that python-
dotenv handles but `source` does not). Fix: a manual export
fallback documented in the runbook —
`export OAUTH_CLIENT_SECRET="$(grep '^OAUTH_CLIENT_SECRET='
/tmp/ohm-rfc-app.env | sed 's/^OAUTH_CLIENT_SECRET=//' |
sed 's/^"//; s/"$//')"`.
9. **SMTP_PASSWORD + WEBHOOK_EMAIL_BOUNCE_SECRET both blank**.
OHM-current reality: Gmail SMTP relay is IP-allowlisted and needs
no password; the inbound bounce webhook isn't wired up yet. The
user picked "mirror reality, skip both." The §13.7 list is
*intended*; what's actually bootstrapped at any time is the subset
with live values. Doc records this with an explanatory note —
future bind happens out-of-band when those keys earn real values.
10. **The bind step failed** with `no Application Default Credentials
available — run 'gcloud auth application-default login'`. The user
had done `gcloud auth login` (since SSH worked) but not the ADC
variant. ADC is a separate credential set used by SDKs like
google-cloud-secret-manager. Fix: run ADC login; the doc's §1
now flags this distinction, and §6 adds a pre-flight
`gcloud auth application-default print-access-token >/dev/null
&& echo "ADC ok" || gcloud auth application-default login`.
11. **Permission-scope nag during ADC**. The browser consent screen
for ADC offers a list of permission scopes. Operators sometimes
click through without selecting all of them — ADC issued without
Secret Manager scope authenticates but the API rejects reads.
Documented in §1 and §6.
12. **The bind still failed** — "secret not readable: secret not
found." The Secret Manager entries themselves hadn't been
created. The bootstrap had only done the IAM grant + ADC; the
`make_secret` loop hadn't run yet. Ran it.
13. **All four bindings succeeded** — `bound GITEA_BOT_TOKEN →
wiggleverse-ohm/ohm-rfc-app-gitea-bot-token@latest` (plus three
others). `flotilla secret list ohm-rfc-app` showed four rows, no
bytes.
14. **Cleanup**: `shred -u /tmp/ohm-rfc-app.env`,
`unset GITEA_BOT_TOKEN OAUTH_CLIENT_SECRET SECRET_KEY
GITEA_WEBHOOK_SECRET …`.
#### 3i. IAM downgrade — wrong email surfaces
`gcloud projects remove-iam-policy-binding wiggleverse-ohm
--member='user:ben@wiggleverse.org' --role='roles/secretmanager.admin'`
failed: "Policy binding with the specified principal, role, and
condition not found." The policy listing showed the actual principal
was `user:ben.stull@wiggleverse.org` (with the dot). Two follow-ups:
- The doc now uses `$OPERATOR_GCP_EMAIL` derived from
`gcloud config get-value account`, not a hard-coded address.
- The policy listing also showed the user has `roles/owner` on the
project. Owner trumps every IAM role — both `secretmanager.admin`
and `secretmanager.secretAccessor` are functionally redundant for
this operator. The doc now flags the owner-shortcut: "if you
already hold `roles/owner`, the IAM dance is cosmetic — `add` /
`remove` is hygiene for an operator without owner."
#### 3j. Dry-run then real deploy
`ohm-rfc-app-flotilla deploy ohm-rfc-app --dry-run` rendered the
9-phase plan with target version 0.2.3, 14 overlay keys, 4 secret
refs (the two blanks correctly absent), no bytes leaked. Clean.
`ohm-rfc-app-flotilla deploy ohm-rfc-app` then executed all 9 phases:
```
opened deploys row id=1; target v0.2.3 (snapshot …)
[1/9] validate: ok
[2/9] preflight: ok
[3/9] fetch+checkout: ok
[4/9] backend deps: ok
[5/9] frontend build: ok
[6/9] write .env: ok
[7/9] restart: ok
[8/9] verify /api/health: ok
[9/9] finalize: ok
ohm: deployed v0.2.3 (verify took …)
```
The first real flotilla deploy ever, and it just worked. `deploy
status` confirmed `HTTP 200 version=0.2.3 status=ok`. `/api/health`
on the live URL returned `{"version":"0.2.3","status":"ok"}`. Slice
2 + Slice 3 + Slice 4 + Slice 5 all validated in a single gesture.
### Arc 4 — `secret set` addition, v1.0.0 cut, second live deploy, roadmap
#### 4a. The user's "how do new features just work?" question
After the deploy succeeded, the user asked: "for ongoing development,
I want all of this to just work. New feature → deploy instructions →
flotilla call to set overlay/secrets → deploy. Anything that won't?"
The honest answer: the per-thing gestures are one-liners (`overlay
set` already is, `secret set` would be once added), so a release's
deploy instructions become a short script the operator copy-pastes.
SPEC §3 invariant 4 already names this — `Upgrade steps:` in rfc-app's
CHANGELOG.
But: flotilla can't pre-validate that the operator did the prep. If
rfc-app v0.4.0 requires a new env var and the operator forgets, phase
7 or 8 will fail after the `.env` was written. Fail-stop preserves
the previous serving version, but the debug happens post-write.
Three options offered:
1. Add as §19.2 candidate, ship v1.0.0.
2. Sketch the manifest now, defer the rfc-app side.
3. Build the contract end-to-end now.
User picked (1). New §19.2 candidate added: "framework-side required-
env manifest" — sketches the fifth versioned contract between
flotilla and rfc-app (a `deploy/required-env.json`, a `GET /api/
required-env`, or machine-parseable upgrade-steps schema) where
flotilla refuses to deploy with missing keys, naming them. Earns its
session when a forgotten-secret incident actually happens.
#### 4b. `flotilla secret set` design + implementation
User: "I expect to be adding lots of secrets (and config overlays)
as we iterate." That tipped the design from "one-shot bootstrap
script" to "single-key incremental verb."
Design:
```
echo "$VALUE" | flotilla secret set <deployment> <ENV_KEY>
```
- Reads bytes from stdin (never argv → never shell history)
- Computes `secret_id` from §7.2 convention (`<deployment>-<env_key>`,
lowercased, underscores→dashes)
- Creates the SM entry if needed, or adds a new version if it exists
(rotation works for free)
- Binds in flotilla at `@latest`
- Project defaults to deployment's `target_vm_project`; `--project`
flag overrides
Implementation:
- `secrets.create_secret_if_absent(project, secret_id) -> bool` —
returns True if newly created, False if AlreadyExists. Both are
success; caller proceeds to add a version unconditionally.
- `secrets.add_secret_version(project, secret_id, payload) -> str`
— returns version number. Bytes argument so caller reads raw
stdin.
- `secrets.derive_secret_id(deployment_name, env_key) -> str` — the
§7.2 convention as pure function.
- `secrets.SecretWriteError` — sibling to SecretReadError, same
auth/permission/API failure shape but for writes.
- CLI verb at `secret set <deployment> <ENV_KEY>` with `--project`
and `--secret-id` overrides.
Tests: derive_secret_id unit tests, FakeSM helper class for CLI-level
integration tests (create + update + empty stdin + unknown deployment
+ permission denied + project override). +9 tests. Added to
`tests/test_bringup_replay.py` so the operator-guide gesture stays
exercised end-to-end. 147 → 156.
SPEC corrections (§19.3 rule 2):
- §7.5 → four verbs (was three).
- §7.6 narrowed: rotation ceremonies + multi-version policy still
§19.2, bytes-level creation now in v1.
- §11.2 IAM widening note.
- §12.1 verb inventory.
Operator guide §1.4 rewritten with two flows: bind for existing
entries (steady-state, accessor IAM), set for new ones (incremental,
admin IAM). secret-bootstrap.md gets a header note pointing at
`secret set` as the preferred ongoing path; manual gcloud dance
preserved for the initial-migration case where partitioning a live
`.env` is still a one-time gesture.
#### 4c. rfc-app scroll fix + 0.3.0 cut
User: "the philosophy doc isn't scrollable. Two fingers gets it
started, then one finger works, but people are getting stuck. I
assume it's a viewport thing?"
Diagnosis: `.app { height: 100vh }` + `body { overflow: hidden }`
is the classic iOS Safari URL-bar trap. `100vh` on iOS measures the
URL-bar-hidden ("largest") viewport — `.app` is taller than what's
visible. Single-finger touches get consumed by the (blocked) page-
level scroll attempt to dismiss the URL bar; two-finger touches
bypass that and engage the nested `.chrome-pane` scroll directly.
Fix: `.app { height: 100vh; height: 100dvh; }` — `dvh` is dynamic
viewport height (adjusts as URL bar shows/hides), supported on iOS
15.4+ and Chrome 108+. `100vh` retained as a fallback for older
browsers (which ignore the unknown `dvh` value).
rfc-app's working tree at session start had a full 0.3.0 release
authored but uncommitted — private-beta gate, allowlist table,
`/admin/allowlist` UI, `/beta-pending` route, anonymous read mode,
beta chips, nginx config rename (rfc.wiggleverse.org →
ohm.wiggleverse.org matching the deprovisioned domain), and a
complete CHANGELOG entry with RFC 2119 upgrade steps. The scroll fix
was added to the same 0.3.0 release (small bug fix, orthogonal to
the substantial 0.3.0 work).
Committed as `21fcbc9 Release 0.3.0: private-beta gate + anonymous
read mode`. Tagged `v0.3.0`. Pushed to both
`git.wiggleverse.org/ben.stull/rfc-app` and the
`git.benstull.org/benstull/rfc-app` mirror.
ohm-rfc/.rfc-app-version bumped from 0.2.3 → 0.3.0. Committed,
pushed.
#### 4d. Second live deploy
`flotilla pin check ohm-rfc-app` → `pin AHEAD of last deploy:
pin=0.3.0, last deployed=v0.2.3 (deploys.id=1)`. Good. Then:
```
ohm-rfc-app-flotilla deploy ohm-rfc-app
```
All 9 phases green again. `ohm: deployed v0.3.0 (verify took 2.5s)`.
Second real flotilla deploy — and the first real version-bump
deploy (the first was v0.2.2 → v0.2.3, this is v0.2.3 → v0.3.0).
`deploys.id=2`.
`deploy status` confirmed `HTTP 200 version=0.3.0 status=ok`. `pin
check` now says `pin matches last deploy (v0.3.0, deploys.id=2)`.
`curl https://ohm.wiggleverse.org/api/health` →
`{"version":"0.3.0","status":"ok"}`. Verified.
The §20.5 exit criterion was met live in-session.
#### 4e. flotilla v1.0.0 commit
`VERSION` updated from `0.4.0` (the stale Slice-4-pending value) to
`1.0.0`. `pyproject.toml` `version = "1.0.0"`. CHANGELOG `1.0.0`
entry replaced "pending operator gesture" with `2026-05-27` plus a
note that the gesture was satisfied during the build session itself
(`deploys.id=2`, all 9 phases, 2.5s verify).
`git add -A` staged 23 files (11 modified, 12 new). Commit
`8b87c27 Release 1.0.0: Slice 4 + Slice 5 — full deploy gesture,
hardening, secret set, operator guide`. Tag `v1.0.0`. Pushed to
`git.wiggleverse.org/wiggleverse/ohm-rfc-app-flotilla` (canonical
only; no personal-mirror for this org-flavored repo per SPEC §2).
#### 4f. Roadmap
User listed 11 features they want to ship at one-per-session
cadence: VM rename, email/OTC login, passcodes, device trust 30d,
CloudFlare verification, cookie opt-in, anonymous-discuss/contribute
off-limits, auto-set RFC owner, owner-only invite, discussion-without-
PR / contribution-needs-PR, Amplitude instrumentation.
Categorized by repo, grouped into four phases (operational/easy wins
→ collaboration model → auth migration block → compliance+data),
with dependencies noted (passcodes depends on OTC, owner-invite
depends on the identity model, etc.). Written to
`ohm-rfc/ROADMAP.md`.
User added a twelfth item mid-write: publicly post the session
transcripts. Added as Phase E with open-questions about where (a
public gitea repo, an OHM site route, both) and what to redact
(probably zero — no secret bytes in transcripts; just a confirming
pass).
## Settlement record
(Decisions that shifted the SPEC or established new operator-doc
patterns, with the §19.3 rule applied where relevant.)
- **§7.5 grows from three to four verbs.** `secret set` added. The
Slice 5 build session is the live evidence the SPEC's read-only
stance was wrong. Implementation surface: `secrets.create_secret_
if_absent`, `secrets.add_secret_version`, `secrets.derive_secret_id`,
`secrets.SecretWriteError`. CLI surface: bytes via stdin, never
via argv.
- **§7.6 narrows.** The "secret creation through flotilla" entry
moves out of v1-deferred (now in v1). Rotation ceremonies and
multi-version pinning policies stay deferred.
- **§11.2 IAM widens for write ops only.** Operators using `secret
set` need `secretmanager.admin` (or the narrower `secretCreator +
secretVersionAdder` pair). Deploy-only operators still only need
`secretAccessor`.
- **§12.1 verb inventory updated.** New `ohm-rfc-app-flotilla secret
set <deployment> KEY` row.
- **§19.2 candidate added: framework-side required-env manifest.**
Sketches the fifth versioned contract between flotilla and
rfc-app. Surfaced by the user's "how do new features just work?"
question and the gap between rfc-app's `Upgrade steps:` prose
contract and what flotilla can pre-validate. Not built in v1.0.0.
- **rfc-app SPEC §3 invariant 4 unchanged.** The four versioned
contracts (GET /api/health, VERSION, .rfc-app-version, upgrade-
steps CHANGELOG) all stayed faithful through the live deploy. The
required-env manifest, if it lands, becomes the fifth — but as a
candidate, not committed.
- **Operator guide is now a first-class artifact.** Sufficient for
laptop re-provisioning end-to-end. Lives at
`flotilla/docs/operator-guide.md` and is exercised by
`tests/test_bringup_replay.py`.
## What didn't happen this session (and why)
- **VM rename `ohm-app` → `ohm-rfc-app`.** Still §19.2. The current
deploy gesture uses the legacy `ohm-app` names; everything works.
The rename is a separate operational gesture (gcloud + useradd +
systemd + nginx + a no-op deploy under the new path), scoped to
its own session. First item in the new ROADMAP.md.
- **Auto-rollback on /api/health verify failure.** Still §19.2. The
two live deploys this session both succeeded; no recovery gesture
needed. Fail-stop is the v1 commitment.
- **Cloud Build / worker pools.** Still §19.2. v1 builds the
frontend on the target VM. Worked fine for both deploys in this
session.
- **Hosted flotilla / web surface.** Still §19.2. Single-operator
laptop-only is correct for OHM right now.
- **Secret rotation ceremonies.** Still §19.2. `secret set` is the
primitive; the full drain-swap-reverify dance is a different
problem.
## Cut state at end-of-session
- **flotilla**: `8b87c27` on `main`, tag `v1.0.0` pushed. 156 tests
passing. The first stable cut.
- **rfc-app**: `21fcbc9` on `main`, tag `v0.3.0` pushed. The 0.3.0
release lives on both canonical (git.wiggleverse.org) and mirror
(git.benstull.org).
- **ohm-rfc**: `6897b3e` on `main`, pin at `0.3.0`. Plus
`ROADMAP.md` (not yet committed — added in this session, will
commit at the next opportunity).
- **OHM VM**: serving rfc-app v0.3.0. /api/health green. Two
`deploys` rows in the local flotilla SQLite (`id=1` for
v0.2.3, `id=2` for v0.3.0).
- **ohm-infra**: this transcript (Session 0008.0) at
`~/git/ohm-infra/SESSION-0008.0-TRANSCRIPT.md`.
## Arc 5 — post-cut: live incident, cleanup, v1.0.1, parallel-fork operating instructions
The session was supposed to end at "v1.0.0 shipped + roadmap committed."
It didn't — the user opened the live site, found two regressions
introduced by the deploy, and the session spent the rest of its
context surfacing the root cause, cleaning up, cutting a patch, and
rewriting the roadmap so future sessions don't have to do the
dispatch by hand.
### 5a. Two reported regressions
User: "The mermaid diagram is missing now" — followed shortly by
"The sign in button also doesn't work."
Initial diagnosis path:
- Mermaid: Philosophy.jsx and MarkdownPreview.jsx unchanged between
v0.2.3 and v0.3.0. The CSS for `.markdown-preview` and `.mermaid-
block` unchanged. So neither code nor CSS regressed.
- The deployed bundle still has mermaid chunks (`architectureDiagram-
*.js`, `blockDiagram-*.js`, etc. all present at
`/opt/ohm-app/frontend/dist/assets/`). nginx serves them with HTTP
200 / `application/javascript`. So the build and the serving layer
are fine.
- The actual cause for mermaid: `/api/philosophy` body has **zero
mermaid fences**. The framework reads PHILOSOPHY.md from disk
(default `/opt/ohm-app/PHILOSOPHY.md` — the framework's own,
shipped with rfc-app, no mermaid). The OHM-specific PHILOSOPHY.md
(with the "Why OHM, in three panels" diagram) lives at
`/opt/ohm-app/meta-content/PHILOSOPHY.md` on the VM, out-of-band.
For the framework to serve it, the `PHILOSOPHY_PATH` env var must
point at it. The previous (pre-flotilla) `.env` had that override;
flotilla's first deploy rewrote the .env from the overlay (which
doesn't carry `PHILOSOPHY_PATH`) and clobbered the override.
Sign-in failure was a stranger story.
### 5b. The CRLF time-bomb surfaced
While inspecting the VM's `.env`, every single value had a literal
trailing `\r` escape sequence:
```
APP_URL="https://ohm.wiggleverse.org\r"
GITEA_URL="https://git.wiggleverse.org\r"
OAUTH_CLIENT_ID="3e85cebb-6515-43a7-9ded-40e111c81f15\r"
OAUTH_CLIENT_SECRET=" gto_oge…\r"
SECRET_KEY="2e05383a849…\r"
...
```
`OAUTH_CLIENT_SECRET` additionally had a **leading space** (artifact
of the manual `grep | sed` extraction from earlier in the bootstrap).
When python-dotenv parsed these on the framework's startup, the `\r`
escape sequences decoded into real carriage-return characters in the
runtime values. `GITEA_URL` became `https://git.wiggleverse.org<CR>`,
which Gitea rejected on the OAuth redirect chain — hence the broken
sign-in. The OAuth client ID and secret were also CR-tainted, so
even if the URL had been clean, the auth would have failed at the
token-exchange step.
Cross-checked against flotilla state via `overlay show --json`:
**every** overlay row had a `\r`-tainted value, and the four Secret
Manager versions stored CR-tainted bytes too (`GITEA_BOT_TOKEN`:
41 bytes instead of 40; `SECRET_KEY`: 65 instead of 64; etc.).
Reconstructed cause: during Arc 3 (the Slice 2 bootstrap), the user
sourced `/tmp/ohm-rfc-app.env` **before** running the `tr -d '\r'`
normalization. The shell variables picked up trailing CRs. The
overlay-reconcile loop and the SM-create loop both ran with the
CR-tainted shell variables, so `flotilla overlay set` and `gcloud
secrets create | flotilla secret bind` faithfully stored the CRs.
The subsequent re-source on the CR-stripped file overwrote the
shell variables but by then the writes had already happened.
This wasn't anticipated by the bootstrap doc — §4 had the CRLF
normalization listed first, but the operator-real ordering put the
errored source attempt before the fix, and the variables retained
their first-source values.
### 5c. Cleanup: Python orchestration script
Wrote `/tmp/cleanup.py` that:
1. Reads every overlay row via `flotilla overlay show … --json`.
2. For each row: strips `\r\n` and surrounding whitespace from the
value. If the cleaned value is empty, `overlay unset` (catches
`SMTP_HOST` and `SMTP_USER` which were blank in the source
but ended up as `\r`-only). Otherwise `overlay set` with the
clean value.
3. For each of the four secret bindings: `gcloud secrets versions
access latest` → `.strip()` → pipe to `flotilla secret set`.
This creates a new SM version with cleaned bytes and rebinds at
`@latest`. Sizes proved the strip worked:
- GITEA_BOT_TOKEN: 41 → 40 bytes
- OAUTH_CLIENT_SECRET: 58 → 56 bytes (stripped leading space too)
- SECRET_KEY: 65 → 64 bytes
- GITEA_WEBHOOK_SECRET: 65 → 64 bytes
4. Adds `PHILOSOPHY_PATH=/opt/ohm-app/meta-content/PHILOSOPHY.md`
to the overlay so the next deploy's .env rewrite carries the
override.
Then `flotilla deploy ohm-rfc-app` again — third deploy in the
session (`deploys.id=3`). All 9 phases green; `/api/health`
verified at v0.3.0 in 2.4s. Same gesture, same shape; the flotilla
deploy path is now well-exercised.
Post-deploy verification confirmed all three fixes:
- `/api/philosophy` body now contains 1 mermaid fence.
- VM `.env` via `cat -A` shows `$` at every line end, no `^M`.
- `/auth/login` 307s to a clean Gitea OAuth URL.
### 5d. v1.0.1 patch — doc + spec hardening
The CRLF incident wasn't a code regression — it was a doc-discipline
failure that the spec hadn't anticipated. Cut as a patch so the
findings live in the canonical history:
- **`docs/secret-bootstrap.md` §4** gains a `tail | od -c`
verification step that catches trailing-CR pollution in shell
variables BEFORE the §5/§6 loops fire. Recovery sequence spelled
out: unset every shell variable → confirm file is CR-free → re-
source → re-check. The check now runs over both the secret
variables AND a couple of overlay variables (GITEA_URL, APP_URL)
to make sure the loop didn't silently miss anything earlier.
- **flotilla SPEC §19.2 candidate added**: "Input sanitization at
the overlay/secret boundary." Open question — should flotilla
reject/normalize CR/LF/whitespace at `overlay set` / `secret set`
inputs as a structural defense? Pro: doc-independent. Con: blocks
deployments that legitimately need multi-line values (a PEM key
in a single env var, say). v1 stays doc-enforced; the candidate
earns its session if a second incident of the same shape happens
or if a deployment wants the opt-out.
Tagged as `v1.0.1` and pushed. No code changes; pure doc + spec.
The OHM live deploy doesn't need re-application — the cleanup in
§5c already fixed the affected state.
### 5e. Roadmap revision: two new items + parallel-fork operating instructions
User reviewed the roadmap and added two new items in Phase C between
email/OTC (#5) and passcodes (now #8):
- **#6 — Open beta-access request (first/last/why on first OTC).**
Anyone with a valid email can sign in via #5's OTC. The previous
v0.3.0 allowlist-of-emails gate becomes vestigial; the new gate
is "has an admin granted you permissions?" New users land in
`pending` permission state with required first name, last name,
and free-text "why I should be included in the beta" captured.
- **#7 — Admin user-management page + new-request notifications.**
Admins get notified (email + in-app inbox if §15 is wired) when
a new request lands; the `/admin/users` page lists every user
with permission state, sign-up reason, and grant/revoke controls.
The v0.3.0 `/admin/allowlist` page either merges in or stays as
a sub-tab.
These two are sequential (#7 needs #6's `users`-row shape to exist)
but the rest of Track C reshuffles: passcodes (#8) and device trust
(#9) become a separate sub-track that can parallel #6/#7 on careful
branches. Cloudflare verification (#10) shifts to better-emphasize
its importance now that anyone can request OTC. Version targets
shifted: passcodes → v0.10.0, device-trust → v0.11.0, cookie →
v0.13.0, owner-invite → v0.14.0, Amplitude → v0.15.0.
Then the user added the structural ask: **"In the next session,
roadmap items should be done in parallel when in possible and all
new roadmap changes should be done in a forked agent with a new
session. That way I can go to bed and progress can be made without
me starting new Claude instances."**
This is the operational shape change that turns the roadmap from a
single-session-per-feature serial queue into an autonomous parallel
pipeline. Added a new section at the bottom of `ohm-rfc/ROADMAP.md`
— **"Operating instructions for the next session (parallel +
autonomous)"** — that codifies:
1. The next session reads the roadmap and identifies the active
wave (items with no unmet dependencies).
2. **Dispatch each shippable item as a separate subagent in
parallel**, using the `Agent` tool with `subagent_type:
general-purpose` and `isolation: worktree` (each subagent gets
an isolated git worktree, removing the "concurrent sessions on
separate branches" ceremony).
3. Each subagent's prompt is self-contained: roadmap item text,
target version, repo path, dependency status, exact ship
gesture (VERSION + pyproject + CHANGELOG with `Upgrade steps:` +
commit + tag + push canonical + mirror if applicable + ohm-rfc
pin bump if it's an rfc-app item).
4. **Subagents tag and push only.** The driver runs `flotilla
deploy ohm-rfc-app` **one at a time** after each subagent
finishes, because the §8.3 lock serializes deploys.
5. Verify after each deploy. Stop the wave on first deploy failure.
6. Pause the wave if a release's `Upgrade steps:` require operator-
provided secret bytes (the driver can't invent them).
7. Update the roadmap (strikethrough the row, link to release tag
and deploys.id) when each item ships.
8. Write a Session 0009.0 transcript at end-of-wave.
Constraints the driver respects: no `git config` edits, no force-
pushes, no `--amend` of pushed commits, no skipping pre-commit
hooks. Match the safety rules from the global instructions.
The roadmap commit (`58e47b9` on `ohm-rfc/main`) carries all of this.
### 5f. Cut state — for real this time
| | |
| --- | --- |
| flotilla | `cad9b5b` tag `v1.0.1` |
| rfc-app | `21fcbc9` tag `v0.3.0` |
| ohm-rfc | `58e47b9` (pin at 0.3.0, ROADMAP updated with #6/#7 + driver instructions) |
| OHM live | `deploys.id=3`, v0.3.0, cleaned values, mermaid back, sign-in working, PHILOSOPHY_PATH override restored |
## Prompt the operator will paste into the next Claude Code session
```
You are the OHM roadmap driver. Read /Users/benstull/projects/wiggleverse/ohm-rfc/ROADMAP.md
end-to-end, then execute the active wave per the "Operating instructions for the next
session" section at the bottom of that file.
Concretely:
1. Identify Wave 1 items that are shippable (no unmet dependency).
2. Dispatch each shippable item as a forked subagent in a single message with multiple
Agent tool uses (subagent_type: general-purpose, isolation: worktree). Each subagent
prompt must be self-contained per the operating-instructions checklist.
3. When subagents finish (tag + push + ohm-rfc pin bump for rfc-app items), run
`flotilla deploy ohm-rfc-app` from /Users/benstull/projects/wiggleverse/ohm-rfc-app-flotilla
one at a time. Verify each deploy.
4. Strikethrough the version-target table row for each shipped item; commit + push ohm-rfc
when the wave is done.
5. Continue to Wave 2 if its dependencies are now met. Keep going until you hit a blocker
(operator-provided secret needed, deploy failure, ambiguity).
6. Write a Session 0009.0 transcript at ~/git/ohm-infra/SESSION-0009.0-TRANSCRIPT.md following the
Session 0008.0 shape.
Stop and wait for the operator if: a deploy fails, a release's Upgrade steps require a
secret you can't generate, or you encounter cross-repo ambiguity. Do not invent secret
values or guess at unclear design decisions.
```
## Wave 1 ready
When the operator pastes the prompt above into a fresh Claude Code
session, that session will dispatch:
- **Session α (Track Ω) — #1 VM rename** (flotilla + ohm-infra; the
§19.2 candidate). Touches `ohm-app` → `ohm-rfc-app` for VM, unix
user, install dir, systemd unit, nginx, and a re-`register-ohm.sh`
on the flotilla side. May ship as flotilla v1.1.0 if any flotilla
code change is needed (the operator-guide updates probably do
require one).
- **Session β (Track A) — #2 Auto-set RFC owner → rfc-app v0.4.0.**
Smallest item. Touches the propose-RFC modal + the corresponding
backend endpoint.
- **Session γ (Track B) — #3 Discussion-without-PR + contribution-
needs-PR → rfc-app v0.5.0.** Largest of the three. Collaboration-
model shift; touches RFC view, comment storage, possibly schema
(new `discussions` table or extension of `comments`).
All three are parallel-safe (different tracks, different code regions).
The driver's first message after reading the roadmap is a single
Agent block with three sub-agent dispatches; then the driver waits
for them to finish, runs three deploys in sequence, verifies each,
and updates the roadmap.
By the time the operator wakes up, OHM should be on `v0.5.0` (or
further if Wave 2 dependencies were met), the flotilla v1.1.0 cut
should have happened if the VM rename produced one, and Session 0009.0's
transcript should record the journey.
If the wave hits a blocker (most likely candidates: the VM rename
needs operator-level GCP gestures that gcloud can't script around,
or the collaboration-model change introduces a schema migration that
needs the operator's blessing), the driver pauses and leaves a clear
"waiting on operator" note in the partial Session 0009.0 transcript.