# Session 0008.0 — Transcript > Date: 2026-05-27 > Goal: Execute Slice 5 of the `ohm-rfc-app-flotilla` SPEC §20 slicing > plan — hardening + first real upgrade + operator guide. Per §20.5: > drive a real rfc-app version bump through `flotilla pin check` → > `flotilla deploy`; tighten failure paths discovered in Slice 4; lock > down `--json` on every read verb; ship a single-page operator guide; > add a bring-up replay CI test. Exit criterion: a real rfc-app upgrade > lands on OHM via flotilla with no manual SSH steps; the operator > guide is sufficient for laptop re-provisioning. Ships as v1.0.0. > > Outcome: **flotilla v1.0.0 shipped end-to-end in-session.** The > §20.5 exit criterion was met live: `flotilla deploy ohm-rfc-app` > drove the rfc-app v0.2.3 → v0.3.0 upgrade against the live OHM VM, > all 9 phases green, `/api/health` verified at v0.3.0 in 2.5 seconds. > 156 tests passing (132 baseline → 156, +24 across hardening, > `secret set`, bring-up replay). Three repos updated in lockstep: > rfc-app v0.3.0 tagged + pushed; ohm-rfc pin bumped to 0.3.0; flotilla > v1.0.0 committed + tagged + pushed. > > Significant additions beyond the §20.5 brief: > > 1. **`flotilla secret set` verb** (§19.3 rule-2 correction to SPEC > §7.5 / §7.6). The Slice-5 live bootstrap of GCP Secret Manager > surfaced enough manual-gcloud friction (six `secrets create` > calls, ADC re-auth, the `${!var}` zsh-history quirk, CRLF source > breakage, `OAUTH_CLIENT_SECRET` lost-from-shell, IAM removal > with wrong email) that the v1 read-only-flotilla stance from §7.6 > became untenable for ongoing iteration. `secret set` reads bytes > from stdin, creates the SM entry (or adds a version), and binds > at `@latest` in one command. Bytes never enter argv or shell > history. IAM widens from `secretmanager.secretAccessor` to > `secretmanager.admin` (or `secretCreator + secretVersionAdder`) > only when using the verb; the bind-only path still only needs > accessor. > > 2. **§19.2 candidate: framework-side required-env manifest.** The > user's "how do new features just work?" question surfaced the gap > between rfc-app's `Upgrade steps:` CHANGELOG prose and what > flotilla can pre-validate. A new §19.2 entry proposes a fifth > versioned contract between flotilla and rfc-app — a machine- > readable manifest of required env vars per release — so the > deploy gesture can refuse to start with missing keys, naming them. > Not built in v1.0.0; documented so the option is ready when a > forgotten-secret incident surfaces it. > > 3. **rfc-app v0.3.0 released** (private-beta gate + anonymous read > mode + iOS-Safari scroll fix). The 0.3.0 WIP was already authored > in the working tree; the scroll fix (`.app: 100vh → 100dvh`) was > added to the same release. Diagnosis: iOS Safari measures `100vh` > as the URL-bar-hidden viewport, so the `.app` overflows what's > visible; combined with `body { overflow: hidden }`, single-finger > touches were consumed by the (blocked) page-level scroll attempt > and never reached the nested `.chrome-pane` scroll. Two-finger > touches bypassed the page-level layer. Switching to `100dvh` > (with `100vh` fallback) fixes the trap on iOS 15.4+ / Chrome 108+. > > 4. **OHM roadmap committed** to `ohm-rfc/ROADMAP.md`. Eleven feature > items from the user spanning auth migration, collaboration model, > compliance, and instrumentation; one item per session/version > discipline. Plus a twelfth: publicly post the session transcripts. > > Five findings worth flagging permanently: > > 1. **`100vh` on iOS Safari is a viewport-trap when combined with > `body { overflow: hidden }`.** The pattern produces a "two-finger > scroll to start, one-finger thereafter" symptom that's hard to > diagnose without knowing the URL-bar interaction. Fix is > `100dvh` with `100vh` fallback. Any future flex-column app-shell > in the rfc-app family should default to `100dvh`. > > 2. **iOS scrollable containers need the dynamic viewport.** Same > finding generalized — every `.chrome-pane`-style nested scroll > container in the rfc-app family inherits its constrained height > from the `.app` root. If that root uses `100vh`, every nested > scroll on iOS gets the trap. The framework's other chrome views > (`/admin/*`, `/settings/notifications`) were also affected and > fixed by the same change. > > 3. **`gcloud auth login` is not `gcloud auth application-default > login`.** Operators with working `gcloud compute ssh` may still > have no ADC, causing flotilla's Secret Manager reads to fail with > "no Application Default Credentials available." The browser > consent screen for ADC must also have **every scope checkbox > selected** — ADC issued without the required scopes will > authenticate but be rejected by the Secret Manager API. > Documented in `docs/secret-bootstrap.md` §1 and §6 pre-flight. > 4. **GCP IAM principal `ben.stull@wiggleverse.org` differs from > Wiggleverse email `ben@wiggleverse.org`.** A `remove-iam-policy- > binding` with the latter silently fails with "Policy binding with > the specified principal, role, and condition not found." The > `gcloud config get-value account` value is canonical for IAM > commands. `docs/secret-bootstrap.md` now uses > `$OPERATOR_GCP_EMAIL` derived from that, not a hard-coded > address. > > 5. **zsh's history-substitution clashes with bash indirect > expansion.** `${!var}` triggers `event not found: var` in zsh > rather than expanding indirectly. Bash idioms in the bootstrap > doc are now wrapped in `bash <<'BASH' … BASH` to sidestep this. > Also surfaced: `source` on a CRLF-line-ending `.env` produces > spurious `command not found: ^M` errors; the doc now leads with > a `tr -d '\r'` normalization step. > > Two §19.3 rule-2 spec corrections in this session: > - §7.5: `secret set` added as the fourth verb. > - §7.6: narrowed (rotation ceremonies + multi-version policy still > deferred; bytes-level creation now in v1). > - §11.2: IAM widening noted for `secret set`. > - §12.1: verb inventory updated. > - §19.2: new candidate "framework-side required-env manifest." --- ## Pre-session state - flotilla repo on `main`, last commit `491e72e Release 0.3.0: Slice 3 — pin reading + dry-run deploy`. - Slice 4 code lived in the working tree, uncommitted: `deploy.py`, `ssh.py`, `health.py`, `deploy_log.py`, migration 004, plus the CLI extensions and 58 new tests. `VERSION` was bumped to 0.4.0; `CHANGELOG.md` had the 0.4.0 entry written. Per Session 0007.0's outcome, the v0.4.0 cut was gated on the first real live deploy — which had not yet happened. - rfc-app at v0.2.3 (latest tag); `/Users/benstull/git/rfc-app` had a large in-progress 0.3.0 release authored but uncommitted — private- beta gate, allowlist table, `BetaPending.jsx`, nginx config rename, admin allowlist UI, anonymous read mode. - ohm-rfc `.rfc-app-version` committed at `0.3.0` — but rfc-app had no `v0.3.0` tag. Pin was ahead of reality (Session 0007.0 had not done a live deploy, so the spec assertion in §13.3 that current pin is 0.3.0 was aspirational). - OHM VM `ohm-app` in `wiggleverse-ohm` running rfc-app **v0.2.2** (the last hand-deployed version). `/api/health` returned 404 — expected, because v0.2.2 predates the route's introduction in v0.2.3. - GCP Secret Manager API on `wiggleverse-ohm`: not enabled. Zero `ohm-rfc-app-*` secrets in the project. The Slice 2 bootstrap had never been performed; the v0.2.0 CHANGELOG entry was still "pending operator gesture." ## Turn-by-turn arc The session moves through four arcs, in order: ### Arc 1 — Slice 5 hardening (code-only, in-flotilla) Started by reading every Slice 4 file end-to-end (`deploy.py`, `ssh.py`, `health.py`, `deploy_log.py`, `cli.py`, the migration, all of `test_*.py`). Surfaced these hardening targets: - **§3 invariant 1 hole** in `deploy.py`: `all_secret_bytes = resolved.all_bytes()` produced a `list[bytes]` snapshot held on the stack across all phases. The `bytes` objects were immutable copies, so `resolved.scrub()` zeroing the underlying bytearrays did not zero them. Bytes survived on the stack until function return. Fix: replace the snapshot with `_ResolvedSecrets.redact_live(text)` that builds the needle list lazily from the live bytearrays at the point of failure. After the call returns and `scrub()` runs, no copies remain. - **`deploy status` silently exits 0 on unhealthy responses.** A 404 from `/api/health` (the exact OHM-current state) produced `HTTP 404 version=None status=None` and a zero exit. Fix: classify by HTTP status — 4xx is `endpoint not reachable (framework pre-0.2.3? proxy mis-routed?)` and exits 1; 5xx is server-side and exits 1; 2xx without a parseable body is malformed and exits 1. Same exit-code discipline under `--json` so scripts can pipe to `jq` and still branch on `$?`. - **`pin check` used `list_deploys(limit=50)` + Python filter.** If 50 failures intervened, the actual last-succeeded would silently drop out of the window. Fix: new `deploy_log.last_succeeded()` — a direct `WHERE outcome='succeeded' ORDER BY id DESC LIMIT 1` query. Test pins this behavior past 60 intervening failures. - **`resolve_operator_account` ignored returncode.** Used `proc. stdout` even when gcloud returned non-zero or when gcloud emitted the literal `(unset)`. Fix: bail to `"unknown"` on non-zero or `(unset)`. - **`--json` gaps**: `pin check` lacked it; `deploy watch` lacked it. Added: `pin check --json` emits a structured comparison (pin, status: matches/ahead/no_successful_deploys, last_succeeded_deploy row); `deploy watch --json` emits NDJSON, one JSON object per reading, suitable for `… | jq -c`. - **`deploy log --limit=0/negative`** wasn't guarded. Tiny — added an explicit refuse-with-message. - Minor: removed unused `field` import in `deploy.py`. Tests for each fix landed in `test_deploy.py`, `test_deploy_log.py`, `test_cli_deploy.py`. 132 tests → 146 tests after hardening. ### Arc 2 — Operator guide + bring-up replay CI test Two §20.5 deliverables that complete the v1.0.0 documentation surface: - **`docs/operator-guide.md`** — single-page bring-up + day-to-day reference. Sections: prerequisites; clone + install; gcloud authentication (with the ADC distinction called out); deployment registration (pointing at `scripts/register-ohm.sh`); secret binding; first-deploy dry-run; the real deploy gesture; verify; watch; recovery patterns; where state lives; JSON output inventory. Sufficient to bring flotilla up on a new laptop without reading the SPEC. The §20.5 exit criterion's "operator-guide-sufficient" bar. - **`tests/test_bringup_replay.py`** — one large test that walks the operator guide's exact verb sequence against stubbed GCP/SSH/ network. Every SPEC §12.1 verb is exercised; a coverage-assertion set at the end catches new verbs that land without being added to the guide. Doc/code drift → CI failure. 146 tests → 147 (the replay test is a single test that exercises many verbs). ### Arc 3 — Live Slice 2 bootstrap + first deploy (where the friction was) Initial plan: cut v1.0.0 immediately and deploy. Reality: every step of the path had latent friction that had to be surfaced and fixed. This arc is the §19.3 rule-2 fuel for the whole session. #### 3a. Pin bump attempt fails because of repository-side state The user attempted the gesture from `ohm-rfc`: ``` echo 0.4.0 > .rfc-app-version && git commit && git push ``` which failed with `nothing to commit` — `git commit` without `-a` or prior `git add` doesn't stage modifications. Fixed in the operator guide §2.1 to show `git commit -am "..."`. But also: `0.4.0` is not a real rfc-app tag. Latest rfc-app tag is `v0.2.3`. Pinning to `0.4.0` would have hit phase 3 (`git checkout v0.4.0`) failure on the VM. The user picked "pin to 0.2.3 (existing tag)" — first real deploy is the upgrade from VM's current v0.2.2 to the framework's actual latest v0.2.3. #### 3b. Flotilla state was empty on the user's laptop `ohm-rfc-app-flotilla deployment list` showed `(no deployments registered)`. The SQLite at `~/.ohm-rfc-app-flotilla/ohm-rfc-app-flotilla.db` existed but had no rows — Slice 1's `scripts/register-ohm.sh` had not been run on this laptop. Ran it, deployment registered. Then `register-ohm.sh` failed with `ohm-rfc-app-flotilla: command not found` — the script calls the bare CLI name but the user hadn't activated the venv. Two fixes documented: `source .venv/bin/activate` first, OR `PATH=.venv/bin:$PATH ./scripts/register-ohm.sh`. The user chose activate. #### 3c. /api/health was 404 — diagnosis The OHM VM serves `{"detail":"Not Found"}` at `/api/health`. That's FastAPI's default 404 body — the framework IS running, the route just doesn't exist. Strong hypothesis: deployed code is pre-0.2.3 (which is when the route was introduced). Confirmed via `gcloud compute ssh` + `git describe`: VM was on `v0.2.2` (`018e323 Release 0.2.2: Philosophy.jsx now renders mermaid`). One version below the `/api/health` introduction. The deploy itself will fix this. The user also tried `gcloud config set project ohm-rfc-app` and got a permission warning — `ohm-rfc-app` is the flotilla **deployment name**, not a GCP project. Three different names share the prefix: the GCP project is `wiggleverse-ohm`, the VM is `ohm-app`, the deployment-record name is `ohm-rfc-app`. The operator guide table now makes this distinction explicit. #### 3d. Slice 2 bootstrap (Secret Manager) — the friction tour Pre-flight check `gcloud secrets list --project=wiggleverse-ohm --filter='name:ohm-rfc-app'` returned `API [secretmanager.googleapis.com] not enabled`. The Slice 2 bootstrap had never been done. Updated the runbook in-session as each friction point surfaced: 1. **`gcloud services enable secretmanager.googleapis.com`** — pre-requisite, done. 2. **IAM grant** — `gcloud projects add-iam-policy-binding wiggleverse-ohm --member='user:ben@wiggleverse.org' --role='roles/secretmanager.admin'`. The user's wiggleverse email `ben@wiggleverse.org` was used. Later turned out the GCP-side principal is actually `ben.stull@wiggleverse.org` — but `add` is idempotent and tolerated the typo by creating a new binding for the (nonexistent-as-IAM-principal) `ben@wiggleverse.org`. The `remove` at the end is where it bit us; see 3i. 3. **Pull the live `.env`** — `gcloud compute ssh ohm-app … 'sudo cat /opt/ohm-app/backend/.env' > /tmp/ohm-rfc-app.env`. 1135 bytes, 39 lines, chmod 600. 4. **`source` errors with CRLF**. First attempt at `source /tmp/ohm- rfc-app.env` produced `command not found: ^M` and `command not found: gto_oge…^M`. The `.env` on the VM had Windows line endings. `sudo cat` preserved them. zsh's `source` parsed each `\r` as a syntax error on otherwise-blank lines; on lines where a value wrapped to the next visual row, the wrapped portion was interpreted as a standalone command. Fix: `tr -d '\r' < /tmp/ohm-rfc-app.env > /tmp/…unix && mv …`. Verified with `file /tmp/ohm-rfc-app.env` → ASCII text (no "with CRLF"). 5. **Heads-up to the user**: the `^M`-broken `source` had echoed the start of `GITEA_BOT_TOKEN` into the terminal scrollback. Worth a `clear` + scrollback-clear at end-of-bootstrap. 6. **Spot-check after re-source**: `GITEA_BOT_TOKEN` length 40, `SECRET_KEY` length 65 — both populated, both confidence-passable without printing the actual bytes. 7. **zsh history-substitution on `${!key}`**. The overlay-reconcile loop uses bash indirect expansion `${!key:-}`. zsh interprets the `!` as history substitution and fails with `event not found: key`. Fix: wrap the loops in `bash <<'BASH' … BASH` to run them under bash regardless of the operator's interactive shell. Doc updated. 8. **OAUTH_CLIENT_SECRET missing in shell.** After re-sourcing, `length=0` for OAUTH_CLIENT_SECRET despite being present in the file. Likely cause: the value's shape conflicted with bash's `source` parser (special characters or multi-line that python- dotenv handles but `source` does not). Fix: a manual export fallback documented in the runbook — `export OAUTH_CLIENT_SECRET="$(grep '^OAUTH_CLIENT_SECRET=' /tmp/ohm-rfc-app.env | sed 's/^OAUTH_CLIENT_SECRET=//' | sed 's/^"//; s/"$//')"`. 9. **SMTP_PASSWORD + WEBHOOK_EMAIL_BOUNCE_SECRET both blank**. OHM-current reality: Gmail SMTP relay is IP-allowlisted and needs no password; the inbound bounce webhook isn't wired up yet. The user picked "mirror reality, skip both." The §13.7 list is *intended*; what's actually bootstrapped at any time is the subset with live values. Doc records this with an explanatory note — future bind happens out-of-band when those keys earn real values. 10. **The bind step failed** with `no Application Default Credentials available — run 'gcloud auth application-default login'`. The user had done `gcloud auth login` (since SSH worked) but not the ADC variant. ADC is a separate credential set used by SDKs like google-cloud-secret-manager. Fix: run ADC login; the doc's §1 now flags this distinction, and §6 adds a pre-flight `gcloud auth application-default print-access-token >/dev/null && echo "ADC ok" || gcloud auth application-default login`. 11. **Permission-scope nag during ADC**. The browser consent screen for ADC offers a list of permission scopes. Operators sometimes click through without selecting all of them — ADC issued without Secret Manager scope authenticates but the API rejects reads. Documented in §1 and §6. 12. **The bind still failed** — "secret not readable: secret not found." The Secret Manager entries themselves hadn't been created. The bootstrap had only done the IAM grant + ADC; the `make_secret` loop hadn't run yet. Ran it. 13. **All four bindings succeeded** — `bound GITEA_BOT_TOKEN → wiggleverse-ohm/ohm-rfc-app-gitea-bot-token@latest` (plus three others). `flotilla secret list ohm-rfc-app` showed four rows, no bytes. 14. **Cleanup**: `shred -u /tmp/ohm-rfc-app.env`, `unset GITEA_BOT_TOKEN OAUTH_CLIENT_SECRET SECRET_KEY GITEA_WEBHOOK_SECRET …`. #### 3i. IAM downgrade — wrong email surfaces `gcloud projects remove-iam-policy-binding wiggleverse-ohm --member='user:ben@wiggleverse.org' --role='roles/secretmanager.admin'` failed: "Policy binding with the specified principal, role, and condition not found." The policy listing showed the actual principal was `user:ben.stull@wiggleverse.org` (with the dot). Two follow-ups: - The doc now uses `$OPERATOR_GCP_EMAIL` derived from `gcloud config get-value account`, not a hard-coded address. - The policy listing also showed the user has `roles/owner` on the project. Owner trumps every IAM role — both `secretmanager.admin` and `secretmanager.secretAccessor` are functionally redundant for this operator. The doc now flags the owner-shortcut: "if you already hold `roles/owner`, the IAM dance is cosmetic — `add` / `remove` is hygiene for an operator without owner." #### 3j. Dry-run then real deploy `ohm-rfc-app-flotilla deploy ohm-rfc-app --dry-run` rendered the 9-phase plan with target version 0.2.3, 14 overlay keys, 4 secret refs (the two blanks correctly absent), no bytes leaked. Clean. `ohm-rfc-app-flotilla deploy ohm-rfc-app` then executed all 9 phases: ``` opened deploys row id=1; target v0.2.3 (snapshot …) [1/9] validate: ok [2/9] preflight: ok [3/9] fetch+checkout: ok [4/9] backend deps: ok [5/9] frontend build: ok [6/9] write .env: ok [7/9] restart: ok [8/9] verify /api/health: ok [9/9] finalize: ok ohm: deployed v0.2.3 (verify took …) ``` The first real flotilla deploy ever, and it just worked. `deploy status` confirmed `HTTP 200 version=0.2.3 status=ok`. `/api/health` on the live URL returned `{"version":"0.2.3","status":"ok"}`. Slice 2 + Slice 3 + Slice 4 + Slice 5 all validated in a single gesture. ### Arc 4 — `secret set` addition, v1.0.0 cut, second live deploy, roadmap #### 4a. The user's "how do new features just work?" question After the deploy succeeded, the user asked: "for ongoing development, I want all of this to just work. New feature → deploy instructions → flotilla call to set overlay/secrets → deploy. Anything that won't?" The honest answer: the per-thing gestures are one-liners (`overlay set` already is, `secret set` would be once added), so a release's deploy instructions become a short script the operator copy-pastes. SPEC §3 invariant 4 already names this — `Upgrade steps:` in rfc-app's CHANGELOG. But: flotilla can't pre-validate that the operator did the prep. If rfc-app v0.4.0 requires a new env var and the operator forgets, phase 7 or 8 will fail after the `.env` was written. Fail-stop preserves the previous serving version, but the debug happens post-write. Three options offered: 1. Add as §19.2 candidate, ship v1.0.0. 2. Sketch the manifest now, defer the rfc-app side. 3. Build the contract end-to-end now. User picked (1). New §19.2 candidate added: "framework-side required- env manifest" — sketches the fifth versioned contract between flotilla and rfc-app (a `deploy/required-env.json`, a `GET /api/ required-env`, or machine-parseable upgrade-steps schema) where flotilla refuses to deploy with missing keys, naming them. Earns its session when a forgotten-secret incident actually happens. #### 4b. `flotilla secret set` design + implementation User: "I expect to be adding lots of secrets (and config overlays) as we iterate." That tipped the design from "one-shot bootstrap script" to "single-key incremental verb." Design: ``` echo "$VALUE" | flotilla secret set ``` - Reads bytes from stdin (never argv → never shell history) - Computes `secret_id` from §7.2 convention (`-`, lowercased, underscores→dashes) - Creates the SM entry if needed, or adds a new version if it exists (rotation works for free) - Binds in flotilla at `@latest` - Project defaults to deployment's `target_vm_project`; `--project` flag overrides Implementation: - `secrets.create_secret_if_absent(project, secret_id) -> bool` — returns True if newly created, False if AlreadyExists. Both are success; caller proceeds to add a version unconditionally. - `secrets.add_secret_version(project, secret_id, payload) -> str` — returns version number. Bytes argument so caller reads raw stdin. - `secrets.derive_secret_id(deployment_name, env_key) -> str` — the §7.2 convention as pure function. - `secrets.SecretWriteError` — sibling to SecretReadError, same auth/permission/API failure shape but for writes. - CLI verb at `secret set ` with `--project` and `--secret-id` overrides. Tests: derive_secret_id unit tests, FakeSM helper class for CLI-level integration tests (create + update + empty stdin + unknown deployment + permission denied + project override). +9 tests. Added to `tests/test_bringup_replay.py` so the operator-guide gesture stays exercised end-to-end. 147 → 156. SPEC corrections (§19.3 rule 2): - §7.5 → four verbs (was three). - §7.6 narrowed: rotation ceremonies + multi-version policy still §19.2, bytes-level creation now in v1. - §11.2 IAM widening note. - §12.1 verb inventory. Operator guide §1.4 rewritten with two flows: bind for existing entries (steady-state, accessor IAM), set for new ones (incremental, admin IAM). secret-bootstrap.md gets a header note pointing at `secret set` as the preferred ongoing path; manual gcloud dance preserved for the initial-migration case where partitioning a live `.env` is still a one-time gesture. #### 4c. rfc-app scroll fix + 0.3.0 cut User: "the philosophy doc isn't scrollable. Two fingers gets it started, then one finger works, but people are getting stuck. I assume it's a viewport thing?" Diagnosis: `.app { height: 100vh }` + `body { overflow: hidden }` is the classic iOS Safari URL-bar trap. `100vh` on iOS measures the URL-bar-hidden ("largest") viewport — `.app` is taller than what's visible. Single-finger touches get consumed by the (blocked) page- level scroll attempt to dismiss the URL bar; two-finger touches bypass that and engage the nested `.chrome-pane` scroll directly. Fix: `.app { height: 100vh; height: 100dvh; }` — `dvh` is dynamic viewport height (adjusts as URL bar shows/hides), supported on iOS 15.4+ and Chrome 108+. `100vh` retained as a fallback for older browsers (which ignore the unknown `dvh` value). rfc-app's working tree at session start had a full 0.3.0 release authored but uncommitted — private-beta gate, allowlist table, `/admin/allowlist` UI, `/beta-pending` route, anonymous read mode, beta chips, nginx config rename (rfc.wiggleverse.org → ohm.wiggleverse.org matching the deprovisioned domain), and a complete CHANGELOG entry with RFC 2119 upgrade steps. The scroll fix was added to the same 0.3.0 release (small bug fix, orthogonal to the substantial 0.3.0 work). Committed as `21fcbc9 Release 0.3.0: private-beta gate + anonymous read mode`. Tagged `v0.3.0`. Pushed to both `git.wiggleverse.org/ben.stull/rfc-app` and the `git.benstull.org/benstull/rfc-app` mirror. ohm-rfc/.rfc-app-version bumped from 0.2.3 → 0.3.0. Committed, pushed. #### 4d. Second live deploy `flotilla pin check ohm-rfc-app` → `pin AHEAD of last deploy: pin=0.3.0, last deployed=v0.2.3 (deploys.id=1)`. Good. Then: ``` ohm-rfc-app-flotilla deploy ohm-rfc-app ``` All 9 phases green again. `ohm: deployed v0.3.0 (verify took 2.5s)`. Second real flotilla deploy — and the first real version-bump deploy (the first was v0.2.2 → v0.2.3, this is v0.2.3 → v0.3.0). `deploys.id=2`. `deploy status` confirmed `HTTP 200 version=0.3.0 status=ok`. `pin check` now says `pin matches last deploy (v0.3.0, deploys.id=2)`. `curl https://ohm.wiggleverse.org/api/health` → `{"version":"0.3.0","status":"ok"}`. Verified. The §20.5 exit criterion was met live in-session. #### 4e. flotilla v1.0.0 commit `VERSION` updated from `0.4.0` (the stale Slice-4-pending value) to `1.0.0`. `pyproject.toml` `version = "1.0.0"`. CHANGELOG `1.0.0` entry replaced "pending operator gesture" with `2026-05-27` plus a note that the gesture was satisfied during the build session itself (`deploys.id=2`, all 9 phases, 2.5s verify). `git add -A` staged 23 files (11 modified, 12 new). Commit `8b87c27 Release 1.0.0: Slice 4 + Slice 5 — full deploy gesture, hardening, secret set, operator guide`. Tag `v1.0.0`. Pushed to `git.wiggleverse.org/wiggleverse/ohm-rfc-app-flotilla` (canonical only; no personal-mirror for this org-flavored repo per SPEC §2). #### 4f. Roadmap User listed 11 features they want to ship at one-per-session cadence: VM rename, email/OTC login, passcodes, device trust 30d, CloudFlare verification, cookie opt-in, anonymous-discuss/contribute off-limits, auto-set RFC owner, owner-only invite, discussion-without- PR / contribution-needs-PR, Amplitude instrumentation. Categorized by repo, grouped into four phases (operational/easy wins → collaboration model → auth migration block → compliance+data), with dependencies noted (passcodes depends on OTC, owner-invite depends on the identity model, etc.). Written to `ohm-rfc/ROADMAP.md`. User added a twelfth item mid-write: publicly post the session transcripts. Added as Phase E with open-questions about where (a public gitea repo, an OHM site route, both) and what to redact (probably zero — no secret bytes in transcripts; just a confirming pass). ## Settlement record (Decisions that shifted the SPEC or established new operator-doc patterns, with the §19.3 rule applied where relevant.) - **§7.5 grows from three to four verbs.** `secret set` added. The Slice 5 build session is the live evidence the SPEC's read-only stance was wrong. Implementation surface: `secrets.create_secret_ if_absent`, `secrets.add_secret_version`, `secrets.derive_secret_id`, `secrets.SecretWriteError`. CLI surface: bytes via stdin, never via argv. - **§7.6 narrows.** The "secret creation through flotilla" entry moves out of v1-deferred (now in v1). Rotation ceremonies and multi-version pinning policies stay deferred. - **§11.2 IAM widens for write ops only.** Operators using `secret set` need `secretmanager.admin` (or the narrower `secretCreator + secretVersionAdder` pair). Deploy-only operators still only need `secretAccessor`. - **§12.1 verb inventory updated.** New `ohm-rfc-app-flotilla secret set KEY` row. - **§19.2 candidate added: framework-side required-env manifest.** Sketches the fifth versioned contract between flotilla and rfc-app. Surfaced by the user's "how do new features just work?" question and the gap between rfc-app's `Upgrade steps:` prose contract and what flotilla can pre-validate. Not built in v1.0.0. - **rfc-app SPEC §3 invariant 4 unchanged.** The four versioned contracts (GET /api/health, VERSION, .rfc-app-version, upgrade- steps CHANGELOG) all stayed faithful through the live deploy. The required-env manifest, if it lands, becomes the fifth — but as a candidate, not committed. - **Operator guide is now a first-class artifact.** Sufficient for laptop re-provisioning end-to-end. Lives at `flotilla/docs/operator-guide.md` and is exercised by `tests/test_bringup_replay.py`. ## What didn't happen this session (and why) - **VM rename `ohm-app` → `ohm-rfc-app`.** Still §19.2. The current deploy gesture uses the legacy `ohm-app` names; everything works. The rename is a separate operational gesture (gcloud + useradd + systemd + nginx + a no-op deploy under the new path), scoped to its own session. First item in the new ROADMAP.md. - **Auto-rollback on /api/health verify failure.** Still §19.2. The two live deploys this session both succeeded; no recovery gesture needed. Fail-stop is the v1 commitment. - **Cloud Build / worker pools.** Still §19.2. v1 builds the frontend on the target VM. Worked fine for both deploys in this session. - **Hosted flotilla / web surface.** Still §19.2. Single-operator laptop-only is correct for OHM right now. - **Secret rotation ceremonies.** Still §19.2. `secret set` is the primitive; the full drain-swap-reverify dance is a different problem. ## Cut state at end-of-session - **flotilla**: `8b87c27` on `main`, tag `v1.0.0` pushed. 156 tests passing. The first stable cut. - **rfc-app**: `21fcbc9` on `main`, tag `v0.3.0` pushed. The 0.3.0 release lives on both canonical (git.wiggleverse.org) and mirror (git.benstull.org). - **ohm-rfc**: `6897b3e` on `main`, pin at `0.3.0`. Plus `ROADMAP.md` (not yet committed — added in this session, will commit at the next opportunity). - **OHM VM**: serving rfc-app v0.3.0. /api/health green. Two `deploys` rows in the local flotilla SQLite (`id=1` for v0.2.3, `id=2` for v0.3.0). - **ohm-infra**: this transcript (Session 0008.0) at `~/git/ohm-infra/SESSION-0008.0-TRANSCRIPT.md`. ## Arc 5 — post-cut: live incident, cleanup, v1.0.1, parallel-fork operating instructions The session was supposed to end at "v1.0.0 shipped + roadmap committed." It didn't — the user opened the live site, found two regressions introduced by the deploy, and the session spent the rest of its context surfacing the root cause, cleaning up, cutting a patch, and rewriting the roadmap so future sessions don't have to do the dispatch by hand. ### 5a. Two reported regressions User: "The mermaid diagram is missing now" — followed shortly by "The sign in button also doesn't work." Initial diagnosis path: - Mermaid: Philosophy.jsx and MarkdownPreview.jsx unchanged between v0.2.3 and v0.3.0. The CSS for `.markdown-preview` and `.mermaid- block` unchanged. So neither code nor CSS regressed. - The deployed bundle still has mermaid chunks (`architectureDiagram- *.js`, `blockDiagram-*.js`, etc. all present at `/opt/ohm-app/frontend/dist/assets/`). nginx serves them with HTTP 200 / `application/javascript`. So the build and the serving layer are fine. - The actual cause for mermaid: `/api/philosophy` body has **zero mermaid fences**. The framework reads PHILOSOPHY.md from disk (default `/opt/ohm-app/PHILOSOPHY.md` — the framework's own, shipped with rfc-app, no mermaid). The OHM-specific PHILOSOPHY.md (with the "Why OHM, in three panels" diagram) lives at `/opt/ohm-app/meta-content/PHILOSOPHY.md` on the VM, out-of-band. For the framework to serve it, the `PHILOSOPHY_PATH` env var must point at it. The previous (pre-flotilla) `.env` had that override; flotilla's first deploy rewrote the .env from the overlay (which doesn't carry `PHILOSOPHY_PATH`) and clobbered the override. Sign-in failure was a stranger story. ### 5b. The CRLF time-bomb surfaced While inspecting the VM's `.env`, every single value had a literal trailing `\r` escape sequence: ``` APP_URL="https://ohm.wiggleverse.org\r" GITEA_URL="https://git.wiggleverse.org\r" OAUTH_CLIENT_ID="3e85cebb-6515-43a7-9ded-40e111c81f15\r" OAUTH_CLIENT_SECRET=" gto_oge…\r" SECRET_KEY="2e05383a849…\r" ... ``` `OAUTH_CLIENT_SECRET` additionally had a **leading space** (artifact of the manual `grep | sed` extraction from earlier in the bootstrap). When python-dotenv parsed these on the framework's startup, the `\r` escape sequences decoded into real carriage-return characters in the runtime values. `GITEA_URL` became `https://git.wiggleverse.org`, which Gitea rejected on the OAuth redirect chain — hence the broken sign-in. The OAuth client ID and secret were also CR-tainted, so even if the URL had been clean, the auth would have failed at the token-exchange step. Cross-checked against flotilla state via `overlay show --json`: **every** overlay row had a `\r`-tainted value, and the four Secret Manager versions stored CR-tainted bytes too (`GITEA_BOT_TOKEN`: 41 bytes instead of 40; `SECRET_KEY`: 65 instead of 64; etc.). Reconstructed cause: during Arc 3 (the Slice 2 bootstrap), the user sourced `/tmp/ohm-rfc-app.env` **before** running the `tr -d '\r'` normalization. The shell variables picked up trailing CRs. The overlay-reconcile loop and the SM-create loop both ran with the CR-tainted shell variables, so `flotilla overlay set` and `gcloud secrets create | flotilla secret bind` faithfully stored the CRs. The subsequent re-source on the CR-stripped file overwrote the shell variables but by then the writes had already happened. This wasn't anticipated by the bootstrap doc — §4 had the CRLF normalization listed first, but the operator-real ordering put the errored source attempt before the fix, and the variables retained their first-source values. ### 5c. Cleanup: Python orchestration script Wrote `/tmp/cleanup.py` that: 1. Reads every overlay row via `flotilla overlay show … --json`. 2. For each row: strips `\r\n` and surrounding whitespace from the value. If the cleaned value is empty, `overlay unset` (catches `SMTP_HOST` and `SMTP_USER` which were blank in the source but ended up as `\r`-only). Otherwise `overlay set` with the clean value. 3. For each of the four secret bindings: `gcloud secrets versions access latest` → `.strip()` → pipe to `flotilla secret set`. This creates a new SM version with cleaned bytes and rebinds at `@latest`. Sizes proved the strip worked: - GITEA_BOT_TOKEN: 41 → 40 bytes - OAUTH_CLIENT_SECRET: 58 → 56 bytes (stripped leading space too) - SECRET_KEY: 65 → 64 bytes - GITEA_WEBHOOK_SECRET: 65 → 64 bytes 4. Adds `PHILOSOPHY_PATH=/opt/ohm-app/meta-content/PHILOSOPHY.md` to the overlay so the next deploy's .env rewrite carries the override. Then `flotilla deploy ohm-rfc-app` again — third deploy in the session (`deploys.id=3`). All 9 phases green; `/api/health` verified at v0.3.0 in 2.4s. Same gesture, same shape; the flotilla deploy path is now well-exercised. Post-deploy verification confirmed all three fixes: - `/api/philosophy` body now contains 1 mermaid fence. - VM `.env` via `cat -A` shows `$` at every line end, no `^M`. - `/auth/login` 307s to a clean Gitea OAuth URL. ### 5d. v1.0.1 patch — doc + spec hardening The CRLF incident wasn't a code regression — it was a doc-discipline failure that the spec hadn't anticipated. Cut as a patch so the findings live in the canonical history: - **`docs/secret-bootstrap.md` §4** gains a `tail | od -c` verification step that catches trailing-CR pollution in shell variables BEFORE the §5/§6 loops fire. Recovery sequence spelled out: unset every shell variable → confirm file is CR-free → re- source → re-check. The check now runs over both the secret variables AND a couple of overlay variables (GITEA_URL, APP_URL) to make sure the loop didn't silently miss anything earlier. - **flotilla SPEC §19.2 candidate added**: "Input sanitization at the overlay/secret boundary." Open question — should flotilla reject/normalize CR/LF/whitespace at `overlay set` / `secret set` inputs as a structural defense? Pro: doc-independent. Con: blocks deployments that legitimately need multi-line values (a PEM key in a single env var, say). v1 stays doc-enforced; the candidate earns its session if a second incident of the same shape happens or if a deployment wants the opt-out. Tagged as `v1.0.1` and pushed. No code changes; pure doc + spec. The OHM live deploy doesn't need re-application — the cleanup in §5c already fixed the affected state. ### 5e. Roadmap revision: two new items + parallel-fork operating instructions User reviewed the roadmap and added two new items in Phase C between email/OTC (#5) and passcodes (now #8): - **#6 — Open beta-access request (first/last/why on first OTC).** Anyone with a valid email can sign in via #5's OTC. The previous v0.3.0 allowlist-of-emails gate becomes vestigial; the new gate is "has an admin granted you permissions?" New users land in `pending` permission state with required first name, last name, and free-text "why I should be included in the beta" captured. - **#7 — Admin user-management page + new-request notifications.** Admins get notified (email + in-app inbox if §15 is wired) when a new request lands; the `/admin/users` page lists every user with permission state, sign-up reason, and grant/revoke controls. The v0.3.0 `/admin/allowlist` page either merges in or stays as a sub-tab. These two are sequential (#7 needs #6's `users`-row shape to exist) but the rest of Track C reshuffles: passcodes (#8) and device trust (#9) become a separate sub-track that can parallel #6/#7 on careful branches. Cloudflare verification (#10) shifts to better-emphasize its importance now that anyone can request OTC. Version targets shifted: passcodes → v0.10.0, device-trust → v0.11.0, cookie → v0.13.0, owner-invite → v0.14.0, Amplitude → v0.15.0. Then the user added the structural ask: **"In the next session, roadmap items should be done in parallel when in possible and all new roadmap changes should be done in a forked agent with a new session. That way I can go to bed and progress can be made without me starting new Claude instances."** This is the operational shape change that turns the roadmap from a single-session-per-feature serial queue into an autonomous parallel pipeline. Added a new section at the bottom of `ohm-rfc/ROADMAP.md` — **"Operating instructions for the next session (parallel + autonomous)"** — that codifies: 1. The next session reads the roadmap and identifies the active wave (items with no unmet dependencies). 2. **Dispatch each shippable item as a separate subagent in parallel**, using the `Agent` tool with `subagent_type: general-purpose` and `isolation: worktree` (each subagent gets an isolated git worktree, removing the "concurrent sessions on separate branches" ceremony). 3. Each subagent's prompt is self-contained: roadmap item text, target version, repo path, dependency status, exact ship gesture (VERSION + pyproject + CHANGELOG with `Upgrade steps:` + commit + tag + push canonical + mirror if applicable + ohm-rfc pin bump if it's an rfc-app item). 4. **Subagents tag and push only.** The driver runs `flotilla deploy ohm-rfc-app` **one at a time** after each subagent finishes, because the §8.3 lock serializes deploys. 5. Verify after each deploy. Stop the wave on first deploy failure. 6. Pause the wave if a release's `Upgrade steps:` require operator- provided secret bytes (the driver can't invent them). 7. Update the roadmap (strikethrough the row, link to release tag and deploys.id) when each item ships. 8. Write a Session 0009.0 transcript at end-of-wave. Constraints the driver respects: no `git config` edits, no force- pushes, no `--amend` of pushed commits, no skipping pre-commit hooks. Match the safety rules from the global instructions. The roadmap commit (`58e47b9` on `ohm-rfc/main`) carries all of this. ### 5f. Cut state — for real this time | | | | --- | --- | | flotilla | `cad9b5b` tag `v1.0.1` | | rfc-app | `21fcbc9` tag `v0.3.0` | | ohm-rfc | `58e47b9` (pin at 0.3.0, ROADMAP updated with #6/#7 + driver instructions) | | OHM live | `deploys.id=3`, v0.3.0, cleaned values, mermaid back, sign-in working, PHILOSOPHY_PATH override restored | ## Prompt the operator will paste into the next Claude Code session ``` You are the OHM roadmap driver. Read /Users/benstull/projects/wiggleverse/ohm-rfc/ROADMAP.md end-to-end, then execute the active wave per the "Operating instructions for the next session" section at the bottom of that file. Concretely: 1. Identify Wave 1 items that are shippable (no unmet dependency). 2. Dispatch each shippable item as a forked subagent in a single message with multiple Agent tool uses (subagent_type: general-purpose, isolation: worktree). Each subagent prompt must be self-contained per the operating-instructions checklist. 3. When subagents finish (tag + push + ohm-rfc pin bump for rfc-app items), run `flotilla deploy ohm-rfc-app` from /Users/benstull/projects/wiggleverse/ohm-rfc-app-flotilla one at a time. Verify each deploy. 4. Strikethrough the version-target table row for each shipped item; commit + push ohm-rfc when the wave is done. 5. Continue to Wave 2 if its dependencies are now met. Keep going until you hit a blocker (operator-provided secret needed, deploy failure, ambiguity). 6. Write a Session 0009.0 transcript at ~/git/ohm-infra/SESSION-0009.0-TRANSCRIPT.md following the Session 0008.0 shape. Stop and wait for the operator if: a deploy fails, a release's Upgrade steps require a secret you can't generate, or you encounter cross-repo ambiguity. Do not invent secret values or guess at unclear design decisions. ``` ## Wave 1 ready When the operator pastes the prompt above into a fresh Claude Code session, that session will dispatch: - **Session α (Track Ω) — #1 VM rename** (flotilla + ohm-infra; the §19.2 candidate). Touches `ohm-app` → `ohm-rfc-app` for VM, unix user, install dir, systemd unit, nginx, and a re-`register-ohm.sh` on the flotilla side. May ship as flotilla v1.1.0 if any flotilla code change is needed (the operator-guide updates probably do require one). - **Session β (Track A) — #2 Auto-set RFC owner → rfc-app v0.4.0.** Smallest item. Touches the propose-RFC modal + the corresponding backend endpoint. - **Session γ (Track B) — #3 Discussion-without-PR + contribution- needs-PR → rfc-app v0.5.0.** Largest of the three. Collaboration- model shift; touches RFC view, comment storage, possibly schema (new `discussions` table or extension of `comments`). All three are parallel-safe (different tracks, different code regions). The driver's first message after reading the roadmap is a single Agent block with three sub-agent dispatches; then the driver waits for them to finish, runs three deploys in sequence, verifies each, and updates the roadmap. By the time the operator wakes up, OHM should be on `v0.5.0` (or further if Wave 2 dependencies were met), the flotilla v1.1.0 cut should have happened if the VM rename produced one, and Session 0009.0's transcript should record the journey. If the wave hits a blocker (most likely candidates: the VM rename needs operator-level GCP gestures that gcloud can't script around, or the collaboration-model change introduces a schema migration that needs the operator's blessing), the driver pauses and leaves a clear "waiting on operator" note in the partial Session 0009.0 transcript.