Files

T

ben.stull 3713299f08 reshape: move OHM transcripts under ohm/ for org-wide multi-app layout; add org-wide root README

2026-06-03 06:54:38 -07:00

11 KiB

Raw Blame History

Session 0030.0 — Transcript

Date: 2026-05-28 Start: 2026-05-28T16-12 (PST implied) End: 2026-05-28T18-41 (PST implied) Subject: Security-audit-0026 FULL remediation — flotilla fixes, live VM hardening, and the rfc-app v0.27.0 security release (shipped + deployed) Status: FINALIZED.

Sanitization note: this transcript is published to the public wiggleverse/ohm-session-history. It references audit findings by ID and general category only — no exploit sketches or step-by-step reproduction. (At the first finalize, several findings were still live and unpatched, which made this mandatory; by this finalize the Critical and the rest of the application findings are remediated + deployed, but the convention holds — L5 SSH remains open, and exploit recipes never belong in a public record.) The full-detail audit (SECURITY-AUDIT-0026-2026-05-28.md) lives only in local ~/git/ohm-infra (not a git repo) and was never published. No secret bytes appear here (§3 invariant 1).

Launch prompt

full remediation of security report at ~/git/ohm-infra

Plus two follow-on operator directives mid-session: "ship v0.25.0" (the security branch) and a request to bind WEBHOOK_EMAIL_BOUNCE_SECRET.

The "security report" is SECURITY-AUDIT-0026-2026-05-28.md (from the parallel audit session 0026): 1 Critical, 1 High, 5 Medium, 9 Low, 6 Info across the rfc-app codebase, the flotilla CLI, and the live GCP/VM.

Outcome in one line

Every actionable audit-0026 finding is remediated and live on production, via three vehicles: flotilla PR #1 (merged + deployed), direct live GCP/VM ops, and the rfc-app v0.27.0 security release (deploys.id=37, verified live). Only the SSH-from-anywhere Low (deferred to a proper IAP rollout) and a few Info items remain.

The "did we leak the audit?" scare (resolved: NO)

Early on the operator worried a transcript containing the audit had been pushed to the public repo before an internet drop. Froze work and verified the entire git history of wiggleverse/ohm-session-history (fresh clone, all branches, git log -S for audit-unique strings): the audit session 0026 had published only its 454-byte placeholder; zero audit-unique strings in any commit ever; the audit file itself lives only in local ~/git/ohm-infra (not a git repo). Nothing leaked. The atomic publish push had failed on the internet drop, leaving only the start-gesture placeholder.

Coordination

The rfc-app application findings were being remediated on an uncommitted feature/v0.25.0-security-hardening branch by a parallel session. With the operator, Session 0030 first scoped to the non-overlapping work (flotilla + live VM). Later the operator directed this session to take over and ship that branch — which it did (as v0.27.0; see below).

Work performed

1. Flotilla repo (PR #1 — merged + deployed)

Branch session-0030/flotilla-security-hardening → PR #1 → merged (7b32a46). 161 tests pass.

L1 — secret redactor threaded into _PhaseRunner; both success and failure phase-detail paths are scrubbed before persistence/printing.
I6 — redact_live matches multiple byte-decodings (utf-8/replace, surrogateescape, latin-1) so a non-ASCII secret can't slip the needle.
L9 — ssh adds StrictHostKeyChecking=accept-new (refuses a changed host key instead of blanket TOFU-accept).
I5 — overlay set warns on unambiguous secret-word key names (not generic *_KEY, to avoid nagging on public site keys).

2. Live GCP/VM ops (project `wiggleverse-ohm`, VM `ohm-rfc-app`)

M3 — data dir → 750, DB + WAL/SHM → 640 (survives reboot + restart). Also secured (did not delete) orphaned ohm-app.db\r* files left by the historical CRLF path bug.
L5 — deleted the dead default-allow-rdp firewall rule. SSH-from- anywhere deferred to an IAP rollout (operator decision).
L6 — created dedicated least-priv SA ohm-rfc-app-vm@… (logging + monitoring writer only; no Secret Manager — the app reads .env), swapped the VM off the default compute SA.
L7 — enabled Shielded-VM Secure Boot (UEFI-verified first; vTPM + integrity monitoring already on).

L6 + L7 applied in one stop/start window (~15s downtime).

3. rfc-app v0.27.0 — the security release (shipped + deployed)

The "v0.25.0" branch was based on v0.24.0, but main/live was already v0.26.0 (#28). Shipping it as v0.25.0 would have downgraded prod, so it was rebased onto main and reversioned to v0.27.0. Then: committed, version-bumped (VERSION + package.json + a CHANGELOG entry with upgrade steps), tested, PR'd (PR #1 on ben.stull/rfc-app), merged (fe044ed), tagged v0.27.0, pin bumped in ohm-rfc/.rfc-app-version (0.26.0→0.27.0), deployed (deploys.id=37), and verified live.

Findings landed: C1 (DOMPurify sanitize chokepoint for every markdown→HTML sink — the Critical), H1 (per-IP ratelimit + per-account OTC lockout, migration 023 auto-applied), M1 (indexed single-row device-trust lookup), M2 (nginx security headers — see below), M4 (session cookie Secure-by-default), M5 (bounce webhook fails closed + secret bound), L2/L3 (throttles), L4 (systemd sandbox), L8/I1 (server_tokens off, legacy TLS removed).

The parallel branch was incomplete — its changes broke 60 backend tests, which this session diagnosed and fixed:

M4 set the session cookie Secure, but TestClient talks plain http, so the cookie was never returned and every authenticated flow failed → the test env now sets SESSION_COOKIE_SECURE=false (the documented dev opt-out).
M5 fail-closed returned 503 in tests → test env sets RFC_APP_INSECURE_BOUNCE_WEBHOOK=1 (mirrors the existing RFC_APP_INSECURE_WEBHOOKS pattern).
The new per-IP rate limiter is a process-global singleton with no per-test reset, so later tests tripped the budget → added ratelimit._reset_all_for_tests() + an autouse conftest.py fixture. Result: 363 backend tests green; frontend builds clean (0 vulns).

4. M2 + L4 applied out-of-band (the deploy gesture does not)

The flotilla deploy does not install nginx/systemd files. The repo's deploy/ templates also carried stale pre-rename paths (User=rfc-app, /opt/rfc-app/…) that would have taken the site down if copied verbatim. So:

M2/L8/I1 — injected the 5 security headers + server_tokens off into the live nginx 443 block (preserving the Certbot TLS config), nginx -t, reload. Verified CSP/HSTS/XFO/XCTO/Referrer live.
L4 — appended the sandbox block (CapabilityBoundingSet=, ProtectKernel*, RestrictAddressFamilies, MemoryDenyWriteExecute, SystemCallFilter=@system-service, UMask=0077, …) to the live ohm-rfc-app.service, with a backup + restart + health-check + auto-revert-on-failure. Service came up clean; no revert needed.

5. M5 secret (operator-run, never in-conversation)

WEBHOOK_EMAIL_BOUNCE_SECRET was confirmed absent from prod .env (the bounce webhook was unauthenticated). The operator minted it themselves (openssl rand | gcloud secrets create … --data-file=-) and bound the reference; v0.27.0's deploy landed it. The endpoint now returns 401 without the header (was 503-unset / would be 200-open). Note: OHM has no bounce source wired today (Google Workspace SMTP relay delivers bounces as DSN email, not webhooks), so the secret simply locks an otherwise-callable endpoint.

Friction / dead-ends (kept honest)

chmod 640 …/data/* first failed: after setting the dir to 750, the non-root SSH user couldn't expand the glob; re-ran under sudo sh -c.
After merging flotilla PR #1, git checkout main && git pull didn't fast-forward (no upstream tracking) and the tree briefly reverted; recovered with git merge --ff-only origin/main.
The operator hit a SyntaxError running secret bind: a separate trial-merge-v1.2.0 branch had a conflicted, unresolved merge of session-0028's flotilla v1.2.0 into the working tree (conflict markers in deploy.py, in the exact _PhaseRunner block L1 touched). Aborted the merge (non-destructive — branch + PR preserved) → CLI imports again.
First v0.27.0 deploy failed at phase 3: the VM pulls code from git.benstull.org/benstull/rfc-app, not the git.wiggleverse.org remote I'd pushed/tagged to. Dual-pushed main + the v0.27.0 tag to the benstull remote; redeploy succeeded.
60 backend test failures from the incomplete parallel branch (see §3).

Findings status at session end

Finding	Status
C1, H1, M1, M2, M4, M5, L2, L3, L4, L8, I1	Remediated + deployed (v0.27.0 / live VM)
L1, I6, L9, I5 (flotilla)	Merged + deployed (PR #1)
M3, L5-RDP, L6, L7 (live VM)	Applied + verified
L5 SSH `0.0.0.0/0`	Deferred to IAP rollout (§19.2)
I2 (webhook replay)	Acceptable (idempotent handlers)
I3 (dead HTML-email branch), I4 (sync siteverify)	Open — minor

Cross-references

flotilla PR #1 (wiggleverse/ohm-rfc-app-flotilla), merge 7b32a46; exercised by OHM deploy deploys.id=35.
rfc-app PR #1 (ben.stull/rfc-app), merge fe044ed, tag v0.27.0; dual-pushed to git.benstull.org/benstull/rfc-app (the VM's deploy source). OHM deploy deploys.id=37.
OHM pin: ohm-rfc/.rfc-app-version 0.26.0 → 0.27.0 (c91f7c0).
GCP: VM ohm-rfc-app/us-central1-a/wiggleverse-ohm; new SA ohm-rfc-app-vm@…; deleted FW rule default-allow-rdp; Secure Boot on.
Source audit: local-only ~/git/ohm-infra/SECURITY-AUDIT-0026-2026-05-28.md.

Operating instructions for the next session

The audit is remediated; the open threads are infra hardening and a stuck flotilla merge:

IAP SSH rollout (§19.2) — closes L5 (default-allow-ssh 0.0.0.0/0) and retires the L9 first-connect-TOFU residue. Steps: add an IAP firewall rule (35.235.240.0/20 → tcp:22), grant roles/iap.tunnelResourceAccessor, verify gcloud compute ssh --tunnel-through-iap works, then remove the 0.0.0.0/0 SSH rule, and flip the flotilla deploy path / deployment record to tunnel_through_iap=true. Don't remove 0.0.0.0/0 until IAP is verified.
Resolve trial-merge-v1.2.0 — to ship flotilla v1.2.0 (session 0028's PR), do a proper 3-way merge: it conflicts with the L1 redaction change in _PhaseRunner.run (both edited the same block). Keep both — the redaction wrapping AND v1.2.0's robustness/record-maintenance verbs.
Info items — I3 (delete the dead text/html email branch or guard it so HTML email can't ship without escaping) and I4 (make the Turnstile siteverify call async so a slow CloudFlare call can't block the event loop). Both small, both rfc-app-core.
Cleanup — sudo rm the orphaned ohm-app.db\r* files on the VM once confirmed orphaned; consider repo-side path-alignment of the stale deploy/nginx + deploy/systemd templates (/opt/rfc-app → /opt/ohm-rfc-app, User=rfc-app → ohm-rfc-app) so a future operator doesn't copy them verbatim.

Suggested next-session launch prompt:

Harden OHM SSH via IAP (§19.2): add the IAP firewall rule + tunnel role, verify IAP SSH, then remove the 0.0.0.0/0 SSH rule and switch flotilla's deploy path to tunnel-through-iap (closes audit-0026 L5 + L9). Then resolve the trial-merge-v1.2.0 conflict to ship flotilla v1.2.0 (it conflicts with the L1 redaction change in _PhaseRunner.run — keep both).

11 KiB Raw Blame History