spec(SD-0002 §§8–11): metrics, telemetry TEL-1..6, alerts ALR-2/3 + runbooks RB-2..4, ops, docs plan DOC-1..4
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -1204,19 +1204,131 @@ reference + sample) published before SLICE-8 exposes the migration path;
|
|||||||
|
|
||||||
## 8. Success Metrics & Measurement
|
## 8. Success Metrics & Measurement
|
||||||
|
|
||||||
*(not yet authored)*
|
### 8.1 Business success metrics
|
||||||
|
|
||||||
|
| Outcome | Measured by (source) | Baseline captured how | Owner | Review cadence |
|
||||||
|
| --- | --- | --- | --- | --- |
|
||||||
|
| BO-1 (catalog adoption) | # storefronts with `product_count > 0`, and upload→confirmed-run conversion time (TEL-1, TEL-2 + DB) | Trivially zero pre-launch (no mechanism exists) | Ben | Monthly |
|
||||||
|
| BO-2 (in-platform upkeep) | # storefronts with ≥ 2 confirmed runs; days between runs (TEL-2) | Zero pre-launch | Ben | Monthly |
|
||||||
|
| BO-3 (no lock-in) | Export availability (uptime of the endpoint) + exports performed (TEL-3) | n/a — exists from SLICE-6 | Ben | Quarterly |
|
||||||
|
| BO-4 (downstream unblocked) | #14 / #15 delivery proceeding against imported catalogs | Tracker record, not telemetry | Ben | Per Feature |
|
||||||
|
|
||||||
|
### 8.2 Product success metrics
|
||||||
|
|
||||||
|
| # | Metric (scores PUC-n) | Target | Fed by | Review cadence |
|
||||||
|
| --- | --- | --- | --- | --- |
|
||||||
|
| PM-1 | Import completion: share of merchants who, having uploaded a file, reach a confirmed run within 24 h (PUC-2→4) | ≥ 80% | TEL-1 + TEL-2 | Monthly |
|
||||||
|
| PM-2 | Row-error rate per confirmed run — proxies how learnable the format and how actionable our errors are (PUC-5, PUC-11) | median < 5% | TEL-2 | Monthly |
|
||||||
|
| PM-3 | Image fetch success per run (PUC-7) | ≥ 95% fetched | TEL-4 | Monthly |
|
||||||
|
| PM-4 | Upload→preview latency, ≤ 5k-row files (PUC-3) | p95 < 10 s | TEL-1 | Monthly |
|
||||||
|
|
||||||
## 9. Telemetry, Observability & Monitoring
|
## 9. Telemetry, Observability & Monitoring
|
||||||
|
|
||||||
*(not yet authored)*
|
Scaled to the single-VM deployment: structured log events + a handful of
|
||||||
|
log-based alerts. Deliberately omitted: metrics stack, tracing, dashboards —
|
||||||
|
the runs list (PUC-8) *is* the import dashboard; revisit at shopper-traffic
|
||||||
|
scale (#24).
|
||||||
|
|
||||||
|
### 9.1 Telemetry
|
||||||
|
|
||||||
|
Log-event telemetry (structured JSON, app log). No file names, no URLs, no
|
||||||
|
catalog content in events — counts and durations only (file names/URLs stay
|
||||||
|
in DB records the merchant owns). No secret bytes ever (hard rule §6.3-handbook).
|
||||||
|
|
||||||
|
| # | Event / signal | Trigger | Payload (shape) | Privacy class / retention | Consumed by |
|
||||||
|
| --- | --- | --- | --- | --- | --- |
|
||||||
|
| TEL-1 | `import_draft_created` | validation completes, draft stored | `storefront_id, dialect, row_count, adds, updates, unchanged, errors, unknown_columns_count, duration_ms` | none / log retention (30 d) | PM-1, PM-4, BO-1 |
|
||||||
|
| TEL-2 | `import_run_completed` | apply transaction commits | `run_id, storefront_id, added, updated, errored, duration_ms` | none / 30 d | BO-1, BO-2, PM-1, PM-2 |
|
||||||
|
| TEL-3 | `catalog_exported` | export streamed | `storefront_id, status_filter, product_count, duration_ms` | none / 30 d | BO-3 |
|
||||||
|
| TEL-4 | `image_phase_completed` | a run's fetch task finishes | `run_id, fetched, rejected, failed, duration_ms` | none / 30 d | PM-3, ALR-3 |
|
||||||
|
| TEL-5 | `image_phase_recovered` | startup recovery resumes a run | `run_id, pending_resumed` | none / 30 d | ALR-3, RB-3 |
|
||||||
|
| TEL-6 | `import_apply_failed` | apply transaction aborts unexpectedly | `draft_id, storefront_id, error_class` | none / 30 d | ALR-2 |
|
||||||
|
|
||||||
|
### 9.2 Observability
|
||||||
|
|
||||||
|
- **Logs:** structured JSON to stdout → journald (Cloud Logging where the VM
|
||||||
|
agent ships them); every run-scoped line carries `run_id` (§6.6); 30-day
|
||||||
|
retention.
|
||||||
|
- **Metrics / traces:** none beyond TEL events — deliberate at this scale.
|
||||||
|
- **Health:** `/healthz` unchanged (process, DB, migrations — SD-0001).
|
||||||
|
Image-task health is *data*, not a health endpoint: runs stuck in
|
||||||
|
`fetching_images` are visible in the runs list and caught by ALR-3.
|
||||||
|
- **Dashboards:** none; "is the import healthy?" → the import history
|
||||||
|
(PUC-8); "why did it fail?" → logs filtered by `run_id`.
|
||||||
|
|
||||||
|
### 9.3 Monitoring & alerts
|
||||||
|
|
||||||
|
SLO/ALR numbering continues app-wide (SLO-1/ALR-1 are SD-0001's health-check
|
||||||
|
pair under the deploy contract).
|
||||||
|
|
||||||
|
| # | SLO | Window | Protects |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
| SLO-2 | 95% of validations (≤ 5k rows) complete < 10 s | 30 d | PUC-3 responsiveness (PM-4) |
|
||||||
|
| SLO-3 | 99% of image phases reach a terminal state < 60 min | 30 d | PUC-7 — outcomes arrive while the merchant still cares |
|
||||||
|
|
||||||
|
| # | Alert / monitor | Condition / threshold | Severity | Notifies | Runbook |
|
||||||
|
| --- | --- | --- | --- | --- | --- |
|
||||||
|
| ALR-2 | Import apply failing | any TEL-6 event | notify (email, log-based alert) | operator (Ben) | RB-2 |
|
||||||
|
| ALR-3 | Image phase stuck | run in `fetching_images` > 2 h, or same run recovered (TEL-5) ≥ 3× | notify | operator (Ben) | RB-3 |
|
||||||
|
|
||||||
|
Pre-v1 honesty: "notify" is a log-based email alert, no paging.
|
||||||
|
|
||||||
## 10. Operations
|
## 10. Operations
|
||||||
|
|
||||||
*(not yet authored)*
|
### 10.1 Operational plan
|
||||||
|
|
||||||
|
- **Operators / roles:** Ben (operator); no on-call — pre-v1 alerts are
|
||||||
|
notify-only.
|
||||||
|
- **Routine tasks:** none daily. Per release: the §7.3 pipeline. Monthly:
|
||||||
|
glance at bucket size/cost against the §10.3 guardrail. One-time per
|
||||||
|
environment: the `provision-bucket` flotilla gesture (SLICE-7).
|
||||||
|
- **Runbooks** (RB-1 is SD-0001's health-restore; numbering continues):
|
||||||
|
|
||||||
|
| # | Runbook | Answers | Lives at |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
| RB-2 | Import apply failed: locate by `draft_id`/`run_id` in logs; confirm rollback left catalog unchanged (INV-11); advise re-upload; file bug | ALR-2 | operator guide (framework repo), DOC-1 |
|
||||||
|
| RB-3 | Image phase stuck: check task log by `run_id`; verify bucket reachability + SA permissions; restart app — startup recovery resumes idempotently (§6.9) | ALR-3 | operator guide, DOC-1 |
|
||||||
|
| RB-4 | Provision the media bucket (the flotilla gesture, per environment) | SLICE-7 stand-up | operator guide, DOC-1 |
|
||||||
|
|
||||||
|
### 10.2 Support & incident response
|
||||||
|
|
||||||
|
Pre-v1, one line: the operator self-supports; problems are filed on the
|
||||||
|
tracker (`wiggleverse/wiggleverse-ecomm` issues); affected merchants (when
|
||||||
|
they exist) hear back on their issue.
|
||||||
|
|
||||||
|
### 10.3 Maintenance & lifecycle
|
||||||
|
|
||||||
|
- **Dependency patching:** Patchwatch → remediation pipeline, as app-wide.
|
||||||
|
Note the new patch-sensitive surface: **Pillow** (image decoding of
|
||||||
|
untrusted bytes) and the HTML sanitizer — prioritize their advisories.
|
||||||
|
- **Backup / restore:** catalog data rides Cloud SQL automated backups
|
||||||
|
(existing). Image objects: immutable keys + GCS soft delete (7 d) are the
|
||||||
|
MVP story; **no cross-region bucket backup yet** — accepted pre-v1 risk
|
||||||
|
(originals are re-importable only while merchant URLs live), revisit at
|
||||||
|
prod stand-up alongside the restore drill (Q-2, §13).
|
||||||
|
- **Capacity & cost:** ~5k images ≈ 10–15 GB ≈ well under $1/mo storage;
|
||||||
|
serving egress rides the VM. Guardrail: the project billing budget alert
|
||||||
|
(scaffold default) flags drift.
|
||||||
|
- **Deprecation / sunset:** this mechanism is the MVP's only mutation path;
|
||||||
|
when a future editing epic arrives, import/export persists as the bulk
|
||||||
|
path (no sunset) — only its "only mechanism" status retires.
|
||||||
|
|
||||||
## 11. Documentation plan
|
## 11. Documentation plan
|
||||||
|
|
||||||
*(not yet authored)*
|
- **Where docs live:** spec — content repo (this file); developer docs +
|
||||||
|
operator guide — framework repo; merchant-facing format docs — served by
|
||||||
|
the app itself (the admin links them, PUC-11).
|
||||||
|
|
||||||
|
| # | Document | Audience | Location | New / updated | Owned by slice |
|
||||||
|
| --- | --- | --- | --- | --- | --- |
|
||||||
|
| DOC-1 | Operator guide: import/export ops — RB-2/3/4, bucket gesture, caps/config surface | operator | framework repo operator guide | updated | every slice (5–8) |
|
||||||
|
| DOC-2 | CSV column reference (every column: meaning, required, accepted values; dialect notes) | merchant | app-served page, source in framework repo | new | SLICE-8 |
|
||||||
|
| DOC-3 | Sample CSV (worked examples incl. variants + images) | merchant | app-served asset (`sample.csv`) | new SLICE-5; finalized SLICE-8 | SLICE-5 / SLICE-8 |
|
||||||
|
| DOC-4 | `products` domain dev notes (canonical model, diff engine, dialect adapters, objectstore/images ports) | developer | framework repo | new | SLICE-5, extended 6–8 |
|
||||||
|
|
||||||
|
DOC-2/3 are the §7.3 launch gate for the migration path: published before
|
||||||
|
SLICE-8 ships. Docs update in the same PR/slice as the change they describe
|
||||||
|
(§4.1-handbook).
|
||||||
|
|
||||||
## 12. Traceability matrix
|
## 12. Traceability matrix
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user