docs(products): image pipeline ops + seams (DOC-1/DOC-4); overlay bucket config; v0.7.0
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -43,6 +43,8 @@ never file names, URLs, catalog content, or secret bytes.
|
||||
| TEL-2 `import_run_completed` | apply transaction commits | `run_id, storefront_id, added, updated, errored, duration_ms` |
|
||||
| TEL-3 `catalog_exported` | export stream completes | `storefront_id, status_filter, product_count, duration_ms` |
|
||||
| TEL-6 `import_apply_failed` | apply transaction aborts unexpectedly | `draft_id, storefront_id, error_class` |
|
||||
| TEL-4 `image_phase_completed` | run's fetch task finishes | `run_id, fetched, rejected, failed, duration_ms` |
|
||||
| TEL-5 `image_phase_recovered` | startup recovery resumes a run | `run_id, pending_resumed` |
|
||||
|
||||
### Export (PUC-9, SLICE-6)
|
||||
|
||||
@@ -56,6 +58,98 @@ re-importing an unmodified export previews as all-unchanged with the import
|
||||
action disabled (PUC-10). TEL-3 (`catalog_exported`) is emitted once the stream
|
||||
completes — counts and duration only, never catalog content.
|
||||
|
||||
### Images (SLICE-7)
|
||||
|
||||
After a merchant confirms an import, the app transitions the run to
|
||||
`fetching_images` and processes image URLs via a bounded thread pool
|
||||
(4 workers). Each image goes through the SSRF guard, is decoded, and up to
|
||||
four WebP renditions (`original`, `thumb`, `card`, `detail`) are written to
|
||||
the objectstore under `product-images/`. Progress is tracked per run in
|
||||
`image_progress` / `image_counts` / `image_outcomes` and visible in the
|
||||
import history detail panel (PUC-8). TEL-4 (`image_phase_completed`) fires
|
||||
when the phase finishes; startup recovery emits TEL-5 (`image_phase_recovered`)
|
||||
when it resumes a stuck run.
|
||||
|
||||
#### `provision-bucket` — one-time gesture per environment (SLICE-7 PPE deploy)
|
||||
|
||||
Before the first SLICE-7 deploy, run the `provision-bucket` skill from the
|
||||
engineering `launch-app` suite. This gesture:
|
||||
|
||||
1. Creates the GCS bucket `wiggleverse-ecomm-ppe-media` in the
|
||||
`wiggleverse-ecomm` GCP project.
|
||||
2. Grants the PPE VM's service account `roles/storage.objectAdmin` on the
|
||||
bucket.
|
||||
3. Sets an `import-drafts/` lifecycle rule (forward-compat for the draft-blob
|
||||
migration deferred past SLICE-7).
|
||||
|
||||
After provisioning, `deployment.toml`'s `[overlay]` already carries:
|
||||
|
||||
```
|
||||
ECOMM_OBJECTSTORE_KIND = "gcs"
|
||||
ECOMM_OBJECTSTORE_BUCKET = "wiggleverse-ecomm-ppe-media"
|
||||
```
|
||||
|
||||
so the next `flotilla deploy` picks them up automatically. GCS auth is the
|
||||
VM service-account ADC — no credential bytes needed.
|
||||
|
||||
### RB-3 — image phase stuck
|
||||
|
||||
Triggered by ALR-3 (run in `fetching_images` > 2 h, or the same run recovered
|
||||
≥ 3×).
|
||||
|
||||
1. **Identify the stuck run.** The import history (PUC-8, `GET
|
||||
/api/products/imports/runs`) lists runs by status; look for a run with
|
||||
`status="fetching_images"` that hasn't advanced.
|
||||
|
||||
```
|
||||
journalctl -u ecomm.service | grep image_phase
|
||||
```
|
||||
|
||||
2. **Restart to trigger recovery.** A process restart causes the startup
|
||||
recovery scan to re-enqueue pending images for any run still in
|
||||
`fetching_images` (TEL-5 confirms the resumption). Restarts are safe —
|
||||
per-image claims are idempotent; already-fetched images are skipped.
|
||||
|
||||
```
|
||||
systemctl restart ecomm.service
|
||||
```
|
||||
|
||||
3. **If a specific image URL is the cause.** The `image_outcomes` field on
|
||||
the run records per-image rejection reasons (SSRF block, resolution
|
||||
rejection, network error, etc.). The merchant can update or remove the
|
||||
offending URL and re-import.
|
||||
|
||||
4. **File a bug** on `wiggleverse/wiggleverse-ecomm` with the `run_id` and
|
||||
the `image_outcomes` content from the log.
|
||||
|
||||
### ALR-3 — the log-based alert for stuck image phases
|
||||
|
||||
Run once per environment, at the SLICE-7 PPE deploy. Mirrors ALR-2's
|
||||
approach — a log metric over the stuck/recovered signal, the existing email
|
||||
channel, and an alert policy in project `wiggleverse-ecomm`.
|
||||
|
||||
```
|
||||
# Select the deployment's gcloud config for this one process (handbook §8.4).
|
||||
export CLOUDSDK_ACTIVE_CONFIG_NAME=wiggleverse-ecomm
|
||||
|
||||
# Log-based metric counting image-phase events (TEL-4/TEL-5 — stuck/recovered).
|
||||
gcloud logging metrics create ecomm_image_phase_events \
|
||||
--description="ecomm TEL-4/TEL-5 image phase completed/recovered (SD-0002 ALR-3)" \
|
||||
--log-filter='resource.type="gce_instance" AND (jsonPayload.message:"image_phase_completed" OR jsonPayload.message:"image_phase_recovered" OR textPayload:"image_phase_completed" OR textPayload:"image_phase_recovered")'
|
||||
```
|
||||
|
||||
Then attach an alert policy — **operator email channel, threshold any event >
|
||||
0 within a 2-hour window, severity notify-only**. List channels first:
|
||||
|
||||
```
|
||||
# Find the operator email channel's id for the policy.
|
||||
gcloud beta monitoring channels list
|
||||
```
|
||||
|
||||
Create the alert policy (Cloud Console or `gcloud alpha monitoring policies
|
||||
create`); condition: `fetching_images` duration > 2 h **or** the same
|
||||
`run_id` appears in TEL-5 ≥ 3 times.
|
||||
|
||||
### RB-2 — import apply failed
|
||||
|
||||
Triggered by ALR-2 (any TEL-6 event). The apply raised mid-transaction and
|
||||
|
||||
Reference in New Issue
Block a user