docs(products): image pipeline ops + seams (DOC-1/DOC-4); overlay bucket config; v0.7.0
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
+84
-10
@@ -112,20 +112,94 @@ same through the UI (export download → re-upload → all-unchanged preview, im
|
||||
disabled). Numeric/`Decimal` fields — the property test's deliberate blind spot
|
||||
(string-form vs value-identity) — get explicit round-trip unit tests.
|
||||
|
||||
## Image pipeline (SLICE-7)
|
||||
|
||||
### `platform/objectstore`
|
||||
|
||||
A two-adapter port (`backend/app/platform/objectstore/`):
|
||||
|
||||
- `local.py` — writes blobs under a configurable local directory; used in
|
||||
tests and local dev (`ECOMM_OBJECTSTORE_KIND=local`).
|
||||
- `gcs.py` — wraps `google-cloud-storage`; bucket name comes from
|
||||
`ECOMM_OBJECTSTORE_BUCKET` (`ECOMM_OBJECTSTORE_KIND=gcs`). Auth is the
|
||||
VM's service-account ADC — no credential bytes in the overlay or secrets.
|
||||
|
||||
Object keys for product images follow the prefix `product-images/` (the
|
||||
`provision-bucket` gesture also sets a lifecycle rule on `import-drafts/` for
|
||||
forward-compat, even though the draft blob remains BYTEA this slice — see
|
||||
seams below).
|
||||
|
||||
### `platform/images`
|
||||
|
||||
`backend/app/platform/images.py` decodes an uploaded or fetched image byte
|
||||
string and produces up to four renditions stored in the objectstore:
|
||||
|
||||
- **`original`** — stored verbatim after the resolution bar passes.
|
||||
- **`thumb`** — 150 px on the shorter side, WebP.
|
||||
- **`card`** — 400 px on the shorter side, WebP.
|
||||
- **`detail`** — 800 px on the shorter side, WebP.
|
||||
|
||||
**Resolution bar (`MIN_IMAGE_SHORT_SIDE = 500`):** images whose shorter side
|
||||
is below 500 px are rejected (Q-3). This is the only hard gate; oversized
|
||||
images are scaled down without rejection. All renditions share the same
|
||||
objectstore key prefix (`product-images/<image_id>/`).
|
||||
|
||||
### `domains/products/imagefetch.py` — the fetch phase
|
||||
|
||||
After `confirm_draft` commits, the run enters `fetching_images` (if it has
|
||||
any pending image URLs) and a `ThreadPoolExecutor(max_workers=4)` processes
|
||||
them concurrently:
|
||||
|
||||
- **SSRF guard (INV-18, extended):** only `http`/`https` schemes are
|
||||
permitted; the resolved IP must be public (RFC-1918, loopback, link-local,
|
||||
and multicast ranges are blocked); redirects are followed only within the
|
||||
same guard — a redirect to a private IP is rejected mid-chain.
|
||||
- **Bounds:** ≤ 20 MB response body, ≤ 30 s per fetch.
|
||||
- **Per-image idempotent claim:** each `CatalogImage` row is claimed before
|
||||
fetch; a restart that finds a row already `claimed` skips it — making the
|
||||
phase resumable.
|
||||
- **Startup recovery scan:** on process start the app scans for runs stuck in
|
||||
`fetching_images` and re-enqueues their pending images (emits TEL-5). This
|
||||
covers crash/restart scenarios without a separate scheduler.
|
||||
- **Progress tracking:** the run row carries `image_progress`,
|
||||
`image_counts`, and `image_outcomes` (updated per image); these fields feed
|
||||
the run-detail UI panel.
|
||||
|
||||
On completion the phase sets the run status to `complete` and emits TEL-4.
|
||||
|
||||
### Image-serving route
|
||||
|
||||
`GET /api/products/images/{id}/{rendition}` — storefront-authorized,
|
||||
streams the objectstore blob. The response sets
|
||||
`Cache-Control: public, immutable, max-age=31536000` (the object key encodes
|
||||
the image id, so the URL is stable for the lifetime of the image). INV-16:
|
||||
an image belongs to exactly one storefront; the route enforces storefront
|
||||
scope before reading from the objectstore.
|
||||
|
||||
### Hosted-URL recognition and INV-12 over images
|
||||
|
||||
`domains/products/hosted.py` provides a predicate that recognises whether an
|
||||
image URL is already served by this app (i.e., a `GET /api/products/images/…`
|
||||
URL at the current `APP_URL`). The diff pre-pass `_resolve_hosted_images`
|
||||
uses it to convert hosted URLs back to their `CatalogImage` FK before hashing
|
||||
the diff fingerprint — this closes the INV-12 round-trip guarantee over
|
||||
image columns (a re-imported export previews as all-unchanged even when
|
||||
images are hosted here).
|
||||
|
||||
## Named seams (what later slices replace)
|
||||
|
||||
- **`import_draft.file_bytes` → objectstore key (SLICE-7).** The upload
|
||||
currently lives as BYTEA on the draft row (`0002_products.sql`); SLICE-7
|
||||
moves the bytes to object storage and stores a key.
|
||||
- **`import_draft.file_bytes` remains BYTEA (deferred past SLICE-7).** Moving
|
||||
the draft blob to object storage was scoped out of SLICE-7. The media
|
||||
bucket holds only `product-images/` objects this slice; the `provision-bucket`
|
||||
script sets the `import-drafts/` lifecycle rule for forward-compat so the
|
||||
migration will be clean when it lands.
|
||||
- **`codec.detect_dialect` → Shopify (SLICE-8).** Today it always returns
|
||||
`"canonical"`; SLICE-8 recognizes Shopify's exact header set here (INV-17).
|
||||
- **Run-status complete shortcut → `fetching_images` (SLICE-7).**
|
||||
`confirm_draft` inserts the run with `status="complete"` directly; SLICE-7
|
||||
inserts it as `fetching_images` and hands off to the image-fetch task.
|
||||
Relatedly, image rows are created with the schema default
|
||||
`status='pending'` **today and stay pending** — placeholder behavior until
|
||||
SLICE-7's fetch phase (the run-detail payload already carries the stable
|
||||
`image_progress` / `image_outcomes` shape, zeroed).
|
||||
- **Run-status complete shortcut → shipped in SLICE-7.**
|
||||
`confirm_draft` now inserts the run as `applying`, then transitions to
|
||||
`fetching_images` (if the run has pending image URLs) or directly to
|
||||
`complete`. The fetch phase fills `image_progress` / `image_counts` /
|
||||
`image_outcomes` and sets `complete` when done.
|
||||
|
||||
## Test map
|
||||
|
||||
|
||||
Reference in New Issue
Block a user