Files
wiggleverse-ecomm-content/plans/2026-06-11-slice-6-export-roundtrip.md
T

1682 lines
66 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# SLICE-6 — Export & the Round-Trip Lock Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Ship the catalog export endpoint and lock the round-trip property — exporting a catalog and re-importing the unmodified file is a visible no-op (INV-12) — completing PUC-9 and PUC-10 of SD-0002.
**Architecture:** A new `serialize.py` in the `products` domain turns the catalog (the same `CatalogProduct` snapshot the diff engine already reads) back into canonical CSV — *one codec, two directions*. The BFF streams it at `GET /api/products/export?status=…`. The frontend's already-built Export button (currently disabled) becomes a status-filter menu that downloads. INV-12 holds because export emits exactly the columns the importer parses and the diff engine, re-reading that file against the live catalog, classifies every product as `unchanged`. PUC-10's no-op preview UX (the `nothing_to_apply` path, the "Nothing to change" note) already shipped in SLICE-5 — this slice adds the export half that makes it reachable, plus a regression that proves the loop.
**Tech Stack:** Python 3 / FastAPI / psycopg 3 / PostgreSQL 16 (backend); React + Vite + TypeScript / Vitest (frontend); Playwright (E2E). Gates: `scripts/check.sh` (lint-imports + pytest + frontend build + vitest) and `bash scripts/e2e.sh` (Playwright).
---
## Context the engineer needs before starting
**Read these first** — the slice extends, never rewrites, the SLICE-5 import spine:
- `wiggleverse-ecomm-content/specs/SD-0002-products-bulk-csv-import-export.md`
§5.2 (Products screen / Export menu), §6.4 (the `GET /api/products/export`
contract row), §6.5.1 (the canonical CSV format — **the serializer's
target**), §6.5.5 (Export), §6.8 (test strategy + the two named E2E
scenarios), §9.1 TEL-3, §7.2 SLICE-6 DoD. **The spec is the source of
truth; if the code proves it wrong, amend the spec then replan.**
- `backend/app/domains/products/models.py``PRODUCT_COLUMNS`,
`VARIANT_COLUMNS`, `OPTION_VALUE_COLUMNS`, `IMAGE_COLUMNS`, `CLEAR_DEFAULTS`.
These dicts map **column name → field name**; the serializer inverts them.
- `backend/app/domains/products/diff.py` — the `CatalogProduct` /
`CatalogVariant` / `CatalogImage` snapshot dataclasses and
`compute_diff()`. The serializer reads the **same** snapshot
`repo.load_catalog()` builds; the round-trip test calls `compute_diff()`.
- `backend/app/domains/products/codec.py` + `validate.py` — the parser the
export must round-trip *through*. Note the parse path: bytes →
`parse_csv``build_products``compute_diff`.
- `backend/app/domains/products/repo.py:load_catalog` — already returns the
catalog keyed by handle in snapshot shape; the export reuses it.
**Key invariants this slice must honor:**
- **INV-12** (the headline): `diff(catalog, import(export(catalog))) = ∅` over
text-field catalogs. *This slice's whole reason for being.*
- **INV-14**: every query storefront-scoped — `load_catalog(conn,
storefront_id)` already is; the export endpoint goes through `_merchant_gate`.
- **INV-10/11**: export is **read-only** — it must never write. (No draft, no
run; just streams.)
**Round-trip subtleties the serializer must get exactly right** (each has a task):
1. **Variant rows vs image rows.** Shopify's grammar (§6.5.1): consecutive
rows share a `Handle`; the first row carries product-level fields + the
first variant + (optionally) the first image; later rows carry additional
variants and/or additional images. A product with more images than variants
emits **image-only rows** (just `Handle` + `Image *`). The serializer must
interleave variants and images into rows the parser regroups identically.
2. **The no-option single variant.** A product with no options has one variant
whose option values are all NULL. On re-import, `validate._build_block`
treats the first row as carrying that single variant (`index == 0 and not
has_options`). The serializer must **not** emit `Option1 Value` etc. for it.
3. **Blank vs absent (§6.5.1).** A column *absent from the file* is untouched;
a cell *present but empty* clears to default. Export emits **every canonical
column** (full header), so on re-import every field is "present". For an
`unchanged` round-trip, an exported empty cell must resolve (via
`CLEAR_DEFAULTS` / parse) to **exactly** the catalog's current value. This
is why the round-trip test uses **text-field catalogs** (§7.2) — fields
whose serialize→parse is identity. Numeric/decimal/bool fields are covered
by explicit unit tests, not the property test's generator (see Task 6 note).
4. **Tags.** Stored as `TEXT[]`; the cell is comma-joined; parse splits on
comma and strips. Round-trips iff tags contain no commas and no
leading/trailing whitespace — the generator respects this.
5. **`Published` / `Status`.** Booleans serialize as `TRUE`/`FALSE`; status as
the lowercase word. Both have `CLEAR_DEFAULTS` entries (`published=True`,
`status="active"`), so the catalog default round-trips even when we emit
the explicit value (we always emit the explicit value — see Task 2).
---
## File Structure
**Backend (create):**
- `backend/app/domains/products/serialize.py` — the canonical serializer:
`catalog_to_csv(products: list[CatalogProduct]) -> Iterator[str]`, streaming
one logical chunk per product (header first). One responsibility: snapshot →
canonical CSV text. DB-free, mirrors `diff.py`'s placement.
- `backend/tests/test_products_serialize.py` — serializer unit tests + the
INV-12 property test (`diff(catalog, import(export(catalog))) = ∅`).
**Backend (modify):**
- `backend/app/domains/products/repo.py` — add `export_catalog(conn,
storefront_id, status_filter)` returning the (status-filtered) snapshot list.
- `backend/app/domains/products/service.py` — add `export_catalog(conn,
storefront_id, status_filter) -> Iterator[str]`; emits TEL-3.
- `backend/app/domains/products/__init__.py` — export the new service function
+ `EmptyCatalog` error.
- `backend/app/domains/products/errors.py` — add `EmptyCatalog`.
- `backend/app/main.py` — add `GET /api/products/export` (streamed,
`_merchant_gate`, `409 empty_catalog`).
**Frontend (modify):**
- `frontend/src/productsApi.ts` — add `EXPORT_STATUSES` + `exportUrl(status)`
helper (the endpoint is a plain download link, not a fetch wrapper).
- `frontend/src/screens/products/ProductsPage.tsx` — replace the disabled
Export button with a real status-filter menu that downloads; disable (with
the existing note) only when the catalog is empty.
- `frontend/src/screens/products/ProductsPage.export.test.tsx` — vitest for the
export menu (enabled/disabled, status → URL).
- `frontend/src/styles/products.css` — minimal styles for the export menu.
**E2E (create):**
- `e2e/tests/export-download.spec.ts` — `e2e_export_download` (status filter
respected).
- `e2e/tests/roundtrip-noop.spec.ts` — `e2e_roundtrip_noop` (export →
re-import → all-unchanged, import action disabled).
**Docs / version (modify):**
- `docs/OPERATIONS.md` — DOC-1: add export ops + TEL-3 row + the export E2E
scenarios.
- `docs/products-domain.md` — DOC-4: the serializer (one codec, two
directions) + INV-12 mechanics.
- `VERSION` + `frontend/package.json` — `0.5.0` → `0.6.0`.
---
## Task 1: Serializer skeleton — header + a single trivial product
**Files:**
- Create: `backend/app/domains/products/serialize.py`
- Create: `backend/tests/test_products_serialize.py`
The canonical CSV column order is fixed and must be a superset covering every
column the importer knows. We derive it once from the model registry so it can
never drift from what the parser accepts.
- [ ] **Step 1: Write the failing test**
```python
# backend/tests/test_products_serialize.py
"""Canonical serializer + INV-12 round-trip lock (SD-0002 §6.5.5, §6.8)."""
import csv
import io
from app.domains.products import serialize
from app.domains.products.diff import CatalogImage, CatalogProduct, CatalogVariant
def _product(**kw) -> CatalogProduct:
"""A minimal no-option catalog product (one all-NULL variant)."""
base = dict(
id=1, handle="moon-mug", title="Moon Mug",
option_names=(None, None, None),
fields={
"title": "Moon Mug", "description_html": None, "vendor": "Acme",
"product_type": "standalone", "google_product_category": None,
"tags": [], "status": "active", "published": True,
},
variants=[CatalogVariant(
id=1, options=(None, None, None), position=1,
fields={"sku": "WG-MUG", "barcode": None, "price": None, "cost": None,
"weight": None, "weight_unit": None, "volume": None,
"volume_unit": None, "tax_id_1": None, "tax_id_2": None,
"inventory_tracker": None, "inventory_qty": None,
"variant_image": None}],
)],
images=[],
)
base.update(kw)
return CatalogProduct(**base)
def _rows(products) -> list[dict]:
text = "".join(serialize.catalog_to_csv(products))
return list(csv.DictReader(io.StringIO(text)))
def test_header_is_full_canonical_set():
text = "".join(serialize.catalog_to_csv([_product()]))
header = next(csv.reader(io.StringIO(text)))
# Handle + Title first; every known column present exactly once.
assert header[0] == "Handle"
assert "Title" in header and "Variant SKU" in header and "Image Src" in header
assert len(header) == len(set(header))
def test_single_no_option_product_one_row():
rows = _rows([_product()])
assert len(rows) == 1
assert rows[0]["Handle"] == "moon-mug"
assert rows[0]["Title"] == "Moon Mug"
assert rows[0]["Variant SKU"] == "WG-MUG"
# A no-option product emits no option values.
assert rows[0]["Option1 Value"] == ""
```
- [ ] **Step 2: Run the test to verify it fails**
Run: `.venv/bin/python -m pytest backend/tests/test_products_serialize.py -q`
Expected: FAIL — `ModuleNotFoundError: No module named 'app.domains.products.serialize'`
- [ ] **Step 3: Write the minimal implementation**
```python
# backend/app/domains/products/serialize.py
"""Canonical serializer — CatalogProduct snapshot → canonical CSV (SD-0002 §6.5.5).
The export half of "one codec, two directions": this writes exactly the columns
codec.py/validate.py parse, in a row grammar (§6.5.1) the validator regroups
identically — so re-importing an unmodified export diffs to nothing (INV-12).
DB-free, like diff.py: it consumes the same CatalogProduct snapshot the diff
engine reads (repo.load_catalog), and the service streams the result.
"""
from __future__ import annotations
import csv
import io
from collections.abc import Iterable, Iterator
from decimal import Decimal
from .diff import CatalogProduct
from .models import (
IMAGE_COLUMNS,
OPTION_VALUE_COLUMNS,
PRODUCT_COLUMNS,
VARIANT_COLUMNS,
)
# Canonical column order: Handle, then product, option-value, variant, image
# columns. A superset of everything the parser knows (models.KNOWN_COLUMNS minus
# the #15-reserved Component columns, which export never emits).
HEADER: list[str] = [
"Handle",
*PRODUCT_COLUMNS,
*OPTION_VALUE_COLUMNS,
*VARIANT_COLUMNS,
*IMAGE_COLUMNS,
]
# field name -> column name, inverting the model registry (the parser reads
# column->field; the serializer writes field->column).
_PRODUCT_FIELD_TO_COL = {field: col for col, field in PRODUCT_COLUMNS.items()}
_VARIANT_FIELD_TO_COL = {field: col for col, field in VARIANT_COLUMNS.items()}
def _cell(value: object) -> str:
"""Serialize one value to its canonical cell text (the parse inverse)."""
if value is None:
return ""
if isinstance(value, bool):
return "TRUE" if value else "FALSE"
if isinstance(value, Decimal):
return str(value)
if isinstance(value, (list, tuple)):
return ", ".join(str(v) for v in value)
return str(value)
def catalog_to_csv(products: Iterable[CatalogProduct]) -> Iterator[str]:
"""Stream canonical CSV text, header first, one product block at a time."""
buf = io.StringIO()
writer = csv.writer(buf)
writer.writerow(HEADER)
yield _drain(buf)
for product in products:
for row in _product_rows(product):
writer.writerow([row.get(col, "") for col in HEADER])
yield _drain(buf)
def _drain(buf: io.StringIO) -> str:
text = buf.getvalue()
buf.seek(0)
buf.truncate(0)
return text
def _product_rows(product: CatalogProduct) -> list[dict[str, str]]:
"""The product's CSV rows (§6.5.1 grammar). Task 1: first variant only."""
base = {"Handle": product.handle, "Title": product.title}
for field, col in _PRODUCT_FIELD_TO_COL.items():
if field in product.fields:
base[col] = _cell(product.fields[field])
for slot, name in enumerate(product.option_names, start=1):
if name:
base[f"Option{slot} Name"] = name
variant = product.variants[0]
row = dict(base)
_write_variant(row, product, variant)
return [row]
def _write_variant(row: dict[str, str], product: CatalogProduct, variant) -> None:
"""Fill a row's option-value + variant columns for one variant."""
for slot, value in enumerate(variant.options, start=1):
# Only emit an option value when the product actually has that option
# (a no-option product's single variant carries all-NULL options).
if product.option_names[slot - 1] and value is not None:
row[f"Option{slot} Value"] = value
for field, col in _VARIANT_FIELD_TO_COL.items():
if field in variant.fields:
row[col] = _cell(variant.fields[field])
```
- [ ] **Step 4: Run the test to verify it passes**
Run: `.venv/bin/python -m pytest backend/tests/test_products_serialize.py -q`
Expected: PASS (2 passed)
- [ ] **Step 5: Commit**
```bash
git add backend/app/domains/products/serialize.py backend/tests/test_products_serialize.py
git commit -m "feat(products): canonical serializer skeleton — header + single product row (SD-0002 §6.5.5)"
```
---
## Task 2: Multi-variant + image interleaving (the row grammar)
**Files:**
- Modify: `backend/app/domains/products/serialize.py` (`_product_rows`)
- Modify: `backend/tests/test_products_serialize.py`
A product emits as many rows as `max(variants, images)`: row *i* carries
variant *i* (if any) and image *i* (if any). Variant *0* shares the product's
first row. This mirrors `validate._build_block`, which reads variants and
images off whatever rows carry them.
- [ ] **Step 1: Write the failing test**
```python
# append to backend/tests/test_products_serialize.py
from app.domains.products.diff import CatalogImage, CatalogVariant
def _star_tee() -> CatalogProduct:
"""A 3-variant, 2-image product (the sample.csv shape)."""
return CatalogProduct(
id=2, handle="star-tee", title="Star Tee",
option_names=("Size", "Color", None),
fields={"title": "Star Tee", "description_html": None, "vendor": "Acme",
"product_type": "standalone", "google_product_category": None,
"tags": ["apparel", "tees"], "status": "active", "published": True},
variants=[
CatalogVariant(id=10, options=("S", "Indigo", None), position=1,
fields={"sku": "WG-TEE-S", "variant_image": None}),
CatalogVariant(id=11, options=("M", "Indigo", None), position=2,
fields={"sku": "WG-TEE-M", "variant_image": None}),
CatalogVariant(id=12, options=("L", "Indigo", None), position=3,
fields={"sku": "WG-TEE-L", "variant_image": None}),
],
images=[
CatalogImage(id=1, source_url="https://x/a.jpg", position=1, alt_text="front"),
CatalogImage(id=2, source_url="https://x/b.jpg", position=2, alt_text="back"),
],
)
def test_multivariant_with_images_interleaves():
rows = _rows([_star_tee()])
assert len(rows) == 3 # max(3 variants, 2 images)
# Product-level fields only on the first row.
assert rows[0]["Title"] == "Star Tee" and rows[1]["Title"] == ""
assert rows[0]["Tags"] == "apparel, tees" and rows[1]["Tags"] == ""
# Option names on row 0; option values on every variant row.
assert rows[0]["Option1 Name"] == "Size" and rows[1]["Option1 Name"] == ""
assert [r["Option1 Value"] for r in rows] == ["S", "M", "L"]
assert [r["Variant SKU"] for r in rows] == ["WG-TEE-S", "WG-TEE-M", "WG-TEE-L"]
# Two images on the first two rows; third row has no image.
assert [r["Image Src"] for r in rows] == ["https://x/a.jpg", "https://x/b.jpg", ""]
assert [r["Image Position"] for r in rows] == ["1", "2", ""]
def test_more_images_than_variants_emits_image_only_rows():
p = _product(images=[
CatalogImage(id=1, source_url="https://x/a.jpg", position=1, alt_text=None),
CatalogImage(id=2, source_url="https://x/b.jpg", position=2, alt_text=None),
CatalogImage(id=3, source_url="https://x/c.jpg", position=3, alt_text=None),
])
rows = _rows([p])
assert len(rows) == 3 # 1 variant, 3 images
assert rows[0]["Variant SKU"] == "WG-MUG"
assert rows[1]["Variant SKU"] == "" and rows[1]["Handle"] == "moon-mug"
assert [r["Image Src"] for r in rows] == ["https://x/a.jpg", "https://x/b.jpg", "https://x/c.jpg"]
```
- [ ] **Step 2: Run to verify it fails**
Run: `.venv/bin/python -m pytest backend/tests/test_products_serialize.py -q`
Expected: FAIL — `test_multivariant_with_images_interleaves` (only 1 row produced).
- [ ] **Step 3: Replace `_product_rows` with the full grammar**
```python
def _product_rows(product: CatalogProduct) -> list[dict[str, str]]:
"""The product's CSV rows (§6.5.1 grammar): product fields + option names on
the first row; one variant per row; images interleaved; image-only rows when
a product has more images than variants."""
base = {"Handle": product.handle, "Title": product.title}
for field, col in _PRODUCT_FIELD_TO_COL.items():
if field in product.fields:
base[col] = _cell(product.fields[field])
for slot, name in enumerate(product.option_names, start=1):
if name:
base[f"Option{slot} Name"] = name
count = max(len(product.variants), len(product.images))
rows: list[dict[str, str]] = []
for i in range(count):
# Row 0 carries the product-level fields; later rows carry only Handle.
row = dict(base) if i == 0 else {"Handle": product.handle}
if i < len(product.variants):
_write_variant(row, product, product.variants[i])
if i < len(product.images):
_write_image(row, product.images[i])
rows.append(row)
return rows
def _write_image(row: dict[str, str], image) -> None:
row["Image Src"] = image.source_url
row["Image Position"] = str(image.position)
if image.alt_text is not None:
row["Image Alt Text"] = image.alt_text
```
Note: the `_product_rows` from Task 1 is fully replaced; `_write_variant`
stays as written in Task 1.
- [ ] **Step 4: Run to verify it passes**
Run: `.venv/bin/python -m pytest backend/tests/test_products_serialize.py -q`
Expected: PASS (4 passed)
- [ ] **Step 5: Commit**
```bash
git add backend/app/domains/products/serialize.py backend/tests/test_products_serialize.py
git commit -m "feat(products): serialize multi-variant + interleaved images (§6.5.1 row grammar)"
```
---
## Task 3: INV-12 property test — `diff(catalog, import(export(catalog))) = ∅`
**Files:**
- Modify: `backend/tests/test_products_serialize.py`
The headline lock. Generate arbitrary **text-field** catalogs, serialize them,
parse the result back through the real importer pipeline, diff against the
generating catalog, and assert every product is `unchanged` (zero adds, zero
updates, zero errors). Text-field per §7.2: the generator varies handles,
titles, vendors, option names/values, SKUs, tags, statuses, published, images —
all fields whose serialize→parse is identity. (Decimal/qty fields get explicit
unit tests in Task 6, not the generator, because e.g. `"18.0"` vs `"18.00"`
parse equal but aren't string-identical — a property-test trap, not a real bug.)
- [ ] **Step 1: Write the failing property test**
```python
# append to backend/tests/test_products_serialize.py
import random
from app.domains.products import codec, diff, validate
def _gen_catalog(seed: int) -> dict:
"""A deterministic random text-field catalog, in load_catalog()'s shape."""
rng = random.Random(seed)
words = ["moon", "star", "river", "cloud", "ember", "fern", "slate", "wave"]
catalog: dict[str, CatalogProduct] = {}
n_products = rng.randint(1, 6)
for pid in range(1, n_products + 1):
handle = "-".join(rng.sample(words, rng.randint(1, 3))) + f"-{pid}"
if handle in catalog:
continue
has_opts = rng.random() < 0.6
option_names = ("Size", "Color", None) if has_opts else (None, None, None)
tags = rng.sample(["apparel", "kitchen", "sale", "new"], rng.randint(0, 3))
fields = {
"title": f"{handle.title()} Thing",
"description_html": rng.choice([None, "<p>hi</p>"]),
"vendor": rng.choice([None, "Acme", "Wiggle Goods"]),
"product_type": "standalone",
"google_product_category": rng.choice([None, "Home & Garden"]),
"tags": tags,
"status": rng.choice(["draft", "active", "archived"]),
"published": rng.choice([True, False]),
}
variants = []
if has_opts:
sizes = rng.sample(["S", "M", "L", "XL"], rng.randint(1, 4))
for vi, size in enumerate(sizes, start=1):
variants.append(CatalogVariant(
id=pid * 100 + vi, options=(size, "Indigo", None), position=vi,
fields={"sku": f"SKU-{pid}-{vi}", "variant_image": None}))
else:
variants.append(CatalogVariant(
id=pid * 100 + 1, options=(None, None, None), position=1,
fields={"sku": f"SKU-{pid}", "variant_image": None}))
images = []
for ii in range(rng.randint(0, 3)):
images.append(CatalogImage(
id=pid * 10 + ii, source_url=f"https://img/{handle}-{ii}.jpg",
position=ii + 1, alt_text=rng.choice([None, f"alt {ii}"])))
catalog[handle] = CatalogProduct(
id=pid, handle=handle, title=fields["title"], option_names=option_names,
fields=fields, variants=variants, images=images)
return catalog
def _roundtrip_diff(catalog: dict) -> diff.DiffResult:
"""export → bytes → import pipeline → diff against the same catalog."""
text = "".join(serialize.catalog_to_csv(catalog.values()))
parsed = codec.parse_csv(text.encode("utf-8"))
products = validate.build_products(parsed)
return diff.compute_diff(catalog, products)
def test_inv12_roundtrip_is_noop_over_generated_catalogs():
for seed in range(200):
catalog = _gen_catalog(seed)
result = _roundtrip_diff(catalog)
assert result.summary["adds"] == 0, f"seed {seed}: {result.summary}"
assert result.summary["updates"] == 0, f"seed {seed}: {result.summary}"
assert result.summary["errors"] == 0, f"seed {seed}: {result.summary}"
assert result.summary["unchanged"] == len(catalog), f"seed {seed}"
```
- [ ] **Step 2: Run to see what breaks**
Run: `.venv/bin/python -m pytest backend/tests/test_products_serialize.py::test_inv12_roundtrip_is_noop_over_generated_catalogs -q`
Expected: This is the integration moment. It may already PASS (Tasks 12 built
the grammar correctly), or FAIL on a specific seed exposing a serialize/parse
asymmetry. **If it fails, that is a real INV-12 bug** — debug with
`superpowers:systematic-debugging`: print the offending product's
`result.records` entry to see which field/variant/image the diff flagged, fix
`serialize.py` (or, if the spec is genuinely wrong, amend the spec then
replan), and re-run. Likely culprits to check in order:
1. **Option values for no-option products** — the all-NULL variant must emit
no `Option*Value` (Task 1 `_write_variant` guards this).
2. **`description_html`** — `nh3.clean("<p>hi</p>")` must equal the stored
value. The generator uses already-sanitized HTML so this holds; if it
flags, the catalog's stored HTML wasn't canonical (not an export bug).
3. **Image position** — exported positions must match stored; the generator
uses sequential 1..n matching `_write_image`.
4. **Tags ordering/whitespace** — `", ".join` then split-and-strip must
reproduce the list; the generator avoids commas-in-tags.
- [ ] **Step 3: Fix any asymmetry found (no fix needed if it already passes)**
Apply the minimal `serialize.py` change the failure points to. Re-run until green.
- [ ] **Step 4: Run the whole serializer file**
Run: `.venv/bin/python -m pytest backend/tests/test_products_serialize.py -q`
Expected: PASS (all serializer tests, incl. INV-12 over 200 catalogs)
- [ ] **Step 5: Commit**
```bash
git add backend/tests/test_products_serialize.py backend/app/domains/products/serialize.py
git commit -m "test(products): INV-12 property test — export round-trips to a no-op (SD-0002 §6.8)"
```
---
## Task 4: Repo — status-filtered catalog export query
**Files:**
- Modify: `backend/app/domains/products/repo.py`
- Modify: `backend/tests/test_products_serialize.py` (add a DB-backed test) — or
a new `backend/tests/test_products_export.py` (preferred; keep serializer
unit tests DB-free).
- Create: `backend/tests/test_products_export.py`
`load_catalog` already builds the full snapshot but takes no status filter.
Rather than thread a filter through it (variants/images join off products),
add a thin `export_catalog` that reuses `load_catalog` and filters products by
status in Python — the catalog fits in memory (≤5k rows, INV-18) and this keeps
one snapshot-builder. Status `all` returns everything.
- [ ] **Step 1: Write the failing test**
```python
# backend/tests/test_products_export.py
"""Export: status-filtered catalog snapshot + the streamed service (PUC-9, TEL-3)."""
import csv
import io
import json
import logging
import psycopg
import pytest
from app.domains import products
from app.domains.products import repo, serialize
from app.platform import db
CSV = (
b"Handle,Title,Vendor,Status,Variant Price\n"
b"active-mug,Active Mug,Acme,active,18.00\n"
b"draft-tee,Draft Tee,Acme,draft,24.00\n"
)
@pytest.fixture()
def migrated_conn(fresh_db_url):
with psycopg.connect(fresh_db_url) as conn:
db.migrate(conn)
yield conn
@pytest.fixture()
def merchant(migrated_conn):
acct = migrated_conn.execute(
"INSERT INTO account (email) VALUES ('m@example.com') RETURNING id").fetchone()[0]
sf = migrated_conn.execute(
"INSERT INTO storefront (name) VALUES ('Shop') RETURNING id").fetchone()[0]
migrated_conn.execute(
"INSERT INTO storefront_membership (account_id, storefront_id) VALUES (%s,%s)", (acct, sf))
migrated_conn.commit()
return {"account_id": acct, "storefront_id": sf}
def _seed(conn, merchant):
draft = products.import_validate(conn, merchant["storefront_id"], merchant["account_id"], "c.csv", CSV)
products.confirm_draft(conn, merchant["storefront_id"], merchant["account_id"], draft["id"])
def test_export_all_returns_both(migrated_conn, merchant):
_seed(migrated_conn, merchant)
snap = repo.export_catalog(migrated_conn, merchant["storefront_id"], "all")
assert {p.handle for p in snap} == {"active-mug", "draft-tee"}
def test_export_status_filter(migrated_conn, merchant):
_seed(migrated_conn, merchant)
active = repo.export_catalog(migrated_conn, merchant["storefront_id"], "active")
assert [p.handle for p in active] == ["active-mug"]
draft = repo.export_catalog(migrated_conn, merchant["storefront_id"], "draft")
assert [p.handle for p in draft] == ["draft-tee"]
```
- [ ] **Step 2: Run to verify it fails**
Run: `.venv/bin/python -m pytest backend/tests/test_products_export.py -q`
Expected: FAIL — `AttributeError: module ... has no attribute 'export_catalog'`
- [ ] **Step 3: Add `export_catalog` to repo.py**
```python
# in backend/app/domains/products/repo.py, after load_catalog()
_EXPORT_STATUSES = ("all", "active", "draft", "archived")
def export_catalog(
conn: psycopg.Connection, storefront_id: int, status_filter: str
) -> list[CatalogProduct]:
"""The storefront's catalog as an ordered snapshot list, optionally filtered
by product status (PUC-9). Reuses load_catalog's snapshot builder; the
catalog fits in memory (≤5k rows, INV-18). Ordered by handle for a stable,
deterministic export."""
catalog = load_catalog(conn, storefront_id)
products = sorted(catalog.values(), key=lambda p: p.handle)
if status_filter and status_filter != "all":
products = [p for p in products if p.fields.get("status") == status_filter]
return products
```
- [ ] **Step 4: Run to verify it passes**
Run: `.venv/bin/python -m pytest backend/tests/test_products_export.py -q`
Expected: PASS (2 passed)
- [ ] **Step 5: Commit**
```bash
git add backend/app/domains/products/repo.py backend/tests/test_products_export.py
git commit -m "feat(products): status-filtered export_catalog snapshot query (PUC-9)"
```
---
## Task 5: Service — streamed export + TEL-3, and the `EmptyCatalog` error
**Files:**
- Modify: `backend/app/domains/products/errors.py`
- Modify: `backend/app/domains/products/service.py`
- Modify: `backend/app/domains/products/__init__.py`
- Modify: `backend/tests/test_products_export.py`
The service streams the serialized CSV and emits TEL-3 once, after the stream
completes, with `product_count` and `duration_ms`. An empty (post-filter)
catalog raises `EmptyCatalog` → the BFF maps it to `409 empty_catalog` (PUC-9:
Export disabled with a note when empty; the endpoint defends the same rule).
Streaming + telemetry-after-completion means the service returns a generator
that emits TEL-3 as its final act. The count is known up front (the snapshot
list), so we capture it before yielding.
- [ ] **Step 1: Write the failing test**
```python
# append to backend/tests/test_products_export.py
@pytest.fixture()
def telemetry_propagation():
"""ecomm.telemetry has propagate=False in app logging; re-enable for caplog."""
lg = logging.getLogger("ecomm.telemetry")
old = lg.propagate
lg.propagate = True
try:
yield
finally:
lg.propagate = old
def test_export_streams_canonical_csv(migrated_conn, merchant):
_seed(migrated_conn, merchant)
text = "".join(products.export_catalog(migrated_conn, merchant["storefront_id"], "all"))
rows = list(csv.DictReader(io.StringIO(text)))
assert {r["Handle"] for r in rows} == {"active-mug", "draft-tee"}
assert rows[0]["Handle"] == "active-mug" # sorted by handle
def test_export_empty_raises(migrated_conn, merchant):
# No catalog at all → empty.
with pytest.raises(products.EmptyCatalog):
list(products.export_catalog(migrated_conn, merchant["storefront_id"], "all"))
def test_export_empty_after_filter_raises(migrated_conn, merchant):
_seed(migrated_conn, merchant) # only active + draft exist
with pytest.raises(products.EmptyCatalog):
list(products.export_catalog(migrated_conn, merchant["storefront_id"], "archived"))
def test_tel3_emitted(migrated_conn, merchant, caplog, telemetry_propagation):
_seed(migrated_conn, merchant)
with caplog.at_level(logging.INFO, logger="ecomm.telemetry"):
list(products.export_catalog(migrated_conn, merchant["storefront_id"], "all"))
events = [json.loads(r.message) for r in caplog.records if r.name == "ecomm.telemetry"]
exported = [e for e in events if e["event"] == "catalog_exported"]
assert len(exported) == 1
assert exported[0]["product_count"] == 2
assert exported[0]["status_filter"] == "all"
assert "duration_ms" in exported[0]
```
- [ ] **Step 2: Run to verify it fails**
Run: `.venv/bin/python -m pytest backend/tests/test_products_export.py -q`
Expected: FAIL — `AttributeError: module 'app.domains.products' has no attribute 'export_catalog'` / `EmptyCatalog`.
- [ ] **Step 3a: Add `EmptyCatalog` to errors.py**
```python
# in backend/app/domains/products/errors.py, after NothingToApply
class EmptyCatalog(ProductsError):
"""PUC-9: nothing to export (no products, or none matching the status filter)."""
```
- [ ] **Step 3b: Add the streaming service to service.py**
```python
# in backend/app/domains/products/service.py
# add to the existing imports:
from collections.abc import Iterator
from . import codec, diff, repo, serialize, validate
from .errors import (
DraftExpired, DraftNotFound, EmptyCatalog, NothingToApply, PreviewStale, RunNotFound,
)
# add this function (e.g. after summary()):
def export_catalog(
conn: psycopg.Connection, storefront_id: int, status_filter: str
) -> Iterator[str]:
"""Stream the storefront's catalog as canonical CSV (PUC-9; INV-12 codec).
Read-only: builds the snapshot, then streams the serializer over it. TEL-3
is emitted once the stream is exhausted, with the product count and elapsed
time. Raises EmptyCatalog before yielding anything if the (filtered) catalog
is empty, so the BFF can answer 409 cleanly with no partial body.
"""
started = time.monotonic()
snapshot = repo.export_catalog(conn, storefront_id, status_filter)
if not snapshot:
raise EmptyCatalog()
def _stream() -> Iterator[str]:
yield from serialize.catalog_to_csv(snapshot)
telemetry.emit(
"catalog_exported",
storefront_id=storefront_id,
status_filter=status_filter,
product_count=len(snapshot),
duration_ms=int((time.monotonic() - started) * 1000),
)
return _stream()
```
Note: `repo.export_catalog` (snapshot, Task 4) and the service `export_catalog`
(stream, here) share a name across layers — same intent, different layer, like
`list_runs`/`get_run` already do. The empty check runs **eagerly** (before the
generator is consumed) so `409` is decided before any bytes stream.
- [ ] **Step 3c: Export from the package surface**
```python
# in backend/app/domains/products/__init__.py
# add EmptyCatalog to the errors import and __all__, and export_catalog to the
# service import and __all__:
from .errors import (
DraftExpired, DraftNotFound, EmptyCatalog, FileRejected, NothingToApply,
PreviewStale, ProductsError, RunNotFound,
)
from .service import (
confirm_draft, discard_draft, export_catalog, get_draft, get_draft_records,
get_run, import_validate, list_runs, summary,
)
# __all__ gains: "EmptyCatalog", "export_catalog"
```
- [ ] **Step 4: Run to verify it passes**
Run: `.venv/bin/python -m pytest backend/tests/test_products_export.py -q`
Expected: PASS (6 passed)
- [ ] **Step 5: Commit**
```bash
git add backend/app/domains/products/errors.py backend/app/domains/products/service.py backend/app/domains/products/__init__.py backend/tests/test_products_export.py
git commit -m "feat(products): streamed export service + TEL-3 + EmptyCatalog (PUC-9, §9.1)"
```
---
## Task 6: Decimal / numeric round-trip unit tests (the property test's blind spot)
**Files:**
- Modify: `backend/tests/test_products_serialize.py`
The INV-12 property test deliberately uses text fields. Numeric fields
(`price`, `cost`, `weight`, `volume`, `inventory_qty`, variant `position`) need
explicit round-trip coverage because their serialize→parse is value-identity,
not string-identity. We assert the **diff** is empty (the real invariant), not
the bytes. This catches e.g. a serializer dropping a Decimal's trailing zero in
a way the parser then reads back differently.
- [ ] **Step 1: Write the failing/covering test**
```python
# append to backend/tests/test_products_serialize.py
from decimal import Decimal
def test_decimal_and_int_variant_fields_roundtrip():
catalog = {
"priced": CatalogProduct(
id=1, handle="priced", title="Priced",
option_names=(None, None, None),
fields={"title": "Priced", "description_html": None, "vendor": None,
"product_type": "standalone", "google_product_category": None,
"tags": [], "status": "active", "published": True},
variants=[CatalogVariant(
id=1, options=(None, None, None), position=1,
fields={"sku": "P1", "price": Decimal("18.00"),
"cost": Decimal("9.5"), "weight": Decimal("0.250"),
"inventory_qty": 40, "variant_image": None})],
images=[],
)
}
result = _roundtrip_diff(catalog)
assert result.summary["unchanged"] == 1, result.records
assert result.summary["updates"] == 0
```
- [ ] **Step 2: Run it**
Run: `.venv/bin/python -m pytest backend/tests/test_products_serialize.py::test_decimal_and_int_variant_fields_roundtrip -q`
Expected: PASS if Decimals round-trip cleanly (psycopg loads `NUMERIC` as
`Decimal`, the diff compares `Decimal == Decimal`). **If it FAILS**, the diff
flagged a numeric field — inspect `result.records[0]["detail"]`; the cause is
almost always a `str(Decimal)` form the parser re-parses to a different
`Decimal` (it shouldn't: `Decimal(str(d)) == d` always). Fix `serialize._cell`
only if a real asymmetry exists; otherwise the test documents the guarantee.
- [ ] **Step 3: (only if it failed) Fix `_cell`**
No change expected; `Decimal(str(d)) == d` holds. If weight stored as
`Decimal("0.250")` re-parses to `Decimal("0.250")` (it does — equal value),
the diff's `!=` is False. Leave `_cell` as is.
- [ ] **Step 4: Run the full serializer suite**
Run: `.venv/bin/python -m pytest backend/tests/test_products_serialize.py -q`
Expected: PASS (all)
- [ ] **Step 5: Commit**
```bash
git add backend/tests/test_products_serialize.py
git commit -m "test(products): decimal/int variant fields round-trip clean (INV-12 coverage)"
```
---
## Task 7: BFF — `GET /api/products/export` (streamed, gated, 409 empty)
**Files:**
- Modify: `backend/app/main.py`
- Modify: `backend/tests/test_products_endpoints.py`
The endpoint: `_merchant_gate` → `products.export_catalog(...)` → stream as
`text/csv` with a `content-disposition` attachment filename. `409 empty_catalog`
on `EmptyCatalog`. The `status` query param is validated to the four allowed
values (default `all`). Uses FastAPI's `StreamingResponse`.
- [ ] **Step 1: Write the failing test**
```python
# append to backend/tests/test_products_endpoints.py
def test_export_returns_canonical_csv(fresh_db_url):
with _merchant_client(fresh_db_url) as client:
draft = _upload(client).json()
client.post(f"/api/products/imports/drafts/{draft['id']}/confirm")
resp = client.get("/api/products/export")
assert resp.status_code == 200
assert resp.headers["content-type"].startswith("text/csv")
assert "attachment" in resp.headers["content-disposition"]
body = resp.text
assert body.splitlines()[0].startswith("Handle,")
assert "moon-mug" in body
def test_export_status_filter_respected(fresh_db_url):
with _merchant_client(fresh_db_url) as client:
# GOOD_CSV's moon-mug has no Status column → defaults to active.
draft = _upload(client).json()
client.post(f"/api/products/imports/drafts/{draft['id']}/confirm")
assert "moon-mug" in client.get("/api/products/export?status=active").text
# No archived products → 409.
assert client.get("/api/products/export?status=archived").status_code == 409
def test_export_empty_catalog_409(fresh_db_url):
with _merchant_client(fresh_db_url) as client:
resp = client.get("/api/products/export")
assert resp.status_code == 409
assert resp.json()["error"]["code"] == "empty_catalog"
def test_export_requires_merchant(fresh_db_url):
with TestClient(create_app(database_url=fresh_db_url)) as client:
assert client.get("/api/products/export").status_code == 401
def test_export_bad_status_422(fresh_db_url):
with _merchant_client(fresh_db_url) as client:
assert client.get("/api/products/export?status=bogus").status_code == 422
```
- [ ] **Step 2: Run to verify it fails**
Run: `.venv/bin/python -m pytest backend/tests/test_products_endpoints.py -k export -q`
Expected: FAIL — 404 (route not defined).
- [ ] **Step 3: Add the endpoint to main.py**
```python
# in backend/app/main.py
# add StreamingResponse to the fastapi.responses import:
from fastapi.responses import JSONResponse, PlainTextResponse, StreamingResponse
# add this route, next to the other /api/products/* routes (e.g. after products_sample_csv
# or before it — order is irrelevant for distinct paths):
@app.get("/api/products/export")
def export_products(
status: str = Query(default="all", pattern="^(all|active|draft|archived)$"),
conn: psycopg.Connection = Depends(get_conn),
sess: dict | None = Depends(get_session),
):
"""Stream the catalog as canonical CSV, optionally status-filtered (§6.4; PUC-9)."""
gate = _merchant_gate(conn, sess)
if isinstance(gate, JSONResponse):
return gate
_account, sf = gate
try:
stream = products.export_catalog(conn, sf.id, status)
except products.EmptyCatalog:
return _error(409, "empty_catalog", "There are no products to export.")
return StreamingResponse(
stream,
media_type="text/csv",
headers={"content-disposition": 'attachment; filename="ecomm-products-export.csv"'},
)
```
Note: `EmptyCatalog` is raised **eagerly** by `export_catalog` (Task 5) before
the generator yields, so the `try/except` around the call catches it and we can
answer `409` with no partial stream. Good.
- [ ] **Step 4: Run to verify it passes**
Run: `.venv/bin/python -m pytest backend/tests/test_products_endpoints.py -k export -q`
Expected: PASS (5 passed)
- [ ] **Step 5: Run the whole backend suite + import linter**
Run: `( cd backend && ../.venv/bin/lint-imports ) && .venv/bin/python -m pytest backend -q`
Expected: PASS — import boundaries clean (serialize.py is in `domains`, imports
only `models`/`diff`), all backend tests green.
- [ ] **Step 6: Commit**
```bash
git add backend/app/main.py backend/tests/test_products_endpoints.py
git commit -m "feat(api): GET /api/products/export — streamed canonical CSV, 409 empty_catalog (§6.4, PUC-9)"
```
---
## Task 8: Frontend API — export URL helper
**Files:**
- Modify: `frontend/src/productsApi.ts`
- Create: `frontend/src/productsApi.export.test.ts`
The export is a **browser download**, not a fetch (we want the browser's native
save dialog and streaming). So the API module contributes a URL builder + the
status list, not a `request()` wrapper.
- [ ] **Step 1: Write the failing test**
```typescript
// frontend/src/productsApi.export.test.ts
import { describe, expect, it } from "vitest";
import { EXPORT_STATUSES, exportUrl } from "./productsApi";
describe("export url", () => {
it("lists the four status filters with 'all' first", () => {
expect(EXPORT_STATUSES.map((s) => s.value)).toEqual(["all", "active", "draft", "archived"]);
});
it("builds the endpoint url with the status query", () => {
expect(exportUrl("all")).toBe("/api/products/export?status=all");
expect(exportUrl("archived")).toBe("/api/products/export?status=archived");
});
});
```
- [ ] **Step 2: Run to verify it fails**
Run: `( cd frontend && npx vitest run src/productsApi.export.test.ts )`
Expected: FAIL — `exportUrl` / `EXPORT_STATUSES` not exported.
- [ ] **Step 3: Add the helper to productsApi.ts**
```typescript
// in frontend/src/productsApi.ts, after dialectLabel()
export type ExportStatus = "all" | "active" | "draft" | "archived";
// PUC-9 status filter, 'all' first (the default). Labels drive the menu.
export const EXPORT_STATUSES: { value: ExportStatus; label: string }[] = [
{ value: "all", label: "All products" },
{ value: "active", label: "Active" },
{ value: "draft", label: "Draft" },
{ value: "archived", label: "Archived" },
];
export function exportUrl(status: ExportStatus): string {
return `/api/products/export?status=${status}`;
}
```
- [ ] **Step 4: Run to verify it passes**
Run: `( cd frontend && npx vitest run src/productsApi.export.test.ts )`
Expected: PASS (2 passed)
- [ ] **Step 5: Commit**
```bash
git add frontend/src/productsApi.ts frontend/src/productsApi.export.test.ts
git commit -m "feat(frontend): export URL helper + status-filter list (PUC-9)"
```
---
## Task 9: Frontend — the Export status-filter menu on the Products page
**Files:**
- Modify: `frontend/src/screens/products/ProductsPage.tsx`
- Modify: `frontend/src/styles/products.css`
- Create: `frontend/src/screens/products/ProductsPage.export.test.tsx`
Replace the disabled Export button (lines ~6671) with a small menu: a primary
"Export" toggle that opens the four status options; choosing one navigates the
browser to `exportUrl(status)` (triggering the download). When the catalog is
empty, keep the **disabled** button + existing note (PUC-9). The menu is a
native `<details>`/`<summary>` disclosure (matches the codebase's no-extra-deps,
keyboard-accessible idiom — §6.6) so it needs no click-outside handler or extra
state.
This is a UI change **within the design system's existing primitives** (buttons,
the `note` class, a `<details>` disclosure already used in ImportPreview) — no
new UI Design required (handbook R4 applies only beyond the design system).
- [ ] **Step 1: Write the failing test**
```tsx
// frontend/src/screens/products/ProductsPage.export.test.tsx
import { render, screen, within } from "@testing-library/react";
import { describe, expect, it, vi, beforeEach } from "vitest";
import ProductsPage from "./ProductsPage";
import * as api from "../../productsApi";
function mockSummary(count: number) {
vi.spyOn(api, "getProductsSummary").mockResolvedValue({
ok: true,
value: { product_count: count, image_problem_count: 0, latest_run_id: null },
});
vi.spyOn(api, "listRuns").mockResolvedValue({ ok: true, value: [] });
}
describe("ProductsPage export menu", () => {
beforeEach(() => vi.restoreAllMocks());
it("offers the four status export links when the catalog is populated", async () => {
mockSummary(3);
render(<ProductsPage />);
const exportToggle = await screen.findByRole("button", { name: "Export" });
expect(exportToggle).toBeEnabled();
// The four status options resolve to the endpoint URLs.
const all = await screen.findByRole("link", { name: "All products" });
expect(all).toHaveAttribute("href", "/api/products/export?status=all");
expect(screen.getByRole("link", { name: "Archived" })).toHaveAttribute(
"href", "/api/products/export?status=archived",
);
});
it("disables export with a note when the catalog is empty", async () => {
mockSummary(0);
render(<ProductsPage />);
expect(await screen.findByRole("button", { name: "Export" })).toBeDisabled();
expect(screen.getByText("Export arrives when you have products")).toBeInTheDocument();
});
});
```
Note: this is the repo's **first React Testing Library test** (SLICE-2's note
said "no RTL in repo"). Verify RTL is available before writing — check
`frontend/package.json` devDependencies for `@testing-library/react` +
`@testing-library/jest-dom` + `jsdom`, and `vitest.config`/`vite.config` for
`environment: "jsdom"` and a jest-dom setup. **If any are missing**, either (a)
add them (`npm i -D @testing-library/react @testing-library/jest-dom jsdom` and
set `test.environment = "jsdom"` + a setup file importing
`@testing-library/jest-dom`), or (b) — to stay consistent with the repo's
"pure-logic tests only" convention — **drop the `.tsx` render test and instead
unit-test a pure helper**: extract an `exportMenuItems()` /
`isExportEnabled(count)` from the component into `productsApi.ts` and test that,
leaving the JSX visual-verified by E2E (Task 11) only. **Prefer (b)** unless RTL
already exists — it matches SLICE-2/3's established pattern and avoids
introducing a test framework as a side effect of this slice. The decision is
the implementer's; log it in the transcript's Deferred decisions.
- [ ] **Step 2: Run to verify it fails (or pivot to 2b per the note)**
Run: `( cd frontend && npx vitest run src/screens/products/ProductsPage.export.test.tsx )`
Expected: FAIL — component still renders the disabled button.
If pivoting to (b), instead create
`frontend/src/screens/products/exportMenu.ts` with
`isExportEnabled(count: number): boolean` + reuse `EXPORT_STATUSES`/`exportUrl`,
and `frontend/src/screens/products/exportMenu.test.ts` testing those pure fns.
- [ ] **Step 3: Implement the menu in ProductsPage.tsx**
Replace the export block (currently):
```tsx
<div className="products__actions">
<div className="products__export">
<button type="button" className="btn-secondary" disabled title="Export arrives in a coming release">
Export
</button>
<span className="note">Export arrives in a coming release</span>
</div>
<a className="btn-primary" href="#/products/import">
Import products
</a>
</div>
```
with:
```tsx
<div className="products__actions">
{empty ? (
<div className="products__export">
<button type="button" className="btn-secondary" disabled>
Export
</button>
<span className="note">Export arrives when you have products</span>
</div>
) : (
<details className="products__export menu">
<summary className="btn-secondary" role="button">
Export
</summary>
<ul className="menu__list">
{EXPORT_STATUSES.map((s) => (
<li key={s.value}>
{/* A real download: the browser navigates to the streamed
endpoint and saves the attachment (PUC-9). */}
<a className="menu__item" href={exportUrl(s.value)} download>
{s.label}
</a>
</li>
))}
</ul>
</details>
)}
<a className="btn-primary" href="#/products/import">
Import products
</a>
</div>
```
Add the imports at the top of the file:
```tsx
import {
dialectLabel,
EXPORT_STATUSES,
exportUrl,
getProductsSummary,
listRuns,
type ProductsSummary,
type RunSummary,
} from "../../productsApi";
```
(`empty` is already computed at line ~57: `const empty = summary.product_count === 0;`.)
- [ ] **Step 4: Add menu styles to products.css**
```css
/* Export status menu (SD-0002 §5.2 — PUC-9). A native <details> disclosure so
it's keyboard-accessible with no extra JS (§6.6). */
.products__export.menu {
position: relative;
display: inline-block;
}
.products__export.menu > summary {
list-style: none;
cursor: pointer;
}
.products__export.menu > summary::-webkit-details-marker {
display: none;
}
.menu__list {
position: absolute;
right: 0;
z-index: 10;
margin: 0.25rem 0 0;
padding: 0.25rem;
list-style: none;
background: var(--surface, #fff);
border: 1px solid var(--border, #d8d3c8);
border-radius: 8px;
box-shadow: 0 6px 20px rgba(0, 0, 0, 0.12);
min-width: 10rem;
}
.menu__item {
display: block;
padding: 0.5rem 0.75rem;
border-radius: 6px;
text-decoration: none;
color: inherit;
}
.menu__item:hover,
.menu__item:focus {
background: var(--surface-hover, #f3efe7);
}
```
(Check `frontend/src/styles/` for the actual CSS custom-property names — the
fallbacks above keep it working even if the vars differ; align them with the
existing tokens if names like `--border` aren't what the design bundle uses.)
- [ ] **Step 5: Run the test + typecheck**
Run: `( cd frontend && npx vitest run && npm run build )`
Expected: PASS — export menu test green, `tsc` clean, Vite build succeeds.
- [ ] **Step 6: Commit**
```bash
git add frontend/src/screens/products/ProductsPage.tsx frontend/src/styles/products.css frontend/src/screens/products/ProductsPage.export.test.tsx
git commit -m "feat(frontend): Export status-filter menu on the Products page (SD-0002 §5.2, PUC-9)"
```
(Adjust the `git add` paths if you pivoted to the pure-helper `exportMenu.ts`
approach in Step 2b.)
---
## Task 10: Full local gate — `check.sh` green
**Files:** none (verification task)
- [ ] **Step 1: Recreate the venv interpreter path expectation**
The repo's `check.sh` picks `$repo_root/.venv/bin/python`. Confirm it exists in
the worktree (init created it). If not: `python3 -m venv .venv && .venv/bin/pip
install -r backend/requirements.txt`.
- [ ] **Step 2: Run the full gate**
Run: `bash scripts/check.sh`
Expected output (tail): `==> all gates green` — import boundaries clean, all
backend pytest green, frontend `tsc`+build clean, vitest green.
- [ ] **Step 3: If anything fails, fix before proceeding**
Use `superpowers:systematic-debugging`. Do not move to E2E with a red unit gate.
- [ ] **Step 4: Commit any fixes**
```bash
git add -A && git commit -m "fix(slice-6): <what you fixed> to green the check.sh gate"
```
---
## Task 11: E2E — `e2e_export_download` and `e2e_roundtrip_noop`
**Files:**
- Create: `e2e/tests/export-download.spec.ts`
- Create: `e2e/tests/roundtrip-noop.spec.ts`
- Modify: `e2e/helpers.ts` (add an export-download helper)
Both scenarios sign up, import `good.csv` (the existing fixture: 2 products,
moon-mug + star-tee), then exercise export. Playwright captures the download via
its `download` event. The round-trip scenario writes the downloaded file to a
temp path and re-uploads it.
- [ ] **Step 1: Add a download helper to helpers.ts**
```typescript
// append to e2e/helpers.ts
import { tmpdir } from "node:os";
import { writeFile } from "node:fs/promises";
// Import good.csv and confirm it, leaving a 2-product catalog. Returns nothing;
// callers continue from the run-detail screen.
export async function importGoodCsv(page: Page) {
await uploadFixture(page, "good.csv");
await expect(page.getByRole("heading", { name: "Import preview — good.csv" })).toBeVisible();
await page.getByRole("button", { name: "Import 2 products" }).click();
await expect(page.getByRole("heading", { level: 1, name: "good.csv" })).toBeVisible();
}
// Click an Export status option and capture the downloaded CSV's text + path.
export async function exportCatalog(page: Page, label: string): Promise<{ text: string; path: string }> {
// Open the <details> menu, then click the status option, capturing the download.
await page.getByRole("button", { name: "Export" }).click();
const [download] = await Promise.all([
page.waitForEvent("download"),
page.getByRole("link", { name: label }).click(),
]);
const stream = await download.createReadStream();
const chunks: Buffer[] = [];
for await (const c of stream) chunks.push(c as Buffer);
const text = Buffer.concat(chunks).toString("utf8");
const path = join(tmpdir(), `export-${Date.now()}.csv`);
await writeFile(path, text, "utf8");
return { text, path };
}
```
Note: the `<summary role="button">` is exposed to Playwright's `getByRole("button",
{ name: "Export" })`. Verify during the run; if the accessible role resolves
differently, target `page.locator(".products__export > summary")` instead.
- [ ] **Step 2: Write `e2e_export_download`**
```typescript
// e2e/tests/export-download.spec.ts
// DoD scenario e2e_export_download (SD-0002 §6.8): export downloads a canonical
// CSV and the status filter is respected.
import { expect, test } from "@playwright/test";
import { exportCatalog, gotoProducts, importGoodCsv, signUpWithStorefront } from "../helpers";
test("e2e_export_download", async ({ page }) => {
await signUpWithStorefront(page);
await gotoProducts(page);
await importGoodCsv(page);
await gotoProducts(page);
// Export "All products" → a canonical CSV with both handles.
const all = await exportCatalog(page, "All products");
expect(all.text.split("\n")[0]).toContain("Handle,");
expect(all.text).toContain("moon-mug");
expect(all.text).toContain("star-tee");
// good.csv's products are active → "Active" exports both, "Draft" is empty/disabled-path.
const active = await exportCatalog(page, "Active");
expect(active.text).toContain("moon-mug");
});
```
- [ ] **Step 3: Write `e2e_roundtrip_noop`**
```typescript
// e2e/tests/roundtrip-noop.spec.ts
// DoD scenario e2e_roundtrip_noop (SD-0002 §6.8, PUC-10): exporting then
// re-importing the unmodified file previews as all-unchanged with the import
// action disabled and the "Nothing to change" note.
import { expect, test } from "@playwright/test";
import { join } from "node:path";
import { exportCatalog, gotoProducts, importGoodCsv, signUpWithStorefront } from "../helpers";
test("e2e_roundtrip_noop", async ({ page }) => {
await signUpWithStorefront(page);
await gotoProducts(page);
await importGoodCsv(page);
await gotoProducts(page);
// Export the catalog, then re-import the unmodified file.
const { path } = await exportCatalog(page, "All products");
await page.getByRole("link", { name: "Import products" }).first().click();
await expect(page.getByRole("heading", { name: "Import products" })).toBeVisible();
await page.locator('input[type="file"]').setInputFiles(path);
// Preview: everything unchanged, nothing to add/update (PUC-10).
await expect(page.getByRole("button", { name: "2 unchanged" })).toBeVisible();
await expect(page.getByRole("button", { name: "0 to add" })).toBeVisible();
await expect(page.getByRole("button", { name: "0 to update" })).toBeVisible();
// The import action is disabled, with the no-op note.
const importBtn = page.getByRole("button", { name: /^Import 0 products$/ });
await expect(importBtn).toBeDisabled();
await expect(
page.getByText("Nothing to change — your catalog already matches this file"),
).toBeVisible();
});
```
Note on the import-preview button label: when `toApply === 0` the button reads
`Import 0 products` and is disabled (ImportPreview.tsx line ~370). Confirm the
exact rendered text during the run and tighten the regex if needed.
- [ ] **Step 4: Run the E2E suite**
Run: `bash scripts/e2e.sh`
Expected: all scenarios PASS — the 4 SLICE-5 scenarios + `e2e_export_download` +
`e2e_roundtrip_noop` (6 total). Requires the dev Postgres up (`docker compose up
-d db`). The harness builds the SPA and serves on :8765.
- [ ] **Step 5: Debug any failures against the real UI**
If `e2e_roundtrip_noop` shows unexpected adds/updates, that's an INV-12
**escape the property test missed** (a field the generator didn't cover but
good.csv exercises — e.g. an image alt text, a specific price format). This is
exactly the bug class the E2E exists to catch. Trace it: download the export,
diff it by eye against expectations, fix `serialize.py`, add a unit test
reproducing it in `test_products_serialize.py`, re-run both gates.
- [ ] **Step 6: Commit**
```bash
git add e2e/tests/export-download.spec.ts e2e/tests/roundtrip-noop.spec.ts e2e/helpers.ts
git commit -m "test(e2e): e2e_export_download + e2e_roundtrip_noop (SD-0002 §6.8, PUC-9/10)"
```
---
## Task 12: Docs — DOC-1 (OPERATIONS.md) + DOC-4 (products-domain.md)
**Files:**
- Modify: `docs/OPERATIONS.md`
- Modify: `docs/products-domain.md`
- [ ] **Step 1: Update OPERATIONS.md (DOC-1)**
In the "Products import/export ops" section, add an export subsection and the
TEL-3 telemetry row. Add after the import telemetry table:
```markdown
### Export (PUC-9, SLICE-6)
`GET /api/products/export?status=all|active|draft|archived` streams the
storefront's catalog as a canonical-format CSV (one codec, two directions — the
same format the importer parses). It is **read-only** (no draft, no run) and
storefront-scoped (INV-14). An empty catalog — no products, or none matching the
status filter — returns `409 empty_catalog`; the Products page disables the
Export action with a note in that case. The round-trip is lossless (INV-12):
re-importing an unmodified export previews as all-unchanged with the import
action disabled (PUC-10).
```
And add the TEL-3 row to the telemetry table:
```markdown
| TEL-3 `catalog_exported` | export stream completes | `storefront_id, status_filter, product_count, duration_ms` |
```
And add `e2e_export_download` + `e2e_roundtrip_noop` to the E2E suite bullet
(now six scenarios):
```markdown
- Lives at `e2e/` — Playwright, Chromium, six scenarios (SLICE-5:
preview/confirm, actionable errors, file rejection, cancel; SLICE-6:
`e2e_export_download`, `e2e_roundtrip_noop`).
```
- [ ] **Step 2: Update products-domain.md (DOC-4)**
Add a serializer/export section documenting the one-codec-two-directions design
and how INV-12 is enforced:
```markdown
## Export & the round-trip (SLICE-6)
`serialize.py` is the export half of "one codec, two directions": it turns the
`CatalogProduct` snapshot (the same one `repo.load_catalog` builds for the diff
engine) back into canonical CSV, writing exactly the columns `codec.py` /
`validate.py` parse, in the §6.5.1 row grammar (`HEADER` is the full canonical
column set; product fields + option names on the first row; one variant per
row; images interleaved; image-only rows when a product has more images than
variants).
`repo.export_catalog` returns the status-filtered snapshot list; the
`service.export_catalog` generator streams `serialize.catalog_to_csv` over it
and emits TEL-3 (`catalog_exported`) once the stream is exhausted. The BFF wraps
it in a `StreamingResponse`; an empty (filtered) catalog raises `EmptyCatalog`
→ `409 empty_catalog` before any bytes stream.
**INV-12** (`diff(catalog, import(export(catalog))) = ∅`) is locked two ways:
a property test (`test_products_serialize.py`) over 200 generated text-field
catalogs runs the real export→parse→diff loop and asserts every product is
`unchanged`; the `e2e_roundtrip_noop` browser scenario does the same through the
UI (export download → re-upload → all-unchanged preview, import disabled).
Numeric/Decimal fields — the property test's blind spot (string-form vs
value-identity) — get explicit round-trip unit tests.
```
- [ ] **Step 3: Verify the docs render (no broken structure)**
Run: `git diff --stat docs/`
Expected: both files modified; eyeball the additions.
- [ ] **Step 4: Commit**
```bash
git add docs/OPERATIONS.md docs/products-domain.md
git commit -m "docs(products): DOC-1 export ops + TEL-3, DOC-4 serializer/round-trip (SLICE-6)"
```
---
## Task 13: Version bump → 0.6.0
**Files:**
- Modify: `VERSION`
- Modify: `frontend/package.json`
The deploy pin checks out the `v<VERSION>` tag and `/healthz` reports it back
(flotilla's verify gate compares them), so VERSION must match the release tag.
- [ ] **Step 1: Bump VERSION**
Set `VERSION` file contents to:
```
0.6.0
```
- [ ] **Step 2: Bump frontend/package.json**
Change `"version": "0.5.0"` to `"version": "0.6.0"`.
- [ ] **Step 3: Verify the backend reads it**
Run: `.venv/bin/python -c "from app.main import _APP_VERSION; print(_APP_VERSION)" 2>/dev/null || ( cd backend && ../.venv/bin/python -c "import app.main as m; print(m._APP_VERSION)" )`
Expected: `0.6.0`
- [ ] **Step 4: Commit**
```bash
git add VERSION frontend/package.json
git commit -m "chore: version 0.6.0 — SLICE-6 export & round-trip"
```
---
## Task 14: Final verification + ship through the §9 pipeline
**Files:** none (verification + deploy)
- [ ] **Step 1: Both gates green from a clean state**
Run: `bash scripts/check.sh && bash scripts/e2e.sh`
Expected: `==> all gates green` then all 6 E2E scenarios pass. **Evidence
before assertion** — do not claim done until you see both.
- [ ] **Step 2: Push the branch + open the PR**
```bash
git push -u origin worktree-slice-6-export-roundtrip
```
Open a PR to `main` citing SD-0002 §7.2 SLICE-6, anchor #13, and the DoD
checklist (BUC-3/4, INV-12 green, both E2E green, TEL-3 emitting, DOC-1 updated,
v0.6.0). Merge it (autonomous posture; the reviewed artifact was the spec).
- [ ] **Step 3: Tag the release**
After merge, on `main`:
```bash
git tag -a v0.6.0 -m "SLICE-6 — export & round-trip lock"
git tag -a "release/$(TZ=America/Los_Angeles date +%Y-%m-%dT%H-%M)" -m "PPE deploy — SLICE-6"
git push origin v0.6.0 && git push origin --tags
```
- [ ] **Step 4: Deploy to PPE via flotilla**
Per `DEPLOY-FLOTILLA.md` + the deployment's operator guide. The deploy checks
out the `v0.6.0` tag. Use `CLOUDSDK_ACTIVE_CONFIG_NAME=wiggleverse-ecomm` for
any gcloud-shelling command:
```bash
flotilla-core deploy ecomm
```
Expected: 9/9 phases green; `/healthz` on `https://ecomm-ppe.wiggleverse.org`
reports `version: 0.6.0`.
- [ ] **Step 5: PPE smoke**
Curl the health endpoint and confirm the version; if practical, walk the export
download + round-trip no-op once against PPE (the §9 PPE browser run is still
manual this slice). TEL-3 (`catalog_exported`) should appear in the VM journal
after an export.
- [ ] **Step 6: Done**
Ticket #13 SLICE-6 is **done** at merge + PPE-green (prod promotion stays the
operator's gate). Finalize archives this plan to the content repo `plans/`.
---
## Self-Review (run before handing off)
**Spec coverage** (SD-0002 §7.2 SLICE-6 DoD):
- ✅ Canonical serializer (one codec, two directions) — Tasks 12, `serialize.py`.
- ✅ Streamed export endpoint + status filter — Tasks 45 (service/repo), 7 (BFF).
- ✅ Status-filter UI — Tasks 89 (Products page menu).
- ✅ `nothing_to_apply` no-op preview behavior — shipped SLICE-5; **reached** by
this slice's export and asserted by `e2e_roundtrip_noop` (Task 11).
- ✅ INV-12 property test over text-field catalogs — Task 3; numeric coverage Task 6.
- ✅ BUC-3 (catalog-wide change) — the import spine (SLICE-5) + this export round-trip.
- ✅ BUC-4 (extraction always available, complete) — Task 7 endpoint + Task 9 UI.
- ✅ E2E `e2e_export_download`, `e2e_roundtrip_noop` — Task 11.
- ✅ TEL-3 `catalog_exported` emitting — Task 5; asserted Task 5 + DOC-1 Task 12.
- ✅ DOC-1 updated — Task 12 (also DOC-4).
- ✅ v0.6.0 shipped through merge + PPE — Tasks 1314.
**Placeholder scan:** every code step carries complete code; no TBD/TODO; the
two adaptive points (Task 9 RTL-vs-pure-helper, Task 3/11 debug-if-fails) give
concrete decision criteria and fallbacks, not hand-waves.
**Type consistency:** `catalog_to_csv` (serialize), `export_catalog` (repo: list;
service: Iterator) used consistently; `EXPORT_STATUSES`/`exportUrl`/`ExportStatus`
consistent frontend⇄tests; `EmptyCatalog`/`empty_catalog` (exception/code) paired
consistently across errors.py → service → __init__ → main.py → tests.
```