wiggleverse/wiggleverse-ecomm-content
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
docs-redirect-stubs
wiggleverse-ecomm-content/plans/2026-06-11-slice-6-export-roundtrip.md
T

66 KiB
Raw Permalink Blame History

SLICE-6 — Export & the Round-Trip Lock Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Ship the catalog export endpoint and lock the round-trip property — exporting a catalog and re-importing the unmodified file is a visible no-op (INV-12) — completing PUC-9 and PUC-10 of SD-0002.

Architecture: A new serialize.py in the products domain turns the catalog (the same CatalogProduct snapshot the diff engine already reads) back into canonical CSV — one codec, two directions. The BFF streams it at GET /api/products/export?status=…. The frontend's already-built Export button (currently disabled) becomes a status-filter menu that downloads. INV-12 holds because export emits exactly the columns the importer parses and the diff engine, re-reading that file against the live catalog, classifies every product as unchanged. PUC-10's no-op preview UX (the nothing_to_apply path, the "Nothing to change" note) already shipped in SLICE-5 — this slice adds the export half that makes it reachable, plus a regression that proves the loop.

Tech Stack: Python 3 / FastAPI / psycopg 3 / PostgreSQL 16 (backend); React + Vite + TypeScript / Vitest (frontend); Playwright (E2E). Gates: scripts/check.sh (lint-imports + pytest + frontend build + vitest) and bash scripts/e2e.sh (Playwright).

Context the engineer needs before starting

Read these first — the slice extends, never rewrites, the SLICE-5 import spine:

  • wiggleverse-ecomm-content/specs/SD-0002-products-bulk-csv-import-export.md — §5.2 (Products screen / Export menu), §6.4 (the GET /api/products/export contract row), §6.5.1 (the canonical CSV format — the serializer's target), §6.5.5 (Export), §6.8 (test strategy + the two named E2E scenarios), §9.1 TEL-3, §7.2 SLICE-6 DoD. The spec is the source of truth; if the code proves it wrong, amend the spec then replan.
  • backend/app/domains/products/models.py — PRODUCT_COLUMNS, VARIANT_COLUMNS, OPTION_VALUE_COLUMNS, IMAGE_COLUMNS, CLEAR_DEFAULTS. These dicts map column name → field name; the serializer inverts them.
  • backend/app/domains/products/diff.py — the CatalogProduct / CatalogVariant / CatalogImage snapshot dataclasses and compute_diff(). The serializer reads the same snapshot repo.load_catalog() builds; the round-trip test calls compute_diff().
  • backend/app/domains/products/codec.py + validate.py — the parser the export must round-trip through. Note the parse path: bytes → parse_csv → build_products → compute_diff.
  • backend/app/domains/products/repo.py:load_catalog — already returns the catalog keyed by handle in snapshot shape; the export reuses it.

Key invariants this slice must honor:

  • INV-12 (the headline): diff(catalog, import(export(catalog))) = ∅ over text-field catalogs. This slice's whole reason for being.
  • INV-14: every query storefront-scoped — load_catalog(conn, storefront_id) already is; the export endpoint goes through _merchant_gate.
  • INV-10/11: export is read-only — it must never write. (No draft, no run; just streams.)

Round-trip subtleties the serializer must get exactly right (each has a task):

  1. Variant rows vs image rows. Shopify's grammar (§6.5.1): consecutive rows share a Handle; the first row carries product-level fields + the first variant + (optionally) the first image; later rows carry additional variants and/or additional images. A product with more images than variants emits image-only rows (just Handle + Image *). The serializer must interleave variants and images into rows the parser regroups identically.
  2. The no-option single variant. A product with no options has one variant whose option values are all NULL. On re-import, validate._build_block treats the first row as carrying that single variant (index == 0 and not has_options). The serializer must not emit Option1 Value etc. for it.
  3. Blank vs absent (§6.5.1). A column absent from the file is untouched; a cell present but empty clears to default. Export emits every canonical column (full header), so on re-import every field is "present". For an unchanged round-trip, an exported empty cell must resolve (via CLEAR_DEFAULTS / parse) to exactly the catalog's current value. This is why the round-trip test uses text-field catalogs (§7.2) — fields whose serialize→parse is identity. Numeric/decimal/bool fields are covered by explicit unit tests, not the property test's generator (see Task 6 note).
  4. Tags. Stored as TEXT[]; the cell is comma-joined; parse splits on comma and strips. Round-trips iff tags contain no commas and no leading/trailing whitespace — the generator respects this.
  5. Published / Status. Booleans serialize as TRUE/FALSE; status as the lowercase word. Both have CLEAR_DEFAULTS entries (published=True, status="active"), so the catalog default round-trips even when we emit the explicit value (we always emit the explicit value — see Task 2).

File Structure

Backend (create):

  • backend/app/domains/products/serialize.py — the canonical serializer: catalog_to_csv(products: list[CatalogProduct]) -> Iterator[str], streaming one logical chunk per product (header first). One responsibility: snapshot → canonical CSV text. DB-free, mirrors diff.py's placement.
  • backend/tests/test_products_serialize.py — serializer unit tests + the INV-12 property test (diff(catalog, import(export(catalog))) = ∅).

Backend (modify):

  • backend/app/domains/products/repo.py — add export_catalog(conn, storefront_id, status_filter) returning the (status-filtered) snapshot list.
  • backend/app/domains/products/service.py — add export_catalog(conn, storefront_id, status_filter) -> Iterator[str]; emits TEL-3.
  • backend/app/domains/products/__init__.py — export the new service function
    • EmptyCatalog error.
  • backend/app/domains/products/errors.py — add EmptyCatalog.
  • backend/app/main.py — add GET /api/products/export (streamed, _merchant_gate, 409 empty_catalog).

Frontend (modify):

  • frontend/src/productsApi.ts — add EXPORT_STATUSES + exportUrl(status) helper (the endpoint is a plain download link, not a fetch wrapper).
  • frontend/src/screens/products/ProductsPage.tsx — replace the disabled Export button with a real status-filter menu that downloads; disable (with the existing note) only when the catalog is empty.
  • frontend/src/screens/products/ProductsPage.export.test.tsx — vitest for the export menu (enabled/disabled, status → URL).
  • frontend/src/styles/products.css — minimal styles for the export menu.

E2E (create):

  • e2e/tests/export-download.spec.ts — e2e_export_download (status filter respected).
  • e2e/tests/roundtrip-noop.spec.ts — e2e_roundtrip_noop (export → re-import → all-unchanged, import action disabled).

Docs / version (modify):

  • docs/OPERATIONS.md — DOC-1: add export ops + TEL-3 row + the export E2E scenarios.
  • docs/products-domain.md — DOC-4: the serializer (one codec, two directions) + INV-12 mechanics.
  • VERSION + frontend/package.json — 0.5.0 → 0.6.0.

Task 1: Serializer skeleton — header + a single trivial product

Files:

  • Create: backend/app/domains/products/serialize.py
  • Create: backend/tests/test_products_serialize.py

The canonical CSV column order is fixed and must be a superset covering every column the importer knows. We derive it once from the model registry so it can never drift from what the parser accepts.

  • Step 1: Write the failing test
# backend/tests/test_products_serialize.py
"""Canonical serializer + INV-12 round-trip lock (SD-0002 §6.5.5, §6.8)."""
import csv
import io

from app.domains.products import serialize
from app.domains.products.diff import CatalogImage, CatalogProduct, CatalogVariant


def _product(**kw) -> CatalogProduct:
    """A minimal no-option catalog product (one all-NULL variant)."""
    base = dict(
        id=1, handle="moon-mug", title="Moon Mug",
        option_names=(None, None, None),
        fields={
            "title": "Moon Mug", "description_html": None, "vendor": "Acme",
            "product_type": "standalone", "google_product_category": None,
            "tags": [], "status": "active", "published": True,
        },
        variants=[CatalogVariant(
            id=1, options=(None, None, None), position=1,
            fields={"sku": "WG-MUG", "barcode": None, "price": None, "cost": None,
                    "weight": None, "weight_unit": None, "volume": None,
                    "volume_unit": None, "tax_id_1": None, "tax_id_2": None,
                    "inventory_tracker": None, "inventory_qty": None,
                    "variant_image": None}],
        )],
        images=[],
    )
    base.update(kw)
    return CatalogProduct(**base)


def _rows(products) -> list[dict]:
    text = "".join(serialize.catalog_to_csv(products))
    return list(csv.DictReader(io.StringIO(text)))


def test_header_is_full_canonical_set():
    text = "".join(serialize.catalog_to_csv([_product()]))
    header = next(csv.reader(io.StringIO(text)))
    # Handle + Title first; every known column present exactly once.
    assert header[0] == "Handle"
    assert "Title" in header and "Variant SKU" in header and "Image Src" in header
    assert len(header) == len(set(header))


def test_single_no_option_product_one_row():
    rows = _rows([_product()])
    assert len(rows) == 1
    assert rows[0]["Handle"] == "moon-mug"
    assert rows[0]["Title"] == "Moon Mug"
    assert rows[0]["Variant SKU"] == "WG-MUG"
    # A no-option product emits no option values.
    assert rows[0]["Option1 Value"] == ""
  • Step 2: Run the test to verify it fails

Run: .venv/bin/python -m pytest backend/tests/test_products_serialize.py -q Expected: FAIL — ModuleNotFoundError: No module named 'app.domains.products.serialize'

  • Step 3: Write the minimal implementation
# backend/app/domains/products/serialize.py
"""Canonical serializer — CatalogProduct snapshot → canonical CSV (SD-0002 §6.5.5).

The export half of "one codec, two directions": this writes exactly the columns
codec.py/validate.py parse, in a row grammar (§6.5.1) the validator regroups
identically — so re-importing an unmodified export diffs to nothing (INV-12).
DB-free, like diff.py: it consumes the same CatalogProduct snapshot the diff
engine reads (repo.load_catalog), and the service streams the result.
"""
from __future__ import annotations

import csv
import io
from collections.abc import Iterable, Iterator
from decimal import Decimal

from .diff import CatalogProduct
from .models import (
    IMAGE_COLUMNS,
    OPTION_VALUE_COLUMNS,
    PRODUCT_COLUMNS,
    VARIANT_COLUMNS,
)

# Canonical column order: Handle, then product, option-value, variant, image
# columns. A superset of everything the parser knows (models.KNOWN_COLUMNS minus
# the #15-reserved Component columns, which export never emits).
HEADER: list[str] = [
    "Handle",
    *PRODUCT_COLUMNS,
    *OPTION_VALUE_COLUMNS,
    *VARIANT_COLUMNS,
    *IMAGE_COLUMNS,
]

# field name -> column name, inverting the model registry (the parser reads
# column->field; the serializer writes field->column).
_PRODUCT_FIELD_TO_COL = {field: col for col, field in PRODUCT_COLUMNS.items()}
_VARIANT_FIELD_TO_COL = {field: col for col, field in VARIANT_COLUMNS.items()}


def _cell(value: object) -> str:
    """Serialize one value to its canonical cell text (the parse inverse)."""
    if value is None:
        return ""
    if isinstance(value, bool):
        return "TRUE" if value else "FALSE"
    if isinstance(value, Decimal):
        return str(value)
    if isinstance(value, (list, tuple)):
        return ", ".join(str(v) for v in value)
    return str(value)


def catalog_to_csv(products: Iterable[CatalogProduct]) -> Iterator[str]:
    """Stream canonical CSV text, header first, one product block at a time."""
    buf = io.StringIO()
    writer = csv.writer(buf)
    writer.writerow(HEADER)
    yield _drain(buf)
    for product in products:
        for row in _product_rows(product):
            writer.writerow([row.get(col, "") for col in HEADER])
        yield _drain(buf)


def _drain(buf: io.StringIO) -> str:
    text = buf.getvalue()
    buf.seek(0)
    buf.truncate(0)
    return text


def _product_rows(product: CatalogProduct) -> list[dict[str, str]]:
    """The product's CSV rows (§6.5.1 grammar). Task 1: first variant only."""
    base = {"Handle": product.handle, "Title": product.title}
    for field, col in _PRODUCT_FIELD_TO_COL.items():
        if field in product.fields:
            base[col] = _cell(product.fields[field])
    for slot, name in enumerate(product.option_names, start=1):
        if name:
            base[f"Option{slot} Name"] = name
    variant = product.variants[0]
    row = dict(base)
    _write_variant(row, product, variant)
    return [row]


def _write_variant(row: dict[str, str], product: CatalogProduct, variant) -> None:
    """Fill a row's option-value + variant columns for one variant."""
    for slot, value in enumerate(variant.options, start=1):
        # Only emit an option value when the product actually has that option
        # (a no-option product's single variant carries all-NULL options).
        if product.option_names[slot - 1] and value is not None:
            row[f"Option{slot} Value"] = value
    for field, col in _VARIANT_FIELD_TO_COL.items():
        if field in variant.fields:
            row[col] = _cell(variant.fields[field])
  • Step 4: Run the test to verify it passes

Run: .venv/bin/python -m pytest backend/tests/test_products_serialize.py -q Expected: PASS (2 passed)

  • Step 5: Commit
git add backend/app/domains/products/serialize.py backend/tests/test_products_serialize.py
git commit -m "feat(products): canonical serializer skeleton — header + single product row (SD-0002 §6.5.5)"

Task 2: Multi-variant + image interleaving (the row grammar)

Files:

  • Modify: backend/app/domains/products/serialize.py (_product_rows)
  • Modify: backend/tests/test_products_serialize.py

A product emits as many rows as max(variants, images): row i carries variant i (if any) and image i (if any). Variant 0 shares the product's first row. This mirrors validate._build_block, which reads variants and images off whatever rows carry them.

  • Step 1: Write the failing test
# append to backend/tests/test_products_serialize.py
from app.domains.products.diff import CatalogImage, CatalogVariant


def _star_tee() -> CatalogProduct:
    """A 3-variant, 2-image product (the sample.csv shape)."""
    return CatalogProduct(
        id=2, handle="star-tee", title="Star Tee",
        option_names=("Size", "Color", None),
        fields={"title": "Star Tee", "description_html": None, "vendor": "Acme",
                "product_type": "standalone", "google_product_category": None,
                "tags": ["apparel", "tees"], "status": "active", "published": True},
        variants=[
            CatalogVariant(id=10, options=("S", "Indigo", None), position=1,
                           fields={"sku": "WG-TEE-S", "variant_image": None}),
            CatalogVariant(id=11, options=("M", "Indigo", None), position=2,
                           fields={"sku": "WG-TEE-M", "variant_image": None}),
            CatalogVariant(id=12, options=("L", "Indigo", None), position=3,
                           fields={"sku": "WG-TEE-L", "variant_image": None}),
        ],
        images=[
            CatalogImage(id=1, source_url="https://x/a.jpg", position=1, alt_text="front"),
            CatalogImage(id=2, source_url="https://x/b.jpg", position=2, alt_text="back"),
        ],
    )


def test_multivariant_with_images_interleaves():
    rows = _rows([_star_tee()])
    assert len(rows) == 3  # max(3 variants, 2 images)
    # Product-level fields only on the first row.
    assert rows[0]["Title"] == "Star Tee" and rows[1]["Title"] == ""
    assert rows[0]["Tags"] == "apparel, tees" and rows[1]["Tags"] == ""
    # Option names on row 0; option values on every variant row.
    assert rows[0]["Option1 Name"] == "Size" and rows[1]["Option1 Name"] == ""
    assert [r["Option1 Value"] for r in rows] == ["S", "M", "L"]
    assert [r["Variant SKU"] for r in rows] == ["WG-TEE-S", "WG-TEE-M", "WG-TEE-L"]
    # Two images on the first two rows; third row has no image.
    assert [r["Image Src"] for r in rows] == ["https://x/a.jpg", "https://x/b.jpg", ""]
    assert [r["Image Position"] for r in rows] == ["1", "2", ""]


def test_more_images_than_variants_emits_image_only_rows():
    p = _product(images=[
        CatalogImage(id=1, source_url="https://x/a.jpg", position=1, alt_text=None),
        CatalogImage(id=2, source_url="https://x/b.jpg", position=2, alt_text=None),
        CatalogImage(id=3, source_url="https://x/c.jpg", position=3, alt_text=None),
    ])
    rows = _rows([p])
    assert len(rows) == 3  # 1 variant, 3 images
    assert rows[0]["Variant SKU"] == "WG-MUG"
    assert rows[1]["Variant SKU"] == "" and rows[1]["Handle"] == "moon-mug"
    assert [r["Image Src"] for r in rows] == ["https://x/a.jpg", "https://x/b.jpg", "https://x/c.jpg"]
  • Step 2: Run to verify it fails

Run: .venv/bin/python -m pytest backend/tests/test_products_serialize.py -q Expected: FAIL — test_multivariant_with_images_interleaves (only 1 row produced).

  • Step 3: Replace _product_rows with the full grammar
def _product_rows(product: CatalogProduct) -> list[dict[str, str]]:
    """The product's CSV rows (§6.5.1 grammar): product fields + option names on
    the first row; one variant per row; images interleaved; image-only rows when
    a product has more images than variants."""
    base = {"Handle": product.handle, "Title": product.title}
    for field, col in _PRODUCT_FIELD_TO_COL.items():
        if field in product.fields:
            base[col] = _cell(product.fields[field])
    for slot, name in enumerate(product.option_names, start=1):
        if name:
            base[f"Option{slot} Name"] = name

    count = max(len(product.variants), len(product.images))
    rows: list[dict[str, str]] = []
    for i in range(count):
        # Row 0 carries the product-level fields; later rows carry only Handle.
        row = dict(base) if i == 0 else {"Handle": product.handle}
        if i < len(product.variants):
            _write_variant(row, product, product.variants[i])
        if i < len(product.images):
            _write_image(row, product.images[i])
        rows.append(row)
    return rows


def _write_image(row: dict[str, str], image) -> None:
    row["Image Src"] = image.source_url
    row["Image Position"] = str(image.position)
    if image.alt_text is not None:
        row["Image Alt Text"] = image.alt_text

Note: the _product_rows from Task 1 is fully replaced; _write_variant stays as written in Task 1.

  • Step 4: Run to verify it passes

Run: .venv/bin/python -m pytest backend/tests/test_products_serialize.py -q Expected: PASS (4 passed)

  • Step 5: Commit
git add backend/app/domains/products/serialize.py backend/tests/test_products_serialize.py
git commit -m "feat(products): serialize multi-variant + interleaved images (§6.5.1 row grammar)"

Task 3: INV-12 property test — diff(catalog, import(export(catalog))) = ∅

Files:

  • Modify: backend/tests/test_products_serialize.py

The headline lock. Generate arbitrary text-field catalogs, serialize them, parse the result back through the real importer pipeline, diff against the generating catalog, and assert every product is unchanged (zero adds, zero updates, zero errors). Text-field per §7.2: the generator varies handles, titles, vendors, option names/values, SKUs, tags, statuses, published, images — all fields whose serialize→parse is identity. (Decimal/qty fields get explicit unit tests in Task 6, not the generator, because e.g. "18.0" vs "18.00" parse equal but aren't string-identical — a property-test trap, not a real bug.)

  • Step 1: Write the failing property test
# append to backend/tests/test_products_serialize.py
import random

from app.domains.products import codec, diff, validate


def _gen_catalog(seed: int) -> dict:
    """A deterministic random text-field catalog, in load_catalog()'s shape."""
    rng = random.Random(seed)
    words = ["moon", "star", "river", "cloud", "ember", "fern", "slate", "wave"]
    catalog: dict[str, CatalogProduct] = {}
    n_products = rng.randint(1, 6)
    for pid in range(1, n_products + 1):
        handle = "-".join(rng.sample(words, rng.randint(1, 3))) + f"-{pid}"
        if handle in catalog:
            continue
        has_opts = rng.random() < 0.6
        option_names = ("Size", "Color", None) if has_opts else (None, None, None)
        tags = rng.sample(["apparel", "kitchen", "sale", "new"], rng.randint(0, 3))
        fields = {
            "title": f"{handle.title()} Thing",
            "description_html": rng.choice([None, "<p>hi</p>"]),
            "vendor": rng.choice([None, "Acme", "Wiggle Goods"]),
            "product_type": "standalone",
            "google_product_category": rng.choice([None, "Home & Garden"]),
            "tags": tags,
            "status": rng.choice(["draft", "active", "archived"]),
            "published": rng.choice([True, False]),
        }
        variants = []
        if has_opts:
            sizes = rng.sample(["S", "M", "L", "XL"], rng.randint(1, 4))
            for vi, size in enumerate(sizes, start=1):
                variants.append(CatalogVariant(
                    id=pid * 100 + vi, options=(size, "Indigo", None), position=vi,
                    fields={"sku": f"SKU-{pid}-{vi}", "variant_image": None}))
        else:
            variants.append(CatalogVariant(
                id=pid * 100 + 1, options=(None, None, None), position=1,
                fields={"sku": f"SKU-{pid}", "variant_image": None}))
        images = []
        for ii in range(rng.randint(0, 3)):
            images.append(CatalogImage(
                id=pid * 10 + ii, source_url=f"https://img/{handle}-{ii}.jpg",
                position=ii + 1, alt_text=rng.choice([None, f"alt {ii}"])))
        catalog[handle] = CatalogProduct(
            id=pid, handle=handle, title=fields["title"], option_names=option_names,
            fields=fields, variants=variants, images=images)
    return catalog


def _roundtrip_diff(catalog: dict) -> diff.DiffResult:
    """export → bytes → import pipeline → diff against the same catalog."""
    text = "".join(serialize.catalog_to_csv(catalog.values()))
    parsed = codec.parse_csv(text.encode("utf-8"))
    products = validate.build_products(parsed)
    return diff.compute_diff(catalog, products)


def test_inv12_roundtrip_is_noop_over_generated_catalogs():
    for seed in range(200):
        catalog = _gen_catalog(seed)
        result = _roundtrip_diff(catalog)
        assert result.summary["adds"] == 0, f"seed {seed}: {result.summary}"
        assert result.summary["updates"] == 0, f"seed {seed}: {result.summary}"
        assert result.summary["errors"] == 0, f"seed {seed}: {result.summary}"
        assert result.summary["unchanged"] == len(catalog), f"seed {seed}"
  • Step 2: Run to see what breaks

Run: .venv/bin/python -m pytest backend/tests/test_products_serialize.py::test_inv12_roundtrip_is_noop_over_generated_catalogs -q Expected: This is the integration moment. It may already PASS (Tasks 1–2 built the grammar correctly), or FAIL on a specific seed exposing a serialize/parse asymmetry. If it fails, that is a real INV-12 bug — debug with superpowers:systematic-debugging: print the offending product's result.records entry to see which field/variant/image the diff flagged, fix serialize.py (or, if the spec is genuinely wrong, amend the spec then replan), and re-run. Likely culprits to check in order:

  1. Option values for no-option products — the all-NULL variant must emit no Option*Value (Task 1 _write_variant guards this).
  2. description_html — nh3.clean("<p>hi</p>") must equal the stored value. The generator uses already-sanitized HTML so this holds; if it flags, the catalog's stored HTML wasn't canonical (not an export bug).
  3. Image position — exported positions must match stored; the generator uses sequential 1..n matching _write_image.
  4. Tags ordering/whitespace — ", ".join then split-and-strip must reproduce the list; the generator avoids commas-in-tags.
  • Step 3: Fix any asymmetry found (no fix needed if it already passes)

Apply the minimal serialize.py change the failure points to. Re-run until green.

  • Step 4: Run the whole serializer file

Run: .venv/bin/python -m pytest backend/tests/test_products_serialize.py -q Expected: PASS (all serializer tests, incl. INV-12 over 200 catalogs)

  • Step 5: Commit
git add backend/tests/test_products_serialize.py backend/app/domains/products/serialize.py
git commit -m "test(products): INV-12 property test — export round-trips to a no-op (SD-0002 §6.8)"

Task 4: Repo — status-filtered catalog export query

Files:

  • Modify: backend/app/domains/products/repo.py
  • Modify: backend/tests/test_products_serialize.py (add a DB-backed test) — or a new backend/tests/test_products_export.py (preferred; keep serializer unit tests DB-free).
  • Create: backend/tests/test_products_export.py

load_catalog already builds the full snapshot but takes no status filter. Rather than thread a filter through it (variants/images join off products), add a thin export_catalog that reuses load_catalog and filters products by status in Python — the catalog fits in memory (≤5k rows, INV-18) and this keeps one snapshot-builder. Status all returns everything.

  • Step 1: Write the failing test
# backend/tests/test_products_export.py
"""Export: status-filtered catalog snapshot + the streamed service (PUC-9, TEL-3)."""
import csv
import io
import json
import logging

import psycopg
import pytest

from app.domains import products
from app.domains.products import repo, serialize
from app.platform import db

CSV = (
    b"Handle,Title,Vendor,Status,Variant Price\n"
    b"active-mug,Active Mug,Acme,active,18.00\n"
    b"draft-tee,Draft Tee,Acme,draft,24.00\n"
)


@pytest.fixture()
def migrated_conn(fresh_db_url):
    with psycopg.connect(fresh_db_url) as conn:
        db.migrate(conn)
        yield conn


@pytest.fixture()
def merchant(migrated_conn):
    acct = migrated_conn.execute(
        "INSERT INTO account (email) VALUES ('m@example.com') RETURNING id").fetchone()[0]
    sf = migrated_conn.execute(
        "INSERT INTO storefront (name) VALUES ('Shop') RETURNING id").fetchone()[0]
    migrated_conn.execute(
        "INSERT INTO storefront_membership (account_id, storefront_id) VALUES (%s,%s)", (acct, sf))
    migrated_conn.commit()
    return {"account_id": acct, "storefront_id": sf}


def _seed(conn, merchant):
    draft = products.import_validate(conn, merchant["storefront_id"], merchant["account_id"], "c.csv", CSV)
    products.confirm_draft(conn, merchant["storefront_id"], merchant["account_id"], draft["id"])


def test_export_all_returns_both(migrated_conn, merchant):
    _seed(migrated_conn, merchant)
    snap = repo.export_catalog(migrated_conn, merchant["storefront_id"], "all")
    assert {p.handle for p in snap} == {"active-mug", "draft-tee"}


def test_export_status_filter(migrated_conn, merchant):
    _seed(migrated_conn, merchant)
    active = repo.export_catalog(migrated_conn, merchant["storefront_id"], "active")
    assert [p.handle for p in active] == ["active-mug"]
    draft = repo.export_catalog(migrated_conn, merchant["storefront_id"], "draft")
    assert [p.handle for p in draft] == ["draft-tee"]
  • Step 2: Run to verify it fails

Run: .venv/bin/python -m pytest backend/tests/test_products_export.py -q Expected: FAIL — AttributeError: module ... has no attribute 'export_catalog'

  • Step 3: Add export_catalog to repo.py
# in backend/app/domains/products/repo.py, after load_catalog()

_EXPORT_STATUSES = ("all", "active", "draft", "archived")


def export_catalog(
    conn: psycopg.Connection, storefront_id: int, status_filter: str
) -> list[CatalogProduct]:
    """The storefront's catalog as an ordered snapshot list, optionally filtered
    by product status (PUC-9). Reuses load_catalog's snapshot builder; the
    catalog fits in memory (≤5k rows, INV-18). Ordered by handle for a stable,
    deterministic export."""
    catalog = load_catalog(conn, storefront_id)
    products = sorted(catalog.values(), key=lambda p: p.handle)
    if status_filter and status_filter != "all":
        products = [p for p in products if p.fields.get("status") == status_filter]
    return products
  • Step 4: Run to verify it passes

Run: .venv/bin/python -m pytest backend/tests/test_products_export.py -q Expected: PASS (2 passed)

  • Step 5: Commit
git add backend/app/domains/products/repo.py backend/tests/test_products_export.py
git commit -m "feat(products): status-filtered export_catalog snapshot query (PUC-9)"

Task 5: Service — streamed export + TEL-3, and the EmptyCatalog error

Files:

  • Modify: backend/app/domains/products/errors.py
  • Modify: backend/app/domains/products/service.py
  • Modify: backend/app/domains/products/__init__.py
  • Modify: backend/tests/test_products_export.py

The service streams the serialized CSV and emits TEL-3 once, after the stream completes, with product_count and duration_ms. An empty (post-filter) catalog raises EmptyCatalog → the BFF maps it to 409 empty_catalog (PUC-9: Export disabled with a note when empty; the endpoint defends the same rule).

Streaming + telemetry-after-completion means the service returns a generator that emits TEL-3 as its final act. The count is known up front (the snapshot list), so we capture it before yielding.

  • Step 1: Write the failing test
# append to backend/tests/test_products_export.py

@pytest.fixture()
def telemetry_propagation():
    """ecomm.telemetry has propagate=False in app logging; re-enable for caplog."""
    lg = logging.getLogger("ecomm.telemetry")
    old = lg.propagate
    lg.propagate = True
    try:
        yield
    finally:
        lg.propagate = old


def test_export_streams_canonical_csv(migrated_conn, merchant):
    _seed(migrated_conn, merchant)
    text = "".join(products.export_catalog(migrated_conn, merchant["storefront_id"], "all"))
    rows = list(csv.DictReader(io.StringIO(text)))
    assert {r["Handle"] for r in rows} == {"active-mug", "draft-tee"}
    assert rows[0]["Handle"] == "active-mug"  # sorted by handle


def test_export_empty_raises(migrated_conn, merchant):
    # No catalog at all → empty.
    with pytest.raises(products.EmptyCatalog):
        list(products.export_catalog(migrated_conn, merchant["storefront_id"], "all"))


def test_export_empty_after_filter_raises(migrated_conn, merchant):
    _seed(migrated_conn, merchant)  # only active + draft exist
    with pytest.raises(products.EmptyCatalog):
        list(products.export_catalog(migrated_conn, merchant["storefront_id"], "archived"))


def test_tel3_emitted(migrated_conn, merchant, caplog, telemetry_propagation):
    _seed(migrated_conn, merchant)
    with caplog.at_level(logging.INFO, logger="ecomm.telemetry"):
        list(products.export_catalog(migrated_conn, merchant["storefront_id"], "all"))
    events = [json.loads(r.message) for r in caplog.records if r.name == "ecomm.telemetry"]
    exported = [e for e in events if e["event"] == "catalog_exported"]
    assert len(exported) == 1
    assert exported[0]["product_count"] == 2
    assert exported[0]["status_filter"] == "all"
    assert "duration_ms" in exported[0]
  • Step 2: Run to verify it fails

Run: .venv/bin/python -m pytest backend/tests/test_products_export.py -q Expected: FAIL — AttributeError: module 'app.domains.products' has no attribute 'export_catalog' / EmptyCatalog.

  • Step 3a: Add EmptyCatalog to errors.py
# in backend/app/domains/products/errors.py, after NothingToApply

class EmptyCatalog(ProductsError):
    """PUC-9: nothing to export (no products, or none matching the status filter)."""
  • Step 3b: Add the streaming service to service.py
# in backend/app/domains/products/service.py
# add to the existing imports:
from collections.abc import Iterator
from . import codec, diff, repo, serialize, validate
from .errors import (
    DraftExpired, DraftNotFound, EmptyCatalog, NothingToApply, PreviewStale, RunNotFound,
)

# add this function (e.g. after summary()):

def export_catalog(
    conn: psycopg.Connection, storefront_id: int, status_filter: str
) -> Iterator[str]:
    """Stream the storefront's catalog as canonical CSV (PUC-9; INV-12 codec).

    Read-only: builds the snapshot, then streams the serializer over it. TEL-3
    is emitted once the stream is exhausted, with the product count and elapsed
    time. Raises EmptyCatalog before yielding anything if the (filtered) catalog
    is empty, so the BFF can answer 409 cleanly with no partial body.
    """
    started = time.monotonic()
    snapshot = repo.export_catalog(conn, storefront_id, status_filter)
    if not snapshot:
        raise EmptyCatalog()

    def _stream() -> Iterator[str]:
        yield from serialize.catalog_to_csv(snapshot)
        telemetry.emit(
            "catalog_exported",
            storefront_id=storefront_id,
            status_filter=status_filter,
            product_count=len(snapshot),
            duration_ms=int((time.monotonic() - started) * 1000),
        )

    return _stream()

Note: repo.export_catalog (snapshot, Task 4) and the service export_catalog (stream, here) share a name across layers — same intent, different layer, like list_runs/get_run already do. The empty check runs eagerly (before the generator is consumed) so 409 is decided before any bytes stream.

  • Step 3c: Export from the package surface
# in backend/app/domains/products/__init__.py
# add EmptyCatalog to the errors import and __all__, and export_catalog to the
# service import and __all__:
from .errors import (
    DraftExpired, DraftNotFound, EmptyCatalog, FileRejected, NothingToApply,
    PreviewStale, ProductsError, RunNotFound,
)
from .service import (
    confirm_draft, discard_draft, export_catalog, get_draft, get_draft_records,
    get_run, import_validate, list_runs, summary,
)
# __all__ gains: "EmptyCatalog", "export_catalog"
  • Step 4: Run to verify it passes

Run: .venv/bin/python -m pytest backend/tests/test_products_export.py -q Expected: PASS (6 passed)

  • Step 5: Commit
git add backend/app/domains/products/errors.py backend/app/domains/products/service.py backend/app/domains/products/__init__.py backend/tests/test_products_export.py
git commit -m "feat(products): streamed export service + TEL-3 + EmptyCatalog (PUC-9, §9.1)"

Task 6: Decimal / numeric round-trip unit tests (the property test's blind spot)

Files:

  • Modify: backend/tests/test_products_serialize.py

The INV-12 property test deliberately uses text fields. Numeric fields (price, cost, weight, volume, inventory_qty, variant position) need explicit round-trip coverage because their serialize→parse is value-identity, not string-identity. We assert the diff is empty (the real invariant), not the bytes. This catches e.g. a serializer dropping a Decimal's trailing zero in a way the parser then reads back differently.

  • Step 1: Write the failing/covering test
# append to backend/tests/test_products_serialize.py
from decimal import Decimal


def test_decimal_and_int_variant_fields_roundtrip():
    catalog = {
        "priced": CatalogProduct(
            id=1, handle="priced", title="Priced",
            option_names=(None, None, None),
            fields={"title": "Priced", "description_html": None, "vendor": None,
                    "product_type": "standalone", "google_product_category": None,
                    "tags": [], "status": "active", "published": True},
            variants=[CatalogVariant(
                id=1, options=(None, None, None), position=1,
                fields={"sku": "P1", "price": Decimal("18.00"),
                        "cost": Decimal("9.5"), "weight": Decimal("0.250"),
                        "inventory_qty": 40, "variant_image": None})],
            images=[],
        )
    }
    result = _roundtrip_diff(catalog)
    assert result.summary["unchanged"] == 1, result.records
    assert result.summary["updates"] == 0
  • Step 2: Run it

Run: .venv/bin/python -m pytest backend/tests/test_products_serialize.py::test_decimal_and_int_variant_fields_roundtrip -q Expected: PASS if Decimals round-trip cleanly (psycopg loads NUMERIC as Decimal, the diff compares Decimal == Decimal). If it FAILS, the diff flagged a numeric field — inspect result.records[0]["detail"]; the cause is almost always a str(Decimal) form the parser re-parses to a different Decimal (it shouldn't: Decimal(str(d)) == d always). Fix serialize._cell only if a real asymmetry exists; otherwise the test documents the guarantee.

  • Step 3: (only if it failed) Fix _cell

No change expected; Decimal(str(d)) == d holds. If weight stored as Decimal("0.250") re-parses to Decimal("0.250") (it does — equal value), the diff's != is False. Leave _cell as is.

  • Step 4: Run the full serializer suite

Run: .venv/bin/python -m pytest backend/tests/test_products_serialize.py -q Expected: PASS (all)

  • Step 5: Commit
git add backend/tests/test_products_serialize.py
git commit -m "test(products): decimal/int variant fields round-trip clean (INV-12 coverage)"

Task 7: BFF — GET /api/products/export (streamed, gated, 409 empty)

Files:

  • Modify: backend/app/main.py
  • Modify: backend/tests/test_products_endpoints.py

The endpoint: _merchant_gate → products.export_catalog(...) → stream as text/csv with a content-disposition attachment filename. 409 empty_catalog on EmptyCatalog. The status query param is validated to the four allowed values (default all). Uses FastAPI's StreamingResponse.

  • Step 1: Write the failing test
# append to backend/tests/test_products_endpoints.py

def test_export_returns_canonical_csv(fresh_db_url):
    with _merchant_client(fresh_db_url) as client:
        draft = _upload(client).json()
        client.post(f"/api/products/imports/drafts/{draft['id']}/confirm")
        resp = client.get("/api/products/export")
        assert resp.status_code == 200
        assert resp.headers["content-type"].startswith("text/csv")
        assert "attachment" in resp.headers["content-disposition"]
        body = resp.text
        assert body.splitlines()[0].startswith("Handle,")
        assert "moon-mug" in body


def test_export_status_filter_respected(fresh_db_url):
    with _merchant_client(fresh_db_url) as client:
        # GOOD_CSV's moon-mug has no Status column → defaults to active.
        draft = _upload(client).json()
        client.post(f"/api/products/imports/drafts/{draft['id']}/confirm")
        assert "moon-mug" in client.get("/api/products/export?status=active").text
        # No archived products → 409.
        assert client.get("/api/products/export?status=archived").status_code == 409


def test_export_empty_catalog_409(fresh_db_url):
    with _merchant_client(fresh_db_url) as client:
        resp = client.get("/api/products/export")
        assert resp.status_code == 409
        assert resp.json()["error"]["code"] == "empty_catalog"


def test_export_requires_merchant(fresh_db_url):
    with TestClient(create_app(database_url=fresh_db_url)) as client:
        assert client.get("/api/products/export").status_code == 401


def test_export_bad_status_422(fresh_db_url):
    with _merchant_client(fresh_db_url) as client:
        assert client.get("/api/products/export?status=bogus").status_code == 422
  • Step 2: Run to verify it fails

Run: .venv/bin/python -m pytest backend/tests/test_products_endpoints.py -k export -q Expected: FAIL — 404 (route not defined).

  • Step 3: Add the endpoint to main.py
# in backend/app/main.py
# add StreamingResponse to the fastapi.responses import:
from fastapi.responses import JSONResponse, PlainTextResponse, StreamingResponse

# add this route, next to the other /api/products/* routes (e.g. after products_sample_csv
# or before it — order is irrelevant for distinct paths):

    @app.get("/api/products/export")
    def export_products(
        status: str = Query(default="all", pattern="^(all|active|draft|archived)$"),
        conn: psycopg.Connection = Depends(get_conn),
        sess: dict | None = Depends(get_session),
    ):
        """Stream the catalog as canonical CSV, optionally status-filtered (§6.4; PUC-9)."""
        gate = _merchant_gate(conn, sess)
        if isinstance(gate, JSONResponse):
            return gate
        _account, sf = gate
        try:
            stream = products.export_catalog(conn, sf.id, status)
        except products.EmptyCatalog:
            return _error(409, "empty_catalog", "There are no products to export.")
        return StreamingResponse(
            stream,
            media_type="text/csv",
            headers={"content-disposition": 'attachment; filename="ecomm-products-export.csv"'},
        )

Note: EmptyCatalog is raised eagerly by export_catalog (Task 5) before the generator yields, so the try/except around the call catches it and we can answer 409 with no partial stream. Good.

  • Step 4: Run to verify it passes

Run: .venv/bin/python -m pytest backend/tests/test_products_endpoints.py -k export -q Expected: PASS (5 passed)

  • Step 5: Run the whole backend suite + import linter

Run: ( cd backend && ../.venv/bin/lint-imports ) && .venv/bin/python -m pytest backend -q Expected: PASS — import boundaries clean (serialize.py is in domains, imports only models/diff), all backend tests green.

  • Step 6: Commit
git add backend/app/main.py backend/tests/test_products_endpoints.py
git commit -m "feat(api): GET /api/products/export — streamed canonical CSV, 409 empty_catalog (§6.4, PUC-9)"

Task 8: Frontend API — export URL helper

Files:

  • Modify: frontend/src/productsApi.ts
  • Create: frontend/src/productsApi.export.test.ts

The export is a browser download, not a fetch (we want the browser's native save dialog and streaming). So the API module contributes a URL builder + the status list, not a request() wrapper.

  • Step 1: Write the failing test
// frontend/src/productsApi.export.test.ts
import { describe, expect, it } from "vitest";
import { EXPORT_STATUSES, exportUrl } from "./productsApi";

describe("export url", () => {
  it("lists the four status filters with 'all' first", () => {
    expect(EXPORT_STATUSES.map((s) => s.value)).toEqual(["all", "active", "draft", "archived"]);
  });
  it("builds the endpoint url with the status query", () => {
    expect(exportUrl("all")).toBe("/api/products/export?status=all");
    expect(exportUrl("archived")).toBe("/api/products/export?status=archived");
  });
});
  • Step 2: Run to verify it fails

Run: ( cd frontend && npx vitest run src/productsApi.export.test.ts ) Expected: FAIL — exportUrl / EXPORT_STATUSES not exported.

  • Step 3: Add the helper to productsApi.ts
// in frontend/src/productsApi.ts, after dialectLabel()

export type ExportStatus = "all" | "active" | "draft" | "archived";

// PUC-9 status filter, 'all' first (the default). Labels drive the menu.
export const EXPORT_STATUSES: { value: ExportStatus; label: string }[] = [
  { value: "all", label: "All products" },
  { value: "active", label: "Active" },
  { value: "draft", label: "Draft" },
  { value: "archived", label: "Archived" },
];

export function exportUrl(status: ExportStatus): string {
  return `/api/products/export?status=${status}`;
}
  • Step 4: Run to verify it passes

Run: ( cd frontend && npx vitest run src/productsApi.export.test.ts ) Expected: PASS (2 passed)

  • Step 5: Commit
git add frontend/src/productsApi.ts frontend/src/productsApi.export.test.ts
git commit -m "feat(frontend): export URL helper + status-filter list (PUC-9)"

Task 9: Frontend — the Export status-filter menu on the Products page

Files:

  • Modify: frontend/src/screens/products/ProductsPage.tsx
  • Modify: frontend/src/styles/products.css
  • Create: frontend/src/screens/products/ProductsPage.export.test.tsx

Replace the disabled Export button (lines ~66–71) with a small menu: a primary "Export" toggle that opens the four status options; choosing one navigates the browser to exportUrl(status) (triggering the download). When the catalog is empty, keep the disabled button + existing note (PUC-9). The menu is a native <details>/<summary> disclosure (matches the codebase's no-extra-deps, keyboard-accessible idiom — §6.6) so it needs no click-outside handler or extra state.

This is a UI change within the design system's existing primitives (buttons, the note class, a <details> disclosure already used in ImportPreview) — no new UI Design required (handbook R4 applies only beyond the design system).

  • Step 1: Write the failing test
// frontend/src/screens/products/ProductsPage.export.test.tsx
import { render, screen, within } from "@testing-library/react";
import { describe, expect, it, vi, beforeEach } from "vitest";
import ProductsPage from "./ProductsPage";
import * as api from "../../productsApi";

function mockSummary(count: number) {
  vi.spyOn(api, "getProductsSummary").mockResolvedValue({
    ok: true,
    value: { product_count: count, image_problem_count: 0, latest_run_id: null },
  });
  vi.spyOn(api, "listRuns").mockResolvedValue({ ok: true, value: [] });
}

describe("ProductsPage export menu", () => {
  beforeEach(() => vi.restoreAllMocks());

  it("offers the four status export links when the catalog is populated", async () => {
    mockSummary(3);
    render(<ProductsPage />);
    const exportToggle = await screen.findByRole("button", { name: "Export" });
    expect(exportToggle).toBeEnabled();
    // The four status options resolve to the endpoint URLs.
    const all = await screen.findByRole("link", { name: "All products" });
    expect(all).toHaveAttribute("href", "/api/products/export?status=all");
    expect(screen.getByRole("link", { name: "Archived" })).toHaveAttribute(
      "href", "/api/products/export?status=archived",
    );
  });

  it("disables export with a note when the catalog is empty", async () => {
    mockSummary(0);
    render(<ProductsPage />);
    expect(await screen.findByRole("button", { name: "Export" })).toBeDisabled();
    expect(screen.getByText("Export arrives when you have products")).toBeInTheDocument();
  });
});

Note: this is the repo's first React Testing Library test (SLICE-2's note said "no RTL in repo"). Verify RTL is available before writing — check frontend/package.json devDependencies for @testing-library/react + @testing-library/jest-dom + jsdom, and vitest.config/vite.config for environment: "jsdom" and a jest-dom setup. If any are missing, either (a) add them (npm i -D @testing-library/react @testing-library/jest-dom jsdom and set test.environment = "jsdom" + a setup file importing @testing-library/jest-dom), or (b) — to stay consistent with the repo's "pure-logic tests only" convention — drop the .tsx render test and instead unit-test a pure helper: extract an exportMenuItems() / isExportEnabled(count) from the component into productsApi.ts and test that, leaving the JSX visual-verified by E2E (Task 11) only. Prefer (b) unless RTL already exists — it matches SLICE-2/3's established pattern and avoids introducing a test framework as a side effect of this slice. The decision is the implementer's; log it in the transcript's Deferred decisions.

  • Step 2: Run to verify it fails (or pivot to 2b per the note)

Run: ( cd frontend && npx vitest run src/screens/products/ProductsPage.export.test.tsx ) Expected: FAIL — component still renders the disabled button.

If pivoting to (b), instead create frontend/src/screens/products/exportMenu.ts with isExportEnabled(count: number): boolean + reuse EXPORT_STATUSES/exportUrl, and frontend/src/screens/products/exportMenu.test.ts testing those pure fns.

  • Step 3: Implement the menu in ProductsPage.tsx

Replace the export block (currently):

        <div className="products__actions">
          <div className="products__export">
            <button type="button" className="btn-secondary" disabled title="Export arrives in a coming release">
              Export
            </button>
            <span className="note">Export arrives in a coming release</span>
          </div>
          <a className="btn-primary" href="#/products/import">
            Import products
          </a>
        </div>

with:

        <div className="products__actions">
          {empty ? (
            <div className="products__export">
              <button type="button" className="btn-secondary" disabled>
                Export
              </button>
              <span className="note">Export arrives when you have products</span>
            </div>
          ) : (
            <details className="products__export menu">
              <summary className="btn-secondary" role="button">
                Export
              </summary>
              <ul className="menu__list">
                {EXPORT_STATUSES.map((s) => (
                  <li key={s.value}>
                    {/* A real download: the browser navigates to the streamed
                        endpoint and saves the attachment (PUC-9). */}
                    <a className="menu__item" href={exportUrl(s.value)} download>
                      {s.label}
                    </a>
                  </li>
                ))}
              </ul>
            </details>
          )}
          <a className="btn-primary" href="#/products/import">
            Import products
          </a>
        </div>

Add the imports at the top of the file:

import {
  dialectLabel,
  EXPORT_STATUSES,
  exportUrl,
  getProductsSummary,
  listRuns,
  type ProductsSummary,
  type RunSummary,
} from "../../productsApi";

(empty is already computed at line ~57: const empty = summary.product_count === 0;.)

  • Step 4: Add menu styles to products.css
/* Export status menu (SD-0002 §5.2 — PUC-9). A native <details> disclosure so
   it's keyboard-accessible with no extra JS (§6.6). */
.products__export.menu {
  position: relative;
  display: inline-block;
}
.products__export.menu > summary {
  list-style: none;
  cursor: pointer;
}
.products__export.menu > summary::-webkit-details-marker {
  display: none;
}
.menu__list {
  position: absolute;
  right: 0;
  z-index: 10;
  margin: 0.25rem 0 0;
  padding: 0.25rem;
  list-style: none;
  background: var(--surface, #fff);
  border: 1px solid var(--border, #d8d3c8);
  border-radius: 8px;
  box-shadow: 0 6px 20px rgba(0, 0, 0, 0.12);
  min-width: 10rem;
}
.menu__item {
  display: block;
  padding: 0.5rem 0.75rem;
  border-radius: 6px;
  text-decoration: none;
  color: inherit;
}
.menu__item:hover,
.menu__item:focus {
  background: var(--surface-hover, #f3efe7);
}

(Check frontend/src/styles/ for the actual CSS custom-property names — the fallbacks above keep it working even if the vars differ; align them with the existing tokens if names like --border aren't what the design bundle uses.)

  • Step 5: Run the test + typecheck

Run: ( cd frontend && npx vitest run && npm run build ) Expected: PASS — export menu test green, tsc clean, Vite build succeeds.

  • Step 6: Commit
git add frontend/src/screens/products/ProductsPage.tsx frontend/src/styles/products.css frontend/src/screens/products/ProductsPage.export.test.tsx
git commit -m "feat(frontend): Export status-filter menu on the Products page (SD-0002 §5.2, PUC-9)"

(Adjust the git add paths if you pivoted to the pure-helper exportMenu.ts approach in Step 2b.)

Task 10: Full local gate — check.sh green

Files: none (verification task)

  • Step 1: Recreate the venv interpreter path expectation

The repo's check.sh picks $repo_root/.venv/bin/python. Confirm it exists in the worktree (init created it). If not: python3 -m venv .venv && .venv/bin/pip install -r backend/requirements.txt.

  • Step 2: Run the full gate

Run: bash scripts/check.sh Expected output (tail): ==> all gates green — import boundaries clean, all backend pytest green, frontend tsc+build clean, vitest green.

  • Step 3: If anything fails, fix before proceeding

Use superpowers:systematic-debugging. Do not move to E2E with a red unit gate.

  • Step 4: Commit any fixes
git add -A && git commit -m "fix(slice-6): <what you fixed> to green the check.sh gate"

Task 11: E2E — e2e_export_download and e2e_roundtrip_noop

Files:

  • Create: e2e/tests/export-download.spec.ts
  • Create: e2e/tests/roundtrip-noop.spec.ts
  • Modify: e2e/helpers.ts (add an export-download helper)

Both scenarios sign up, import good.csv (the existing fixture: 2 products, moon-mug + star-tee), then exercise export. Playwright captures the download via its download event. The round-trip scenario writes the downloaded file to a temp path and re-uploads it.

  • Step 1: Add a download helper to helpers.ts
// append to e2e/helpers.ts
import { tmpdir } from "node:os";
import { writeFile } from "node:fs/promises";

// Import good.csv and confirm it, leaving a 2-product catalog. Returns nothing;
// callers continue from the run-detail screen.
export async function importGoodCsv(page: Page) {
  await uploadFixture(page, "good.csv");
  await expect(page.getByRole("heading", { name: "Import preview — good.csv" })).toBeVisible();
  await page.getByRole("button", { name: "Import 2 products" }).click();
  await expect(page.getByRole("heading", { level: 1, name: "good.csv" })).toBeVisible();
}

// Click an Export status option and capture the downloaded CSV's text + path.
export async function exportCatalog(page: Page, label: string): Promise<{ text: string; path: string }> {
  // Open the <details> menu, then click the status option, capturing the download.
  await page.getByRole("button", { name: "Export" }).click();
  const [download] = await Promise.all([
    page.waitForEvent("download"),
    page.getByRole("link", { name: label }).click(),
  ]);
  const stream = await download.createReadStream();
  const chunks: Buffer[] = [];
  for await (const c of stream) chunks.push(c as Buffer);
  const text = Buffer.concat(chunks).toString("utf8");
  const path = join(tmpdir(), `export-${Date.now()}.csv`);
  await writeFile(path, text, "utf8");
  return { text, path };
}

Note: the <summary role="button"> is exposed to Playwright's getByRole("button", { name: "Export" }). Verify during the run; if the accessible role resolves differently, target page.locator(".products__export > summary") instead.

  • Step 2: Write e2e_export_download
// e2e/tests/export-download.spec.ts
// DoD scenario e2e_export_download (SD-0002 §6.8): export downloads a canonical
// CSV and the status filter is respected.
import { expect, test } from "@playwright/test";
import { exportCatalog, gotoProducts, importGoodCsv, signUpWithStorefront } from "../helpers";

test("e2e_export_download", async ({ page }) => {
  await signUpWithStorefront(page);
  await gotoProducts(page);
  await importGoodCsv(page);
  await gotoProducts(page);

  // Export "All products" → a canonical CSV with both handles.
  const all = await exportCatalog(page, "All products");
  expect(all.text.split("\n")[0]).toContain("Handle,");
  expect(all.text).toContain("moon-mug");
  expect(all.text).toContain("star-tee");

  // good.csv's products are active → "Active" exports both, "Draft" is empty/disabled-path.
  const active = await exportCatalog(page, "Active");
  expect(active.text).toContain("moon-mug");
});
  • Step 3: Write e2e_roundtrip_noop
// e2e/tests/roundtrip-noop.spec.ts
// DoD scenario e2e_roundtrip_noop (SD-0002 §6.8, PUC-10): exporting then
// re-importing the unmodified file previews as all-unchanged with the import
// action disabled and the "Nothing to change" note.
import { expect, test } from "@playwright/test";
import { join } from "node:path";
import { exportCatalog, gotoProducts, importGoodCsv, signUpWithStorefront } from "../helpers";

test("e2e_roundtrip_noop", async ({ page }) => {
  await signUpWithStorefront(page);
  await gotoProducts(page);
  await importGoodCsv(page);
  await gotoProducts(page);

  // Export the catalog, then re-import the unmodified file.
  const { path } = await exportCatalog(page, "All products");
  await page.getByRole("link", { name: "Import products" }).first().click();
  await expect(page.getByRole("heading", { name: "Import products" })).toBeVisible();
  await page.locator('input[type="file"]').setInputFiles(path);

  // Preview: everything unchanged, nothing to add/update (PUC-10).
  await expect(page.getByRole("button", { name: "2 unchanged" })).toBeVisible();
  await expect(page.getByRole("button", { name: "0 to add" })).toBeVisible();
  await expect(page.getByRole("button", { name: "0 to update" })).toBeVisible();

  // The import action is disabled, with the no-op note.
  const importBtn = page.getByRole("button", { name: /^Import 0 products$/ });
  await expect(importBtn).toBeDisabled();
  await expect(
    page.getByText("Nothing to change — your catalog already matches this file"),
  ).toBeVisible();
});

Note on the import-preview button label: when toApply === 0 the button reads Import 0 products and is disabled (ImportPreview.tsx line ~370). Confirm the exact rendered text during the run and tighten the regex if needed.

  • Step 4: Run the E2E suite

Run: bash scripts/e2e.sh Expected: all scenarios PASS — the 4 SLICE-5 scenarios + e2e_export_download + e2e_roundtrip_noop (6 total). Requires the dev Postgres up (docker compose up -d db). The harness builds the SPA and serves on :8765.

  • Step 5: Debug any failures against the real UI

If e2e_roundtrip_noop shows unexpected adds/updates, that's an INV-12 escape the property test missed (a field the generator didn't cover but good.csv exercises — e.g. an image alt text, a specific price format). This is exactly the bug class the E2E exists to catch. Trace it: download the export, diff it by eye against expectations, fix serialize.py, add a unit test reproducing it in test_products_serialize.py, re-run both gates.

  • Step 6: Commit
git add e2e/tests/export-download.spec.ts e2e/tests/roundtrip-noop.spec.ts e2e/helpers.ts
git commit -m "test(e2e): e2e_export_download + e2e_roundtrip_noop (SD-0002 §6.8, PUC-9/10)"

Task 12: Docs — DOC-1 (OPERATIONS.md) + DOC-4 (products-domain.md)

Files:

  • Modify: docs/OPERATIONS.md

  • Modify: docs/products-domain.md

  • Step 1: Update OPERATIONS.md (DOC-1)

In the "Products import/export ops" section, add an export subsection and the TEL-3 telemetry row. Add after the import telemetry table:

### Export (PUC-9, SLICE-6)

`GET /api/products/export?status=all|active|draft|archived` streams the
storefront's catalog as a canonical-format CSV (one codec, two directions — the
same format the importer parses). It is **read-only** (no draft, no run) and
storefront-scoped (INV-14). An empty catalog — no products, or none matching the
status filter — returns `409 empty_catalog`; the Products page disables the
Export action with a note in that case. The round-trip is lossless (INV-12):
re-importing an unmodified export previews as all-unchanged with the import
action disabled (PUC-10).

And add the TEL-3 row to the telemetry table:

| TEL-3 `catalog_exported` | export stream completes | `storefront_id, status_filter, product_count, duration_ms` |

And add e2e_export_download + e2e_roundtrip_noop to the E2E suite bullet (now six scenarios):

- Lives at `e2e/` — Playwright, Chromium, six scenarios (SLICE-5:
  preview/confirm, actionable errors, file rejection, cancel; SLICE-6:
  `e2e_export_download`, `e2e_roundtrip_noop`).
  • Step 2: Update products-domain.md (DOC-4)

Add a serializer/export section documenting the one-codec-two-directions design and how INV-12 is enforced:

## Export & the round-trip (SLICE-6)

`serialize.py` is the export half of "one codec, two directions": it turns the
`CatalogProduct` snapshot (the same one `repo.load_catalog` builds for the diff
engine) back into canonical CSV, writing exactly the columns `codec.py` /
`validate.py` parse, in the §6.5.1 row grammar (`HEADER` is the full canonical
column set; product fields + option names on the first row; one variant per
row; images interleaved; image-only rows when a product has more images than
variants).

`repo.export_catalog` returns the status-filtered snapshot list; the
`service.export_catalog` generator streams `serialize.catalog_to_csv` over it
and emits TEL-3 (`catalog_exported`) once the stream is exhausted. The BFF wraps
it in a `StreamingResponse`; an empty (filtered) catalog raises `EmptyCatalog`
→ `409 empty_catalog` before any bytes stream.

**INV-12** (`diff(catalog, import(export(catalog))) = ∅`) is locked two ways:
a property test (`test_products_serialize.py`) over 200 generated text-field
catalogs runs the real export→parse→diff loop and asserts every product is
`unchanged`; the `e2e_roundtrip_noop` browser scenario does the same through the
UI (export download → re-upload → all-unchanged preview, import disabled).
Numeric/Decimal fields — the property test's blind spot (string-form vs
value-identity) — get explicit round-trip unit tests.
  • Step 3: Verify the docs render (no broken structure)

Run: git diff --stat docs/ Expected: both files modified; eyeball the additions.

  • Step 4: Commit
git add docs/OPERATIONS.md docs/products-domain.md
git commit -m "docs(products): DOC-1 export ops + TEL-3, DOC-4 serializer/round-trip (SLICE-6)"

Task 13: Version bump → 0.6.0

Files:

  • Modify: VERSION
  • Modify: frontend/package.json

The deploy pin checks out the v<VERSION> tag and /healthz reports it back (flotilla's verify gate compares them), so VERSION must match the release tag.

  • Step 1: Bump VERSION

Set VERSION file contents to:

0.6.0
  • Step 2: Bump frontend/package.json

Change "version": "0.5.0" to "version": "0.6.0".

  • Step 3: Verify the backend reads it

Run: .venv/bin/python -c "from app.main import _APP_VERSION; print(_APP_VERSION)" 2>/dev/null || ( cd backend && ../.venv/bin/python -c "import app.main as m; print(m._APP_VERSION)" ) Expected: 0.6.0

  • Step 4: Commit
git add VERSION frontend/package.json
git commit -m "chore: version 0.6.0 — SLICE-6 export & round-trip"

Task 14: Final verification + ship through the §9 pipeline

Files: none (verification + deploy)

  • Step 1: Both gates green from a clean state

Run: bash scripts/check.sh && bash scripts/e2e.sh Expected: ==> all gates green then all 6 E2E scenarios pass. Evidence before assertion — do not claim done until you see both.

  • Step 2: Push the branch + open the PR
git push -u origin worktree-slice-6-export-roundtrip

Open a PR to main citing SD-0002 §7.2 SLICE-6, anchor #13, and the DoD checklist (BUC-3/4, INV-12 green, both E2E green, TEL-3 emitting, DOC-1 updated, v0.6.0). Merge it (autonomous posture; the reviewed artifact was the spec).

  • Step 3: Tag the release

After merge, on main:

git tag -a v0.6.0 -m "SLICE-6 — export & round-trip lock"
git tag -a "release/$(TZ=America/Los_Angeles date +%Y-%m-%dT%H-%M)" -m "PPE deploy — SLICE-6"
git push origin v0.6.0 && git push origin --tags
  • Step 4: Deploy to PPE via flotilla

Per DEPLOY-FLOTILLA.md + the deployment's operator guide. The deploy checks out the v0.6.0 tag. Use CLOUDSDK_ACTIVE_CONFIG_NAME=wiggleverse-ecomm for any gcloud-shelling command:

flotilla-core deploy ecomm

Expected: 9/9 phases green; /healthz on https://ecomm-ppe.wiggleverse.org reports version: 0.6.0.

  • Step 5: PPE smoke

Curl the health endpoint and confirm the version; if practical, walk the export download + round-trip no-op once against PPE (the §9 PPE browser run is still manual this slice). TEL-3 (catalog_exported) should appear in the VM journal after an export.

  • Step 6: Done

Ticket #13 SLICE-6 is done at merge + PPE-green (prod promotion stays the operator's gate). Finalize archives this plan to the content repo plans/.

Self-Review (run before handing off)

Spec coverage (SD-0002 §7.2 SLICE-6 DoD):

  • ✅ Canonical serializer (one codec, two directions) — Tasks 1–2, serialize.py.
  • ✅ Streamed export endpoint + status filter — Tasks 4–5 (service/repo), 7 (BFF).
  • ✅ Status-filter UI — Tasks 8–9 (Products page menu).
  • ✅ nothing_to_apply no-op preview behavior — shipped SLICE-5; reached by this slice's export and asserted by e2e_roundtrip_noop (Task 11).
  • ✅ INV-12 property test over text-field catalogs — Task 3; numeric coverage Task 6.
  • ✅ BUC-3 (catalog-wide change) — the import spine (SLICE-5) + this export round-trip.
  • ✅ BUC-4 (extraction always available, complete) — Task 7 endpoint + Task 9 UI.
  • ✅ E2E e2e_export_download, e2e_roundtrip_noop — Task 11.
  • ✅ TEL-3 catalog_exported emitting — Task 5; asserted Task 5 + DOC-1 Task 12.
  • ✅ DOC-1 updated — Task 12 (also DOC-4).
  • ✅ v0.6.0 shipped through merge + PPE — Tasks 13–14.

Placeholder scan: every code step carries complete code; no TBD/TODO; the two adaptive points (Task 9 RTL-vs-pure-helper, Task 3/11 debug-if-fails) give concrete decision criteria and fallbacks, not hand-waves.

Type consistency: catalog_to_csv (serialize), export_catalog (repo: list; service: Iterator) used consistently; EXPORT_STATUSES/exportUrl/ExportStatus consistent frontend⇄tests; EmptyCatalog/empty_catalog (exception/code) paired consistently across errors.py → service → init → main.py → tests.

Reference in New Issue View Git Blame Copy Permalink