117 lines
6.0 KiB
Markdown
117 lines
6.0 KiB
Markdown
# products domain — developer notes
|
||
|
||
DOC-4 (SD-0002 §11): the import/export spine as built in SLICE-5, extended by
|
||
SLICE-6–8. Operator-facing material is in [`OPERATIONS.md`](./OPERATIONS.md);
|
||
the spec is SD-0002 in the content repo.
|
||
|
||
## Layout and pipeline
|
||
|
||
`backend/app/domains/products/` is layered like the rest of the app
|
||
(main → domains → platform, enforced by import-linter):
|
||
|
||
- `models.py` — canonical row model + the column registry.
|
||
- `codec.py` — bytes → `ParsedFile`; file-level gates only.
|
||
- `validate.py` — rows → `CanonicalProduct` blocks + per-row errors.
|
||
- `diff.py` — catalog × canonical products → apply plan + preview records.
|
||
- `repo.py` — SQL only: catalog snapshot, draft/run CRUD, apply primitives.
|
||
Never commits or rolls back.
|
||
- `service.py` — use-case orchestration; owns every transaction boundary and
|
||
emits the TEL events via `backend/app/platform/telemetry.py`.
|
||
|
||
```mermaid
|
||
flowchart LR
|
||
subgraph validate_path [import_validate]
|
||
A[parse_csv] --> B[build_products] --> D[compute_diff] --> E[(import_draft)]
|
||
C[load_catalog] --> D
|
||
end
|
||
subgraph confirm_path [confirm_draft]
|
||
E --> F[re-derive: parse → build → diff] --> G{fingerprint match?}
|
||
G -- yes --> H[one-transaction apply<br/>run + products + delete draft]
|
||
G -- no --> I[PreviewStale<br/>draft kept]
|
||
end
|
||
```
|
||
|
||
Confirm re-derives everything from the draft's stored `file_bytes` against the
|
||
live catalog, then checks the fingerprint — so what lands is exactly what the
|
||
preview showed, or the confirm refuses (`preview_stale`). A confirm with no
|
||
adds and no updates refuses with `nothing_to_apply`.
|
||
|
||
## Canonical model and blank-vs-absent
|
||
|
||
`models.py` is the one model every dialect maps to (INV-17). `KNOWN_COLUMNS`
|
||
is the registry header detection, unknown-column warnings, and validation all
|
||
read; `CLEAR_DEFAULTS` holds the reset values for clearable fields
|
||
(`status`, `published`, `product_type`, `tags`).
|
||
|
||
The §6.5.1 cell semantics, as implemented:
|
||
|
||
- **Absent column** → the field never enters `fields{}` → untouched by diff
|
||
and apply (never a change).
|
||
- **Present-but-empty cell** → `fields[name] = None` (an explicit clear) →
|
||
resolved at diff time to its `CLEAR_DEFAULTS` entry, or NULL where none
|
||
exists.
|
||
- **The position exception:** a cleared `Variant Position` has no
|
||
`CLEAR_DEFAULTS` entry — `diff.resolved_variant_fields` resolves it to the
|
||
variant's 1-based file order within its product, never to NULL.
|
||
|
||
Option names live both in `CanonicalProduct.option_names` (the values) and in
|
||
`fields{}` (the file-presence marker the diff needs for the absent-vs-clear
|
||
distinction).
|
||
|
||
## Error granularity
|
||
|
||
`validate.py` never raises on a row problem: every violation is recorded as a
|
||
merchant-language `RowError` and **poisons its whole product block** — the
|
||
product previews as `kind="error"` and is excluded from apply, while parsing
|
||
continues so one pass yields a complete accounting (BUC-1a). On apply, error
|
||
rows are recorded per line in `import_run_error`; `rows_errored` on the run is
|
||
the **error-row count**, while the preview's errors tile counts error
|
||
**products** — the two numbers legitimately differ.
|
||
|
||
File-level problems (`not_csv`, `missing_required_column`, `too_many_rows`,
|
||
`file_too_large`) raise `FileRejected` in `codec.py` instead: no draft is
|
||
created. The BFF adds an early 413 for oversized uploads (`main.py`).
|
||
|
||
## INV-11 mechanics
|
||
|
||
`compute_diff` makes **one walk** that produces two views of the same
|
||
computation: the typed apply `plan` (resolved natives — `Decimal`, `bool`,
|
||
lists) that `confirm_draft` executes, and the JSON-safe preview `records`
|
||
stored as draft JSONB and served verbatim to the SPA. Because both derive from
|
||
the same walk they cannot diverge. The `fingerprint` is
|
||
`sha256(json.dumps(records, sort_keys=True, separators=(",", ":")))`; a
|
||
mismatch at confirm means the catalog drifted since preview → `PreviewStale`
|
||
(409 `preview_stale`, draft kept for re-validation). The whole apply — run row,
|
||
product/variant/image writes, error rows, draft delete — is one transaction;
|
||
any exception rolls it back and emits TEL-6.
|
||
|
||
## Named seams (what later slices replace)
|
||
|
||
- **`import_draft.file_bytes` → objectstore key (SLICE-7).** The upload
|
||
currently lives as BYTEA on the draft row (`0002_products.sql`); SLICE-7
|
||
moves the bytes to object storage and stores a key.
|
||
- **`codec.detect_dialect` → Shopify (SLICE-8).** Today it always returns
|
||
`"canonical"`; SLICE-8 recognizes Shopify's exact header set here (INV-17).
|
||
- **Run-status complete shortcut → `fetching_images` (SLICE-7).**
|
||
`confirm_draft` inserts the run with `status="complete"` directly; SLICE-7
|
||
inserts it as `fetching_images` and hands off to the image-fetch task.
|
||
Relatedly, image rows are created with the schema default
|
||
`status='pending'` **today and stay pending** — placeholder behavior until
|
||
SLICE-7's fetch phase (the run-detail payload already carries the stable
|
||
`image_progress` / `image_outcomes` shape, zeroed).
|
||
|
||
## Test map
|
||
|
||
| File | Covers |
|
||
| --- | --- |
|
||
| `backend/tests/test_products_codec.py` | file-level gates: parse, caps (INV-18), required columns, dialect |
|
||
| `backend/tests/test_products_validate.py` | every §6.5.1 row-error rule, one fixture each |
|
||
| `backend/tests/test_products_diff.py` | classification, blank-vs-absent, option matching, fingerprint |
|
||
| `backend/tests/test_products_service.py` | draft lifecycle: validate/preview/discard, expiry, TEL-1 |
|
||
| `backend/tests/test_products_invariants.py` | INV-10 (never deletes), INV-14 (storefront isolation), apply transactionality, TEL-6 |
|
||
| `backend/tests/test_products_endpoints.py` | §6.4 API scenarios + auth/storefront gates |
|
||
| `e2e/tests/import-preview-confirm.spec.ts` | happy path: upload → preview → confirm → history |
|
||
| `e2e/tests/import-errors.spec.ts` | actionable row errors at preview and on the run report |
|
||
| `e2e/tests/import-file-rejected.spec.ts` | file-level rejection, picker stays live, no trace |
|
||
| `e2e/tests/import-cancel.spec.ts` | cancel at preview leaves no trace |
|