Fix the quality gates, the dead pytest module, and conventions.json packaging (#1) - #2
Merged
Conversation
The committed tree failed several `npm run validate` gates: - prettier: seven src files were committed unformatted - markdownlint: ARCHITECTURE.md had list-spacing and trailing-newline errors - knip: `@registry/*` (a docs vite alias, not a package) was reported as an unlisted dependency, and the entry/project config belonged under a `workspaces` block now that the repo has one Also drop `docker-compose` and `ruff` from knip's ignoreBinaries (neither is invoked anywhere) and tighten the lint gate to `--max-warnings 0`, which the tree already satisfies.
The two files under skills/cdl-survey-types/references/ are codegen output, so their markdownlint errors could only be fixed at the source: - consecutive per-type warnings were emitted as separate blockquotes separated by a blank line (MD028); they are now one `>`-joined blockquote - the names/choice-codes list had no blank line above it (MD032) - blocks each carry a trailing newline and are then joined with another, which left double blank lines at the seams (MD012); a new `_tidy()` normalises the whole document once instead of making every emitter track blank lines Splitting the two emitters keeps them under the C901 complexity limit that `ruff check` enforces: `_question_types_md` now delegates to `_variants_by_base`, `_constraint_lines` and `_type_section`, and `_syntax_md` walks a (rule, section-builder) table instead of one long chain of `if` blocks. Regenerated output is byte-identical apart from the whitespace fixes above.
7a892c3 restructured registry/entities into definition/fixtures/generated and moved each entity's xlsform.xlsx into generated/, but test_xlsform_pyxform.py still looked for it at the exampleDir root. The parametrisation collapsed to zero cases, so its guard test failed and none of the 14 fixtures were actually validated. `generated/` is gitignored codegen output, so the guard now skips (rather than fails) when no generated/ directory exists at all, and still fails if the directories are there but hold no xlsx. Also extract helpers from test_testB_question_order_in_groups to clear its C901 complexity error: dict-or-object field access, the gid/question groupings, the per-group order assertion, and the ordered-titles lookup were all inline.
formtransform-app still runs two unrelated conversion engines: the old xlsform2lstsv npm package for the TSV path, and Pyodide plus the survey2ddi Python wheel for the DDI paths. This library covers all of that except the response-data CSV, so the app can drop both engines for a single dependency. The document records the verified API surface the app needs, the one gap that blocks deleting Pyodide (buildDdiXml uses `submissions` for caseQnty only — there is no TS equivalent of survey2ddi's build_data_csv), and links the sequenced issues filed in CorrelAid/formtransform-app (#4-#10). It also carries the scope notice the app's site is missing: the site presents FormTransform as an XLSForm converter, while the library deliberately rejects anything outside the registered CDL subset. Stating that on the page turns a confusing error into an expected one.
Neither repo runs a competing converter, so unlike formtransform-app this is a drift problem rather than a dependency one: both carry hand-maintained copies of knowledge the registry owns. qwac renders qwacback data — whose types come from here — using labels derived by string surgery on the type name, and encodes the type list a second time in its preview components, with nothing to catch a registry type that has no component. formulaid's skill teaches ~1000 lines of general XLSForm specification, well beyond what the pipeline accepts, and its app builds workbooks with xlsx and never validates them. The document records what the package can hand over today, the one thing it cannot (no labelled type catalogue: prefLabel/useWhen live in registry/, which "files": ["dist"] keeps out of consumers' node_modules), and links the sequenced issues filed in CorrelAid/qwac (#9-#12) and CorrelAid/formulaid (#9-#12). It also flags the three details that went stale in qwacback's own untracked brief.
Fixes #1. Every module that needed the generated conventions imported them as `generated/conventions.json` with an import attribute. That made the published package depend on two things outside its control: tsc copying the JSON into `dist/` (it only does so as a side effect of some .ts file importing it), and the consumer's bundler understanding `with { type: 'json' }`. A consumer installing from GitHub reported `dist/generated/conventions.json` missing and the bundle failing to resolve it. codegen now emits `src/generated/conventions.ts` alongside the JSON — same payload, same inferred types — and the library imports that. tsc always emits it and every bundler resolves it, so neither failure mode is reachable any more. The JSON stays as the artifact for non-TypeScript consumers, and `npm run build` copies it into `dist/generated/` after tsc so it is present in an installed package regardless of what imports it. Verified: clean `npm run build` produces dist/generated/{conventions.js, conventions.json}, no import attributes remain in dist, and `bun build dist/index.js --target=browser` bundles 55 modules without error.
The combined document made a reader of either repo skim past half of it, and the two plans have little in common beyond one shared upstream gap: qwac needs a labelled type catalogue for its UI, formulaid needs it for a type union. HANDOVER_QWAC.md now carries the label/preview-component drift, the DDI upload page's silence about what it validates, and the qwacback notes (Go-fmt-serialised DDI fields, structured validation findings, the stale details in that repo's own brief). HANDOVER_FORMULAID.md carries the skill's over-broad XLSForm reference, the unvalidated workbook output, and the cdl-survey-types bundling contract. Each document repeats the allowlist framing and the missing-catalogue constraint so it stands alone, and cross-links the other. The eight issues in the two repos were edited to point at their own document.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The tree was clean but every quality gate was red, one pytest module had been silently dead since a fixture move, and a consumer hit a real packaging bug (#1). This branch fixes all of it and adds the handover documents for the three downstream repos.
Fixes
ARCHITECTURE.mdhad 9 markdownlint errors, knip reported@registry/*(a docs Vite alias, not a package) as an unlisted dependency, andruff check .failed on three C901 complexity errors.--max-warningstightened from 12 to 0, which the tree already satisfied.conventions.jsonpackaging (fixes tsc build does not emit dist/generated/conventions.json —preparescript leaves dist incomplete #1) — the library imported generated JSON at runtime with an import attribute, so whetherdist/generated/conventions.jsonexisted depended ontsccopying it as a side effect, and whether a consumer could bundle it depended on their bundler understandingwith { type: 'json' }. codegen now emitssrc/generated/conventions.tsalongside the JSON and the library imports that;npm run buildcopies the JSON intodist/regardless.test_xlsform_pyxform.py— looked for<exampleDir>/xlsform.xlsx, which7a892c3moved to<exampleDir>/generated/xlsform.xlsx. The parametrisation collapsed to zero cases, so its guard failed and none of the 14 fixtures were validated. The guard now skips when nogenerated/directory exists at all, since that is gitignored codegen output.skills/cdl-survey-types/references/are generated, so their lint errors were fixed incodegen/emit_skill.pyrather than by hand: warnings emit one>-joined blockquote (MD028), a blank line precedes the naming list (MD032), and a new_tidy()collapses seam blank lines (MD012). The two emitters were split to clear C901 at the same time. Regenerated output is byte-identical apart from the whitespace fixes.Documents
HANDOVER_FORMTRANSFORM_APP.md,HANDOVER_QWAC.mdandHANDOVER_FORMULAID.md— one per downstream repo, each linking a sequenced set of issues filed in that repo (formtransform-app #4–#10, qwac #9–#12, formulaid #9–#12). They record the verified API surface each consumer needs and the two real gaps: no response-data CSV emitter (keeps Pyodide alive in the app), and no labelled machine-readable type catalogue (prefLabel/useWhenlive inregistry/, which"files": ["dist"]keeps out of consumers'node_modules).Verification
npm run validate(7 gates), 707 vitest, 33 pytest,uv run ruff check .,ruff format --check, andscripts/check-drift.shall pass. For #1 specifically: pristine clone →npm run build→dist/generated/holds bothconventions.jsandconventions.json; no import attributes remain indist/;bun build dist/index.js --target=browserbundles 55 modules;npm pack --dry-runlists both files.Not verified here: a fresh
npm installof the package end to end, because it fetchesxlsxfromcdn.sheetjs.comand remote tarballs are blocked in the environment this was built in.🤖 Generated with Claude Code