Skip to content

fix: include exact gestational chart boundaries - #101

Open
pacharanero wants to merge 3 commits into
livefrom
fix/exact-chart-boundaries
Open

fix: include exact gestational chart boundaries#101
pacharanero wants to merge 3 commits into
livefrom
fix/exact-chart-boundaries

Conversation

@pacharanero

@pacharanero pacharanero commented Aug 27, 2026

Copy link
Copy Markdown
Member

Clinical/reference-data decision

Following @eatyourpeas's response, the implementation now makes the distinction explicit:

  • reference curves start at the exact first ages for which LMS data exists: 23 weeks for weight/OFC, 25 weeks for length, and 42 weeks for BMI
  • measurements below those reference boundaries remain plottable at their exact corrected age with the raw observation value
  • no SDS or centile is fabricated where reference data is absent

Implementation

  • use named, full-precision gestational-age constants at the 23-, 25-, and 42-week chart boundaries
  • reuse those named constants for equivalent UK90/WHO threshold definitions
  • preserve the existing exact-age LMS endpoint clamp from PR fix: restore both age-two WHO chart points #102
  • preserve both sides of the 2-year and 4-year chart seams
  • document that the 42-week BMI point is WHO-sourced because UK90 preterm has no BMI LMS values

Clinical safety

This does not extrapolate reference data or broaden the range in which SDS can be calculated. A 22-week observation remains available in plottable_data with its exact corrected x-coordinate and raw y-value, while SDS and centile remain None with the below-reference explanation.

Testing

  • s/lint --running
  • focused boundary and chart-contract suite: 102 passed
  • full suite: 145,286 passed
  • no skipped tests

@pacharanero

Copy link
Copy Markdown
Member Author

@eatyourpeas this is a question for you really. I'm reviewing as much of the dGC Python/API/Chart/Demo code as possible, bit by bit. Where I find possible inconsistencies I'm flagging them, in case they were not deliberate. This one could have been deliberate - if it was then that's fine we can close the PR and make some comments in the code that explain the decision.

@eatyourpeas

Copy link
Copy Markdown
Member

I am not sure i understand this but yes, 25 weeks should be exactly 25 weeks and so on. It is important that the charting allows plotting of values below these gestations as babies these days are born and save at 22 w, so they need to be plottable even if there is no reference data down there to generate and SDS.

@pacharanero

Copy link
Copy Markdown
Member Author

Review: the truncated boundary literals, and a regression the branch introduces

This one took me (and some very experienced growth-calculations colleagues) several passes to fully understand, so this comment deliberately spells the numerics out. The core idea of this PR is right, and it does restore real boundary points - but as written it also silently drops the age-2 and age-4 chart points that PR #102 added. Detail and evidence below.

What the recurring decimal actually is

0.038329911 is the decimal age of exactly 42 weeks gestation:

(42 x 7 - 40 x 7) / 365.25 = 14 / 365.25 = 0.038329911019849415...

It legitimately appears in three different roles in the codebase, which is why it "turns up more than once":

  1. UK_90_PRETERM_AGES[-1] - the last chart age of the UK90 preterm series.
  2. WHO_2006_UNDER_TWOS_AGES[0] - the first chart age of the WHO infant series. It is the same physical point (42 weeks), which is where the preterm and infant datasets hand over to each other.
  3. should_default_to_youngest_reference() in global_functions.py - the handover logic that lets the centile-line generator continue through the 42-week disjunction using the younger reference.

Because UK90 preterm data and WHO 2006 infant data both contain a 42-week row, this single age is the seam between two reference datasets. That seam is the whole story of this PR.

Why the truncation matters: floating-point comparison against the data tables

The reference data tables (uk90_preterm.json) store the boundary ages at full precision:

uk90_preterm.json row:  decimal_age: 0.038329911019849415   (17 s.f.)
age_constants.py:       0.038329911019849415                (exact same float)
old chart AGES list:    0.038329911                          (truncated to 9 d.p.)

The truncated literal is a different float from the constant (they differ by ~2e-11). Whether that matters depends on the sign of the error, and here it bites at the lower boundary:

  • -0.325804244 (truncated 23 weeks) is less than the true -0.3258042436687201, so reference_data_absent() rejects the truncated literal as "below 23 weeks" and the chart silently loses its first weight/OFC point.
  • The 42-week end of the preterm series similarly never matched the should_default_to_youngest_reference comparison in a way that produced the intended handover point for BMI.

The chart code rounds x-coordinates to 4 d.p. for display, so the rendered points look fine - what goes missing is the first and last data points of each preterm series. Concretely, on live today:

Chart point live this PR
Weight preterm start -0.3066 (24 wk) -0.3258 (23 wk) - gains a point
OFC preterm start -0.3066 (24 wk) -0.3258 (23 wk) - gains a point
Height preterm start -0.2683 (26 wk) -0.2875 (25 wk) - gains a point
BMI infant start 0.0575 0.0383 (42 wk) - gains a point

So: the fixes themselves are real and clinically meaningful - the charts genuinely were missing their outermost preterm boundaries.

The regression: age-2 and age-4 points disappear

This PR removes _age_for_lms_lookup() from global_functions.py. That helper was added by PR #102 (merged into live after this branch was cut) to solve a related but different boundary problem:

  • The WHO daily tables end at 1.9986310746 (just under 2.0 years) and the child table starts at 2.0013689254 (just over).
  • When the centile generator asks for a point at exactly 2.0, interpolation against either table is out of range.
  • The helper clamps exactly-2.0 (and exactly-4.0) onto the nearest table endpoint so the disjunction points survive.

Without the clamp, measurement_from_sds(age=2.0) raises and the chart silently drops the point. Verified by running the chart builder on both branches:

Series live this PR
uk_who_infant points (weight) 36, ends at 2.0 35, ends at 1.9167 - age-2 point lost
uk_who_child points (weight) 25, starts at 2.0 24, starts at 2.0833 - age-2 point lost
Same for height, BMI, OFC, and the 4.0 disjunction present lost

The same failure is reproducible directly:

measurement_from_sds(UK_WHO, "weight", 0, "female", 2.0, default_youngest_reference=True)
  live:    11.4741   (clamped onto the table endpoint)
  branch:  IndexError: list index out of range

Why the branch's own tests do not catch it

test_chart_boundary_ages.py asserts the new first points appear, which they do. But nothing asserts that the existing boundary points at 2.0 and 4.0 survive, and the 1,700 chart test cases that would exercise adjacent ages are collection-filtered by design (see AGENTS.md). So the suite passes (145105 passed, 1700 skipped when I ran the branch) while the disjunction points silently vanish.

Suggested resolution

  1. Rebase onto live (the branch predates PR fix: restore both age-two WHO chart points #102, chore(release): v4.6.1 #103, feat: identify Python provenance engine #104, chore(release): v4.6.2 #105).
  2. Keep _age_for_lms_lookup() - the 42-week literals and the 2.0/4.0 clamp solve different ends of the same boundary problem and are needed together.
  3. Extend the new test file to assert the age-2 and age-4 points exist in every UK-WHO component, so this class of regression is pinned down.

One smaller question: the new BMI test expects a point at uk90_preterm x=0.0383, but the UK90 preterm BMI table contains no LMS values at all (all rows are empty-string sentinels), so that point is only produced via the default_youngest_reference handover from WHO infants. If that is the intended semantics - the 42-week BMI point is WHO-sourced, not UK90-sourced - it would be worth a comment in the test so the next reader does not have to rediscover it.

None of this changes the fact that the 23/25/42-week restoration is correct and valuable - it just needs to land on top of the age-two fix rather than instead of it.

@pacharanero

Copy link
Copy Markdown
Member Author

Updated this branch on top of current live to reflect the clinical distinction in your reply. The tests now prove that a 22-week observation remains plottable at its exact corrected age and raw value while SDS/centile stay unavailable, and separately prove that reference curves begin at exact 23/25/42-week boundaries. The existing 2-year and 4-year handover behavior is retained and tested.

I kept broader decimal-grid generation out of this PR: some lists intentionally preserve source rounding, so that deserves a separate, list-preserving review rather than being mixed into this boundary fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants