Skip to content

[DOCS] Add a gallery example on per-diagonal and total calendar-year tests - #1562

Open
scks1 wants to merge 4 commits into
casact:mainfrom
scks1:docs-valuation-correlation-caution
Open

scks1 wants to merge 4 commits into
casact:mainfrom
scks1:docs-valuation-correlation-caution

Conversation

@scks1

@scks1 scks1 commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Summary of Changes

Adds a gallery example, "Calendar Year Tests: Per Diagonal or in Total". It styles the S and L marks of the tail_sample paid triangle with Triangle.style, then splits the prism claims into 100 random samples and counts how often ValuationCorrelation flags them at p_critical=0.1, per diagonal and in total, at annual, semiannual and quarterly grain. The gallery index and the user guide's correlation section link to it. The ValuationCorrelation docstring drops the caution and links the example instead, keeping two wording fixes: it names total=True as the default, and it no longer reads a pass as proof that there is no calendar-year effect.

Related GitHub Issue(s)

Follows #1540 and #1544.

AI/LLM Usage

This change was made with Árbol, an AI assistant running on Muse, working under my direction. It wrote the example and the docstring changes, ran the notebook through Jupyter Book and the docstring's doctests locally, and drafted this description. I reviewed the diff before submitting.

Additional Context for Reviewers

The notebook takes about 21 seconds to execute in a local Jupyter Book build.

Submitter's Checklist

  • I have reviewed and am adhering to the standards outlined in the project Governing Doc.
  • The PR subject title summarizes the changes, with one proper prefix ([FIX], [FEAT], [DOCS], [TST], [CHORE], or [BRK]).
  • I am a human (not a bot), and this PR form is written by a human.

Reviewer's Checklist

  • The implementation addresses the associated issue(s).
  • The implementation is appropriate, maintainable, and follows ARCHITECTURE.md.
  • PR subject title has the proper prefix and the subject is appropriate.
  • Relevant issue(s) are linked.
  • AI/LLM usage is disclosed and appropriate.
  • Documentation and tests are appropriate.
  • CI tests passed, or any failures are acceptable.
  • Leave a comment with the final recommendation (e.g. approve as is, request a secondary review, or flag an area for more review).

@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown

Pyright Type Completeness

View the full pyright --verifytypes output for this commit

Project (full chainladder package, at this PR's head): 16.7% of exported symbols fully typed (265 / 1586)

Known Ambiguous Unknown Total
Project (head) 265 101 1220 1586

Other symbols referenced but not exported by chainladder: 24

Known Ambiguous Unknown Total
Other (head) 11 1 12 24

Symbols without documentation:

  • Functions without docstring: 326
  • Functions without default param: 0
  • Classes without docstring: 10

Patch (exported symbols added or changed by this PR): no exported symbol type-completeness changes detected.

@codecov

codecov Bot commented Nov 9, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.76%. Comparing base (8e2e53c) to head (8067896).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1562      +/-   ##
==========================================
+ Coverage   94.59%   94.76%   +0.16%     
==========================================
  Files          98       98              
  Lines        6233     6318      +85     
  Branches      799      804       +5     
==========================================
+ Hits         5896     5987      +91     
+ Misses        216      212       -4     
+ Partials      121      119       -2     
Flag Coverage Δ
unittests 94.76% <ø> (+0.16%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@henrydingliu

Copy link
Copy Markdown
Member

i'm leaning towards mack's caution being too in-depth for docstrings. i also think a caution is more persuasive when accompanied by a concrete example. historically we've approached this via either the user guide or the example gallery.

The example simulates triangles with no calendar-year effect and counts
how often ValuationCorrelation flags them at p_critical=0.1. At least one
diagonal is flagged in about 15% of 10-year triangles and 77% of 30-year
triangles, while the total test stays near 10%. The tail_sample paid
triangle shows the two forms disagreeing on real data.

Developed with an AI assistant (Árbol, running Muse).
Name total=True as the default, say that a pass does not prove there is
no calendar-year effect, and link the gallery example that compares the
per-diagonal and total forms.

Developed with an AI assistant (Árbol, running Muse).
Developed with an AI assistant (Árbol, running Muse).
@scks1
scks1 force-pushed the docs-valuation-correlation-caution branch from 6ddf178 to fbfd709 Compare November 10, 2026 11:10
@scks1 scks1 changed the title [DOCS] Add Mack's caution on per-diagonal calendar-year tests [DOCS] Add a gallery example on per-diagonal and total calendar-year tests Nov 10, 2026
@scks1

scks1 commented Nov 10, 2026

Copy link
Copy Markdown
Contributor Author

Agreed. I moved the caution into a gallery example, "Calendar Year Tests" (docs/gallery/plot_valuation_correlation.ipynb). It simulates 1,000 triangles with no calendar-year effect at each of four sizes and counts false alarms at p_critical=0.1: at least one diagonal is flagged in 15% of 10-year triangles and 77% of 30-year ones, while the total test flags 8.6% to 11.7% at every size. It opens with tail_sample's paid triangle, where one diagonal flags and the total test does not. The docstring keeps the two small wording fixes and links the example in one sentence, and the user guide's correlation section links it too. I replaced the two earlier commits instead of stacking reverts on them.

(This comment was made with Árbol, an AI assistant, under my direction.)

@henrydingliu

Copy link
Copy Markdown
Member

i'm good with the docstring changes.

on the top half of the notebook, i'm thinking we can leverage the recently implement triangle.styler to make it really pop. can you use color coding to help the read visualize the S's and L's of the tail_sample triangle? i have an inkling that it will make the 2008 calendar year easily stand out.

on the bottom half of the notebook, i'm wondering if we can take the same premise about too many diagonals and make it about triangle grain. for example, take the prism triangle, take random samples of it (the dataset is at a claim level), then compare the quarterly, semiannual, and annual grains across all samples.

The top half of the calendar-year gallery example now shows the S and L
marks of the tail_sample paid triangle with Triangle.style, letters kept
in the cells. The 2008 diagonal holds six L marks, two medians and no S.

The bottom half replaces the simulated triangles with prism: the claims
are split at random into 100 samples, and each sample's paid triangle is
tested at annual, semiannual and quarterly grain. At least one diagonal
is flagged in 12, 49 and 77 of the 100 samples, while the total test
flags 12, 7 and 11. The docstring's pointer to the example, the card alt
text and the thumbnail follow the new content.

Developed with an AI assistant (Árbol, running Muse).
@scks1

scks1 commented Nov 11, 2026

Copy link
Copy Markdown
Contributor Author

Thanks, both halves are reworked in 8067896.

The top half now styles the tail_sample paid triangle with Triangle.style: L cells red, S cells blue, with the letters kept in the cells and * for the column medians. 2008 does stand out. Its eight link ratios are six L's and two medians, with no S, so the per-diagonal test flags 2008 (probability 1 in 32), while the total test does not (Z = 12, inside the band of 9.5 to 15.5).

The bottom half now uses prism instead of the simulated triangles. It splits the 34,244 claims at random into 100 disjoint samples of 342 or 343 claims (seed 2026) and tests each sample's paid triangle at annual, semiannual and quarterly grain. At least one diagonal is flagged in 12, 49 and 77 of the 100 samples, no single diagonal in more than 10, and the total test flags 12, 7 and 11. Four other seeds, and 50 samples of about 685 claims, show the same rise. The notebook runs in about 21 seconds.

Since the simulation is gone, the docstring line that points to the example now says it compares the two forms across grains, and the gallery thumbnail shows the new chart.

(This comment was made with Árbol, an AI assistant, under my direction.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants