refactor: remove rho-stats from shapepipe (closes #710)#715
Merged
martinkilbinger merged 1 commit intodevelopfrom Apr 23, 2026
Merged
refactor: remove rho-stats from shapepipe (closes #710)#715martinkilbinger merged 1 commit intodevelopfrom
martinkilbinger merged 1 commit intodevelopfrom
Conversation
Per #657: rho-stats are obsolete in shapepipe and have been superseded by sp_validation / cosmo_val, which use shear_psf_leakage directly. Removes: - rho_stats() + Rho{1..5}SysTest classes (~990 lines in mccd_plot_utilities.py) and the rho-stats branch of mccd_plots_runner - stile, shear_psf_leakage, and treecorr as dependencies (Dockerfile, environment.yml, environment-dev.yml, install_shapepipe, and the python_example_runner depends list) - PLOT_RHO_STATS, RHO_STATS_STYLE, RHO_STATS_YLIM_{L,R} config keys from 10 example configs; mccd_plots_runner retained where needed for meanshapes Closes #710 (treecorr-5 incompatibility, fixed by removal). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributor
|
Looking good! Let's keep meanshapes, this is very useful and can be run on merged star and PSF catalogues as well. Not sure whether this is what @fabianhervaspeters is using for the 2D plots in the catalogue paper. |
martinkilbinger
approved these changes
Apr 23, 2026
cailmdaley
added a commit
that referenced
this pull request
Apr 24, 2026
#715 removed treecorr from environment.yml, environment-dev.yml, Dockerfile, and install_shapepipe, but left the dependency row in docs/source/dependencies.md and the now-orphan `jarvis:04` BibTeX entry in docs/source/refs.bib (no other references). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This was referenced Apr 24, 2026
Merged
cailmdaley
added a commit
to martinkilbinger/shapepipe-1
that referenced
this pull request
Apr 24, 2026
Commit 78c2668 deleted read_ext_cat_package/ and read_ext_cat_runner.py while renaming consumers to read_ext_sexcat_runner, but never added the renamed module. Likewise the commit moved Vizier retry logic out of create_star_cat.py and mask.py into shapepipe.utilities.vizier but that module was never created. This left the imports broken on v2.0: - mask.py:22 and create_star_cat.py:30 import shapepipe.utilities.vizier (ImportError on module load — mask is a core pipeline stage) - config_tile_Uc.ini, test_tile_det.py, and run_job_sp_canfar_v2.0.bash all reference read_ext_sexcat_runner (module-not-found at pipeline run) Changes: * shapepipe.utilities.vizier — new module exposing query_vizier(ra, dec, radius_arcmin, cat_id), consolidating the retry logic from mask.py and create_star_cat.py (server fallback, timeout escalation, random stagger, double-precision coord fix inherited from mask.py). * read_ext_sexcat_package/ + read_ext_sexcat_runner.py — restored from the deleted read_ext_cat_package with: - Renamed symbols (read_ext_sexcat, make_ldac_from_ascii, etc). - Tile-id overflow guard: the encoding tile_id = RRR*1000 + DDD silently collided for dec_idx >= 1000; extracted into _tile_id_from_file_number_string with an explicit bounds check. * pyproject.toml: dropped dead deps — treecorr and Stile (removed in CosmoStat#715), sip_tpv (removed in CosmoStat#716, merged today). These were stale holdovers on v2.0 that never got swept after the corresponding develop cleanups. * Dockerfile.jupyter: FROM points at ghcr.io/cosmostat/shapepipe:v2.0 instead of the martinkilbinger fork tag, so upstream builds work. Addresses the two new blockers flagged in PR CosmoStat#706 re-review: CosmoStat#706 (comment) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rho-stats are obsolete inside ShapePipe — superseded by
sp_validation/cosmo_val, which useshear_psf_leakagedirectly. Per @martinkilbinger's comment on #657.The
mccd_plots_runnermodule stays alive for meanshapes plots; only the rho-stats branch is removed. Thestiletreecorr-5 incompatibility (#710) is fixed by removal.Changes
mccd_plot_utilities.pyrho_stats(),new_BaseCorrelationFunctionSysTest,Rho1SysTest,DESRho{2..5}SysTest,get_params_rho,neg_dash, and related importsmccd_plots_runner.pystile/treecorrimports,has_stile/has_treecorrguardsmccd_package/__init__.pyPLOT_RHO_STATS/RHO_STATS_STYLE/RHO_STATS_YLIM_{L,R}docstring entriesstile,shear_psf_leakage,treecorrfromDockerfile,environment.yml,environment-dev.yml,install_shapepipe, andpython_example_runnerdepends listmccd_plots_runnerfor meanshapes; 2 (config_Pl_psfex{,_conv}.ini, which hadPLOT_MEANSHAPES = False) retired the runner entirelyQuestions for @martinkilbinger
Should meanshapes also be deleted? Kept it per your literal "option 1, delete" on #657, but if meanshapes is also obsolete by the same cosmo_val reasoning, happy to nuke the whole
mccd_plots_runnerin a follow-up — just say the word.Treecorr: after this PR, no runtime code in shapepipe imports
treecorr. Dropped it from deps alongside stile. Flagging in case any downstream production code relies on it being in the shapepipe container.Disclosure
Authored in collaboration with Claude. Code removal and config edits were done by sub-agents; I reviewed the diffs before committing. Docker image build was not tested locally — relying on CI for that.
Closes #710. Progresses #657.