Update float type for window cutting and docs - #353
Merged
Conversation
Signed-off-by: Yiyu Ni <niyiyu@uw.edu>
niyiyu
requested review from
carlosgjs,
chengxinjiang and
mdenolle
as code owners
May 21, 2026 18:07
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #353 +/- ##
=======================================
Coverage 84.84% 84.84%
=======================================
Files 9 9
Lines 1986 1986
Branches 298 298
=======================================
Hits 1685 1685
Misses 191 191
Partials 110 110 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Signed-off-by: Yiyu Ni <niyiyu@uw.edu>
mdenolle
approved these changes
May 27, 2026
niyiyu
pushed a commit
that referenced
this pull request
Aug 3, 2026
…fft, rfft whitening) — timestamps stay float64 (#356) * Performance fixes from the 2026-08 audit of the S1 pipeline - cut_trace_make_stat: waveform windows back to float32 (the float64 flip in PR #353 was not needed for issue #352 and doubled FFT/memory cost); timestamps dataS_t STAY float64 — the actual #352 fix — now locked in by a regression test asserting exactly regular 300 s steps at a 2025 epoch. Drop the redundant mad() gate (std check suffices). - correlate(): replace the per-window loop of individual complex scipy.fftpack.ifft calls with one batched hermitian scipy.fft.irfft (multi-threaded, releases the GIL); accumulate the frequency-domain mean in float64; fix the remaining float32 t_corr in the substack_len branch (same #352 failure mode); remove the dead np.zeros(nwin*Nfft2) allocation; take an explicit is_autocorr flag (iiS == iiR) from the call site instead of comparing the full amplitude spectra. - whiten(): delete the dead arr_out allocations (433 MB transient in the 2D branch at campaign geometry, result discarded). - whiten_2D: rfft instead of full complex fftn — the legacy code zeroed everything outside the whitened positive-frequency band anyway. - moving_ave_2D: scipy.ndimage.uniform_filter1d instead of padded convolve2d (numerically identical for odd and even N). - detrend: vectorize the per-row loop into two matrix products. Guard rails: tests/test_ccf_regression.py runs the full pipeline A/B against verbatim copies of the legacy float64 implementations on a seeded synthetic day (per-window CCF correlation > 0.99999, stacked r = 1.0 to 9 decimals, identical stretching dv/v on a ±5% grid), and the #352 timestamp-regularity test in tests/test_noise_module.py. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Address Copilot review: preserve autocorr fallback, fix debug log placeholder - cross_corr(): default is_autocorr to None instead of False so external callers that don't pass the flag still get correlate()'s legacy spectra-comparison fallback; the production call site passes iiS == iiR explicitly. - cut_trace_make_stat(): the skipped-window debug log had a %s placeholder with no argument; log the trace id. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
stepshould be stepping size, not overlapping size between windows.