You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
load_data: bare except hides prep_*.py failures while the run reports success #1516
try:
prep_module.main(iargs)
except:
warnings.warn('prep_nisar.py failed. Assuming its result exists and continue...')
Two consequences:
The cause is never reported. The warning says the step failed, but not why. If stack files from an earlier run happen to be present, load_data continues on stale inputs and smallbaselineApp goes on to print Normal end, so the real failure only surfaces much later as empty or wrong products. [Feature] Silent all-zero output from invert_network with custom LiCSAR GeoTIFF reader #1491 is a report of that same shape from further down the pipeline.
A bare except: also catches KeyboardInterrupt and SystemExit. Ctrl-C during a long prep_* run is swallowed and turned into a warning.
Minimal reproduction
No NISAR data is needed -- a syntactically valid but empty HDF5 is enough to make prep_nisar fail.
processor : nisar
SAR platform/sensor : unknown from project name "None"
--------------------------------------------------
prepare metadata files for nisar products
prep_nisar.py -i "./gunw_*.h5" -d ./dem.vrt --frequency auto
update mode: True
Found 1 unwrapped files
.../mintpy/load_data.py:677: UserWarning: prep_nisar.py failed. Assuming its result exists and continue...
warnings.warn('prep_nisar.py failed. Assuming its result exists and continue...')
exit=0
Running the same prep_nisar invocation by hand shows what was discarded:
$ prep_nisar.py -i "./gunw_*.h5" -d ./dem.vrt --frequency auto
...
ValueError: NISAR auto (frequencyA) data for polarization 'HH' was not found in
./gunw_20260101_20260113.h5. Missing path:
/science/LSAR/GUNW/grids/frequencyA/unwrappedInterferogram.
Use --frequency B for frequencyB products.
That message already names the file, the missing dataset path, and the flag that would fix it -- prep_nisar did the diagnostic work. None of it reaches the user through load_data, and the exit status is 0. The same shape applies to the isce and gmtsar branches.
Suggested fix
Keep the deliberate continue-on-failure behaviour, but narrow the catch and surface the cause. script_name is already in scope (set at load_data.py#L602), so the hardcoded module name in the message can go too:
exceptExceptionase:
warnings.warn(f'{script_name} failed ({type(e).__name__}: {e}). ''Assuming its result exists and continue...')
Happy to open a PR if that shape looks right, or to leave it if you would rather handle it differently. One thing to note: #1510 adds an isce3 branch to the same part of prepare_metadata(), so I would rebase around it in whichever order is more convenient for #1510.
👋 Thanks for opening your first issue here! Please filled out the template with as much details as possible. We appreciate that you took the time to contribute!
Make sure you read our contributing guidelines.
Currently, load_data still works as long as the loaded inputs directory and HDF5 files exist, even if the upstream data files are missing, which was my common case where I generate the ifg stack on the server and analyze the TS on my laptop.
@s-sasaki-earthsea-wizard A PR that keeps this old behavior and prints out a more accurate msg would be great! Please feel free to go ahead; the #1510 PR will take some time as we need to go through testing.
Thanks again for the green light @yunjunz — opened #1517 with the fix.
The continue-on-failure behaviour is preserved as discussed; the warning now reports the exception type and message. Verified on all three affected branches (nisar / isce / gmtsar) with minimal reproductions, included in the PR description.
Description
prepare_metadata()wraps three of theprep_*calls in a bareexcept:that discards the exception and continues:load_data.py#L674-L677--nisarload_data.py#L722-L725--isceload_data.py#L790-L793--gmtsarTwo consequences:
load_datacontinues on stale inputs andsmallbaselineAppgoes on to printNormal end, so the real failure only surfaces much later as empty or wrong products. [Feature] Silent all-zero output from invert_network with custom LiCSAR GeoTIFF reader #1491 is a report of that same shape from further down the pipeline.except:also catchesKeyboardInterruptandSystemExit. Ctrl-C during a longprep_*run is swallowed and turned into a warning.Minimal reproduction
No NISAR data is needed -- a syntactically valid but empty HDF5 is enough to make
prep_nisarfail.Output (MintPy 1.6.4,
mainat06a98622):Running the same
prep_nisarinvocation by hand shows what was discarded:That message already names the file, the missing dataset path, and the flag that would fix it --
prep_nisardid the diagnostic work. None of it reaches the user throughload_data, and the exit status is 0. The same shape applies to theisceandgmtsarbranches.Suggested fix
Keep the deliberate continue-on-failure behaviour, but narrow the catch and surface the cause.
script_nameis already in scope (set atload_data.py#L602), so the hardcoded module name in the message can go too:Happy to open a PR if that shape looks right, or to leave it if you would rather handle it differently. One thing to note: #1510 adds an
isce3branch to the same part ofprepare_metadata(), so I would rebase around it in whichever order is more convenient for #1510.