Skip to content

load_data: bare except hides prep_*.py failures while the run reports success #1516

Description

Description

prepare_metadata() wraps three of the prep_* calls in a bare except: that discards the exception and continues:

try:
    prep_module.main(iargs)
except:
    warnings.warn('prep_nisar.py failed. Assuming its result exists and continue...')

Two consequences:

  1. The cause is never reported. The warning says the step failed, but not why. If stack files from an earlier run happen to be present, load_data continues on stale inputs and smallbaselineApp goes on to print Normal end, so the real failure only surfaces much later as empty or wrong products. [Feature] Silent all-zero output from invert_network with custom LiCSAR GeoTIFF reader #1491 is a report of that same shape from further down the pipeline.
  2. A bare except: also catches KeyboardInterrupt and SystemExit. Ctrl-C during a long prep_* run is swallowed and turned into a warning.

Minimal reproduction

No NISAR data is needed -- a syntactically valid but empty HDF5 is enough to make prep_nisar fail.

mkdir repro && cd repro

python -c "
import h5py
with h5py.File('gunw_20260101_20260113.h5', 'w') as f:
    f.create_group('science/LSAR/GUNW')
"
touch dem.vrt
python -c "
import mintpy, os, shutil
shutil.copy(os.path.join(os.path.dirname(mintpy.__file__), 'defaults/smallbaselineApp.cfg'), '.')
"

cat > repro.txt <<'EOF'
mintpy.load.processor = nisar
mintpy.load.unwFile   = ./gunw_*.h5
mintpy.load.demFile   = ./dem.vrt
EOF

load_data.py -t smallbaselineApp.cfg repro.txt; echo "exit=$?"

Output (MintPy 1.6.4, main at 06a98622):

processor : nisar
SAR platform/sensor : unknown from project name "None"
--------------------------------------------------
prepare metadata files for nisar products
prep_nisar.py -i "./gunw_*.h5" -d ./dem.vrt --frequency auto
update mode: True
Found 1 unwrapped files
.../mintpy/load_data.py:677: UserWarning: prep_nisar.py failed. Assuming its result exists and continue...
  warnings.warn('prep_nisar.py failed. Assuming its result exists and continue...')
exit=0

Running the same prep_nisar invocation by hand shows what was discarded:

$ prep_nisar.py -i "./gunw_*.h5" -d ./dem.vrt --frequency auto
...
ValueError: NISAR auto (frequencyA) data for polarization 'HH' was not found in
./gunw_20260101_20260113.h5. Missing path:
/science/LSAR/GUNW/grids/frequencyA/unwrappedInterferogram.
Use --frequency B for frequencyB products.

That message already names the file, the missing dataset path, and the flag that would fix it -- prep_nisar did the diagnostic work. None of it reaches the user through load_data, and the exit status is 0. The same shape applies to the isce and gmtsar branches.

Suggested fix

Keep the deliberate continue-on-failure behaviour, but narrow the catch and surface the cause. script_name is already in scope (set at load_data.py#L602), so the hardcoded module name in the message can go too:

except Exception as e:
    warnings.warn(f'{script_name} failed ({type(e).__name__}: {e}). '
                  'Assuming its result exists and continue...')

Happy to open a PR if that shape looks right, or to leave it if you would rather handle it differently. One thing to note: #1510 adds an isce3 branch to the same part of prepare_metadata(), so I would rebase around it in whichever order is more convenient for #1510.

Activity

  1. welcome commented on Aug 20, 2026

    @welcome

    👋 Thanks for opening your first issue here! Please filled out the template with as much details as possible. We appreciate that you took the time to contribute!
    Make sure you read our contributing guidelines.

  2. yunjunz commented on Aug 20, 2026

    @yunjunz
    Member

    Currently, load_data still works as long as the loaded inputs directory and HDF5 files exist, even if the upstream data files are missing, which was my common case where I generate the ifg stack on the server and analyze the TS on my laptop.

    @s-sasaki-earthsea-wizard A PR that keeps this old behavior and prints out a more accurate msg would be great! Please feel free to go ahead; the #1510 PR will take some time as we need to go through testing.

  3. s-sasaki-earthsea-wizard commented on Aug 23, 2026

    @s-sasaki-earthsea-wizard
    ContributorAuthor

    Thanks again for the green light @yunjunz — opened #1517 with the fix.

    The continue-on-failure behaviour is preserved as discussed; the warning now reports the exception type and message. Verified on all three affected branches (nisar / isce / gmtsar) with minimal reproductions, included in the PR description.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions