StaMPS-PyRefactor

StaMPS-PyRefactor is a Python port and workflow refactor of the MATLAB StaMPS Persistent Scatterer InSAR processing chain. The repository provides the translated Python implementation under stamps_python/. The original MATLAB StaMPS source is used as a local translation reference, but is not vendored in this Git repository.

The current focus is PS-InSAR processing for ISCE/ISCE2 stack outputs. The goal is not to reproduce the MATLAB plotting GUI, but to make the processing and export pipeline runnable from Python on Windows/Linux with HDF5-based intermediate products.

This is an independent refactor/translation project and is not the official StaMPS distribution.

License

StaMPS is licensed under the GNU General Public License v3.0. This Python refactor is derived from the StaMPS workflow and translation reference, so the project is released under the same license: GPL-3.0.

See LICENSE.

Repository Layout

stamps_python/      Python implementation of preprocessing, Steps 1-8, and export
OPERATION_MANUAL.md Historical operation notes

Large datasets, generated HDF5/MAT files, GeoPackages, Shapefiles, logs, and validation runs are intentionally ignored by Git.

Main Python Entry Points

Script	Purpose
stamps_python/define_sb_pairs.py	Define SB pair graphs before patch preparation
stamps_python/prep_isce.py	Read ISCE/ISCE2 stack products and prepare Python-compatible StaMPS patch data
stamps_python/build_sb_phase.py	Build SB PATCH_*/pscands.1.ph from co-registered SLC pairs
stamps_python/stamps_main.py	Run StaMPS Steps 1-8
stamps_python/export_results.py	Export velocity and displacement time series to GeoPackage or Shapefile
stamps_python/validate_outputs.py	Helper checks for generated outputs

Completed Work

• ISCE/ISCE2 PS preprocessing has been migrated from mt_prep_isce concepts to Python-oriented outputs.
• Patch preparation can produce Python-compatible HDF5 inputs and no longer depends on MATLAB .mat products for the main flow.
• ISCE2 merged directories can now be used directly as preprocessing input. The Python preprocessor can generate reference-date metadata (reference_day.1.in plus the legacy-compatible master_day.1.in), day.1.in, bperp.1.in, heading.1.in, lambda.1.in, parms.json, and localparms.json without creating or reading a MATLAB/StaMPS INSAR_YYYYMMDD directory.
• StaMPS PS workflow Steps 1-8 are implemented in Python:
  • Step 1: initial data loading
  • Step 2: gamma estimation
  • Step 3: PS selection, including gamma re-estimation
  • Step 4: weeding
  • Step 5: phase correction and patch merge
  • Step 6: unwrapping with snaphu
  • Step 7: SCLA estimation and smoothing
  • Step 8: SCN filtering
• Full-chain PS processing has been run on a real ISCE2 merged stack dataset.
• Step 3 gamma re-estimation performance has been improved:
  • unique grid-cell reuse
  • faster spectral smoothing
  • large HDF5 bperp_mat reads avoid slow fancy indexing
• Step 4/5 save and merge bottlenecks have been optimized.
• Incidence-angle extraction from ISCE2 incLocal.rdr.full is supported.
• ISCE2 shared-stack preprocessing can create PS and SB views. SB pair graphs can be defined first with define_sb_pairs.py, then consumed by prep_isce.py for patch/candidate preparation and by build_sb_phase.py for SLC-native phase extraction.
• Result export is implemented:
  • velocity in mm/yr
  • displacement time series in mm
  • ps_deramp support before export
  • GeoPackage output by default
  • Shapefile output retained as an option

Current Validation Status

The PS-InSAR workflow has been run through Step 8 on a real ISCE2 SLC stack. The validated merged outputs include:

• ps2.h5
• ph2.h5
• pm2.h5
• phuw2.h5
• scla2.h5
• scla_smooth2.h5
• scn2.h5

The exported GeoPackage test completed with:

• 1,651,830 point features
• vel plus 45 date fields
• default correction v-dso: SCLA + SCN correction followed by deramp

The SLC-native Small Baseline workflow has also been run on a real ISCE2 merged stack through Steps 1-8. In this workflow, SB interferometric phase is built from co-registered SLC pairs, then inverted to single-master unwrapped date phases for SCLA, SCN, and export. Exported SB time series use local interpolation at the reference/master acquisition so that the master date does not appear as an artificial displacement spike.

Validation datasets and generated outputs are not committed to Git because of size and because each user will have their own ISCE/StaMPS project layout.

The preprocessing bootstrap path has also been validated with a clean input view containing only:

• SLC/
• geom_reference/
• baselines/
• input_file_YYYYMMDD

That validation intentionally excluded INSAR_YYYYMMDD, parms.mat, and localparms.mat. The generated Python metadata matched the native ISCE/StaMPS reference values for heading and perpendicular baseline.

Basic Usage

Install dependencies in a Python environment:

pip install -r requirements.txt

Inspect preprocessing options:

python stamps_python/prep_isce.py --help

Prepare an ISCE2 SLC stack for the PS-InSAR workflow. The input directory should be the ISCE2 merged directory produced by the SLC stack workflow, for example:

merged/
  baselines/
  geom_reference/
    hgt.rdr.full
    incLocal.rdr.full
    lat.rdr.full
    lon.rdr.full
  SLC/
    20200101/
      20200101.slc.full
      20200101.slc.full.xml
    20200113/
      20200113.slc.full
      20200113.slc.full.xml

Run preprocessing from the repository root:

python stamps_python/prep_isce.py path/to/merged \
  --output validation_runs/prep_isce_ps \
  --reference-date 20200101 \
  --bootstrap-metadata \
  --range-patches 8 \
  --azimuth-patches 2 \
  --da-thresh 0.4 \
  --run-calamp \
  --run-select \
  --run-extract \
  --phase-from-slcs \
  --write-step1

This command discovers co-registered SLCs from SLC/YYYYMMDD/*.slc.full, infers width.txt and len.txt, reads the acquisition dates, computes perpendicular baselines from baselines/YYYYMMDD/YYYYMMDD, computes heading from geom_reference/los.rdr.full, reads wavelength from input_file_*, and extracts longitude/latitude/DEM/incidence angle from geom_reference/. --bootstrap-metadata writes Python-native parms.json and localparms.json alongside the StaMPS-style text metadata files. --reference-date should be passed explicitly for reproducible PS processing. The older --master-date spelling is retained only as a backward-compatible alias. --da-thresh controls the amplitude-dispersion threshold used during PS candidate selection; the default is 0.4.

To only test metadata generation without running candidate selection or phase extraction:

python stamps_python/prep_isce.py path/to/merged \
  --output validation_runs/prep_isce_metadata_probe \
  --reference-date 20200101 \
  --bootstrap-metadata \
  --metadata-only \
  --range-patches 8 \
  --azimuth-patches 2

Example command matching the project full-stack validation settings, using relative paths:

python stamps_python/prep_isce.py data/isce2_stack/merged \
  --output runs/prep_isce_ps_8x2 \
  --reference-date 20250911 \
  --bootstrap-metadata \
  --range-patches 8 \
  --azimuth-patches 2 \
  --da-thresh 0.4 \
  --run-calamp \
  --run-select \
  --run-extract \
  --phase-from-slcs \
  --write-step1

Small Baseline Workflow

The recommended SB workflow uses the same ISCE2 merged stack as the PS workflow and does not require an original StaMPS INSAR_YYYYMMDD directory. It is split into three preprocessing commands:

1. define the SB pair graph;
2. prepare patches, candidates, and geometry;
3. build wrapped SB phase from the co-registered SLC pairs and write Step-1 HDF5.

If the PS workflow has already been run in a directory such as path/to/merged/stampsFix, create SB under a sibling/subdirectory such as path/to/merged/stampsFix/sb. The PS products do not need to be regenerated.

First define the pair graph:

python stamps_python/define_sb_pairs.py \
  --slc-root path/to/merged \
  --output path/to/merged/stampsFix/sb \
  -c 3 \
  --write-pscphase

Here -c 3 means each acquisition connects to the next three acquisitions in time. Increase it for a denser SB graph; decrease it for fewer interferograms. Keep this setting in the pair-definition command because both candidate selection and phase extraction use the same pair list.

Then prepare SB patches and candidates. This is the step that controls patch layout:

python stamps_python/prep_isce.py path/to/merged \
  --output path/to/merged/stampsFix/sb \
  --prepare sb \
  --reference-date YYYYMMDD \
  --bootstrap-metadata \
  --sb-pair-source list \
  --sb-pair-list path/to/merged/stampsFix/sb/ifgday.1.in \
  --range-patches 8 \
  --azimuth-patches 2 \
  --da-thresh 0.4 \
  --run-calamp \
  --run-select \
  --run-extract \
  --skip-phase

--skip-phase is intentional here: prep_isce.py prepares the patch layout, candidate files, and geometry rasters, while build_sb_phase.py fills the SB phase columns afterwards. --da-thresh controls SB candidate selection from pairwise SLC amplitude statistics. A larger value keeps more candidates.

Then build PATCH_*/pscands.1.ph from the same SLC stack and write SB Step-1 HDF5 products:

python stamps_python/build_sb_phase.py \
  --prepared path/to/merged/stampsFix/sb \
  --slc-root path/to/merged \
  --write-step1

Run StaMPS Steps 1-8 on the SB project:

python stamps_python/stamps_main.py \
  --start 1 \
  --end 8 \
  --config path/to/merged/stampsFix/sb

By default, SB patch merging follows the original StaMPS-style merge_resample_size=100, which averages dense candidates into one representative point per 100 m grid cell. To keep a denser SB result, rerun from Step 5 with a smaller merge grid:

python stamps_python/stamps_main.py \
  --start 5 \
  --end 8 \
  --config path/to/merged/stampsFix/sb \
  --merge-resample-size 50

Use --merge-resample-size 25 for a denser output, or 0 to disable merge resampling. Disabling resampling can preserve millions of points, but Steps 6-8 and export will be much heavier.

For a Windows path, the same sequence looks like:

python stamps_python\define_sb_pairs.py `
  --slc-root C:\data\isce_stack\merged `
  --output C:\data\isce_stack\merged\stampsFix\sb `
  -c 3 `
  --write-pscphase

python stamps_python\prep_isce.py C:\data\isce_stack\merged `
  --output C:\data\isce_stack\merged\stampsFix\sb `
  --prepare sb `
  --reference-date YYYYMMDD `
  --bootstrap-metadata `
  --sb-pair-source list `
  --sb-pair-list C:\data\isce_stack\merged\stampsFix\sb\ifgday.1.in `
  --range-patches 8 `
  --azimuth-patches 2 `
  --da-thresh 0.4 `
  --run-calamp `
  --run-select `
  --run-extract `
  --skip-phase

python stamps_python\build_sb_phase.py `
  --prepared C:\data\isce_stack\merged\stampsFix\sb `
  --slc-root C:\data\isce_stack\merged `
  --write-step1

python stamps_python\stamps_main.py `
  --start 1 `
  --end 8 `
  --config C:\data\isce_stack\merged\stampsFix\sb

Parameter Files

The Python workflow uses JSON parameter files instead of MATLAB parms.mat for the main path:

• parms.json is the base project parameter file generated by prep_isce.py --bootstrap-metadata.
• localparms.json is the local override file. Values here take precedence over parms.json.
• Code defaults are applied after loading both files for any missing parameters.

The effective priority is:

localparms.json > parms.json > Python defaults

For manual tuning, prefer editing localparms.json. This keeps generated base metadata separate from experiment-specific parameter overrides. Example:

{
  "select_reest_gamma_flag": "y",
  "density_rand": 20,
  "weed_standard_dev": 1.0,
  "scla_deramp": "y",
  "ref_lon": [120.12, 120.18],
  "ref_lat": [30.21, 30.26]
}

Reference Area Parameters

Reference PS selection supports two coordinate styles:

• ref_lon / ref_lat: longitude/latitude bounds in degrees. This is usually the easiest and safest way to define a reference area.
• ref_x / ref_y: local StaMPS xy bounds in metres. These coordinates are generated during Step 1 and are not image row/column indices.

If ref_x and ref_y are both present, they are used first. Otherwise the workflow falls back to ref_lon and ref_lat. If no PS are found inside the specified reference area, the workflow falls back to using all PS as reference.

Example local-coordinate reference box:

{
  "ref_x": [-5000, 5000],
  "ref_y": [10000, 18000]
}

To inspect coordinate ranges after Step 1 or a full run:

python - <<'PY'
import h5py
from pathlib import Path

p = Path("path/to/prepared_project/ps2.h5")
with h5py.File(p, "r") as f:
    xy = f["xy"][:]
    lonlat = f["lonlat"][:]

print("x range:", xy[:, 1].min(), xy[:, 1].max())
print("y range:", xy[:, 2].min(), xy[:, 2].max())
print("lon range:", lonlat[:, 0].min(), lonlat[:, 0].max())
print("lat range:", lonlat[:, 1].min(), lonlat[:, 1].max())
PY

When changing reference-area parameters after a completed run, rerun the affected later steps before exporting. A conservative choice is rerunning from Step 7 through Step 8 and then rerunning vector export.

Export deramped velocity and displacement time series to GeoPackage:

python stamps_python/export_results.py \
  --input-path path/to/prepared_project \
  --output-dir path/to/prepared_project/export \
  --format gpkg \
  --correction v-dso

For SB projects, use the SB project directory as --input-path. The exporter uses phuw2.h5 generated by SB inversion when it is available; if only phuw_sb2.h5 is present, it falls back to inverting SB IFG phases for export. GeoPackage fields include vel and one displacement column per acquisition date.

Optional Shapefile export:

python stamps_python/export_results.py \
  --input-path path/to/prepared_project \
  --output-dir path/to/prepared_project/export \
  --format shp \
  --correction v-dso

Important Notes

• The Python workflow targets PS-InSAR and SLC-native Small Baseline processing from ISCE/ISCE2 merged stacks. Full-chain validation should still be repeated for each new stack geometry before production use.
• prep_isce.py can discover dates and metadata from ISCE2 .vrt SLC mosaics, but the current calamp and candidate extractors read flat complex binary rasters. Translate VRT-only SLC mosaics to binary rasters before running --run-calamp or --run-select.
• The project is designed around HDF5 outputs (*.h5) and JSON parameter files (parms.json, localparms.json) rather than MATLAB .mat files for the main Python path.
• For the direct-merged PS workflow, the preprocessor does not require the original make_single_reference_stack_isce output directory (INSAR_YYYYMMDD). It uses SLC/, geom_reference/, baselines/, and input_file_* directly.
• snaphu must be installed and available on PATH, or configured through the StaMPS parameters, for Step 6 unwrapping.
• Full processing can be CPU-, memory-, and disk-intensive. Generated products can be hundreds of MB to multiple GB.
• Shapefile has format limitations and is not ideal for dense time-series exports. GeoPackage is the recommended vector format.
• Some documentation files were created during iterative validation and may lag behind the code. Treat stamps_python/ as the implementation source of truth.

Known Gaps / Not Yet Complete

• Small Baseline processing has been validated on one real ISCE2 stack, but still needs broader cross-stack and cross-processor validation.
• MATLAB plotting functions are not ported as GUI/figure tools. Only the non-plotting velocity/time-series export path has been extracted.
• TRAIN/tropospheric correction integrations are not ported end to end.
• Cross-platform validation has focused mainly on Windows with Python 3.11.
• Numerical equivalence to MATLAB is validated progressively but not guaranteed for every branch and optional correction mode.
• Packaging is not yet formalized as an installable Python package.
• Automated tests are still limited compared with the size of the workflow.

Citation / Attribution

This project is based on StaMPS (Stanford Method for Persistent Scatterers) and uses the original MATLAB implementation as a translation reference. Please cite the appropriate StaMPS publications and respect the GPL-3.0 license when using or distributing this work.