Skip to content
BranchesTags

Repository files navigation

ISCE2 topsStack InSAR Processing Wrapper

License: MIT DOI Download ZIP

A collection of fault-tolerant Bash wrapper scripts for running the ISCE2 topsStack Sentinel-1 InSAR processing pipeline. Each script checks for existing outputs before reprocessing, enabling clean restarts after partial failures without re-running completed steps.

The scripts support both topsStack workflows generated by stackSentinel.py:

  • Interferogram stack (-W interferogram, default) — runs steps 1–16, producing unwrapped interferograms
  • SLC stack (-W slc) — runs steps 1–12 only, producing coregistered, merged SLCs in merged/SLC/

Because each script parses the run_files/ content dynamically (by config key, not by assumed file layout), the same wrapper scripts work for both workflows with no modification.

Authors and Attribution

Primary author: Manudeo Singh
Royal Society Newton International Fellow & Alexander von Humboldt Fellow
Department of Geography and Earth Sciences, Aberystwyth University
ORCID: 0000-0002-3511-8362
GitHub: manudeo.github.io

Acknowledgement: Portions of the swath/burst detection logic were adapted from scripts originally developed by my Humboldt Postdoc host, Professor Dr. Bodo Bookhagen (University of Potsdam). I am deeply grateful for his contributions to the open InSAR community and for his mentorship during my Humboldt Postdoc, which significantly shaped my understanding of InSAR techniques.

Overview

The topsStack processor in ISCE2 generates a series of run_files/run_NN_* scripts that must be executed in order. The problem is that when a job fails mid-way (e.g. due to memory, disk, or cluster time limits), the default workflow requires re-running all steps from scratch. These wrapper scripts solve that by:

  1. Inspecting output files — each script checks whether the expected output already exists for each date or pair.
  2. Collecting only failed/missing jobs — those that need reprocessing are written to a rerun_NN_missing.txt file.
  3. Parallelising remaining jobs — a configurable number of parallel processes ($processors) is used via Bash backgrounding and wait fences.
  4. Logging everything — all stdout/stderr is captured to log/rerun_NN_missing.log.

Prerequisites

  • ISCE2 (with topsStack processor) installed and on your $PATH
  • Sentinel-1 SLC data downloaded (e.g. via ASF Vertex or Copernicus Data Space)
  • A working ISCE2 topsStack configuration — i.e., run_files/ must already exist (generated by stackSentinel.py or equivalent)
  • Bash ≥ 4.0

Quick Start

Step 1 — Generate ISCE2 run files

Run stackSentinel.py (or SentinelWrapper.py if using a custom config) to generate the run_files/ directory.

Interferogram stack (full pipeline through unwrapping):

stackSentinel.py -s /path/to/slcs -d /path/to/dem -W interferogram [other options]
# or with a custom wrapper:
SentinelWrapper.py -c /path/to/configs/config_reference

SLC stack (coregistered SLCs only, no interferograms):

stackSentinel.py -s /path/to/slcs -d /path/to/dem -W slc [other options]

This creates the run_files/ directory. For an interferogram stack this goes up to run_16_unwrap; for an SLC stack it ends at run_12_merge_reference_secondary_slc.

Step 2 — Run Step 1 manually

Run 01 unpacks the reference SLC and is typically fast enough to run directly:

./run_files/run_01_unpack_topo_reference 2>&1 | tee log/run_01.log

Make sure the log/ directory exists:

mkdir -p log

Step 3 — Run Steps 2–16 using these wrapper scripts

Each wrapper script accepts an optional argument specifying the number of parallel processes (default: 1).

Get the scripts

In your InSAR working directory:

Option A — Clone (recommended, easiest to update later):

git clone https://github.com/manudeo/isce2-topsstack-wrapper.git
cp isce2-topsstack-wrapper/r*.sh isce2-topsstack-wrapper/run_pipeline.sh .
chmod +x r*.sh run_pipeline.sh

Option B — Download ZIP without Git:

wget https://github.com/manudeo/isce2-topsstack-wrapper/archive/refs/heads/main.tar.gz -O - | tar -xz --wildcards --strip-components=1 '*/r*.sh' '*/run_pipeline.sh'
chmod +x r*.sh run_pipeline.sh

Option C — Download via browser: Click Code → Download ZIP on the repository page, unzip, and copy the r*.sh and run_pipeline.sh files into your working directory.

Place the scripts in your ISCE2 WORK_DIR alongside run_files/ before proceeding. After placing the files, run:

chmod +x r*.sh run_pipeline.sh

Interferogram stack — run all steps:

bash r2.sh  [N_PROCESSORS]
bash r3.sh  [N_PROCESSORS]
bash r4.sh               # no parallelism — runs serially
bash r5.sh  [N_PROCESSORS]
bash r6.sh  [N_PROCESSORS]
bash r7.sh  [N_PROCESSORS]
bash r8.sh               # no parallelism — runs serially
bash r9.sh  [N_PROCESSORS]
bash r10.sh [N_PROCESSORS]
bash r11.sh              # no parallelism — runs serially
bash r12.sh [N_PROCESSORS]
bash r13.sh [N_PROCESSORS]
bash r14.sh [N_PROCESSORS]
bash r15.sh [N_PROCESSORS]
bash r16.sh [N_PROCESSORS]

SLC stack — stop after step 12 (no interferograms are formed):

bash r2.sh  [N_PROCESSORS]
bash r3.sh  [N_PROCESSORS]
bash r4.sh
bash r5.sh  [N_PROCESSORS]
bash r6.sh  [N_PROCESSORS]
bash r7.sh  [N_PROCESSORS]
bash r8.sh
bash r9.sh  [N_PROCESSORS]
bash r10.sh [N_PROCESSORS]
bash r11.sh
bash r12.sh [N_PROCESSORS]
# Stop here — merged/SLC/<date>/<date>.slc.full.xml is your final output

Example with 8 parallel processes:

bash r2.sh 8

Or run the full interferogram pipeline in one command (see run_pipeline.sh):

bash run_pipeline.sh 8

Script Reference

Script ISCE2 Step Description Workflow Parallel? Output check location
r2.sh run_02_unpack_secondary_slc Unpack secondary SLCs from raw SAFE files Both ✅ secondarys/<date>/<IW>.xml + burst_NN.slc.vrt
r3.sh run_03_average_baseline Compute perpendicular baselines for all secondary dates Both ✅ baselines/<ref>_<date>/<ref>_<date>.txt
r4.sh run_04_extract_burst_overlaps Extract burst overlap regions for NESD Both ❌ (runs full file)
r5.sh run_05_overlap_geo2rdr Geo-to-radar mapping for overlap regions Both ✅ coreg_secondarys/<date>/overlap/<IW>/range_top_01_02.off.vrt
r6.sh run_06_overlap_resample Resample secondary SLCs in overlap regions Both ✅ coreg_secondarys/<date>/overlap/<IW>_top.xml + _bottom.xml
r7.sh run_07_pairs_misreg Estimate azimuth/range misregistration per pair using NESD Both ✅ misreg/azimuth/pairs/<pair>/<pair>.txt + range equivalent
r8.sh run_08_timeseries_misreg Invert misregistration time series Both ❌ (runs full file)
r9.sh run_09_fullBurst_geo2rdr Geo-to-radar for full bursts (coregistration offsets) Both ✅ coreg_secondarys/<date>/<IW>/burst_01.slc.xml + last burst
r10.sh run_10_fullBurst_resample Resample full bursts to reference geometry Both ✅ coreg_secondarys/<date>/<IW>/burst_NN.slc + .vrt
r11.sh run_11_extract_stack_valid_region Determine valid pixel region across the stack Both ❌ (runs full file)
r12.sh run_12_merge_reference_secondary_slc Merge burst SLCs into full-swath SLCs Both ✅ merged/SLC/<date>/<date>.slc.full.xml ⬅ final output for SLC stack
r13.sh run_13_generate_burst_igram Form burst-level interferograms Interferogram only ✅ interferograms/<pair>/<IW>/fine_NN.int.vrt
r14.sh run_14_merge_burst_igram Merge burst interferograms Interferogram only ✅ merged/interferograms/<pair>/fine.int.xml
r15.sh run_15_filter_coherence Goldstein filter + coherence estimation Interferogram only ✅ merged/interferograms/<pair>/filt_fine.int + .cor
r16.sh run_16_unwrap Phase unwrapping (SNAPHU) Interferogram only ✅ merged/interferograms/<pair>/filt_fine.unw.xml

SLC Stack vs Interferogram Stack

stackSentinel.py can generate two different processing workflows from the same Sentinel-1 data. These wrappers support both.

How the scripts know what to run

The scripts do not hard-code file paths or assume a workflow type. Instead, each script:

  1. Splits the corresponding run_files/run_NN_* file into one-job-per-file chunks
  2. Greps each chunk for a workflow-specific config key (e.g. config_merge_, config_fullBurst_resample_) to extract the date or date pair
  3. Checks whether the expected output file already exists for that date/pair
  4. Adds missing jobs to the rerun queue

Because stackSentinel.py -W slc generates identically named and structured run_files/ for steps 1–12 (with the same config key patterns), the wrappers parse and check them correctly without any changes. Steps 13–16 simply don't exist in run_files/ for an SLC stack run, so those wrapper scripts are never called.

Workflow comparison

Aspect Interferogram stack (-W interferogram) SLC stack (-W slc)
stackSentinel.py flag -W interferogram (or omit, it is default) -W slc
Steps covered 1–16 1–12
Wrapper scripts used r2.sh – r16.sh r2.sh – r12.sh
Final outputs merged/interferograms/<pair>/filt_fine.unw merged/SLC/<date>/<date>.slc.full
Typical use case Time-series deformation (MintPy, GIAnT) External interferogram formation (e.g. custom pair networks, ISCE3 workflows)

When to use SLC stack

SLC stack is useful when you want to keep coregistration separate from interferogram formation — for example:

  • Forming a custom pair network not supported by stackSentinel.py directly
  • Using the coregistered SLCs as input to a different InSAR time-series processor
  • Running MintPy with a different coherence mask or filtering strategy applied externally

The scripts automatically detect the number of IW swaths (1, 2, or 3) and the number of bursts per swath by inspecting the reference/ directory. This means they work for all valid Sentinel-1 IW acquisitions regardless of the AOI. The relevant variables computed internally are:

  • IW_num — last swath (e.g. IW3)
  • IW_num1 — first swath (e.g. IW1)
  • IW_num2 — middle swath (e.g. IW2)
  • last_burst_num, last_burst_num1, last_burst_num2 — zero-padded burst indices (e.g. 05)
  • burst_bot_num*, azimuth_bot_num* — overlap-region file name fragments

Restarting After Failure

Each parallelised script follows this pattern:

  1. Strips & and wait lines from the run_files/ script.
  2. Splits it into single-job files in a working directory (e.g. run_02_unpack_secondary_slc/).
  3. Iterates over all jobs, checking for expected outputs.
  4. Collects only the missing jobs into rerun_NN_missing.txt.
  5. Adds & (parallel background) and wait fences at N_PROCESSORS intervals → rerun_NN_missing.cmd.
  6. Executes the command file and tees output to log/rerun_NN_missing.log.

To restart after a failure, simply re-run the same script. Already-completed dates will be skipped automatically.

run_pipeline.sh

A convenience wrapper that runs all steps (2–16) in order for the interferogram stack workflow. For the SLC stack workflow, run r2.sh through r12.sh manually as shown above.

bash run_pipeline.sh [N_PROCESSORS]

Steps 4, 8, and 11 run serially regardless (they are fast and single-threaded by design in ISCE2).

Expected Directory Structure

Before running, your working directory should look like:

<WORK_DIR>/
├── run_files/
│   ├── run_01_unpack_topo_reference
│   ├── run_02_unpack_secondary_slc
│   ├── ...
│   ├── run_12_merge_reference_secondary_slc   ← last step for SLC stack
│   ├── ...
│   └── run_16_unwrap                          ← last step for interferogram stack
├── reference/
│   ├── IW1/
│   ├── IW2/    (if applicable)
│   ├── IW3/    (if applicable)
│   └── IW*.xml
├── configs/
│   └── config_*
├── log/        ← must exist before running
├── r2.sh
├── r3.sh
├── ...
└── r16.sh

Known Limitations and Notes

  • Run 01 is not wrapped here. Run it directly as shown in Quick Start.
  • The scripts assume the run_files/ have already been generated by stackSentinel.py or equivalent.
  • Scripts that run serially (r4.sh, r8.sh, r11.sh) do not check individual outputs — they simply re-run the corresponding run_files/ script. If idempotency is needed, ISCE2 handles this internally for those steps.
  • Setting N_PROCESSORS too high relative to available RAM may cause out-of-memory failures, particularly during steps 9, 10, 13, and 16 (unwrapping). A safe starting point is 4–8 on a workstation with ≥64 GB RAM.
  • SNAPHU (step 16) is single-threaded per interferogram; parallelism here means multiple interferograms unwrapped simultaneously.
  • These scripts are specific to the ISCE2 topsStack workflow and are not compatible with ISCE3, which uses a fundamentally different YAML-based configuration and execution framework.

Citation

If you use these scripts in your research, please cite this repository and acknowledge ISCE2:

Singh, M. (2026). ISCE2 topsStack InSAR Processing Wrapper (v1.0.0). Zenodo.
https://doi.org/10.5281/zenodo.19294500

For ISCE2 itself, please cite:

Rosen, P. A., et al. (2012). The InSAR Scientific Computing Environment. EUSAR 2012; 9th European Conference on Synthetic Aperture Radar, 730–733.

Contributing

Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.

License

This project is licensed under the MIT License — see the LICENSE file for details. The software is provided as-is with no warranties or guarantees of any kind.

About

Fault-tolerant Bash wrappers for ISCE2 topsStack Sentinel-1 InSAR processing

Resources

Readme
License
Activity

Stars

1 star

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages