Visual Relocalization from Sparse Views in Aliased and Low-Texture Environments via Novel View Synthesis

Official code for our IROS 2026 paper.

Maria Peribáñez¹, Javier Civera¹, Rudolph Triebel²·³, Riccardo Giubilato² ¹ University of Zaragoza · ² German Aerospace Center (DLR), Institute of Robotics and Mechatronics · ³ Karlsruhe Institute of Technology (KIT)

Paper

Method

• Offline map building. RGB keyframes and LiDAR are grouped into submaps. Rather than modelling the whole sequence, retrieval (DINOv2 + SALAD) selects the submaps relevant to the query set and we train one 3D Gaussian Splatting model per selected submap with geometry-aware supervision: the standard photometric loss plus (i) MVS depth/normal alignment from MVSAnywhere and (ii) a LiDAR-guided Chamfer loss.
• Online relocalization. Each query retrieves its top-k candidate submaps, and its 6-DoF pose is estimated directly against the trained 3DGS with 6DGS.

Repository structure

mvsanywhere/        # geometry-aware 3DGS training (patch on Niantic's regsplatfacto)
place_recognition/  # submap preprocessing (MPRF retrieval → training/pose)
6dgs/               # 6-DoF pose estimator: fork (git submodule; method unchanged)
baselines/          # PnP (code) + nerfstudio vanilla 3DGS (documented)
scripts/            # Training orchestration + evaluation
sample_data/        # Minimal example data (formats + runnable slice)
results/            # Paper metrics and figures

Installation

The pipeline builds on three external projects (four with the optional nerfstudio baseline); our own code needs only standard scientific Python (numpy, pandas, opencv, plyfile).

Project	Role	Setup
mvsanywhere	regsplatfacto + MVS depth/normal supervision (pulls in nerfstudio as training framework)	clone + apply our patch (see below)
6dgs (our fork)	6-DoF pose estimation	ships as the 6dgs/ submodule; conda env create -f environment.yml
MPRF	place recognition / retrieval (DINOv2 + SALAD)	clone (external, DLR-RM)
nerfstudio	vanilla 3DGS baseline (splatfacto) — only for --all	clone + install (optional)

Clone this repository recursively to pull the 6dgs fork and its CUDA submodules:

git clone --recursive https://github.com/DLR-RM/multimodal-gsplat-relocalization.git
# or, if already cloned: git submodule update --init --recursive

mvsanywhere can't be redistributed (non-commercial Niantic code), so we ship a patch instead of a fork — rebuild it in a Python 3.10 env:

git clone https://github.com/nianticlabs/mvsanywhere.git mvsanywhere
cd mvsanywhere && git checkout $(cat <repo>/mvsanywhere/BASE_COMMIT.txt)
git apply <repo>/mvsanywhere/changes.patch
pip install -e .   # pulls the pinned deps

Training also needs their mvsanywhere_hero.ckpt weights, at the path MVS_CKPT points to in scripts/config.sh.

Data

Our data: the moon_lake sequence of the DLR S3LI Vulcano Dataset, preprocessed into per-submap nerfstudio scenes. Download: 10.5281/zenodo.21843676: submaps.zip, 2.1 GB unpacked.

It holds the 103 submaps the pipeline reads: the 98 that get a 3DGS (listed in place_recognition/submap_train_lists/) and 5 more whose transforms.json only supplies a query's ground-truth pose at evaluation time. Unzip it onto SUBMAP_ROOT (scripts/config.sh):

mkdir -p "$PROJECT_ROOT/s3li" && cd "$PROJECT_ROOT/s3li"
unzip submaps.zip    # -> submapImages/submap_<id>/

sample_data/ documents the per-submap format and ships one real submap plus the 50 query images. Everything else: cached MVS predictions, trained models, baseline depth maps... the pipeline writes as it runs.

Pipeline

Set PROJECT_ROOT and CONDA_BASE in scripts/config.sh, which centralizes all paths, environments and hyperparameters (a few are hardcoded elsewhere — grep -rn /path/to .).

0. Retrieval → submap list — (optional; already done for the shipped inputs). Run MPRF on the queries and build the training top-k retrievals submaps list:

cd place_recognition
python map_queries_to_submaps.py    # MPRF rankings → query↔keyframe matches (+ manual curation → *_kfmatch_good.csv)
python make_submap_train_lists.py --csv <kfmatch_good.csv> --out_dir submap_train_lists --k 5

1. Train geometry-aware 3DGS per selected submap — Each run is timed and evaluated (PSNR/Chamfer → metrics.json); existing results are skipped.

bash scripts/train_gs_top1_submaps.sh        # our method only (3DGS + MVSA + LiDAR + Chamfer)
bash scripts/train_gs_top1_submaps.sh --all  # also the paper baselines + ablations (needs vanilla nerfstudio)

2. Prepare + pretrain the 6DGS pose estimator — Exports each trained 3DGS to the 6DGS format (+ point cloud) and pretrains the 6DGS identification module, for the GS_TAG variant selected in config.sh (default: our method).

bash scripts/train_6dgs_top1_submaps.sh

3. Estimate 6-DoF poses — For each query, 6DGS estimates the pose against its retrieved submaps → per-query poses + error tables in results/pose_estimation/.

bash scripts/run_pose_estimation.sh

4. Evaluate / reproduce the figures

python scripts/batch_metrics.py     # reconstruction: PSNR / Chamfer → results/all_submaps_metrics.csv 
python scripts/plot_rank5_bars.py   # pose error (rank ≤ 5) → results/{rotation,translation}_bar_rank5.png 

Baselines (comparison only) — see baselines/README.md: classical PnP (SuperPoint + SuperGlue + RANSAC over LiDAR-derived depth) and vanilla photometric 3DGS (splatfacto).

Training hyperparameters (paper): regsplatfacto 20k iters, mvsanywhere_hero.ckpt, λ_MVSD=0.05, λ_MVSN=0.1; Chamfer on at iter 2000, ramped to 8000, λ_Ch=5e-5. Photometric baseline: splatfacto 30k iters.

Citation

@inproceedings{peribanez2026relocalization,
  title     = {Visual Relocalization from Sparse Views in Aliased and Low-Texture Environments via Novel View Synthesis},
  author    = {Periba{\~n}ez, Maria and Civera, Javier and Triebel, Rudolph and Giubilato, Riccardo},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
  year      = {2026}
}

Acknowledgements

Built on nerfstudio, MVSAnywhere, 6DGS and MPRF. Data: DLR S3LI Vulcano Dataset.