GeeWarp

GeeWarp implements the Feature Space Curvature Map (FSCM) methodology for the analysis and homogenization of HPC performance feature spaces extracted from Paraver traces.

The implementation extends the method proposed in:

Feature Space Curvature Map: A Method to Homogenize Cluster Densities

The project supports:

• GSOM-based Feature Space Curvature Modeling
• FSCM projection through inverse bilinear mapping
• Density homogenization of HPC performance feature spaces
• Configurable DBSCAN clustering on original and transformed spaces
• Automatic reintegration into Paraver traces

---

Installation

1. Clone the repository

git clone https://gitlab.bsc.es/performance-tools/geewarp.git
cd geewarp

2. Create a virtual environment (recommended)

python3 -m venv venv
source venv/bin/activate

3. Install dependencies

pip install -r requirements.txt

---

Dependencies

Python >= 3.9
NumPy >= 2.0
Matplotlib >= 3.7
scikit-learn >= 1.3
MiniSom >= 2.3.1
pandas >= 2.0

---

4. Add the repository to PATH

To execute the scripts directly from any location, add the repository to your PATH.

Example:

export PATH=$PATH:/path/to/geewarp

You can add this line to your shell configuration file:

• ~/.bashrc

Then reload the shell:

source ~/.bashrc

After that, the tools can be executed directly:

run_pipeline.py trace.prv

---

Quick Start

The recommended way to use GeeWarp is through the GeeWarp Clustering Pipeline.

The complete workflow can be executed using a single command:

python3 run_pipeline.py trace.prv

The pipeline automatically performs:

1. Trace preprocessing
2. Feature extraction
3. GSOM training
4. FSCM projection (2D feature spaces)
5. DBSCAN clustering
6. Reintegration of cluster identifiers into a new Paraver trace

---

Required Input Files

The pipeline expects the standard Paraver trace files:

trace.prv
trace.pcf
trace.row

These files must be located in the same directory.

---

Typical Outputs

Running the pipeline produces:

• clustered Paraver trace
• GSOM model
• FSCM projection
• DBSCAN visualizations
• CSV files containing the original and transformed feature spaces
• diagnostic information about the clustering process

The generated trace is immediately ready for visualization inside Paraver.

---

GeeWarp Clustering Pipeline

The pipeline consists of four major stages:

1. Trace extraction

   • Extract execution intervals and hardware counters from Paraver traces.

2. FSCM Phase 1: Feature Space Curvature Modeling (GSOM)

   • Train a Gravitational Self-Organizing Map.
   • Learn the density structure of the feature space.
   • Construct the Feature Space Curvature (FSC) model.

3. FSCM Phase 2: Curvature Mapping

   • Apply inverse bilinear FSCM projection over the GSOM mesh.
   • Project data from the curved feature space to a new Euclidean space.
   • Homogenize cluster densities while preserving the learned topology.

4. Clustering and reintegration

   • Apply DBSCAN clustering.
   • Inject cluster identifiers back into the Paraver trace.

The repository provides an end-to-end pipeline for clustering Paraver traces.

---

Running a Basic Example

python3 run_pipeline.py trace.prv

---

Explicit Feature Selection

Users can explicitly define the feature space.

Example: Temporal Analysis

python3 run_pipeline.py trace.prv \
    --features Duration TaskId

Example: Memory Behavior Analysis

python3 run_pipeline.py trace.prv \
    --features d_PAPI_TOT_INS x_PAPI_L3_TCM

---

Derived Features

In addition to raw hardware counters and temporal metrics, the pipeline also supports derived performance metrics computed automatically from the extracted trace data.

Currently supported derived features are:

Feature	Formula
CPI	d_PAPI_TOT_CYC / d_PAPI_TOT_INS
branch_miss_rate	x_PAPI_BR_MSP / x_PAPI_BR_INS
stall_ratio	x_RESOURCE_STALLS / d_PAPI_TOT_CYC
l3_miss_rate	x_PAPI_L3_TCM / d_PAPI_TOT_INS

Derived features can be combined with regular features.

Derived metrics are computed automatically during GSOM input generation and therefore do not need to exist explicitly in the extracted CSV file.

Example: Speculation Analysis

python3 run_pipeline.py trace.prv \
    --features d_IPC branch_miss_rate

---

Recommended Feature Spaces

Analysis Goal	Suggested Features
Computational behavior	d_IPC, d_PAPI_TOT_INS
Memory behavior	d_PAPI_TOT_INS, x_PAPI_L3_TCM
Temporal behavior	Duration, TaskId
Phase analysis	Duration, Begin_Time
Speculation analysis	d_IPC, branch_miss_rate
Stall/starvation analysis	d_IPC, stall_ratio
Cache pressure analysis	d_IPC, l3_miss_rate

---

Accelerating GSOM Training

For very large traces, GSOM training may become the dominant execution cost.

The option

--gsom-sample-size N

limits the number of samples used during GSOM training while preserving the complete dataset for the subsequent FSCM projection and DBSCAN clustering.

Example:

python3 run_pipeline.py trace.prv \
    --gsom-sample-size 10000

This option considerably reduces the training time while preserving the global topology learned by GSOM, making it particularly useful for traces containing hundreds of thousands of execution intervals.

Duration-Based Filtering

For traces without hardware counters, filtering can be performed using execution duration.

Example:

python3 run_pipeline.py trace.prv \
    --features Duration TaskId \
    --min-duration 1000000

---

Non-Counter Traces

Some traces do not contain hardware counters or IPC information.

For such traces, use:

--ignore-use-column
--ignore-instruction-filters

Example:

python3 run_pipeline.py trace.prv \
    --features Duration TaskId \
    --min-duration 10000000 \
    --min-tot-ins 0 \
    --ignore-use-column \
    --ignore-instruction-filters

---

Pipeline Outputs

The pipeline generates:

File	Description
*.clustered.prv	Clustered Paraver trace
*.clustered.pcf	Paraver configuration
*.clustered.row	Row description
original_units_with_clusters.csv	Original feature values with cluster IDs
dbscan_X_and_Xprime.png	DBSCAN clustering on the normalized feature space and the
	FSCM-transformed space
dbscan_original_units.png	DBSCAN results visualized in the original feature units
final_map_curvature.png	Feature Space Curvature (FSC) model learned during FSCM Phase 1
grid_comparison.png	Initial vs deformed grid

---

Output Interpretation

The pipeline produces clustering results in both the transformed and the original feature spaces.

• The transformed space (X') is intended to assess the effectiveness of FSCM in homogenizing cluster densities.
• The original feature space is intended for performance analysis, allowing clusters to be interpreted in terms of the original performance metrics while preserving the clustering obtained after FSCM.

DBSCAN Visualization

For two-dimensional feature spaces, the pipeline automatically generates two complementary visualizations:

• Normalized feature space (dbscan_X_and_Xprime.png), useful for understanding the FSCM transformation.

• Original feature space (dbscan_original_units.png), where clusters are projected onto the original performance metrics (e.g., IPC vs. Instructions), facilitating interpretation by performance analysts.

Paraver Integration

Clusters are injected using the Paraver event:

EVENT_TYPE
9   90000001   Cluster ID

The generated traces follow the same semantics used by Clustering_suite, including:

• End states
• Missing intervals
• Range-filtered intervals
• Noise intervals
• Cluster IDs starting at value 6

This enables direct comparison with Clustering_suite results inside Paraver.

---

Example Pipeline Execution

python3 run_pipeline.py \
    ../original-traces/SPHEXA.prv \
    --feature-set memory \
    --alpha 0.1

---

Example DBSCAN Output

Projection report:
found_cell=100.0%
u@border=0.0
v@border=0.0

DBSCAN@X
eps≈0.007
clusters=16
noise=2.4%
silhouette=0.7349

DBSCAN@X'
eps≈0.021
clusters=15
noise=2.9%
silhouette=0.6779

---

Standalone GSOM + FSCM Usage

The standalone mode applies GSOM and FSCM directly to CSV datasets.

Basic Example

python3 curvature_map.py \
    --data datasets/synthetic_data_syn10-paper.csv \
    --epochs 1000 \
    --alpha 0.10 \
    --radius-cutoff \
    --outdir results_gsom_fscm

---

Common Options

Argument	Description
--data	Input CSV dataset
--grid	SOM grid size (RxC, N, or auto)
--grid-scale	Scaling factor for automatic grid generation
--epochs	Number of GSOM iterations
--alpha	Initial learning rate
--sigma	Initial neighborhood radius
--warmup-iters	Classic SOM warm-up iterations
--warmup-alpha	Warm-up learning rate
--warmup-sigma	Warm-up sigma
--zcap	Maximum gravitational amplification
--radius-cutoff	Restrict updates to local radius
--seed	Random seed
--outdir	Output directory
--fscm-mode	hybrid, robust, or none
--plot-area-heatmap	Generate Jacobian distortion heatmap
--no-dbscan	Disable DBSCAN diagnostics
--gsom-sample-size	Maximum number of samples used during GSOM
	training. Larger datasets are randomly
	sampled to accelerate model construction.

---

Multi-Dimensional GSOM Support

The current implementation supports arbitrary-dimensional GSOM training.

However, the FSCM curvature mapping phase currently supports only 2D feature spaces because the multilinear transformation is implemented through bilinear cell mapping.

Examples:

• 2D:

  • IPC vs Instructions
  • Instructions vs L3 misses

• 3D:

  • Duration + IPC + Instructions

• Higher-dimensional:

  • Full hardware counter sets

When the input dimensionality is greater than 2:

• GSOM training still runs normally
• DBSCAN clustering is still performed
• FSCM projection is automatically disabled

---

Current Limitations

FSCM dimensionality

FSCM projection currently supports only 2D feature spaces.

Runtime scalability

Large traces may require significant runtime due to:

• GSOM training (partially mitigated through random sampling)
• FSCM projection over the GSOM mesh
• DBSCAN clustering

Further optimization of the FSCM projection phase is ongoing to improve scalability for large HPC traces.

DBSCAN sensitivity

DBSCAN results may be sensitive to:

• feature selection
• scaling
• epsilon estimation
• temporal-only feature spaces

Feature engineering sensitivity

Clustering quality strongly depends on the selected feature space.

Raw hardware counters may produce fragmented clusters dominated by workload scale, while normalized or derived metrics often expose more semantically meaningful execution behaviors.

---