issue: 5094361 Enable relaxed ordering for XLIO memory registrations by BasharRadya · Pull Request #656 · Mellanox/libxlio

BasharRadya · 2026-08-31T13:05:03Z

Request IBV_ACCESS_RELAXED_ORDERING for XLIO-owned host memory registrations so per_mkey mode can improve performance without a device-wide mlxconfig change.

Keep the runtime switch disabled by default to preserve existing ordering behavior. Older verbs builds continue with regular ordering, while modern libibverbs treats the flag as optional when kernel support is unavailable.

Document both configuration paths and their user-visible behavior.

Description

Please provide a summary of the change.

What

Subject: what this PR is doing in one line.

Why ?

Justification for the PR. If there is existing issue/bug please reference.

How ?

It is optional but for complex PRs please provide information about the design,
architecture, approach, etc.

Change type

What kind of change does this PR introduce?

Check list

Code follows the style de facto guidelines of this project
Comments have been inserted in hard to understand places
Documentation has been updated (if necessary)
Test has been added (if possible)

Summary by CodeRabbit

New Features
- Added an optional relaxed-ordering setting for XLIO-managed memory registration.
- Configure it through hardware_features.memory_registration.relaxed_ordering or XLIO_ENABLE_RELAXED_ORDERING.
- The setting is disabled by default and applies only when supported by the system.
Documentation
- Updated the configuration reference with the new setting, default behavior, scope, and support details.
Diagnostics
- Startup output now reports the relaxed-ordering configuration and memory-registration logs include access flags.

coderabbitai · 2026-08-31T13:05:18Z

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Enterprise

Run ID: 06cf7d20-511a-492c-9d2f-782c858d95e8

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Enterprise

Run ID: 918425f1-1e94-4f6c-be86-5f4560ba6536

📥 Commits

Reviewing files that changed from the base of the PR and between 9934438 and cd82b13.

📒 Files selected for processing (11)

config/m4/verbs.m4
docs/xlio_config_reference.md
src/core/config/config_var_definitions.h
src/core/config/descriptor_providers/xlio_config_schema.json
src/core/config/mappings.py
src/core/dev/ib_ctx_handler.cpp
src/core/ib/base/verbs_extra.h
src/core/main.cpp
src/core/util/sys_vars.cpp
src/core/util/sys_vars.h
src/core/util/sys_vars_configurator.cpp

Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review.

📝 Walkthrough

Walkthrough

XLIO adds the hardware_features.memory_registration.relaxed_ordering setting. The setting maps to XLIO_ENABLE_RELAXED_ORDERING, defaults to false, and conditionally adds IBV_ACCESS_RELAXED_ORDERING to XLIO-owned host memory registrations.

Changes

Relaxed Ordering Support

Layer / File(s)	Summary
Configuration contract and documentation `config/m4/verbs.m4`, `src/core/config/...`, `src/core/util/sys_vars.h`, `docs/xlio_config_reference.md`	The new boolean setting, environment mapping, default, verbs capability probe, and configuration documentation define relaxed-ordering support.
Runtime configuration loading `src/core/util/sys_vars_configurator.cpp`, `src/core/util/sys_vars.cpp`, `src/core/main.cpp`	The runtime registry and environment parser initialize `enable_relaxed_ordering`. Global diagnostics report its configured state.
Memory registration access flags `src/core/ib/base/verbs_extra.h`, `src/core/dev/ib_ctx_handler.cpp`	XLIO conditionally maps the verbs flag and applies it during XLIO-owned memory registration. Registration diagnostics include the access flags.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: 🔵 Low · up to cd82b

The change may apply relaxed ordering to TX user-buffer registrations even though that path does not depend on the affected ordering behavior, creating a bounded risk of an unnecessary runtime behavior change. The PR is otherwise mergeable with explicit owner follow-up.

Sequence Diagram(s)

sequenceDiagram
  participant ConfigurationRegistry
  participant mce_sys_var
  participant mem_reg
  participant ibv_reg_mr
  ConfigurationRegistry->>mce_sys_var: Initialize enable_relaxed_ordering
  mce_sys_var->>mem_reg: Provide relaxed-ordering setting
  mem_reg->>ibv_reg_mr: Register memory with access flags
  ibv_reg_mr-->>mem_reg: Return registration result

Suggested reviewers: omriritblat, pasis

Poem

A rabbit found a setting bright,
That lets memory hop just right.
The verbs flag joins the registration trail,
With strict defaults when support may fail.
Logs now show each flag in sight.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 4 functions across 8 files. (3 skipped: 3 …	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change: enabling relaxed ordering for XLIO memory registrations. The issue identifier adds useful tracking context.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

Full details: Docstring Coverage

Explanation

Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 4 functions across 8 files. (3 skipped: 3 unsupported.)

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands.}

greptile-apps · 2026-08-31T13:07:11Z

Greptile Summary

This PR adds an opt-in relaxed-ordering mode for XLIO memory registrations while retaining strict-ordering requests by default.

Adds build-time detection for IBV_ACCESS_RELAXED_ORDERING.
Wires the setting through legacy and structured configuration paths.
Applies the optional access flag during memory registration.
Adds startup diagnostics and generated configuration documentation.

Confidence Score: 5/5

The PR appears safe to merge because no blocking failure remains in the eligible follow-up review scope.

No blocking failure remains.

Important Files Changed

Filename	Overview
config/m4/verbs.m4	Adds configure-time detection for the relaxed-ordering verbs access flag.
src/core/config/descriptor_providers/xlio_config_schema.json	Defines the structured boolean setting, default, and user-facing behavior.
src/core/util/sys_vars.cpp	Initializes and reads the legacy relaxed-ordering environment setting.
src/core/util/sys_vars_configurator.cpp	Registers the setting with the structured runtime configuration registry.
src/core/dev/ib_ctx_handler.cpp	Conditionally adds the relaxed-ordering access flag to memory registrations and logs the resulting access mask.
src/core/main.cpp	Reports whether relaxed ordering was requested during startup.
docs/xlio_config_reference.md	Documents configuration paths, defaults, scope, and compatibility behavior.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
    A[Legacy environment or structured config] --> B[enable_relaxed_ordering]
    B --> C{Enabled and verbs flag available?}
    C -- Yes --> D[Add IBV_ACCESS_RELAXED_ORDERING]
    C -- No --> E[Keep existing access flags]
    D --> F[ibv_reg_mr]
    E --> F

_{Reviews (3): Last reviewed commit: "issue: 5094361 Enable relaxed ordering f..." | Re-trigger Greptile}

pasis

Looks good in general. Small changes needed.

pasis · 2026-08-31T13:13:32Z

  - [`core.syscall.fork_support`](#coresyscallfork_support) — Enable fork support
  - [`core.syscall.sendfile_cache_limit`](#coresyscallsendfile_cache_limit) — Sendfile byte limit
 - **[HARDWARE_FEATURES](#hardware_features)**
+  - [`hardware_features.memory_registration.relaxed_ordering`](#hardware_featuresmemory_registrationrelaxed_ordering) — Enable relaxed ordering for memory registration


[comment] Please see whether we already have existing category related to memory registration, PCI or similar. Create new category only if the new config doesn't fit anything existing.

no related category, i can simply remove the memory_Registeration and put relaxed oredering underas hardware features
WDYT?

Request IBV_ACCESS_RELAXED_ORDERING for XLIO-owned host memory registrations so per_mkey mode can improve performance without a device-wide mlxconfig change. Expose the request through XLIO_RELAXED_ORDERING and the structured hardware configuration, disabled by default to preserve existing MR access flags. Report when the request cannot be applied and distinguish XLIO's request from device-wide force_relaxed behavior. Signed-off-by: Bashar Abdelgafer <babdelgafer@nvidia.com>

BasharRadya requested review from OmriRitblat and pasis August 31, 2026 13:05

pasis reviewed Aug 31, 2026

View reviewed changes

BasharRadya force-pushed the relaxed_ordering_5094361 branch from d0dcce2 to 2ed87a0 Compare August 31, 2026 14:14

Navigation Menu

Uh oh!

issue: 5094361 Enable relaxed ordering for XLIO memory registrations - #656

issue: 5094361 Enable relaxed ordering for XLIO memory registrations#656
BasharRadya wants to merge 1 commit into
Mellanox:vNextfrom
BasharRadya:relaxed_ordering_5094361

BasharRadya commented Aug 31, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented Aug 31, 2026 •

edited

Loading

Review skipped

Walkthrough

Changes

Sequence Diagram(s)

❌ Failed checks (1 warning)

Uh oh!

greptile-apps Bot commented Aug 31, 2026 •

edited

Loading

Uh oh!

pasis left a comment

Uh oh!

Uh oh!

pasis Aug 31, 2026

Uh oh!

BasharRadya Aug 31, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

BasharRadya commented Aug 31, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

What

Why ?

How ?

Change type

Check list

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Aug 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Walkthrough

Changes

Sequence Diagram(s)

❌ Failed checks (1 warning)

Uh oh!

greptile-apps Bot commented Aug 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Flowchart

Uh oh!

pasis left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

pasis Aug 31, 2026

Choose a reason for hiding this comment

Uh oh!

BasharRadya Aug 31, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

BasharRadya commented Aug 31, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Aug 31, 2026 •

edited

Loading

greptile-apps Bot commented Aug 31, 2026 •

edited

Loading