build(deps): add s3 and gcs extras and floor fsspec (#1256)

[chethanuk]

Problem

Curator has no cloud-storage extras, so users who want s3:// or gs:// paths install s3fs or
gcsfs themselves — and nothing stops the version they get from being one Curator cannot use.
Fixes #1256.

Root cause

[project.dependencies] declares a bare, unbounded "fsspec". The >=2025.3.0 floor everyone
treats as Curator's pinned range lives only in [tool.uv] override-dependencies, which pip never
reads and which uv applies by replacing every declared fsspec requirement. Two consequences:

• a pip user installing s3fs/gcsfs alongside Curator pulls fsspec down to that backend's
  fsspec==<CalVer> pin with nothing objecting;
• under uv, uv.lock already ships s3fs 2024.12.0 (which requires fsspec==2024.12.0) beside
  fsspec 2026.3.0 — a pairing s3fs itself declares invalid.

The missing extras are the symptom. Without the floor, new extras would advertise a range whose
lowest member is broken.

Change

• pyproject.toml — floor fsspec>=2025.3.0 in [project.dependencies]; add s3 (s3fs) and
  gcs (gcsfs) optional-dependencies, both >=2025.3.0; roll both into all; bump the
  interleaved_cpu and test-group s3fs entries from >=2024.12.0 to >=2025.3.0 — that
  interleaved_cpu entry is what pins the lock to the broken s3fs today.
• uv.lock — regenerated.
• fern/.../get-started/installation.mdx — two rows in the Package Extras table. Users are told
  to pass s3:///gs:// paths with no hint a backend is needed.

2025.3.0 rather than 2024.12.0: s3fs 2024.12.0 requires fsspec==2024.12.0, which
contradicts the floor. Extra names match NVIDIA-NeMo/Nemotron; no gs alias, same as there.
The [tool.uv] override at line 354 is left alone — it exists for nemo-toolkit[asr]==2.7.2 vs
data-designer-engine==0.5.5.

Evidence

Ran with uv 0.12.1 (required-version = ">=0.12.0").

Before, on clean main:

$ uv sync --locked --extra s3 --dry-run
Resolved 627 packages in 128ms
error: Extra `s3` is not defined in the `optional-dependencies` table for `nemo-curator`

Same for --extra gcs. uv lock --check passes on clean main, so the lock churn here comes
from this diff alone.

Lock regenerated in two steps, because plain uv lock takes the new s3fs/gcsfs but leaves fsspec
at 2026.3.0:

$ uv lock
Added gcsfs v2026.8.0
Updated s3fs v2024.12.0 -> v2026.7.0
$ uv lock --upgrade-package fsspec
Resolved 630 packages in 38.18s
Updated fsspec v2026.3.0 -> v2026.7.0
$ uv lock --check
Resolved 630 packages in 275ms

After:

$ uv sync --locked --extra s3 --dry-run
Resolved 630 packages in 9ms
 + fsspec==2026.7.0
 + s3fs==2026.7.0

$ uv sync --locked --extra gcs --dry-run
Resolved 630 packages in 57ms
 + gcsfs==2026.8.0

$ uv sync --locked --extra all --dry-run     # what install-test.yml runs
Resolved 630 packages in 89ms

627 → 630 packages: adds gcsfs 2026.8.0, google-auth-oauthlib, google-cloud-storage-control;
bumps s3fs, fsspec, and google-auth 2.45.0 → 2.57.1. uv.lock is +83/−15, not the ~900
lines estimated on the issue.

Limitations

• No test covers this. Nothing in the repo imports s3fs or gcsfs — they are reached only through
  fsspec (nemo_curator/utils/file_utils.py:54-58) — and no test exercises an s3:// path, so
  there is nothing a Python test could assert. The failing check above is the resolver.
• Whether s3fs 2024.12.0 → 2026.7.0 changes interleaved-stage behaviour is unverified;
  confirming it needs S3 credentials.
• This makes the s3fs/gcsfs/fsspec trio internally valid. The lock as a whole still relies on the
  fsspec override, which masks nemo-toolkit 2.7.2 (fsspec==2024.12.0) and datasets 4.0.0
  (fsspec[http]<=2025.3.0, uv.lock:2063). main already exceeded that cap at fsspec 2026.3.0;
  this widens it to 2026.7.0. Version-scoping that override is a separate change.
• The s3fs entry in the test dependency-group is dead — no test references s3fs, moto, or an
  s3:// URL. Bumped for consistency rather than deleted; happy to drop it if maintainers prefer.
• s3/gcs are not added to CPU_EXTRAS in install-test.yml; --extra all already covers
  them through the rollup.

Upstream issue: #1256

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[greptile-apps]

Greptile Summary

This PR adds supported cloud-storage installation extras and aligns their fsspec dependency floor.

• Adds s3 and gcs extras backed by s3fs and gcsfs.
• Includes both cloud backends in the all extra.
• Raises the project and development-group fsspec/s3fs floors to 2025.3.0.
• Regenerates the lockfile with updated cloud-storage dependencies.
• Documents the new extras and their corresponding URI schemes.

Confidence Score: 5/5

The PR appears safe to merge; no concrete correctness, compatibility, security, or documentation defect was established.

The extras are represented consistently in project metadata, lock metadata, and documentation, and the changed dependencies support the declared Python range. Existing broader fsspec override conflicts are not introduced by this PR.

Important Files Changed

Filename	Overview
pyproject.toml	Adds S3/GCS optional dependencies, rolls them into all, and aligns the fsspec and s3fs minimum versions.
uv.lock	Regenerates the resolved dependency graph for the new extras and upgraded storage packages without an established regression.
fern/versions/main/pages/get-started/installation.mdx	Documents installation of the S3 and GCS backends with the correct fsspec URI schemes.

_{Reviews (1): Last reviewed commit: "build(deps): add s3 and gcs extras and f..." | Re-trigger Greptile}