Skip to content

FEAT: Add Python 3.15 Conda support - #839

Merged
Jahnvi Thakkar (jahnvi480) merged 30 commits into
mainfrom
jahnvi/conda-python-3-15
Oct 12, 2026
Merged

Jahnvi Thakkar (jahnvi480) merged 30 commits into
mainfrom
jahnvi/conda-python-3-15

Conversation

@jahnvi480

@jahnvi480 Jahnvi Thakkar (jahnvi480) commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Work Item / Issue Reference

AB#48877

GitHub Issue: N/A - related pull request linked below.


Summary

Add ordinary CPython 3.15 to the existing combined binding/ODBC/RS Conda package without removing older supported versions.

  • Extend build and release defaults to 34 variants: Python 3.10-3.15 on five subdirs and 3.12-3.15 on win-arm64. Free-threaded Python and musllinux remain outside the Conda matrix.
  • Use the target Windows site-packages directory for extraction and validation: lib/python/site-packages on 3.15, retaining Lib/site-packages on older Python.
  • Pin conda-build 26.9.1 only for Windows legs containing 3.15, because it reads the target Python's recorded layout. Other legs retain the existing <26 constraint.
  • Use microsoft, then conda-forge, for the win-arm64 3.15 build and verification dependencies. Preserve the exact older-version channel policies and the 3.12 ARM64 ABI workaround.

Source RS/ODBC versions, optional dependencies, OS floors and publisher tools are unchanged.

User-directed PR smoke follow-up: keep current-checkout source tests blocking, but obtain the real Linux CPython 3.11 smoke inputs directly from PyPI using fetch-wheels --published-smoke. Select PyPI's current published binding and its exact ODBC/RS dependencies, rather than unpublished source versions or independently mixing latest components. No smoke versions, filenames or SHA256 values are hardcoded. Downloads still require PyPI-provided hashes and actual METADATA/WHEEL/RECORD/dependency checks. Missing or mismatched inputs fail; no skip, older-release fallback or release-policy relaxation. The workflow supports both embedded-core and separate-RS published bindings without substituting source RS pins. This published packaging control does not qualify current-source RS or CPython 3.15. Production/default source-bound selection and trigger scoping are unchanged. The hardcoded digest literals reported by DevSkim have been removed, not suppressed.

Builds on merged #817 and now targets main.

Validation

.\myvenv\Scripts\python.exe -m pytest --noconftest -o addopts='' tests\test_027_conda_release_metadata.py tests\test_029_bundled_binary_audit.py tests\test_030_pe_machine_assert.py tests\test_033_driver_load_probe.py tests\test_034_conda_verify_cwd.py tests\test_035_conda_macho_assert.py tests\test_036_conda_provenance.py tests\test_037_rs_wheel_download.py tests\test_038_conda_archive_limits.py -q

Result: 1,388 passed on the initial Conda support commit. These are source-only packaging/tooling tests, not native driver or SQL integration tests.

.\myvenv\Scripts\python.exe -m pytest --noconftest -o addopts='' tests\test_036_conda_provenance.py tests\test_027_conda_release_metadata.py tests\test_034_conda_verify_cwd.py -q

Result: 1,073 passed on the PyPI-selection follow-up, covering explicit published/source selection, unavailable or invalid published dependencies, target restriction, embedded/separate RS and existing digest/dependency/provenance/trigger controls. Live read_published_versions() selected the current PyPI binding and its exact ODBC requirement successfully. Earlier local wheel-download attempts hit TLS handshake errors on Windows and a bounded Linux container; no TLS bypass was used. The normal post-push hosted conda-audit run validates actual wheel downloads and the build/audit for this revision.

.\myvenv\Scripts\python.exe -m pre_commit validate-config
.\myvenv\Scripts\python.exe -m pre_commit run black-check --all-files --hook-stage pre-push
git diff --check

Result: all passed. Normal commit and push hooks also passed.

Earlier bounded local experiments solved the full ordinary-3.15 runtime dependency closure on all six subdirs. Synthetic packaging with actual Conda interpreters and conda-build 26.9.1 passed for win-64 3.10/3.15 and cross-built win-arm64 3.15, including ABI, PE and installed-component checks.

Not end-to-end qualification: this host's NoDefaultCurrentDirectoryInExePath setting blocks conda-build's normal bare batch-file launch, including the production build_packages invocation. The synthetic packaging results used an explicit-path generated-batch launch followed by conda-build --post, without changing host policy or installed libraries. They do not establish normal pipeline success or native ARM64 execution. Approved-agent execution, real candidate wheels, native platform/SQL/TLS/authentication/bulk-copy and optional-feature qualification remain pending. No ADO pipeline was queued and nothing was published to Anaconda.

Add Python 3.15 RC2 preview legs on Windows, macOS, and Linux x64/ARM64 so the existing PR validation flow can expose product and packaging blockers before GA. Skip Arrow-only tests on 3.15 until PyArrow publishes compatible wheels.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
AB#48377

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Give each concurrent connect call its own configured native connection mock so MagicMock child creation cannot race during close.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
AB#48377

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Use the pinned PyArrow nightly wheel set and skip every non-3.15 pipeline leg so the validation result measures only 3.15 readiness.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Select compatible abi3 Rust wheels for newer CPython versions and include Python.h before libc-consuming standard headers.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Force Python.h ahead of libc headers in every native translation unit, align the GC stress assertion with Python 3.15 finalization ordering, and preserve focused-pipeline contracts.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Suppress Decimal conversion causes that can retain rejected input and make the cyclic-finalizer stress test explicitly release its synthetic self-cycle.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Keep Python 3.15 preview coverage additive while preserving every existing validation leg.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

# Conflicts:
#	mssql_python/pybind/CMakeLists.txt
#	tests/test_040_native_binary_hardening.py
….15-validation

# Conflicts:
#	eng/scripts/install-mssql-py-core.ps1
#	eng/scripts/install-mssql-py-core.sh
#	eng/versions/mssql-python-rs-nuget.version
#	eng/versions/mssql-python-rs.version
#	tests/test_038_mssql_odbc_daily_validation.py
Drop the stable-ABI wheel selection now provided by PR #804 and refresh the pinned Python 3.15 PyArrow nightly after the previous build was pruned.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Use RC3 on Windows and macOS while the official Linux Docker image remains on RC2.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Use Python 3.15 GA on hosted agents and stable PyArrow 26.0.0, while retaining RC3 Linux images until Docker publishes GA.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Preserve isolated-wheel skips, fail container setup immediately, align Decimal privacy documentation, and pin stable PyArrow 26.0.0.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pin PyArrow 25.0.1 below Python 3.15 and PyArrow 26.0.0 on Python 3.15+, keeping the existing 3.10 release locks satisfiable.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Remove the native-hardening and auth-test changes owned by PR #810, retaining only the Python 3.15 header-order compatibility flag and its focused contract.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add the marker-scoped PyArrow 26.0.0 hashes while preserving PyArrow 25.0.1 for older Python versions.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Remove the three added source-text tests, assert suppressed Decimal causes in the existing test, and propagate container setup failures.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
### Work Item / Issue Reference

> #817

-------------------------------------------------------------------
### Summary

Extends the governed build and release pipeline to produce Python 3.15
wheels without changing Conda packaging.

- Adds Python 3.15 x64 and ARM64 Windows stages.
- Adds a Python 3.15 macOS universal2 stage.
- Adds `cp315` to all manylinux and musllinux x64/ARM64 build loops.
- Extends release-lock compatibility validation to Python 3.15, allowing
source builds where upstream wheels are unavailable.
- Raises the consolidated wheel completeness gate from 34 to 41 wheels.

### Validation

- YAML and whitespace validation passed.
- Hosted universal-lock generation completed successfully for Linux,
macOS, Windows, and ODBC.
- The full build-release pipeline is the validation for native wheel
production, tests, signing, and artifact completeness.

### Scope

This PR covers PyPI wheel production only. Conda Python 3.15 packaging
is intentionally excluded.

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Raise the metadata-only error after leaving the conversion handler, and assert absent context in the three existing privacy tests.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Retain existing supported variants while adding ordinary CPython 3.15 across the six Conda subdirs. Align Windows layout, scoped builder selection and ARM64 channels, with focused packaging regressions.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@github-actions github-actions Bot added the pr-size: medium Moderate update size label Oct 12, 2026
Keep current-checkout source tests blocking while selecting an explicit Linux CPython 3.11 binding/ODBC baseline independent of unpublished source releases. Reuse strict PyPI metadata and dependency validation and additionally enforce reviewed wheel filenames and SHA256 pins. Production and release source pins are unchanged.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@github-actions github-actions Bot removed the pr-size: medium Moderate update size label Oct 12, 2026
@github-actions github-actions Bot added the pr-size: large Substantial code update label Oct 12, 2026
@jahnvi480
Jahnvi Thakkar (jahnvi480) changed the base branch from bewithgaurav/python-3.15-validation to main October 12, 2026 08:49
@github-actions

github-actions Bot commented Oct 12, 2026 •

Copy link
Copy Markdown

PR Performance Report

✅ No regression detected

No consistent slowdowns detected across all 2 environments.

0 IMPROVEMENTS 0 SLOWDOWNS 2/2 ENVIRONMENTS

Coverage: 2 of 2 environments completed. Advisory result; does not block merging.

Performance diagnostics

Phase times are inclusive diagnostics and must not be added together. They identify where measured time changed, not why it changed.

No affected phases or call-count changes were recorded.

All database tasks and timings

Unix / SQL Server 2022

Database task Before After Paired change Result
Connection opening 10.912 ms 11.065 ms -0.4% no signal
SELECT queries 1.088 ms 1.056 ms -2.8% no signal
Row insertion 31.687 ms 32.233 ms +1.7% no signal
Executemany inserts 138.048 ms 136.918 ms +0.1% no signal
Fetch-all queries 120.171 ms 119.347 ms -0.5% no signal
Row-by-row fetching 12.581 ms 12.705 ms +0.7% no signal
Batched row fetching 111.546 ms 112.009 ms -0.3% no signal
Transaction commit and rollback 100.784 ms 99.837 ms -1.0% no signal
Arrow row fetching 90.937 ms 90.728 ms -0.3% no signal
100,000-row insertion 404.912 ms 403.543 ms +0.0% no signal
Row fetching in batches of 100 107.868 ms 108.291 ms +0.6% no signal
Row fetching in batches of 10,000 128.140 ms 137.764 ms +7.5% no signal
Repeated positional queries 30.125 ms 30.497 ms -0.2% no signal
Repeated named-parameter queries 32.551 ms 32.643 ms +0.3% no signal
Legacy 100,000-row insertion 313.512 ms 311.508 ms +0.6% no signal
Insertion with explicit input sizes 453.638 ms 452.435 ms -5.1% no signal
Joined aggregation queries 183.203 ms 185.427 ms -0.3% no signal
Large joined-result fetching 185.830 ms 186.871 ms -0.4% no signal
1.2-million-row fetching 3484.803 ms 3475.001 ms -0.2% no signal
Common table expression queries 5.665 ms 5.628 ms -1.1% no signal
256 KiB VARCHAR(MAX) / fetchall() 1.272 ms 1.273 ms -4.0% no signal
10,000 scalar values / fetchval() (debug disabled) 101.077 ms 101.351 ms -0.2% no signal

Unix / SQL Server 2025

Database task Before After Paired change Result
Connection opening 95.455 ms 95.349 ms -0.2% no signal
SELECT queries 1.024 ms 1.029 ms -0.4% no signal
Row insertion 34.959 ms 33.956 ms -3.6% no signal
Executemany inserts 148.340 ms 149.790 ms +0.5% no signal
Fetch-all queries 117.953 ms 117.946 ms -0.0% no signal
Row-by-row fetching 14.367 ms 14.084 ms -2.3% no signal
Batched row fetching 114.188 ms 114.515 ms +0.4% no signal
Transaction commit and rollback 114.460 ms 112.828 ms -1.3% no signal
Arrow row fetching 91.873 ms 92.110 ms +0.9% no signal
100,000-row insertion 432.209 ms 447.266 ms -1.1% no signal
Row fetching in batches of 100 118.332 ms 120.768 ms +1.6% no signal
Row fetching in batches of 10,000 128.451 ms 132.553 ms +9.6% no signal
Repeated positional queries 33.182 ms 33.202 ms -0.2% no signal
Repeated named-parameter queries 36.021 ms 35.642 ms -2.1% no signal
Legacy 100,000-row insertion 337.899 ms 338.119 ms -0.1% no signal
Insertion with explicit input sizes 508.538 ms 478.420 ms -5.6% no signal
Joined aggregation queries 157.835 ms 161.747 ms +3.6% no signal
Large joined-result fetching 172.408 ms 173.072 ms +0.1% no signal
1.2-million-row fetching 3460.218 ms 3468.385 ms +0.3% no signal
Common table expression queries 5.029 ms 5.081 ms +1.0% no signal
256 KiB VARCHAR(MAX) / fetchall() 1.481 ms 1.507 ms +1.1% no signal
10,000 scalar values / fetchval() (debug disabled) 106.595 ms 106.192 ms +0.0% no signal
Build and measurement details

ADO build 182163

PR head: db5a336129007e3e333965a98955be1f8a22c3d1
Base: 2eb7929d6dea9cecd9cd36c334394872e623fedd
Measured merge: cb76f6c42aaa642174a5f9fb1d06abf1c08c23b5

  • Unix / SQL Server 2022: Python 3.12.3, x86_64, SQL 16.0.4295.3; 5 paired comparisons and 1 warmup.
  • Unix / SQL Server 2025: Python 3.12.3, x86_64, SQL 17.0.5005.3; 5 paired comparisons and 1 warmup.

A consistent change requires more than 20% median paired movement, at least 1 ms between the median runtimes, and at least 80% of pairs exceeding the relative threshold in the same direction. A slowdown without enough pair agreement is reported as inconsistent.

The displayed change is the median of paired before-and-after ratios. It is not recalculated from the two displayed median runtimes.

Both revisions use profiling-enabled builds on the same agent and database, with alternating order and discarded warmups. Results are diagnostic and do not represent production-wheel latency.

Raw samples and logs are attached to the ADO run as profiler-* artifacts.

Comment thread eng/conda_tools/inputs.py Fixed
Comment thread eng/conda_tools/inputs.py Fixed
Remove hardcoded smoke versions and SHA256 literals. Select the current published binding and its exact ODBC/RS requirements from PyPI while retaining runtime PyPI digest and wheel metadata validation. Keep production source-bound inputs unchanged.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 12, 2026 08:55

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Cross-platform packaging, channel resolution, and native ARM64 behavior still require approved-agent validation.

0 open findings

What changed in this PR

Adds ordinary CPython 3.15 to the Conda packaging matrix while preserving existing versions and published-wheel smoke validation.

Changes:

  • Expands Conda builds and release validation to 34 variants.
  • Handles Python 3.15’s Windows site-packages layout and builder/channel requirements.
  • Makes the Linux smoke test use the current published binding and exact dependencies.
File Description
.github/​workflows/​conda-audit.yml Uses published CPython 3.11 smoke inputs.
OneBranchPipelines/​conda-build-pipeline.yml Adds 3.15 build variants.
OneBranchPipelines/​jobs/​consolidate-conda-artifacts-job.yml Updates expected package counts.
OneBranchPipelines/​steps/​conda-build-validate-step.yml Documents the expanded Windows matrix.
OneBranchPipelines/​steps/​conda-publish-step.yml Publishes against the 3.15-inclusive matrix.
OneBranchPipelines/​steps/​conda-release-step.yml Validates the expanded release matrix.
conda/​README.md Documents 3.15 packaging and smoke policy.
conda/​mssql-python/​bld.bat Uses the target-provided site-packages path.
conda/​mssql-python/​meta.yaml Updates the supported build range.
eng/​conda_tools/​build.py Selects channels and builder by target version.
eng/​conda_tools/​contracts.py Defines target-specific site-packages roots.
eng/​conda_tools/​environment.py Pins conda-build 26.9.1 for Windows 3.15.
eng/​conda_tools/​inputs.py Adds published-smoke dependency selection.
eng/​conda_tools/​release.py Expands default release policy to 3.15.
eng/​conda_tools/​verify.py Uses conda-forge for win-arm64 3.15.
tests/​test_027_conda_release_metadata.py Tests matrix defaults and package counts.
tests/​test_030_pe_machine_assert.py Tests the new Windows package layout.
tests/​test_034_conda_verify_cwd.py Tests ABI, channels, and builder selection.
tests/​test_036_conda_provenance.py Tests published smoke and 3.15 provenance.

🧠 Review effort: Balanced


💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@github-actions

Copy link
Copy Markdown

📊 Code Coverage Report

🔥 Diff Coverage

100%


🎯 Overall Coverage

85%


📈 Total Lines Covered: 9443 out of 11092
📁 Project: mssql-python


Diff Coverage

Diff: main...HEAD, staged and unstaged changes

No lines with coverage information in this diff.


📋 Files Needing Attention

📉 Files with overall lowest coverage (click to expand)
mssql_python.pybind.performance_counter.hpp: 0.7%
mssql_python.pybind.logger_bridge.cpp: 57.9%
mssql_python.pybind.ddbc_bindings.h: 62.6%
mssql_python.pybind.logger_bridge.hpp: 70.8%
mssql_python.pybind.connection.connection_pool.cpp: 82.3%
mssql_python.pybind.connection.connection.cpp: 83.1%
mssql_python.logging.py: 86.2%
mssql_python.pooling.py: 90.1%
mssql_python.pybind.fetch_temporal.hpp: 92.1%
mssql_python.cursor.py: 92.5%

🔗 Quick Links

⚙️ Build Summary 📋 Coverage Details

View Azure DevOps Build

Browse Full Coverage Report

@jahnvi480
Jahnvi Thakkar (jahnvi480) merged commit acdb42c into main Oct 12, 2026
38 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-size: large Substantial code update

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants