Merge main into fix/issue-1885-default-interpreter-path-not-venv

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Author: eleanorjboyd

.github/instructions/testing-workflow.instructions.md

@@ -20,6 +20,7 @@ This guide covers the full testing lifecycle:
 
 - Pip commands that return JSON must pass `--disable-pip-version-check`; the process helper combines stderr with stdout, so update notices can otherwise make valid JSON unparseable (1).
 - When a view subscribes to a newly added provider event, TypeMoq-based view tests must return a real `EventEmitter.event`; an unstubbed event yields an undefined disposable and fails during teardown (1).
+- Test agent selections across a full reload with `python.defaultInterpreterPath` set. An effective manager value equal to the extension default does not prove a workspace value was saved; tool-owned persistence must inspect `workspaceValue` or startup can restore the global interpreter (1).
 
 ### When to Use This Guide
 

CONTRIBUTING.md

@@ -98,6 +98,152 @@ The npm package under [`api/`](./api) is the public API facade other extensions
 - `src/api.ts`, `src/types.ts`, and `src/publicErrors.ts` are validated on every PR by the extension's own lint and TypeScript compile.
 - **Versioning and compatibility:** the published package version in [`api/package.json`](./api/package.json) is maintained independently of the extension version in [`package.json`](./package.json) — the two do not need to match. Compatibility is based on the API shape exported by the installed Python Environments extension at runtime. Package updates must preserve backwards-compatible contracts unless the API package version intentionally communicates a breaking change; consumers should treat newly added members as optional when they may run against older installed extension versions. Any PR that edits `src/api.ts`, `src/types.ts`, or `src/publicErrors.ts` must bump `api/package.json` (use the `skip api version` label to bypass) and add an entry to [`api/CHANGELOG.md`](./api/CHANGELOG.md) (use the `skip api changelog` label to bypass).
 
+## Internal Python agent-tool bridge
+
+The extension's existing **flat** export also provides `__pythonTools.version === 1`.
+This is an internal convention, not a security boundary or a supported public API.
+Its types live only in `src/internal/pythonToolsApi.ts`; do not copy them into
+`src/types.ts`, `src/api.ts`, or the published API package. Consumers must check the
+version and all three methods at runtime before using the private API. The Python
+consumer uses its previous public Environments integration when that capability
+is absent or incompatible; that compatibility route can still prompt. It never
+falls back after a private operation starts, including errors, partial results,
+timeouts, or cancellation. Environments-disabled users retain the legacy Python
+route.
+Explicit compatibility selection persists through the public Environments setter, not
+just the Python extension's cached interpreter path, so later package operations
+use the same environment. Reusing an already-selected environment is read-only;
+it does not reselect or wait for an interpreter-change event.
+
+| Method | Request | Result |
+| --- | --- | --- |
+| `configureEnvironment(request, token)` | `{ resourcePath?: string, pythonPath?: string }` | Select an explicit interpreter exactly; otherwise reuse an isolated environment or create one for the project. Success includes `created`. |
+| `getEnvironment(request, token)` | `{ resourcePath?: string, includePackages?: boolean }` | Resolve the real selected environment, not the public API's short-timeout cache. `includePackages: true` returns `{ name, version?: string }[]`, including `[]`, or an error. |
+| `installPackages(request, token)` | `{ resourcePath?: string, packages: string[] }` | Install into an isolated environment. Does not create or change selection implicitly, or modify any global/base interpreter. |
+
+Automatic configuration uses isolation by default, regardless of whether a global
+interpreter was selected automatically or by a person. Virtual environments are
+recognized by `pyvenv.cfg`; non-base Conda and inline-script environments are also
+isolated. Existing isolated environments outside the workspace remain reusable.
+When no isolated environment is selected, configuration can resolve Poetry/Pipenv
+caches through the native finder's project association. It does not infer ownership
+from the cache location or record selection history; ambiguous project matches
+require an explicit `pythonPath`. Read-only queries do not select these candidates.
+There is no selection-origin bookkeeping or persistence. Old origin records are
+ignored; ordinary current-interpreter selection and its persistence are unchanged.
+An explicit `pythonPath` selects that interpreter without creating an environment.
+This can select global Python for execution or queries, but does not authorize
+package installation into it: `ENVIRONMENT_NOT_ISOLATED` directs the caller to
+configure without `pythonPath` or select an existing isolated environment.
+
+All methods require a VS Code `CancellationToken` and a trusted workspace.
+`resourcePath` accepts an absolute path or a `file:` URI within an open workspace.
+The Python tools default an omitted or empty path to the first workspace folder and
+pass it explicitly. They disclose the target in preparation/results and instruct agents
+to reuse the returned `resourcePath`. Editor focus does not change this default.
+At the direct private API boundary, omission selects the only workspace root;
+zero/multiple roots still produce actionable errors. The consumer default does not
+relax that backend validation.
+Other explicit input is never replaced by that default. Because the route is
+workspace scoped, read-only tools keep their previous Environments behaviour when
+no folder is open. Configuration and installation with a compatible private API
+report `NO_WORKSPACE` instead of opening a picker or modifying global Python.
+Nested projects and per-script selections retain their scope; a file is
+never used as an environment creation directory. There is no User/global settings
+fallback. An explicit `pythonPath` can switch away from an unavailable manager.
+New nested targets persist as exact project entries without changing root defaults;
+their automatic dependency discovery is restricted to that target directory.
+Tool selections at the workspace root persist an explicit workspace manager, even
+when it matches the extension default. This prevents `python.defaultInterpreterPath`
+from restoring global Python on reload without rewriting that interpreter setting.
+Ordinary human selections retain their existing settings-write behavior.
+
+Results are `{ status: 'success', environment, resourcePath?, created?, packages? }`
+or `{ status: 'error', code, message, environment?, resourcePath? }`. Errors retain
+an already-created environment when subsequent package installation or selection
+fails; retries reuse a resolvable `.venv`/`.conda` rather than creating a suffixed
+directory. Selecting an existing environment does not reinstall project
+dependencies. A resolvable `.venv`/`.conda` is only reused when it actually contains
+the resolved isolated environment. A new subdirectory can reuse an isolated parent-project
+selection; `resourcePath` chooses the target, not a requirement to create a fresh
+environment.
+Register an independent project when it needs a separate selection.
+Reads and writes for the same effective target share a cancellable
+queue; waiting behind an operation does not use the discovery timeout. Different
+targets remain independent. Discovery timeouts report `NOT_READY`; a configured
+environment manager that never registers reports `UNSUPPORTED_MANAGER` naming the id;
+subprocess
+timeouts report `TIMEOUT`. Cancellation throws `CancellationError` after owned
+process cleanup. Cancellation is cooperative, not a rollback: completed writes or
+package changes are not undone. A completed action remains successful if a
+cancellation request arrives on its return. If cleanup fails,
+`PROCESS_TERMINATION_FAILED` takes precedence:
+the process may still be modifying files, so consumers must not blindly retry.
+Cleanup waits for process closure and owned POSIX group completion; an exited
+Windows parent cannot turn a failed tree termination into successful cancellation.
+If a Windows parent has already exited when cancellation starts, descendant
+cleanup cannot be confirmed; the tool reports that uncertainty instead of
+signalling a potentially reused PID.
+Only internal `toolExecution` calls enable these subprocess policies. Public
+progress tokens do not enable agent timeouts, input suppression, or process-tree
+cleanup semantics. Public package calls retain their existing execution defaults.
+
+Internal symbol capabilities opt built-in managers into prompt-free operations;
+they are unrelated to the human UI's `quickCreateConfig`. Creation supports venv,
+Conda, and enabled PEP 723 inline scripts (the latter retain their existing cache
+ownership/metadata rules). System and Pyenv base interpreters route creation to
+venv; Conda base routes to an isolated Conda prefix. Poetry and Pipenv environments
+can be reused, but unsupported creation and contributed managers without this
+capability fail explicitly. Private package operations use the existing headless pip/uv, Conda, and
+Poetry helpers with the real token and strict, uncached package listing. Venv
+dependency validation fails rather than asking to continue. No tool installs a
+base Python automatically. Conda response descriptors clone `execInfo.activatedRun`
+to a standalone `conda run --prefix ... --no-capture-output ...` command; shared
+public descriptors are not changed.
+Venv creation uses the selected Python 3 as its base, falling back to the latest
+discovered Python 3 only when none is selected. Conda creation requests the selected
+Python major/minor version. This does not add a project version-constraint resolver
+for `pyproject.toml` or `.python-version`. To choose a particular base, select it
+first and then configure without `pythonPath`; the path parameter itself is an
+exact selection request, not a venv base-interpreter argument.
+Bare Conda commands configured through `python.condaPath` are resolved through
+PATH for private execution and returned command prefixes.
+Conda base is isolated even when inherited through `CONDA_PREFIX`, and failed
+Conda discovery does not authorize replacement creation. Public Conda refresh remains best-effort while
+the private readiness check retains the discovery error. Poetry verifies the
+requested project's actual environment prefix before package operations. Agent
+inventory uses `poetry run pip list --format=json`, including when Poetry supplies
+pip itself; the public lockfile-based `poetry show` path is unchanged.
+Private Poetry installs also apply the lockfile with `install --no-root`, since
+`add` alone skips already-declared dependencies missing from the environment.
+Direct installs into shared PEP 723
+caches return `IMMUTABLE_ENVIRONMENT`; edit script metadata and configure again.
+With the inline-script feature enabled, configuring a Python file with PEP 723
+metadata can create its cache on the first tool invocation, without first using a
+CodeLens or registering a script project manually. Invalid metadata fails rather
+than falling back to ordinary project creation. This opt-in configuration route
+does not change human or read-only routing, which still requires a validated
+association. Target the project directory or supply `pythonPath` for ordinary
+project/exact-interpreter configuration instead.
+An agent joining a human-owned inline cache build gets `ENVIRONMENT_BUSY` without
+waiting for or cancelling the human operation.
+
+Private discovery is separate from public initialization and its onboarding.
+Agent reads never start or await installation or missing-manager questions, even
+when a concurrent public initialization is waiting for input. Ordinary startup,
+public API, and human workflows retain their prompts; independently triggered
+startup UI can still appear while a tool runs. There is no global "suppress UI"
+switch. Exact nested-project persistence is also opt-in to the private setter;
+ordinary manager-setting updates retain their existing defaults.
+Human venv initialization retries discovery and base preparation after a failure,
+including failures after discovery has completed. A partial initialization is not
+reused as a successful cache on the next public lookup; private discovery still
+does not wait for human onboarding.
+
+Keep tests for both internal and human routes when changing these helpers. The
+focused suites are in `src/test/internal`, with shared creation/package and
+inline-script regressions under `src/test/managers`.
+
 ## Questions or Issues?
 
 - **Questions**: Start a [discussion](https://github.com/microsoft/vscode-python/discussions/categories/q-a)

README.md

@@ -130,7 +130,7 @@ All commands can be accessed via the Command Palette (`ctrl/cmd + Shift + P`):
 | pythonProjects               | `[]`                      | A list of Python workspaces, specified by the path, in which you can set particular environment and package managers. You can set information for a workspace as `[{"path":  "/path/to/workspace", "envManager": "ms-python.python:venv", "packageManager": "ms-python.python:pip"]}`.                                                                                                                                                                                                           |
 | terminal.showActivateButton  | `false`                   | (experimental) Show a button in the terminal to activate/deactivate the current environment for the terminal. This button is only shown if the active terminal is associated with a project that has an activatable environment.                                                                                                                                                                                                                                                                 |
 | terminal.autoActivationType  | `"command"`               | Specifies how the extension can activate an environment in a terminal. Accepted values: `command` (execute activation command in terminal), `shellStartup` (`terminal.integrated.shellIntegration.enabled` successfully enabled or we may modify shell startup scripts ), `off` (no auto-activation). Shell startup is only supported for: zsh, fish, pwsh, bash, cmd. **Takes precedence over** `python.terminal.activateEnvironment`. Restart terminals after changing this setting. To revert shell startup changes, run `Python Envs: Revert Shell Startup Script Changes`. |                                                                                        
-| alwaysUseUv                  | `true`                    | When `true`, [uv](https://github.com/astral-sh/uv) will be used to manage all virtual environments if available. When `false`, uv will only manage virtual environments explicitly created by uv. The extension prefers `uv` on its PATH; if unavailable in a trusted workspace, it also checks that workspace's `.pyprojectx/main/uv` (`uv.exe` on Windows) for environments inside that workspace. External environments still require uv on PATH. This does not alter the integrated terminal's PATH. |
+| alwaysUseUv                  | `true`                    | When `true`, [uv](https://github.com/astral-sh/uv) will be used to manage all virtual environments if available. When `false`, uv will only manage virtual environments explicitly created by uv. The extension uses `uv` on its PATH, or the executable it installed during the current session. If unavailable in a trusted workspace, it also checks that workspace's `.pyprojectx/main/uv` (`uv.exe` on Windows) for environments inside that workspace. External environments require uv on PATH or a current-session installation. This does not alter the integrated terminal's PATH. |
 | globalSearchPaths            | `[]`                      | Global search paths for Python environments. Array of absolute directory paths to search for environments at the user level. This setting is merged with the legacy `python.venvPath` and `python.venvFolders` settings.                                                                                                                                                                                                                                                                        |
 | workspaceSearchPaths         | `[]`                      | Workspace search paths for Python environments. Can be absolute paths or relative directory paths searched within the workspace.                                                                                                                                                                                                                                                                                                                                                                 |
 

package-lock.json

@@ -2,7 +2,7 @@
     "name": "vscode-python-envs",
     "displayName": "Python Environments",
     "description": "Provides a unified python environment experience",
-    "version": "1.39.0",
+    "version": "1.41.0",
     "publisher": "ms-python",
     "engines": {
         "vscode": "^1.110.0-20260204"

package.json

@@ -3,7 +3,7 @@
 
 import * as tomljs from '@iarna/toml';
 import * as fs from 'fs/promises';
-import { Uri } from 'vscode';
+import { l10n, Uri } from 'vscode';
 import { traceVerbose, traceWarn } from '../logging';
 import { PythonVersion } from '../pythonVersion';
 import { PythonVersionSpecifier } from '../pythonVersionSpecifier';
@@ -654,8 +654,13 @@ export function sliceHeaderBytes(text: string): string {
  *  - any I/O error (logged at `traceVerbose`);
  *  - any of the malformed-metadata cases handled by
  *    `readInlineScriptMetadata`.
+ * @param uri The local script to read.
+ * @param strict Throw on I/O errors or invalid metadata instead of treating them as absent.
  */
-export async function readInlineScriptMetadataFromFile(uri: Uri): Promise<InlineScriptMetadata | undefined> {
+export async function readInlineScriptMetadataFromFile(
+    uri: Uri,
+    strict = false,
+): Promise<InlineScriptMetadata | undefined> {
     if (uri.scheme !== 'file') {
         traceVerbose(`inline script metadata: skipping non-file URI scheme '${uri.scheme}'`);
         return undefined;
@@ -671,11 +676,22 @@ export async function readInlineScriptMetadataFromFile(uri: Uri): Promise<Inline
             await handle.close();
         }
     } catch (err) {
+        if (strict) {
+            throw err;
+        }
         traceVerbose(`inline script metadata: failed to read ${uri.fsPath}:`, err);
         return undefined;
     }
 
-    return readInlineScriptMetadata(text, uri.fsPath);
+    const result = parseInlineScriptMetadata(text, uri.fsPath);
+    if (
+        strict &&
+        (result.kind === 'invalid' ||
+            (result.kind === 'parsed' && result.problems.some((problem) => problem.severity === 'error')))
+    ) {
+        throw new Error(l10n.t('Fix the PEP 723 metadata in {0} before configuring its environment.', uri.fsPath));
+    }
+    return result.kind === 'parsed' ? result.metadata : undefined;
 }
 
 /**

src/common/inlineScript/metadata.ts

@@ -8,6 +8,7 @@ import * as path from 'path';
 export interface AcquireFileLockOptions {
     readonly timeoutMs: number;
     readonly retryIntervalMs: number;
+    readonly checkCancellation?: () => void;
 }
 
 export interface AcquiredFileLock {
@@ -52,6 +53,7 @@ export async function acquireFileLock(filePath: string, options: AcquireFileLock
     const deadline = Date.now() + options.timeoutMs;
 
     while (true) {
+        options.checkCancellation?.();
         try {
             await fsapi.mkdir(lockPath);
             try {