fix: clean up inline-script environments after activation (#1794)

## Summary

Inline-script cache cleanup previously ran before the first environment
creation in a window. Users who did not create another environment could
keep old cached environments indefinitely, and cleanup could delay
unrelated interpreter lookups.

- Run one delayed activation sweep instead of triggering cleanup from
`create()`. Keep the existing 14-day expiration policy, reclaim
incomplete setups after a one-day grace period, and remove at most three
entries per sweep. There is no daily sweep.
- Keep candidate scans and cleanup-result publication outside the
interpreter-lookup barrier. Retain per-entry locks, physical-path
ownership checks, reference checks, and an under-lock age recheck before
deletion.
- Refresh cached environments' last-used timestamps on lookup. Keep
optional bookkeeping off the critical path, use fail-fast read-path
metadata writes, and withhold an old entry when its protection cannot be
established rather than return an unsafe last-known descriptor.
- Preserve the selected interpreter through temporary unavailability,
expose setup for an explicit retry, and publish recovery events after
bounded background revalidation.
- Preserve retry deadlines and budgets across saves that leave the
inline requirements unchanged. Cancel stale recovery when requirements
or the stored association change, and recheck selection after
asynchronous timestamp work.
- Recover dead-owner cache locks and reconcile the environment catalog
after deletion without discarding a replacement rebuilt at the same
path.
- Add regression coverage and document the cleanup and recovery
behavior.

The feature remains behind the existing
`python-envs.inlineScripts.enabled` flag. This does not add a worker
process, recurring cleanup, dependencies, or a new cache format.

## Validation

Lint, type checking, the full unit suite, and the production bundle were
repeated after integrating the latest `main` API-facade refactor and
resolving the import in the relocated `src/extensionApi.ts`. Obsolete
compiled files from the renamed modules were removed before
recompilation. Cache-deletion, save/recovery, and ordinary-API scenarios
were also rerun against the integrated implementation.

- `npm run lint`
- `npm run compile-tests`
- `npm run unittest -- --reporter=dot --no-colors`: 2,409 passing, 6
pending.
- Production webpack bundle.
- Independent Windows walkthroughs using real Python virtual
environments, cache metadata, and entry locks:
- The actual 96-126 second activation timer removed an old orphan
without calling the manager's `create()` and did not schedule another
sweep.
- Exactly three old orphaned entries were removed; referenced, recent,
live-locked, newer-schema, and redirected entries were preserved.
- A normal project `.venv` and the base Python installation remained
usable.
  - Another window's use after eviction planning prevented deletion.
- Temporary write failures and locks recovered with and without an
intervening same-requirements save.
- A pending timestamp write did not return a subsequently unset
interpreter.
- A healthy lookup completed while an unrelated cache inspection was
deliberately paused.
- HEAD/current shared-API comparisons retained ordinary-manager routing,
venv/conda timeout fallbacks, explicit non-inline overrides, and
neighboring-file isolation.

The walkthroughs adapt the editor/discovery-service boundaries; they are
not full live VS Code/Pylance GUI or macOS/Linux validation. Usage
timestamps are not process-lifetime leases.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Author: StellaHuang95

docs/managing-python-projects.md

@@ -113,6 +113,10 @@ Stop runs or debug sessions using the environment before deleting it. The extens
 
 On Windows, changing only the letter casing of a script's filename keeps its existing environment association. A rename does not validate unsaved dependency edits or install packages.
 
+Unused cached script environments are cleaned up once per window, about two minutes after the extension activates, rather than being triggered by environment creation. Cleanup considers entries unused for more than 14 days and incomplete setups older than one day, removes at most three entries, and skips environments referenced by this workspace or entries it cannot safely inspect. There is no daily sweep.
+
+The background cache scan does not hold up interpreter lookups. If another window briefly locks an entry, or its last-used time cannot be updated safely, the extension retries the script association in the background with bounded delays. Saving without changing the inline requirements does not cancel, postpone, or reset those retries; changing the requirements or stored inline-environment association cancels outdated recovery work. A temporarily unavailable selected environment keeps its association instead of silently switching execution to another interpreter, and the setup action is available for an explicit retry. If automatic recovery does not succeed, use that action to retry. Cleanup never installs packages.
+
 ## Assigning Environments to Projects
 
 Each project can have its own Python environment. This is the core benefit of project management.

src/common/inlineScript/cacheLayout.ts

@@ -3,6 +3,7 @@
 
 import * as crypto from 'crypto';
 import * as fsapi from 'fs-extra';
+import { rename as renameWithoutRetry } from 'fs/promises';
 import * as path from 'path';
 import { Uri } from 'vscode';
 import type { PythonEnvironment } from '../../api';
@@ -258,11 +259,22 @@ async function inspectMetaJsonFile(metaPath: string): Promise<InlineScriptMetaRe
  * hold the cache-entry file lock, which serializes this operation across
  * extension-host processes.
  */
-export function writeMetaJson(envDir: Uri, meta: InlineScriptEnvMeta): Promise<void> {
+export interface WriteMetaJsonOptions {
+    /**
+     * Fail immediately instead of letting graceful-fs retry a Windows sharing violation on the
+     * rename for a full minute. Read-path bookkeeping must never hold the shared cache-entry lock
+     * that long: other windows cannot tell it apart from a build or a deletion.
+     */
+    readonly failFast?: boolean;
+}
+
+export function writeMetaJson(envDir: Uri, meta: InlineScriptEnvMeta, options?: WriteMetaJsonOptions): Promise<void> {
     const finalPath = getMetaJsonPath(envDir).fsPath;
     const key = normalizePath(path.resolve(finalPath));
     const previous = pendingMetaJsonWrites.get(key) ?? Promise.resolve();
-    const operation = previous.catch(() => undefined).then(() => writeMetaJsonOnce(envDir, meta, finalPath));
+    const operation = previous
+        .catch(() => undefined)
+        .then(() => writeMetaJsonOnce(envDir, meta, finalPath, options?.failFast === true));
     let queued: Promise<void>;
     queued = operation.finally(() => {
         if (pendingMetaJsonWrites.get(key) === queued) {
@@ -273,7 +285,13 @@ export function writeMetaJson(envDir: Uri, meta: InlineScriptEnvMeta): Promise<v
     return queued;
 }
 
-async function writeMetaJsonOnce(envDir: Uri, meta: InlineScriptEnvMeta, finalPath: string): Promise<void> {
+async function writeMetaJsonOnce(
+    envDir: Uri,
+    meta: InlineScriptEnvMeta,
+    finalPath: string,
+    failFast: boolean,
+): Promise<void> {
+    const rename = failFast ? renameWithoutRetry : fsapi.rename;
     await fsapi.ensureDir(envDir.fsPath);
     const tmpSuffix = crypto.randomBytes(6).toString('hex');
     const tmpPath = `${finalPath}.tmp-${tmpSuffix}`;
@@ -285,18 +303,20 @@ async function writeMetaJsonOnce(envDir: Uri, meta: InlineScriptEnvMeta, finalPa
     try {
         await fsapi.writeFile(tmpPath, payload, 'utf8');
         try {
-            await fsapi.rename(tmpPath, finalPath);
+            await rename(tmpPath, finalPath);
             finalKnownToExist = true;
             return;
         } catch (err) {
             const code = (err as NodeJS.ErrnoException | undefined)?.code;
-            if (!['EPERM', 'EEXIST', 'EBUSY'].includes(code ?? '')) {
+            // Read-path bookkeeping must leave the old sidecar in place rather than enter
+            // the backup/restore path, whose recovery may itself need a retrying rename.
+            if (failFast || !['EPERM', 'EEXIST', 'EBUSY'].includes(code ?? '')) {
                 throw err;
             }
         }
 
         try {
-            await fsapi.rename(finalPath, backupPath);
+            await rename(finalPath, backupPath);
             hasBackup = true;
         } catch (err) {
             if (!isFileNotFoundError(err)) {
@@ -305,7 +325,7 @@ async function writeMetaJsonOnce(envDir: Uri, meta: InlineScriptEnvMeta, finalPa
         }
 
         try {
-            await fsapi.rename(tmpPath, finalPath);
+            await rename(tmpPath, finalPath);
             finalKnownToExist = true;
         } catch (replaceError) {
             if (hasBackup) {

src/common/inlineScript/routingRegistry.ts

@@ -41,6 +41,7 @@ interface ScriptRoutingState {
     readonly metadataIdentity?: string;
     readonly metadataRevision: number;
     readonly validatedAssociation: boolean;
+    readonly environmentUnavailable?: boolean;
 }
 
 export class InlineScriptRoutingRegistry implements Disposable {
@@ -49,6 +50,9 @@ export class InlineScriptRoutingRegistry implements Disposable {
     private readonly setupOutcomes = new Map<string, InlineScriptSetupOutcome>();
     private readonly _onDidChangeRouteability = new EventEmitter<InlineScriptRouteabilityChangeEvent>();
     private readonly _onDidChangeMetadata = new EventEmitter<InlineScriptMetadataChangeEvent>();
+    private readonly _onDidChangeAvailability = new EventEmitter<Uri>();
+
+    public readonly onDidChangeAvailability: Event<Uri> = this._onDidChangeAvailability.event;
 
     public readonly onDidChangeRouteability: Event<InlineScriptRouteabilityChangeEvent> =
         this._onDidChangeRouteability.event;
@@ -73,6 +77,8 @@ export class InlineScriptRoutingRegistry implements Disposable {
                     metadataRevision,
                     validatedAssociation:
                         state.metadataIdentity === metadataIdentity ? state.validatedAssociation : false,
+                    environmentUnavailable:
+                        state.metadataIdentity === metadataIdentity ? state.environmentUnavailable : false,
                 };
             },
             true,
@@ -129,9 +135,25 @@ export class InlineScriptRoutingRegistry implements Disposable {
             ...state,
             uri: script instanceof Uri ? script : state.uri,
             validatedAssociation,
+            environmentUnavailable: validatedAssociation ? state.environmentUnavailable : false,
         }));
     }
 
+    /** Keep temporary I/O failure separate from interpreter selection, while allowing setup to be retried. */
+    public setEnvironmentUnavailable(uri: Uri, unavailable: boolean): void {
+        const scriptPath = getInlineScriptRoutingKey(uri);
+        if (scriptPath) {
+            this.update(scriptPath, (state) => ({ ...state, uri, environmentUnavailable: unavailable }));
+        }
+    }
+
+    /** Whether a selected inline environment is temporarily withheld by a lookup. */
+    public isEnvironmentUnavailable(uri: Uri): boolean {
+        const scriptPath = getInlineScriptRoutingKey(uri);
+        const state = scriptPath ? this.states.get(scriptPath) : undefined;
+        return this.isRouteable(state) && state?.environmentUnavailable === true;
+    }
+
     public hasValidatedAssociation(script: Uri | string): boolean {
         const scriptPath = getInlineScriptRoutingKey(script);
         return scriptPath ? this.states.get(scriptPath)?.validatedAssociation === true : false;
@@ -177,6 +199,7 @@ export class InlineScriptRoutingRegistry implements Disposable {
         this.setupOutcomes.clear();
         this._onDidChangeMetadata.dispose();
         this._onDidChangeRouteability.dispose();
+        this._onDidChangeAvailability.dispose();
     }
 
     private update(
@@ -189,6 +212,7 @@ export class InlineScriptRoutingRegistry implements Disposable {
             validatedAssociation: false,
         };
         const previousRouteable = this.isRouteable(previous);
+        const previouslyUnavailable = previousRouteable && previous.environmentUnavailable === true;
         const next = updater(previous);
 
         if (!next.metadata && !next.validatedAssociation) {
@@ -214,6 +238,9 @@ export class InlineScriptRoutingRegistry implements Disposable {
                 routeable,
             });
         }
+        if (previouslyUnavailable !== (routeable && next.environmentUnavailable === true) && next.uri) {
+            this._onDidChangeAvailability.fire(next.uri);
+        }
     }
 
     private isRouteable(state: ScriptRoutingState | undefined): boolean {

src/extensionApi.ts

@@ -35,6 +35,7 @@ import type {
     SetEnvironmentScope,
 } from './types';
 import { PackageVersionLookupNotSupportedError } from './publicErrors';
+import { INLINE_SCRIPT_MANAGER_ID } from './common/constants';
 import { traceError, traceInfo } from './common/logging';
 import { pickEnvironmentManager } from './common/pickers/managers';
 import { timeout } from './common/utils/asyncUtils';
@@ -269,6 +270,13 @@ export class PythonEnvironmentApiImpl implements PythonEnvironmentApi {
         // Keep the background resolution alive so the cache/last-known value gets populated and the
         // change event fires once it finishes.
         resolution.catch((ex) => traceError('Failed to resolve environment in background', ex));
+        // Inline-script environments are reclaimed from a shared cache, and the manager withholds
+        // one it cannot prove is still safe. Serving the last-known value here would hand back the
+        // descriptor that decision just rejected, so only the timeout is skipped for them; every
+        // other manager keeps the fast fallback.
+        if (this.envManagers.getEnvironmentManager(currentScope)?.id === INLINE_SCRIPT_MANAGER_ID) {
+            return resolution;
+        }
         return this.envManagers.getLastKnownEnvironment(currentScope);
     }
     onDidChangeEnvironment: Event<DidChangeEnvironmentEventArgs> = this._onDidChangeEnvironment.event;

src/features/envManagers.ts

@@ -233,6 +233,13 @@ export class PythonEnvironmentManagers implements EnvironmentManagers {
                         traceError('Failed to refresh inline-script routing:', error),
                     );
                 }),
+                this.inlineScriptRouting.onDidChangeAvailability((uri) => {
+                    if (this.getEnvironmentManager(uri)?.id === INLINE_SCRIPT_MANAGER_ID) {
+                        void this.refreshEnvironment(uri, true).catch((error) =>
+                            traceError('Failed to refresh inline-script availability:', error),
+                        );
+                    }
+                }),
             );
         }
     }
@@ -891,7 +898,11 @@ export class PythonEnvironmentManagers implements EnvironmentManagers {
             return undefined;
         }
 
-        return manager.get(scope);
+        const environment = await manager.get(scope);
+        if (manager.id === INLINE_SCRIPT_MANAGER_ID && this.getEnvironmentManager(scope) !== manager) {
+            return this.getEnvironment(scope);
+        }
+        return environment;
     }
 
     /**
@@ -902,8 +913,9 @@ export class PythonEnvironmentManagers implements EnvironmentManagers {
      *
      * Unlike getEnvironment(), this IS a mutation — it updates internal state.
      * Unlike setEnvironment(), it does NOT call manager.set() or persist to settings.
+     * Availability recovery may republish an unchanged descriptor so consumers retry a failed lookup.
      */
-    async refreshEnvironment(scope: GetEnvironmentScope): Promise<void> {
+    async refreshEnvironment(scope: GetEnvironmentScope, notifyIfUnchanged = false): Promise<void> {
         const manager = this.getEnvironmentManager(scope);
         if (!manager) {
             return;
@@ -918,7 +930,10 @@ export class PythonEnvironmentManagers implements EnvironmentManagers {
         }
 
         const oldEnv = this._activeSelection.get(key);
-        if (this.isSameEnvironment(oldEnv, newEnv) || !this.commitSelectionOperation(key, operation)) {
+        if (
+            (this.isSameEnvironment(oldEnv, newEnv) && !notifyIfUnchanged) ||
+            !this.commitSelectionOperation(key, operation)
+        ) {
             return;
         }
         this._activeSelection.set(key, newEnv);

src/features/inlineScript/codeLens.ts

@@ -46,6 +46,7 @@ export class InlineScriptCodeLensProvider implements CodeLensProvider, Disposabl
     ) {
         this.subscriptions.push(
             this.routing.onDidChangeRouteability(() => this._onDidChangeCodeLenses.fire()),
+            this.routing.onDidChangeAvailability(() => this._onDidChangeCodeLenses.fire()),
             // Only metadata arriving or changing can add/replace a lens; a scan that finds no metadata
             // (the common case for ordinary .py files) needs no refresh. Hiding a lens for an
             // edited/removed block is handled by VS Code re-querying on the document change itself.
@@ -89,7 +90,7 @@ export class InlineScriptCodeLensProvider implements CodeLensProvider, Disposabl
         const offset = metadata.sourceRange?.start ?? metadata.range.start;
         const position = document.positionAt(offset);
         const range = new Range(position, position);
-        if (this.routing.shouldRoute(uri)) {
+        if (this.routing.shouldRoute(uri) && !this.routing.isEnvironmentUnavailable(uri)) {
             // A validated inline-script environment matching the current metadata already exists.
             const key = getInlineScriptRoutingKey(uri);
             const confirmation = key ? this.readyConfirmations.get(key) : undefined;

src/features/inlineScript/setupCodeAction.ts

@@ -92,7 +92,7 @@ export class InlineScriptSetupCodeActionProvider implements CodeActionProvider {
         if (!getInlineScriptRoutingKey(uri)) {
             return [];
         }
-        if (this.routing.shouldRoute(uri)) {
+        if (this.routing.shouldRoute(uri) && !this.routing.isEnvironmentUnavailable(uri)) {
             return [];
         }
         if (!readInlineScriptMetadata(sliceHeaderBytes(document.getText()), uri.fsPath)) {