Fall back to backup universe files in live trading when the expected ones are unavailable

[jhonabreul]

Description

In live trading, file-based universe selection (option and futures chains, coarse fundamental, ETF constituents) depends on the daily universe file for the previous tradable date being on disk. If the data pipeline hasn't produced that file yet by the time the market is about to open, selection silently doesn't happen.

The change adds a safety net: a backup universe file (<universe file>.backup), when present, is used as a last resort:

• LiveCustomDataSubscriptionEnumeratorFactory owns the fallback (GetUniverseFileBackupSourceAdjustment), evaluated at the same rate-limited cadence as source refreshes and only for LocalFile sources. It is enabled with a new opt-in fallBackToBackupUniverseFiles constructor flag, off by default so custom data and custom universe subscriptions never probe backup files.
• LiveTradingDataFeed.CreateUniverseSubscription enables it for every universe served by the file-based branch: option chains, futures chains, fundamental/coarse and ETF constituents.
• When the market is open or within universe-file-backup-fallback-minutes (new config, default 30) of the next market open, if the expected universe file can't be fetched, the subscription falls back to <file>.backup (logged).
• The expected file is always preferred and re-checked on every refresh; far from market open no backup probing happens.

Related Issue

N/A

Motivation and Context

The daily chain universe files are written by a data pipeline shortly before the market opens. If a file is late or missing, live algorithms get no chain selection for the day. With this change, a previously generated backup file keeps selection working until the expected file shows up.

Requires Documentation Change

No

How Has This Been Tested?

• New LiveCustomDataSubscriptionEnumeratorFactoryTests: FallsBackToBackupUniverseFileWhenExpectedSourceIsNotAvailable (fallback fires, checks are rate limited like refreshes, and the expected file is preferred again once it becomes available), DoesNotFallBackToBackupUniverseFileFarFromMarketOpen and DoesNotFallBackToBackupUniverseFileWhenNotConfigured (no availability probing in either case).
• New LiveTradingDataFeedTests.UniverseSelectionFallsBackToBackupUniverseFileCloseToMarketOpen: 8 cases — equity option chain, index option chain, coarse fundamental and ETF constituents universes, each with the expected file available and missing. Selection succeeds from the backup only when the expected file is missing; the backup is never touched when the expected file exists.
• New LiveTradingDataFeedTests.ChainSelectionDoesNotFallBackToBackupUniverseFileFarFromMarketOpen: no backup probing outside the pre-open window (the gating is shared by all universe types).
• All 30 existing LiveChainSelection cases, the full LiveCustomDataSubscriptionEnumeratorFactoryTests fixture, and the *ImmediateSelection/FundamentalScheduleSelection tests pass; the new feed-level tests passed repeat runs with no flakiness.
• Live-paper deployment (local Launcher, FakeDataQueue, test-only shifted engine clock, fabricated GOOG universe files): deployed pre-open with only a backup file on disk, the backup was correctly not probed outside the window, then read on the first refresh inside it — the option filter selected all 2192 contracts from the backup, zero errors.
• Second live-paper deployment with both files on disk (backup trimmed to 20 contracts as a decoy) and a 1-minute refresh cadence: selection used the expected file (2192 contracts) immediately, and across ~6 refresh cycles the backup was never probed, zero errors.
• The same live-paper deployment pair for a coarse fundamental universe: with only a backup coarse file on disk, the first refresh outside the pre-open window correctly read nothing and the first refresh inside it fell back and selected from the backup; with both files on disk, selection used the expected file (7068 entries, not the 20-row decoy backup) within seconds of deployment and the backup was never probed.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• Refactor (non-breaking change which improves implementation)
• Performance (non-breaking change which improves performance. Please add associated performance test and results)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Non-functional change (xml comments/documentation/etc)

Checklist:

• My code follows the code style of this project.
• I have read the CONTRIBUTING document.
• I have added tests to cover my changes.
• All new and existing tests passed.
• My branch follows the naming convention bug-<issue#>-<description> or feature-<issue#>-<description>