Skip to content

TESTING.md

Latest commit

 

History

History
114 lines (96 loc) · 11.4 KB

TESTING.md

File metadata and controls

114 lines (96 loc) · 11.4 KB
Raw

Testing Strategy & Fixture Backlog

This document tracks ThunderKit's automated test coverage: what exists, what's worth adding, and why. It is the living backlog for EditMode test fixtures.

Tests live in Tests/Editor/ and run via the Unity Test Runner (NUnit, EditMode). CI runs them across the Unity matrix defined in .github/workflows/main.yml.

What CI gives you, in two parts:

  1. Compile validation — running EditMode tests forces a clean compile of every assembly in the test assembly's dependency graph (ThunderKit.Common, ThunderKit.Markdown, ThunderKit.Core) on every Unity version. This catches version-specific compile breaks (e.g. the #if UNITY_6000_4_OR_NEWER EndNameEditAction → AssetCreationEndAction split). It does not compile assemblies outside that graph — notably ThunderKit.Addressable.*, ThunderKit.Thunderstore, ThunderKit.SpaceDock.
  2. Behavioral validation — only the fixtures below.

Conditional-compilation caveat: a #if UNITY_X branch is only compiled on a matrix version that defines UNITY_X. Branches gated on a Unity version absent from the matrix are never validated by CI.

Status legend: ✅ done · 🟡 partial · ⬜ not started

Current coverage (existing fixtures)

Fixture Covers Status
PackageSourceSettingsTests Package-source add/remove + Package Manager window sync ✅
PackageHelperTests GetCleanPackageName, GetStringHashUTF8, GetCleanedStringHashUTF8 ✅
PathExtensionsTests PathExtensions.Combine slash normalization ✅
PathReferenceTests ResolvePath no-token + unknown-token throws 🟡
PackageGroupTests HasString, version indexer, equality ✅
PackageVersionTests Equality / hashset dedup by dependencyId ✅
EnumExtensionsTests GetDescription attribute + fallback ✅
UnityPathUtilityTests IsAssetDirectory ✅
FileIdUtilTests Compute determinism / distinctness (MD4) ✅
ComposableObjectTests InsertElement / RemoveElement incl. type guard + index mismatch ✅
ConstantTests Constant path component ✅
ManifestNameTests ManifestName path component + null fallback ✅
SelfDestructingActionAssetTests Delegate invocation + self-destruct ✅
StageThunderstoreManifestTests RenderJson round-trip + dependency formatting ✅
ClassDataManagerTests TPK acquisition policy: PlanAcquisition coverage/throttle/download branches, SelectBestVersion closest-match selection, TryParseUnityVersion, IsThrottled, marker parse ✅
PlayerDataResolverTests PlayerDataResolver player-layout selection (loose globalgamemanagers vs data.unity3d), bundle entry lookup, and header version read, driven against a synthesized UnityFS bundle. Offline, no fixtures. ✅
ClassDataVersionCoverageTests Acquired tpk covers the running Unity version ([Explicit], network) 🟡
ImportProjectSettingsTests Real ImportProjectSettings.ExportProjectSettings over per-version globalgamemanagers fixtures + classdata.tpk: tpk version-resolution + YAML export on every matrix editor; core-setting correctness on the editor matching a fixture. Fixtures are generated by generate-project-settings-fixtures. Offline, runs in CI. 🟡

Backlog

Tier 1 — Pure logic, low effort, high value

Deterministic functions, no Editor state. Cheapest wins; run on every Unity version.

  • ✅ PackageHelper.GetCleanPackageName — PackageHelper.cs:31
  • ✅ PackageHelper.GetStringHashUTF8 / GetCleanedStringHashUTF8 — PackageHelper.cs:136
  • ✅ PathExtensions.Combine — PathExtensions.cs:8
  • 🟡 PathReference.ResolvePath — PathReference.cs:24. Has no-token + unknown-token-throws. Gap: multi-token, nested-token, and resolution-cache reuse cases.
  • ⬜ Manifest.EnumerateManifests — Manifest.cs:30. Assert dependency-graph dedup (HashSet), cycle tolerance (no stack overflow), and throw-on-null Identity/Dependencies/entry.
  • ⬜ VersionIdToGroupId — ThunderstoreSource.cs:81, LocalThunderstoreSource.cs:37. Author-Pkg-1.2.3→Author-Pkg, multi-dash names. Latent bug: input with no - → LastIndexOf("-") is -1 → Substring(0,-1) throws; pin the intended behavior. Logic is duplicated in two sources — candidate to hoist to the base class.
  • ✅ PackageGroup.HasString + version indexer — PackageGroup.cs:41
  • ✅ PackageVersion / PackageGroup equality contracts — PackageGroup.cs:79
  • ⬜ Pipeline.SupportsType — Pipeline.cs:301. [PipelineSupport] attribute routing, inheritance, non-job types → false.
  • ✅ EnumExtensions.GetDescription — EnumExtensions.cs:12
  • ✅ UnityPathUtility.IsAssetDirectory — Helpers/UnityPathUtility.cs
  • ✅ FileIdUtil.Compute / MD4 — FileIdUtil.cs

Tier 2 — Editor-coupled, high value (EditMode)

  • ✅ ComposableObject.InsertElement / RemoveElement — ComposableObject.cs:20
  • ✅ SelfDestructingActionAsset.Action + cleanup — SelfDestructingActionAsset.cs
  • 🟡 PathComponent leaf classes — Paths/Components/. Done: Constant, ManifestName. Gap: GamePath, ManifestVersion, OutputReference, Resolver, incl. ManifestIndex == -1 vs valid bounds.
  • ⬜ FlowPipelineJob.Execute manifest filtering — FlowPipelineJob.cs:16. PerManifest off→once; WhiteList/BlackList filtering; ManifestIndex set per-iteration then reset to -1.
  • ⬜ ImportConfiguration chain (LoadImportExtensions, ImportGame) — ImportConfiguration.cs:218. Executors ordered by Priority desc + dedup; index progression, disabled-skip, exception→index=-1, completion→Cleanup.
  • 🟡 ImportProjectSettings.ExportProjectSettings — ImportProjectSettings.cs. Covered by ImportProjectSettingsTests. Fixtures are produced by the generate-project-settings-fixtures workflow (one globalgamemanagers per matrix Unity version, built by that exact editor) and committed under Tests/Editor/Fixtures/GlobalGameManagers/<version>/. The test auto-discovers them; on the editor matching a fixture it asserts the core stable settings export, on every other editor it asserts resolution + no-crash. To populate / refresh: run the workflow from the Actions tab — no manual project or build work. Gap: the matching-editor assertion checks a small universal core set (AudioManager, TimeManager, TagManager, DynamicsManager, QualitySettings), not every version-specific setting; pin a fixture's own expectedSettings in its fixture.json to assert more. PlayerSettings/GraphicsSettings are excluded by design (layout shifts across patch releases).
  • ⬜ TransformHierarchyTreeView construction — TransformHierarchyTreeView.cs:21. Build in-memory GameObject tree; assert depth/parent/leaf + transformLookup id mapping. Also the only place CI would compile the PR #116 Addressables EntityId path — but only if an Addressables-referencing test assembly exists (see cross-cutting #2).
  • ⬜ Thunderstore / SpaceDock PackagesResponse parsing — PackagesResponse.cs, SpaceDock/PackagesResponse.cs. The array-wrap parse ({"results":[…]}) and empty-results case. Parsing is separable from the HTTP fetch — test the parser; skip the network.

Tier 3 — Valuable but needs a test seam first

Add internal + [assembly: InternalsVisibleTo("ThunderKit.Core.Tests")] to unlock these. This single refactor also removes the existing fixture's reflection hack.

  • ⬜ PackageSource.EnumerateDependencies (private) — PackageSource.cs:253. Recursive dependency resolution + cycle detection. High value (install correctness).
  • ⬜ PackageSource.LoadPackages dependency-map building / latest fallback.
  • 🛠️ Replace the reflection into private DeferredRefresh/RefreshList in PackageSourceSettingsFixture.cs:67 with the same seam — removes a latent break (renaming those privates currently breaks pre-2021 jobs).

Tier 4 — Defer to integration / skip

Filesystem-driven path components (FindFile/FindDirectory), pipeline jobs (Copy/Delete/ExecuteProcess), BatchModeExecutePipeline arg parsing, FixMisreferencedAssets (YAML + reflection), live network fetches, and global-state side-effect code (ScriptingSymbolManager, tree-view click handlers). HARD to isolate; only worth it as integration tests later.

Cross-cutting recommendations

  1. One InternalsVisibleTo seam unlocks all of Tier 3 and cleans up the existing fixture's reflection — highest-leverage single change. Done: [assembly: InternalsVisibleTo("ThunderKit.Core.Tests")] is now in Editor/Core/AssemblyInfo.cs (added for the ClassDataManager tests); the Tier 3 items below and the fixture reflection cleanup can now reuse it.
  2. A "compile-everything" test assembly referencing ThunderKit.Addressable.*, ThunderKit.Thunderstore, and ThunderKit.SpaceDock (even with one trivial [Test]) turns CI into a full compile gate. Today those assemblies are never compiled by CI, so a version-specific break in them (e.g. the PR #116 TransformHierarchyTreeView EntityId changes) would ship undetected.
  3. Re-enable code coverage once GameCI's coverage package stops failing to compile on newer Unity (it currently uses the obsolete non-generic TreeView API; see the coverageOptions: '' note in the workflow). Then gate coverage with assemblyFilters:+ThunderKit.* for a real coverage number.