chore(next): carry #1676 and #1677 for the next build, do not merge by Moep90 · Pull Request #1678 · kapicorp/kapitan

Moep90 · 2026-11-07T06:58:40Z

This branch only exists for the next build (#1637). Please review #1676 and #1677 instead, not this PR. I'll close it once those two are merged.

next merges open PRs in number order and leaves out the ones that conflict. #1676 and #1677 conflict with earlier open PRs there, so this branch carries them already resolved:

feat(fetch): reload the inventory after fetching, so fetched classes take effect #1676 against Fix #1379: Handle missing parameters.kapitan.compile gracefully #1388, feat: preserve exception chains and improve reclass error messages #1486, feat: discover generator-provided schema.json for dependency validation #1488, refactor(cached): drop orphaned KMS globals + trim worker payload (#1510 tail, PR-4+5) #1531, feat: add --path-traversal-mode to warn or fail when file accessors leave their search paths #1566 and docs: fix stale dependency, topics and omegaconf notes; validate links #1634, with the resolutions from feat(fetch): reload the inventory after fetching, so fetched classes take effect #1641
feat(omegaconf): confine from_file to the project #1677 against fix(omegaconf): fail clearly on the unsupported filename and path resolvers #1654, keeping both resolver changes
a test fix for fix(compile): report an inventory without targets #1656: the mock in test_compile_targets_logs_error_without_targets now accepts the keyword arguments the reload passes to get_inventory

I replayed the next assembly locally with the current open PRs. #1676 and #1677 are left out as conflicting, this PR merges cleanly after them, and the full suite passes on the result (971 passed, 4 skipped).

- Skip targets without compile configuration instead of raising FileNotFoundError - Add validation in load_target_inventory() to check for compile key - Add safety check before copying temp compile path - Log informational message when no compilable targets found - Update coverage requirement from 63% to 65% Tests: - Add test for empty parameters.kapitan - Add test for missing compile key - Add test for mixed valid/invalid targets Fixes: #1379

Improve error reporting when the reclass or reclass-rs inventory backend fails by: - Using `raise InventoryError(...) from e` in both backends so the original exception (and its traceback) is preserved. - Removing the `NotFoundError` special-case in the reclass backend that replaced specific errors with a generic "inventory not found" message. - Logging `InventoryError` in `get_inventory()` before `sys.exit(1)` so users see the actual error instead of a silent exit. - Catching `KapitanError` explicitly in `trigger_compile()` and printing the message to stderr before exiting. - Adding `from e` to the `CompileError` wrapper in `compile_targets()`. Fixes #1172

Adds pin-down tests covering behaviors that are currently untested but will be touched by the planned cached.py global-state refactor (issue #1510), so a behavior change cannot ship silently: - parallel==serial golden compile (--parallelism 4 vs 1) across all three inventory backends, exercising the multiprocessing worker-seeding path - null_presenter / --yaml-dump-null-as-empty output (characterized, not fixed) - gkms/awskms/azkms accessor construct-once + memoize contract (SDKs mocked) - resources.inventory() -> cached.args compose_target_name wiring seam No production code changes.

Address review feedback: skipping targets without kapitan/kapitan.compile config is now visible at info level so a forgotten class (e.g. kapitan.common) isn't silently masked.

- Restore typo protection: requesting an unknown target via -t now errors with CompileError instead of silently skipping and exiting 0. - Fix 5 test_targets.py tests broken by the merge with #1502: MockTarget now carries a compile config so valid targets load; rewrite the empty- kapitan test to assert skip-not-raise, add a missing-compile skip test. - Add a CLI test that an unknown -t target exits non-zero. - Drop redundant hasattr() guard (compile is a guaranteed pydantic field) and an always-true target_objs check after the early return.

Use the isolated_compile_dir fixture and create_test_inventory() helper instead of hand-rolled tempfile + os.chdir + inline YAML, per CLAUDE.md testing guidelines (CompileTestHelper / fixtures, no manual chdir).

fix(schema-validation): address review feedback for generator schema validation

- Restore unrelated OCI test test_nested_artifact_without_subpath_emits_warning that was deleted; its code path in base.py is still live. - Replace manual os.chdir/try-finally in compile e2e tests with monkeypatch.chdir, and parametrize them over all inventory backends (reclass, reclass-rs, omegaconf), skipping unavailable backends. - Make SourceTrackerCache public since targets.py imports it across modules.

…ached globals Moves the memoization of the gkms/awskms/azkms secret-handler clients off the kapitan.cached module globals onto module-local @functools.cache factories (first step of #1510, decoupling secret handlers from the global state bag). - gkms_obj/awskms_obj/azkms_obj now delegate to a memoized _*_client() factory; azkms is keyed on key_id so distinct keys get distinct clients. - cached.{gkms,awskms,azkms}_obj are kept as backward-compatible mirrors so pool seeding and serialization (cached.as_dict/from_dict) are unaffected; they become vestigial and are removed in a later cleanup step. - reset_cache() flushes the new factory caches via a small clearer registry, preserving the previous "reset clears handlers" behavior and preventing cross-run client reuse. gpg_obj is intentionally left untouched (its *args/**kwargs first-wins semantics need separate care) and follows in a later step. No call-site or public-signature changes.

…to cached.args Part of #1510: lets get_inventory be driven by explicit arguments instead of reading the parsed CLI args out of the global cache. - get_inventory gains keyword-only params backend_id, compose_target_name, migrate, enable_class_wildcards. When None (the default), each falls back to kapitan.cached.args, so every existing call site keeps working unchanged. - Library callers can now load an inventory without populating cached.args. - Drops a redundant duplicate read of compose_target_name; the --compose-node-name deprecation warning now only fires on the cached.args fallback path (the explicit API does not carry the deprecated flag). cached.inv / cached.global_inv writes are retained as the inventory store (removed in a later cleanup step). Scoped to inventory *loading*; the cached.inv *access* in kadet/inputs is a separate follow-up. Adds tests/test_get_inventory_explicit.py proving the explicit path drives backend + compose selection with an empty cached.args (reclass + reclass-rs).

… without globals Part of #1510: decouples two leaf helpers from kapitan.cached so they can be unit-tested without mutating global state. - jinja2_filters: add make_reveal_maybe(reveal, revealer) factory returning a closure bound to explicit inputs; reveal_maybe(ref_tag) becomes a thin cache-reading shim over it, so the registered filter and its behavior are unchanged. - utils.null_presenter: read the flag from the dumper instance (dumper.yaml_dump_null_as_empty) when set, falling back to cached.args otherwise. Setting the attribute on a dumper makes the representer testable without touching the global cache. Both global reads are retained as fallbacks (removed in a later cleanup step); no call-site or public-signature changes. Adds tests/test_helpers_explicit.py exercising the explicit, global-free paths.

Jsonnet's import and file_read natives, the jinja2 render callback, the yaml and directory natives, and input glob expansion all resolve a caller-supplied name against search paths with os.path.join. A name containing ".." or a symlink can resolve outside the inventory and read an unrelated file such as an SSH key, and nothing flagged it. Add warn_on_path_traversal in utils: it compares the resolved realpath against the allowed roots and logs a warning when the path escapes all of them. Wire it into the resource native callbacks and the input path loop. The check is warn-only, so the read still succeeds and no existing setup breaks; the escape is now visible in the logs.

Add --path-traversal-mode, also settable in the .kapitan compile section, so a resolved path escaping its search paths can warn (default), fail the compile, or be ignored. check_path_traversal now honours the mode; workers re-apply it in compile_target so it survives the spawn start method. Memoise realpath of the roots since they repeat across a compile. Document the option on the compile command reference, and cover the error, off and invalid modes plus the search_imports multi-root wiring in tests.

MagicMock args returned a Mock for path_traversal_mode, which failed validation in set_path_traversal_mode (expected warn/error/off). Set it explicitly so the worker re-apply step gets a real value.

…ventory Stacks #1641 on #1531, which contains #1530. get_inventory() keeps partial as a positional parameter before the keyword-only flags of #1530. Signed-off-by: Moep90 <3042152+Moep90@users.noreply.github.com>

…nventory Stacks #1641 on #1566. Only the kapitan.utils import conflicted. Signed-off-by: Moep90 <3042152+Moep90@users.noreply.github.com>

…reload-inventory Stacks #1641 on #1634. The new "Fetching classes" section stays, the stale "Caching" section after it goes, as #1634 intends. Signed-off-by: Moep90 <3042152+Moep90@users.noreply.github.com>

…olvers `${filename:}`, `${path:}`, `${parent_filename:}` and `${parent_path:}` read OmegaConf node flags that nothing sets, so they always resolved to null. They came from the omegaconf backend proposal that set string flags through a patched OmegaConf fork. Upstream OmegaConf only allows boolean flags: `_get_flag()` asserts on other types and merges fail on string flags. The four resolvers now raise an InventoryError that names the resolver, and the docs no longer list them. Signed-off-by: Moep90 <3042152+Moep90@users.noreply.github.com>

…take effect kapitan compile --fetch now loads the inventory partially, fetches, drops the cached inventory and reloads, until a round adds no new or changed dependency (at most 5 rounds). A partial load skips missing classes and references that don't resolve yet and keeps only the dependency entries and labels. The last load is a full one and reports errors as before. Plain kapitan compile does the same after fetching force_fetch entries. Fetched classes are limited: dependencies declared in a class fetched into inventory/classes must write below that class's output_path, and two entries fetching different sources into one output_path fail. Fixes #1638 Signed-off-by: Moep90 <3042152+Moep90@users.noreply.github.com>

${from_file:...} now only reads a file that resolves, after following symlinks, below the working directory or the inventory path. Any other path fails the inventory render with an error naming the path. A custom from_file in resolvers.py still replaces the built-in one for projects that read outside on purpose. Fixes #1639 Signed-off-by: Moep90 <3042152+Moep90@users.noreply.github.com>

Signed-off-by: Moep90 <3042152+Moep90@users.noreply.github.com> #1676 is the reload commit of #1641 on master. This branch already has it, resolved against the stacked PRs, so the merge keeps that resolution.

Signed-off-by: Moep90 <3042152+Moep90@users.noreply.github.com>

Signed-off-by: Moep90 <3042152+Moep90@users.noreply.github.com> # Conflicts: # kapitan/inventory/backends/omegaconf/resolvers.py

Signed-off-by: Moep90 <3042152+Moep90@users.noreply.github.com>

…uments With the reload after fetching (#1676), compile_targets calls get_inventory with ignore_class_not_found and partial. The mock from #1656 only took the path, so the test failed once both are in. Signed-off-by: Moep90 <3042152+Moep90@users.noreply.github.com>

github-actions · 2026-11-07T06:58:56Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

Merged: kapicorp#1388 kapicorp#1440 kapicorp#1446 kapicorp#1449 kapicorp#1459 kapicorp#1465 kapicorp#1484 kapicorp#1486 kapicorp#1488 kapicorp#1497 kapicorp#1501 kapicorp#1516 kapicorp#1518 kapicorp#1527 kapicorp#1528 kapicorp#1529 kapicorp#1530 kapicorp#1531 kapicorp#1539 kapicorp#1540 kapicorp#1541 kapicorp#1542 kapicorp#1543 kapicorp#1544 kapicorp#1545 kapicorp#1546 kapicorp#1547 kapicorp#1554 kapicorp#1555 kapicorp#1559 kapicorp#1565 kapicorp#1566 kapicorp#1598 kapicorp#1600 kapicorp#1606 kapicorp#1609 kapicorp#1612 kapicorp#1614 kapicorp#1615 kapicorp#1623 kapicorp#1624 kapicorp#1625 kapicorp#1627 kapicorp#1629 kapicorp#1631 kapicorp#1633 kapicorp#1634 kapicorp#1635 kapicorp#1636 kapicorp#1637 kapicorp#1640 kapicorp#1642 kapicorp#1643 kapicorp#1646 kapicorp#1648 kapicorp#1650 kapicorp#1652 kapicorp#1654 kapicorp#1656 kapicorp#1659 kapicorp#1660 kapicorp#1668 kapicorp#1669 kapicorp#1670 kapicorp#1671 kapicorp#1674 kapicorp#1678 Conflicting: kapicorp#1658 kapicorp#1662 kapicorp#1664 kapicorp#1666 kapicorp#1676 kapicorp#1677 Next-Fingerprint: f03513f 169c870 1388 e047547 1440 624bbb4 1446 8600881 1449 916c63d 1459 519a90b 1465 7f6b85b 1484 2543d11 1486 6a7be30 1488 9991a7c 1497 96cdde6 1501 2706f7e 1516 2485def 1518 24bb6ad 1527 2f4e188 1528 02e2e25 1529 bb4fa93 1530 d499bb7 1531 6e6afff 1539 ee8bb7b 1540 84c0929 1541 46d0f7a 1542 4a4d562 1543 5cd2a5c 1544 d27945f 1545 ee6e895 1546 0902bbc 1547 094c999 1554 ccb0486 1555 f5cbd07 1559 79d42fa 1565 07d2b6d 1566 7ec811a 1598 570ab14 1600 5cf9875 1606 942fdeb 1609 1dccc5e 1612 9cd6b5c 1614 782afcb 1615 0beed2b 1623 9e88063 1624 872197a 1625 9267812 1627 531002f 1629 75ae30a 1631 5ef2dca 1633 4af5f87 1634 822a175 1635 7327e46 1636 b6fb808 1637 1c2d6dc 1640 b909052 1642 ea824d8 1643 67c0727 1646 5a60b7f 1648 a126868 1650 c970796 1652 8314139 1654 7f1adca 1656 a174d8e 1658 a9ec63f 1659 a59f187 1660 28869c7 1662 235ae8b 1664 929dec3 1666 4a23d70 1668 2f07d58 1669 991f82e 1670 9eb680b 1671 d761ef1 1674 2d40202 1676 f835258 1677 3bbaedf 1678 a14e904

Moep90 and others added 30 commits January 17, 2026 14:57

fix: lower coverage fail_under threshold to 64

31af33b

Merge branch 'master' into fix/issue-1379-missing-compile-config

bc9a956

Merge branch 'master' into fix/issue-1379-missing-compile-config

70c9f63

Merge master into fix/issue-1379-missing-compile-config

084821e

Merge master into feat/better-reclass-error-messages

c1a534f

Merge branch 'master' into fix/issue-1379-missing-compile-config

68064f6

Log skipped targets at info level instead of debug

97ea8bb

Address review feedback: skipping targets without kapitan/kapitan.compile config is now visible at info level so a forgotten class (e.g. kapitan.common) isn't silently masked.

Align #1379 compile tests with repo test conventions

dd67f4c

Use the isolated_compile_dir fixture and create_test_inventory() helper instead of hand-rolled tempfile + os.chdir + inline YAML, per CLAUDE.md testing guidelines (CompileTestHelper / fixtures, no manual chdir).

Merge branch 'master' into refactor/cached-pr0-safety-net

d73179a

Merge branch 'master' into refactor/cached-pr0-safety-net

56b296b

Merge branch 'master' into fix/issue-1379-missing-compile-config

a01f200

feat: discover generator-provided schema.json for dependency validation

4ba2553

fix(schema-validation): address review feedback for generator schema validation

docs: add SEO frontmatter to generator schema validation page

5a05186

test: set path_traversal_mode on mocked args in CompileTargetTest

705c650

MagicMock args returned a Mock for path_traversal_mode, which failed validation in set_path_traversal_mode (expected warn/error/off). Set it explicitly so the worker re-apply step gets a real value.

Merge branch 'master' into refactor/cached-pr1-kms-factories

572645d

Merge branch 'master' into refactor/cached-pr3-explicit-inventory

a8b9ddf

Merge branch 'master' into refactor/cached-pr2-testable-helpers

a6ffb71

Merge branch 'master' into fix/issue-1379-missing-compile-config

14c66e1

Merge branch 'master' into harden/path-traversal-warning

a72acb4

Merge branch 'master' into refactor/cached-pr3-explicit-inventory

6cb9ad3

Moep90 added 11 commits November 5, 2026 05:56

Merge branch 'harden/path-traversal-warning' into feat/fetch-reload-i…

057cbe6

…nventory Stacks #1641 on #1566. Only the kapitan.utils import conflicted. Signed-off-by: Moep90 <3042152+Moep90@users.noreply.github.com>

Merge branch 'docs/fix-stale-dependency-topics-docs' into feat/fetch-…

9db5007

…reload-inventory Stacks #1641 on #1634. The new "Fetching classes" section stays, the stale "Caching" section after it goes, as #1634 intends. Signed-off-by: Moep90 <3042152+Moep90@users.noreply.github.com>

Merge #1676 into integration/reload-and-from-file

d4394d9

Signed-off-by: Moep90 <3042152+Moep90@users.noreply.github.com> #1676 is the reload commit of #1641 on master. This branch already has it, resolved against the stacked PRs, so the merge keeps that resolution.

Merge #1654 into integration/reload-and-from-file

7ee1d4d

Signed-off-by: Moep90 <3042152+Moep90@users.noreply.github.com>

Merge #1677 into integration/reload-and-from-file

b10623c

Signed-off-by: Moep90 <3042152+Moep90@users.noreply.github.com> # Conflicts: # kapitan/inventory/backends/omegaconf/resolvers.py

Merge #1656 into integration/reload-and-from-file

66b48c3

Signed-off-by: Moep90 <3042152+Moep90@users.noreply.github.com>

github-actions Bot added size/xl documentation Anything related to documentation python Pull requests that update Python code reclass issue regarding reclass and removed size/xl labels Nov 7, 2026

kadet-hub-kapitan-next Bot pushed a commit to kadet-hub/kapitan that referenced this pull request Nov 7, 2026

Merge kapicorp#1678 into next

0b96c3d

github-actions Bot mentioned this pull request Nov 7, 2026

next: integration status kadet-hub/kapitan#1

Open

kadet-hub-kapitan-next Bot pushed a commit to kadet-hub/kapitan that referenced this pull request Nov 7, 2026

Merge kapicorp#1678 into next

3fb611e

kadet-hub-kapitan-next Bot pushed a commit to kadet-hub/kapitan that referenced this pull request Nov 7, 2026

Merge kapicorp#1678 into next

656f068

kadet-hub-kapitan-next Bot pushed a commit to kadet-hub/kapitan that referenced this pull request Nov 7, 2026

Merge kapicorp#1678 into next

014bf06

kadet-hub-kapitan-next Bot pushed a commit to kadet-hub/kapitan that referenced this pull request Nov 7, 2026

Merge kapicorp#1678 into next

bfcb490

kadet-hub-kapitan-next Bot pushed a commit to kadet-hub/kapitan that referenced this pull request Nov 8, 2026

Merge kapicorp#1678 into next

22caff8

Navigation Menu

Repository navigation

chore(next): carry #1676 and #1677 for the next build, do not merge - #1678

chore(next): carry #1676 and #1677 for the next build, do not merge#1678
Moep90 wants to merge 80 commits into
masterfrom
integration/reload-and-from-file

Moep90 commented Nov 7, 2026

Uh oh!

github-actions Bot commented Nov 7, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Moep90 commented Nov 7, 2026

Uh oh!

github-actions Bot commented Nov 7, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants