Skip to content

feat(deployment-operator): implement monty-based python interpreter for values templating - #4054

Open
floreks wants to merge 7 commits into
masterfrom
sebastian/prod-5160-implement-python-equivalent-in-deployment-operator
Open

feat(deployment-operator): implement monty-based python interpreter for values templating#4054
floreks wants to merge 7 commits into
masterfrom
sebastian/prod-5160-implement-python-equivalent-in-deployment-operator

Conversation

@floreks

@floreks floreks commented Aug 25, 2026

Copy link
Copy Markdown
Member

Test Plan

Test environment: https://console.plrl-dev-aws.onplural.sh/

Checklist

  • I have added a meaningful title and summary to convey the impact of this PR to a user.
  • If required, I have updated the Plural documentation accordingly.
  • I have added tests to cover my changes.
  • I have deployed the agent to a test environment and verified that it works as expected (required only when changing agent code).

Plural Flow: console

- Add support for Python script resources (`PythonScript`, `PythonFile`, `PythonFolder`) in Helm value generation.
- Extend Helm render logic to merge Python and Lua values.
- Implement Python tests for template values.
- Introduce Python runtime utils and Gomonty integration.
- Update Dockerfile to include necessary environment variables and dependencies for Python execution.
- Add comprehensive unit tests for manager wait and Python templating functionalities.
…-implement-python-equivalent-in-deployment-operator
@floreks floreks self-assigned this Aug 25, 2026
@floreks floreks added the enhancement New feature or request label Aug 25, 2026
@linear

linear Bot commented Aug 25, 2026

Copy link
Copy Markdown

PROD-5160

@soffi-ai

soffi-ai Bot commented Aug 25, 2026

Copy link
Copy Markdown
Soffi AI Summary

This PR introduces a Python-based values templating engine for Helm deployments in the deployment operator, as an alternative to the existing Lua scripting support. The motivation is to give operators a more familiar and expressive scripting language (Python via a "monty" interpreter pool) for generating Helm values dynamically — matching capabilities already available through Lua scripts.

The implementation adds a managed Python runtime pool (go/deployment-operator/pkg/python/pool.go) that executes user-provided Python scripts in a sandboxed, timeout-controlled manner. The Helm rendering pipeline is extended to accept three new input modes — an inline PythonScript, a single PythonFile, or a PythonFolder (whose .py files are concatenated in sorted order) — and merges the resulting values and values-file paths with those from the existing Lua and static values layers. The Python pool is initialized at agent startup and wired into the service reconciler. The GraphQL client schema and generated models are updated to expose the new pythonScript, pythonFile, and pythonFolder fields on the Helm service spec. A manifest cache fix is also bundled in, along with logging improvements for render inputs and a refactor of values-file merging to use slices.Concat.

Changes

Python values templating for Helm deployments

  • Integrated the Python runtime pool into the Helm template engine. Added pythonValues and pythonFolder methods that read Python scripts from inline strings, single files, or entire folders, and merged their output into the Helm values pipeline alongside the existing Lua layer. (00374cc)
  • Refactored the values-file list merging step to use slices.Concat for cleaner combination of Lua and Python values-file paths before passing them to the Helm renderer. (da04cb6)
  • Fixed a manifest cache bug that was encountered during development and integration of the Python templating feature. (927c9e0)
  • Cleaned up a stale error-handling path that was no longer needed after the Helm rendering refactor. (0efc136)
  • Added structured logging for Helm render inputs and a utility helper function to improve observability of the templating pipeline during debugging and production runs. (144612f)

Updated: 2026-08-26 15:36 UTC

Deploy in Soffi

@socket-security

socket-security Bot commented Aug 25, 2026

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addedgolang/​github.com/​ewhauser/​gomonty@​v0.0.1499100100100100

View full report

@greptile-apps

greptile-apps Bot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Adds Monty-based Python execution for Helm values templating in the deployment operator.

  • Extends the Console service deployment query and generated client models with Python script, file, and folder fields.
  • Introduces a bounded Python worker pool with execution, memory, recursion, cancellation, and output-shape limits.
  • Integrates Python-generated values and values files into Helm rendering while retaining Lua support.
  • Initializes and shuts down the runtime with the operator lifecycle and updates the container image for the native runtime.
  • Adds focused tests for execution isolation, resource limits, path confinement, merge precedence, concurrency, and shutdown behavior.

Confidence Score: 5/5

The PR appears safe to merge; no concrete blocking or independently actionable non-blocking issue was identified.

The Python runtime is bounded and lifecycle-managed, filesystem inputs are confined to the fetched source root, generated values preserve explicit precedence, and the dependency security leads either predate this change or lack a reachable changed-code path.

Important Files Changed

Filename Overview
go/deployment-operator/pkg/python/pool.go Implements a bounded, cancellable pool of isolated Monty REPLs with strict JSON input/output and resource limits; no actionable defect was identified.
go/deployment-operator/pkg/manifests/template/helm.go Adds path-confined Python script loading and merges generated files and values in an explicitly tested Lua-then-Python order.
go/deployment-operator/cmd/agent/main.go Initializes the Python pool after configuration loading and waits for reconcilers before deferred runtime cleanup.
go/deployment-operator/pkg/controller/controller_manager.go Adds an idempotent shutdown-completion signal used to coordinate runtime cleanup with controller termination.
go/deployment-operator/Dockerfile Enables the Monty musl build, installs its runtime library, and provides a writable FFI cache directory for the non-root operator user.
go/client/graph/deployment.graphql Requests the three new Helm Python configuration fields through the agent-facing service deployment fragment.

Reviews (1): Last reviewed commit: "Merge remote-tracking branch 'origin/mas..." | Re-trigger Greptile

floreks and others added 5 commits August 25, 2026 13:33
- Replace manual append method with `slices.Concat` for combining `luaValuesFiles` and `pythonValuesFiles`.
…an/prod-5160-implement-python-equivalent-in-deployment-operator
- Introduce `helmRenderInputs` to generate Helm values files and configuration keys
- Log Helm render input details including values files and their existence status
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants