Skip to content

Update data representation management at stage and pipeline level and fix service docs#934

Merged
thehrh merged 27 commits into
masterfrom
stages_supported_reps_and_doc_format
Jun 12, 2026
Merged

Update data representation management at stage and pipeline level and fix service docs#934
thehrh merged 27 commits into
masterfrom
stages_supported_reps_and_doc_format

Conversation

@thehrh

@thehrh thehrh commented Apr 29, 2026

Copy link
Copy Markdown
Collaborator

Key changes and fixes:

  • remove consistency checks on apply_modes and prepare defining sum as default translation mode for certain container keys -> address in another PR (see discussion Re-define and document supported combinations of calc_mode and apply_mode in pipeline configurations #933)
  • expose whether service overrides setup_function, compute_function, or apply_function via has_setup, has_compute, and has_apply function -> default deduction of allowed representations at Stage level
  • services with restrictions on their data representations now explicitly declare which ones they support
    • removed need for given service to specify each dict value as a list (e.g. None instead of [None])
  • removed instances of superfluous representation setting, when the Pipeline already takes care of that
  • a recently introduced dangerous bug is fixed that allowed running with a representation set to None even though the corresponding function is implemented (see new tests in core.pipeline module)
  • representation consistency is now validated separately in setup(), compute(), and apply() functions in case the Stage somehow became misconfigured (unlikely to have any significant performance impact on realistic pipelines IMO)
  • give pisa_tests/test_services.py the ability to generate a service implementation reference (table/matrix) in markdown & csv format, and include said markdown table in notebook pisa_examples/pisa_modes.ipynb

Documentation improvements:

  • fixed and standardised docstring formatting across all services (in particular parameter & notes—expected container keys—sections)
  • clean up "PISA Pi" mentions
  • updated copyright years

@thehrh thehrh changed the title Definition of supported representations by services and service docs fixes Detect service implementation of compute+setup and apply, definitions of supported representations and service docs fixes May 24, 2026
thehrh and others added 9 commits May 27, 2026 03:33
…nding function isn't implemented (for now, possibly change later) [no ci]
…d include md output in stage modes notebook for now (TODO: fix notebook) [no ci]
…tion_modes -> translation_modes; representation management unit test; update container module docs
….hist assertion fails otherwise, and adapt pipeline unit test temporarily
@thehrh thehrh requested a review from JanWeldert June 3, 2026 15:29
@thehrh thehrh marked this pull request as ready for review June 3, 2026 15:29
@thehrh thehrh added this to the 4.3 milestone Jun 4, 2026
@thehrh thehrh merged commit 8942e34 into master Jun 12, 2026
@thehrh thehrh changed the title Detect service implementation of compute+setup and apply, definitions of supported representations and service docs fixes Update data representation management at stage and pipeline level and fix service docs Jun 12, 2026
@thehrh thehrh deleted the stages_supported_reps_and_doc_format branch June 12, 2026 13:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants