Skip to content

Remove auto-generated protobuf files from version control - #78

Closed
chenkasirer with Copilot wants to merge 8 commits into
mainfrom
copilot/remove-generated-proto-files
Closed

chenkasirer with Copilot wants to merge 8 commits into
mainfrom
copilot/remove-generated-proto-files

Conversation

Copilot AI commented Jan 29, 2026 •

Copy link
Copy Markdown
Contributor

Auto-generated protobuf Python files (*_pb2.py, *_pb2.pyi) were tracked in git despite being generated from .proto definitions. This creates maintenance overhead and diff noise.

Changes

Generation workflow:

  • Added .gitignore patterns for generated files
  • CI workflows now use compas-actions.build@v4.1.0 with built-in run_prebuild support
  • Added pre-build invoke task to generate proto files automatically
  • README.md documents generation requirement for developers

Plugin compatibility:

  • Kept invocations.py in src/compas_pb/ to maintain backward compatibility with plugins
  • Implemented lazy imports in tasks.py to avoid circular dependency during proto generation
  • Created wrapper tasks that import from compas_pb.invocations at runtime
  • Added src/compas_pb/generated/__init__.py to maintain package structure

Code quality:

  • Fixed typos (class_assests → class_assets)
  • Added path traversal protection in tar extraction
  • Improved error handling for unsupported target languages
  • Unified GitHub Actions versions to v5

Developer workflow

git clone https://github.com/gramaziokohler/compas_pb.git
cd compas_pb
pip install -e ".[dev]"
invoke pre-build  # Required after clone

Generated files are included in wheel distributions automatically.

What type of change is this?

  • Bug fix in a backwards-compatible manner.
  • New feature in a backwards-compatible manner.
  • Breaking change: bug fix or new feature that involve incompatible API changes.
  • Other (e.g. doc update, configuration, etc)

Checklist

  • I added a line to the CHANGELOG.md file in the Unreleased section under the most fitting heading (e.g. Added, Changed, Removed).
  • I ran all tests on my computer and it's all green (i.e. invoke test).
  • I ran lint on my computer and there are no errors (i.e. invoke lint).
  • I added new functions/classes and made them available on a second-level import, e.g. compas.datastructures.Mesh.
  • I have added tests that prove my fix is effective or that my feature works.
  • I have added necessary documentation (if appropriate)
Original prompt

This section details on the original issue you should resolve

<issue_title>remove generated proto files from repository</issue_title>
<issue_description> The generater proto python files used in the protobuf wrapper of this library shouldn't be committed to this codebase because they are auto-generater from the .proto files included in the repo.

Figure out how these can be removed and remove them.

  • They should be generated using the task included in tasks.py before running the tests on the CI
  • They should be packages along with the built wheel when built for distribution</issue_description>

<agent_instructions>please remove these auto-generated python modules</agent_instructions>

Comments on the Issue (you are @copilot in this section)


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI and others added 4 commits January 29, 2026 11:40
- Add generated proto files to .gitignore
- Move invocations.py to root to avoid circular dependency during generation
- Update CI workflows to generate proto files before testing
- Update README with development installation instructions
- Update pyproject.toml to explicitly include generated files in package
- Remove generated files from git tracking

Co-authored-by: chenkasirer <3398309+chenkasirer@users.noreply.github.com>
The files were previously tracked in git. This commit removes them from the index so they are properly ignored.

Co-authored-by: chenkasirer <3398309+chenkasirer@users.noreply.github.com>
This ensures the generated directory is tracked in git and recognized as a Python package, even when the generated proto files are not present.

Co-authored-by: chenkasirer <3398309+chenkasirer@users.noreply.github.com>
- Fix typos: 'class_assests' -> 'class_assets'
- Add missing f-string prefix for dist_dir message
- Move shutil import to top of file
- Add return statement for unsupported target language
- Add path traversal protection in tar extraction
- Update GitHub Actions to use consistent versions (v5)

Co-authored-by: chenkasirer <3398309+chenkasirer@users.noreply.github.com>
Copilot AI changed the title [WIP] Remove generated proto files from repository Remove auto-generated protobuf files from version control Jan 29, 2026
Copilot AI requested a review from chenkasirer January 29, 2026 11:48
Comment thread .github/workflows/build.yml Outdated

steps:
- uses: compas-dev/compas-actions.build@v4
- uses: actions/checkout@v4

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot check out version 4.1.0 of the build task, there's a pre-build task invocation there that might be useful for this. would be cool if we don't have to reimplement the build action

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated to use compas-actions.build@v4.1.0 with the run_prebuild parameter. Created a new pre-build invoke task that generates the protobuf files. This simplifies the workflow by leveraging the built-in pre-build support instead of reimplementing all the build steps. Commit: 3501071

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot if possible, leave this module where it is. it's relied upon by plugins of this project

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved invocations.py back to src/compas_pb/ to maintain plugin compatibility. Used lazy imports in tasks.py to avoid circular dependency issues during proto file generation. Commit: 33c768f

Comment thread .github/workflows/release.yml Outdated

steps:
- uses: compas-dev/compas-actions.build@v4
- uses: actions/checkout@v4

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot same here, see if you can use the new version of the build workflow

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Already using compas-actions.build@v4.1.0 with run_prebuild: true from commit 3501071. No changes needed here.

- Added pre-build invoke task to generate protobuf files
- Updated build.yml to use compas-actions.build@v4.1.0 with run_prebuild parameter
- Updated release.yml to use compas-actions.build@v4.1.0 with run_prebuild parameter
- Updated README to reference invoke pre-build instead of longer command
- This leverages the built-in pre-build support instead of reimplementing the build action

Co-authored-by: chenkasirer <3398309+chenkasirer@users.noreply.github.com>
- Moved invocations.py from root back to src/compas_pb/ as requested
- Added lazy imports in tasks.py to avoid circular dependency
- Created wrapper tasks that import from compas_pb.invocations at runtime
- This maintains backward compatibility for plugins that rely on this module

Co-authored-by: chenkasirer <3398309+chenkasirer@users.noreply.github.com>
@gonzalocasas

Copy link
Copy Markdown
Member

@chenkasirer should we close this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

remove generated proto files from repository

3 participants