Guard optional bz2/gzip/lzma imports so pooch imports without them (fixes #468)#540
Open
gaoflow wants to merge 2 commits into
Open
Guard optional bz2/gzip/lzma imports so pooch imports without them (fixes #468)#540gaoflow wants to merge 2 commits into
gaoflow wants to merge 2 commits into
Conversation
bz2, gzip and lzma are optional features of the Python standard library: an interpreter can be built without them (e.g. when the bzip2/lzma/zlib development headers are missing). pooch.processors imported all three at module level, so importing pooch failed entirely on such interpreters with ModuleNotFoundError (e.g. 'No module named _lzma'), even when no decompression was needed. Guard the three imports so a missing module becomes None and is only required when the matching Decompress method is actually used, in which case a clear error is raised naming the missing module. Fixes fatiando#468
mypy flags 'bz2 = None' as incompatible with the Module type bound by 'import bz2'. Mark the None fallbacks with 'type: ignore[assignment]' (intentional) so the 'types' CI job passes without changing behavior.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #468
Problem
bz2,gzipandlzmaare optional features of the Python standardlibrary — an interpreter can be built without them when the corresponding
bzip2/lzma/zlibdevelopment headers are missing (reported on aself-compiled Python 3.13 and on a custom Python 3.9 on RHEL in #468).
pooch/processors.pyimported all three at module level:so importing any part of pooch failed entirely on such interpreters:
even for users who never touch
Decompress.Change
As suggested in the issue, guard the three imports so a missing module
becomes
Noneinstead of crashing at import time:The module is then only required when the matching
Decompressmethod isactually used.
Decompress._compression_modulenow raises a clearValueErrornaming the missing module instead of failing later with anAttributeErroronNone:When the modules are present (the normal case) behaviour is unchanged.
Tests
test_decompress_unavailable_module— monkeypatches the module table toNone(simulating a Python built without the module) and checks the clearerror for the
lzma/xz/bzip2/autopaths.test_processors_import_without_optional_modules— spawns a subprocessthat blocks importing
lzma/bz2and assertsimport pooch.processorsstill succeeds (faithful reproduction of
lzmaimport failed in Python 3.13 #468).Both fail on
mainand pass with this change; the fulltest_processors.pysuite (44 tests) passes andruffis clean.