Skip to content

Guard optional bz2/gzip/lzma imports so pooch imports without them (fixes #468)#540

Open
gaoflow wants to merge 2 commits into
fatiando:mainfrom
gaoflow:fix-468-optional-compression-imports
Open

Guard optional bz2/gzip/lzma imports so pooch imports without them (fixes #468)#540
gaoflow wants to merge 2 commits into
fatiando:mainfrom
gaoflow:fix-468-optional-compression-imports

Conversation

@gaoflow

@gaoflow gaoflow commented Jun 2, 2026

Copy link
Copy Markdown

Fixes #468

Problem

bz2, gzip and lzma are optional features of the Python standard
library — an interpreter can be built without them when the corresponding
bzip2/lzma/zlib development headers are missing (reported on a
self-compiled Python 3.13 and on a custom Python 3.9 on RHEL in #468).

pooch/processors.py imported all three at module level:

import bz2
import gzip
import lzma

so importing any part of pooch failed entirely on such interpreters:

File ".../pooch/processors.py", line 15, in <module>
    import lzma
ModuleNotFoundError: No module named '_lzma'

even for users who never touch Decompress.

Change

As suggested in the issue, guard the three imports so a missing module
becomes None instead of crashing at import time:

try:
    import lzma
except ImportError:
    lzma = None

The module is then only required when the matching Decompress method is
actually used. Decompress._compression_module now raises a clear
ValueError naming the missing module instead of failing later with an
AttributeError on None:

Could not decompress 'data.xz' because the 'lzma' module is not available
in this Python installation. This usually means Python was built without
'lzma' support. ...

When the modules are present (the normal case) behaviour is unchanged.

Tests

  • test_decompress_unavailable_module — monkeypatches the module table to
    None (simulating a Python built without the module) and checks the clear
    error for the lzma/xz/bzip2/auto paths.
  • test_processors_import_without_optional_modules — spawns a subprocess
    that blocks importing lzma/bz2 and asserts import pooch.processors
    still succeeds (faithful reproduction of lzma import failed in Python 3.13 #468).

Both fail on main and pass with this change; the full
test_processors.py suite (44 tests) passes and ruff is clean.

gaoflow added 2 commits June 2, 2026 05:33
bz2, gzip and lzma are optional features of the Python standard library:
an interpreter can be built without them (e.g. when the bzip2/lzma/zlib
development headers are missing). pooch.processors imported all three at
module level, so importing pooch failed entirely on such interpreters with
ModuleNotFoundError (e.g. 'No module named _lzma'), even when no
decompression was needed.

Guard the three imports so a missing module becomes None and is only
required when the matching Decompress method is actually used, in which
case a clear error is raised naming the missing module.

Fixes fatiando#468
mypy flags 'bz2 = None' as incompatible with the Module type bound by
'import bz2'. Mark the None fallbacks with 'type: ignore[assignment]'
(intentional) so the 'types' CI job passes without changing behavior.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

lzma import failed in Python 3.13

1 participant