torch.compile() may fail in some configuration

[tardyp]

Bug

Docling fails to convert a PDF on Apple Silicon macOS when Torch model compilation is enabled by default.

The conversion reaches the layout-detection stage, then fails inside PyTorch TorchInductor with:

  torch._inductor.exc.InductorError:
  TypeError: BooleanAtom not allowed in this context.

Relevant traceback path:

  Docling layout model
    -> Transformers object-detection engine
      -> torch.compile()
        -> TorchDynamo/TorchInductor
          -> SymPy symbolic shape handling
            -> BooleanAtom not allowed in this context

Disabling Docling’s Torch compilation allows the same PDF to convert successfully:

  DOCLING_INFERENCE_COMPILE_TORCH_MODELS=false \
  uvx --native-tls docling \
    --to md \
    -v \
    --image-export-mode referenced \
    <mypdf>.pdf

The workaround produced valid Markdown output and referenced image artifacts. Therefore, the failure appears limited to the compiled inference path rather than PDF parsing or the conversion pipeline itself.

It would be useful if Docling either:

• Automatically fell back to eager inference when torch.compile()/TorchInductor fails; or
• Disabled model compilation by default for unsupported or unstable platform/backend combinations.

Steps to reproduce

1. Use an Apple Silicon Mac.
2. Obtain a PDF with the relevant characteristics. The original test file was a 64-page encrypted PDF.
3. Run:

     uvx --native-tls docling \
       --to md \
       -v \
       --image-export-mode referenced \
       <mypdf>.pdf

4. Observe that Docling selects the MPS accelerator:

     Accelerator device: 'mps'
   

5. Observe that it initializes the Transformers object-detection layout engine.
6. Observe conversion failure during layout processing:

     Stage layout failed
     TypeError: BooleanAtom not allowed in this context.
   

7. Run the same conversion with model compilation disabled:

     DOCLING_INFERENCE_COMPILE_TORCH_MODELS=false \
     uvx --native-tls docling \
       --to md \
       -v \
       --image-export-mode referenced \
       <mypdf>pdf

8. Observe that conversion succeeds.

Docling version

  Docling version: 2.119.0
  Docling Core version: 2.91.0
  Docling IBM Models version: 3.14.0
  Docling Parse version: 7.12.1
  Python: cpython-312 (3.12.11)
  Platform: macOS-26.5.2-arm64-arm-64bit

Python version

  Python 3.12.11

Additional environment information:

  PyTorch: 2.13.0
  Device selected by Docling: mps
  OS: macOS arm64

The exact error is:

  torch._inductor.exc.InductorError: TypeError: BooleanAtom not allowed in this context.

[dosubot]

@tardyp This is closely related to #3956, which tracks the same class of problem. torch.compile() is lazy — it returns an OptimizedModule immediately and defers actual kernel compilation to the first forward pass, so errors like your BooleanAtom not allowed in this context don't surface until inference starts rather than at model initialization [1].

Two open PRs are directly addressing this:

• #3964 — adds a fallback in TransformersObjectDetectionEngine: on a TorchDynamoException during the first forward pass, it logs a warning and permanently swaps back to the uncompiled model, retrying the batch [1].
• #3962 — disables torch.compile() by default on Windows by setting compile_torch_models to sys.platform != "win32" [2]. Your case (MPS/macOS) suggests a similar per-platform gate for darwin may be warranted.

The existing version gates only check Python/PyTorch versions, not the OS or backend [3], so this MPS + PyTorch 2.13 failure mode on macOS arm64 isn't covered by any current guard.

Until one of those PRs lands, your workaround (DOCLING_INFERENCE_COMPILE_TORCH_MODELS=false) is the correct approach [4]. You can also set it in Python before creating the converter:

from docling.datamodel.settings import settings
settings.inference.compile_torch_models = False

It would be worth noting here that the BooleanAtom error (MPS + PyTorch 2.13 on macOS arm64) is a distinct failure mode from the Windows MSVC one in #3956, which would strengthen the case for either extending #3964's fallback to catch it or adding darwin to the platform gate in #3962.

To reply, just mention @dosu.

---

Docs are dead. Just use Dosu.