Move kernels-data Python binding into kernels

[danieldk]

This simplifies installs from the development tree. It also avoids having to maintain a separate dependency on kernels-data. The binding bits will always be in sync with kernels.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[sayakpaul]

Nice. My only concern is that the kernels source is now getting a bit harder to navigate.

Also, is there any benefit of having another directory with rust, called kernels-data and put the respective files there? This will have a benefit if we ever wanted to add any more such Rust-based components to the kernels source.

[danieldk]

Also, is there any benefit of having another directory with rust, called kernels-data and put the respective files there? This will have a benefit if we ever wanted to add any more such Rust-based components to the kernels source.

Since it's internal now, we can always rename things. The module is _data now, but we can always rename it _rust in the future to be more generic. I think there is a benefit to having all the Rust bits in a single module (smaller compile/link time). To that end, I think I want to rename kernels-data (the Rust part) next to make it more generic, since we might also get more non-data stuff there.

[github-actions]

Coverage report — kernels/

Measured on: Python 3.10 / Torch 2.13.0.
Other CI configurations are not included in this number.
Hardware-gated code paths (ROCm/XPU/NPU/Darwin/Windows) are excluded or unreachable on the Linux+CUDA runner.

Total coverage: 87.1% — threshold: 80% — ✅

Per-file breakdown

Name	Stmts	Miss	Cover	Missing
src/kernels/__init__.py	14	0	100%
src/kernels/_system.py	6	1	83%	10
src/kernels/_versions.py	78	9	88%	48, 54-55, 58-59, 98, 120, 131, 137
src/kernels/archs.py	56	1	98%	94
src/kernels/backends.py	213	62	71%	42, 46, 50-53, 70, 92, 110, 119, 123, 127-129, 150, 159, 163, 167-169, 190, 201, 203, 210-213, 226, 230, 234-254, 262, 285-305
src/kernels/compat.py	8	1	88%	5
src/kernels/deps.py	65	1	98%	58
src/kernels/hf_hub.py	63	2	97%	20, 22
src/kernels/importer.py	44	5	89%	82, 86, 89, 103-104
src/kernels/install.py	21	7	67%	76-100
src/kernels/layer/__init__.py	6	0	100%
src/kernels/layer/_interval_tree.py	103	4	96%	23, 52, 147, 150
src/kernels/layer/device.py	48	14	71%	42, 47-49, 91, 96-98, 101, 149, 152, 155-157
src/kernels/layer/func.py	88	6	93%	92, 125, 198, 318, 344, 374
src/kernels/layer/globals.py	5	0	100%
src/kernels/layer/kernelize.py	80	8	90%	258, 293, 301-302, 308, 312, 328-330
src/kernels/layer/layer.py	218	14	94%	189, 236, 262, 396, 476-477, 498, 506, 517, 546, 550, 563, 616, 646
src/kernels/layer/mode.py	14	0	100%
src/kernels/layer/repos.py	144	42	71%	27, 33, 36-43, 63-64, 70, 73-76, 90, 94, 103-104, 110, 113-116, 123-124, 130, 133-136, 143-144, 150, 153-156, 163-164, 170, 173-176, 257
src/kernels/load.py	70	2	97%	329, 368
src/kernels/locking.py	89	64	28%	35-83, 91-98, 102-125, 137, 152-159, 165-175, 179-186
src/kernels/python_deps.py	58	6	90%	59-60, 64-65, 101, 104
src/kernels/resolver.py	156	2	99%	220, 226
src/kernels/status.py	50	2	96%	25, 79
src/kernels/validate.py	54	1	98%	85
src/kernels/variants.py	278	19	93%	65, 96, 117, 147, 256-257, 299-302, 304, 388-394, 400-406, 437-443, 455-461
src/kernels/verify.py	88	1	99%	33
TOTAL	2117	274	87%

Updated by the Test kernels workflow on commit 18e7bf88ada3bdbfdb2d41a66abcce72e45c237b.