Skip to content

Commit c93ffa3

Browse files
authored
fix(python): translate Polars decimal literals (#10187)
Polars decimal comparisons fail because `Scalar.Decimal` is unrecognized. Preserve the serialized unscaled integer, precision, and scale in a native Vortex decimal literal, without floating-point conversion. This is a direct expression translation; it performs no eager evaluation and does not require literal arguments. Literal-specific PRs decode the scalar representation itself. Validation: 1 targeted regression case(s) passed with this PR’s converter. With only `polars_.py` restored to the base revision and the same tests retained, all 1 failed with Polars `ComputeError` wrapping the unsupported-expression `ValueError` or `NotImplementedError`. The tests compare complete dataframes from Polars alone and a Vortex-backed Polars scan, and check explicit row expectations. No lint or broader suite was run. Signed-off-by: Daniel King <dan@spiraldb.com>
1 parent c00e06c commit c93ffa3

2 files changed

Lines changed: 15 additions & 0 deletions

File tree

‎vortex-python/python/vortex/polars_.py‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -101,6 +101,9 @@ def _polars_to_vortex(expr: dict[str, Any]) -> ve.Expr:
101101
dtype = "Int64"
102102
elif "Time" in scalar:
103103
return ve.literal(_dtype.time("ns"), scalar["Time"])
104+
elif "Decimal" in scalar:
105+
value, precision, scale = scalar["Decimal"]
106+
return ve.literal(_dtype.decimal(precision=precision, scale=scale), value)
104107
else:
105108
raise ValueError(f"Cannot convert to Vortex: unsupported Polars scalar value type {scalar}")
106109

‎vortex-python/test/test_polars_.py‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@
44
import math
55
import os
66
from datetime import time
7+
from decimal import Decimal
78

89
import polars as pl
910
import pyarrow as pa
@@ -102,3 +103,14 @@ def test_polars_conditional(tmp_path):
102103
actual = vx.open(str(path)).to_polars().filter(expr).collect()
103104
assert_frame_equal(actual, expected_frame)
104105
assert actual["id"].to_list() == [1, 2]
106+
107+
108+
def test_polars_decimal_literals(tmp_path):
109+
frame = pl.DataFrame({"id": [0, 1, 2, 3], "x": [Decimal("1.24"), Decimal("1.25"), None, Decimal("1.26")]})
110+
expr = pl.col("x") >= Decimal("1.25")
111+
path = tmp_path / "decimal_literals.vortex"
112+
vx.io.write(vx.array(frame.to_arrow()), str(path))
113+
expected_frame = frame.lazy().filter(expr).collect()
114+
actual = vx.open(str(path)).to_polars().filter(expr).collect()
115+
assert_frame_equal(actual, expected_frame)
116+
assert actual["id"].to_list() == [1, 3]

0 commit comments

Comments
 (0)