Skip to content

Bloom filter keys floats by raw bytes: -0.0 and +0.0 land in different buckets #10276

Description

@jackylee-ch

The bloom-filter aggregate keys float values by their raw little-endian bytes, on both the
accumulate side (vortex-layout/src/layouts/zoned/aggregates/bloom_filter/canonical/primitive.rs:45
and :52) and the probe side (bloom_filter/partial/scalar.rs:45 and :53), with no
normalization. Since -0.0 is 0x8000…0 and +0.0 is 0x0000…0, the two hash to different
buckets even though they compare equal under both IEEE 754 and SQL.

Once the filter drives pruning (via the write interface in #9413), a chunk that stored +0.0
would be skipped for a predicate like col = -0.0, dropping rows that should match — a silent
wrong result rather than a conservative over-fetch.

Distinct NaN bit patterns are kept distinct on purpose (the nan_bit_patterns_are_distinct_members
test fixes that, and NaN != NaN makes bloom membership moot for NaN), so this is specifically
about signed zero. Canonicalizing -0.0 to +0.0 before keying on both sides would fix it while
leaving NaN handling untouched.

The aggregate is marked unstable (#9753), so filing rather than patching in case the keying is
already being reworked as part of stabilization.

AI assistance

Found and written with agentic AI assistance; both keying sites were read on develop before filing.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions