Skip to content

Commit 87719b7

Browse files
committed
Count valid rows from the validity array's Sum statistic
`valid_count` uses an exact cached `NullCount` first (`len - null count`). Otherwise, for array-backed validity, it now reads the validity array's `Sum` statistic, which is the number of valid rows and can be cached or computed by an encoding kernel, as develop did with the `sum` aggregate. It counts the bitmap only when the session registers no `Sum` aggregate. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ps4mp7KF5c4p66aoSoXx5j Signed-off-by: Claude <noreply@anthropic.com>
1 parent 15225c8 commit 87719b7

1 file changed

Lines changed: 10 additions & 1 deletion

File tree

‎vortex-array/src/array/erased.rs‎

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -361,7 +361,16 @@ impl ArrayRef {
361361
let count = match self.validity()? {
362362
Validity::NonNullable | Validity::AllValid => len,
363363
Validity::AllInvalid => 0,
364-
Validity::Array(a) => a.execute::<BoolArray>(ctx)?.to_bit_buffer().true_count(),
364+
// The number of valid rows is the number of `true` values in the validity array, which
365+
// its `Sum` statistic gives. Count the bitmap if the session has no `Sum` aggregate.
366+
Validity::Array(a) => match a
367+
.statistics()
368+
.compute_stat(Stat::Sum, ctx)?
369+
.and_then(|sum| sum.as_primitive().as_::<usize>())
370+
{
371+
Some(count) => count,
372+
None => a.execute::<BoolArray>(ctx)?.to_bit_buffer().true_count(),
373+
},
365374
};
366375
vortex_ensure!(count <= len, "Valid count exceeds array length");
367376

0 commit comments

Comments
 (0)