Skip to content

Commit ac87fe0

Browse files
authored
AllInvalid arrays have non empty distinct info (#7369)
This is required to handle cases where we are attempting to dict compress and a sample ends up being all null Signed-off-by: Robert Kruszewski <github@robertk.io> Signed-off-by: Robert Kruszewski <github@robertk.io>
1 parent d296528 commit ac87fe0

2 files changed

Lines changed: 13 additions & 2 deletions

File tree

‎vortex-compressor/src/stats/float.rs‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -179,7 +179,13 @@ where
179179
null_count: u32::try_from(array.len())?,
180180
value_count: 0,
181181
average_run_length: 0,
182-
erased: TypedStats { distinct: None }.into(),
182+
erased: TypedStats {
183+
distinct: Some(DistinctInfo {
184+
distinct_values: HashSet::with_capacity_and_hasher(0, FxBuildHasher),
185+
distinct_count: 0,
186+
}),
187+
}
188+
.into(),
183189
});
184190
}
185191

‎vortex-compressor/src/stats/integer.rs‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -338,7 +338,12 @@ where
338338
erased: TypedStats {
339339
min: T::max_value(),
340340
max: T::min_value(),
341-
distinct: None,
341+
distinct: Some(DistinctInfo {
342+
distinct_values: HashMap::with_capacity_and_hasher(0, FxBuildHasher),
343+
distinct_count: 0,
344+
most_frequent_value: T::zero(),
345+
top_frequency: 0,
346+
}),
342347
}
343348
.into(),
344349
});

0 commit comments

Comments
 (0)