fix(dict-writer): stop DictStreamState::encode from looping on values that overflow an empty dictionary

[wahajahmed010]

Summary

Tracking Issue: #10397

When every value in a chunk was longer than max_bytes - 16 (the most an empty dictionary can hold once the 16-byte BinaryView header is included), BytesDictBuilder::encode_value returned None for every row. DictStreamState::encode then produced 0 codes from encode_chunk, remainder returned the entire unencoded chunk, and the loop re-fed the same chunk indefinitely. Each pass also pushed empty codes/values chunks into res, so memory grew without bound until the process was killed.

Changes

Detect the case where encode_chunk (or start_encoding) returned EncodingState::Continue with a zero-length encoded array while the chunk still had rows. In both arms (self.encoder.take() == None and Some(encoder)) the function now returns Ok(res) immediately and lets the encoder drop. The outer loop never re-feeds the chunk that cannot be encoded.

While here, also skip the inner while body when remaining is already empty (the caller's contract guarantees that branch is unreachable today, but the guard makes the loop self-consistent if it is reached).

Two regression tests in mod tests:

• test_dict_stream_state_encodes_value_filling_an_empty_dictionary — a 48-byte string under a 64-byte dictionary limit, which should still encode successfully.
• test_dict_stream_state_does_not_loop_on_oversized_value — a 49-byte string under the same 64-byte dictionary limit. The writer must return within 2 s (the test runs DictStreamState::encode in a spawned thread with an mpsc channel + recv_timeout, so a missing fix hangs the thread instead of the suite).

API Changes

None. The bug is observable on the dict layout writer; no public signatures change.

AI assistance disclosure: this patch was drafted with help from an LLM-based coding assistant. The fix design, code, and tests were reviewed and edited manually before submission.

[robert3005]

see #10400