Claude/paperless odf phase 1 rnyzcu - #64
Closed
theolivenbaum wants to merge 416 commits into
Closed
Conversation
Two bytes in BIFF8 — which is a property of the HEADER record rather than of the format, so LibreOffice picks between ReadByteString and ReadUniString on the generation (xipage.cxx:114). Reading it wrong loses the header, which loses nothing visible: the page count cannot catch it, because with the header gone the top margin and the band move by the same amount in opposite directions and the printable height is unchanged. Asserted directly instead, alongside the BIFF column-width conversion — 2953 256ths of a character at 111 twips a digit comes out at 1279 twips and not 1280, because the conversion subtracts half a twip before truncating, and LibreOffice's own rendering puts the columns 63.95 points apart. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
The restart rule was read from all four formats and applied by nothing. It is genuinely circular -- a note's number under a per-page restart is its position within its page, the citation's width follows from the number, and where the citing line breaks follows from the width -- and Writer does not look for a fixed point. It numbers over pages that already exist (SwRootFrame::UpdateFootnoteNums, ftnfrm.cxx:971; SwPageFrame::UpdateFootnoteNum, ftnfrm.cxx:2564) and then damps: txtftn.cxx:560 validates the frame one iteration stale under the comment "We break the oscillation". Layout/NoteRenumbering.cs does the same -- paginate, renumber per page, lay out once more, stop. Three things writing it turned up: A renumbering is a text edit, not a number. LibreOffice draws a note's number twice and all four readers emit it twice, so the pass rewrites paragraphs -- text, runs, frame anchors and the offsets of the notes after it. PageNote had to learn the class's whole NoteNumbering (the restart alone cannot say what sequence to write the new number in), the citation as read, and where that citation sits in the note's own body. The last is zero in three formats and not in DOCX, where a w:footnoteRef marks the place and a note beginning with a tab puts it at one. The rewritten blocks have to reach the caller, or pagination indexes lines into a list the reader still holds and the old numbering is drawn. Each page names its own list and Paginator.Blocks reports it. Guarded on NoteRenumbering.Applies, so every document whose notes do not restart pays one walk and takes no second pagination. Verified against LibreOffice in all four formats: page one cites 1,2,3,4 and page two 1,2,3,4 again, at the anchor and at the head of the note, and the drawn text is character for character LibreOffice's. It cited 5,6,7,8 before. 1627 passing, 0 failed, 0 skipped. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
The rendering half of the TODO: every rule with its citation, every number with the reference it was measured against, and every gap with what would settle it. Also the two decks that are not in the fidelity comparison and what they currently measure at, since that is where the next work is. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
The 16 pt heading baseline is fixed, and the rule is not where the symptom was. The leading proportional line spacing adds above a paragraph's first line belongs to the paragraph *above* it, measured against that paragraph's last line: SwTextFormatter::CalcRealHeight guards the whole switch with if(!IsParaLine()) and says outright that "for the _first_ line the line spacing of the previous paragraph is applied in SwFlowFrame::CalcUpperSpace()". That is why nothing saw it. Between two paragraphs at the same spacing the two readings agree exactly — the leading one paragraph withholds is the leading the next hands back — so every baseline lands where it did and the block is the same height. It is visible only across a size change, and only as an absolute baseline. Before: 35.250/22.400 against LibreOffice's 37.200/20.450. After: 37.250/20.400, and every baseline of paginated.* in all four formats now agrees absolutely to 0.051 pt, the constant this tree already lives with. The footnotes.rtf separator turns out to be LibreOffice's defect, not ours, and the attribution is measured rather than asserted: its RTF import loses the face the note states, rendering the built-in Footnote style inheriting Liberation Serif where the file says Carlito twice over. Bisected on hand-written RTF — the trigger is the \sN reference, not the group. Liberation Serif is 13 twips taller a line at ten point; two notes, 26 twips, 1.30 pt against the 1.286 measured. Recorded as a known deviation with a test that fails if it stops holding. Note-restart application follows Writer's shape exactly: renumber per page over finished pages, lay out once more, stop. A renumbering turned out to be a text edit rather than a number, since all four readers emit the citation into the text twice. Merging this against the frame work needed the two rules combined rather than either picked: a line beside a frame clear of both margins is several boxes on one baseline sharing one box's worth of geometry, so the leading has to come off whichever stretch is the one whose height is counted, and Fit measures the line from its last stretch. 1649 tests pass, 0 skipped. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
Slides lay out and draw. PPTX and ODP decks emit background, per-shape fill, outline and text into IDrawingSink; no backend was written, because the PDF and raster ones landed earlier today. The finding that would otherwise have cascaded through every deck: a slide's line height is not the font's. pptshapecontext.cxx sets FontIndependentLineSpacing, and impedit3.cxx then makes ascent one em and the line 1.2 em — so against Liberation Sans's own ascent of 0.905 em the font's metrics are 1.705 pt out on every line of every shape. And flip-before-rotation is proved rather than assumed: an rtTriangle with flipH and rot=5400000 lands on the reference's exact vertices, where the other order gives a different triangle in a different corner. Right-to-left text draws in visual order, consuming the levels the itemiser was already computing. A right-to-left paragraph is a mirror rather than a special case, which is what Writer does in SwTextFrame::SwitchLTRtoRTL. L2 moved into one generic TextItemiser.ReorderVisually so sub-runs and glyph runs cannot be reordered by two implementations. Its named trap cost hours and is worth keeping: ParagraphAdjust::START only exists from LibreOffice 26.2, so the installed 24.2 renders a start-aligned right-to-left paragraph against the *left* margin and rewrites fo:text-align="right" as "end" on export. The corpus document states alignment physically for that reason. The three-way merge needed combining rather than picking twice. In Paginator, a line beside a frame is several boxes on one baseline sharing one box's worth of geometry, so the leading rule had to come off whichever stretch is the one whose height is counted. In ParagraphLayouter, one agent hoisted the justification to a local and the other added the stretch flag; both are kept, and the hoist was checked to be per-site rather than a behaviour change, since the two call sites measure different text. 1725 tests pass, 0 failed, 0 skipped, across all ten projects. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
…-rnyzcu Claude/paperless odf phase 1 rnyzcu
… not load A FontReference whose FaceKey names a family rather than a path never loads, and the writer then had no hmtx to state a /Widths from, so it stated zeros. Every glyph therefore sat a whole advance from where the widths put the pen and the content stream corrected each one with a TJ of the entire advance: -722 and -556 thousandths between adjacent glyphs on sheet-print-xlsx.xlsx. An adjustment that large is how a PDF spells a word break, so pdftotext got 13255 words out of that file's fourteen pages against LibreOffice's 2281 -- one per character -- while every operator-level comparison passed, because the pages did look right. The display list already carries each glyph's advance, so that is the fallback. With it, the same file extracts as 2281 words, exactly LibreOffice's count. This is a fallback, not the fix: the face is still not embedded. Three callers build the reference this way -- Paperless.Spreadsheets' SheetText.Describe, Paperless.Presentations' SlideTextLayout.Reference and Paperless.WordProcessing's PageDrawing.Reference -- and each should carry the resolver's own key instead. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
…age sequence A DrawingML table decomposes into the shapes a slide already draws: one per visible cell carrying its fill and its text, then one per consolidated grid line carrying only a pen — cells first and borders after, which is what LibreOffice's own table view contact does 'to get the correct overlapping'. Measured against LibreOffice 24.2.7.2's PDF for a three-row, two-column table whose every cell states a one-point red edge: seven strokes in the same order, positions within 0.013 pt, pen 0.85 against 0.85009, cell fills and every baseline within 0.015 pt. The page sequence now skips hidden slides, which is what the reference PDF does; extraction still reports and flags them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
…gns it
Cell fonts, alignment (including the general default that is the cell's
type rather than a constant), vertical alignment, indent, wrap,
shrink-to-fit and the `###`/General-shrink a too-narrow numeric cell
shows, ported from sc/source/ui/view/output2.cxx.
Two measured findings drove the geometry. The sheet path built its
FontReference from the face's family name rather than from the
resolver's key, so the PDF backend could not load the face, had no
/Widths to advance the pen with, and corrected every glyph with a TJ
adjustment of about -700 thousandths — which pdftotext reads as a word
break, turning a fourteen-page workbook into 13 255 one-character
words. And LibreOffice's output lands on whole twips and then on whole
hundredths of a millimetre, truncating positions and rounding font
heights, which accumulates to 1.5 pt down an eighty-row page.
sheet-print-{xlsx,ods} now agree with LibreOffice's own PDF on all 2281
positioned runs, within 0.006 pt across and 0.024 pt down.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
Both backends could state only a solid paint. A GradientPaint drew as its middle stop and a BitmapPaint as nothing, and nothing decoded a picture at all. Gradients. Linear becomes a PDF axial shading (/ShadingType 2) and a Skia linear shader; radial and elliptical become /ShadingType 3 and a radial shader, the ellipse's squash carried by the gradient's own transform. The colour ramp is one exponential function per pair of stops, stitched with a /FunctionType 3 when there are more than two -- which is what a:gsLst and loext:gradient-stop can say and a two-colour attribute cannot. Conical and rectangular have no native form in either backend, so both expand them through one shared band decomposition ported from Gradient::AddGradientActions, so the picture cannot depend on the backend. Bitmap fills. One image draw per tile inside a clip in PDF, an image shader with a repeat matrix in Skia, both laid on the grid Fills.Tiles computes. That is what LibreOffice writes: its PDF of the new corpus slide clips the shape and emits 47 "q ... cm /Im10 Do Q" groups sharing one 8x8 XObject. Pictures. RasterImageDecoder turns encoded bytes into the display list's RasterImage over Skia, sniffing the format by content so a mislabelled part still passes a JPEG through to DCTDecode. It is public and lives in Paperless.Rendering because Paperless.Core must stay dependency-free; a family library that wants it needs a ProjectReference it does not have yet. Measured against LibreOffice 24.2.7.2 on tests/corpus/features/paint-fills.fodp, 150 dpi, per channel: our raster vs its rendering page 1 mae 0.0016, page 2 mae 0.0018 our PDF vs its PDF, one page 1 mae 0.0007, page 2 mae 0.0000 rasteriser reading both (identical, pixel for pixel) Its page-one content stream is 91602 bytes and ours 2570: Impress decomposes every shape gradient into flat bands before its PDF writer sees it (tdf#150551), so its own shading writer is unreachable and an operator comparison of a gradient is not a question that can be asked. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
…he deck that does it
slide-table-grid.pptx is hand-written because LibreOffice's own PPTX export
writes a:noFill on every edge of every cell, so no converted deck can state a
border at all. Two slides: a fully bordered grid with three junctions whose two
cells disagree, and one with spans, all three anchors and a row that has to grow
to its text.
What the measurement changed:
- A grid line runs past its own end by half the width of the line *crossing*
it there, not half its own. The reference draws a one-point horizontal from
71.121 to 612.879 on a table spanning 72 to 612, because the verticals it
meets are two points wide.
- An Impress line box is ascent plus descent with no external leading, and
both are rounded to a whole 1/100 mm first. Liberation Sans declares a line
gap of 67/2048, so keeping it made an 18 pt line 20.70 pt where LibreOffice
draws 20.154.
Twelve strokes on the first slide and nine on the second, in the same order,
every number within 0.045 pt of the reference. PdfStrokes now takes a stroking
colour set inside the same q-group, which is where our own writer puts it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
Unit tests ask the question an image diff is worst at -- given one fill at one place, is the colour at a stated point the one the stop list says -- and ask it of both backends on the same paints, since two renderings of one IR that disagree is the bug worth catching. The fidelity tests ask whether the picture agrees with LibreOffice's, on a new corpus slide stating a linear, an axial and a radial gradient, a tiled bitmap fill and an embedded picture. The corpus slide is committed in both flat ODF and the PPTX LibreOffice exports from it, which keeps all three gradients as a:gradFill and the tile as a:blipFill with a:tile -- so the OOXML reader has a file to aim at rather than a hypothesis. PdfPaints is a new TestKit reader. It is separate from PdfFills rather than an extension of it because PdfFills finds a page by looking for BT, and a slide of four filled shapes has none; changing that rule would renumber the pages every existing comparison reports. It walks the page objects properly instead, which also fixes a trap worth naming: a shading dictionary contains the string "/ShadingType", so picking the resource list as "the dictionary mentioning /Shading" silently selects the last shading in the file and every lookup then returns nothing. 1806 passing, 0 failed, 0 skipped. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
…s agree A PDF shading's colour space is DeviceRGB and it has no alpha channel, so until now a GradientPaint whose stops carried transparency drew translucent on a PNG -- Skia's shader colours have an alpha for nothing -- and fully opaque in a PDF. One display list, two pictures. An alpha shared by every stop is now a constant /ca in an ExtGState. An alpha that varies along the ramp is a luminosity soft mask: a second shading written in DeviceGray with each stop's alpha as its grey level, painted into a transparency group whose brightness the mask reads, which is the same construction LibreOffice uses for a FLOATTRANSPARENT action (research/06-rendering.md E.3). Measured on a red-to-transparent ramp across 24 cm: our PDF rasterised by poppler against our own raster of the same fill, mean absolute difference 0.0003, ink ratio 0.999. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
Cell backgrounds, cell borders with the shared edge settled, the printed
grid, the row and column headings, and header and footer text with their
fields expanded — from ODS, XLSX and XLS alike.
The shared-edge rule is ported rather than invented: Calc resolves an edge
two neighbours both state as std::max over svx::frame::Style::operator<
(framelinkarray.cxx:796, framelink.cxx:306), which orders by total width,
then double over single, then the narrower gap, and only settles two
one-twip hairlines by their pattern. Colour is not in the ordering, so an
equal pair goes to the cell being asked about.
Unlike a Writer table, Calc emits one stroke per cell edge rather than one
per grid line, and extends an end by half the width of the perpendicular
border it meets rather than by half its own. Both measured in LibreOffice's
own PDF of the new corpus file.
Corpus: sheet-decor-{ods,xlsx,xls}, one page carrying three fills, a shared
edge stated both ways round on each axis, a printed grid and headings, and
a header and footer using &A, &P, &N and &F.
Two reader fixes the measurements forced. An ODF header band's declared
height already includes its gap to the body, so adding the gap
double-counted it by 0.25 cm a page; and a BIFF header band has a floor the
OOXML one does not, because the BIFF filter never writes an ATTR_PAGE_SIZE
and Calc's own 0.75 cm default stands.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
XLSX's styles.xml fonts and cellXfs alignment (Ooxml/XlsxCellFormats, XlsxSheetFormats), BIFF's FONT, XF, PALETTE, ROW and COLINFO (MsBinary/XlsCellFormats), and ODF's cell styles through the existing OdfStyles cascade (OpenDocument/OdsCellFormats). Each reader resolves its own style chain and hands layout a SheetCellFormat; the formats are pooled by value and kept apart from the cells, which is how a uniformly-formatted million-cell region stays cheap. An indent level is not one thing: BIFF makes it a flat 200 twips and SpreadsheetML three space widths of the workbook's default font, which for ten-point Liberation Sans is 330 twips. Both filters are right and the same workbook saved both ways indents differently. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
Phase 3 in dotnet/TODO.md and the whole of src/Paperless.Rendering/TODO.md. Recorded rather than left to be rediscovered: LibreOffice writes no shading dictionary for a shape gradient at all, so the comparison had to become picture for picture; the two ODF radial mappings a reader will need, both invisible until the colours are compared; and three new known deviations with their evidence -- one shading against hundreds of bands, poppler's uninterpolated magnification against Skia's and LibreOffice's own smooth one, and widths taken off the display list when a face's file will not load. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
…does not The rules with their citations and the measurements behind them: why a shared edge goes to the heavier border and why that is not the Writer rule, why the printed grid is black and sits at the far edge of each column, and why "thin" strokes at three quarters of a point. The named trap is the one that cost an hour: saving the corpus source as ODS moved a cell's fill onto its whole column and cancelled it seven rows down with table:style-name="Default", so "names the default style" and "names no style" are different answers and a reader that conflates them paints a cell LibreOffice leaves blank. What is left is recorded with its cost rather than as a wish: the dynamic header band is 1.03 pt on sheet-decor-xlsx.xlsx and needs font metrics in the reader; the 1/100 mm quantisation is 0.03 pt a row and needs every sheet measurement moved off EMUs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
…t row The far edge of a cell is drawn only where nothing follows to cover it, and "nothing follows" is not "last on the page": a repeated header band is placed above a block it is not adjacent to, and a hidden row leaves the same gap. Calc reaches the same answer by drawing each band through its own ScOutputData, so the last row of a repeat band keeps its bottom border. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
…he rules A corpus document exercising alignment, wrap, indent, shrink-to-fit, rotation and both ways a too-narrow number is drawn — hand-written as a flat ODF with its header and footer switched off, because Calc recomputes a dynamic band's height from the text in it and that is 1.3 pt of vertical offset on every row of a workbook that has one — plus its XLSX and XLS conversions, so all three readers answer the same questions. SheetTextComparisonTests compares operator for operator against soffice's own PDF; SheetCellTextTests covers the decisions no document demonstrates one at a time, chiefly the width-dependent General rendering and the two device-unit roundings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
… and auto numbers
The six presets that were transcribed by hand are replaced by the evaluator the
preset file is written for: guide formulas over the bounding box and the
adjustment handles, then a path of moves, lines, arcs and Beziers. The
definitions come from LibreOffice's own presetShapeDefinitions.xml, reduced to
the four things drawing needs and embedded as 110 kB of data — the same file
also serves a:custGeom, which needs the path builder and none of the formulas.
Three findings, all measured on slide-shape-features.pptx:
- An arcTo angle is a *direction*, not an ellipse parameter, and on a
non-circular arc the two differ. A 240 degree sweep on a 3:2 box ends at
249 degrees of the ellipse's own parameter — 7.6 pt from where the stated
angle alone puts it. Invisible on every rounded corner, which is why the six
hand-transcribed shapes agreed without it.
- A preset dash states no lengths: `dash` is four pen widths of ink and three
of gap, and the pen width itself rounds into 1/100 mm first, so a stated
three points is a 3.00467 pt pen and a 12.0187 pt dash.
- An arrowhead is a filled polygon and the shaft is shortened by the marker's
length less a fifteenth of its width — a deliberate overlap, so a notched
marker has no gap behind it.
Every one of the twelve preset shapes agrees with the reference: the six
straight-edged ones vertex for vertex, the six curved ones by bounding box.
Both arrowheads, three dash arrays and twelve numbered runs likewise.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
…s cost British spelling, why over what, and every measurement that settled a rule: the border-width chain that turns one point into 0.85, the arc angle that is a direction rather than an ellipse parameter, the grid line that overshoots by half of what crosses it, and the line gap Impress does not add. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
It still said the OOXML and legacy spreadsheet and presentation formats did not read at all. Every format in scope now extracts except XLSB, all three families paginate, and all three render to PDF. Records the end-to-end check that is worth repeating and that no component test makes: our PDF against LibreOffice PDF of the same file on page count *and* pdftotext word count. A document gives 41/41 and a workbook 2281/2281 — the check that catches a PDF whose glyphs land correctly but whose text cannot be extracted, which is exactly the defect it found. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
…n slides --convert-to png and --convert-to jpg both write one file for a deck, its first slide, so the PDF export is the only reference that says which slides are pages. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
The DOC layout assembler added a paragraph to the open cell only when it had text or closed the cell, so a cell padded with a run of blank paragraphs laid out as a single line. The extraction pass never dropped them, so the two passes disagreed about the same document with nothing to say which was right. Blank paragraphs in a cell are not whitespace. Each is a line of the cell's own height, and padding a cell with twenty of them is how a Word document makes a block fill the rest of its page - common in exactly the documents where pagination matters most: forms, checklists and equipment lists. Dropping them deletes the document's own vertical spacing. words/batch-018/doc/A_320.doc is the case that showed it. It pads each of its 106 equipment entries with twenty-one blank paragraphs so that each takes a page of its own, which is what LibreOffice renders: 106 of its 150 pages carry exactly one entry. Paperless carried them on 73 pages and produced 100. It now produces 135, with all 106 entries on pages of their own. The layout engine was never at fault here, and the way to establish that was to render LibreOffice's own flat-ODF export of the same document: through the ODF reader Paperless already gave every entry its own page. That is worth recording as a technique - converting a document to .fodt and rendering both tells you which side of the reader/layout boundary a difference lives on, in one step. Across the words track: total absolute page error 260 -> 223. One document, words/batch-004/doc/07-04.doc, crosses from 3/3 to 4/3 and costs a match; it was passing because a lost blank line cancelled something else that is still wrong. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
Paperless already carried LibreOffice's substitution chains from VCL.xcu and
walked them in order. What it did not carry was the other half of each entry —
`FontType` — so once a chain named nothing installed, the shape of the
substitute was guessed from the family name. On a Linux box that is the common
case rather than the exception: Tahoma's chain names fourteen faces and a
machine with only the Liberation and DejaVu families has none of them.
The guess was wrong twice over. It defaulted to serif, and it read the name for
hints ("sans", "gothic", ...) that the families which matter do not carry —
nothing in "Tahoma", "Verdana" or "Segoe UI" says grotesque. So a sans-serif
document was set in a roman, and then in Liberation rather than DejaVu.
Measured against the installed reference rather than argued from the source.
Fifty-five families rendered through LibreOffice 24.2.7.2 and through Paperless,
comparing the face each actually embedded:
agreement with LibreOffice 6/55 -> 47/55
Three changes, each reading data already in the tree:
* `FontType` is now emitted alongside the chain, as `FontFamilyClass`. 188 of
the 309 entries carry one. It replaces the name-sniffing guess outright.
* The generic fallback lists lead with DejaVu instead of Liberation. This is
where LibreOffice stops consulting its own table and asks fontconfig, whose
generic families are DejaVu on a stock configuration. Every one of the
fifty-five probes that reached this path landed on DejaVu under LibreOffice
and none on Liberation. Liberation still wins wherever a chain names it, so
Arial, Times New Roman and Courier New keep their metric-compatible
substitutes untouched.
* A blank family name is answered from `DefaultFonts`/`LATIN_TEXT`, not from
the substitution path. These are different questions: a request for a face
nobody has is a substitution, a request for nothing is the default template
speaking, and that list is serif-headed where the substitution path ends at
fontconfig's generic sans. Conflating them set every document specifying no
font in DejaVu Sans; it cost two picture-anchor tests and would have
reflowed a large part of the corpus.
The generator also no longer hardcodes an absolute checkout path, which made it
write its output into whichever tree the path named rather than the worktree it
was run from.
Corpus, at 9ac8a07, each track swept whole with frozen before/after binaries:
words match 107 -> 113 exact page count 119 -> 125
total |page error| 297 -> 231
17 documents moved: 16 closer to the reference, 0 further,
6 gained, 0 lost
slides match 133 -> 132 (ppt 42 -> 43, pptx 91 -> 89)
total |word error| 12359 -> 9331
9 gained, 10 lost, every change word-count only
The slides count falls by one while the continuous measure improves by a
quarter, and the ten losses are not a font-resolution error. Nine of the ten
now draw *fewer* words than the reference on an unchanged page count, which on
a deck means text overflowing its box rather than reflowing. The worst of them
is dominated by `MS Mincho`, 467 occurrences — and LibreOffice resolves
`MS Mincho` to DejaVu Sans here, which is exactly what this change now does and
the opposite of the Liberation Serif it used to pick. So the substitution got
closer, the text got wider by the correct amount, and what that exposes is
downstream: LibreOffice's autofit shrinks the text to fit the box where we let
it overflow and lose the tail.
That is the compensating-error pattern this corpus keeps producing — a
too-narrow face was masking an autofit shortfall, and fixing one reveals the
other. The autofit half belongs to the slides work, not here, and is left
untouched rather than papered over by keeping a font substitution we can
demonstrate is wrong.
Tests 2759, 0 skipped (2745 known-good plus 14 here).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
The brief's premise was wrong and the agent said so. Paperless already had the VCL.xcu chains and already walked them first; the defect was one layer down, in carrying only half of each entry. FontType was dropped, so once a chain named nothing installed — the common case on Linux, where Tahoma's fourteen candidates are all absent — the shape was guessed from the family name. Verifying against the installed soffice rather than its source inverted the diagnosis a second time: LibreOffice's answer for an unknown family here IS fontconfig's, and Paperless was the one hard-coding a serif/Liberation list. All 55 probes that reached the generic path landed on DejaVu under LibreOffice and none on Liberation. Chains still win where they name Liberation, so Arial, Times and Courier keep their metric-compatible substitutes. Face agreement with LibreOffice over 55 families goes 6/55 to 47/55. Words 107 to 113 matches, 119 to 125 page-exact, and total absolute page error 297 to 231 — a 22% cut, the largest single move so far. Slides falls by one, and the agent declined to revert on that. Nine of the ten losses draw fewer words on an unchanged page count, which on a deck means overflow rather than reflow: total absolute word error fell 24% over the same change. The substitution got wider by the correct amount and what it exposes is that LibreOffice's autofit shrinks text where we overflow and drop the tail. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
Three fixes to under-pagination, the largest cluster left. The header lead held and was the biggest of them. Per-page body geometry in Paginator is what makes it work: the body area now settles at the top of each page from the head that page actually draws, while the section geometry is kept for placing the furniture. An earlier attempt took the tallest header across a section's slots and over-grew two documents; this one moves page two's body from 70.5 to 97.0 pt against LibreOffice's 98.2. A_320's in-cell line pitch measurement was right and its stated cause was wrong again — there is no sprmPDyaLine anywhere in that document, and LibreOffice's own flat-ODF export of it carries no line spacing on any paragraph involved. The cause is WW8Dop.fUsePrinterMetrics, which formats against a 300 dpi printer instead of the virtual device, rounding every metric through that grid twice. The model reproduces LibreOffice on six measured face-and-size pairs exactly. The fourth fix was not on the list: blank paragraphs were dropped from DOC table cells by the layout assembler while extraction kept them, so the two passes disagreed about the same document. A_320 pads each of its 106 entries with 21 blank paragraphs so each takes a page; we were carrying them on 73. It costs one match — a document that passed because a dropped blank line cancelled something else — and removes 37 pages of error. Net for the three: +1 match, +3 page-exact, and total absolute page error 297 to 223. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
Four findings, each of which either found a defect other approaches had missed or refuted a plausible theory before it cost a round. Round-tripping a document through soffice to flat ODF and rendering that splits reading from layout in one command: LibreOffice does the reading, Paperless only the layout, so a correct render exonerates the layout engine and a wrong one exonerates the reader. It localised a WW8 bug in one step that had survived two rounds of layout theorising. A line advance can be inverted into a face. 12.65 pt at 11 pt is 2355/2048, Liberation Serif's line box; 13.45 is 2500/2048, Carlito's. That is how the largest defect in the project was identified — not a line-height rule at all, but the paragraph being laid out in the wrong font. A rule error gives a clean ratio; a substitution error gives an arbitrary one that resolves exactly against some installed font's tables. The C++ in this tree is a development branch and the soffice making the references is a release. They disagree, and the source is the more persuasive of the two, which makes it the more dangerous: the source says an unknown family falls back through LibreOffice's own generic lists, and the running binary demonstrably takes fontconfig's answer. Read the source for what mechanism exists, measure the binary for what it does. Extraction and layout are separate paths over one reader and can hold different views of the same file. Which of them has the text says whether the defect is in the assembler, in the text representation, or in reading — one command that eliminates two thirds of the search space. Also recorded: ask what a document actually contains before believing a theory about it (three diagnoses here named a mechanism the file does not use), and check whether a field is parsed but consumed by nothing, which is invisible to unit tests and how TabStop.Leader left every dotted contents line blank in 51 of 136 files. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
Match counts rose to 114/202 and 135/163, but the number to read this round by is total absolute page error: 297 to 149. The corpus moved from wrong by a lot to wrong by one page. 47 of the 88 remaining words failures are now off by exactly one page and only 12 documents are off by three or more. Also records the two premises I got wrong in the briefs, both caught by agents measuring rather than reading. The VCL table was already present and already consulted first — the defect was that it carried only half of each entry, dropping FontType, so a chain naming nothing installed fell back to guessing the shape from the family name. And the fallback direction was backwards: LibreOffice's answer for an unknown family is fontconfig's, and Paperless was the one hard-coding a serif list. Reading the source alone would have produced a confident, well-cited, wrong fix. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
A DOC's header subdocument holds six stories per section whether the section uses them or not, so the reader has to tell "no such header" from "an empty one", and emptiness is the only thing that distinguishes them. The test it made was on the paragraph's text — and a paragraph holding nothing but an inline picture reads back with no text at all, because the U+0001 that stands for the picture is consumed by the frame it makes (CollectFrame returns whether a frame was made, and that is what decides whether the anchor character stays in the string). So a logo-only running head was discarded outright, and with it the room it takes. On words/batch-018/doc/120509coss.doc — a 97.7 pt header logo and a 60.7 pt footer logo, no words in either — the body ran from the top margin to the bottom of the sheet and the document came out 20 pages against LibreOffice's 25. The test is now "no text and no frames", which leaves the placeholder paragraph Word writes for an unused slot being dropped exactly as before. tests/corpus/features/picture-furniture.doc is generated: a flat-ODF document whose header and footer each hold one PNG and nothing else, converted with `soffice --convert-to doc`. LibreOffice renders it in two pages with the body starting at 115.6 pt, which is the header's logo plus the margin above it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
…plied `IsCapitalised` was read by the DOCX extraction path and consumed by nothing, and the WW8 sprm constants for the same two toggles (0x083B, 0x083A) were declared and never matched. The layout path had no notion of case at all, so a heading Word displays as HEADING was drawn in whatever case the file stores it in. This is the same shape as the tab-leader defect: a property every reader collects and no consumer reads. It is not rare either — 43 of the corpus's 136 DOCX files declare `w:caps` and 28 declare `w:smallCaps`. The mapping is resolved in the readers rather than in the layout engine, because the engine measures and draws from `PageParagraph.Text`; a run left mapped would be measured in one case and drawn in another. `CaseMapping.Apply` does it once for all four formats, so DOC's sprms, RTF's `\caps`/`\scaps` and ODF's `fo:text-transform`/`fo:font-variant` land beside DOCX's. Two details are load-bearing. **The mapping is per character and never changes a length.** A paragraph's text is indexed by its runs, note anchors, frame anchors and bookmarks, all recorded against the stored text, so the German sharp s becoming SS would move every one of them. `char.ToUpperInvariant` is the per-character map and leaves it alone — the same guard LibreOffice spells `bCaseMapLengthDiffers`. **A small capital is drawn smaller and does not shorten its line.** Writer builds the four-fifths font locally inside `SwSubFont::DoOnCapitals` (`SMALL_CAPS_PERCENTAGE`, `include/editeng/svxfont.hxx:35`) and leaves `SwFont::GetHeight` — which is what `SwLineLayout::CalcLine` asks — reporting the unshrunken one. So `FormattedRun` grows a `MetricEmSize`: without it a heading set entirely in lowercase small capitals would draw its line four fifths as tall as the reference and repaginate the document under it. Measured across the whole words track, 202 documents: match 114 → 113, page-exact 127 → 126, total absolute page error 149 → 151. Ten documents moved. The change is right on its own evidence and the numbers say so where they can be read directly: `HC-Bulletin-template.docx` drew `Affected displaced refugees injured deaths` where the reference draws `AFFECTED DISPLACED REFUGEES INJURED DEATHS`, and its extracted word count goes 511 → 514 against the reference's 514 — exact. That document is also the one match lost, to a page count that went 5 → 6 once the text was the right width. This is the cancelling-errors case the corpus-batches skill describes, so it is committed on its own: two pagination agents are live and this moves their numbers. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
The first whole-track sweep of the sheets corpus — 174 documents, never measured before — came out at 53 of 174 matching, and the dominant failure was not a wrong page but a wrong number of them: 40 documents paginated more than five pages over the reference and 20 of those by more than a hundred. The worst gave 1170 pages against LibreOffice's 220. Three of the four causes produce blank paper, and none of them is where a page-count problem is usually looked for. A wrapping cell was widening the print area. SheetTextOverflow measured every text cell against its column and extended the printed block to cover the spill, without asking whether the cell wraps; ScColumn::GetNeededSize refuses in one line, `if ( bWidth && bBreak ) return 0;` (column2.cxx:226). A wrapping column is usually a wide column of prose, so its strings measure to thousands of points, and every point of that became empty columns and every band of empty columns a band of pages holding nothing. AFS-400_Contacts.xlsx: 340 pages against 48, 289 of ours blank and none of LibreOffice's. "Is there any cell to the left" was standing in for a measurement. That was the right trade while the measurement above was wrong; with it fixed, Calc's own test ports as written — re-run ExtendPrintArea over the block's rows from column zero and keep the page when the extension reaches it (documen9.cxx:486-500). The conservative version keeps every column band of every row band that has anything in column A, which on a wide sheet is most of the paper: RCO_VOR_Master_List_082824.xlsx, 183 pages against 80, 103 blank. A background was being treated as a border. HasAttrFlags::Lines tests the four edges of ATTR_BORDER and nothing else (attarray.cxx:1279-1284); a fill is not a border and does not keep a page. grants-2005.xls, whose shading reaches far past its cells: 1170 pages against 220, 949 blank. The fourth is the one that is not about blank pages. A BIFF SETUP marked invalid loses its scale as well as its paper: one assignment sets both flags (xipage.cxx:68) and ATTR_PAGE_SCALE is written only under `else if (maData.mbValid)` (:274-276). Twenty-three of the corpus's 87 .xls files set the bit, and the scale beside it is arbitrary — 255, 285, 300, once 20480 — so honouring it multiplies the sheet in both directions. P1636e.xls states 285 and came out on twelve pages against LibreOffice's two. Measured after, on the same documents: AFS-400 51 against 48, RCO_VOR 80 against 80, grants-2005 219 against 220, P1636e 2 against 2, UASEventsNov2014-Aug2015 523 against 523, ZenithAviation 33 against 33. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
The mirror of the header push-down that landed last round, and the same mechanism read at the other end of the page. SectionPropertyMap::PrepareHeaderFooterProperties makes the bottom margin w:footer and the footer frame's height w:bottom - w:footer, with a 1 mm floor and dynamic spacing; SwHeadFootFrame::FormatPrt then lets the footer eat the gap above it and keep growing, and it grows *upwards* because a footer frame's lower edge is pinned at the footer distance. So the body ends at min(pageHeight - w:bottom, pageHeight - w:footer - footer height). The DOC reader reaches the same pair of numbers through dyaBottom and dyaHdrBottom (wwSectionManager::GetPageULData, ww8par6.cxx:581). Paperless ended the body at the bottom margin regardless, so a footer taller than the room reserved for it was drawn over the last lines of the page and every page held three or four rows too many. The offset is in the sum because an ODF footer is top-aligned below the body rather than bottom-aligned against the page, so what it needs is the spacing above it plus its own content; a Word footer states no offset and needs only its content. Fixed heights are excluded, at both ends. Writer only grows a frame whose size is SwFrameSize::Minimum with eat-spacing on, which is what the Word importers always set — but ODF says which it means, svg:height against fo:min-height, and a fixed-height footer holding one ordinary line already overruns the room its own style reserves. Growing it shortened every page of the three ODF fidelity fixtures that state one. PageGeometry now carries HasFixedHeaderHeight and HasFixedFooterHeight, and the ODF reader is the only one that sets either. Measured on words/batch-018/doc/120509coss.doc, whose header is a 97.7 pt logo and whose footer a 60.7 pt one: with the reader fix before this it was 24 pages of LibreOffice's 25 and its body ran to 735.3 pt; it is now 25 pages with the body from 133.2 to 695.8 pt, which is LibreOffice's own body area to the twip. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
The header side of this landed last round; the footer side is the same mechanism upside down and was left out. `SectionPropertyMap::Prepare- HeaderFooterProperties` handles the two ends symmetrically — the bottom margin becomes `w:footer`, the footer frame gets `w:bottom - w:footer` with a 1 mm floor, and both get dynamic height and dynamic spacing — and `SwHeadFootFrame::FormatPrt` is written once for both. So the body's last line can sit no lower than `pageHeight - w:footer - footer height`. Measured against the reference binary rather than derived from the code alone: five synthetic DOCX files varying the footer's line count against `w:bottom`/`w:footer` reproduce LibreOffice's body-line count exactly, including the case that turns out to matter most on real files — a document whose `w:footer` *exceeds* its `w:bottom`, so the footer starts above where the body was allowed to end. Eight of the corpus's 202 word-processing documents are shaped that way and every page of each ran several points long. Words track, over the 200 documents that remain after two were withdrawn for carrying personal information: 113 -> 116 matches, 126 -> 129 page-exact, total absolute page error 147 -> 142. Six documents gained the page they were missing; four lost one, all of them documents where our own measurement of the footer's height is too large — an error this change exposes rather than introduces, since nothing consumed that number before. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
WW8 spells a hard page break and a section break with the same character,
and says which by whether a section ends at that position. This reader
took every one of them for a section break, so a Ctrl+Enter in a DOC was
silently dropped and the paragraph after it carried on down the same page.
LibreOffice looks the position up in the section PLCF and, finding no
boundary, puts an ordinary `SvxBreak::PageBefore` on the paragraph that
follows (`ww8par.cxx`:4097). The same question is asked here as "does the
next paragraph belong to another section", which needs no character-
position arithmetic: a section-terminating U+000C is the last character of
its section, so the mark after it is in the next one.
The second half is what stops that from over-paginating. A U+000C ends a
paragraph only when one is under way — `HandlePageBreakChar` adds a
paragraph end exactly under `!m_bWasParaEnd` and otherwise lets the break
settle on the empty paragraph the mark before it already opened. Closing
one regardless put a blank line above every hard page break and every
section break in the document, which is a line of height per break and
eventually a page. Both passes get the rule, because a document whose
extracted paragraphs and laid-out paragraphs disagree is a defect this
reader has shipped before.
Measured on the 66 DOC files of the words track, against reference PDFs
from LibreOffice 24.2.7.2:
match page-exact abs page error
before 38 41 60
page break only 44 47 67
with the paragraph rule 45 48 59
The whole words track — the 200 documents that remain after two were
withdrawn for carrying personal information — goes 116 -> 123 matches,
129 -> 136 page-exact, and 142 -> 141 total absolute page error.
Unproven: no committed fixture contains a U+000C that is not a section
boundary — LibreOffice's own DOC export writes a page break as
`sprmPFPageBreakBefore` rather than as the character, so one cannot be
generated here. The evidence for this is the corpus measurement above and
the reference comparisons in `SectionComparisonTests`, which cover the
section-break half.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
`PARA_SPACE_MAX_AT_PAGES` reads like a switch and is not one. It decides whether the question is asked at all; `SwFlowFrame::HasParaSpaceAtPages` (`flowfrm.cxx`:1415) then answers it per paragraph, and in the document body it grants the space only on the first page and after an explicit break, taking it away at every automatic one. Outside the body — a header, a footer, a table cell — it grants it everywhere, which is why `FlowLayouter` is untouched. Measured against LibreOffice 24.2.7.2 rather than read off the source: a DOCX whose paragraphs carry 20 pt of space-before and nothing else, A4 with a one inch margin, puts page one's first line at 92.03 pt and page two's at 72.03 pt. Paperless put both at 92, so every page after the first started 20 pt low. A caveat worth recording, because it nearly inverted this: the synthetic that establishes it needs a `word/settings.xml`, even an empty one. Without that part LibreOffice never applies its OOXML compatibility defaults, and the same file then keeps the space on page two *and* sums paragraph spacings instead of taking the larger — which would have read as evidence that `CollapsesSpacing` is backwards. Adding an empty settings part reversed both readings. Words track, over the 200 documents that remain after two were withdrawn for carrying personal information: 123 -> 124 matches, 136 -> 137 page-exact, total absolute page error 141 -> 146. The error metric moving the wrong way is worth stating plainly. Seven documents changed and every one of them lost pages, which is the change working. Five moved towards the reference (one to an exact match). The other two are `150-5370-10H.docx` and `AC-150-5370-10G-updated-201604.docx` — 721 and 697 reference pages — which were already 7 and 3 pages short and are now 11 and 8. Those two account for the whole of the +5, and their remaining error is not this rule's. Cumulative over the three changes of this round, same 200 documents: 113 -> 124 matches, 126 -> 137 page-exact, 147 -> 146 total absolute page error. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
A hand-built DOCX without word/settings.xml does not get LibreOffice's OOXML compatibility defaults, so a synthetic written to settle a compatibility question answers a different one. It fails quietly: five fixtures gave clean, consistent, mutually corroborating and wrong answers to two separate questions, and adding an empty settings part reversed both readings. The general form is worth stating — a fixture minimal enough to be obviously correct may be minimal enough to be answering a different question. And a green test that proves nothing is worse than no test. Verify a new test by reintroducing the bug and watching it fail; one written this round still passed with the defect deliberately put back and was deleted rather than shipped. Where a fix genuinely cannot be unit-tested — LibreOffice's DOC export never writes the construct the bug concerned, so no fixture could be generated — saying so and resting on a corpus measurement is a complete answer. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
The sheets track's first whole-track sweep — 174 documents, never measured before — came out at 53 matching, and the dominant failure was not a wrong page but a wrong number of them: 40 documents over five pages past the reference, 20 of those by more than a hundred, the worst giving 1170 pages against LibreOffice's 220. Three of the four causes produce blank paper, and none is where a page-count problem is usually looked for. A wrapping cell was widening the print area, because the overflow measurement never asked whether the cell wraps — a wrapping column is usually prose, so its strings measure to thousands of points and every point became empty columns. "Is there any cell to the left" was standing in for a measurement, which was the right trade only while that measurement was wrong. And a background fill was being treated as a border. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
Two agents wrote the footer eat-spacing rule independently — the second time this round that opposite-facing clusters converged on one defect. The already-merged version wins: it takes the page as well as the geometry, it was verified against five synthetics varying footer lines against w:bottom and w:footer, and it was measured at +3 matches on the corpus. Resolving the two conflicts to it drops only the duplicate. What this branch adds that the other did not is the running head whose only content is a picture. It was being discarded as empty, so a document whose letterhead is a logo lost it from every page. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
Every reader parsed them into IsCapitalised and nothing applied it — the same shape as the tab-leader defect found two rounds ago, and the second property this corpus has caught being read but never used. It was deferred twice for being risky beside live pagination agents, since capitalising a run changes glyph widths and therefore line breaks. Landing it now that those agents are merged is the right order. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
All three tracks measured at one commit for the first time — slides 135/163, words 126/200, sheets 82/171. Sheets had never been measured before this round. Its baseline was 53/174 and its first fix took it to 82, the largest single jump any track has had, because its dominant defect was a whole class rather than a scatter: not a wrong page but a wrong number of them, 20 documents over a hundred pages past the reference and the worst giving 1170 against 220. Three of four causes produce blank paper. Total page error is still 1120 there against 141 on words, so the class is not exhausted. Words has changed shape. 41 of its 74 remaining failures are off by exactly one page and total page error is down from 385 to 141 — metric errors give deltas proportional to length and these do not, so what is left is a boundary rule rather than a measurement. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
…ot draw them `sprmPFDyaBeforeAuto` (0x245B) and `sprmPFDyaAfterAuto` (0x245C) were decoded by nothing, so every paragraph a Word-or-HTML-sourced DOC marks "auto" was laid out with no margin at all where LibreOffice gives it fourteen points. Found by round-tripping `A380MaenPressRel.doc` through `soffice` to flat ODF: every paragraph in the export carries `fo:margin-top="0.1945in"` — 280 twips — and nothing in our reading of the binary produced it. That is `GetParagraphAutoSpace` (`sw/source/filter/ww8/ww8par6.cxx` :4609), whose two branches are 280 twips and, once `fDontUseHTMLAutoSpacing` is set, 100. The margin is applied in file order beside `sprmPDyaBefore`, the way `Read_ParaAutoBefore` and `Read_UL` compose on one `SvxULSpaceItem`, so whichever the document states last wins. Switched *off*, the sprm states nothing about the margin and only clears the flag. Reading the sprms alone over-paginates, because Word draws the margin at neither edge of a table cell, nor at the top of a flow, nor between two items of one list — measured: `RNW167A-150428.doc` went 9 pages to 11 and `FlightLaws.doc` 4 to 5 with only the decode in place. So all four suppressions come with it, from `FinalizeTextNode` (`ww8par.cxx`:2627-2681) and `WW8TabDesc::SetPamInCell` (`ww8par2.cxx`:2896-2935), including their ordering: the cell edges run after the list rules and overrule them. Measured over the corpus's 64 DOC files: exact page counts 47 -> 48, total absolute page error 54 -> 53, and the only document whose pagination moved is the one the round-trip named — `A380MaenPressRel.doc`, 2 pages against the reference's 3 before and 3 after. The decode itself is untested and cannot be tested from a fixture: LibreOffice reads these two sprms and never writes them, so no DOC exercising them can be produced with `soffice`, and none in `tests/corpus` contains one. The rules the decode feeds are covered instead, driven from hand-built blocks, and each was checked by putting the defect back and watching the right tests fail. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
EE_CHAR_KERNING, which DrawingML spells a:rPr/@spc and a word processor spells w:spacing: not the face's own pair kerning but a constant per character, commonly negative, that a designer uses to pull a heading in. Half the slides corpus's PPTX files state it. It is a distance *between* characters, so a run of n carries n - 1 — what SvxFont::QuickGetTextSize measures, adding one per distinct advance and then taking the trailing one back off. A prefix table can express that exactly for a paragraph measured whole and not for an arbitrary range, since the count of gaps inside a range depends on the range; charging the gap before each character is the reading that is right to the last unit for the whole and one unit generous for a line starting part-way in. The clamp afterwards is not tidiness. Every width read out of the table is a difference of two entries, and a value more negative than a character is wide would walk it backwards — a negative width fits any line. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
`SheetEmptyPages.IsBlank` returned "not blank" for any page carrying a repeated band, on the reasoning that the band does print there. Calc does not count it: `IsPrintEmpty` is asked for the page's own block alone — `IsPrintEmpty(getStartColumn(), nPageStartRow, getEndColumn(), nRow-1, …)` and the same range through `lcl_SetHidden` (sc/source/ui/view/printfun.cxx:3174, :3053) — and `PrintPage` adds the repeated band afterwards, so it never enters the question. One `if` therefore disabled the whole empty-page class for every sheet declaring `_xlnm.Print_Titles`. fy20-may20-sep20.xlsx repeats row 1 and its column F reaches only row 76, so its second column band is two pages to Calc and was 103 blank ones here: 233 pages against 96, now 118. The note this contradicts is also corrected. It claimed `bSkipEmpty` is off unless a caller passes it "and its PDF export does not"; both halves are false. `ScPrintOptions::SetDefaults` sets it true (sc/source/core/tool/printopt.cxx:38) and the render path reads the module's options straight out of it (docuno.cxx:266-268). Across sheets/batch-001…018: total absolute page error 1120 -> 1007. It costs one match — FAA-2019-0995-0002_attachment_2.xlsx goes 33 -> 31 against 33, because a page whose block holds only styled-but-empty cells is blank to us and is not to Calc. That narrowness is pre-existing and already recorded; it was masked by the guard removed here. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
`XlsxDrawings` took `xdr:sp` so the print area would be right and dropped its `xdr:txBody` on the floor, so a text box put nothing on the paper. A text box is the one thing on a sheet that no walk of the cells can find, so no check built on the grid could see it either way — the words were simply absent from every rendering. Ten of the corpus's xlsx workbooks carry shape text and 1083 words of it between them. On SSRO_Quarterly_Statistical_Bulletin_Q3201617_DATA.xlsx, whose entire methodology note is one text box, that was 163 words against LibreOffice's 550; it is now 386. Every one of the ten moved toward the reference: their total absolute word error goes 1316 -> 502, and three of them become exact matches. Across sheets/batch-001…018 the track goes 82/171 to 84/171. `Layout/SheetShapeText.cs` models the body — paragraphs, run sizes, the insets, the wrap flag and the anchor — and `Layout/SheetShapePainter.cs` draws it, wrapping by whole words and scaling the type by the print zoom as `SheetChart` already does. What it still gets wrong is one thing: the runs name `+mn-lt`, the theme's minor Latin face, and the sheet path can shape in one face only, so they are set in Liberation Sans where LibreOffice uses Carlito. Carlito is about 9% narrower, so our lines run further right and more of them fall off the paper — that is the whole of the remaining 386 against 550. It is the module's existing single-face limitation rather than a new one. `vertOverflow="clip"` is likewise not honoured; honouring it before the face is fixed would make the rendering better and the word count worse. Both are recorded in the module TODO. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
Three readings the slides track was missing, measured together on all 163 documents: 135 matched before and 146 after, with total absolute word error 8273 -> 7420 and not one previously-matching document lost. **A master's txStyles names its face indirectly, and only extraction was following it.** Two readers resolve a typeface — DrawingCharacterStyle for the content tree and PptxTextBody for the page — and the second took `<a:latin typeface="+mn-lt"/>` at face value. That is not a family, so every body placeholder in such a deck fell through to the generic sans: DejaVu Sans against the reference's Carlito, two fifths wider, so each line broke early and the tail of a full placeholder overflowed off the slide and was clipped away by the page. Nine PPTX documents were failing on it alone. It is invisible in extraction, which is why it survived. **a:rPr/@spc is the run's tracking**, in hundredths of a point and usually negative — 54 of the 112 corpus PPTX files state it, and at the commonest value of -20 a fifty-character line is ten points narrower than we drew it. **A binary deck asks for shrink-to-fit by holding outline text, not by saying so.** The format has no a:normAutofit; svdfppt.cxx turns TextFitToSizeType_AUTOFIT on for a TextHeaderAtom naming Body, HalfBody or QuarterBody and off for a title, a subtitle and an ordinary shape, then takes it away again for a shape that grows around its text instead — fFitShapeToText, or a box whose lines never wrap. berlin.ppt, 29 slides of outline placeholders, drew 1356 of the reference's 1395 words with its page count already exactly right; the missing 39 were running off the bottom of the slide. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
A sheet has one paper size and one set of margins, so a section break that does not start a page cannot re-cut the page it lands on. We switched geometry at the break, which narrows or widens the rest of that page part way down it. Writer's model forces the other answer: page geometry lives on a page style, and `SectionPropertyMap::CloseSectionGroup` gives a continuous section no page style of its own, so `InheritOrFinalizePageStyles` hands it the previous section's (`sw/source/writerfilter/dmapper/PropertyMap.cxx`:1309-1323, 1722). The paginator already reasons this way about the running head a few lines above — "a sheet has one running head" — and this is the same argument about the sheet itself. Measured on `b050-19.docx`, whose first section is one empty half-inch-margined paragraph and whose continuous second section is inch-margined. LibreOffice sets page one's text from 36 pt to 574 pt and pages two and three from 72 pt to 539 pt; we set all three from 72 pt to 539. A tenth of the measure lost on every line of page one rewrapped it and spilled the document onto a fourth page the reference does not have. Columns are the exception and still change at the break, because Writer does start a text section for them mid-page — which is also what keeps `section-columns.doc` passing. Six corpus documents declare a continuous section whose geometry differs from the one before it; four of them were failing. `b050-19.docx` 4 pages -> 3, `hdss-bulletin-issue-285-25-june-2025.docx` 11 -> 10, `Press release_EUREKA labels ITEA 3 Cluster.docx` now matches, and the two that were already passing still do. `ABCD-SDE-23-00` is still +1, from something else. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
…ofit 135/163 to 146/163, with no regressions and every page count still exact. The largest was a theme font. PptxTextBody took <a:latin typeface="+mn-lt"/> literally; that is not a family, so every body placeholder whose master states it fell through to the generic sans — DejaVu Sans against the reference's Carlito, about 39% wider — breaking every line early and pushing the tail off the slide. Extraction resolved it correctly all along, which is exactly why it survived five rounds: invisible in text, decisive on the page. Nine documents. The autofit lead held, but not where the brief pointed: PPTX autofit was already implemented, and the binary path had none. PPT never says normAutofit — svdfppt.cxx derives it from the kind of text a shape holds, a TextHeaderAtom naming Body or HalfBody getting AUTOFIT, then suppresses it for a shape that grows around its text instead. Also a:rPr/@spc, character tracking, stated by 54 of 112 corpus pptx files and never read. Its corpus contribution is unmeasured and possibly zero — only two decks state spc without also stating a theme reference, both at 0.01 pt — so it rests on unit tests and on LibreOffice's own flat-ODP export writing fo:letter-spacing for it. The agent declined to claim more. One correction to the record: Sylva introduction session.pptx had been written down twice as "the reference is wrong". It was ours — the theme font — and now matches exactly at 1115/1115. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
Two findings from the round that took slides to 146/163, both worth more than the number. The word-count gate has a ceiling and the slides track has reached it. Of 7420 residual word error, 6311 is text we draw and the reference does not, and only 267 is text we miss: on the worst page of nine documents LibreOffice rasterises an embedded object's replacement metafile, or draws nothing at all, where we emit real vector glyph runs. Our output is arguably better and wc -w scores it as failure. When over-drawing dominates under-drawing by an order of magnitude the gate is measuring a difference in rendering strategy rather than a defect, and the answer is to change instrument rather than to drive the number down by making the output worse. And "the reference is wrong" needs a higher bar than it gets. One document was recorded as "not our defect, confirmed twice" by two independent agents. It was ours — a theme font reference taken literally — and it now matches exactly. Two rounds skipped that file because the record said not to look. Name the mechanism before concluding it; "I cannot explain our output" is not evidence the other side is wrong. Also records the shape both of this round's largest defects shared: a value that only the rendering path resolves wrongly, while extraction resolves it correctly, is invisible to every text comparison and survives indefinitely. A document whose extracted text is perfect and whose page is wrong is the tell. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
`w:footer` states the footer's distance from the page's bottom edge, so it fixes the rectangle's *bottom*; the top is what gives way. We grew the rectangle downwards from where the body's text area ends instead, which is the same rectangle whenever the body ends above the footer and the wrong one when it does not. Nothing forbids a document from stating a footer distance larger than its bottom margin. `easa-form-1.docx` says `w:bottom="357" w:footer="488"`, which puts the body's edge below the footer's: the height clamped to nought and the rectangle landed at the body's edge, 131 twips too low. Measured against the reference on that document, LibreOffice ends the footer's text at 570.9 pt of a 595.35 pt landscape A4 page — the stated 488 twips from the edge — where we drew it at 577.3. After the change we draw it at 570.7. Twenty-six corpus DOCX declare a header or footer whose stated distance leaves its margin less than LibreOffice's 1 mm floor, and eleven of those cross it outright. This is a drawing fix and not a pagination one: the body's own bottom is decided by `Paginator.PulledUpBy`, which reads `FooterDistance` and the footer's advance directly and is unchanged. The corpus page and word counts are expected to be identical, and the existing `AFooterWithNoReservedHeightKeepsItsPlace` covers the case where the two edges merely meet — the new test covers the case where they cross, and fails when the old anchoring is put back. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
126/200 to 129/200, three fixed and none regressed. sprmPFDyaBeforeAuto and AfterAuto were never decoded. Found by round-tripping a document to flat ODF and seeing every paragraph come back with a 280-twip top margin we produced nowhere. The decode alone over-paginates, so the four suppressions came with it — flow top, both cell edges, inside a list — in LibreOffice's own order. A continuous section was re-cutting the sheet it lands on. A sheet has one paper size and one set of margins; Writer cannot switch mid-page because a continuous section gets no page style of its own and inherits the previous one. Columns still change at the break. And w:footer fixes the footer rectangle's bottom edge, not its top: growing it downward from the body's edge breaks whenever w:footer exceeds w:bottom. The reusable artefact from this round is not a fix. Round-tripping every failure through soffice classified all 74 in one background run — 17 reader-side, 52 layout-side, 5 page-correct — which is the split that should shape the next round rather than another page-delta histogram. Two more notes in the record turned out wrong as stated: the four "footer regressions" are heterogeneous and three are reader-side, one having a floating table in its footer and no tabs at all; and easa-form-1 is not a footer-height problem, its page-1 body being identical to the reference. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCFkGZ2rzTHM5gqrwG1P7U
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.