Text2Store

Developer notes

The 84,000-message thread

How webview PDF export fails at scale (slowly, then silently, then fatally) and the chunk-gate-merge architecture that fixed it, with the PDF internals Acrobat forced us to learn.

The field report that started this chapter: a friend's iPhone with an 84,000-message conversation on it, ten years of texting. Export to PDF. Crash. A second report from another real archive: an 8,400-message thread produced an 811-page PDF that was fine until page 583 and blank after it. No error anywhere. The renderer just stopped painting and kept counting pages.

Why webview PDF export breaks at scale

Like most apps with a pretty transcript, we render messages as HTML and ask the platform webview to print to PDF. That machinery was built for web pages. Hand it a single document with tens of thousands of messages and thousands of images and it fails in escalating stages: first slowly, then by silently emitting blank pages as its memory tightens, and finally by the web content process dying outright. The silent stage is the dangerous one. Blank pages after page 583 means a user hands an incomplete document to whoever needed it, and nobody knows.

Chunk, gate, merge

  • Chunk. The transcript renders at most 1,500 messages per pass; each pass prints its own part file. No single print operation is ever large enough to reach the cliff.
  • Gate. Every written part is parsed and inspected: if no page paints anything (each content stream literally the empty q Q), the part is discarded, retried once, and if still blank, split in half and re-attempted, recursively. Nondeterministic blanks heal on retry; deterministic ones isolate down to the single message that triggers them, and the export fails naming it rather than shipping a defective document.
  • Merge. The parts are stitched into one PDF in Rust, and the merged file is re-verified at write time before we call it done.
The export menu on a conversation, with PDF, CSV, plain text, verified export, and RSMF.
The user sees one menu item. Everything in this post happens between that click and the file appearing.

A multi-part export announces itself honestly in each part's header, so a reader always knows what they are holding:

Part 3 of 9 · messages 12,001–13,500 of 84,000

What Acrobat taught us about PDF internals

The merge sounds trivial and is not, because the strictest PDF reader in the world is the one lawyers use. Three lessons, each paid for with a real broken export:

  • Write a classic xref table, not a cross-reference stream. Our PDF library's default 1.5-style xref stream omitted the object-0 free-list head from its index. Preview opened the file happily. Acrobat refused it entirely.
  • Never trust a library's max object id. Real webview print output contains broken zero-offset xref entries; deriving the next id range from max-id under-reports, the next part's renumbered objects silently overwrite the previous part's, and you get a corrupt hybrid page tree. We derive ranges from the objects that actually exist and abort on any collision.
  • Rebuild the page tree from scratch. Past roughly ten pages, WebKit emits nested page trees whose intermediate nodes carry parent references; naively merging dictionaries gave the root a parent array, which Acrobat rightly rejects. The merged root is built fresh: type, count, kids, nothing else.

The write-time verifier now checks every one of those invariants on every merged file, so a regression fails in milliseconds on our machine instead of in a law office.

The numbers

The 84,000-message thread exports to a single readable PDF. Same thread, early version: 1.05 GB. Current engine: 164 MB. A photo-heavy 2,500-message conversation exports in about 17 seconds. Multi-part exports label each part by its message range, and if the document omits anything (recoverable deleted messages, a date limit), it says so on the document itself.

One more honest note: the interactive print dialog has the same single-pass limits, so for very long threads the app routes you to PDF export instead of letting the dialog fail. And all of this was still not the end of the story: fixing exports revealed a memory problem that took a week of process-lifecycle archaeology, which is the next chapter.