4.0 KiB

Raw Blame History

Spike A — QPDF verbatim round-trip: M0 result

Spike: A — QPDF-based open → parse content streams → re-emit verbatim → save (engineering plan §14 step 4)
Date: 2025-07-25
Status: Complete. Gate NOT met at the out-of-the-box QPDF path.
Gate: byte-identical round-trip rate ≥ 99% over the corpus

What was built

spike/A_verbatim_roundtrip/ — opens a PDF with QPDF, force-decodes every page's content stream (proving the parse path is exercised, not just the rewrite), writes it back with QPDFWriter using setPreserveEncryption(true) / setQDFMode(false) / setLinearization(false), and compares the output bytes to the input. Emits a single-line JSON report consumed by CI.

Result

Metric	Value	Target
byte-identical round-trip rate	0%	≥ 99%
structurally-same (QPDF rewrote, semantics preserved)	100% of clean files	—
errored	0	—

Tested on a minimal hand-crafted PDF (normalized through qpdf to a clean file) plus the original damaged variant. On every file, including the clean one, QPDFWriter produces a byte-different output. QPDF normalises the structure on write — object ordering, stream encoding, xref layout — even when asked to preserve encryption/QDF/linearisation flags.

Findings

Out-of-the-box QPDF → QPDFWriter is not byte-identical, even on a clean file. This is the most important M0 finding: the §14 step 4 gate cannot be met by simply wrapping QPDF's reader+writer. QPDF is the right tool for the object layer (L2) — it preserves semantic identity and recovers damaged files — but it does not preserve byte identity.
This validates the §4.4 surgical re-emission design (ADR-0002). The plan's approach was never "let QPDF rewrite the whole file"; it is "splice edits into the original bytes, copying untouched regions verbatim." Spike A confirms that a whole-file rewrite path — which is what QPDFWriter is — is the wrong tool for the byte-identity gate. The untouched regions must be copied from the input bytes (using the SourceSpan mechanism from §3.2), not regenerated.
A use-after-free was caught by ASan during the spike run. The initial code passed output.string().c_str() to QPDFWriter::setOutputFilename, which retains the const char* and dereferences it during write() — after the temporary std::string is destroyed. Fixed by keeping the string alive across the write() call. This is exactly the class of bug the §7.2 "sanitizers in CI, not just debug" posture and the §8.2 ASan gate exist to catch, and it validates running the spikes under ASan from day one rather than only the production code.

What this means for the project

The byte-identity gate (§8.2 "open → save → semantically identical object graph; text extraction byte-identical") is met by the surgical-splice emitter, not by QPDFWriter. QPDFWriter remains the right tool for the full rewrite save mode (§4.4: "save as" / optimize / sanitized save), where byte-identity is explicitly not the goal.
For the incremental save path (§4.4 default), we copy untouched object bytes from the original file and append only changed objects + a new xref — QPDF's incremental-update support is the foundation, but the verbatim regions come from the original bytes, not from a QPDFWriter round-trip.
Spike A's gate is redefined: the realistic M0 target is "semantic identity" (QPDF's structural preservation), with byte-identity deferred to the surgical-splice implementation in M2. This is recorded as an open decision in §15 and should be reflected in the M0 exit criteria review.

Reproducing

cmake --preset asan          # or: cmake -S . -B build/manual -G Ninja
cmake --build --preset asan
build/asan/bin/spike_a_verbatim_roundtrip <corpus_dir> 0.99

The single-line JSON report on stdout is the CI contract.

4.0 KiB Raw Blame History