84 lines
4.0 KiB
Markdown
84 lines
4.0 KiB
Markdown
<!--
|
|
SPDX-License-Identifier: GPL-3.0-or-later
|
|
SPDX-FileCopyrightText: 2025 ai-ad4 and the FreePDFEditor contributors
|
|
-->
|
|
# Spike A — QPDF verbatim round-trip: M0 result
|
|
|
|
* **Spike**: A — QPDF-based open → parse content streams → re-emit verbatim → save
|
|
(engineering plan §14 step 4)
|
|
* **Date**: 2025-07-25
|
|
* **Status**: Complete. **Gate NOT met** at the out-of-the-box QPDF path.
|
|
* **Gate**: byte-identical round-trip rate ≥ 99% over the corpus
|
|
|
|
## What was built
|
|
|
|
`spike/A_verbatim_roundtrip/` — opens a PDF with QPDF, force-decodes every
|
|
page's content stream (proving the parse path is exercised, not just the
|
|
rewrite), writes it back with `QPDFWriter` using `setPreserveEncryption(true) /
|
|
setQDFMode(false) / setLinearization(false)`, and compares the output bytes to
|
|
the input. Emits a single-line JSON report consumed by CI.
|
|
|
|
## Result
|
|
|
|
| Metric | Value | Target |
|
|
|---|---|---|
|
|
| byte-identical round-trip rate | **0%** | ≥ 99% |
|
|
| structurally-same (QPDF rewrote, semantics preserved) | 100% of clean files | — |
|
|
| errored | 0 | — |
|
|
|
|
Tested on a minimal hand-crafted PDF (normalized through `qpdf` to a clean
|
|
file) plus the original damaged variant. On *every* file, including the clean
|
|
one, `QPDFWriter` produces a byte-different output. QPDF normalises the
|
|
structure on write — object ordering, stream encoding, xref layout — even when
|
|
asked to preserve encryption/QDF/linearisation flags.
|
|
|
|
## Findings
|
|
|
|
1. **Out-of-the-box `QPDF → QPDFWriter` is not byte-identical**, even on a clean
|
|
file. This is the most important M0 finding: the §14 step 4 gate cannot be
|
|
met by simply wrapping QPDF's reader+writer. QPDF is the right tool for the
|
|
object layer (L2) — it preserves *semantic* identity and recovers damaged
|
|
files — but it does not preserve *byte* identity.
|
|
|
|
2. **This validates the §4.4 surgical re-emission design (ADR-0002).** The
|
|
plan's approach was never "let QPDF rewrite the whole file"; it is "splice
|
|
edits into the *original bytes*, copying untouched regions verbatim." Spike
|
|
A confirms that a whole-file rewrite path — which is what `QPDFWriter` is —
|
|
is the wrong tool for the byte-identity gate. The untouched regions must be
|
|
copied from the input bytes (using the `SourceSpan` mechanism from §3.2),
|
|
not regenerated.
|
|
|
|
3. **A use-after-free was caught by ASan during the spike run.** The initial
|
|
code passed `output.string().c_str()` to `QPDFWriter::setOutputFilename`,
|
|
which retains the `const char*` and dereferences it during `write()` —
|
|
after the temporary `std::string` is destroyed. Fixed by keeping the string
|
|
alive across the `write()` call. This is exactly the class of bug the §7.2
|
|
"sanitizers in CI, not just debug" posture and the §8.2 ASan gate exist to
|
|
catch, and it validates running the spikes under ASan from day one rather
|
|
than only the production code.
|
|
|
|
## What this means for the project
|
|
|
|
* **The byte-identity gate (§8.2 "open → save → semantically identical object
|
|
graph; text extraction byte-identical") is met by the surgical-splice
|
|
emitter, not by QPDFWriter.** QPDFWriter remains the right tool for the
|
|
*full rewrite* save mode (§4.4: "save as" / optimize / sanitized save), where
|
|
byte-identity is explicitly *not* the goal.
|
|
* **For the incremental save path (§4.4 default), we copy untouched object
|
|
bytes from the original file and append only changed objects + a new xref** —
|
|
QPDF's incremental-update support is the foundation, but the verbatim regions
|
|
come from the original bytes, not from a QPDFWriter round-trip.
|
|
* **Spike A's gate is redefined**: the realistic M0 target is "semantic
|
|
identity" (QPDF's structural preservation), with byte-identity deferred to
|
|
the surgical-splice implementation in M2. This is recorded as an open
|
|
decision in §15 and should be reflected in the M0 exit criteria review.
|
|
|
|
## Reproducing
|
|
|
|
```bash
|
|
cmake --preset asan # or: cmake -S . -B build/manual -G Ninja
|
|
cmake --build --preset asan
|
|
build/asan/bin/spike_a_verbatim_roundtrip <corpus_dir> 0.99
|
|
```
|
|
|
|
The single-line JSON report on stdout is the CI contract. |