freepdfeditor/docs/adr/0005-rust-leaf-decoders.md

171 lines
8.7 KiB
Markdown

# ADR-0005 — Rust for the image-codec leaf decoders; C++ throughout elsewhere
* **Status**: Accepted
* **Date**: 2026-07-25
* **Plan reference**: §7.2, §15, §2.4
* **Supersedes**: the deferred-decision note in ADR-0003 (which is now resolved).
## Context
§15 lists this as an open decision with a hard deadline: "End of M0. Decide
once; do not revisit at M4." The decision must be made before M2 begins,
because the L1/L2 leaf decoders (filters, image codec glue, CMap parsing)
are written in M2, and retrofitting a second language mid-project is worse
than either choice made early (§7.2).
The threat model (§7.1) identifies parser memory corruption as the main risk.
The leaf decoders — the code that actually decompresses and decodes the bytes
of a hostile PDF's streams and images — are where the historical CVEs live.
The question is whether the memory-safety benefit of writing *those* leaves in
Rust (behind a C ABI) justifies the build-friction cost of a second language in
the tree.
ADR-0003 recorded the posture that holds regardless of this decision
(hardening on in release, bounded views, sanitizers in CI, hardened allocator,
resource budgets). This ADR records the language choice.
## Evidence gathered at M0
### Leaf-decoder CVE history (the §15 "CVE history" input)
Surveyed via the CVE record database (2026-07-25):
* **OpenJPEG** (the JPX/JPEG2000 decoder): **~40 buffer-overflow CVEs**, spanning
2016 → 2024, including `CVE-2024-56827` and `CVE-2024-56826` (heap buffer
overflow, Red Hat CNAs, recent). Several Chrome CVEs are explicitly
"OpenJPEG, as used in PDFium" (`CVE-2016-5157`, `CVE-2016-5158`,
`CVE-2016-5159`) — the *exact* leaf decoder the plan bundles (§2.4:
"JBIG2 and JPX come from PDFium's bundled decoders"). This is a high-CVE,
still-actively-found-bug surface.
* **libpng / zlib-ng / libjpeg-turbo**: comparatively mature, few recent
memory-safety CVEs (libpng: 2 CVEs in the survey, both old; zlib-ng is a
hardening fork of zlib which itself has had ~no memory-safety CVEs in years).
* **CMap parsing / PDF object parsing**: smaller surface, but it is *our* new
code (no upstream CVE history to lean on) — and it is the most reachable
surface from a hostile PDF.
### Build-friction cost (measured empirically, not estimated)
`spike/F_rust_ffi_probe/` builds a Rust leaf decoder as a `staticlib` behind a
C ABI and links it into a C++ driver:
* Rust 1.85 `cargo build --release` produces `libfpe_rust_leaf.a` in **~10.5 s**.
* The C++ driver links the `.a` by absolute path; the FFI is one `extern "C"`
function with bounded (ptr, len) arguments matching the §7.2 bounded-view
rule.
* The round-trip (C++ → Rust → C++) is **clean under ASan+UBSan** — the FFI
boundary is a normal memory-safe-by-construction interface.
* The CMake integration is one `add_custom_command` invoking `cargo` plus a
`target_link_libraries` with the `.a` path. Corrosion/FetchContent is
*not* required for a staticlib leaf.
* `panic = "abort"` + `overflow-checks = true` in the release profile mirrors
the C++ side's `-fstack-protector-strong` / overflow checks.
The measured friction is **a `cargo build` step and a `.a` link** — real, but
not the multi-day toolchain fight a full Rust integration used to be. The
decision is not blocked by build cost.
## Decision
**Rust for the image-codec leaf decoders; C++ throughout everywhere else.**
Concretely:
1. **In Rust** (built as `staticlib`, behind a C ABI, linked from C++):
the **image codec glue** that wraps OpenJPEG, libjpeg-turbo, libpng,
OpenJPEG/JBIG2 — the leaf where the CVE history is concentrated. The Rust
layer owns the byte-buffer sizing, the allocation, and the call into the C
codec; the C codec's output is bounded-checked on the Rust side before it
crosses back. This is the narrowest place a second language buys the most:
it sits between hostile input and the C decoders that have the CVEs.
2. **In C++** (the rest of the tree, unchanged): the L1 file layer, the L2
object model, the L3 content model, the L4 layout engine, the L5 semantic
model, the L6 command layer, the UI, the rendering, the sandbox, the IPC.
No second language enters these. CMap parsing and the PDF object parser
stay in C++ with the ADR-0003 hardening (bounded views, sanitizers, checked
arithmetic) — they are our new code with no upstream CVE history, and the
bounded-view discipline plus fuzzing (§8.2) is the right posture for them.
3. **The image codec glue is the *only* Rust in the tree.** No Rust in the UI,
no Rust in the rendering pipeline, no Rust in the model. The build stays
single-language for everyone who isn't touching the codec glue.
## Consequences
**Positive.**
* The highest-CVE leaf (the image decoders, especially OpenJPEG/JPX where the
40-CVE history lives and where "as used in PDFium" appears) gets a
memory-safe wrapper. A buffer overflow in the C codec is caught at the
Rust boundary's bounds check rather than corrupting the document process
heap. This is a real reduction in the §7.1 "parser memory corruption" risk
for exactly the surface that has historically been most exploited.
* The decision is narrow and defensible: a second language only where its
benefit (memory safety for the highest-CVE leaf) clearly exceeds its cost.
The rest of the tree pays no Rust tax.
* The build-friction cost was measured, not guessed: ~10 s for a `cargo
build` of a leaf crate, one `.a` link. CI gets one more build step; local
builds for engineers not touching the codec glue are unaffected (the Rust
staticlib is a build dependency like any other).
**Negative.**
* A second language enters the tree. CI needs `cargo` on the runners (the
Gitea Actions matrix already runs on Debian where `apt install rustc cargo`
is one line; the nightly packaging pipeline must bundle the Rust staticlib
into the artifacts). Contributor friction for the small number of people
who touch the codec glue: they need both toolchains.
* Debugging across the FFI boundary is harder than debugging within one
language — stack traces cross the boundary, and a `panic = "abort"` Rust
leaf crashes the process without unwinding. The §7.2 sanitizers + Crashpad
still work; the crash is just louder. This is acceptable for a leaf that
should never panic in practice (it does bounded byte work).
* The Rust leaf must be kept *leaf*: if it grows a dependency on the C++
object model it stops being a leaf and the boundary erodes. CODEOWNERS and
review enforce this; the C ABI is the contract.
**Neutral.**
* `panic = "abort"` matches the C++ side's no-unwinding-across-FFI stance.
* The bounded-view (ptr, len) FFI shape is the same shape the §7.2 rule
mandates for C++ parsing, so the two sides agree on the discipline.
## Alternatives considered
* **C++ throughout (no Rust).** Rejected. The OpenJPEG CVE history
(~40 buffer overflows, including 2024 CVEs, several "as used in PDFium") is
exactly the surface the §7.1 threat model worries about, and it is the
surface where C++ has demonstrably, repeatedly failed. The measured
build-friction cost (~10 s `cargo build`, one `.a` link) does not justify
forgoing the memory-safe wrapper for that specific leaf. The §15 instruction
to "decide once" means the C++-throughout choice would also be permanent;
accepting 40 more years of OpenJPEG-class CVEs in our highest-reach surface
is the wrong permanent choice.
* **Rust for all of L1/L2 (the whole parser surface).** Rejected. CMap
parsing and the object parser are *our* new code with no upstream CVE
history; the ADR-0003 hardening posture (bounded views, sanitizers,
resource budgets, fuzzing) is the right tool for new C++ code, and rewriting
it all in Rust would spread the second language across the most-coupled
part of the tree (the object model touches everything). The marginal
safety benefit over hardened+sanitized+fuzzed new C++ does not justify the
coupling cost. The leaf is where the benefit concentrates; this decision
keeps Rust there.
* **Rust for the whole document process.** Rejected strongly; Qt, Skia,
PDFium, HarfBuzz, FreeType, ICU, OpenSSL are all C/C++. A Rust document
process would FFI into all of them anyway, gaining nothing over a C++
process with a Rust leaf, and losing the Qt/Skia integration the UI needs.
## Reproducing the evidence
```bash
# Build-friction probe:
cmake --build build/manual --target spike_f_rust_ffi
build/manual/bin/spike_f_rust_ffi 1024 # round-trips 1024 bytes via Rust C ABI
# CVE history (surveyed 2026-07-25):
# OpenJPEG: https://www.cve.org/CVERecord/SearchResults?query=openjpeg+buffer+overflow (~40)
# libpng: https://www.cve.org/CVERecord/SearchResults?query=libpng+memory+corruption (2)
```