ai-ad4
/
freepdfeditor
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
freepdfeditor/docs/adr/0005-rust-leaf-decoders.md

8.7 KiB
Raw Blame History

ADR-0005 — Rust for the image-codec leaf decoders; C++ throughout elsewhere

  • Status: Accepted
  • Date: 2026-07-25
  • Plan reference: §7.2, §15, §2.4
  • Supersedes: the deferred-decision note in ADR-0003 (which is now resolved).

Context

§15 lists this as an open decision with a hard deadline: "End of M0. Decide once; do not revisit at M4." The decision must be made before M2 begins, because the L1/L2 leaf decoders (filters, image codec glue, CMap parsing) are written in M2, and retrofitting a second language mid-project is worse than either choice made early (§7.2).

The threat model (§7.1) identifies parser memory corruption as the main risk. The leaf decoders — the code that actually decompresses and decodes the bytes of a hostile PDF's streams and images — are where the historical CVEs live. The question is whether the memory-safety benefit of writing those leaves in Rust (behind a C ABI) justifies the build-friction cost of a second language in the tree.

ADR-0003 recorded the posture that holds regardless of this decision (hardening on in release, bounded views, sanitizers in CI, hardened allocator, resource budgets). This ADR records the language choice.

Evidence gathered at M0

Leaf-decoder CVE history (the §15 "CVE history" input)

Surveyed via the CVE record database (2026-07-25):

  • OpenJPEG (the JPX/JPEG2000 decoder): ~40 buffer-overflow CVEs, spanning 2016 → 2024, including CVE-2024-56827 and CVE-2024-56826 (heap buffer overflow, Red Hat CNAs, recent). Several Chrome CVEs are explicitly "OpenJPEG, as used in PDFium" (CVE-2016-5157, CVE-2016-5158, CVE-2016-5159) — the exact leaf decoder the plan bundles (§2.4: "JBIG2 and JPX come from PDFium's bundled decoders"). This is a high-CVE, still-actively-found-bug surface.
  • libpng / zlib-ng / libjpeg-turbo: comparatively mature, few recent memory-safety CVEs (libpng: 2 CVEs in the survey, both old; zlib-ng is a hardening fork of zlib which itself has had ~no memory-safety CVEs in years).
  • CMap parsing / PDF object parsing: smaller surface, but it is our new code (no upstream CVE history to lean on) — and it is the most reachable surface from a hostile PDF.

Build-friction cost (measured empirically, not estimated)

spike/F_rust_ffi_probe/ builds a Rust leaf decoder as a staticlib behind a C ABI and links it into a C++ driver:

  • Rust 1.85 cargo build --release produces libfpe_rust_leaf.a in ~10.5 s.
  • The C++ driver links the .a by absolute path; the FFI is one extern "C" function with bounded (ptr, len) arguments matching the §7.2 bounded-view rule.
  • The round-trip (C++ → Rust → C++) is clean under ASan+UBSan — the FFI boundary is a normal memory-safe-by-construction interface.
  • The CMake integration is one add_custom_command invoking cargo plus a target_link_libraries with the .a path. Corrosion/FetchContent is not required for a staticlib leaf.
  • panic = "abort" + overflow-checks = true in the release profile mirrors the C++ side's -fstack-protector-strong / overflow checks.

The measured friction is a cargo build step and a .a link — real, but not the multi-day toolchain fight a full Rust integration used to be. The decision is not blocked by build cost.

Decision

Rust for the image-codec leaf decoders; C++ throughout everywhere else.

Concretely:

  1. In Rust (built as staticlib, behind a C ABI, linked from C++): the image codec glue that wraps OpenJPEG, libjpeg-turbo, libpng, OpenJPEG/JBIG2 — the leaf where the CVE history is concentrated. The Rust layer owns the byte-buffer sizing, the allocation, and the call into the C codec; the C codec's output is bounded-checked on the Rust side before it crosses back. This is the narrowest place a second language buys the most: it sits between hostile input and the C decoders that have the CVEs.

  2. In C++ (the rest of the tree, unchanged): the L1 file layer, the L2 object model, the L3 content model, the L4 layout engine, the L5 semantic model, the L6 command layer, the UI, the rendering, the sandbox, the IPC. No second language enters these. CMap parsing and the PDF object parser stay in C++ with the ADR-0003 hardening (bounded views, sanitizers, checked arithmetic) — they are our new code with no upstream CVE history, and the bounded-view discipline plus fuzzing (§8.2) is the right posture for them.

  3. The image codec glue is the only Rust in the tree. No Rust in the UI, no Rust in the rendering pipeline, no Rust in the model. The build stays single-language for everyone who isn't touching the codec glue.

Consequences

Positive.

  • The highest-CVE leaf (the image decoders, especially OpenJPEG/JPX where the 40-CVE history lives and where "as used in PDFium" appears) gets a memory-safe wrapper. A buffer overflow in the C codec is caught at the Rust boundary's bounds check rather than corrupting the document process heap. This is a real reduction in the §7.1 "parser memory corruption" risk for exactly the surface that has historically been most exploited.
  • The decision is narrow and defensible: a second language only where its benefit (memory safety for the highest-CVE leaf) clearly exceeds its cost. The rest of the tree pays no Rust tax.
  • The build-friction cost was measured, not guessed: ~10 s for a cargo build of a leaf crate, one .a link. CI gets one more build step; local builds for engineers not touching the codec glue are unaffected (the Rust staticlib is a build dependency like any other).

Negative.

  • A second language enters the tree. CI needs cargo on the runners (the Gitea Actions matrix already runs on Debian where apt install rustc cargo is one line; the nightly packaging pipeline must bundle the Rust staticlib into the artifacts). Contributor friction for the small number of people who touch the codec glue: they need both toolchains.
  • Debugging across the FFI boundary is harder than debugging within one language — stack traces cross the boundary, and a panic = "abort" Rust leaf crashes the process without unwinding. The §7.2 sanitizers + Crashpad still work; the crash is just louder. This is acceptable for a leaf that should never panic in practice (it does bounded byte work).
  • The Rust leaf must be kept leaf: if it grows a dependency on the C++ object model it stops being a leaf and the boundary erodes. CODEOWNERS and review enforce this; the C ABI is the contract.

Neutral.

  • panic = "abort" matches the C++ side's no-unwinding-across-FFI stance.
  • The bounded-view (ptr, len) FFI shape is the same shape the §7.2 rule mandates for C++ parsing, so the two sides agree on the discipline.

Alternatives considered

  • C++ throughout (no Rust). Rejected. The OpenJPEG CVE history (~40 buffer overflows, including 2024 CVEs, several "as used in PDFium") is exactly the surface the §7.1 threat model worries about, and it is the surface where C++ has demonstrably, repeatedly failed. The measured build-friction cost (~10 s cargo build, one .a link) does not justify forgoing the memory-safe wrapper for that specific leaf. The §15 instruction to "decide once" means the C++-throughout choice would also be permanent; accepting 40 more years of OpenJPEG-class CVEs in our highest-reach surface is the wrong permanent choice.

  • Rust for all of L1/L2 (the whole parser surface). Rejected. CMap parsing and the object parser are our new code with no upstream CVE history; the ADR-0003 hardening posture (bounded views, sanitizers, resource budgets, fuzzing) is the right tool for new C++ code, and rewriting it all in Rust would spread the second language across the most-coupled part of the tree (the object model touches everything). The marginal safety benefit over hardened+sanitized+fuzzed new C++ does not justify the coupling cost. The leaf is where the benefit concentrates; this decision keeps Rust there.

  • Rust for the whole document process. Rejected strongly; Qt, Skia, PDFium, HarfBuzz, FreeType, ICU, OpenSSL are all C/C++. A Rust document process would FFI into all of them anyway, gaining nothing over a C++ process with a Rust leaf, and losing the Qt/Skia integration the UI needs.

Reproducing the evidence

# Build-friction probe:
cmake --build build/manual --target spike_f_rust_ffi
build/manual/bin/spike_f_rust_ffi 1024        # round-trips 1024 bytes via Rust C ABI

# CVE history (surveyed 2026-07-25):
#   OpenJPEG:   https://www.cve.org/CVERecord/SearchResults?query=openjpeg+buffer+overflow  (~40)
#   libpng:     https://www.cve.org/CVERecord/SearchResults?query=libpng+memory+corruption  (2)