1048 lines
66 KiB
Markdown
1048 lines
66 KiB
Markdown
# FreePDFEditor — Engineering Plan
|
||
|
||
A cross-platform native desktop PDF editor with a true WYSIWYG interface and **full content editing**:
|
||
edit existing text with reflow, replace images, manipulate vector objects, plus annotations,
|
||
forms, signatures, and page assembly.
|
||
|
||
**Stack:** C++20 + Qt 6.7 (Widgets for the shell, custom canvas), CMake, vcpkg.
|
||
**Targets:** Windows 10+, macOS 12+ (Intel + Apple Silicon), Linux (X11/Wayland).
|
||
**Distribution:** Linux `.deb` / `.AppImage` / portable tarball, Windows `.msi` / `.exe`, macOS
|
||
`.dmg` / `.pkg` — see §12.
|
||
**Source control:** every code change is pushed to `gitea.lm.je` — see §13.
|
||
|
||
---
|
||
|
||
## 1. Product definition
|
||
|
||
### 1.1 What "full content editing" means here
|
||
|
||
The editor parses each page's content stream into a semantic document model, lets the user
|
||
manipulate that model directly on the rendered page, and **re-emits** the content stream on save.
|
||
This is fundamentally different from the common "overlay" approach (cover old text with a white
|
||
rectangle, draw new text on top), which produces files that look right but are broken for search,
|
||
copy-paste, accessibility, and redaction.
|
||
|
||
Consequences of choosing the hard path:
|
||
- We own a content-stream interpreter *and* emitter, and both must be lossless.
|
||
- We own font subsetting and glyph→Unicode mapping.
|
||
- We own text layout (shaping + line breaking) for edited regions.
|
||
- We own the structure tree when content moves under it (§3.4).
|
||
- Fidelity regressions are the #1 product risk and need continuous automated measurement.
|
||
|
||
### 1.2 Feature scope
|
||
|
||
**Tier 1 — must ship at 1.0**
|
||
- View: all PDF 1.0–2.0 features, ≥99.5% pixel-parity with reference renderers on the test corpus.
|
||
- Text: click into any text block, edit inline, reflow within the block, styling (font, size,
|
||
color, spacing, alignment), find & replace across the document.
|
||
- Images: select, move/scale/rotate/crop, replace, delete, extract, adjust resolution.
|
||
- Vectors: select paths, move/scale/rotate, edit fill/stroke, delete; node-level editing.
|
||
- Pages: insert, delete, reorder, rotate, crop/resize, split, merge, extract, N-up; create a new
|
||
blank document; build a document from images; combine files by drag-and-drop.
|
||
- Annotations: highlight/underline/strikeout, notes, freehand ink, shapes, text boxes, stamps,
|
||
measurement; full comment thread panel with reply/resolve; import/export FDF/XFDF.
|
||
- Forms: fill AcroForm fields; create/edit fields (text, checkbox, radio, combo, list, button,
|
||
signature); tab order; JavaScript-free validation/format actions; export/import form data.
|
||
- Redaction: true content removal (glyphs, image pixels, paths, annotations, hidden metadata).
|
||
- Signatures: validate existing (PAdES B-B..B-LTA), sign via PKCS#11 / OS keystore.
|
||
- Security: open encrypted (RC4 40/128, AES-128, AES-256 R6, legacy R5 read-only), set
|
||
permissions/passwords, remove security when the owner password is known.
|
||
- Structure: bookmarks/outline, links and actions, document metadata (Info + XMP), attachments,
|
||
optional content groups (visibility, rename, delete, assign objects to a layer).
|
||
- **Tag preservation**: editing text, images or paths never silently drops or corrupts the
|
||
structure tree, `ActualText`/`Alt`, or the tab order of a tagged PDF (§3.4). *Authoring* tags is
|
||
Tier 2; *not destroying* them is Tier 1.
|
||
- Search: cross-document, diacritic- and ligature-folded, regex option, results panel.
|
||
- Clipboard: copy/paste of text, images and objects within and between documents; paste from and
|
||
copy to external apps as text / PNG / SVG.
|
||
- Print: page scaling, booklet, N-up, tiling, "print as image" fallback, per-printer profiles.
|
||
- Export: PDF, PDF/A-2b & PDF/A-3b, PNG/JPEG/TIFF, plain text, HTML, DOCX (best-effort).
|
||
|
||
**Tier 2 — post-1.0**
|
||
- Cross-page text stories (reflow overflowing into the next frame/page).
|
||
- OCR for scanned pages (Tesseract) producing an invisible text layer.
|
||
- Tagged-PDF structure *authoring* and a full accessibility checker (PDF/UA).
|
||
- Table detection and cell-level editing.
|
||
- Compare-documents view; batch/CLI processing; plugin API; scanner acquisition.
|
||
|
||
**Explicit non-goals for 1.0**
|
||
- XFA forms. Dynamic XFA is not renderable without an XFA engine, and there is no credible
|
||
GPL-compatible one. Behaviour: detect XFA, render the AcroForm fallback appearance streams,
|
||
open the document **read-only** with a clear banner explaining why. We never write a file that
|
||
silently invalidates its XFA packet.
|
||
- Embedded PDF JavaScript execution (we parse and preserve actions but never run them).
|
||
- PDF Portfolios/Collections as a browsable UI — we display the cover sheet, list the embedded
|
||
files, and preserve the collection dictionary on save.
|
||
- Real-time multi-user collaboration.
|
||
- Full print-production preflight (Acrobat/PitStop territory).
|
||
|
||
---
|
||
|
||
## 2. Architecture
|
||
|
||
### 2.1 Process model
|
||
|
||
Two processes. This is a security requirement, not an optimization: PDF parsers are among the most
|
||
heavily exploited attack surfaces in desktop software, and we are writing a new one.
|
||
|
||
```
|
||
┌───────────────────────────────┐ ┌──────────────────────────────────┐
|
||
│ UI process (Qt) │ │ Document process (sandboxed) │
|
||
│ - canvas, panels, tools │ ◄────► │ - file I/O, parsing │
|
||
│ - command dispatch / undo │ IPC │ - object model, content streams │
|
||
│ - render surface compositing │ │ - rasterization │
|
||
│ - no untrusted parsing │ │ - re-emission and save │
|
||
└───────────────────────────────┘ └──────────────────────────────────┘
|
||
```
|
||
|
||
- IPC: local socket, Cap'n Proto messages, shared memory for raster tiles (no copies on the hot
|
||
path).
|
||
- Sandbox: seccomp-bpf + user namespaces (Linux), App Sandbox with a minimal entitlement set
|
||
(macOS), AppContainer + low-integrity token (Windows). The document process gets **no** network
|
||
access and no filesystem access beyond fds handed to it by the UI process.
|
||
- One document process per open document. A crash loses one tab, not the app; the UI process
|
||
restarts it and replays the journal (§6.4).
|
||
|
||
**The IPC boundary is a trust boundary in both directions.** The document process handles hostile
|
||
input, so anything it sends is itself untrusted: the UI validates every message (bounds, counts,
|
||
string encodings, shared-memory extents) and must never index, allocate, or size-compute directly
|
||
from a document-supplied number. Cap'n Proto's arena reader limits are set explicitly rather than
|
||
left at defaults. Fuzz the UI-side deserializer as its own harness (§8.2) — a compromised document
|
||
process attacking the unsandboxed UI is the obvious escape path and the one people forget.
|
||
|
||
### 2.2 Layer stack (document process)
|
||
|
||
```
|
||
L6 Command layer transactional edits, undo/redo, journaling
|
||
L5 Semantic model text frames, paragraphs, images, path objects, groups
|
||
L4 Layout engine shaping (HarfBuzz), line breaking, frame reflow
|
||
L3 Content model operator stream ↔ display list, graphics state machine
|
||
L2 Object model PDF objects, xref, streams, filters, encryption
|
||
L1 File layer byte access, incremental updates, repair
|
||
L0 Platform mmap I/O, threads, allocators
|
||
```
|
||
|
||
Rendering hangs off L3: the display list feeds the rasterizer. Rendering is **never** driven by L5
|
||
directly — see §2.5.
|
||
|
||
### 2.3 Threading model
|
||
|
||
Inside the document process:
|
||
- **One model thread** owns L1–L6. All mutation happens here; the model is single-writer and needs
|
||
no internal locking. Commands arrive on a queue.
|
||
- **A render worker pool** (N = cores − 1) holds immutable *snapshots*. A snapshot is the
|
||
copy-on-write object graph at a given content revision plus the emitted page bytes; workers never
|
||
see a partially-applied edit. Snapshots are refcounted and released when their tiles age out.
|
||
- **One I/O thread** for read-ahead, journal fsync batching, and save staging.
|
||
- Cancellation is cooperative and revision-based: when revision R+1 lands, in-flight tiles for R
|
||
are abandoned unless they are the only thing on screen.
|
||
|
||
The UI process is Qt-standard: everything on the GUI thread except the tile decompressor.
|
||
Long-running document operations (save, export, OCR) show progress and are cancellable; nothing
|
||
blocks the GUI thread for more than one frame.
|
||
|
||
### 2.4 Third-party components and licensing
|
||
|
||
The project ships under **GPL-3.0-or-later** (Qt LGPL-3 dynamically linked), so components must be
|
||
GPL-3-compatible. Deliberately avoided: MuPDF and Poppler (AGPL / GPL-2-only, the latter creating a
|
||
compatibility headache and a hard dependency on someone else's model we can't extend).
|
||
|
||
| Concern | Choice | License | Why |
|
||
|---|---|---|---|
|
||
| PDF object/file layer | **QPDF** | Apache-2.0 | Best-in-class object access, preserves structure, real incremental-update support, excellent recovery on damaged files. Does *not* touch content streams — exactly the boundary we want. |
|
||
| Rasterization | **PDFium** | BSD-3 | Battle-tested (Chrome), fast, permissive. Used only as a renderer over the bytes *we* emit. |
|
||
| 2D backend | **Skia** | BSD-3 | Comes with PDFium; also backs our annotation/overlay drawing, the typing fast path (§2.5), and the GPU compositor. |
|
||
| Text shaping | **HarfBuzz** | MIT | Complex scripts, OpenType features; `hb-subset` gives us production-grade font subsetting for free. |
|
||
| Font rasterizing/parsing | **FreeType** | FTL | Glyph metrics, cmap access, sfnt table surgery. FTL is GPLv3-compatible (it is *not* GPLv2-compatible — another reason the project is GPL-3, not GPL-2). |
|
||
| Unicode | **ICU** | Unicode-3.0 | Bidi, normalization, line-break classes (UAX #14), collation for search. |
|
||
| Color | **Little-CMS 2** | MIT | ICC profiles, separations, rendering intents. |
|
||
| Image codecs | libjpeg-turbo, libpng, OpenJPEG, libtiff | permissive | **JBIG2 and JPX come from PDFium's bundled decoders**, not jbig2dec — jbig2dec is AGPL-3 (combinable under GPLv3 §13, but it drags AGPL obligations into the tree for no benefit when we already link PDFium). |
|
||
| Crypto/signing | **OpenSSL 3** | Apache-2.0 | CMS, timestamps, OCSP/CRL, PKCS#11 via provider. Apache-2.0 is GPL-3-compatible; it would not have been under GPL-2. |
|
||
| Hyphenation / spelling | **Hunspell** | LGPL-2.1/GPL-2/MPL-1.1 tri-license | Use under LGPL-2.1, dynamically linked. |
|
||
| Compression | zlib-ng, brotli, liblzma | permissive | Flate is 95% of streams. |
|
||
| OCR (Tier 2) | Tesseract 5 | Apache-2.0 | |
|
||
|
||
**Test-only tools are not linked and do not constrain the license**: Ghostscript (AGPL) and veraPDF
|
||
(MPL/GPL dual) are invoked as external reference binaries in CI. The license scanner (§11) must be
|
||
configured to understand that distinction or it will produce a permanent false positive.
|
||
|
||
**Font embedding legality** is a real constraint: check `OS/2.fsType` on every font we embed or
|
||
re-subset. `fsType` bit 1 (restricted, `0x0002`) blocks embedding — surface a clear error and offer
|
||
substitution rather than silently producing a file that violates the foundry license. Bits 8
|
||
(no-subsetting) and 9 (bitmap-only) are enforced too.
|
||
|
||
**Dependency duplication note:** PDFium bundles its own FreeType, libjpeg, libpng and OpenJPEG.
|
||
Decide at M1 whether to build PDFium against our system copies (fewer bytes, single CVE surface, more
|
||
build friction) or accept the duplicates. Recommendation: unbundle on Linux, accept duplicates on
|
||
Windows/macOS where we ship everything anyway.
|
||
|
||
### 2.5 The core architectural rule: *render what you will save*
|
||
|
||
The single biggest failure mode for WYSIWYG PDF editors is divergence between the preview and the
|
||
saved file. We eliminate it structurally:
|
||
|
||
1. Every edit mutates L5 (semantic model).
|
||
2. The dirty page's content stream is **re-emitted immediately** (§4.4) into an in-memory buffer.
|
||
3. The rasterizer renders *that buffer*, not the model.
|
||
|
||
So the preview is, by construction, a render of the exact bytes that saving would produce.
|
||
|
||
**The typing fast path.** Rule 3 taken literally means re-parsing and re-rendering a whole page
|
||
through PDFium on every keystroke. That is fine for a light page and far too slow for a dense one,
|
||
so during an active edit gesture we use a two-tier scheme:
|
||
|
||
- **Predictive tier (every keystroke, budget < 8 ms):** render only the dirty text frame's damage
|
||
rectangle from our own L3 display list via Skia, composited over the cached page raster.
|
||
- **Authoritative tier (on 120 ms idle, on gesture end, and always before save):** re-emit and
|
||
re-render through PDFium, then diff against what the predictive tier drew. A mismatch above
|
||
threshold repaints and logs a fidelity event with the page and revision.
|
||
|
||
This preserves the guarantee — nothing is ever *saved* that hasn't been rendered from its own bytes
|
||
— while keeping keystroke latency bounded. It also gives us a continuous, free, in-production
|
||
consistency check between our interpreter and PDFium's, which is exactly the divergence we are most
|
||
exposed to (§11, "two interpreters").
|
||
|
||
Re-emission must still be fast — target < 5 ms for a typical page — which drives the design in §4.4
|
||
(surgical re-emission, not full rebuild).
|
||
|
||
### 2.6 Two interpreters, one truth
|
||
|
||
We necessarily run two content-stream interpreters (ours at L3, PDFium's in the rasterizer) and two
|
||
file parsers (QPDF and PDFium's). Where they disagree, the user sees one thing and we model another.
|
||
Rules that keep this bounded:
|
||
|
||
- **PDFium never sees the original bytes of a damaged file.** On open, if QPDF performed any
|
||
recovery (rebuilt xref, repaired streams, resolved a broken trailer), we hand PDFium the
|
||
QPDF-normalized bytes, not the file on disk. Otherwise the preview can reflect a different
|
||
document than the one we will save.
|
||
- **Our Skia display-list renderer is a first-class artifact**, not a debugging toy: it powers the
|
||
predictive tier and the CI cross-check gate (§8.2). Divergence between it and PDFium on the corpus
|
||
is a tracked metric with a budget, not an anomaly.
|
||
- Any operator our L3 interpreter does not understand marks its page **splice-unsafe**, forcing
|
||
verbatim copy for that region (§4.4 rule 4). We never guess.
|
||
|
||
---
|
||
|
||
## 3. Document model
|
||
|
||
### 3.1 L2 — object layer
|
||
|
||
Wraps QPDF. Responsibilities: xref/xref-stream parsing, object streams, stream filters, encryption
|
||
handshake, damaged-file reconstruction, and the object graph with reference resolution and cycle
|
||
safety.
|
||
|
||
Guarantees we add on top:
|
||
- **Provenance tracking**: every object records whether it is untouched, modified, or new. Untouched
|
||
objects are written back byte-identical.
|
||
- **Copy-on-write** semantics so edits are cheap and undo is a pointer swap.
|
||
- **Lazy page loading**: pages materialize on demand; a 5,000-page document opens with only the
|
||
page tree resident.
|
||
- **Resource budgets**: hard caps on decompressed stream size, nesting depth, object count, and
|
||
total resident bytes, enforced at the L2 boundary. Exceeding one aborts the operation with a
|
||
diagnostic instead of dying by OOM-killer — decompression bombs are a corpus reality, not a
|
||
hypothetical.
|
||
|
||
### 3.2 L3 — content model
|
||
|
||
The content-stream interpreter turns operators into a **display list**, maintaining the full
|
||
graphics state machine (CTM, color spaces, ExtGState, clipping, transparency groups, soft masks,
|
||
text state, optional content membership, marked content / structure tags).
|
||
|
||
```cpp
|
||
struct DisplayItem {
|
||
enum class Kind { Glyphs, Path, Image, Shading, XObjectForm, MarkedContent } kind;
|
||
GraphicsState gs; // resolved, not deltas
|
||
Matrix ctm;
|
||
ClipId clip; // interned clip stack
|
||
OCGId layer;
|
||
StructElemId tag; // for tagged PDFs
|
||
MCID mcid; // marked-content id, needed to rewrite /K arrays (§3.4)
|
||
SourceSpan span; // byte range in the original stream — critical, see §4.4
|
||
std::variant<GlyphRun, PathGeom, ImageRef, ShadingRef, FormRef> payload;
|
||
};
|
||
|
||
struct GlyphRun {
|
||
FontId font;
|
||
float size, charSpacing, wordSpacing, horizScale, rise;
|
||
RenderMode mode;
|
||
Matrix textMatrix;
|
||
std::vector<Glyph> glyphs; // {gid, code, advance, offset, unicode, confidence, origin}
|
||
};
|
||
```
|
||
|
||
`SourceSpan` is the key to lossless round-tripping: it lets us splice edits back into the original
|
||
operator byte stream instead of regenerating everything. Note that a page's content may be an
|
||
**array of streams** and operators can straddle the boundary — spans are (streamIndex, offset,
|
||
length) triples over the logical concatenation, and re-emission must preserve the array structure.
|
||
|
||
### 3.3 L5 — semantic model
|
||
|
||
Built by the reconstruction pass (§4.1). This is what tools manipulate.
|
||
|
||
```cpp
|
||
struct TextFrame { // an editable block: paragraph, column, cell, caption…
|
||
QuadTree<LineId> lines;
|
||
Polygon shape; // may be non-rectangular (wrap-around)
|
||
Direction baseDir; // ICU bidi
|
||
std::vector<Paragraph> paras;
|
||
ColumnLink next; // Tier 2: story continuation
|
||
Provenance origin; // which DisplayItems produced this
|
||
};
|
||
|
||
struct Paragraph {
|
||
std::vector<StyleRun> runs; // {text, FontId, size, color, tracking, features…}
|
||
ParaStyle style; // align, indents, leading, spaceBefore/After, tabs
|
||
bool leadingIsUniform; // if false, reflow preserves per-line baselines
|
||
};
|
||
```
|
||
|
||
Every semantic object keeps a bidirectional link to the display items it came from. Edits mark
|
||
those items dirty; untouched items are re-emitted verbatim.
|
||
|
||
### 3.4 Structure tree (tagged PDF) maintenance
|
||
|
||
A tagged PDF carries a parallel tree of `StructElem` objects whose leaves point at content via MCIDs
|
||
(and at annotations/XObjects via OBJR). An editor that reflows text without touching that tree
|
||
produces a file that still *looks* right and is no longer accessible — the reading order, the
|
||
`Alt` text, and the tab order all decay silently. Given the EU Accessibility Act obligations now in
|
||
force for many of our users' documents, quietly destroying tags is a product defect, not a
|
||
limitation.
|
||
|
||
Rules:
|
||
1. **MCID stability.** Re-emission preserves each surviving marked-content sequence's MCID. New
|
||
content inside an existing `BDC` region gets a new MCID appended to the parent `StructElem`'s
|
||
`/K` array at the correct position; deleted content's MCID is removed from `/K` and any now-empty
|
||
`StructElem` is pruned (with its `/P` chain fixed up).
|
||
2. **Split and merge.** Splitting a paragraph splits its `P` element; merging merges them, keeping
|
||
the first element's attributes and logging when the second's differed.
|
||
3. **`ActualText` and `Alt` survive**: they are carried on the semantic run, edited through the
|
||
inspector, and re-emitted. If the user edits text whose `ActualText` no longer matches, the
|
||
fidelity panel flags it rather than guessing.
|
||
4. **`ParentTree` and `StructParents`** are rebuilt on save; number-tree consistency is asserted.
|
||
5. **Tab order** (`/Tabs /S`) is preserved on page reorder, and annotation `StructParent` entries are
|
||
remapped.
|
||
6. If a document's structure tree is already inconsistent on open, we do **not** attempt repair;
|
||
we mark it untagged-in-practice, tell the user, and preserve the objects verbatim.
|
||
|
||
CI gate: tag round-trip on a tagged subcorpus (§8.2).
|
||
|
||
---
|
||
|
||
## 4. The hard parts
|
||
|
||
### 4.1 Text reconstruction (content stream → editable paragraphs)
|
||
|
||
PDF has no concept of a word, a line, or a paragraph. Every one of those must be inferred. Pipeline:
|
||
|
||
**Step 1 — glyph→Unicode.** In priority order:
|
||
1. `ToUnicode` CMap (correct when present and non-broken; validate against surrogate/`FFFD` junk).
|
||
2. Simple-font `Encoding` + `Differences` → glyph names → Adobe Glyph List → Unicode.
|
||
3. Standard/Symbolic font built-in encodings; `Identity-H` + CIDSystemInfo registry ordering
|
||
(Adobe-Japan1 etc.) via the standard CMap tables.
|
||
4. Reverse lookup through the embedded font's `cmap` table.
|
||
5. Heuristic fallback: OCR the rendered glyph bitmap against a shape database.
|
||
|
||
Every character carries a **confidence** value through the whole model. Low-confidence characters
|
||
are shown with a subtle underline in edit mode, because editing near them can change visible output.
|
||
Ligature decomposition (`fi` → `f` + `i`), soft hyphens, and `ActualText` overrides are resolved here
|
||
so that the text the user edits is the text a copy-paste would produce.
|
||
|
||
**Step 2 — run assembly.** Merge adjacent glyph runs sharing font/size/color/CTM into logical runs.
|
||
Handle `TJ` kerning adjustments: a negative adjustment larger than ~20% of the space width means an
|
||
inter-word gap, not kerning.
|
||
|
||
**Step 3 — line detection.** Cluster by baseline in the *text space* of the run, tolerance
|
||
±0.25 × font size, handling rotation (cluster along the baseline direction vector, not page Y) and
|
||
superscript/subscript rise. Merge runs whose baselines agree and whose horizontal gap is < 2 space
|
||
widths.
|
||
|
||
**Step 4 — reading order + columns.** Recursive XY-cut with a whitespace-density projection, with
|
||
fallback to a topological sort of line-adjacency when the cut fails (common in magazine layouts).
|
||
When the PDF is tagged, **use the structure tree instead** — it's authoritative and far more
|
||
accurate.
|
||
|
||
**Step 5 — paragraph grouping.** Merge consecutive lines when: leading is consistent (±10%),
|
||
horizontal extents overlap, no bullet/numbering prefix starts a new line, indentation pattern is
|
||
consistent, and the previous line ends near the right margin (i.e. it wrapped rather than ended).
|
||
Detect list structures and preserve their markers as paragraph properties.
|
||
|
||
**Step 6 — frame geometry.** Frame shape = union of line bounding boxes, expanded to the inferred
|
||
text column, minus any obstacles (images/paths that text visibly wraps around).
|
||
|
||
Reconstruction is **lazy and cached**: it runs per page on first edit-mode entry or first search,
|
||
not on open, and its result is invalidated by content revision. Budget: < 30 ms for a typical page,
|
||
< 200 ms for a dense one.
|
||
|
||
Accuracy is measured, not asserted: a labelled corpus of ~2,000 pages with ground-truth
|
||
paragraph/column segmentation, scored by boundary F1, gates every release (§8.2).
|
||
|
||
### 4.2 Font handling
|
||
|
||
The hardest practical problem in PDF editing. A page's embedded font is typically a **subset**
|
||
containing only the glyphs originally used. Type "Ω" into a paragraph and that glyph does not
|
||
exist.
|
||
|
||
Resolution ladder, applied per missing character:
|
||
1. **Grow the subset.** If the full font is installed on the system and matches by name +
|
||
`head.checkSumAdjustment` / panose / glyph-hash comparison, pull the glyph from it and re-subset
|
||
with `hb-subset`.
|
||
2. **Font-service fallback.** Optional, opt-in, offline: a bundled set of metric-compatible open
|
||
fonts (Liberation, Croscore, Noto family) mapped to the common commercial faces.
|
||
3. **Substitute.** Pick the closest available face by panose/OS-2 class, weight, width, italic
|
||
angle, and x-height ratio; embed a fresh subset. The substituted run is flagged in the UI with
|
||
a "font substituted" badge and listed in a pre-save report.
|
||
4. **Refuse.** For fonts with `fsType` restricted embedding, we never embed — offer substitution.
|
||
|
||
Also required:
|
||
- Rewrite `ToUnicode` CMaps for every re-subset font (searchability depends on it).
|
||
- Handle Type3 fonts (glyph procedures) — editable only as whole-run replacement.
|
||
- Preserve CID ordering and `CIDToGIDMap` for CJK; never renumber GIDs of untouched runs.
|
||
- Vertical writing mode (`Identity-V`, `WMode 1`) in shaping and line breaking.
|
||
- **Variable fonts**: instance to a static named/custom instance before embedding; PDF 2.0 allows
|
||
variable font embedding but consumer support is poor.
|
||
- **Never mutate a font program in place.** Growing a subset produces a *new* font object with a new
|
||
subset tag (`ABCDEF+Name`); pages referencing the old object are untouched. This keeps undo cheap
|
||
and prevents one page's edit from breaking another's rendering.
|
||
- Standard-14 fonts (non-embedded Helvetica etc.): treat as substitution candidates with their
|
||
canonical metrics; if the user edits such a run and we must embed, say so.
|
||
|
||
### 4.3 Layout and reflow
|
||
|
||
When a text frame changes, re-layout it:
|
||
1. **Shape** each style run with HarfBuzz against the actual embedded font, with the original
|
||
OpenType features preserved where detectable.
|
||
2. **Break lines** with Knuth–Plass (total-fit) using ICU UAX #14 break opportunities, hyphenation
|
||
via Hunspell patterns, and the frame polygon for available width per line.
|
||
3. **Justify** using the original method: word-spacing-only, or word+letter spacing, matched to what
|
||
the source document did (measured from the original inter-word gap variance).
|
||
4. **Fit** to the frame. If the text overflows: (a) 1.0 default — grow the frame downward if the
|
||
space below is empty, otherwise show an overset indicator and let the user resize/link;
|
||
(b) Tier 2 — flow into the linked next frame, cascading across pages.
|
||
|
||
**Reflow fidelity guard:** for unedited paragraphs we do not re-layout at all. For an edited
|
||
paragraph, we first attempt a *minimal* reflow — re-break only from the edited word to the end of
|
||
the paragraph — and verify that lines before the edit are byte-identical to the original. If not,
|
||
we fall back to full paragraph re-layout and log a fidelity event.
|
||
|
||
**Bidi editing** is a distinct problem from bidi rendering: caret movement (visual vs logical),
|
||
selection contiguity across direction runs, and cursor placement at direction boundaries all need
|
||
explicit UX decisions. Follow the UAX #9 + platform conventions and test with a dedicated RTL/mixed
|
||
corpus; this is easy to get 80% right and very visibly wrong at the last 20%.
|
||
|
||
### 4.4 Surgical re-emission
|
||
|
||
Naïvely regenerating a page's content stream loses everything we didn't model: obscure operators,
|
||
`BDC` marked-content nesting, transparency groups, printer-specific `DP` properties, comments.
|
||
Instead:
|
||
|
||
```
|
||
original stream bytes: [ ..A.. ][ ..B.. ][ ..C.. ][ ..D.. ]
|
||
edits touch B and C: [ ..A.. ][ new B'C' ][ ..D.. ] ← A and D copied verbatim
|
||
```
|
||
|
||
Rules:
|
||
1. Each `DisplayItem` carries its `SourceSpan`. Dirty items' spans are coalesced into replacement
|
||
regions.
|
||
2. Graphics state at a region boundary is reconstructed and **explicitly re-established** inside the
|
||
replacement (`q ... Q` wrapping) so the following verbatim bytes still see the state they expect.
|
||
This is verified by an assertion pass that re-interprets the emitted stream and diffs the
|
||
graphics state at every splice point against the original.
|
||
3. Newly created objects append after the last region.
|
||
4. If a page's structure is too tangled to splice safely (deeply interleaved marked content across
|
||
a dirty region, an unrecognised operator inside it, or a `q`/`Q` imbalance), fall back to a full
|
||
re-emit of that page and record it as a fidelity event.
|
||
5. Resources (`/Font`, `/XObject`, `/ExtGState`…) added by an edit are merged into the page's
|
||
resource dictionary with fresh, non-colliding names; the resource dictionary may be inherited
|
||
from an ancestor `Pages` node, in which case we copy it down to the page before mutating it —
|
||
never edit an inherited dictionary in place, it is shared.
|
||
|
||
Save modes:
|
||
- **Incremental update** (default when the file is signed or when the user chose "save"): append
|
||
changed objects + new xref. Preserves existing signatures on prior revisions and is fast on
|
||
large files.
|
||
- **Full rewrite** (on "save as", or when the user asks to optimize): garbage-collect unused
|
||
objects, deduplicate identical streams and fonts, recompress, optionally linearize for fast web
|
||
view.
|
||
- **Sanitized save** (explicit user action): full rewrite plus removal of JavaScript, embedded
|
||
files, external references, and all prior revisions.
|
||
|
||
Version handling: the output PDF version is `max(input version, minimum required by features used)`.
|
||
If an edit needs a feature the input's version doesn't allow (e.g. AES-256 in a 1.4 file), we prompt
|
||
before upgrading, because some downstream consumers care.
|
||
|
||
Both paths run a **verification pass** before the file is handed over: reparse the output, re-render
|
||
every changed page, and compare against the pre-save preview raster. A mismatch above threshold
|
||
blocks the save with a diagnostic rather than writing a corrupted file. Text extraction of changed
|
||
pages is compared too — a page can be pixel-identical and semantically ruined.
|
||
|
||
### 4.5 Redaction
|
||
|
||
Redaction that only draws black boxes is a recurring real-world data breach. Ours removes:
|
||
- Glyphs whose bounding boxes intersect the region (partial glyphs → the run is split and re-emitted
|
||
without them; a glyph straddling the boundary is removed entirely).
|
||
- Image pixels: decode, blank the region, re-encode (lossless where the source was lossless). If the
|
||
image is referenced by more than one page, it is **copied first** — redacting a shared XObject in
|
||
place silently redacts other pages.
|
||
- Path geometry: clip and re-emit the remaining segments.
|
||
- Annotations, form field values, and their appearance streams.
|
||
- Embedded font subsets: if removing the only occurrence of a glyph leaves it in the subset, re-subset
|
||
so the glyph itself is gone (fonts leak surprisingly often).
|
||
- Document metadata, XMP, attachments, and *all prior incremental revisions* (redaction forces a
|
||
full rewrite — never incremental).
|
||
- Optional-content groups whose hidden content intersects the region.
|
||
Then re-render and OCR-check the redacted region as a final assertion that no text survives.
|
||
Redaction is irreversible and says so, with a preview of exactly what will be destroyed.
|
||
|
||
### 4.6 Signatures, encryption and editing
|
||
|
||
These interact badly and users get hurt when a tool is vague about it.
|
||
|
||
- **Editing a signed document.** Any content edit invalidates signatures that cover the edited
|
||
content. On the first edit we explain this once, per document, in plain language, and offer:
|
||
save as a new revision (incremental — earlier signatures remain valid over their revisions, the
|
||
latest becomes "signed but modified") or save a copy. `DocMDP` permission level is honoured:
|
||
at level 1 we refuse content edits outright, at level 2 we allow form fill and signing only, at
|
||
level 3 we additionally allow annotations.
|
||
- **Signing.** Byte-range placeholder, two-pass write, PKCS#11 / OS keystore (CryptoAPI, Keychain,
|
||
p11-kit) for the private key — we never handle raw private key material. RFC 3161 timestamps;
|
||
LTV by embedding OCSP/CRL and building B-LT/B-LTA.
|
||
- **Validation** reports the chain, revocation status, timestamp, coverage (does the signature cover
|
||
the whole file?) and what changed after signing — not a green tick.
|
||
- **Encryption.** Re-encrypt on save with the document's existing algorithm unless the user changes
|
||
it. Owner-password-only documents open with full editing (permission bits are advisory and we say
|
||
so honestly, rather than pretending to enforce them, but we do preserve them). Encrypted
|
||
attachments and unencrypted-wrapper documents (PDF 2.0) are handled explicitly.
|
||
- The journal and autosave snapshots for an encrypted document are themselves encrypted at rest with
|
||
a key held only in memory; a crashed session's recovery data is unreadable without reopening the
|
||
original.
|
||
|
||
---
|
||
|
||
## 5. Rendering and printing
|
||
|
||
- **Tile-based**: 512×512 tiles in device space, rendered on a worker pool, cached in an LRU keyed
|
||
by (page, zoom bucket, rotation, layer set, content revision).
|
||
- **Progressive**: low-res preview immediately, refine to full res; text-only quick pass first for
|
||
perceived responsiveness during scroll.
|
||
- **HiDPI**: render at device pixel ratio; independent zoom levels per split view.
|
||
- **GPU path**: Skia Ganesh/Graphite for annotation overlay, selection chrome, the typing fast path,
|
||
and page compositing; page content stays on CPU (PDFium) for fidelity. Fall back to full CPU
|
||
compositing when GPU init fails, and blocklist known-bad driver/GPU combinations.
|
||
- **Editing overlay** (selection handles, text carets, guides, snapping) draws in a separate layer
|
||
above the raster, so it never invalidates the content cache.
|
||
- **Color management**: soft-proof mode with output-intent simulation; correct handling of
|
||
Separation/DeviceN, overprint preview, and rendering intents.
|
||
|
||
**Printing** goes through the same emitter, not a separate path: we produce the PDF we would save,
|
||
then hand it to the platform print system (CUPS/IPP Everywhere, `IPrintDocumentPackageTarget`,
|
||
`NSPrintOperation`). For drivers that mangle PDF, offer "print as image" at a chosen DPI. Booklet,
|
||
N-up, tiling, scale-to-fit, and duplex/margin handling are computed by us so output is identical
|
||
across platforms. Print previews are the actual raster, not an approximation.
|
||
|
||
Performance targets (P95, on a 2020-class laptop):
|
||
|
||
| Operation | Target |
|
||
|---|---|
|
||
| Open 1,000-page / 100 MB file → first page visible | < 800 ms |
|
||
| Page render at 100% zoom, cached fonts | < 40 ms |
|
||
| Keystroke → updated pixels in text editing (predictive tier) | < 16 ms |
|
||
| Authoritative re-render after idle | < 120 ms |
|
||
| Text reconstruction, typical page (first edit-mode entry) | < 30 ms |
|
||
| Content-stream re-emission, typical page | < 5 ms |
|
||
| Scroll at 60 fps with progressive tiles | sustained |
|
||
| Search across a 1,000-page document (first result) | < 500 ms |
|
||
| Memory, 1,000-page document, 20 pages viewed | < 600 MB |
|
||
| Save (incremental) 100 MB file | < 500 ms |
|
||
| Cold app start → empty window interactive | < 700 ms |
|
||
|
||
---
|
||
|
||
## 6. Editing infrastructure
|
||
|
||
### 6.1 Command pattern
|
||
|
||
Every mutation is a `Command` with `apply` / `revert` / `merge` / `serialize`. Commands compose into
|
||
transactions; a transaction is the undo granularity. Consecutive typing commands merge on a 500 ms
|
||
idle or word boundary.
|
||
|
||
```cpp
|
||
class Command {
|
||
public:
|
||
virtual Result apply(Document&) = 0;
|
||
virtual void revert(Document&) = 0;
|
||
virtual bool mergeWith(const Command&) { return false; }
|
||
virtual void serialize(Journal&) const = 0; // for crash recovery
|
||
virtual PageSet dirtyPages() const = 0; // drives re-emission + repaint
|
||
};
|
||
```
|
||
|
||
Undo stack is unlimited by default, spilling command payloads to disk beyond 200 MB. `revert` must
|
||
restore the object graph *and* the structure tree; every command type has a property-based test
|
||
asserting apply→revert returns the document to a byte-identical emitted state.
|
||
|
||
### 6.2 Selection model
|
||
|
||
Three selection modes, one shared hit-test index (per-page R-tree over display items):
|
||
- **Object selection** — images, paths, groups, annotations, form fields.
|
||
- **Text selection** — caret + range within a `TextFrame`, with word/line/paragraph expansion,
|
||
rectangular (column) selection on Alt-drag, and bidi-correct visual↔logical mapping.
|
||
- **Node selection** — path anchor points and control handles in vector-edit mode.
|
||
|
||
### 6.3 Tools
|
||
|
||
Modeless where possible. Double-click any text enters edit mode in place; double-click an image
|
||
opens crop/replace handles. Explicit tools for annotate, redact, form-field, shape, and node edit.
|
||
Snapping to object edges/centers, page margins, baselines, and a configurable grid, with
|
||
alignment/distribute operations and smart guides.
|
||
|
||
### 6.4 Crash recovery and autosave
|
||
|
||
Commands are appended to a journal (fsync'd, batched at 200 ms) alongside a periodic snapshot of the
|
||
document delta. After a crash, the UI process replays the journal onto the reopened original file
|
||
and offers the recovered state. The original file is never modified in place — saves go to a temp
|
||
file on the same filesystem, fsync, then atomic rename, preserving ownership, permissions, xattrs
|
||
and (on macOS) resource forks.
|
||
|
||
Recovery must be safe against *poisoned* journals: a journal that crashes replay is quarantined
|
||
after two attempts and the user is offered the last good snapshot instead of an infinite crash loop.
|
||
|
||
**External modification**: we hold the file's identity (device+inode / file id) and mtime+size; if it
|
||
changes underneath us we warn before saving and offer reload/compare/save-as. Two windows on the same
|
||
file share one document process.
|
||
|
||
### 6.5 Text input
|
||
|
||
The canvas is custom-drawn, so we get none of Qt's line-edit behaviour for free. Required:
|
||
- **IME support** via `QInputMethodEvent` — preedit string rendered inline with the correct styling,
|
||
candidate window positioned at the caret, and commit handled as a single undoable command. Test
|
||
on Windows IME, macOS input sources, and ibus/fcitx on Linux (all three behave differently).
|
||
- Dead keys, compose sequences, and Unicode hex input.
|
||
- Platform-native caret behaviour, word-boundary navigation (ICU word breaks, not `isspace`), and
|
||
overwrite/insert modes.
|
||
- Spell-check underlining via Hunspell with per-document language detection; no autocorrect,
|
||
ever — silently changing the text in someone's legal document is unacceptable.
|
||
- Drag-and-drop of text and objects, with autoscroll.
|
||
- Accessibility: expose the edit session through `QAccessible` text interfaces so a screen reader
|
||
can read and navigate the text being edited (§9.3).
|
||
|
||
---
|
||
|
||
## 7. Security and privacy
|
||
|
||
### 7.1 Threat model
|
||
|
||
| Asset | Adversary | Vector | Mitigation |
|
||
|---|---|---|---|
|
||
| User's machine | Attacker sending a malicious PDF | Parser memory corruption | Sandboxed document process (§2.1), sanitizers in CI, continuous fuzzing, resource budgets (§3.1) |
|
||
| User's other files | Same | Sandbox escape via IPC to the UI process | IPC treated as untrusted in both directions (§2.1); UI deserializer fuzzed; no path handling in the document process |
|
||
| User's network / data exfiltration | Same | PDF actions: `SubmitForm`, `URI`, `GoToR`, remote XObjects, embedded file launch | Document process has no network. The UI never opens a URL, launches a file, or submits a form without an explicit user gesture and a dialog showing the full target. Remote resources are never fetched during rendering. |
|
||
| Document confidentiality | Local attacker | Autosave journal, temp files, crash dumps | Encrypted-at-rest journal for encrypted documents (§4.6); temp files mode 0600 in a per-user dir; crash dumps never include document bytes (§7.3) |
|
||
| Update channel | Network attacker | Malicious update | Ed25519-signed updates verified before applying, pinned key, TLS on top (§12.4) |
|
||
| Release integrity | Supply chain | Compromised dependency or build | Pinned vcpkg baseline + lockfile, SBOM per release, reproducible builds where the toolchain allows, signed tags |
|
||
|
||
Explicitly *not* claimed: we do not defend against a compromised OS account, and permission bits in
|
||
a PDF are not a security boundary (we say so in the UI).
|
||
|
||
### 7.2 Memory safety strategy
|
||
|
||
We are writing a new parser in C++, which the threat model says is the main risk. Beyond "be
|
||
careful":
|
||
- All parsing operates on `std::span`-like bounded views; raw pointer arithmetic over input bytes is
|
||
banned by clang-tidy and enforced in review.
|
||
- Build with `-D_GLIBCXX_ASSERTIONS` / `_LIBCPP_HARDENING_MODE=fast`, `-fstack-protector-strong`,
|
||
CFI and shadow-stack/CET where available, in **release** builds — not just debug.
|
||
- A hardened allocator (scudo or hardened_malloc) in the document process.
|
||
- Integer overflow is a build error in the parsing layers (`-ftrapv`-equivalent via UBSan in CI,
|
||
checked arithmetic helpers in code).
|
||
- Evaluate at M0 whether L1/L2 primitive decoders (filters, image codecs glue, CMap parsing) are
|
||
worth writing in Rust behind a C ABI. They are self-contained, leaf-level, and where most of the
|
||
historical CVEs live. Decide once, at M0 — retrofitting a second language mid-project is worse
|
||
than either choice made early.
|
||
|
||
### 7.3 Telemetry, crash reporting and privacy
|
||
|
||
The default posture is that a PDF editor sees people's most sensitive documents and must be boring
|
||
about it.
|
||
|
||
- **No telemetry by default.** Opt-in only, asked once, never re-nagged. When enabled it sends
|
||
counters and timings — never file names, paths, content, or URLs.
|
||
- **Crash reporting** (Crashpad) is opt-in and uploads minidumps with **document memory excluded**:
|
||
the document heap lives in a dedicated allocator region that is stripped from the dump, and the
|
||
dump is filtered client-side before upload. Show the user the report contents before the first
|
||
send.
|
||
- **Fidelity events** (§2.5, §4.3, §4.4) are recorded locally and visible in the fidelity panel;
|
||
they are only transmitted if the user explicitly submits a diagnostic bundle, and the bundle
|
||
never includes the document unless the user attaches it deliberately.
|
||
- No network connections at all except: update check, timestamp/OCSP/CRL during signing, and
|
||
user-initiated link opening. All are individually disableable by policy (§12.2).
|
||
- A **privacy policy** and a documented list of every outbound connection ship in the repo.
|
||
|
||
### 7.4 Security disclosure
|
||
|
||
`SECURITY.md` with a published contact, PGP key, and a 90-day coordinated-disclosure policy.
|
||
Pre-1.0: an external security audit of the parser and sandbox, closed before release (§10, M9).
|
||
OSS-Fuzz integration from M2, not from M8.
|
||
|
||
---
|
||
|
||
## 8. Testing and quality
|
||
|
||
### 8.1 Corpora
|
||
|
||
- **govdocs1** subset (~50k real-world PDFs) — parser robustness and crash-freedom.
|
||
- **PDF Association test suite** + veraPDF corpus — spec conformance, PDF/A, PDF/UA.
|
||
- **pdf.js + PDFium regression corpora** — known-bad renderers.
|
||
- **Internal labelled corpus** (~2,000 pages) — ground-truth paragraph/column segmentation and
|
||
glyph→Unicode mappings, for reconstruction scoring.
|
||
- **Tagged subcorpus** (~300 documents) — structure-tree round-trip and edit-under-tags.
|
||
- **Script corpus** — Arabic/Hebrew (RTL), Devanagari, Thai, CJK horizontal and vertical, mixed-
|
||
direction, for shaping, bidi editing, and line breaking.
|
||
- **Generated adversarial corpus** — malformed xrefs, cyclic object graphs, 10k-deep nesting,
|
||
decompression bombs, hostile encryption dictionaries.
|
||
|
||
Corpus provenance and licensing are tracked; anything with unclear redistribution rights stays out
|
||
of the public corpus repo and is referenced by hash for local-only use.
|
||
|
||
### 8.2 Automated gates (every PR)
|
||
|
||
| Gate | Threshold |
|
||
|---|---|
|
||
| Render pixel-diff vs Ghostscript+PDFium reference, 5 zoom levels | ≤ 0.1% differing pixels, no new failures |
|
||
| **Own Skia renderer vs PDFium** on the corpus (§2.6) | ≤ 0.3% differing pixels; regressions block |
|
||
| Round-trip: open → save (no edits) → reparse | semantically identical object graph; text extraction byte-identical |
|
||
| Reconstruction: paragraph boundary F1 | ≥ 0.93, no regression |
|
||
| Text edit → save → extract | edited text matches expected, neighbours unchanged |
|
||
| **Tag round-trip**: edit a tagged doc → save → structure tree valid, reading order preserved | no regression; veraPDF PDF/UA checks no worse than input |
|
||
| Splice-point graphics-state invariant (property-based) | zero violations |
|
||
| Command apply→revert byte-identity | zero violations |
|
||
| ASan/UBSan/TSan on the full unit + corpus suite | clean |
|
||
| Fuzzing (libFuzzer, per-layer harnesses **incl. the UI-side IPC deserializer**) | no new crashes in 30 min/PR; continuous via OSS-Fuzz |
|
||
| Performance benchmarks | no >5% regression on the target table |
|
||
| Memory ceiling on the 1,000-page fixture | < 600 MB |
|
||
| License scan (REUSE + scancode) | no non-GPL-3-compatible linked dependency |
|
||
|
||
### 8.3 Other testing
|
||
|
||
- Differential testing: our text extraction vs pdftotext/PDFium on the whole corpus; disagreements
|
||
are triaged, not auto-accepted (we are sometimes right).
|
||
- Property-based tests for the graphics state machine and the splice-point invariant (§4.4 rule 2).
|
||
- **Consumer-compatibility matrix**: every release's output opened in Acrobat, Preview, Chrome,
|
||
Firefox/pdf.js, Edge and Foxit. "It renders in our app" is not the bar; a file our editor produces
|
||
that Acrobat refuses is a release blocker.
|
||
- Manual exploratory passes each milestone.
|
||
- Accessibility: keyboard-only operation of every feature, screen reader (NVDA/VoiceOver/Orca)
|
||
labelling of canvas objects, high-contrast and reduced-motion support.
|
||
- **Upgrade/soak tests**: install old version → open documents → auto-update → verify settings,
|
||
recovery journals and licences survive.
|
||
|
||
### 8.4 Release bug bar
|
||
|
||
Blocking a release: any data loss, any file corruption, any crash reproducible from a corpus file,
|
||
any redaction failure, any signature mis-validation, any sandbox escape, any accessibility
|
||
regression on a Tier-1 flow, any perf target missed by >20%.
|
||
|
||
---
|
||
|
||
## 9. UI design
|
||
|
||
```
|
||
┌────────────────────────────────────────────────────────────────────────┐
|
||
│ menu bar [search] │
|
||
├──────────┬──────────────────────────────────────────────┬──────────────┤
|
||
│ tool │ │ inspector │
|
||
│ palette │ page canvas │ (context- │
|
||
│ │ (continuous / facing / single) │ sensitive: │
|
||
│ [thumbs] │ │ text, obj, │
|
||
│ [outline]│ ┌────────────────────┐ │ page, doc) │
|
||
│ [layers] │ │ floating format │ │ │
|
||
│ [comment]│ │ bar on selection │ │ [history] │
|
||
│ [fields] │ └────────────────────┘ │ [fidelity] │
|
||
├──────────┴──────────────────────────────────────────────┴──────────────┤
|
||
│ status: page 4/128 · 100% · 2 substituted fonts · unsaved changes │
|
||
└────────────────────────────────────────────────────────────────────────┘
|
||
```
|
||
|
||
### 9.1 Principles
|
||
|
||
- **Direct manipulation first.** Text edits happen on the page, not in a side panel. Format controls
|
||
appear as a floating bar near the selection.
|
||
- **Honest feedback.** A persistent, non-modal "fidelity" indicator surfaces substituted fonts,
|
||
low-confidence character mappings, pages that required full re-emission, and tag-structure
|
||
warnings. Users of a PDF editor need to know when the output may not match the input.
|
||
- **Nothing is silently destructive.** Redaction, flattening, downsampling, signature invalidation
|
||
and tag loss always show what will be permanently removed, with a preview.
|
||
- **No modal progress for anything the user could keep working through.**
|
||
- Dark mode, per-monitor DPI, tabbed documents, split view, session restore, recent files with
|
||
pinning, and custom keymaps (Acrobat-compatible preset shipped).
|
||
|
||
### 9.2 Internationalization
|
||
|
||
- The UI is fully translatable from M1: Qt Linguist `.ts` files, no concatenated strings, plural
|
||
forms, and no text baked into images. Ship with a Weblate/Transifex-style community workflow.
|
||
- Launch languages: English, German, French, Spanish, Portuguese (BR), Italian, Polish, Russian,
|
||
Japanese, Simplified Chinese. Others as community translations reach 90%.
|
||
- RTL UI mirroring (`LayoutDirection::RightToLeft`) is tested, not assumed — the canvas itself never
|
||
mirrors, only the chrome.
|
||
- Locale-correct number/date/measurement formatting; unit preference (pt/mm/inch/pica) is explicit
|
||
and per-document-independent.
|
||
|
||
### 9.3 Accessibility
|
||
|
||
- Every function reachable by keyboard, with a discoverable shortcut list.
|
||
- The canvas exposes an accessibility tree: pages, objects with names/roles, and a text interface
|
||
during editing so screen readers can read, navigate and edit text (§6.5).
|
||
- Respect OS settings for contrast, reduced motion, font scaling, and cursor size.
|
||
- Target WCAG 2.2 AA for the application chrome. We ask users to care about accessible PDFs; the
|
||
editor has to hold itself to the same standard.
|
||
|
||
### 9.4 Settings and policy
|
||
|
||
Settings live in a documented, human-readable file (INI/JSON) under the platform config dir, with a
|
||
machine-wide policy layer that overrides them (Windows ADMX/registry, macOS configuration profile,
|
||
Linux `/etc/freepdfeditor/policy.json`). Policy-controlled keys: updates, telemetry, crash reporting,
|
||
default save location, network access, sanitized-save enforcement, allowed signature roots.
|
||
|
||
---
|
||
|
||
## 10. Roadmap
|
||
|
||
Assumes a team of **6–8 engineers** (2 PDF core, 2 text/layout/fonts, 2 UI, 1 rendering/perf,
|
||
1 QA/infra) plus a designer. Durations are calendar time.
|
||
|
||
| # | Milestone | Duration | Exit criteria |
|
||
|---|---|---|---|
|
||
| **M0** | Feasibility spikes | 8 wks | Prototypes proving: content-stream splice round-trips losslessly on 500 real PDFs; text reconstruction hits F1 ≥ 0.85; PDFium renders our re-emitted bytes identically; our Skia display-list renderer matches PDFium closely enough for the predictive tier. Rust-vs-C++ decision for L1/L2 decoders made. Kill/redesign decision point. |
|
||
| **M1** | Viewer | 10 wks | Open/render/navigate/search/print any corpus file; tiling + caching; sandboxed process split; i18n scaffolding; perf targets for viewing met. |
|
||
| **M2** | Object model + commands | 8 wks | L2/L3/L6 complete; incremental + full save with verification; page assembly ops; undo/redo; crash recovery; OSS-Fuzz live. |
|
||
| **M3** | Annotations & forms | 10 wks | All Tier-1 annotation types, comment panel, AcroForm fill + field authoring, appearance stream generation. Ships as a **usable public beta**. |
|
||
| **M4** | Text editing, single-line | 12 wks | Reconstruction pipeline, glyph→Unicode, in-place editing without reflow, styling, font subset growth, IME support. |
|
||
| **M5** | Reflow | 12 wks | Paragraph reflow with Knuth–Plass, frame fitting, font substitution ladder, bidi + CJK vertical, tag maintenance under edits, find & replace across document. |
|
||
| **M6** | Images & vectors | 8 wks | Image replace/crop/resample, path selection and node editing, colorspace-correct handling. |
|
||
| **M7** | Security & compliance | 8 wks | Redaction with verification, signing + validation (PAdES), encryption, PDF/A export via veraPDF gate, policy layer. |
|
||
| **M8** | Performance, a11y, polish | 10 wks | All perf targets green; keyboard-complete; screen reader support; localization at launch-language coverage; installers, auto-update, crash reporting. |
|
||
| **M9** | 1.0 hardening | 8 wks | 4 weeks bug-fix-only; OSS-Fuzz clean for 30 days; external security audit closed; consumer-compatibility matrix green. |
|
||
|
||
**≈ 94 weeks (~22 months) to 1.0**, with a public beta at M3 (~9 months) and text editing in
|
||
beta hands at M5 (~15 months). Milestones M4–M6 are the ones that slip; budget accordingly.
|
||
|
||
### 10.1 If the team is one or two people
|
||
|
||
The plan above is staffed for 6–8 engineers. At 1–2 people the same scope is a 5+ year project and
|
||
the honest move is to change the *shape* of it, not the estimate:
|
||
|
||
1. **Ship a viewer first, publicly** (M0 + M1, ~6 months at this size). A fast, sandboxed,
|
||
correct viewer is a real product and it de-risks everything downstream.
|
||
2. **Then annotations + forms + page assembly** (M2 + M3). This is where most "PDF editor" users
|
||
actually stop, and it is 20% of the engineering of text editing.
|
||
3. **Then text editing, narrowly**: in-place single-line editing (M4) with *no* reflow, gated behind
|
||
a clearly-labelled beta. Refuse to edit anything the reconstructor is not confident about rather
|
||
than shipping bad output.
|
||
4. **Reflow last** (M5), and only if steps 1–3 have users.
|
||
|
||
Non-negotiable even at small scale, because retrofitting them is worse than building them:
|
||
the process split (§2.1), the render-what-you-save rule (§2.5), the command/journal layer (§6.1),
|
||
the packaging pipeline (§12), and the CI gates for round-trip and pixel-diff (§8.2).
|
||
|
||
Deferrable at small scale: GPU compositing, PDF/A export, LTV signing, OCR, DOCX export, the
|
||
localization breadth in §9.2, and the `.rpm`/Flatpak/winget/Homebrew channels.
|
||
|
||
---
|
||
|
||
## 11. Risks
|
||
|
||
| Risk | Impact | Mitigation |
|
||
|---|---|---|
|
||
| Text reconstruction accuracy is the whole product; real PDFs are pathological | High | M0 spike gates the project on measured F1; labelled corpus scored every PR; per-document confidence surfaced in UI rather than hidden. |
|
||
| Re-emission silently corrupts complex pages | High | Splice-point graphics-state assertion; pre-save render *and text-extraction* verification; per-page full-rebuild fallback; fidelity events reported and triaged. |
|
||
| **Two interpreters (ours and PDFium's) diverge** | High | Cross-renderer CI gate (§8.2); predictive-vs-authoritative diff in production (§2.5); unknown operators force verbatim splice. |
|
||
| Font substitution produces visibly wrong output | Medium | Subset-growth before substitution; metric-compatible fallback set; explicit UI badge and pre-save report. |
|
||
| Editing silently destroys accessibility tags | Medium-High | Tag maintenance is Tier 1 (§3.4) with its own CI gate; fidelity panel warns. |
|
||
| Security vulnerability in the parser | High | Sandboxed document process from M1 (not retrofitted); hardened build settings in release (§7.2); sanitizers in CI; OSS-Fuzz from M2; external audit before 1.0. |
|
||
| Sandbox escape through the IPC layer into the UI process | High | Bidirectional trust boundary (§2.1); UI deserializer fuzzed as a first-class harness. |
|
||
| Scope creep into Acrobat-parity print production | Medium | Tier-2 list is contractual; 1.0 scope frozen at M3. |
|
||
| Font embedding license violations | Medium | `fsType` enforced at the embedding call site, no bypass path. |
|
||
| GPL-incompatible dependency creeps in | Medium | CI license scan (REUSE + scancode) fails the build on any new non-compatible dependency; test-only tools explicitly allowlisted. |
|
||
| PDFium is a large upstream we must track | Medium | Pin a revision, update on a schedule with the full corpus gate; keep our patches minimal and upstreamable. |
|
||
| Schedule optimism on M4–M6 | High | These are staffed with the two most senior engineers; M0 spike results recalibrate the estimate before commitment. Small-team variant in §10.1. |
|
||
| Our own output rejected by Acrobat/Preview | High | Consumer-compatibility matrix as a release gate (§8.3), run from M2 — not discovered at 1.0. |
|
||
|
||
---
|
||
|
||
## 12. Distribution and packaging
|
||
|
||
Every platform gets first-class native packaging. Packaging is **not** an M8 afterthought: the
|
||
pipeline is stood up in M1 producing nightly artifacts for all targets, so integration problems
|
||
(signing, notarization, dependency bundling, Wayland quirks) surface early and continuously.
|
||
|
||
Payload note: ICU's full data is ~30 MB. Build a trimmed data bundle (the collations, break
|
||
iterators and normalization we actually use) as part of the packaging pipeline, or the download size
|
||
doubles for nothing.
|
||
|
||
### 12.1 Linux
|
||
|
||
| Format | Details |
|
||
|---|---|
|
||
| **`.deb`** | Debian 12+ / Ubuntu 22.04+. Built via `dpkg-deb` from a CPack DEB generator. Depends on system Qt 6 where the distro ships ≥ 6.7, otherwise bundles it under `/opt/freepdfeditor`. Ships `.desktop` entry, hicolor icons (16–512 px), MIME association for `application/pdf`, AppStream metainfo XML, man page, and shell completions. Published to an APT repo (`deb.freepdfeditor.org`) signed with the project GPG key; `stable` and `nightly` suites. |
|
||
| **`.AppImage`** | The universal fallback: built inside a manylinux-style container against an old glibc (2.31) for maximum reach, bundling Qt, Skia, ICU, and all codecs. Includes `AppRun` with Wayland/X11 auto-detection, zsync file for delta updates, and desktop-integration prompt via `appimaged` conventions. Runs on any glibc ≥ 2.31 distro with no root. **Caveat:** AppImage cannot use bubblewrap-style user namespaces on distros that restrict them, so the sandbox falls back to seccomp-only; the app reports its actual sandbox level in About, and refuses to silently pretend. |
|
||
| **Portable tarball** | `freepdfeditor-<ver>-linux-x86_64.tar.xz` — the same self-contained tree as the AppImage but unpacked, for sandboxed/immutable environments, CI use, and users who want the raw binary. Contains an `install.sh` that only creates symlinks and desktop entries under `~/.local`. |
|
||
| *(Also planned)* | **`.rpm`** for Fedora/openSUSE via the CPack RPM generator, and a **Flatpak** on Flathub (`org.freepdfeditor.FreePDFEditor`) using **only** the document and file-chooser portals — no `--filesystem=host`, which would defeat the point of the sandbox and is increasingly penalised by Flathub's own review. Both are best-effort community-supported, not release blockers. |
|
||
|
||
Architectures: `x86_64` for all formats; `aarch64` for `.deb` and tarball.
|
||
All Linux artifacts are reproducible-build-verified where the toolchain allows, and shipped with
|
||
detached GPG signatures plus SHA-256 sums.
|
||
|
||
### 12.2 Windows
|
||
|
||
| Format | Details |
|
||
|---|---|
|
||
| **`.msi`** | The enterprise/managed-deployment path, built with WiX Toolset v4. Per-machine install, ADMX templates for the §9.4 policy set, silent install (`msiexec /qn`), proper upgrade codes for in-place version upgrades, and clean uninstall. This is what IT departments deploy via Group Policy / Intune / SCCM. |
|
||
| **`.exe`** | The consumer installer (Inno Setup or NSIS wrapping the same payload). Per-user install without admin rights by default, elevating only if the user chooses all-users. Handles file associations, "Open with", Explorer preview handler and thumbnail provider registration, and offers the portable mode opt-out. |
|
||
| **Portable `.zip`** | No installer, no registry writes, settings stored beside the executable. |
|
||
| *(Also planned)* | `winget` manifest and a Chocolatey package pointing at the `.exe`. MSIX/Microsoft Store is deferred past 1.0: its sandbox complicates our document-process sandbox, *and* the Store's terms are widely read as incompatible with GPL-3 distribution. |
|
||
|
||
Both installers are **Authenticode-signed** with an EV certificate (in HSM/cloud KMS, never on a
|
||
build machine), which also builds SmartScreen reputation. Architectures: `x64` and `arm64`.
|
||
|
||
### 12.3 macOS
|
||
|
||
| Format | Details |
|
||
|---|---|
|
||
| **`.dmg`** | Primary consumer distribution. Universal 2 binary (x86_64 + arm64), drag-to-Applications layout with a styled background, hardened runtime enabled, code-signed with a Developer ID Application certificate and **notarized + stapled** (unnotarized builds are a support disaster on macOS 15+). |
|
||
| **`.pkg`** | For managed/MDM deployment (Jamf, Kandji): signed with Developer ID Installer, supports `installer -pkg` silent install and configuration profiles for the same policy set as the Windows ADMX. |
|
||
| *(Also planned)* | Homebrew Cask pointing at the `.dmg`. The **Mac App Store is not possible**, not merely unplanned: the App Store's terms impose usage restrictions that GPL-3 forbids. The sandbox-entitlement friction with §2.1 is a secondary reason. |
|
||
|
||
macOS-specific integration: Quick Look preview extension, Services menu entries, Continuity/Handoff
|
||
of the open document, and correct behaviour under App Nap and Stage Manager.
|
||
|
||
### 12.4 Auto-update
|
||
|
||
One update mechanism, three transports: Sparkle (macOS), WinSparkle or a custom updater service
|
||
(Windows), zsync/AppImageUpdate + APT (Linux). Updates are delta where possible, signed with the
|
||
project's Ed25519 release key and verified before applying. Channels: `stable`, `beta`, `nightly`.
|
||
Enterprise builds ship with updates disabled by policy. The update check sends no identifiers beyond
|
||
version and platform, and is disableable.
|
||
|
||
### 12.5 Release pipeline
|
||
|
||
`git tag v*` on the release branch triggers: cross-platform build matrix → full test gates (§8.2) →
|
||
artifact packaging for all nine formats → signing/notarization → SBOM generation (CycloneDX) →
|
||
checksum + signature manifest → upload to the Gitea release (§13) and the public mirror.
|
||
A release is only publishable if **every** platform artifact built and passed its smoke test —
|
||
no partial releases. Signing keys live in an HSM/cloud KMS; no key material ever reaches a runner's
|
||
filesystem.
|
||
|
||
---
|
||
|
||
## 13. Source control workflow
|
||
|
||
**All code created or changed must be committed and pushed to the project Gitea instance.** This is
|
||
a hard rule for every contributor and every automated agent working on the codebase, not a
|
||
best-effort convention — nothing stays only on a local disk.
|
||
|
||
### 13.1 Remote
|
||
|
||
| | |
|
||
|---|---|
|
||
| **Host** | `https://gitea.lm.je` |
|
||
| **Username** | `ai-ad4` |
|
||
| **Credential** | Read from the environment variable `GITEA_PASSWORD_AI_AD4`. Never hardcode it, never commit it, never echo it into logs or CI output. |
|
||
| **Repository** | `ai-ad4/freepdfeditor` (create on first push if absent) |
|
||
|
||
> **Note on the variable name:** the underscores are deliberate — the username is `ai-ad4`, but a
|
||
> hyphen is not legal in a shell identifier, so the variable form of the name substitutes
|
||
> underscores. Do not "correct" it back to match the username.
|
||
|
||
Configure the remote without embedding the secret in `.git/config`:
|
||
|
||
```bash
|
||
# credential helper reads the password from the environment at call time
|
||
git config --local credential.helper \
|
||
'!f() { echo "username=ai-ad4"; echo "password=$GITEA_PASSWORD_AI_AD4"; }; f'
|
||
git remote add origin https://gitea.lm.je/ai-ad4/freepdfeditor.git
|
||
```
|
||
|
||
The variable is exported from the developer's shell profile for local work. Interactive shell
|
||
profiles are **not** inherited by CI runners, cron jobs, or GUI-launched processes — those take the
|
||
password from the Gitea repository secret store (§13.3) instead.
|
||
|
||
An SSH deploy key is preferred over password auth once the repo exists; the password path stays as
|
||
the bootstrap and CI fallback.
|
||
|
||
### 13.2 Rules
|
||
|
||
1. **Push after every change.** Any session that creates or modifies code ends with that work
|
||
committed and pushed. Work in progress goes to a branch — it does not sit uncommitted.
|
||
2. **Never commit to `main` directly.** Branch (`feat/`, `fix/`, `spike/`, `chore/`), push, open a
|
||
pull request. `main` is protected and requires the §8.2 gates to pass.
|
||
3. **Conventional Commits** (`feat:`, `fix:`, `perf:`, `refactor:`, `test:`, `docs:`, `build:`) with
|
||
the affected layer as a scope, e.g. `fix(L3): restore graphics state at splice boundary`.
|
||
4. **No secrets in history.** `gitleaks` runs as a pre-commit hook and again in CI; a hit fails the
|
||
build. `GITEA_PASSWORD_AI_AD4` and all signing keys live in the environment or the CI secret
|
||
store only.
|
||
5. **Large binaries via Git LFS.** The test corpora (tens of GB of PDFs) live in a separate
|
||
`freepdfeditor-corpus` repository with LFS, referenced as a submodule pinned by commit — the main
|
||
repo stays fast to clone.
|
||
6. **Tags are releases.** Annotated, signed (`git tag -s`), semver, and they drive §12.5.
|
||
7. **This plan is versioned with the code**, at `docs/plan.md`. It is a living document; changes to
|
||
architecture land as PRs against it in the same commit as the code that implements them.
|
||
|
||
### 13.3 CI on Gitea
|
||
|
||
Gitea Actions runners (Linux x64/arm64, Windows, macOS) execute the §8.2 gate matrix on every push
|
||
and pull request, and the §12.5 packaging pipeline on tags. Runner registration tokens and signing
|
||
credentials come from Gitea repository secrets, never from the repo tree. Runners for release builds
|
||
are ephemeral and isolated from PR builds — a PR must never be able to reach a signing key.
|
||
|
||
### 13.4 Project governance
|
||
|
||
- **License:** GPL-3.0-or-later, `REUSE`-compliant headers, `LICENSES/` directory.
|
||
- **Contributions:** DCO sign-off (not a CLA). `CONTRIBUTING.md` covers build setup, the gate
|
||
matrix, and the coding standard (clang-format enforced, clang-tidy set checked in).
|
||
- **Security:** `SECURITY.md` per §7.4.
|
||
- **Decisions:** architecture decision records in `docs/adr/`, numbered, immutable once accepted.
|
||
§2.5, §4.4 and §7.2 each get one at M0.
|
||
- **Trademark:** the name and logo are held separately from the code license so forks can't ship
|
||
malware under our name; policy documented in `TRADEMARK.md`.
|
||
|
||
---
|
||
|
||
## 14. Immediate next steps (first 4 weeks)
|
||
|
||
1. Stand up the repo **on `gitea.lm.je` (§13) before any code is written**: CMake + vcpkg (pinned
|
||
baseline + lockfile), branch protection on `main`, Gitea Actions runners for all three platforms,
|
||
CI matrix (Win/macOS/Linux × Debug/Release/ASan), clang-format, clang-tidy, gitleaks, license
|
||
scanner, REUSE headers, `SECURITY.md` / `CONTRIBUTING.md` / `docs/adr/`.
|
||
2. Skeleton packaging for all nine artifact formats (§12) producing nightly builds from day one,
|
||
even if the app only shows an empty window — signing and notarization are discovered problems,
|
||
not implemented ones.
|
||
3. Assemble the corpora and the pixel-diff harness against Ghostscript + PDFium references — this
|
||
infrastructure gates everything, so it comes early.
|
||
4. **Spike A**: QPDF-based open → parse content streams → re-emit verbatim → save. Measure
|
||
byte-identical round-trip rate over 500 corpus files. Target ≥ 99%.
|
||
5. **Spike B**: glyph→Unicode + line/paragraph reconstruction on the labelled corpus. Report F1.
|
||
6. **Spike C**: `hb-subset` growth of an existing embedded subset font with a new glyph; verify the
|
||
result renders in Acrobat, Preview, and Chrome.
|
||
7. **Spike D**: our own Skia display-list renderer vs PDFium on 500 corpus pages. Report pixel
|
||
divergence — this decides whether the §2.5 predictive tier is viable or whether keystroke latency
|
||
needs a different answer.
|
||
8. **Spike E**: sandbox bring-up on all three platforms with a trivial parser, including the IPC
|
||
layer. Measure the cost of the process split on open latency. Sandboxing is the one thing that is
|
||
genuinely painful to retrofit.
|
||
9. Design review of §4.4 splice invariants before any production code lands, recorded as an ADR.
|
||
|
||
---
|
||
|
||
## 15. Open decisions
|
||
|
||
Things this plan deliberately does not settle, with the point at which they must be:
|
||
|
||
| Decision | Deadline | Notes |
|
||
|---|---|---|
|
||
| Rust for L1/L2 leaf decoders, or C++ throughout | End of M0 | §7.2. Decide once; do not revisit at M4. |
|
||
| Team size and therefore roadmap shape (§10 vs §10.1) | Before M1 | Changes the milestone order, not just the dates. |
|
||
| Qt Widgets vs QML for the shell | End of M1 | Widgets assumed. QML buys animation and mobile-ish polish, costs accessibility maturity and adds a runtime. |
|
||
| Unbundle PDFium's vendored libraries, or accept duplicates | M1 | §2.4. |
|
||
| Whether "grow the subset from a system font" is legally defensible in all jurisdictions | Before M4 ships | Get an actual opinion; `fsType` is a signal, not a licence. |
|
||
| Funding/sustainability model for a GPL-3 desktop app | Before public beta (M3) | Donations, paid support, enterprise-signed builds — pick one before users arrive, not after. |
|
||
| Hosted crash/telemetry backend, or none at all | M8 | §7.3 default is "none"; shipping a backend has ongoing privacy obligations. |
|