152 lines
8.4 KiB
Markdown
152 lines
8.4 KiB
Markdown
<!--
|
||
SPDX-License-Identifier: GPL-3.0-or-later
|
||
SPDX-FileCopyrightText: 2025 ai-ad4 and the FreePDFEditor contributors
|
||
-->
|
||
# Spike E — sandbox + IPC bring-up: M0 result
|
||
|
||
* **Spike**: E — sandbox + IPC bring-up (engineering plan §14 step 8)
|
||
* **Date**: 2025-07-25 (M0); **updated 2026-07-27** — Cap'n Proto socketpair
|
||
stall resolved, RPC transport wired in.
|
||
* **Status**: Complete. **Gate MET.**
|
||
* **Gate**: the sandboxed document process round-trips IPC requests with the
|
||
seccomp filter installed (proving the allow-list is correct and the channel
|
||
works under the sandbox).
|
||
|
||
## What was built
|
||
|
||
`spike/E_sandbox/` implements the two-process model from ADR-0004 (§2.1):
|
||
|
||
- `Sandbox.cpp` — a seccomp-bpf allow-list filter for the document process.
|
||
Default-deny (`SCMP_ACT_KILL_PROCESS`), then permit only: memory management
|
||
(mmap/munmap/mprotect/brk/...), thread sync (futex), the already-open IPC
|
||
socket fds (read/write/close/readv/writev/poll/epoll/eventfd/timerfd/dup),
|
||
`ioctl` (FIONBIO, needed by KJ's `wrapSocketFd` to set non-blocking mode),
|
||
and exit/signal. **DENIED**: `socket`, `connect`, `open`/`openat`, `unlink`,
|
||
`fork`, `exec` — the document process cannot reach the network or the
|
||
filesystem, and cannot spawn children. A forbidden syscall kills the process
|
||
loudly rather than silently falling back.
|
||
- `ipc.capnp` — a Cap'n Proto schema for the IPC protocol. **Now the actual
|
||
transport** (see "Findings" for the resolution of the original stall).
|
||
- `DocumentProcess.cpp` — the sandboxed child: installs the seccomp filter,
|
||
then serves Parse requests over the socketpair via Cap'n Proto two-party
|
||
RPC. Uses the low-level `LowLevelAsyncIoProvider::wrapSocketFd(fd)` +
|
||
`TwoPartyVatNetwork(SIDE_SERVER)` + `makeRpcServer(network, bootstrap)` path
|
||
(NOT `EzRpcServer` — see Findings). The `DocumentProcess::Server` capability
|
||
bounds-checks the payload length (16 MiB cap) per ADR-0004's "the document
|
||
process is untrusted" rule.
|
||
- `UIProcess.cpp` — the UI process: uses `EzRpcClient(fd)` (designed for an
|
||
already-connected socket), calls `parse()` N times, measures round-trip
|
||
latency, and **validates every response** (request id, byte count, ok flag)
|
||
per ADR-0004's "IPC is a trust boundary in both directions" rule.
|
||
- `main.cpp` — forks, sets up the socketpair, runs both sides, measures.
|
||
|
||
## Result
|
||
|
||
| Metric | Value | Target |
|
||
|---|---|---|
|
||
| round-trip success rate | **1.0** (1000/1000) | 1.0 |
|
||
|
||
| Configuration | avg latency | min | max | child exit |
|
||
|---|---|---|---|---|
|
||
| sandbox ON, Release, 1KB payload, 1000 reqs (Cap'n Proto RPC) | **~58 µs** | ~28 µs | ~350 µs | 0 (clean) |
|
||
| sandbox OFF, Release, 1KB payload, 1000 reqs (Cap'n Proto RPC) | ~50 µs | ~28 µs | ~1.5 ms | 0 |
|
||
| sandbox OFF, ASan+UBSan, 512B payload, 200 reqs (Cap'n Proto RPC) | ~480 µs | ~100 µs | ~4.3 ms | 0 |
|
||
|
||
The sandbox adds no measurable latency overhead. The Cap'n Proto RPC framing
|
||
(~50–58 µs/round-trip) is ~3× the raw length-prefixed protocol it replaced
|
||
(~18 µs) — expected, since RPC adds message framing, the bootstrap
|
||
handshake, and the KJ event loop — and still far inside the §5 budget (the
|
||
document process's open latency budget is < 800 ms for a 1000-page file →
|
||
first page; the per-request IPC cost is negligible against that). Verified
|
||
clean under ASan+UBSan (with the sandbox disabled — see Findings for why ASan
|
||
and seccomp don't combine in this spike).
|
||
|
||
## Findings
|
||
|
||
1. **The process split works and is cheap.** ~50 µs average round-trip per
|
||
Cap'n Proto RPC request over the socketpair, well under the §5 performance
|
||
targets. The sandbox is free. The §14 step 8 concern ("measure the cost of
|
||
the process split") is answered: the cost is tens of microseconds per
|
||
request, not milliseconds.
|
||
|
||
2. **The seccomp allow-list is correct for the RPC event loop.** The document
|
||
process runs the KJ event loop (epoll/eventfd/timerfd), reads/writes the
|
||
IPC socket, allocates memory, and exits — all under the filter, with no
|
||
denials. A forbidden syscall (`open`/`socket`/`connect`/`fork`/`exec`)
|
||
would trigger `SCMP_ACT_KILL_PROCESS`, crashing the process loudly. The
|
||
one addition over the raw-protocol variant was `ioctl` (FIONBIO), which
|
||
KJ's `wrapSocketFd` calls to set non-blocking mode; strace identified it
|
||
as the denied syscall in the first sandboxed run.
|
||
|
||
3. **The Cap'n Proto socketpair stall is RESOLVED.** The original M0 spike
|
||
recorded that `EzRpcServer(int fd, ...)` over a socketpair stalled: the
|
||
server processed requests but responses never reached the client. Root
|
||
cause, confirmed with a standalone reproduction: **`EzRpcServer(int fd,
|
||
...)` expects a LISTENING socket and calls `accept()` on it.** A
|
||
socketpair end is already CONNECTED, so `accept()` fails with `EINVAL`
|
||
("Invalid argument") and the bootstrap never completes — the server
|
||
aborts and the client sees a disconnect. The fix is the low-level path on
|
||
the server side:
|
||
```cpp
|
||
kj::AsyncIoContext io = kj::setupAsyncIo();
|
||
kj::Own<kj::AsyncIoStream> stream = io.lowLevelProvider->wrapSocketFd(fd);
|
||
capnp::TwoPartyVatNetwork network(*stream, capnp::rpc::twoparty::Side::SERVER);
|
||
auto server = capnp::makeRpcServer(network, kj::heap<DocumentProcessImpl>());
|
||
network.onDisconnect().wait(io.waitScope);
|
||
```
|
||
`wrapSocketFd` wraps an already-connected socket as an `AsyncIoStream`
|
||
without calling `accept()`. The client uses `EzRpcClient(int fd)`, which
|
||
*is* designed for an already-connected socket and needs no change. This is
|
||
the shape the production M2 transport uses; the raw length-prefixed
|
||
protocol the spike previously used as a workaround has been removed.
|
||
|
||
4. **ASan and seccomp don't combine in this spike.** ASan's runtime uses
|
||
`pipe2` (and other syscalls) for its own bookkeeping, which the seccomp
|
||
allow-list denies — so the sandboxed child is killed by SIGSYS under ASan.
|
||
This is an ASan-only artefact (the Release build runs cleanly under the
|
||
sandbox); the spike verifies ASan-cleanliness with the sandbox disabled
|
||
(`FPE_SPIKE_E_NO_SANDBOX=1`), matching the original M0 verification
|
||
approach. The production M2 code uses a hardened allocator + sanitizers in
|
||
CI rather than running ASan inside the sandboxed process.
|
||
|
||
5. **ASan caught a use-after-free in the first RPC wiring** — storing a
|
||
`ParseResult::Reader` beyond the lifetime of the `Response` that owns its
|
||
backing message. Fixed by validating the response inside the `Response`'s
|
||
scope. This validates the §7.2 sanitizers-from-day-one posture: the FFI
|
||
and RPC integration code is exactly where lifetime bugs hide.
|
||
|
||
6. **The bidirectional trust boundary is exercisable.** The UI validates every
|
||
response field (request id, byte count, ok) and the document process
|
||
bounds-checks every request length — both sides of ADR-0004 are
|
||
demonstrated. The production M2 code fuzzes the UI-side deserializer as a
|
||
first-class harness (§8.2); this spike is the foundation for that.
|
||
|
||
## What this means for the project
|
||
|
||
- **The process split is viable and cheap.** The §2.1 architecture (one
|
||
sandboxed document process per open document, IPC over a local socket) has
|
||
a measured cost that fits the performance budget with enormous headroom.
|
||
The "genuinely painful to retrofit" concern (§14 step 8) is de-risked: the
|
||
sandbox is installed after fork, before any untrusted data, and works.
|
||
- **The Cap'n Proto RPC transport is proven over a socketpair.** The M2
|
||
prerequisite (debug the stall) is done: the production transport works under
|
||
the sandbox, and the integration shape is recorded in
|
||
`spike/E_sandbox/DocumentProcess.cpp`. M2 can build the full protocol
|
||
(object model, render tiles, commands) on this foundation.
|
||
- **The sandbox is the security foundation** (ADR-0004). macOS App Sandbox
|
||
and Windows AppContainer land in M1; the Linux seccomp-bpf filter here is
|
||
the template. The §7.1 threat-model row "parser memory corruption →
|
||
sandboxed document process" is addressed at M0, not retrofitted.
|
||
|
||
## Reproducing
|
||
|
||
```bash
|
||
cmake -S . -B build/manual -G Ninja -DCMAKE_BUILD_TYPE=Release
|
||
cmake --build build/manual --target spike_e_sandbox
|
||
build/manual/bin/spike_e_sandbox 1000 1024 # <n_requests> <payload_bytes>
|
||
# To diagnose without the sandbox (debug aid; the gate requires it ON):
|
||
FPE_SPIKE_E_NO_SANDBOX=1 build/manual/bin/spike_e_sandbox 1000 1024
|
||
```
|
||
|
||
Exit 0 if all requests round-trip under the sandbox, 1 otherwise. Also
|
||
verified clean under ASan+UBSan (sandbox disabled). |