8.4 KiB

Raw Permalink Blame History

Spike E — sandbox + IPC bring-up: M0 result

Spike: E — sandbox + IPC bring-up (engineering plan §14 step 8)
Date: 2025-07-25 (M0); updated 2026-07-27 — Cap'n Proto socketpair stall resolved, RPC transport wired in.
Status: Complete. Gate MET.
Gate: the sandboxed document process round-trips IPC requests with the seccomp filter installed (proving the allow-list is correct and the channel works under the sandbox).

What was built

spike/E_sandbox/ implements the two-process model from ADR-0004 (§2.1):

Sandbox.cpp — a seccomp-bpf allow-list filter for the document process. Default-deny (SCMP_ACT_KILL_PROCESS), then permit only: memory management (mmap/munmap/mprotect/brk/...), thread sync (futex), the already-open IPC socket fds (read/write/close/readv/writev/poll/epoll/eventfd/timerfd/dup), ioctl (FIONBIO, needed by KJ's wrapSocketFd to set non-blocking mode), and exit/signal. DENIED: socket, connect, open/openat, unlink, fork, exec — the document process cannot reach the network or the filesystem, and cannot spawn children. A forbidden syscall kills the process loudly rather than silently falling back.
ipc.capnp — a Cap'n Proto schema for the IPC protocol. Now the actual transport (see "Findings" for the resolution of the original stall).
DocumentProcess.cpp — the sandboxed child: installs the seccomp filter, then serves Parse requests over the socketpair via Cap'n Proto two-party RPC. Uses the low-level LowLevelAsyncIoProvider::wrapSocketFd(fd) + TwoPartyVatNetwork(SIDE_SERVER) + makeRpcServer(network, bootstrap) path (NOT EzRpcServer — see Findings). The DocumentProcess::Server capability bounds-checks the payload length (16 MiB cap) per ADR-0004's "the document process is untrusted" rule.
UIProcess.cpp — the UI process: uses EzRpcClient(fd) (designed for an already-connected socket), calls parse() N times, measures round-trip latency, and validates every response (request id, byte count, ok flag) per ADR-0004's "IPC is a trust boundary in both directions" rule.
main.cpp — forks, sets up the socketpair, runs both sides, measures.

Result

Metric	Value	Target
round-trip success rate	1.0 (1000/1000)	1.0

Configuration	avg latency	min	max	child exit
sandbox ON, Release, 1KB payload, 1000 reqs (Cap'n Proto RPC)	~58 µs	~28 µs	~350 µs	0 (clean)
sandbox OFF, Release, 1KB payload, 1000 reqs (Cap'n Proto RPC)	~50 µs	~28 µs	~1.5 ms	0
sandbox OFF, ASan+UBSan, 512B payload, 200 reqs (Cap'n Proto RPC)	~480 µs	~100 µs	~4.3 ms	0

The sandbox adds no measurable latency overhead. The Cap'n Proto RPC framing (~50–58 µs/round-trip) is ~3× the raw length-prefixed protocol it replaced (~18 µs) — expected, since RPC adds message framing, the bootstrap handshake, and the KJ event loop — and still far inside the §5 budget (the document process's open latency budget is < 800 ms for a 1000-page file → first page; the per-request IPC cost is negligible against that). Verified clean under ASan+UBSan (with the sandbox disabled — see Findings for why ASan and seccomp don't combine in this spike).

Findings

The process split works and is cheap. ~50 µs average round-trip per Cap'n Proto RPC request over the socketpair, well under the §5 performance targets. The sandbox is free. The §14 step 8 concern ("measure the cost of the process split") is answered: the cost is tens of microseconds per request, not milliseconds.
The seccomp allow-list is correct for the RPC event loop. The document process runs the KJ event loop (epoll/eventfd/timerfd), reads/writes the IPC socket, allocates memory, and exits — all under the filter, with no denials. A forbidden syscall (open/socket/connect/fork/exec) would trigger SCMP_ACT_KILL_PROCESS, crashing the process loudly. The one addition over the raw-protocol variant was ioctl (FIONBIO), which KJ's wrapSocketFd calls to set non-blocking mode; strace identified it as the denied syscall in the first sandboxed run.
The Cap'n Proto socketpair stall is RESOLVED. The original M0 spike recorded that EzRpcServer(int fd, ...) over a socketpair stalled: the server processed requests but responses never reached the client. Root cause, confirmed with a standalone reproduction: EzRpcServer(int fd, ...) expects a LISTENING socket and calls accept() on it. A socketpair end is already CONNECTED, so accept() fails with EINVAL ("Invalid argument") and the bootstrap never completes — the server aborts and the client sees a disconnect. The fix is the low-level path on the server side:
```
kj::AsyncIoContext io = kj::setupAsyncIo();
kj::Own<kj::AsyncIoStream> stream = io.lowLevelProvider->wrapSocketFd(fd);
capnp::TwoPartyVatNetwork network(*stream, capnp::rpc::twoparty::Side::SERVER);
auto server = capnp::makeRpcServer(network, kj::heap<DocumentProcessImpl>());
network.onDisconnect().wait(io.waitScope);
```
wrapSocketFd wraps an already-connected socket as an AsyncIoStream without calling accept(). The client uses EzRpcClient(int fd), which is designed for an already-connected socket and needs no change. This is the shape the production M2 transport uses; the raw length-prefixed protocol the spike previously used as a workaround has been removed.
ASan and seccomp don't combine in this spike. ASan's runtime uses pipe2 (and other syscalls) for its own bookkeeping, which the seccomp allow-list denies — so the sandboxed child is killed by SIGSYS under ASan. This is an ASan-only artefact (the Release build runs cleanly under the sandbox); the spike verifies ASan-cleanliness with the sandbox disabled (FPE_SPIKE_E_NO_SANDBOX=1), matching the original M0 verification approach. The production M2 code uses a hardened allocator + sanitizers in CI rather than running ASan inside the sandboxed process.
ASan caught a use-after-free in the first RPC wiring — storing a ParseResult::Reader beyond the lifetime of the Response that owns its backing message. Fixed by validating the response inside the Response's scope. This validates the §7.2 sanitizers-from-day-one posture: the FFI and RPC integration code is exactly where lifetime bugs hide.
The bidirectional trust boundary is exercisable. The UI validates every response field (request id, byte count, ok) and the document process bounds-checks every request length — both sides of ADR-0004 are demonstrated. The production M2 code fuzzes the UI-side deserializer as a first-class harness (§8.2); this spike is the foundation for that.

What this means for the project

The process split is viable and cheap. The §2.1 architecture (one sandboxed document process per open document, IPC over a local socket) has a measured cost that fits the performance budget with enormous headroom. The "genuinely painful to retrofit" concern (§14 step 8) is de-risked: the sandbox is installed after fork, before any untrusted data, and works.
The Cap'n Proto RPC transport is proven over a socketpair. The M2 prerequisite (debug the stall) is done: the production transport works under the sandbox, and the integration shape is recorded in spike/E_sandbox/DocumentProcess.cpp. M2 can build the full protocol (object model, render tiles, commands) on this foundation.
The sandbox is the security foundation (ADR-0004). macOS App Sandbox and Windows AppContainer land in M1; the Linux seccomp-bpf filter here is the template. The §7.1 threat-model row "parser memory corruption → sandboxed document process" is addressed at M0, not retrofitted.

Reproducing

cmake -S . -B build/manual -G Ninja -DCMAKE_BUILD_TYPE=Release
cmake --build build/manual --target spike_e_sandbox
build/manual/bin/spike_e_sandbox 1000 1024    # <n_requests> <payload_bytes>
# To diagnose without the sandbox (debug aid; the gate requires it ON):
FPE_SPIKE_E_NO_SANDBOX=1 build/manual/bin/spike_e_sandbox 1000 1024

Exit 0 if all requests round-trip under the sandbox, 1 otherwise. Also verified clean under ASan+UBSan (sandbox disabled).

8.4 KiB Raw Permalink Blame History Unescape Escape