Wire protocol
Two wire formats. Media frames are binary with a fixed-width header, because that is the hot path and it must stay allocation-free. Control and input are JSON, because they are low-rate and ergonomics wins there.
The media header
Section titled “The media header”Every media WebSocket message is exactly one frame: 16 bytes of header, then the payload. Nothing else, ever.
| Offset | Size | Field | Meaning |
|---|---|---|---|
| 0 | 2 | Magic |
0x4442 — ASCII DB. A frame is self-identifying when logged or teed |
| 2 | 1 | Kind |
which stream this frame belongs to |
| 3 | 1 | Format |
payload container |
| 4 | 4 | Seq |
producer sequence number; gaps mean dropped frames |
| 8 | 4 | Stamp |
producer monotonic milliseconds |
| 12 | 4 | Length |
payload bytes following the header |
Kind travels in-band even though the port already implies it, so a captured
frame stays interpretable outside the connection that carried it.
| Value | Name | Direction |
|---|---|---|
| 1 | mic |
harness → app |
| 2 | speaker |
app → harness |
| 3 | cam |
harness → app |
| 4 | screen |
app → harness |
Format
Section titled “Format”| Value | Name | Payload |
|---|---|---|
| 1 | PCM16 |
interleaved signed 16-bit little-endian |
| 2 | F32 |
interleaved 32-bit float |
| 10 | RawRGB |
packed RGB frame |
| 11 | RawBGRA |
packed BGRA frame — the native macOS capture format |
| 12 | JPEG |
JPEG-compressed frame |
Format only distinguishes the container. The concrete parameters — sample rate,
channel count, resolution, frame rate — are negotiated once on the control
channel, not repeated per frame.
Why Stamp exists
Section titled “Why Stamp exists”It is a producer monotonic millisecond counter, and it is the whole latency story. A harness subtracts it from its own monotonic clock on receipt and gets the transport cost with no wall clock and no clock-synchronisation assumption. Spike S0’s 0.047 ms audio and 5.6 ms video figures came out of this field.
Control messages
Section titled “Control messages”JSON over the control port (default 9330). The first message must be hello.
{ "type": "hello", "token": "…" }A wrong or absent token closes the connection. The compare is constant-time.
{ "type": "list" }{ "type": "start", "stream": "screen", "video": { "width": 1280, "height": 720, "fps": 30 } }{ "type": "start", "stream": "speaker", "audio": { "rate": 48000, "channels": 2 } }{ "type": "stop", "stream": "screen" }Stream names in control messages are the canonical ones — mic, speaker,
cam, screen, input. The CLI’s audio-in / audio-out / video-in /
video-out aliases are resolved before they reach the wire; see
Streams & ports.
start binds a port at that moment, returns the port that was actually
allocated, and mints a one-time ticket the media port will demand. stop
closes and releases the port.
Why not JSON, or gRPC, per frame
Section titled “Why not JSON, or gRPC, per frame”Rejected in ADR 0002, for the reasons the measurements supported:
- One multiplexed port for everything — demux overhead plus head-of-line blocking between audio and video on the same socket.
- gRPC / protobuf per frame — serialization cost on the hot path buying nothing a fixed 16-byte header does not already give.
- Raw TCP with custom framing — loses the uniform WebSocket surface that makes the same client work locally and through a tunnel.