Optimizations I: Protocols & Serialization

Why we chose Protobuf over FlatBuffers, heartbeat coalescing, and Zstandard compression. The wire format decisions that saved 40% bandwidth.

Overview

This episode covers the first half of our optimization journey: protocol design, serialization choices, and compression strategies.

Key Topics

• Protobuf vs FlatBuffers: why absent-field-is-free matters for sparse updates
• Heartbeat coalescing: 2-4 byte idle tick frames
• Zstandard streaming compression: ~40% bandwidth reduction
• Wire format evolution: from JSON to binary Protobuf

Protobuf vs FlatBuffers

Characteristic	Protobuf	FlatBuffers
Fixed overhead	None per table	~2-4 bytes per table
Sparse updates	Absent fields = 0 bytes	Tables always allocated
2 Hz deltas	Mostly absent fields	Full table each time
Our winner	✅ Protobuf	❌ FlatBuffers

FlatBuffers is excellent for game saves (large, complete state). Protobuf wins for sparse event deltas.

Timestamps

Wire Format Structure

message ClientFrame {
  uint32 sequence = 1;           // Tick sequence number
  repeated EntityDelta deltas = 2;  // Sparse entity updates
  repeated Event events = 3;     // Combat, movement, etc.
  bool heartbeat_only = 4;       // True if no changes
}

message EntityDelta {
  uint64 entity_id = 1;
  optional Position position = 2;
  optional uint32 hp = 3;
  optional uint32 shield = 4;
  // Only changed fields present
}

Idle tick: sequence (varint) + heartbeat_only=true (1 byte) = ~4 bytes

Compression Results

Compression	CPU Impact	Bandwidth Reduction
None	Baseline	Baseline
Deflate	High	35%
LZ4	Low	25%
Zstandard	Medium	40%
Zstd dict	Medium	45%

We chose Zstandard with pre-trained dictionary (message schemas).

Next Episode

Episode 7 covers the remaining optimizations: reconnect storms, passive mode, and edge caching.