BLACK SKIES ARCHITECTURE · EPISODE 3 OF 7
Designing Fan-Out for 10,000 Real-Time Players
· 25:00 ·
#distributed-systems#redis#pub-sub#fan-out
Case study of tile processor sharding, backpressure, and delivery guarantees. How Redis sharded pub/sub enables viewport-relevant event delivery.
Overview
This episode covers the fan-out problem: how do you deliver tick updates to 10,000 clients without overwhelming your network or degrading latency?
Key Topics
- The math of fan-out: 10K clients × 2 Hz × event size = bandwidth reality
- Viewport-based interest management: clients only receive relevant events
- Relay pod coalescing: merging all updates into one frame per tick
- Redis 7 sharded pub/sub: removing the cluster-wide broadcast ceiling
The Fan-Out Equation
| Scenario | Clients | Tick Rate | Event Size | Bandwidth |
|---|---|---|---|---|
| Naive broadcast | 10,000 | 2 Hz | 600 B | ~12 MB/s |
| Viewport-filtered | 10,000 | 2 Hz | 600 B | ~1.2 MB/s |
| With coalescing | 10,000 | 2 Hz | 2 KB | ~40 MB/s |
| Passive (0.5 Hz) | 9,000 | 0.5 Hz | 200 B | ~0.9 MB/s |
Timestamps
0:00 · The fan-out problem 5:30 · Viewport filtering 12:15 · Relay architecture 18:45 · Redis sharded pub/sub
Redis Sharded Pub/Sub
Redis 7’s SPUBLISH/SSUBSCRIBE pins channels to hash slots:
# Each channel mapped to slot by hash(channel_name)
# SPUBLISH only broadcasts to subscribers on that shard
# Eliminates cluster-wide bus for hot channels
This is load-bearing for our architecture. Without it, a single hot tile would saturate the entire Redis cluster bus.
Backpressure Handling
When relay pods are overwhelmed:
- Level 1: Drop cosmetic events (particles, animations)
- Level 2: Reduce update frequency for distant entities
- Level 3: Delta-only updates (skip unchanged fields)
- Level 4: Request client viewport reduction
Combat damage events are never dropped - they’re prioritized above all else.
Next Episode
Episode 4 covers ingress: how client actions flow into the tile processor and get validated.