EP16: Handling sudden bursts of high concurrency¶

Open on YouTube

Recap	EP15 "The relationship between horizontal scaling and low latency"
Next	EP17 "Deploy from scratch: run your own CDN in 15 minutes"

0. Goals of this episode¶

By the end of this episode, viewers should be able to do four things:

Tell who a "burst of high concurrency" actually stresses: it's the pressure on viewer pull (downstream distribution), not on publisher push (upstream) — this distinction sets the direction for every capacity-planning decision that follows.
Understand an Edge node's two capacity tiers: the engineering ceiling ("full capacity") versus the "configured capacity" actually run in production, why the production baseline leaves elastic headroom, and what to do once concurrency nears configured capacity.
Understand the fallback for when scaling can't keep up: once the superadmin turns on the "business degrade permission," the capacity alarm auto-triggers a 1080p→720p downgrade — see how it buys a window of buffer time between a second-scale burst and minute-scale scaling.
Run the numbers with real AWS specs: how many viewers a single node, and a single Region, can actually hold, and whether this margin is enough for a typical game-streaming peak.

1. First, locate it: a burst stresses the "viewer", not the "publisher"¶

A live stream has two completely different load paths:

flowchart LR
    OBS["Publisher OBS push"] -->|"Upstream: 1 stream, roughly constant bitrate<br/>simulcast 3-layer packaging, doesn't vary with viewer count"| Origin["Origin"]
    Origin --> Edge1["Edge node 1"]
    Origin --> Edge2["Edge node 2"]
    Origin --> EdgeN["Edge node N"]
    Edge1 -->|"Downstream"| V1["Viewers × some"]
    Edge2 -->|"Downstream"| V2["Viewers × some"]
    EdgeN -->|"Downstream: one egress stream per viewer<br/>grows linearly with viewer count"| VN["Viewers × some"]

Upstream (publish): there's only one link from publisher to Origin; the bitrate is set by the simulcast layers and the HEVC/H264 multi-track together, and doesn't change whether viewers go from 10 to 100,000. This uplink uses the publisher's own bandwidth; PPCDN does not bill for uplink and needs no cloud transcoding.
Downstream (pull): every viewer pulls a copy of the media stream from some Edge; an Edge's egress bandwidth, connection count, and CPU/memory usage all grow linearly with viewer count — this is the part that actually gets crushed by a "burst."

In one line: whenever you discuss a "burst of high concurrency," the first move is always to focus on downstream distribution at viewer scale; the upstream publish link runs on an entirely separate load model that doesn't change no matter how much the viewer count spikes.

2. Edge node capacity: full capacity vs. configured capacity¶

Every Edge node has two distinct capacity concepts — don't conflate them:

Concept	Meaning	Value (2vCPU/4GB node)
Full capacity	The engineering ceiling — push concurrency any higher and the node is the first thing to give out	250 concurrent pulls
Configured capacity	The admission threshold actually set in production; the scheduler makes admission decisions against this number	100 concurrent pulls
Elastic headroom	The buffer left between full capacity and configured capacity	150 concurrent pulls

250 isn't a number pulled out of thin air: it's the measured ceiling EP14 hit during a stepped load test on a 2vCPU/2GB instance — a clean pass, 0 disconnects (the next step up, 300 concurrent, is the failure point where memory runs out, and doesn't count as capacity). Production actually runs on the more memory-comfortable 2vCPU/4GB instance, so using a number verified on tighter hardware as the full-capacity figure already carries its own safety margin — there's no need to re-run a separate load test for the 4GB instance.

Single-node concurrency capacity (2vCPU / 4GB)
Full capacity (engineering ceiling)   ██████████████████████████████  250
Configured capacity (production baseline) ████████████                      100   ← day-to-day operation runs below this line
Elastic headroom                     ██████████████████         150   ← reserved for burst peaks

Why does configured capacity leave headroom instead of running straight up to the 250 full capacity?

Viewers arrive in pulses: the instant a stream goes live, or when popular content gets shared around, the inflow is far faster than the scaling process.
If the daily baseline already runs near full capacity, the moment a peak hits you're stuck waiting for a new node to come online — that wait can't be skipped.
Setting the daily baseline at 100 effectively reserves 150 of emergency headroom on every node ahead of time, so when a burst hits, the node itself can absorb more right away without waiting on a machine.

Configured capacity is only the ceiling of the "comfort zone," not a hard cap:

flowchart TD
    A["Viewer concurrency keeps climbing"] --> B{"Nearing configured capacity (100/node)?"}
    B -- "Not yet" --> A
    B -- "Near/reached" --> C["Capacity alarm fires (80% warning / 95% critical)"]
    C --> D["Start adding Edge nodes to share load<br/>new nodes come in at the same configured-capacity standard"]
    D --> E["Overall configured capacity scales linearly with node count"]

Once concurrency nears configured capacity, the move is to add nodes — not let a single node grind all the way up to full capacity. The headroom reserved by full capacity is there for that brief window where "adding a node isn't fast enough" — that's exactly what the next section covers.

3. The fallback for when scaling can't keep up: business degrade permission + automatic capacity-alarm linkage¶

Scaling is minute-scale (requesting a machine, registering it, joining the node pool), but viewers can flood in at second-scale. That gap in between is covered by an authorization flag the superadmin pre-enables — the business degrade permission.

This switch in the ppcenter admin console isn't a button that directly executes a downgrade — it's a permission flag for whether a capacity alarm is allowed to auto-trigger a downgrade. Once the superadmin turns on "business degrade permission," if scaling can't keep up, the capacity alarm will automatically cap platform-wide playback quality at 720p; if the permission is off, the capacity alarm only goes through the normal manual-scaling process and won't auto-downgrade quality.

stateDiagram-v2
    state "Normal: 1080p available, concurrency ≤ configured capacity (100/node)" as Normal
    state "Alert: capacity alarm fires (80%/95% threshold)" as Alert
    state "Auto-degrade: business degrade permission=true → platform-wide playback cap locks at 720p" as Degrade
    state "Manual scaling only: business degrade permission=false, no auto-degrade" as ManualOnly
    state "Recover: new node online, concurrency falls back, cap lifted" as Recover
    [*] --> Normal
    Normal --> Alert: concurrency nears/exceeds configured capacity
    Alert --> Degrade: business degrade permission = true
    Alert --> ManualOnly: business degrade permission = false
    Degrade --> Recover: scaling complete + concurrency falls back
    ManualOnly --> Recover: scaling complete + concurrency falls back
    Recover --> Normal: cap lifted gradually, back to 1080p

Why dropping to 720p is a lifesaver: for the same bandwidth and the same number of connection slots, carrying 720p viewers is lighter than carrying 1080p viewers; temporarily capping playback quality by one notch pushes a node's "effective viewer capacity" from the configured 100 toward the full-capacity 250. Once the business degrade permission is on, this step doesn't need a new machine or a human to click anything — the moment a capacity alarm fires, this switch immediately frees up headroom on the node.

Degradation and scaling are two lines racing in parallel, not a sequence:

sequenceDiagram
    participant V as Viewer influx
    participant M as Capacity alarm
    participant A as Business degrade permission (superadmin preset)
    participant P as ppcenter auto-degrade (second-scale)
    participant O as Node scaling (minute-scale)
    V->>M: Concurrency spikes, hits configured capacity
    M->>M: capacity_high alarm (80%/95%)
    M->>A: Check business degrade permission
    A-->>M: Permission = true
    M->>P: Auto-dispatch disable1080p
    P->>P: Broadcast to all Edge nodes over the existing heartbeat channel
    Note over P: Takes effect in seconds, playback cap locked at 720p, headroom freed
    M->>O: Simultaneously trigger the scaling process
    Note over O: Minute-scale: request machine → register → join node pool
    O->>M: New Edge node online, overall configured capacity expands
    M->>P: Concurrency falls back below configured capacity, 720p cap lifted gradually

Degradation is responsible for "holding the line for those few minutes," scaling is responsible for "actually raising the capacity baseline" — only together do they make the "configured capacity plus elastic headroom" design actually pay off; and the precondition for this whole automatic linkage is that the superadmin has already turned on "business degrade permission."

4. AWS deployment specs and headroom: is this enough for typical game-streaming capacity?¶

The single-node (2vCPU/4GB) full capacity / configured capacity stays the same:

Scale	Full capacity	Configured capacity	Elastic headroom
Single node (2vCPU/4GB)	250 concurrent pulls	100 concurrent pulls	150 concurrent pulls

But taking this single-node figure and simply multiplying it by the Edge count to extrapolate a single-Region total is the method an earlier version of this episode (and EP15) both used — "250×30=7500." That method is wrong, and needs correcting first.

Correction: single-Region capacity cannot be linearly extrapolated as "single-node capacity × Edge count." The real measurement later added to docs/deployment/ppcdn-production-deployment-plan.md §4.8 overturns this method: Origin uses forwardMmx to broadcast every live stream in full to every single Edge, so Origin's own egress bandwidth = concurrent-stream count × Edge count × per-stream bitrate — a figure that's completely unrelated to viewer count, and it saturates Origin's own network interface long before the Edge count climbs anywhere near 30. The real scaling bottleneck isn't on the Edge side at all — it's at Origin.

Measured against the full 7.5Mbps simulcast ladder (a viewer actually only pulls one layer of it, evaluated at 1.5Mbps/viewer), for an Origin on a 5Gbps network tier:

Edge count	Nature	Peak viewers (`$12` tier Edge)	Peak viewers (`$24` tier Edge)
15	5 Gbps Origin safe value	~2,700	~3,300
22	5 Gbps Origin theoretical ceiling	~4,000	~4,800

Single-Region peak viewers (bounded by Origin's NIC, not a linear single-node extrapolation)
15 Edge nodes ($24 tier) ████████████████████████████████          ~3,300
22 Edge nodes ($24 tier) ████████████████████████████████████████  ~4,800
                     (for comparison: the old 250×30 linear extrapolation of 7500 doesn't hold)

These numbers are much smaller than the old "250×30=7500," for a direct reason: a single stream alone costs Origin Edge count × 7.5Mbps of egress; as Edge goes from 15 to 22 nodes, Origin's egress load climbs nearly 50% in lockstep, and the thing that hits the ceiling first is Origin's own network interface, not Edge's CPU/memory.

There's another layer in instance-size choice: under a 1.5Mbps viewer load, the $12 tier (2vCPU/2GB) hits a memory bottleneck first on a single instance; the $24 tier (2vCPU/4GB) has double the memory and 33% more traffic allowance for the same price, so a single instance can carry noticeably more viewers — for the same budget, the $24 tier is the better deal, which is where the gap between the table's two columns comes from.

To break past this roughly 15-to-22-Edge ceiling, there are three paths: change forwardMmx from "full broadcast" to "on-demand forwarding" (only forward to Edges that actually have viewers watching — the most effective option), use multiple Origins in a layered fan-out, or move to an Origin instance with a bigger network tier.

(The capacity estimates above assume each viewer pulls 1.5Mbps of H264 and do not count the Edge egress savings from HEVC and P2P; EP20 gives how those are computed.)

A typical game stream's single-event peak viewership ranges from a few hundred to the low thousands; only a handful of top-tier tournaments push into five figures. Measured against that scale: 15 Edge nodes ($24 tier) at a ~3,300 peak already covers most events; for a top-tier event genuinely approaching or exceeding that scale, the first move shouldn't be simply adding more Edge nodes — past this scale, adding Edge count stops being the most effective scaling lever, and on-demand forwarding or multi-Origin layering needs to come first.

5. Stringing the three lines of defense together¶

flowchart TD
    A["Viewer concurrency starts climbing"] --> B["Line 1: configured capacity<br/>daily baseline, leaves elastic headroom for bursts"]
    B -->|"Nears/reaches configured capacity"| C["Capacity alarm (80% warning / 95% critical)"]
    C --> D["Line 2: auto-degrade (second-scale)<br/>business degrade permission=true → platform-wide playback cap 720p<br/>frees headroom, pushes toward full capacity"]
    C --> E["Line 3: node scaling (minute-scale)<br/>new Edge joins the node pool at the same configured-capacity standard"]
    D --> F["Concurrency falls back below configured capacity"]
    E --> F
    F --> G["720p cap lifted gradually, back to 1080p"]

The three lines of defense divide the work cleanly: configured capacity decides how much headroom the daily baseline leaves, auto-degrade buys time at second-scale (provided the business degrade permission is already on), and node scaling actually raises the capacity baseline. Only together do they cover the scenario that "full capacity 250 / configured capacity 100 / elastic headroom 150" is really designed for.

6. Wrap-up and next episode (script notes)¶

To handle a burst of high concurrency, first separate the two load paths of "viewer" and "publisher" — the one that actually gets driven up is the viewer-pull downstream link. An Edge node's configured capacity leaves elastic headroom; nodes get added once concurrency nears configured capacity; once the superadmin turns on "business degrade permission," if scaling can't keep up, the capacity alarm automatically caps platform-wide playback quality at 720p, trading a second-scale degradation for the time minute-scale scaling needs. Landing it on real AWS specs: a single node at 2vCPU/4GB has a full capacity of 250; but a single Region's peak can't be obtained by simply multiplying that number by the Edge count — Origin's own network interface is the real scaling bottleneck, with 15 Edge nodes ($24 tier) peaking around 3,300 and 22 nodes around 4,800. This margin is enough for typical game-streaming capacity, but traffic at the level of a top-tier tournament needs on-demand forwarding or multi-Origin layering planned ahead of time.

Next episode, we'll deploy the entire system from scratch and run today's mechanisms end to end in 15 minutes.