The CPU in the Inference Era — What Is Measured Fact and What Is Forecast - Research Report

Reading depth

1. Asking the Question Precisely — 'Inference' Is Not One Block'How much CPU does inference need' answers only by segment: modest for serving, memory-bound for disaggregated, truly heavy only for agentic.Jump to section
2. The Journey of One Request — Breaking Down the Inference WorkloadThe CPU is the kitchen manager beside the cooking GPU. A slow manager stalls even a genius chef, and adding managers costs far less than chefs.Jump to section
3. Three Tiers of Evidence — Fact, Forecast, OffsetTiering the evidence clarifies it: neutral measurement points to 8-28 CPU cores per GPU, and CPU dominance is confined to agentic workloads.Jump to section
4. The CPU's Real Role — Three MechanismsThe CPU governs utilization, not replaces the GPU. It works across coordination, memory, and new workloads, a story of attach volume, not unit price.Jump to section
5. Competitive Landscape — the ARM Ecosystem and Attach DynamicsLong-term share goes to ARM, near-term pricing to x86 (AMD). But the essence of ARM's growth is GPU attach, which must not be forgotten.Jump to section
6. Investment Landscape — Three Defensible Pillars and the Korean Memory DerivativeThe defensible thesis rests on three pillars: attach volume, agentic demand, supply-driven pricing. Only the Korean memory derivative holds without a magnitude bet.Jump to section
7. What to TrackSix signals decide where and how real inference CPU demand is. Magnitude shows in agentic and CXL; the Korean benefit, in memory guidance.Jump to section
8. Conclusion — Where Measurement and Forecast Part'The CPU in the inference era' is a theme read by segment, not one stock. Primary: ARM, NVIDIA, AMD; derivative: Korean memory; true only in agentic.Jump to section

1. Asking the Question Precisely — 'Inference' Is Not One Block

The question 'does need this much CPU' cannot be answered as stated. That is because 'inference' lumps together three segments of utterly different CPU intensity. Separate the three and the claim sorts into where it holds and where it is overstated.

Segment	The CPU's role	CPU intensity
Mainstream serving	Tokenization, kernel launch, request scheduling (control plane)	Low to medium — but fatal if starved
Disaggregated inference +	Managing the decode-side memory tier (CPU DRAM, )	Memory-centric — not CPU compute
/	Tool execution, code compilation, verification sandbox	High — only here is 'this much' true

Segment

Mainstream token serving

The CPU's role: Tokenization, kernel launch, request scheduling (control plane)
CPU intensity: Low to medium — but fatal if starved

Segment

Disaggregated inference + KV offloading

The CPU's role: Managing the decode-side memory tier (CPU DRAM, CXL)
CPU intensity: Memory-centric — not CPU compute

Segment

Agentic / RL

The CPU's role: Tool execution, code compilation, verification sandbox
CPU intensity: High — only here is 'this much' true

The prior version merged the three into 'inference' and projected the intensity of #2 and #3 onto #1. This version treats the evidence level segment by segment.

'How much CPU does inference need' only yields an answer once you split the segments. Modestly for mainstream serving, as memory for disaggregated, truly for agentic. Bundle them into one block and the claim overstates itself.

The CPU in the Inference Era — What Is Measured Fact and What Is Forecast

Reader's Brief — 30-second TL;DR

1. Asking the Question Precisely — 'Inference' Is Not One Block

Full access requires 🥉 Bronze tier

Comments