AMD is betting that the next phase of AI infrastructure will need far more than GPU acceleration. Its Helion rack blueprint dedicates up to 32,768 EPYC CPU cores to agent execution, while leaving the system’s power efficiency, topology and real-world economics unproven.
AMD has introduced Helion, a rack-scale computing blueprint designed to address a growing challenge in artificial intelligence infrastructure: the computational work surrounding a model can become almost as important as the model itself.
As AI systems evolve from single-prompt interactions into agents that retrieve information, execute code, query databases, invoke external tools, and coordinate multistep workflows, the infrastructure supporting them must do more than accelerate matrix operations. It must manage concurrent processes, sustain application logic, move data between services and coordinate repeated interactions with inference engines.
AMD’s Helion architecture attempts to turn that emerging requirement into a distinct infrastructure tier. Its hyperscale configuration can scale to 32,768 CPU cores using sixth-generation AMD EPYC processors, with agent execution handled separately from the infrastructure serving AI models. An alternative configuration combines EPYC CPU nodes, AMD Instinct MI350P GPU nodes and general-purpose computing within a rack-scale design intended for enterprise deployments.
The announcement is significant because it challenges the increasingly familiar assumption that expanding AI capacity primarily means adding more accelerators. But it also raises questions that AMD has not yet answered: How much CPU capacity does a production agent workload actually require? What does the rack consume at full operating load? And does separating agent execution from model inference improve performance and economics enough to justify another layer of infrastructure?
For now, Helion is a design announcement, not a validated performance result.
The hidden computing workload behind AI agents
The distinction between model inference and agent execution is central to understanding Helion.
A conventional inference workload sends input to a trained model and processes the resulting output. Depending on the model and serving configuration, the dominant computational demands can include matrix multiplication, attention operations, memory bandwidth, and moving model parameters through accelerator memory.
Agentic systems introduce additional work around that inference cycle.
An agent may interpret a request, select a tool, retrieve documents, execute a program, query a database, inspect a result, and call a model again before completing a task. Multiple agents may perform these operations concurrently, with each workflow generating its own scheduling, data-access, and coordination demands.
Much of this activity involves general-purpose computation rather than sustained GPU-intensive numerical operations.
CPU resources can be required for:
- Workflow orchestration: Scheduling tasks, managing dependencies and coordinating multistep execution.
- Tool execution: Running application code, invoking APIs and interacting with databases or enterprise services.
- Retrieval and data preparation: Processing requests, filtering results, coordinating storage access and preparing context for model inference.
- Concurrent session management: Maintaining execution state and coordinating large numbers of active workflows.
- Systems integration: Connecting model-serving infrastructure to existing software, security controls and operational services.
These operations do not all have identical resource requirements. Some are CPU-intensive; others are constrained by memory latency, network performance, storage access or external service response times. A database query waiting on a remote system, for example, does not necessarily become faster simply because additional CPU cores are available.
That distinction matters because AMD’s maximum core count is a capacity specification, not a measurement of completed agent tasks.
The underlying architectural thesis is nevertheless credible: when AI applications generate large numbers of tool calls and control-flow operations, allocating every computing resource to GPU inference may be an inefficient way to build the entire system.
The harder question is how to size that additional CPU capacity for real workloads.
Two configurations, two different infrastructure strategies
AMD describes two Helion configurations that address different deployment requirements.
The hyperscale approach: separate agent execution from inference
The CPU-only configuration uses sixth-generation AMD EPYC processors and supports a disclosed maximum of 32,768 CPU cores. It is intended to run agent-execution workloads while connecting to separate infrastructure responsible for model inference.
This arrangement follows a disaggregated computing model. Instead of treating CPUs and GPUs as components that must always reside in the same rack, operators can allocate resources to separate functional tiers.
A CPU-heavy execution pool could manage workflow logic and tool interactions, while an accelerator pool handles model requests. The two systems would communicate over a network fabric, allowing each tier to be provisioned according to its own utilization and capacity requirements.
In principle, this separation could improve resource utilization. CPU capacity would not have to scale in lockstep with GPU capacity, and an organization could expand its agent-execution tier without purchasing additional accelerators whenever orchestration demand increases.
It could also create new operational challenges.
The network between the execution and inference tiers becomes part of the application’s performance path. Each model request may involve request serialization, network transmission, inference queueing, and response handling. Agent workflows that repeatedly alternate between local computation and remote model calls could become sensitive to communication latency.
Storage and retrieval traffic add further complexity. A system may have abundant CPU capacity but still spend substantial time waiting for data, database responses, or inference capacity.
Consequently, the relevant performance measurement is not simply how many agent operations the CPU rack can execute. It is how quickly the complete system can finish a defined workload under realistic concurrency, network conditions, and inference demand.
AMD has not disclosed a Helion-specific benchmark establishing that result.
The enterprise approach: CPUs and GPUs in one rack-scale design
The second configuration combines EPYC CPU nodes, Instinct MI350P GPU nodes and general-purpose compute within a single rack-scale concept.
This integrated approach addresses organizations that prefer a more consolidated deployment rather than maintaining physically separate agent-execution and inference pools.
Combining these resources may simplify certain infrastructure decisions and reduce the operational distance between application execution and model serving. It also gives system integrators room to configure the balance between CPU capacity, accelerator capacity, and general-purpose computing according to deployment requirements.
However, an integrated rack is not automatically a more efficient rack.
The optimal CPU-to-GPU ratio depends on the application mix, model sizes, request rates, concurrency levels, and the proportion of time agents spend executing tools versus waiting for inference results. A configuration optimized for retrieval-heavy enterprise agents may be poorly matched to a workload dominated by repeated, large-model inference.
The same variability affects memory capacity, storage throughput and network requirements.
AMD has not disclosed a complete Helion bill of materials, including exact processor models, socket and node counts, memory configurations, or rack-level power limits. Without those details, customers cannot yet determine how closely the reference design matches their intended workload.
The 32,768-core figure needs engineering context
The headline number is impressive, but raw core counts are a poor substitute for a complete system specification.
AMD says the CPU-only configuration can scale to 32,768 cores. That tells prospective customers something about the design’s intended computational density. It does not reveal how those cores are distributed across servers, how much memory is available to them, or how the system behaves under sustained load.
The launch announcement does not specify the exact Helion node topology, processor SKU, socket count, memory capacity, rack dimensions, or maximum power envelope.
Those omissions matter in high-performance computing because system behavior depends on much more than aggregate processing resources.
Memory bandwidth determines how quickly cores can access data. Memory capacity constrains the number and size of resident workloads. NUMA topology affects access latency and the cost of moving data between processor domains. Network bandwidth and latency determine how efficiently distributed nodes coordinate with external services and inference clusters.
Software scheduling also matters. A large collection of cores is useful only if the workload can exploit the available parallelism without creating excessive synchronization, memory contention, or scheduling overhead.
Agent execution presents a particularly uneven workload. Some agents may spend most of their time waiting for external services, while others may perform substantial data processing or code execution. A large core allocation can help when enough independent CPU work is available, but it can also leave expensive resources underutilized when the workload is dominated by latency or serialized dependencies.
The rack’s power envelope is equally important.
A system with tens of thousands of cores must be evaluated against the facility infrastructure required to support it, including power distribution, cooling, networking, and serviceability. Air-cooled and liquid-cooled versions could offer different deployment options, but AMD has not provided the configuration-level power and cooling data needed to establish their relative density or efficiency.
The distinction is straightforward: 32,768 cores describe the scale of the proposed system, not the amount of useful work it will deliver per watt or per dollar.
The token router could matter as much as the hardware
Helion also includes an integrated token router that AMD says can direct model calls across frontier and open-source models.
The concept addresses another important feature of agentic AI infrastructure. A production application does not necessarily need the same model for every operation. A relatively simple classification or extraction task may be suitable for a smaller model, while complex reasoning or specialized generation may require a more capable system.
Routing requests across available models could, in principle, help operators balance capability, latency, cost and capacity.
But the announcement provides no detailed architecture, routing-policy description, latency measurements, or cost data for Helion’s token router. It also does not establish which routing decisions are supported or how the component integrates with model-serving infrastructure.
Those details are consequential.
A useful router must do more than direct traffic to an available endpoint. Depending on its intended role, it may need to account for model capability, request complexity, queue depth, endpoint availability, context requirements, and service-level objectives. Enterprise deployments may additionally require policies governing data privacy, model authorization, auditability, and which workloads can use external services.
Routing decisions can also change the economics of an agent workload. Sending more requests to smaller models may reduce inference expenditure, but only if the resulting answers remain sufficiently accurate and do not trigger additional retries, escalations or corrective work.
AMD has not yet published enough information to establish whether its router delivers these benefits or how it compares with existing orchestration and model-routing approaches.
For customers evaluating Helion, the software layer deserves scrutiny alongside the hardware specification. A powerful CPU rack cannot compensate for inefficient workflow design, poorly selected models or an orchestration system that generates unnecessary inference requests.
The open-infrastructure promise is not yet an accepted standard
AMD is positioning Helion as an open, rack-scale blueprint based on Open Compute Project specifications and says it plans to contribute the Helion specification to the Open Compute Project.
That distinction is important.
A design based on existing OCP specifications is not the same thing as a newly accepted OCP standard. The planned contribution represents a future step toward formalizing the design, not evidence that the complete Helion architecture has already been approved through that process.
Openness could nevertheless become a meaningful commercial advantage.
Rack-scale infrastructure involves coordination across processors, servers, power distribution, cooling, network fabrics and system-management software. A sufficiently detailed, interoperable blueprint could give manufacturers a common starting point while allowing them to customize configurations for specific customers.
AMD identifies support from eight major system and infrastructure companies: Cisco Compute, Dell Technologies, Hon Hai Technology Group, HPE, Lenovo, QCT, Supermicro and Wiwynn.
That roster is a meaningful signal that established manufacturers are interested in the design and may be preparing implementations.
It is not proof of customer adoption.
Supporting statements do not establish purchase orders, completed validation, production deployments, or a common availability date across the participating vendors. The announcement does not identify a named end customer operating a Helion system in production.
Nor does the participation of multiple manufacturers guarantee that their resulting configurations will deliver comparable performance, cooling characteristics, or operational behavior. Those outcomes depend on implementation details and validation.
For customers, the useful milestone will be a documented reference architecture with specific component configurations, supported software, interoperability testing and measurable operating characteristics.
November availability remains a target
AMD says partner-built Helion configurations are under development and planned for November 2026.
That language should not be interpreted as confirmation that every configuration will be orderable or deployable next month.
The announcement does not establish which partners will ship first, which configurations will be available, where they will be offered, or whether November refers to orderability, initial shipments, or customer deployment.
Those distinctions are especially important for rack-scale systems. Procuring servers is only one part of the process. Customers must also account for facility readiness, power and cooling capacity, network integration, software qualification and operational acceptance.
Until vendors publish specific product information and orderability details, November should be treated as a planned milestone rather than a confirmed delivery schedule.
The benchmark AMD still needs to publish
The most important missing evidence is a reproducible comparison of complete agent workloads.
AMD’s announcement does not provide independent benchmarks, defined workload characteristics, utilization measurements, power figures, or a cost model specific to Helion. Earlier modeled comparisons involving agentic-AI workloads should not be treated as measurements of this newly announced design.
A credible evaluation would need to define what the system is actually doing.
For example, a benchmark could measure completed agent tasks per second, end-to-end task latency, successful task completion rates, and tail latency under increasing concurrency. It should document the number and type of model calls, the proportion of tool execution, the retrieval workload, the database behavior, and the network configuration.
Power measurements should cover the relevant computing infrastructure rather than relying on processor specifications alone. For a disaggregated deployment, that means accounting for the CPU execution tier and the resources required to serve its model requests. For a consolidated configuration, it means measuring the combined system under a clearly defined workload.
Cost comparisons should include more than hardware acquisition. They should consider the cost of delivering a completed task, including accelerator utilization, electricity, cooling, networking, software operations, and the capacity needed to meet service-level requirements.
The comparison also needs a meaningful baseline. A CPU-led architecture should be tested against a realistic alternative configured for the same task and completion criteria, not an artificially underprovisioned system.
Finally, performance should be evaluated across different agent workload profiles. A retrieval-heavy workflow, a code-execution agent, and a model-intensive reasoning agent may produce very different results on the same hardware.
Until such measurements are available, claims of superior performance per watt, lower total cost or a general CPU advantage would be premature.
A credible architectural thesis, not yet a proven advantage
Helion reflects a broader shift in AI infrastructure thinking. As organizations move from isolated model queries toward systems that coordinate tools, data, and multistep execution, the supporting compute architecture may need to evolve beyond the accelerator-centric designs that have dominated much of the current AI investment cycle.
AMD’s proposal is that high-density CPU capacity can become a distinct resource for this work, either operating alongside a separate inference cluster or sharing a rack-scale design with Instinct accelerators.
That is a technically credible direction. It could allow operators to provision agent execution independently, give system builders more flexibility in balancing compute resources, and create an infrastructure framework for applications that place substantial demands on general-purpose processing.
But the economic case remains unproven. Agent workloads vary widely, CPU resources can be underutilized when applications are waiting on external systems, and disaggregation introduces network and operational requirements of its own. A large core count does not resolve those challenges automatically.
For SuperComputingOnline’s audience of HPC engineers, systems architects and AI infrastructure operators, the next phase of the story should focus on the details AMD has yet to publish: the reference topology, rack power, memory and network configuration, token-router implementation, vendor-specific availability and measured performance on representative workloads.
Helion is worth watching because it makes the CPU’s role in agentic AI an explicit rack-scale design decision. Whether it becomes an important new class of AI infrastructure will depend on evidence AMD has not yet supplied: useful work completed, at what latency, with what power consumption, and at what total cost.
