AMD MI455X doubles memory capacity, Hugging Face tests confirm 3x request throughput

At AMD's Advancing AI 2026 event, the company unveiled the Instinct MI455X, a datacenter GPU with 432 GB of HBM4 memory and 23.3 TB/s of bandwidth. That's roughly 2.25 times the 192 GB of the MI300 generation. For teams running large language models, the immediate effect is fewer cards to hold a model and more room for the key-value cache that grows with each request, a pattern linked to the broader memory shortage driving demand for such chips.

Hugging Face got early access to a four-accelerator system and ran a preliminary capacity test. They loaded a 64 GB Qwen3-32B BF16 model on a single MI455X and increased concurrent requests until memory ran out. The MI300 hit its limit first; the MI455X handled more than three times the requests before it reached its own ceiling. The difference comes almost entirely from the larger KV cache capacity after the model weights are loaded, addressing the infrastructure bottleneck that memory upgrades aim to solve.

The team also ran the Transformers test suite against 24 important model architectures including encoders, decoders, vision and audio models, and multimodal systems. After fixes contributed alongside AMD, the success rate reached 99.5 percent, putting it in the same range as the MI300 at 99.4 percent and NVIDIA's A10 at 99.1 percent. Getting there required enabling a stable Flash Attention path, adding torchcodec support for audio and video inputs, and resolving output comparison issues uncovered during testing.

Memory has become the bottleneck for many inference and serving setups, especially with long-context workloads that stress memory capacity. Long-context workloads, large batch sizes, and multi-turn conversations all drive KV cache growth. A single card that can hold more of that state means fewer nodes, simpler networking, and lower latency from reduced cross-device communication. AMD has not shared clock or architecture details yet, so the MI455X does not claim to be faster per FLOP than the MI300, but the memory upgrade alone shifts the economics for anyone packing many small to medium models on one machine.

What the test does not yet cover is raw throughput. Hugging Face said it will follow up with speed comparisons as testing progresses. Without that data, the MI455X is clearly a density play, fitting into the broader trend where orchestration and density matter more than raw size. It is designed for scenarios where the GPU runs out of memory before it runs out of compute. For model serving, that is often the case.

Hugging Face plans to add the MI455X to its Transformers continuous integration pipeline and expand validation across more libraries. The company is also working with AMD's AITER team to bring optimized kernels to the Hugging Face Kernel Hub, which would let users invoke them directly from Transformers workflows. These steps matter because compatibility with the existing ecosystem is what turns a hardware spec into something a data science team can actually use on Monday morning.