How NVLink Fusion Connects d-Matrix Inference Chips to Nvidia Server Racks

Data center operators can now slot third-party artificial intelligence inference chips directly into Nvidia's proprietary server racks, splitting the workload of generating text between different processor architectures to speed up response times. The integration, announced on September 10, 2026, connects d-Matrix's upcoming Raptor processors to Nvidia's infrastructure using a technology called NVLink Fusion. By opening up the high-speed interconnect fabric that previously only linked Nvidia's own graphics processing units, the collaboration allows facilities to mix and match hardware without rewiring their data centers. The move signals a shift in how the industry approaches the physical infrastructure of artificial intelligence, moving away from homogeneous clusters toward specialized, disaggregated systems.[1][2]

The architectural shift addresses a fundamental bottleneck in how large language models serve users in production environments. When a person prompts an artificial intelligence system, the underlying neural network performs two distinct computational tasks to deliver the answer. The first step is reading and processing the input prompt, known as the prefill phase. The second step is generating the response word by word, known as the decode phase. Because these two phases place entirely different demands on the hardware, running both of them on the same processor forces data center operators to compromise on either speed or cost, creating inefficiencies as models scale to serve millions of simultaneous users.[5]

The prefill phase is highly parallel and compute-intensive, making it an ideal workload for the massive parallel processing capabilities of graphics processing units like Nvidia's Vera Rubin architecture. The decode phase, however, is sequential and heavily memory-bound. The model must load its entire weight matrix into memory for every single token it generates, creating a latency bottleneck where the processor sits idle waiting for data to arrive from the memory chips. Because the compute cores can process data much faster than the memory can supply it, the decode phase effectively starves the processor, wasting the expensive computational power that makes graphics processing units so valuable for training.[3]

To solve this memory wall, data center operators are moving toward a deployment model called heterogeneous disaggregation. Instead of forcing a single graphics processing unit to handle both phases inefficiently, the workload is split across specialized hardware operating in tandem. Nvidia processors handle the heavy prefill phase, computing the context of the prompt, and then hand that context over to specialized inference processing units designed specifically to accelerate the memory-bound decode phase. This division of labor ensures that each piece of silicon is performing the exact mathematical operation it was engineered to execute, maximizing the overall throughput of the server rack.[1]

The primary technical challenge of disaggregation is moving the data between the graphics processor and the inference processor fast enough that the handoff does not erase the speed gains. Standard Ethernet connections introduce too much latency for real-time applications, creating a bottleneck at the network switch. Nvidia's NVLink Fusion solves this by extending its proprietary scale-up fabric—previously reserved strictly for Nvidia's own silicon—to third-party processors. By adopting the standard, external chip designers can allow their hardware to communicate with Nvidia processors as if they were part of the same physical chip, bypassing the traditional network stack entirely.[4][6]

NVLink Fusion provides a high-bandwidth, low-latency domain within the server rack, effectively pooling the memory of multiple processors. According to Nvidia, the sixth-generation NVLink delivers three terabytes per second of all-to-all bandwidth per inference processing unit. The company states this architecture provides packet rates ten times higher and latency three times lower than off-the-shelf Ethernet networking, enabling the rapid context switching required for disaggregated inference. By eliminating the network bottleneck, the fabric allows the prefill and decode phases to operate continuously without stalling, keeping the utilization rates of the expensive graphics processing units as high as possible.[1][3]

Beyond the interconnect, the collaboration integrates d-Matrix chips directly into Nvidia's MGX rack architecture. MGX is a modular reference design that standardizes the physical trays, power delivery, and liquid cooling infrastructure across data centers, allowing facilities to upgrade their compute without rebuilding the room. "Demand for inference is soaring, but capital, time and energy remain finite," said Sid Sheth, cofounder and CEO of d-Matrix. "With NVLink Fusion and MGX, we can integrate our Raptor XPUs into a broadly deployed, liquid-cooled architecture, giving customers a faster, lower-risk path to deploy and scale ultralow-latency inference."[1]

The Raptor inference processing unit itself tackles the decode memory bottleneck through a novel physical design. Rather than placing memory chips next to the compute cores on a two-dimensional circuit board—the standard approach for most accelerators—d-Matrix utilizes a three-dimensional stacking approach. The company previewed the architecture at the 2026 Hot Chips conference, detailing how the physical arrangement of the silicon alters the speed at which data can be retrieved. By changing the geometry of the chip, the design minimizes the physical distance the electrical signals must travel during the repetitive memory fetches required to generate text.[2][5]

Specifically, the Raptor package stacks a dynamic random-access memory chip directly on top of a static random-access memory compute chip, forming a single two-story package connected by microscopic vertical pathways. This physical proximity drastically reduces the distance data must travel, lowering the power consumption required to push the signal and increasing the speed at which the compute cores can fetch the model weights during the sequential decode phase. Because the decode phase is entirely dependent on memory bandwidth, this vertical integration directly translates to faster generation speeds for the end user.[2]

The resulting architecture targets what d-Matrix calls the "premium token economy." These are enterprise applications where interactivity is the primary value proposition and customers are willing to pay a premium for instantaneous speed. Examples include real-time voice agents that must respond without awkward pauses, live customer service chatbots, and artificial intelligence coding assistants that generate suggestions as the programmer types. In these use cases, the latency introduced by traditional hardware configurations breaks the illusion of a seamless conversation, making the specialized decode acceleration a strict requirement rather than a luxury.[2][5]

For data center operators, the integration offers a way to scale their inference capacity without purchasing additional flagship graphics processing units. By slotting Raptor inference processors into the same liquid-cooled racks alongside Nvidia central processing units, BlueField data processing units, and Spectrum-X Ethernet networking gear, hyperscalers can optimize their hardware mix for the specific economic demands of inference workloads. This flexibility allows cloud providers to offer tiered pricing models, routing batch-processing tasks to standard hardware while reserving the disaggregated, high-speed racks for customers demanding real-time performance.[3][4]

The physical hardware required to execute this disaggregated architecture is not yet in mass production. The Raptor chip is expected to complete its final design stage, known in the semiconductor industry as tape-out, before the end of 2026. Following manufacturing and testing, initial availability of the integrated inference processors within the Nvidia MGX rack ecosystem is projected for the fourth quarter of 2027, pending final validation by the frontier artificial intelligence laboratories currently evaluating the silicon.[5]

While d-Matrix is an early adopter of the standard, the success of NVLink Fusion depends on broader ecosystem uptake and complex software integration. Nvidia has announced a roster of other hardware partners, including Astera Labs for custom connectivity solutions, but the long-term viability of heterogeneous disaggregation relies on software frameworks seamlessly managing the handoff between competing silicon architectures at scale. If the software layer cannot route the prefill and decode tasks efficiently, the theoretical hardware speeds will not materialize in production environments.[3][6]