Running OpenAI workloads on NVL72 at Nvidia SuperLab

Nvidia isn’t just building faster chips; they are fundamentally redesigning the architecture of the data center to handle the demands of agentic AI. Recently, the company invited journalists to witness this revolution firsthand at their Engineering SuperLab, offering an unfiltered look behind the curtain at how the Vera Rubin platform and next-generation power systems are being brought to life.

Stepping into the lab provided a stark contrast to the sleek keynote presentations. Instead of pristine showrooms, engineers were working with hardware in a live environment—demonstrating real workloads running within actual racks. This is where the complexity of building massive AI infrastructure meets the gritty reality of physical engineering. It’s a space filled with organized chaos, reflecting the intense effort required to manage staggering amounts of computation.

At the heart of this new platform is the Vera Rubin NVL72 rack design, representing Nvidia’s most complex and integrated AI and HPC system to date. The reveal showcased not just a tray, but an entire infrastructure built for scale. Engineers demonstrated how the trays slide together with retention arms in minutes, emphasizing the streamlined assembly process that underpins this massive hardware.

The design philosophy prioritizes efficiency and density. Unlike previous generations, the Vera Rubin architecture eliminates traditional cooling fans entirely, opting instead for full liquid cooling. This focus on thermal management is critical, as the racks employ a system called “dry cooling,” which allows the entire system to be cooled efficiently by heat exchangers, promising a future where data centers can operate much more quietly and with less energy spent on auxiliary systems.

But power delivery poses another monumental challenge. A single Vera Rubin NVL72 rack demands over 200 kW of power, pushing the limits of traditional infrastructure. To address this, Nvidia is pioneering an 800VDC power delivery system. This shift is designed to dramatically reduce energy loss associated with converting power and deliver massive wattage to compute racks more efficiently than ever before.

To achieve this, Nvidia demonstrated an 800VDC “sidecar”—a dedicated rack filled solely with power equipment—that acts as a retrofit for modern data centers. This sidecar rectifies the facility’s AC power into DC power and feeds adjacent racks, illustrating a critical step toward a more efficient electrical grid for AI infrastructure.

Communication is just as vital as compute. The internal architecture leverages Nvidia’s sixth-generation NVLink technology, which connects all chips in the rack—and across multiple racks—with unprecedented bandwidth. This communication fabric is supported by high-speed interfaces like the ConnectX-9 NICs and the Spectrum-X CPO switch trays, demonstrating how data can flow at lightning speed between processors.

Furthermore, Nvidia is tackling memory limitations head-on with its homegrown SOCAMM2 LPDDR5X memory system. This modular approach allows for greater flexibility in scaling memory capacity, offering density and modality similar to traditional DIMMs while reducing power costs significantly. The goal is clear: to build not just faster processors, but an entire interconnected, hyper-efficient ecosystem capable of powering the next wave of artificial intelligence.