Sandisk/SK hynix unveil GPU tech for terabytes of memory

Bridging the Gap: How New Storage Tech is Fueling the AI Revolution

The race to power artificial intelligence is constantly hitting a bottleneck: memory bandwidth. As computing demands grow exponentially, the speed at which data can move from storage to the processor dictates the limits of what AI systems can achieve. Enter High Bandwidth Flash (HBF), a groundbreaking technology developed through a collaboration between Sandisk and SK hynix, poised to redefine this crucial link.

This new specification aims to seamlessly merge the non-volatility of 3D NAND flash with the blistering speed capabilities of High Bandwidth Memory (HBM). The result? A storage medium designed specifically for the demanding requirements of AI inference systems where massive, fast memory pools close to the compute unit are essential.

To bring this concept into the real world, Sandisk and SK hynix formally introduced HBF through the Open Compute Project (OCP), establishing it as an open standard. This collaborative approach ensures that the technology can be adopted widely across the industry, rather than remaining locked in proprietary interfaces.

The core promise of HBF lies in its exceptional performance potential. Initial specifications define HBF packages with capacities up to 512GB, utilizing specialized NAND die stacks. While these are not traditional 3D NAND structures, they are engineered to deliver superior access speeds necessary for next-generation workloads.

Performance is measured across a wide spectrum, offering bandwidth grades ranging from approximately 0.4 TB/s up to 3.0 TB/s. This impressive range signals that HBF is designed with a multi-year roadmap, allowing for multiple generations and implementations tailored to various computing needs.

In an eye-opening comparison, the most capable implementation of HBF targets speeds of 3 TB/s, potentially outperforming the memory bandwidth of a single HBM4 stack. While it may not match the latency advantages of HBM4, HBF delivers unparalleled capacity and near-memory bandwidth crucial for massive AI data processing.

The complexity behind this speed is fascinating. Achieving high bandwidth from limited storage requires sophisticated engineering. For instance, extracting 400 GB/s from a single 512GB package necessitates complex architecture, potentially leveraging advanced interfaces like the Universal Chiplet Interconnect Express (UCIe) to manage data flow efficiently.

This innovation positions HBF as a vital new memory tier, capable of providing substantially larger memory pools than current solutions. Where an HBM4 stack might cap out at 64GB, an HBF stack can offer up to 512GB, providing the necessary data volume for complex AI inference tasks.

The next big question is adoption: who will lead this revolution? Although Sandisk and SK hynix have set ambitious goals for the technology’s future, only a select few industry giants have signaled direct interest in participating in the HBF consortium. While some key players are exploring the potential, major semiconductor companies such as AMD, Nvidia, Intel, and others have not yet expressed interest in this specific direction.