Cloudflare saves 100TB RAM by shrinking DNS cache
Cloudflare Unlocks 100TB of RAM Through Clever Memory Optimization
In the world of hyperscale data centers, efficiency is king, and Cloudflare has just demonstrated a masterful way to reclaim staggering amounts of memory without touching a single physical RAM module. By rethinking the very structure of how its DNS cache is stored, the company has freed up approximately 100TB of RAM across its global fleet.
This massive reclamation effort wasn’t about swapping out hardware; it was a triumph of clever software engineering. Systems engineer Sebastiaan Neuteboom details how the team performed a deep dive into the Big Pineapple platform, the engine behind the highly popular 1.1.1.1 resolver, making subtle but profound changes at the Rust level.
The core of the optimization involved shrinking each cached DNS entry. Previously, these entries consumed a considerable amount of space; the team successfully reduced the size of each entry from 953 bytes down to just 420 bytes. This seemingly small reduction multiplied across the system, yielding monumental results.
The impact of this design shift was immediate and dramatic. By eliminating wasted space—just one wasted byte per entry costs the fleet more than 250GB of memory, given the sheer volume of cached entries—Cloudflare recovered memory equivalent to the combined DDR5 used in 130 of its most powerful 768GB Gen 13 servers.
Beyond saving raw memory, the optimization delivered significant performance boosts. The redesigned structure made the cache faster, improving insert throughput by up to 43% and reducing lookup latency by 19%. This means that users experience lightning-fast DNS resolution while the underlying infrastructure operates with unprecedented efficiency.
The team achieved this by implementing several strategic changes. They replaced traditional growable containers with fixed-size boxed slices, collapsed multiple record lists into a single indexed buffer, and streamlined data storage by packing record data contiguously. These changes also improved CPU cache locality, allowing data to be copied directly instead of being re-serialized field by field.
During the rollout, which ran from mid-May to early July, the efficiency gains were tangible. The p99 resident memory per instance dropped significantly, moving from 9.3GB down to 5.3GB. This effort marks the second major memory reclamation project for Cloudflare in the last year, following their successful FL2 rewrite of their request-handling layer.
Crucially, the liberated RAM isn’t just sitting idle. Cloudflare plans to strategically reintroduce this freed memory back into larger DNS caches. This approach is designed to raise hit rates and further cut the query traffic sent upstream to authoritative servers, further solidifying the company’s commitment to hyper-efficient, scalable infrastructure.
Buy on Amazon
Credit
