Cloudflare trims cache memory by 100 TB via hash-map overhaul
Cloudflare reduced its server hash entries from 100,000 to 10,000, cutting memory usage by 100 TB. The optimization targets the Ketama algorithm used in its Pingora framework for routing URLs to cache servers. This follows a previous speed-focused optimization and addresses memory bloat in large-scale backend routing.
The optimization centers on how Cloudflare routes incoming URLs to cached content across its global server fleet. The Ketama algorithm assigns each server multiple hash points in a circular space, so when a server fails, its load spreads across neighbors rather than overwhelming one. Cloudflare had been using 100,000 hash points per machine, but statistical analysis showed 10,000 delivered nearly identical distribution quality.
The memory savings came from trimming 2 bytes per hash-map entry within Rust's data structures. With billions of records across Cloudflare's infrastructure, those small per-entry reductions compounded into the 100 TB total. This follows an earlier speed-focused optimization to DNS cache entries, showing the company's ongoing effort to squeeze efficiency from its edge network.
The efficiency gain could lower operational costs for Cloudflare and its customers, as less RAM means fewer servers or smaller instances needed for the same workload. For everyday internet users, this may translate into marginally faster content delivery and potentially lower cloud service prices over time. However, the impact is largely invisible to end users, who are unlikely to notice changes in page load times. The broader significance lies in how large-scale infrastructure optimizations compound—small per-request savings across billions of requests can yield substantial environmental and economic benefits.