Cloudflare just reclaimed more than 100 terabytes of RAM across its entire network — and the fix was a single algorithm tweak, not a new server rack. The company’s engineers found excessive memory usage in one of its load-balancing services, fixed it with better math, and freed up enough space to measure in terabytes.
The story comes from a blog post by Cloudflare, which explains how the company’s Pingora-based backend router was holding onto far more memory than it needed. The culprit was a library called pingora-ketama, which handles consistent hashing — a way of spreading requests across servers so that adding or removing a machine doesn’t break everything.
What consistent hashing does
Consistent hashing works by turning every server and every request into a number on a line. Each server gets a spot on that line, and requests land with the server to its left. The trick is that hashes are random numbers, so the spots aren’t equal — some servers end up with vastly more requests than others.
Cloudflare’s engineers noticed this imbalance in a ticket filed by Ivan. The service was using “significantly more memory than expected” in structures tied to pingora-ketama. The fix wasn’t a bigger server farm. It was a deeper understanding of how the numbers were landing.
The math behind the usage
The key insight is that hash functions produce random numbers, and random numbers don’t distribute evenly. The blog post walks through the stats: expected value tells you where the middle of a distribution sits, and standard deviation tells you how spread out the rest is. For a hundred servers, the math showed that each server should handle about 0.99% of the total load, with most falling within 1% of that mark.
But that’s a fraction of the total. The blog post describes calculating the coefficient of variation, which measures how much the actual sizes vary from the expected ones. The principle: because hashes are essentially random, the sizes of the regions assigned to each server will naturally drift from what the expected value predicts.
Fixing it with more hashes
The solution isn’t a bigger machine. It’s more hashes. By giving each server multiple spots on the number line, Cloudflare spread the load more evenly. The blog describes it as turning every tool into a hammer — the problem is always about how the hashes land.
The result is a Pingora-based service that uses less memory overall. Cloudflare didn’t buy new hardware. It just made the numbers land better.
The scale of the win
Cloudflare runs thousands of servers with petabytes of RAM and millions of CPU cores. Even a small improvement at that scale adds up fast. The company says small 1%-at-a-time gains are worth celebrating, and this one went further: more than 100TB of RAM reclaimed, on top of the 100TB the DNS team shed last month.
That’s a real number. It’s not a chart or a graph — it’s actual capacity freed up by a few lines of code. The Performance team helps maintain equitable resource sharing between teams, and this fix helps keep the resources fair.
Key facts
- Memory freed: More than 100TB of RAM globally
- Previous savings: 100TB freed by the DNS team last month
- Algorithm involved: pingora-ketama, a consistent-hashing library
- Fix approach: Adding more hashes to balance the load
- Found by: Ivan, who filed a ticket noting excessive memory usage
Timeline
| Date | Event |
|---|---|
| Last month | DNS team frees 100TB of RAM |
| Recent | Ivan files ticket on excessive memory |
| Recent | Fix lands, freeing >100TB of RAM |
This is the kind of engineering win that rarely makes headlines, but it matters deeply to companies that run at Cloudflare’s scale. The memory is now available for other services, and the fix is a reminder that sometimes the best optimization isn’t a new server — it’s a better way to assign the work.
Source material: “Saving another 100TB of RAM,” cloudflare.com.
Get the Notebook.
The day's best stories and every fresh verdict, in plain English, in your inbox by seven. One email a day, no more.