Cyclone vs. the file cache: benchmarking a memory-mapped page cache

Every cache backend behind ModPageSpeed sits under one interface, so the optimizer above it never changes. That makes a direct question easy to ask: how much does the storage engine itself matter? We put Cyclone, the memory-mapped cache in mod_pagespeed 1.15 and ModPageSpeed 2.0, against the classic file-per-entry disk cache it replaced, and measured both across concurrency, realistic mixed traffic, latency tails, and eviction, on two very different machines.

The most useful result is a trend. Cyclone’s advantage is smallest on a laptop with a fast NVMe SSD and grows to 4x-6x on the kind of storage a real server runs on. The faster the disk, the more the old design’s per-file cost stays hidden.

Two engines, one interface

Cyclone keeps one memory-mapped file per volume. A read returns a pointer into mapped memory, with no open/read/close and no heap copy. An in-process RAM tier holds the hot set, eviction runs inline as entries are written, and the admission policy resists one-hit-wonders flushing the working set.

The file-per-entry cache stores one file per entry. Every read is open, then read, then close, plus a copy into a fresh buffer; every write creates a file. Eviction is deferred to a periodic janitor that walks the whole cache directory, an O(n) scan that grows with the file count.

Everything below is the median of repeated runs on quiesced machines. Where the file cache wins, the numbers say so.

Concurrency: the file cache stops scaling

A live server answers many requests at once. Ramping concurrent readers from 1 to 64 against a warm cache, Cyclone’s mapped reads carry no per-operation lock or system call, so aggregate throughput climbs with cores. The file cache peaks near 4-8 threads, then declines as lock and system-call contention take over.

Aggregate reads per second at 64 threads, laptop (fast NVMe SSD):

Object size	File cache	Cyclone	Improvement
100 B	601,358	2,715,255	4.5x
10 KB	592,166	2,090,796	3.5x
1 MB	20,897	84,704	4.1x

And on the workstation (realistic server storage):

Object size	File cache	Cyclone	Improvement
100 B	273,742	1,533,871	5.6x
10 KB	268,719	1,252,859	4.7x
1 MB	41,184	164,090	4.0x

One case runs the other way. At a single thread, the file cache can be faster, on the laptop it actually wins for 10 KB and 1 MB objects, because a warm read is just a page-cache copy and nothing more. That edge disappears the moment real concurrency arrives. The next section explains why.

Both fit in RAM. Why is memory-mapped still faster?

A fair objection: the readers above run against a warm cache, so the file cache’s bytes are already in the OS page cache. They are in RAM. Both engines serve from memory, and no request touches the SSD. So why is the file cache 4x-6x slower at 64 threads?

Because a file-cache hit is not free even when nothing reads the disk. Every hit still pays open, read, close: several system calls, each a round trip across the user/kernel boundary, plus a copy_to_user that physically copies the object out of the page cache into your buffer. A memory-mapped hit pays none of that. It is a load from a page already resident in RAM, with at most a one-time minor fault the first time a page is touched.

For large objects the copy is most of the cost. For small ones the real tax is contention: those same system calls force every thread to touch shared kernel structures, the directory and file dentry locks, the inode, the process file-descriptor table, and under many cores those cache lines ping-pong between them. That is why the file cache tops out around 4-8 threads and then gets slower as you add more, while the memory-mapped path, read-only shared pages with no cross-core writes, keeps scaling with cores.

The honest tell is the single-thread number, where the file cache can win. With no concurrency there is nothing to serialize, and a lone warm read plus copy is a very tight path, tighter than mmap’s first-touch fault and TLB setup. The advantage only appears under concurrency, which is precisely the server workload that matters.

Realistic traffic, under pressure

A Zipfian mix of reads, writes, and deletes over a heavy-tailed object-size distribution, with a working set about twice the cache size so eviction runs continuously, the way it does on a busy site. Eight worker threads, a 1 GB cache with a 128 MB RAM tier.

Metric (workstation)	Cyclone	File cache
Throughput	111k ops/s	27k ops/s
p99 latency	1.0 ms	4.2 ms
p999 tail	2.1 ms	8.4 ms

On this storage Cyclone runs 4.1x the throughput with roughly a quarter of the p999 tail. The file cache often posts a higher hit rate, because it admits everything and lets its footprint grow to do it, but it serves what it holds far more slowly. On the laptop’s fast NVMe SSD the throughput gap narrows to 1.6x, which is the point: the slower and more realistic the disk, the wider the margin.

The full latency distribution, machine by machine, is in the interactive data explorer. The curves make the tail difference obvious in a way percentiles alone do not.

The through-line: a slower disk amplifies the win

Here is every headline result as a ratio, Cyclone over the file cache, for both machines. The laptop is the conservative case, with cheap per-file cost, so the file cache looks its best. The workstation’s realistic storage is closer to production. In every pairing, the slower disk makes the gap larger.

Test	Laptop (fast SSD)	Workstation (realistic)
Concurrency, 64 threads	4.5x	5.6x
Realistic, under pressure	1.6x	4.1x
Eviction sweep, 100k files	2.0x	6.0x
Overflow write rate	2.9x	5.6x

The practical lesson: benchmarks run on developer laptops with fast local SSDs understate what a memory-mapped cache buys in production.

Eviction: a flat line vs. a stall

Stream 100,000 files into a small cache and watch write throughput batch by batch. Cyclone evicts inline and holds steady. The file cache defers to its janitor; each time the scan fires, writers stall, and the walk gets more expensive as the file count climbs. On the workstation the file cache finished the same sweep 6x slower.

There is a real-world version of this. The file cache also writes a small file for every miss it remembers, so every not-found input URL becomes an inode the janitor later has to walk. On a busy server that churn inflates the very scan above. Cyclone’s single-file design and selective admission do not accumulate that debris.

Staying inside the cap

Write far more than a 16 MB cache holds, and the difference in discipline shows. Cyclone evicts inline and holds the cap at 15.5 MB. The file cache’s deferred janitor lets the directory balloon well past the limit before it catches up: 5x over on the workstation, 9x on the laptop. Predictable disk usage is worth as much to an operator as raw speed.

Sizing: keep the hot working set in RAM

A memory-mapped cache is demand-paged, so only the pages you actually touch need to be resident, the hot working set, not the whole file. The cache can safely be larger than RAM: the kernel keeps hot pages in memory and pages cold ones in on access. You pay for extra disk reads only when the working set itself outgrows RAM. We measured that deliberately, a 4 GB cache in a 2 GB budget with a working set about twice the cache, and throughput dropped roughly 1.8x. That is graceful degradation, not a cliff, and even while paging, Cyclone still ran 1.5x-3.5x faster than the file cache at the same oversized size.

Machine	Engine	Fits in RAM (ops/s)	Oversized, paging to disk (ops/s)
Laptop	Cyclone	132,504	69,524
Laptop	File cache	78,524	46,993
Workstation	Cyclone	114,974	65,161
Workstation	File cache	27,142	18,340

This is not a tax Cyclone introduces. The file cache is fast for exactly the same reason, its bytes live in the OS page cache, so any disk-backed cache slows down once its hot set spills past memory; the file cache degrades here too, and from a lower starting point. The guidance is unchanged and cheap: size RAM to your hot working set, not to the entire cache. It is spelled out in the production deployment guide.

Where this lands in the products

mod_pagespeed 1.15 already reads from Cyclone with no copy: a hit is a pointer into the mapped file, not a system call and a buffer allocation, which is exactly the read path these numbers measure. ModPageSpeed 2.0 carries the memory-mapped path further into the request, serving cached bytes toward the socket without an intermediate copy. How it does that, streaming large hits with kernel sendfile straight from the shared cache file, copying small ones because the syscall would cost more, and staying correct while the cache changes underneath a slow client, is its own deep-dive: zero-copy serving between nginx and the worker. If you run a file-cache configuration under real concurrency or eviction pressure, the update is worth taking.

A fair boundary on the claim: this is a cache benchmark, not an end-to-end page-serving benchmark. It shows the cache component is faster, not that a page renders some fixed percentage sooner. On most requests, origin fetch and optimization work dominate the clock. The cache win reaches the visitor precisely when a server is busy, with many concurrent hits, a large working set, and eviction running hot. That is also when a server most needs the help.

Frequently asked questions

Is Cyclone faster than the old file-per-entry cache? Under concurrency, eviction pressure, and on realistic server storage, yes: 4x to 6x on throughput in our tests, with a tighter latency tail. On an idle, over-provisioned cache backed by a fast NVMe SSD, the two are close and the file cache can edge ahead.

Does mod_pagespeed 1.15 benefit from Cyclone? Yes. 1.15 already ships Cyclone and reads from it zero-copy: a cache hit is a pointer into the mapped file rather than a system call plus a heap copy. That read path is exactly what this benchmark measures.

Will a faster cache make my site load faster? It helps most when the cache is on the hot path: high concurrency, lots of already-optimized assets, and cache eviction under load. On a lightly loaded site the cache was rarely the bottleneck, so the end-to-end gain is smaller.

How large should I size the cache? Aim to keep the hot working set in RAM. A memory-mapped cache is demand-paged, so the cache can be larger than physical memory; only the pages in active use need to be resident. Push the working set itself far past RAM and it pages to disk, about 1.8x slower in our test, and still faster than the file cache at that size. The file cache depends on the OS page cache the same way, so this holds for any disk-backed cache.

The machines

Two deliberately different machines. The specs that matter for a cache benchmark are core count (concurrency), memory (the RAM tier and mapping headroom), and above all the storage stack and filesystem, which set the per-file system-call cost.

Laptop. Apple-silicon, 10 cores, 16 GB, fast NVMe SSD (native APFS, multi-GB/s sequential reads). A very fast core and low-latency local flash, but few cores. This is where per-file cost is cheapest, so it is the conservative case for every filesystem claim here.
Workstation. 64-core x86, 96 GB, ext4 on virtualized storage. Many threads draw the clean concurrency curve, and the virtualized storage adds the kind of filesystem-metadata overhead a production server actually pays.

Explore the numbers yourself, with per-machine toggles and the full latency distribution, in the data explorer.

In memory of Alan M. Carroll. Some of Cyclone’s design grew out of conversations with Alan, known to the Apache Traffic Server community as SolidWallOfCode and among the people who understood its architecture most deeply. His thinking about how a cache lays bytes out on disk and reclaims space without stalling helped shape how we thought about ours, and he shared it generously. We remember him with gratitude. Alan’s memorial at the Apache Software Foundation.