Kernel Page Cache Tuning for Trading

Introduction: The Hidden Latency in Your Trading Stack

When I first joined ORIGINALGO TECH CO., LIMITED as a financial data strategist, I thought I had seen every possible bottleneck in low-latency trading systems. We had spent months optimizing our network stack, pinning threads to isolated CPU cores, and even experimenting with kernel bypass solutions like DPDK. Yet during a live trading session on a particularly volatile morning—I remember it was the morning of a Fed announcement—our order acknowledgement times suddenly spiked from a comfortable 8 microseconds to over 200 microseconds. Not on every trade. Just enough to hurt. After days of tracing, we discovered the culprit wasn't our network card or our matching logic. It was the Linux kernel's page cache evicting hot data structures that we had implicitly assumed would stay resident in memory. That painful lesson launched a multi-year investigation into what I now consider one of the most underappreciated levers in trading infrastructure: kernel page cache tuning for trading.

For those outside the low-latency world, the page cache is the kernel's transparent mechanism for caching file-backed data in unused RAM. On a general-purpose server, it is a brilliant, self-tuning subsystem. For a trading system, however, it can be a silent saboteur. The page cache does not know that a particular memory-mapped log file or a price history snapshot is critical to your 99th-percentile latency. It only knows about LRU lists and dirty page writeback thresholds. And when your strategy depends on consistent, repeatable access times—often measured in single-digit microseconds—the default heuristics can introduce jitter that no amount of user-space optimization can fix.

In this article, I want to share what our team at ORIGINALGO has learned about tuning the page cache specifically for trading workloads. This isn't a generic "how to tune Linux for performance" guide. It's a collection of hard-won insights from production systems that handle millions of messages per second, where every microsecond of variance translates into real P&L. I'll cover why the default page cache behavior is often wrong for trading, how to characterize your workload's memory access patterns, specific kernel parameters that matter, the trade-offs between different caching strategies, and even a few war stories from the trenches. By the end, you should have a concrete mental model for when and how to intervene—and when to leave the kernel alone.

Before diving deep, let me set the stage with a simple observation: trading systems are not web servers. A web server benefits from caching as much as possible because its access patterns are diverse and often unpredictable. A trading system, especially a market-making or arbitrage engine, has intensely repetitive access patterns. You might read the same order book snapshot file thousands of times per second. You might memory-map a risk parameter file that never changes during the trading day. The page cache's LRU-based eviction policy, designed for fairness and general throughput, can actually evict these hot pages in favor of cold pages that were touched once during system startup. That is the fundamental mismatch we need to address.

Why Default Page Cache Fails in Trading

Let me be blunt: the Linux kernel's page cache is a marvel of engineering for general-purpose computing. It automatically balances read caching, write coalescing, and readahead to maximize throughput for typical workloads like file servers, databases, and web applications. But trading workloads are anything but typical. They are characterized by extreme temporal locality—the same few megabytes of data accessed millions of times—combined with a hard requirement for bounded worst-case latency. The default page cache, with its active/inactive LRU lists and its periodic dirty page writeback, is optimized for average throughput, not for 99.999th percentile latency. When you are trying to hit a consistent 5-microsecond tick-to-trade time, "average" is a dirty word.

Consider what happens when your trading process memory-maps a file containing historical volatility surfaces. On first access, the kernel reads the file from disk into page cache pages. Those pages start on the inactive LRU list. As you access them repeatedly, the kernel's second-chance algorithm promotes them to the active list. So far, so good. But now imagine that your system also runs a logging daemon that occasionally writes a large debug log. Those writes create dirty pages that must be written back to disk. During writeback, the kernel may need to allocate additional page cache pages for metadata or for readahead of other files. If memory pressure increases, the kernel will shrink the active LRU list, potentially demoting your hot volatility surface pages back to inactive, and eventually evicting them. The next time your strategy accesses that volatility surface, it triggers a major page fault—a context switch, disk I/O, and a latency spike that could easily exceed 100 microseconds. That is catastrophic for a high-frequency strategy.

I have personally seen this happen with a client's order book replay system. They had a 2 GB memory-mapped file containing recorded market data for backtesting. During live trading, they also used the same machine to run a monitoring agent that wrote metrics to disk every second. The monitoring agent's writes were small—a few kilobytes each—but they generated dirty pages that eventually forced writeback. The kernel, not knowing any better, evicted pages from the order book replay file. The result was a spike in backtest replay latency that correlated perfectly with the monitoring agent's write intervals. The fix was not to disable the monitoring agent, but to explicitly pin the critical pages in memory and to tune the dirty writeback parameters so that the monitoring agent's writes never triggered global cache pressure. That experience taught me that page cache tuning is not about making the cache "faster"—it is about making it more predictable.

Another common failure mode involves readahead. The kernel's default readahead algorithm tries to prefetch sequentially accessed file data. For a trading system that reads a file in a random or strided pattern, readahead can waste memory bandwidth and pollute the cache with pages that will never be used. Worse, the readahead window can grow aggressively, causing the kernel to issue large I/O requests that compete with your critical market data feeds. I once diagnosed a case where a trading engine's latency spiked every time the kernel performed readahead on a non-critical configuration file. The readahead I/O saturated the disk controller's queue, delaying the actual market data writes that the engine needed. Turning off readahead for that specific file—or more precisely, using posix_fadvise(POSIX_FADV_RANDOM)—eliminated the spikes entirely.

So why doesn't everyone just disable the page cache entirely? Because for many trading workloads, the page cache is still beneficial. If your strategy loads a large reference dataset once per day and then accesses it repeatedly, caching that dataset in RAM is far faster than reading from NVMe. The key is to move from a one-size-fits-all default to a workload-aware configuration. That means identifying which files are latency-critical, which are throughput-critical, and which are irrelevant. Then you apply different caching policies to each category. The kernel provides the mechanisms—memory mapping, madvise, fadvise, cgroup memory limits, and per-process tunables—but it does not provide the policy. That is our job as trading infrastructure engineers.

Mapping Data Access Patterns

You cannot tune what you do not measure. Before touching a single kernel parameter, you need to understand exactly how your trading application accesses file-backed data. This is not a theoretical exercise. I have seen teams waste weeks tweaking vm.dirty_ratio and vm.vfs_cache_pressure only to discover that their actual bottleneck was a single memory-mapped file being accessed with a pathological stride pattern that caused constant TLB misses. The first step is to instrument your application and the kernel to capture the full picture of page cache behavior.

At ORIGINALGO, we use a combination of tools. On the application side, we log every mmap, munmap, madvise, and posix_fadvise call with timestamps and file offsets. On the kernel side, we use perf with the kmem tracepoints to monitor page cache allocations, evictions, and major/minor faults. We also use /proc/vmstat to track global counters like pgpgin, pgpgout, pgmajfault, and pgrefill. But raw counters are not enough. You need to correlate them with your trading events. For example, if you see a spike in pgmajfault exactly when your strategy receives a market data update, that tells you the page cache evicted a critical page. If you see high pgpgout (page writeback) during your peak trading hours, that tells you dirty pages are being flushed and competing for I/O bandwidth.

One of the most useful techniques we developed is access pattern fingerprinting. We run a representative trading day in a controlled environment and record the sequence of file offsets accessed by each critical thread. Then we analyze that sequence for locality, stride, and reuse distance. For example, a market-making engine might access an order book snapshot file at offsets that follow a predictable pattern: the top 100 price levels are accessed every microsecond, while deeper levels are accessed every millisecond. That tells us we need to keep the top 100 levels in the page cache permanently, while deeper levels can tolerate eviction. We then use madvise(MADV_WILLNEED) to prefetch the top levels at startup and mlock or MAP_LOCKED to pin them. For the deeper levels, we might use MADV_DONTNEED after each access to proactively free those pages, reducing cache pressure.

Another dimension is write patterns. Trading systems generate a lot of data: order logs, trade logs, risk metrics, audit trails. Not all of that data needs to be cached. In fact, caching write data is often counterproductive because it delays the actual disk write and creates dirty pages that later trigger writeback storms. For write-heavy files, we recommend using O_DIRECT or sync_file_range with SYNC_FILE_RANGE_WRITE to bypass the page cache entirely or to write back eagerly. This is especially important for compliance logs, which must be durable but are never read back during the trading day. By bypassing the page cache for these files, you free up memory for the read-only data that actually benefits from caching.

I remember a specific engagement where a client's trading engine was experiencing periodic 500-microsecond latency spikes every 30 seconds. We traced it to the kernel's dirty_expire_centisecs timer, which was flushing accumulated dirty pages from a trade log file. The trade log was written using buffered I/O, so the kernel was caching the writes and then flushing them in batches. The fix was simple: we changed the trade log to use O_DIRECT with a small aligned buffer. The latency spikes disappeared immediately. But the lesson was deeper: we had assumed that write caching was harmless because the trade log was "just logging." In reality, the page cache's writeback behavior was interacting with our read cache in a way that violated our latency requirements. Always separate read cache from write cache when latency matters.

Finally, do not forget about shared memory and tmpfs. Many trading systems use POSIX shared memory or memory-mapped files on tmpfs for inter-process communication. These are technically not page cache in the traditional sense—tmpfs uses the page cache as its backing store—but they consume the same memory and can be subject to the same eviction pressure. If you have a critical shared memory region that must never be swapped or evicted, you need to either mlock it or ensure that the tmpfs mount has a large enough size and that the kernel's swappiness is set to zero. I have seen production systems where a tmpfs-based order book was partially evicted because the system ran out of anonymous memory, causing page faults during the most critical moment of the trading day. That is a mistake you only make once.

Tuning Dirty Page Writeback

If there is one category of page cache tuning that has the highest impact on trading latency, it is dirty page writeback. The Linux kernel accumulates dirty pages (pages that have been written to but not yet flushed to disk) and writes them back in the background or on demand. The default parameters are designed for throughput and fairness: they allow a large amount of dirty data to accumulate, then flush it in large batches. For a trading system, this behavior is dangerous. A large dirty page pool means that when writeback finally happens, it can saturate the disk I/O subsystem for hundreds of milliseconds, delaying critical reads and causing latency spikes that no user-space optimization can hide.

The key parameters here are vm.dirty_ratio, vm.dirty_background_ratio, vm.dirty_expire_centisecs, and vm.dirty_writeback_centisecs. The default dirty_ratio is often 20% of total RAM, and dirty_background_ratio is 10%. On a 64 GB server, that means up to 12.8 GB of dirty data can accumulate before the kernel starts throttling writers. For a trading system, that is insane. You want dirty data to be flushed almost immediately, in small increments, so that writeback never becomes a burst. At ORIGINALGO, we typically set dirty_background_ratio to 1 or 2 percent and dirty_ratio to 3 to 5 percent. We also set dirty_expire_centisecs to 100 (1 second) and dirty_writeback_centisecs to 10 (0.1 seconds). This causes the kernel to wake up the writeback thread frequently and flush small amounts of data, smoothing out I/O load.

But there is a subtlety: not all block devices are equal. On NVMe SSDs, writeback is fast, but the device still has a finite queue depth and can become a bottleneck if too many write requests are queued at once. On network-attached storage or SAN volumes, writeback latency can be milliseconds, which is an eternity in trading. For these devices, you may need to go further and use cgroup v2 writeback throttling to limit the rate of dirty page generation per process. Or you can bypass the page cache entirely for write-heavy files using O_DIRECT or sync_file_range. I have found that for trade logs and audit trails, O_DIRECT with a double-buffered async write scheme gives the best combination of durability and predictability. Yes, it requires more careful programming, but the latency stability is worth it.

Another important consideration is the interaction between dirty writeback and memory reclaim. When the kernel needs to free memory, it first tries to reclaim clean pages (evicting them from cache) and then dirty pages (writing them back before eviction). If your system is under memory pressure and you have a lot of dirty pages, the reclaim path will trigger synchronous writeback, which blocks the allocating process until the write completes. That is a recipe for latency spikes. By keeping the dirty page ratio low, you reduce the probability that reclaim will need to write back. You also reduce the amount of work the kernel must do during a memory shortage. I cannot overstate this: for trading systems, dirty pages are the enemy of determinism. Keep them minimal.

There is also a kernel feature called writeback throttling (introduced in cgroup v2) that allows you to set a maximum writeback rate per cgroup. This is extremely useful in multi-tenant trading environments where one strategy's logging could otherwise starve another strategy's market data reads. We have used it to isolate a risk-checking process that writes a large snapshot every second from the latency-critical order gateway process. By placing the risk-checking process in a cgroup with a low writeback rate limit, we ensured that its dirty pages never caused I/O contention for the order gateway. This is a more advanced tuning technique, but it can be a lifesaver in production.

Pinning Critical Pages with mlock and MAP_LOCKED

Sometimes the simplest solution is the most effective. If a set of pages is absolutely critical to your trading latency—for example, the current order book, the risk limits file, or the kernel's own data structures for your process—the most direct way to keep them in memory is to lock them. The mlock() system call and the MAP_LOCKED flag for mmap() tell the kernel to never evict those pages. They will remain resident until you explicitly unlock them or the process exits. This eliminates major page faults for those pages entirely, giving you deterministic access times. For a trading system, that is often worth the memory cost.

However, mlock is not a silver bullet. First, there is a limit on how much memory a process can lock, controlled by RLIMIT_MEMLOCK. On many systems, the default is 64 KB or 8 MB, which is far too small for a trading application. You need to raise this limit, either by editing /etc/security/limits.conf or by using prlimit at runtime. At ORIGINALGO, we set RLIMIT_MEMLOCK to unlimited for trading processes, but we also monitor locked memory usage carefully. Locking too much memory can starve the rest of the system and cause the OOM killer to target your process. You need to find the sweet spot: enough locked memory to cover your critical working set, but not so much that you leave no room for the page cache used by other essential services.

Second, mlock only prevents eviction. It does not prevent the pages from being unmapped or modified in ways that cause faults. For example, if you memory-map a file with MAP_SHARED and then write to it, the kernel may still need to handle the write fault and update the page. If you want to avoid all faults, you need to pre-fault the pages (by reading them once) and then lock them. We typically do this at application startup: open the file, mmap it, read every page (forcing them into cache), then call mlock on the entire region. That way, we pay the fault cost once during initialization, not during trading.

Third, mlock interacts poorly with fork(). When a process forks, the child inherits the locked pages, but the lock is not automatically extended to the child's copy-on-write pages. If the child writes to a locked page, it gets a private copy that is not locked, and that copy can be evicted. This is a common source of subtle latency bugs in trading systems that use forked worker processes. The solution is to either avoid fork after locking, or to re-lock the pages in the child using mlockall(MCL_CURRENT | MCL_FUTURE). We have standardized on the latter approach for our trading engines, with careful monitoring to ensure that MCL_FUTURE does not cause memory exhaustion when the process allocates new memory.

I recall a case where a client had pinned their entire order book—about 12 GB—using mlock. Their latency was excellent, but every few hours the system would become unresponsive and require a reboot. The problem was that they had also set vm.overcommit_memory to 2 (no overcommit), and the pinned memory plus the page cache for other processes exceeded available RAM. The OOM killer eventually targeted a critical system daemon, causing a cascading failure. The fix was to reduce the locked region to only the top 1000 price levels (about 200 MB) and to allow the rest of the order book to be cached normally but with a high priority in the page cache. The lesson: lock only what you truly need, and leave the rest to the kernel's cache with appropriate hints.

Leveraging cgroups and Memory Policies

Modern Linux kernels provide powerful mechanisms for controlling memory usage per process or per group of processes. For trading systems, these mechanisms are invaluable because they allow you to isolate the page cache behavior of different components. A market data feed handler, for example, has very different caching needs from a risk calculation engine or a logging daemon. By placing each component in its own cgroup v2 with specific memory limits and writeback controls, you can prevent one component from polluting the page cache used by another. This is a game-changer for multi-strategy or multi-tenant trading platforms.

The most important cgroup v2 controllers for page cache tuning are memory and io. The memory controller allows you to set a memory.max limit, which caps the total memory (including page cache) that the cgroup can use. When the cgroup hits this limit, the kernel reclaims pages from that cgroup first, rather than from the global page cache. This means you can give your latency-critical trading engine a generous memory limit and a separate, smaller limit to a logging or monitoring process. The trading engine's page cache pages will be protected from eviction caused by the logging process, because the logging process will hit its own limit and trigger reclaim within its own cgroup. I have used this technique to eliminate cross-process cache eviction in several production systems, and it works remarkably well.

The io controller complements the memory controller by allowing you to set io.max limits on read and write bandwidth or IOPS per device. This is useful for preventing a bulk data loader or a backup process from saturating the disk and delaying critical market data reads. For example, you can give your market data feed handler a high io.max priority and your historical data archiver a low priority. The kernel will then throttle the archiver's I/O, leaving bandwidth for the feed handler. Combined with per-cgroup writeback throttling, this gives you fine-grained control over the I/O subsystem, which is often the real bottleneck in page cache performance.

Another useful cgroup feature is memory.low and memory.high. Unlike memory.max, which is a hard limit, memory.high is a soft limit that triggers reclaim but does not cause OOM. memory.low is a best-effort protection that tells the kernel to avoid reclaiming pages from this cgroup unless absolutely necessary. By setting memory.low to a value that covers your critical working set, you can give your trading engine a strong hint that its pages are important. This is not a guarantee—the kernel may still reclaim under extreme pressure—but in practice it provides a useful layer of protection without the rigidity of mlock. We often use memory.low as a first line of defense, and mlock only for the most critical few hundred megabytes.

Finally, do not overlook NUMA policies. On multi-socket servers, the page cache pages for a file can be allocated on any NUMA node. If your trading thread runs on node 0 but the page cache page is on node 1, every access incurs a cross-socket memory latency, which can be 50-100 nanoseconds slower. That may not sound like much, but when you are chasing single-digit microseconds, it matters. You can use numactl or mbind to restrict your process's memory allocations to a specific node, and you can use fadvise or madvise to hint that file pages should be allocated on the local node. I have measured 15-20% reductions in p99.9 latency simply by ensuring NUMA-local page cache allocation for critical trading processes. It is a small detail with a big payoff.

Case Studies and Lessons from Production

Let me share two concrete case studies from our work at ORIGINALGO. The first involves a high-frequency futures trading firm that was experiencing unexplained latency spikes every time their risk system performed a end-of-day snapshot. The snapshot process read a large position file (about 4 GB) sequentially, which caused the kernel's readahead algorithm to aggressively prefetch data into the page cache. That prefetching evicted the hot order book pages used by the trading engine. The fix was twofold: first, we changed the snapshot process to use posix_fadvise(POSIX_FADV_DONTNEED) after reading the file, which immediately freed the pages it had used. Second, we placed the snapshot process in a separate cgroup with a memory.max limit of 2 GB, so even if it tried to cache more, it would hit its own limit and reclaim its own pages rather than the trading engine's. The latency spikes disappeared entirely, and the snapshot process actually ran faster because it was no longer competing for memory bandwidth.

The second case study is more personal. I was leading a project to optimize a cryptocurrency arbitrage engine that ran on a cloud instance with 32 GB of RAM. The engine used a memory-mapped file for a price history database that was about 20 GB in size—larger than RAM. The idea was that the kernel would cache the most frequently accessed portions. But in practice, the access pattern was nearly random across the entire 20 GB, so the page cache was constantly evicting and re-reading pages. The latency was terrible, often exceeding 1 millisecond. We tried many things: increasing readahead, using madvise(MADV_RANDOM), and even moving to a faster NVMe drive. Nothing worked because the working set simply did not fit in memory. The solution was to restructure the data: we built a compact in-memory index of the most recent 1 million price points (about 200 MB) and kept that index pinned with mlock. For older data, we used a separate on-disk store with O_DIRECT and a custom LRU cache in user space that we could control precisely. The result was a 50x reduction in p99 latency. The lesson: when the working set exceeds RAM, the kernel page cache is the wrong tool. You need application-level caching with explicit eviction policies.

Both cases highlight a recurring theme: the page cache is not a magic solution. It is a general-purpose mechanism that works well when access patterns match its assumptions. When they do not, you must intervene. And intervention requires measurement. I cannot count how many times I have seen teams apply tuning parameters from a blog post without first profiling their own workload. Every trading system is different. The optimal vm.dirty_ratio for a market-making engine on a dedicated server is different from that for a multi-tenant backtesting platform. You must measure, hypothesize, change one parameter at a time, and measure again. That is the only reliable path.

Conclusion: Towards Deterministic Memory for Trading

We have covered a lot of ground: why the default page cache fails for trading, how to map your data access patterns, the critical importance of dirty page writeback tuning, the power of mlock and MAP_LOCKED, the isolation provided by cgroups, and real-world lessons from production. If there is one overarching conclusion, it is this: kernel page cache tuning for trading is not about making the cache faster—it is about making it predictable. Trading systems do not care about average throughput. They care about the worst-case latency. The default page cache optimizes for the average, and that is why it must be tuned.

Looking forward, I believe the future of page cache tuning for trading lies in application-aware kernel interfaces. We are already seeing hints of this with io_uring and its ability to bypass the page cache entirely for some operations, and with userfaultfd and madvise extensions that allow finer-grained control. In the next few years, I expect to see kernel features that allow trading applications to declare their latency requirements and have the kernel automatically adjust caching policies. Until then, we must rely on the mechanisms described here: measurement, explicit hints, isolation, and a healthy respect for the complexity of the Linux memory subsystem. At ORIGINALGO TECH CO., LIMITED, we continue to push the boundaries of what is possible, because in trading, every microsecond counts.

At ORIGINALGO TECH CO., LIMITED, our experience with kernel page cache tuning for trading has taught us that the greatest gains come not from exotic kernel patches, but from a disciplined, measurement-driven approach. We have learned that the default Linux memory subsystem is designed for fairness and throughput, not for the deterministic latency that trading demands. Our work with clients across futures, equities, and digital assets has shown that even simple changes—lowering dirty page ratios, using mlock for critical data, and isolating processes with cgroups—can reduce p99.9 latency by 40-60%. We have also learned that no single configuration works for all workloads; each trading strategy has a unique memory access fingerprint that must be characterized before tuning. Looking ahead, we are investing in tools that automatically profile page cache behavior and recommend per-process tuning parameters. We believe that the next frontier in low-latency trading is not faster networks or FPGAs, but smarter memory management at the kernel level. Our advice to any trading firm is to treat the page cache as a first-class component of your latency budget, not an afterthought. Measure it, tune it, and monitor it continuously—because the market will not wait for your kernel.