At various points in my career, I have created or managed systems which need to perform calculations on enormous amounts of numeric data in real-time. Whether I was running a Varnish cluster that served 30,000 requests per second or writing financial analytics which needed to calculate aggregates on terabytes of price and return data in a few seconds, a common need emerged: how can I calcalate aggregates on large amounts of numeric data in a single pass?
In the fall of 2017, I was researching a performance problem with a job queueing system. I was trying to figure out how I could better visualize the response time of the system in order to understand what was going on and I ran across the excellent article Brandon Gregg’s Latency Heat Maps. I could not find an easy-to-use, reusable latency heat map chart component, so I decided to make my own in d3.
Many versions of UNIX have a useful concept built in to their dynamic linker: by setting an environment variable (usually LD_PRELOAD), a user can force a shared library to load before all others. While I wasn’t able to figure out how to exactly replicate this functionality on Windows, I found a snippit on page 794 of Programming Applications for Microsoft Windows: Fourth Edition, by Jeffrey Richter, on how to come close.