The Nukemblog
We all want fast services, but how fast is fast? Would you work hard to shave off a millisecond off the mean latency? The 99th percentile? If aiming for 300ms latency, you might answer “probably not.” However, due to various phenomena collectively known as “latency amplification,” a single millisecond deep in your stack can turn into a large increase in user-visible latency—and this is very common in microservices-based systems. What is the true cost of a millisecond?