# performance analysis Published articles for performance analysis. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Stack walking: space and time trade-offs DevFeed: [Stack walking: space and time trade-offs]() Original publisher: [Read original article]() Published: 2025-10-26T07:00:00Z Content type: article Language: en Sources: [MaskRay]() Topics: [LLVM](), [Linux](), [C++]() Tags: [article](), [gcc](), [linux](), [llvm](), [performance-analysis](), [platforms](), [required](), [sframe](), [update]() ### AI overview This article examines space and time trade-offs among stack-walking mechanisms on Linux, including DWARF, frame pointers, SFrame, LLVM's Compact Unwinding Format, Last Branch Record, and CET Shadow Stack. It analyzes the size overhead of enabling non-DWARF mechanisms when building LLVM executables; runtime performance analysis is deferred to a future update. ### Source excerpt On most Linux platforms (except AArch32, which uses .ARM.exidx), DWARF .eh_frame is required for C++ exception handling and stack unwinding to restore callee-saved registers. While .eh_frame can be used for call trace recording, it is often criticized for its runtime overhead. As an alternative, developers can enable frame pointers, or adopt SFrame, a newer format designed specifically for profiling. This article examines the size overhead of enabling non-DWARF stack walking mechanisms when building several LLVM executables. Runtime performance analysis will be added in a future update. ## Thread Count Scaling Part 5. Summary DevFeed: [Thread Count Scaling Part 5. Summary]() Original publisher: [Read original article]() Author: Denis Bakhvalov Published: 2024-05-10T04:00:00Z Content type: article Language: en Sources: [Denis Bakhvalov]() Topics: [Concurrency](), [Concurrent Programming](), [benchmarking](), [cpu](), [Hardware](), [Caching](), [Deadlock]() Tags: [benchmarks](), [book-chapters](), [cache](), [concurrent](), [cpu](), [hardware](), [performance](), [performance-analysis](), [thread](), [threads]() ### AI overview This summary reviews thread-count scaling in throughput-oriented applications. It identifies frequency throttling, differences between performant and energy-efficient cores, shared-resource limits such as memory bandwidth and L3 cache, and thread synchronization as factors that can limit scaling. It also references SPEC CPU 2017 benchmarks and reports differing scaling behavior for integer and floating-point workloads. ### Source excerpt Subscribe to my newsletter, support me on Patreon, Github, or by PayPal donation. This blog is an excerpt from the book. More details in the introduction. Summary In the case study, we have analyzed several throughput-oriented applications with varying thread count scaling characteristics. Here is a quick summary of our findings: - Frequency throttling is a major roadblock to achieving good thread count scaling. This affects all the benchmarks that we've analyzed. ## Thread Count Scaling Part 2. Blender and Clang DevFeed: [Thread Count Scaling Part 2. Blender and Clang]() Original publisher: [Read original article]() Author: Denis Bakhvalov Published: 2024-05-10T04:00:00Z Content type: article Language: en Sources: [Denis Bakhvalov]() Topics: [Benchmark](), [benchmarking](), [Concurrency](), [C++](), [Processes]() Tags: [benchmark](), [blender](), [book-chapters](), [c-plus-plus](), [clang](), [compilation](), [concurrency](), [floating-point](), [parallel](), [parallelism](), [performance](), [performance-analysis](), [process](), [scale](), [thread](), [threads]() ### AI overview This article examines thread-count scaling in Blender and Clang. Blender scales across the available threads because its rendering workload is highly parallelizable, but scaling declines as E-cores and SMT sibling threads are used. Clang compilation is also massively parallel, though its scaling is affected by cache and branch behavior, core differences, SMT, and frequency throttling. ### Source excerpt Subscribe to my newsletter, support me on Patreon, Github, or by PayPal donation. This blog is an excerpt from the book. More details in the introduction. Blender is the only benchmark in our suite that continues to scale up to all 16 threads in the system. The reason for this is that the workload is highly parallelizable. The rendering process is divided into small tiles, and each tile can be rendered independently. However, even with this high level of parallelism, the scaling is only . ## Thread Count Scaling Part 4. CloverLeaf and CPython DevFeed: [Thread Count Scaling Part 4. CloverLeaf and CPython]() Original publisher: [Read original article]() Author: Denis Bakhvalov Published: 2024-05-10T04:00:00Z Content type: article Language: en Sources: [Denis Bakhvalov]() Topics: [benchmarking](), [Concurrency]() Tags: [book-chapters](), [case-study](), [dram](), [hpc](), [memory](), [metric](), [performance](), [performance-analysis](), [scale](), [speed](), [thread]() ### AI overview The article examines CloverLeaf and CPython thread-count scaling. It reports that CloverLeaf performance stops increasing after three threads because memory bandwidth becomes the limiting factor. Replacing two memory modules with faster DDR4 modules improves performance by 10% to 33% as thread count increases. ### Source excerpt Subscribe to my newsletter, support me on Patreon, Github, or by PayPal donation. This blog is an excerpt from the book. More details in the introduction. CloverLeaf is a hydrodynamics workload. We will not dig deep into the details of the underlying algorithm as it is not relevant to this case study. CloverLeaf uses OpenMP to parallelize the workload. Similar to other HPC workloads, we should expect CloverLeaf to scale well. ## Thread Count Scaling Part 3. Zstandard DevFeed: [Thread Count Scaling Part 3. Zstandard]() Original publisher: [Read original article]() Author: Denis Bakhvalov Published: 2024-05-10T04:00:00Z Content type: article Language: en Sources: [Denis Bakhvalov]() Topics: [Compression](), [Concurrent Programming](), [data]() Tags: [benchmarks](), [book-chapters](), [case-study](), [compression](), [fragmentation](), [memory](), [memory-pool](), [performance](), [performance-analysis](), [thread](), [threads]() ### AI overview This article examines why Zstandard compression does not scale linearly with thread count. It discusses compression-level trade-offs, worker-thread interactions, waiting periods, input-buffer shortages, and memory-pool fragmentation, based on an instrumented timeline. ### Source excerpt Subscribe to my newsletter, support me on Patreon, Github, or by PayPal donation. This blog is an excerpt from the book. More details in the introduction. Next on our list is the Zstandard compression algorithm, or Zstd for short. When compressing data, Zstd divides the input into blocks, and each block can be compressed independently. This means that multiple threads can work on compressing different blocks simultaneously. ## Thread Count Scaling Part 1. Introduction DevFeed: [Thread Count Scaling Part 1. Introduction]() Original publisher: [Read original article]() Author: Denis Bakhvalov Published: 2024-05-10T04:00:00Z Content type: tutorial Language: en Sources: [Denis Bakhvalov]() Topics: [Scalability](), [C++](), [Concurrency](), [systems](), [Linux]() Tags: [benchmarks](), [book-chapters](), [c-plus-plus](), [case-study](), [linux](), [performance-analysis](), [scalability](), [systems]() ### AI overview This case study explains thread count scaling for multithreaded applications. It examines how applications use multicore systems and how hardware configuration and Linux scheduling affect scaling results. ### Source excerpt Subscribe to my newsletter, support me on Patreon, Github, or by PayPal donation. I would love to hear your feedback! I wrote this blog series for the second edition of my book titled "Performance Analysis and Tuning on Modern CPUs". It is open-sourced on Github: perf-book. The book primarily targets mainstream C and C++ developers who want to learn low-level performance engineering, but devs in other languages may also find a lot of useful information. ## Memory Profiling Part 2. Memory Usage Case Study DevFeed: [Memory Profiling Part 2. Memory Usage Case Study]() Original publisher: [Read original article]() Author: Denis Bakhvalov Published: 2024-02-12T05:00:00Z Content type: tutorial Language: en Sources: [Denis Bakhvalov]() Topics: [Benchmark](), [Linux](), [Ubuntu](), [Windows](), [cpu](), [intel]() Tags: [benchmark](), [book-chapters](), [case-study](), [intel](), [linux](), [performance-analysis](), [processor](), [profiling](), [ubuntu](), [windows]() ### AI overview This article presents a memory-usage case study of Stockfish's built-in benchmark using heaptrack on Linux, with comparisons to Mtuner on Windows. It examines memory consumption and allocation patterns, including long-lived and temporary allocations. ### Source excerpt Subscribe to my newsletter, support me on Patreon or by PayPal donation. Now, let's take a look at how to profile the memory usage of a real-world application. We will use heaptrack, an open-sourced heap memory profiler for Linux developed by KDE. Ubuntu users can install it very easily with . Heaptrack can find places in the code where the largest and most frequent allocations happen among many other things. On Windows, you can use Mtuner which has similar1 capabilities as Heaptrack. ## Memory Profiling Part 3. Memory Footprint with SDE DevFeed: [Memory Profiling Part 3. Memory Footprint with SDE]() Original publisher: [Read original article]() Author: Denis Bakhvalov Published: 2024-02-12T05:00:00Z Content type: tutorial Language: en Sources: [Denis Bakhvalov]() Topics: [Code](), [Cache](), [Instrumentation](), [Emulator](), [x86](), [Software]() Tags: [book-chapters](), [cache](), [code](), [emulator](), [instrumentation](), [performance-analysis](), [software](), [x86]() ### AI overview This tutorial explains how to estimate the memory footprint of a simple 4Kx4K matrix multiplication program. It compares naive matrix multiplication with a loop-interchanged version that improves memory access by preserving row-major traversal, then introduces Intel SDE for measurement while noting its substantial runtime cost. ### Source excerpt Subscribe to my newsletter, support me on Patreon or by PayPal donation. Now let's take a look at how we can estimate the memory footprint. In part 3, we will warm up by measuring the memory footprint of a simple program. In part 4, we will examine the memory footprint of four production workloads. Consider a simple naive matrix multiplication code presented in the listing below on the left. The code multiplies two square 4Kx4K matrices and and writes the result into square 4Kx4K matrix . ## Memory Profiling Part 4. Memory Footprint Case Study DevFeed: [Memory Profiling Part 4. Memory Footprint Case Study]() Original publisher: [Read original article]() Author: Denis Bakhvalov Published: 2024-02-12T05:00:00Z Content type: article Language: en Sources: [Denis Bakhvalov]() Topics: [intel](), [benchmarking](), [Benchmark](), [Optimization](), [Ray Tracing](), [App]() Tags: [applications](), [benchmark](), [book-chapters](), [cpu](), [intel](), [optimization](), [performance-analysis](), [production](), [ray-tracing]() ### AI overview This case study uses Intel SDE to examine the memory footprints of Blender ray tracing, Stockfish, Clang++ compilation, and AI_bench PSPNet segmentation. It compares their behavior over one-billion-instruction intervals and explains how instruction timelines can be approximately converted to seconds using IPC and processor frequency. ### Source excerpt Subscribe to my newsletter, support me on Patreon or by PayPal donation. In this case study we will use the Intel SDE tool to analyze the memory footprint of four production workloads: Blender ray tracing, Stockfish chess engine, Clang++ compilation, and AI_bench PSPNet segmentation. We hope that this study will give you an intuition of what you could expect to see in real-world applications. ## Finding Java Thread Leaks With JDK Flight Recorder and a Bit Of SQL DevFeed: [Finding Java Thread Leaks With JDK Flight Recorder and a Bit Of SQL]() Original publisher: [Read original article]() Published: 2023-02-28T21:16:10Z Content type: tutorial Language: en Sources: [Gunnar Morling]() Topics: [JDK Flight Recorder](), [Java](), [Concurrency](), [Memory Leaks](), [SQL]() Tags: [java](), [jdk](), [jdk-flight-recorder](), [memory-leak](), [performance-analysis](), [root-cause-analysis](), [sql](), [thread](), [threads]() ### AI overview This article explains how to investigate Java thread leaks with JDK Flight Recorder. It describes why thread leaks can cause memory and CPU problems, why thread dumps are limited, and how JFR events can reveal changing thread counts and the source of newly created threads. ### Source excerpt The other day at work, we had a situation where we suspected a thread leak in one particular service, i.e. code which continuously starts new threads, without taking care of ever stopping them again. Each thread requires a bit of memory for its stack space, so starting an unbounded number of threads can be considered as a form of memory leak, causing your application to run out of memory eventually. In addition, the more threads there are, the more overhead the operating system incurs for scheduling them, until the scheduler itself will consume most of the available CPU resources. Thus it's vital to detect and fix this kind of problem early on. ## Merge Data Challenge Results DevFeed: [Merge Data Challenge Results]() Original publisher: [Read original article]() Author: Rodrigo Vasquez Published: 2022-12-05T00:00:00Z Content type: release Language: en Sources: [Ethereum Foundation Blog]() Topics: [Ethereum](), [data](), [Monitoring](), [Open Source]() Tags: [analysis](), [analysts](), [ecosystem-support-program](), [ethereum](), [event](), [monitoring](), [open-source](), [performance-analysis]() ### AI overview The Ethereum Foundation announces the winners of the Merge Data Challenge, which ran for about nine weeks around the Merge. Participants submitted 45 blog posts containing data, analyses, tools, and visualizations, with continuing work needed to monitor and understand Ethereum's proof-of-stake network. ### Source excerpt The Ethereum Foundation is excited to announce the winners of the Merge Data Challenge 🐼. The challenge ran for ~9 weeks surrounding the Merge, allowing for data analysts to gather and review information both before and after the big event. Participants submitted a treasure trove of data and analysis... ## Gradle vs Bazel for JVM Projects DevFeed: [Gradle vs Bazel for JVM Projects]() Original publisher: [Read original article]() Author: Piotr Jagielski Published: 2020-06-30T04:00:00Z Content type: article Language: en Sources: [The Gradle Blog]() Topics: [Gradle](), [toolchain](), [Scalability](), [Scala](), [Maven]() Tags: [bazel](), [build-system](), [comparison](), [comparisons](), [ecosystem](), [github](), [gradle](), [jvm](), [kotlin](), [maintenance](), [performance-analysis](), [projects](), [scalability](), [speed](), [toolchain]() ### AI overview This article compares Gradle and Bazel for JVM projects. It reports that Gradle performed better in nearly all tested scenarios, while Bazel optimization required substantially more build authoring and maintenance effort. The article also evaluates build-system capabilities, performance factors, and common JVM use cases. ### Source excerpt Introduction Gradle has emerged as the build tool of choice for projects within the JVM ecosystem, including Kotlin. It is the most popular build tool for open source JVM projects on GitHub. It is downloaded on average more than 15 million times per month and has been counted in the Top 20 Most Popular Open Source Projects for IT by Techcrunch. Many popular projects have migrated from Maven to Gradle, with Spring Boot being a prominent example. Recently, we have received inquiries about the suitability of Google's Bazel build tool for usage within JVM environments. What follows is a detailed comparative performance analysis and evaluation of key capabilities, that arrives at three major conclusions: Despite Bazel's strong and well-deserved reputation for performance and scalability, Gradle outperforms Bazel in almost every scenario we tested. Optimizing projects for Bazel comes at a significant cost for build authoring and maintenance. Gradle provides more compelling features and conveniences for common use cases in JVM projects. In summary, the data and analysis indicates clearly that Gradle is a better choice than Bazel for most JVM projects. We will provide an equivalent comparison for Android projects in a follow up article. At the same time, recognizing that individual tools have unique strengths in addressing the needs and requirements of specific developer ecosystems and specific use cases, we expect that build tool specialization and fragmentation will continue to be the norm across and even within these ecosystems (e.g. Gradle and Maven). In recognition of this, Gradle Enterprise provides analytics and acceleration for more than just Gradle. This allows users to benefit from faster and more reliable builds without migrating to any specific build tool. Today this includes Gradle and Maven, but will include Bazel and other tools in the future. To learn more about the acceleration technology supported in Gradle Enterprise for the Gradle, Maven, and soon Bazel ## HTTP Toolkit Pro Adds Context-Rich HTTP Debugging and Automated Performance Analysis DevFeed: [HTTP Toolkit Pro Adds Context-Rich HTTP Debugging and Automated Performance Analysis]() Original publisher: [Read original article]() Author: HTTP Toolkit; Tim Perry Published: 2019-04-24T12:00:00Z Content type: release Language: en Sources: [HTTP Toolkit]() Topics: [HTTP](), [debugging](), [Tooling](), [API](), [Caching](), [OpenAPI Specification](), [Compression](), [Open Source](), [cross-platform]() Tags: [announcements](), [api](), [caching](), [compression](), [cross-platform](), [debugging](), [http](), [interception](), [open-source](), [openapi](), [performance-analysis](), [tools]() ### AI overview This release introduces new HTTP Toolkit Pro features for understanding HTTP traffic. It adds API-aware request analysis using OpenAPI data, inline parameter documentation, validation warnings, and automated performance analysis covering response time, compression, and caching. ### Source excerpt HTTP Toolkit is a suite of open-source & cross-platform tools for developing, debugging & testing anything using HTTP. It lets you intercept HTTP(S) with one click, and explore, examine & understand all your traffic, to spot bugs, fix bugs, and build better software. The free release of the first version (HTTP View) has been available for a little while, and today I've got some new killer features launching as part of HTTP Toolkit Pro, to let you look deep inside your HTTP traffic, with all the context you need to understand everything your application is doing. API integrations for 1400+ APIs Understanding your traffic takes more than just the raw data. Using OpenAPI and the OpenAPI directory, HTTP Toolkit can work out exactly which API every single request is talking to, for APIs from AWS to Stripe to Github, and a whole lot more. With that, there's a lot of cool things we can do. For example: Here we've taken a request to the YouTube API, and immediately worked out what operation it's doing, interpreted the parameters to provide inline documentation, pointed out that one parameter has an invalid value, and spotted another required parameter that's missing. Debugging tools with real context - tools that really understand what you're trying to do - let you take your development skills to a whole new level. Performance analysis, tips & warnings Performance is hard. There's a huge number of ways to tweak & tune the speed of your application with HTTP, a lot of confusing specs (what's the difference between a no-store and no-cache cache-control header?), and not a lot of advice. For most applications though, the two most important things to focus on are compression, and caching. Streaming large responses hugely slows down client applications, and caching lets them avoid request round trips entirely. You can now get automated performance analysis for all HTTP responses, including the response time itself, but also the details of the request & response body compression ## Tracing Gradle task execution DevFeed: [Tracing Gradle task execution]() Original publisher: [Read original article]() Published: 2018-08-01T00:00:00Z Content type: article Language: en Sources: [Jake Wharton]() Topics: [Gradle](), [tracing](), [ci](), [Java]() Tags: [analysis](), [build](), [ci](), [concurrent](), [flamegraph](), [gradle](), [jdk](), [performance](), [performance-analysis](), [profile](), [tracing]() ### AI overview This article explains ways to trace Gradle task execution and analyze build performance. It compares Gradle's built-in profiling and scanning mechanisms with local approaches using Gradle profiler integrations, Java Flight Recorder, Java Mission Control, command-line conversion, and flamegraphs, while noting limitations for CI builds and daemon-backed tasks. ### Source excerpt Gradle provides two built-in mechanisms for tracing your build: --profile and --scan. The former produces a simple HTML report of task execution times. You can get a rough idea of where time was spent but are unlikely to glean any real insights. The latter sends a detailed report to Gradle's servers (or to a Gradle Enterprise installation) with much more granular information. Task details are rendered on a concurrent timeline corresponding to their execution. For CI builds, I tend to want something more granular than --profile but I don't like the idea of sending details of every build to Gradle with --scan. It seems entirely needless considering their plugin has all of that information locally but chooses to render it remotely. The Gradle profiler project started a few years ago as a way to deterministically measure build speeds. By creating scenarios such as an ABI-breaking change, ABI-compatible change, Android resource change, etc., the tool can run these scenarios multiple times to first warm up the JVM and then to produce an accurate picture of what gets executed. It offers integrations and outputs for use with popular JVM-based performance analysis tools such as YourKit and Java Flight Recorder. For CI builds, executing through the Gradle profiler would be an annoying abstraction to use. We can instead use it for inspiration and run its integrations on individual builds. Java Flight Recorder can be used on individual Gradle builds with the jcmd binary in the JDK and with flags to java specified on the org.gradle.jvmargs in your gradle.properties. There are even Gradle plugins which offer to start and stop the recording automatically. We can then open the resulting .jfr file in Java Mission Control or use a command-line tool to convert it into a flamegraph. The flamegraph can show where time is being spent inside of tasks over the course of the build. The stacks aren't correlated to a task, though, so it's important to remember that you're looking at the large ## Scaling Django Admin Date Hierarchy DevFeed: [Scaling Django Admin Date Hierarchy]() Original publisher: [Read original article]() Author: Haki Benita Published: 2017-10-05T21:00:00Z Content type: article Language: en Sources: [Haki Benita]() Topics: [Django](), [Database]() Tags: [articles](), [django](), [django-admin](), [performance](), [performance-analysis](), [usability]() ### AI overview This article examines the performance cost of Django Admin's date hierarchy on large tables. It describes the database queries used to populate the hierarchy and presents a package that replaces those queries with generated date ranges under stated assumptions, improving page load performance and usability. ### Source excerpt The date hierarchy is a great feature but it comes at a price. On very large tables, the way date hierarchy is implemented can make an admin page nearly unusable. In this article we describe the limitations of the date hierarchy, and suggest a way to overcome them. ## Open Sourcing Daytona: A Framework For Automated and Application-agnostic Performance Analysis DevFeed: [Open Sourcing Daytona: A Framework For Automated and Application-agnostic Performance Analysis]() Original publisher: [Read original article]() Author: mikesefanov Published: 2017-05-23T16:00:06Z Content type: release Language: en Sources: [Yahoo]() Topics: [Testing](), [benchmarking](), [Framework](), [Open Source](), [Tool](), [ui](), [Command-line interface](), [CI/CD](), [Cloud](), [on-prem]() Tags: [architecture](), [ci-cd](), [cloud](), [cloud-services](), [command-line](), [data](), [framework](), [metrics](), [on-prem](), [open-source](), [performance](), [performance-analysis](), [performance-testing](), [systems-engineering](), [tool](), [ui](), [yahoo](), [yahoo-engineering]() ### AI overview Yahoo presents Daytona as an open-source, application-agnostic framework for automated performance testing and analysis. It provides repeatable test execution, standardized reporting, profiling, a unified interface, command-line script support, scheduling, CI/CD integration, and deployment on-premises or in the cloud. ### Source excerpt By Sapan Panigrahi and Deepesh Mittal Today, we are pleased to offer Daytona, an open-source framework for automated performance testing and analysis, to the community. Daytona is an application-agnostic framework to conduct integrated performance testing and analysis with repeatable test execution, standardized reporting, and built-in profiling support. Daytona gives you the capability to build a customized test harness in a single, unified framework to test and analyze the performance of any application. You'll get easy repeatability, consistent reporting, and the ability to capture trends. Daytona's UI accepts a performance testing script that can run on a command line. This includes websites, databases, networks, or any workload you need to test and tune for performance. You can submit tests to the scheduler queue from the Daytona UI or from your CI/CD tool. You can deploy Daytona as a hosted service in your on-prem environment or on the public cloud of your choice. In fact, you can even host test harnesses for multiple applications with a single centralized service so that developers, architects, and systems engineers from different parts of your organization can work together on a unified view and manage your performance analysis on a continuous basis. Daytona's differentiation lies in its ability to aggregate and present essential aspects of application, system, and hardware performance metrics with a simple and unified user interface. This helps you maintain your focus on performance analysis without changing context across various sources and formats of data. The overall goal of performance analysis is to find ways of maximizing application throughput with minimum hardware resource and the best user experience. Metrics and insights from Daytona help achieve this objective. Prior to Daytona, we created multiple, heterogenous performance tools to meet the specific needs of various applications. This meant that we often stored test results inconsistently, maki ## Beyond iostat: Storage performance analysis with blktrace DevFeed: [Beyond iostat: Storage performance analysis with blktrace]() Original publisher: [Read original article]() Author: Marc Brooker Published: 2013-07-14T00:00:00Z Content type: article Language: en Sources: [Marc Brooker's Blog](), [Marc Brooker's Blog]() Topics: [IO](), [Linux](), [Monitoring](), [Processes](), [benchmarking]() Tags: [io](), [linux](), [monitoring](), [performance](), [performance-analysis](), [processes](), [smart](), [tool]() ### AI overview This article explains how to investigate Linux storage I/O performance beyond the aggregate metrics provided by iostat. It introduces blktrace and related tools for capturing detailed disk I/O traces, then using blkparse to analyze them and identify process-level or time-sensitive performance issues. ### Source excerpt Beyond iostat: Storage performance analysis with blktrace An under appreciated set of IO analysis tools. If you've spent much time at all investigating IO performance on Linux, you're no doubt already familiar with iostat from the venerable sysstat package. iostat is the go-to tool for Linux storage performance monitoring with good reason: it's available nearly everywhere, it works on the vast majority of Linux machines, and it's relatively easy to use and understand. Some of what it measures can be subtle, and the exact definitions of its measurements can be confusing, and even contentious, but it's still a great start. Sometimes, though, you need to go into more detail than iostat can provide. The aggregate view from iostat is simple, but makes it difficult to tell which processes are doing which IOs. Averaging over a period of time can hide subtle performance issues and the real causes of may IO-related problems. To get around these issues, you'll want to go deeper. If you have the guts for it, a recent kernel, and a good understanding of IO performance issues, you'll want to reach for blktrace and friends. The blktrace toolkit provides an extremely powerful way to look at the exact IO performance of a Linux machine, at a wide range of levels of detail, and is vastly more capable than the simple iostat. For a start, let's look at the performance of a random read workload to a magnetic drive, with 16k IOs. The manufacturer's spec sheet says this drive should be delivering about 120 IOs per second on a completely random load. iostat -x has this to say about the drive: Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz sdb 0.00 0.00 124.67 0.00 3989.33 0.00 32.00 avgqu-sz await svctm %util 1.00 8.01 8.02 100.00 As expected, we're doing about 125 random IOs per second, each at 16k (32.00 512 byte sectors), at a mean service time of around 8ms. That's pretty much exactly what we would expect from a 7200 RPM magnetic drive. Nothing to see there, then. Next up, is a d