In any system under performance consideration, there are 3 important timescales.
- The timescale at which the CPU works
- The timescale at which the Memory can be accessed
- The timescale at which data can be stored on disk
These three have widely different timescales in a traditional computer setup. CPU runs at nano second resolution, memory at micro-second and Disks at milli-second.
This makes it difficult to estimate the true efficiency of code. Because, the rate at which the instructions can be processed also now depends on the rate at which the memory is accessed and the disk is utilized. Hence, to truly measure the code efficiency one needs to make sure that the disk response times do not play a major role in the flow of instructions. In other words on e should take care of removing the IO bottlenecks before analyzing the performance of code.
In presence of strong influence of disk IOs the measurements will be biased by the characteristics of the storage system. The order of magnitude difference between the response times of disks and memory and memory and CPU makes it even more difficult to remove the IO bottleneck. While the CPUs have become faster in the last few years and there are now CPUs with multiple cores the memory and disk access speeds haven’t kept pace. As a result the contrast between the response times of CPU, Memory and Storage has widened.