Posts tagged ‘file system’

April 8, 2011

Performance Engineering – SSD file systems.

Solid state device threatens to challenge and change existing computing paradigms.

While traditional disk access times are of the order of a few milli seconds, ssd access times are under 100 micro seconds for reads and writes respectively. Speeding up by a factor of 100. And that is significant.

Operating system components have evolved over the last 4 decades at a much slower speed. For an enterprise platform like IBM AIX or HPUX it takes A few years (my guess is a minimum 6) to push a new technology. The cycle is as follows – a new hardware technology is invented. OS vendors take a few years to adopt and evangelize. Enterprise customers longer to test, adopt and deploy.

SSD promises to deliver better performance by lowering IO latency and increasing throughput. File systems have evolved to do the same. Specialized caches have been invented to speed up performance. For example directory name lookup cache, page cache, inode cache, large directory cache, buffer cache and so on.

Quite a lot of focus is on being clever with reads and writes of application data. Engineers go to great extents to squeeze the last bit of performance out of the system. Sadly performance is not a main consideration during implementation (functionality is) and is often times applied as an afterthought.

The result is hacks rather than elegant solutions to performance issues.

Coming back to SSDs – a large portion of the file system implementation has to be re looked at and parts of it have to be thrown away completely. Especially true when complete filesystems are laid out on SSD. We need to look at how filesystems can take advantage of SSDs.

April 7, 2011

Balance File System Caching and Flushing policies

Most file-systems have intelligent caching policies. The policies are designed to increase the throughput and decrease IO latency rates thereby providing faster service times to the users and applications. Based on the nature of the workload, appropriate caching policies can be set to achieve maximum cache-hit rates.

However, this works as long as there is enough memory to store revisited pages. Once the number of pages cached exceeds the set memory limit, the file-system decides to reclaim space by flushing older pages to the storage. If flushing is not done frequently, a lot of data may suddenly be dumped on the disk leading to storage bottleneck.

Depending on storage bandwidth, flushing large amount of file-system data can lead to very large service times for the users or the application. For all-round good performance one thus needs to also look at how frequently and how much of data is flushed to the storage. Just like caching policies should take into account the nature of workload, flushing policies should take into account the nature of storage and storage i/o bandwidths.

A good sustained file-system performance is possible only when both, caching and flushing, policies are set optimally.