Andrew's Blog

A tool I discovered: apachetop can read apache log files, and show performance data in a human readable format. apachetop will use the default apache log files, and apachetop -f /path/to/logfile will show stats for a selected log file. (continue reading…)

Linux IO is a pretty big topic, and this is just a quick entry with some links and notes about tools I have found, I hope to do a more in depth look later.

First up Network IO:
The command ifstat gives you a pretty decent look at current bandwidth usage. ifstat -l will also show loopback devices

Next Up Disk IO, this is a pretty big one, as this is probably a big cause of system load in my environments.
The command iostat gives a basic run down of the disk usage.
Using the eXtended option iostat -x gives you a more detailed look, and is much more useful in finding issues.
Running with an interval at the end will re-run the stats using live data, which again is more useful, and will refresh the display: iostat -x 1 will refresh the data every 1 second.

1. r/s and w/s are the requests by a process to read and write to the device
2. rsec/s and wsec/s sectors per second.
3. avgrq-sz is the average sectors per request (both read an write). Divide by 2 to get bytes.
4. avgqu-sz is the average queue length
5. await is the average wait time for a request, includes the queue time and the service time
6. svctm is the average time spent servicing requests, the actual read/write time
7. %util is the utilisation percentage, not quite accurate on shared storage/multiple disk raid

To calculate the queue time, take the svctm from await.
If the queue time is greater then the service time, then the device is probably overloaded. Ideally the await should be pretty much equal to the svctm figure, as then there are not many requests being queued.

Some good references:
Quick Reference about iostat
http://www.pythian.com/news/247/basic-io-monitoring-on-linux/
and a detailed blog entry on iostat, and linux disk IO in general…
http://blog.serverfault.com/post/777852755/interpreting-iostat-output

As a final note, for VMWare ESX, have a look at esxtop, and use the d and v and u keys to display various disk related stats, see man esxtop for more, and I hope to cover more later…