How to identify processes that are consuming lots of CPU power

Here is one way to identify processes that are consuming lots of CPU on your sever that I have used successfully in the past.

First, use the mpstat command, which will print the CPU statistics n times with n seconds interval.
For example, mpstat 5 5
would yield the following output:

CPU  minf  mjf  xcal  intr  ithr  csw  icsw  migr  smtx  srw  syscl  usr  sys  wt  idl 
0  1  0  0  345  224  589  220  0  0  0  799  29  1  0  70 
0  1  0  0  302  200  752  371  0  0  0  1191  99  1  0  0 
0  0  0  0  341  221  767  375  0  0  0  1301  100  0  0  0 
0  0  0  0  411  256  776  378  0 0  0  1313  99  1  0  0 
0  0  0  0  382  241  738  363  0  0  0  1163  97  3  0 0 

In the output sample above, 4 of the 5 samples have CPU 0 with

* a combined user time and system time at 100
and
* idle time at 0 (column headings usr, sys, idl).

This indicates that the CPU is completely consumed on this system.

After gathering the data from mpstat, which indicates that the system CPU resources are overtaxed, you can use prstat to identify which processes are consuming the CPU resources.

The prstat -s cpu -n 5 command is used to list the five processes that are consuming the most CPU resources.

* The -s cpu flag tells prstat to sort the output by CPU usage.
* The -n 5 flag tells prstat to restrict the output to the top five processes.

prstat -s cpu -n 5 

   PID USERNAME  SIZE   RSS STATE  PRI NICE      TIME  CPU PROCESS/NLWP
 25906 oracle   2692M 1912M cpu2     0    0   0:34:39 3.1% oracle/11
 25472 oracle    133M  130M cpu1     0    0   0:00:47 0.4% emagent/7
  8709 oracle     33M   19M cpu28   10    0   0:00:03 0.4% perl/1
 19404 root       88M   24M sleep   21    0 141:23:44 0.3% vxconfigd/1
  9767 root     5752K 5224K sleep   59    0   0:00:01 0.2% vmstat/1