How do we determine processor contention?
Posted: April 21st, 2008 | Author: TnT Admin | Filed under: Analyze | Tags: Bottleneck, processor | No Comments »Simplest way, use the Processor(_Total)\% Processor Time which measures the average processor utilization of your machine (i.e. utilization averaged over all processors not a specified processor). You can further break down the usage by examining which instance/process is hogging the processor with Processor(instance)\% Processor Time.
For example, instances/processes such as IIS and MS Exchange can be examined with Process(inetinfo)\% Processor Time and Process(store)\% Processor Time respectively. While WebLogic Server can be examined with Process(java.exe)\% Processor Time. (Note: WLS may generate more than one java.exe which you will have to be sure of the correct instance). A rule of thumb, keep the Processor(_Total)\% Processor Time under 85%. However, there maybe cases of spikes due to backup jobs, which you can determined with patterns of constant periodic spikes or known backup tasks schedules.
If the processor is running at around 70% to 80%, its normally a good sign and means your machine is handling its load effectively and not under utilized. Average processor utilization of around 20% or 30% may suggests your machine is under utilized and maybe a wise to do server consolidation using VMWare, or allowing more applications to be house in it. (Source: Mitch Tulloch, Windows Networking)
Going down a step deeper, we can determine if the processor is busy working on kernel-related task or user-related task with the use of Processor(_Total)\% Privileged Time and Processor(_Total)\% User Time respectively. Kernel-related task will comprises of OS tasks to ensure the functionality of the OS while user-related task comprises of application tasks not native to the OS (e.g. WebLogic Servers as java.exe). I will discuss it in details in future posts.
If kernel mode utilization is high, your machine is likely underpowered as it’s too busy handling basic OS housekeeping functions to be able to effectively run other applications. It may be wiser to (1) reduce the amount of OS tasks and (2) services or distribute the schedules in non-peak timing. If user mode utilization is high, it may be your server running too many specific roles which you may consider (1) scaling the hardware (adding more or better processor), (2) reduce the number of user applications or (3) horizontal scaling of the server architecture. (Source: Mitch Tulloch, Windows Networking)
In addition, you can use System\Processor Queue Length to determine processor contention. Use this counter when the processor utilization peaks at a period of 100%. If this counter is consistently higher than around 5 when processor utilization approaches 100%, then this is a good indication that there is more work (active threads) available (ready for execution) than the machine’s processors are able to handle.
Mitch highlighted that it is not a definitive way of determining the bottleneck for some services like IIS 6 pool (similarly for JVM which spawns threads from the java.exe process) which manage their own worker threads. For example, on a busy IIS, you would want to look at other counters like ASP\Requests Queued or ASP.NET\Requests Queued as well . Also, the larger the number of active services and applications running on your server will have a busier the processor queue then norm. On such server running near 100% utilization, contention may only be a significant factor when System\Processor Queue Length exceeds more than the recommended queue length of 5 . (Source: Mitch Tulloch, Windows Networking)
Similarly, Microsoft TechNet suggests that different types of multiprocessor to have different guidelines. The suggestion was 3 threads in the queue. That is, if there are 4 processors, the threshold should be 12 for all processors. (Source: Microsoft TechNet)
Leave a Reply