Slow computers: Performance issues and Event Tracing

[Christy]

A lot of people complain about slow computes, & generally they are provided with advice such as using System Restore, Antivirus tools ,Ccleaner like tools, Defragging their hard disk & uninstalling unwanted software. In most of the cases such as a virus or trojan infection finding the infection solves the issue. But, ever wondered why uninstalling certain software improves system performance at times? Drivers. How do we know if and which driver is causing issues. This post starts where such general fixes don't fix the problem.

Event tracing: The basic idea here is to trace the event using up CPU time, leaving lesser resources to other processes on the system, resulting in what people call a "Slow computer". here I'm using RATTV3: A Trace Processing Tool. RATTV3 is a Microsoft Windows tool for auditing execution times of interrupt service routines (ISRs), deferred procedure calls (DPCs), and timer DPCs. RATTV3 is designed to help developers of drivers and other kernel mode components audit the ISR and DPC execution time of their components. However, we use it a bit differently.We'll use RATT to find out where to start looking for the problem driver.

ISRs and DPCs (including timer DPCs) typically run for relatively short periods of time and are not problematic. However, some drivers and other kernel mode components do generate ISRs and DPCs that are long enough to cause system wide problems such as:

• Audio and video "glitches"
• Problems with other devices by overly delaying the execution of other ISRs and DPCs
• Interference with applications due to over use of CPU time.

Don't bother if you don't know what ISR and DPC is. I`m writing this so that anybody can use the tool. Leaving aside the mumbo-jumbo lets get to work.


The RATTV3 installer can be downloaded from the Microsoft website under the following url:
http://msdn.microsoft.com/en-us/wind.../gg487354.aspx

After installation, follow these steps:

• Right click the RATTV3 Symbol in the Taskbar and turn off "Loop Mode".
• Right click the RATTV3 Symbol in the Taskbar again and select "Stop monitoring".
• Browse to the following folder on your computer:"C:\WINDOWS\system32\LogFiles\RATTV3" [for Win Xp]
• Delete all files in the folder.
• Right click the RATTV3 Symbol in the Taskbar again and select "Start monitoring".
• Let the system program run for 3-4 minutes(with the suspect application if any running).
• From the folder "C:\WINDOWS\system32\LogFiles\RATTV3" open the file <machine-name>.cswa-accumulator-report.txt with the Windows application application "WordPad". Other text editors might not show a formatting.
• For each driver section(Heading), look for the last entry within each driver section and check if you can find a value which is higher than 1 ms in the "Label" column as shown in the following log file below.
• Note down the driver name.

OR
Inside this file you'll find a section for each kernel mode driver (DRIVERNAME.SYS). Each section is a histogram of DPC times. There are 1109 possible "buckets" a driver call can fall into. A bucket is a range of duration times. What we are looking for is when calls to a given driver take longer than 1 millisecond. When we find that, we have identified a driver that could be behaving better.

The easiest way to spot the misbehavior is to look at the first column Buckets no, the buckets we don not want will be numbered above 100. Ignore anything with a bucket number under 50. Those are too fast to matter. Any driver with items in buckets numbered over 109 is doing DPCs that take longer than a millisecond (1000 microseconds), and may be contributing to our performance issues. Further troubleshooting will be needed to determine that.

A Healthy Driver has a histogram like this:
Driver Name: afd.sys

Cumulative Histogram for: afd.sys        (\SystemRoot\System32\drivers\afd.sys)
  #,                  Range, Label,       ISR Count,  DPC Count,  DPCTmr Count
  0,    0.00us to    0.99us, <1.00us,             0,          0,             5
  1,    1.00us to    1.99us, <2.00us,             0,          0,            36
  2,    2.00us to    2.99us, <3.00us,             0,          0,             1
                              TOTALS,             0,          0,            42

A possible Sick Driver (deserves further scrutiny) would appear like this:
Driver Name: RtkHDAud.sys

Cumulative Histogram for:  RtkHDAud.sys  (\SystemRoot\system32\drivers\RtkHDAud.sys)
  #,                  Range, Label,       ISR Count,  DPC Count,  DPCTmr Count
  0,    0.00us to    0.99us, <1.00us,             0,      12434,             0
  1,    1.00us to    1.99us, <2.00us,             0,       2042,             0
  2,    2.00us to    2.99us, <3.00us,             0,       1187,             0
  3,    3.00us to    3.99us, <4.00us,             0,       1138,             0
  4,    4.00us to    4.99us, <5.00us,             0,        213,            20
  5,    5.00us to    5.99us, <6.00us,             0,         88,           290
  6,    6.00us to    6.99us, <7.00us,             0,         30,          1448
137,    1.28ms to    1.29ms, <1.29ms,             0,          1,             0
211,    2.02ms to    2.03ms, <2.03ms,             0,          1,             0
239,    2.30ms to    2.31ms, <2.31ms,             0,          1,             0
244,    2.35ms to    2.36ms, <2.36ms,             0,          1,             0
250,    2.41ms to    2.42ms, <2.42ms,             0,          1,             0
251,    2.42ms to    2.43ms, <2.43ms,             0,          1,             0
259,    2.50ms to    2.51ms, <2.51ms,             0,          1,             0
266,    2.57ms to    2.58ms, <2.58ms,             0,          1,             0
292,    2.83ms to    2.84ms, <2.84ms,             0,          1,             0
489,    4.80ms to    4.81ms, <4.81ms,             0,          1,             0
551,    5.42ms to    5.43ms, <5.43ms,             0,          1,             0
642,    6.33ms to    6.34ms, <6.34ms,             0,          1,             0
664,    6.55ms to    6.56ms, <6.56ms,             0,          1,             0
971,    9.62ms to    9.63ms, <9.63ms,             0,          1,             0
1009,   10.00ms to   19.99ms, <20.00ms,            0,          1,             0
1010,   20.00ms to   29.99ms, <30.00ms,            0,          4,             0
1011,   30.00ms to   39.99ms, <40.00ms,            0,          1,             0
1012,   40.00ms to   49.99ms, <50.00ms,            0,          1,             0
1015,   70.00ms to   79.99ms, <80.00ms,            0,          1,             0
                              TOTALS,             0,      34350,          2514

From bucket no 137-1015, Label>1ms.
RtkHDAud.sys has locked the CPU for 80 ms.
It is a Realtek HD Audio Driver.


Generally, Low bucket no's are Good & High bucket no's are Bad.
The 3 times analysed by RATT: ISR, DPC and Timer DPC are effectively covered in the above rule.

It's important to note that long CPU times aren't the problem here, they are a clue to the real issue. They drivers should be used to narrow down the search field for your cause of your performance issue. It's still up to you to determine the fix.

What do we do after we have isolated suspect Drivers?

If you have values above 1ms, check to which devices the affected driver belong(Google), then disable the affected devices in Windows Device Manager if possible. If you are unsure if the device can be disabled, then you should try to search the web for the driver's name in order to find out if the associated device can safely be disabled.

Important: Here is a list of devices that you should NEVER deactivate since these are essential components required by Windows: System timer, Keyboard, System CMOS/real time clock, Microsoft ACPI-Compliant System, Numeric data processor, Primary IDE Channel, Secondary IDE Channel, Graphics Controller, Ultra ATA Storage Controllers. In general, you should not deactivate anything which is listed in the branch "System Devices".

How to De-activate devices in the Windows Device Manager?

• Right click the "Computer" icon on your desktop. Then select Properties > Device Manager
• In the Device Manager, locate the devices RATTV3 reported as problematic. If this is a device that is not essential for the basic operation of your computer, right click on the component and choose "Deactivate" ,(NOT "Uninstall"). Once you disabled the problematic devices (according to RATTV3), test if this resolved the performance issue.

• Selectively disable multiple suspect devices one at a time with device manager to see if your issues get cleared up.
• Additional devices that can often be disabled to further preserve system resources.
• Get updated drivers for the component if possible.
• Check the hardware behind the driver for performance issues.

PS:Tested on Windows XP32bit
RATT doesn't seem to work on 64bit systems

Alternative: Kernate, Don't know if it works on 64bit systems , For those who wish to try: http://www.microsoft.com/download/en...ng=en&id=24853

[Bearcat]

Thanks for the great tutorial Christy. It was very informative and I going to use RATTV3 to check out my PC.

[BigGuy]

Thank you Christy. I am going try out RATTV3 for myself.

[lello.siddi]

You say: "Browse to the following folder on your computer:"
But do not mention the folder...

[Raymond]

It should logically be at C:\WINDOWS\system32\LogFiles\RATTV3
You have to delete the auto generated logs and then start over again to create new logs.

[lello.siddi]

It should logically be at C:\WINDOWS\system32\LogFiles\RATTV3
You have to delete the auto generated logs and then start over again to create new logs.

Thank you! I didn't have the all picture...

[Christy]

You say: "Browse to the following folder on your computer:"
But do not mention the folder...

Misd it, edited it way too many times, missed that.

[lello.siddi]

Misd it, edited it way too many times, missed that.

In my case, that was not a sterile question! I was following your instructions and just didn't know how to go ahead...
I can continue now!
Thank you for your work,
Cheers.

[sujay]

Shouldn't it be under tutorials section?

Great tut Christy..