Analyzing Tracepoint Output
Last week I had this displeasure of tracking down a fairly unpleasant reference counting bug. I wound up having to solve the problem by brute force:
- Set a breakpoint in the constructor of leaking object
- Set a data breakpoint on the object's reference count variable
- Change the data breakpoint to a tracepoint that prints the callstack (see dialog)
- Match up the callstacks.
However, my tracepoint was hit so often that matching up callstacks by hand proved to be impossible. To help, I wrote a little tool that I decided to post. The tool does some basic analysis of the tracepoint output - compute a database of all callstacks with their hit count, and compute a calltree with hit count. While matching up the callstacks was still not easy, it did help.
Basically, the tool gave me two things:
- In places where an AddRef and Release where matched with the same callstack, I could bucket all of these callstacks together and eliminate them quickly
- If CFoo::CFoo called AddRef and CFoo::~CFoo called release, I could easily filter out the callstacks that contained neither CFoo::CFoo or CFoo::~CFoo. If the resulting callstacks balanced out (just as many CFoo::CFoo as CFoo::~CFoo) then I could throw out all these callstacks quickly.
The code for my tool is in the attached ‘TracepointAnalysis.cs’ file. Here is some example code to call it:
static void Main(string[] args)
{
List<Callstack> stacks;
List<CallTreeNode> callTreeRoots;
CallstackFilter filter = delegate(CallstackKey key) {
return key.Contains("CFoo::CFoo") || key.Contains("CFoo::~CFoo");
};
TracepointAnalysis.ReadFile(@"c:\log.txt", filter, out stacks,
out callTreeRoots);
DumpStacks(stacks);
DumpCallTree(callTreeRoots);
}
private static void DumpCallTree(List<CallTreeNode> roots)
{
foreach (CallTreeNode root in roots)
{
root.Dump(0);
}
}
private static void DumpStacks(List<Callstack> list)
{
// Sort the stacks by their hit count
Comparison<Callstack> comparison = delegate(Callstack a, Callstack b)
{
return a.HitCount - b.HitCount;
};
list.Sort(comparison);
foreach (Callstack item in list)
{
item.Dump();
}
}