Visualizing Silverlight / .NET managed memory using Pivot

I got to see this amazing video on Pivot from Gary Flake.  Pivot is about visualizing data. It’s primarily for visualizing patterns and trends. What other way I would use other than analyzing managed memory using pivot. I spend a lot of time in windbg to look at memory for patterns, analyzing memory leaks.  And I also know not many of them want to learn the arcane commands of windbg. So I went ahead took the data from managed memory and ported it in to pivot format , which is cxml. I use the “!Traverseheap –xml” command from sos to generate the xml ,which I port it to cxml format.My initial thought was use dgml to visualize the data, but pivot is way better. Pivot can only be used in Vista+  OS.

FYI I know this is not for everyone, there are geeks who wouldn’t want to look at memory in a visual format, they prefer looking at in cdb.  I took Facebook Silverlight client app and got a snapshot of the memory for this demonstration

The few things that can be done with this are

  • Drill down objects by type with just one click : What I mean by this is , you could probably have 6000 odd types of objects and not instances (instances could be in thousands), but if you wanted to look at all the instance of customer objects  , it is as easy as one click from the filter.

Here is an example of filtering securitypermission objects in memory

  • Drill down members: The objects members can be looked up. So if you are looking at the customer object then you can view  strings, ints and all its members. It WILL NOT have the value for these members. It is not provided by the traverseheap command

  • Filter objects by namespace– All the objects belonging to namespaces can be drilled down.

  • Filter by object size

void Main()
 var startString =@"<?xml version=""1.0"" encoding=""utf-8""?>
<?xml-stylesheet type=""text/xsl"" href=""../pivot.xsl""?>
<Collection xmlns:xsi="""" xmlns:xsd=""""
 Name=""Memory Visualization"" SchemaVersion=""1"" d1p1:Icon=""icon.ico""
 xmlns:d1p1="""" xmlns="""">
 <FacetCategory Name=""Type"" Type=""String"" d1p1:IsFilterVisible=""true"" d1p1:IsMetaDataVisible=""true"" d1p1:IsWordWheelVisible=""true"" />
 <FacetCategory Name=""Address"" Type=""String"" d1p1:IsFilterVisible=""true"" d1p1:IsMetaDataVisible=""true"" d1p1:IsWordWheelVisible=""true"" />
 <FacetCategory Name=""Size""  Type=""Number"" d1p1:IsFilterVisible=""true"" d1p1:IsMetaDataVisible=""true"" d1p1:IsWordWheelVisible=""true"" />
 <FacetCategory Name=""Namespaces"" Type=""Link"" d1p1:IsFilterVisible=""true"" d1p1:IsMetaDataVisible=""true"" d1p1:IsWordWheelVisible=""true"" />
 <FacetCategory Name=""Members"" Type=""Link"" d1p1:IsFilterVisible=""true"" d1p1:IsMetaDataVisible=""true"" d1p1:IsWordWheelVisible=""true"" />
 <Items ImgBase=""test.gif"">";

 var endString = @"</Items>

 var counter = 0;
 var outputFileName = "sl.cxml";
 var dict = new Dictionary<string,string>() { {"String","img5.jpg"},{"Object","img6.jpg"}};
 var getimg = new Func<string,string>((t) => dict.ContainsKey(t) ? dict[t] : "img8.jpg");

 var doc = XDocument.Load(@"c:\Temp\fb.xml");

 var types = (from i in doc.Descendants().Descendants().Descendants("type").AsParallel()
 let ns = i.Attribute("name").Value.SplitEx().ToList()
 select new {Id = int.Parse( i.FirstAttribute.Value),Name = ns.Last(),
 Namespaces = ns.Count()  > 2 ? ns.Take(ns.Count () -2) : ns.Take(ns.Count -1)}).ToList();
 var y = from i in doc.Descendants().Descendants().Descendants("object").AsParallel()
 let type = types.First (t => t.Id== int.Parse( i.Attribute("typeid").Value))
 select new {Address = i.FirstAttribute.Value,
 Type = type.Name,
 Namespaces = type.Namespaces,
 Size = int.Parse( i.LastAttribute.Value),
 children = from k in i.Descendants().AsParallel()
 from o in doc.Descendants().Descendants().Descendants("object").AsParallel()
 where k.FirstAttribute.Value == o.FirstAttribute.Value
 select new {Address = k.FirstAttribute.Value,Type = types.First (t => t.Id == int.Parse( o.Attribute("typeid").Value) ).Name}

 var x = from i in y.Take(2000).AsParallel()
 let increment = new Func<int>(() => Interlocked.Increment(ref counter))
 select new XElement("Item",new XAttribute("Img",getimg(i.Type)),new XAttribute("Id",increment()),new XAttribute("Name",i.Type),
 new XElement("Facets",
 new XElement("Facet",new XAttribute("Name","Type"),
 new XElement("String",new XAttribute("Value",i.Type))),
 new XElement("Facet",new XAttribute("Name","Size"),
 new XElement("Number",new XAttribute("Value",i.Size.ToString()))),
 new XElement("Facet",new XAttribute("Name","Address"),
 new XElement("String",new XAttribute("Value",i.Address))),
 new XElement("Facet",new XAttribute("Name","Namespaces"),
 i.Namespaces.Select (n =>new XElement("Link",new XAttribute("Name",n),new XAttribute("Href",string.Format("{0}#Link=EQ.{1}",outputFileName,n))))),
 i.children.Any() == false ? default(XElement):
 new XElement("Facet",new XAttribute("Name","Members"),
 i.children.Select (n =>new XElement("Link",new XAttribute("Name",n.Type),new XAttribute("Href",string.Format("{0}#Address=EQ.{1}",outputFileName,n.Address)))))
 File.WriteAllText(@"c:\temp\sl.cxml",string.Format("{0}{1}{2}",startString,x.Select (s => s).Aggregate(new StringBuilder(), (ag, n) => ag.Append(n)).ToString(),endString));
 Console.WriteLine ("Done");

static class Foo
 public static IEnumerable<string> SplitEx(this string s)
 if (s.Contains("`") == false)
 return s.Split(new []{'.'});
 var stringList = new List<string>();
 var index =0;
 var counter = 0;
 var splitString = "";
 while (counter < s.Length)
 index = s.IndexOf(".",counter);
 if (index < 0)
 splitString = s.Substring(counter,index -counter );
 if (splitString.Contains("`"))
 counter = index +1;
 return stringList;

The above code converts the xml format from !traverseheap to pivot xml format. This is CPU intensive and I have not profiled and fine tuned it. I use plinq for conversion primarily because I have quad-core box. I use pictures from powerpoint for visualizing  objects and I also wrote SplitEx method to avoid splitting namespaces that are part of the type name within generics “GenericEqualityComparer`1[[System.String, mscorlib]]”

And here is the command to get the xml input for the above code as input

!TraverseHeap -xml c:\temp\mem.xml

2 thoughts on “Visualizing Silverlight / .NET managed memory using Pivot

  1. Pingback: DotNetShoutout

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s