Show POH PerHeapHistoryData in Raw XML view

[cshung]

This change allows us to view the POH data in the raw XML view.

@brianrob
@Maoni0

[Maoni0]

other than what I mentioned in this comment it LGTM!

[Maoni0]

LGTM

[brandonh-msft]

@cshung & @brianrob

This PR performed the following breaking change:

We have code in our codebase like this:

        public TraceGCEx(TraceGC gc)
        {
            Number = gc.Number;
            RelativeTimestamp = gc.StartRelativeMSec;
            Generation = gc.Generation;
            Reason = gc.Reason.ToString();
            Type = gc.Type.ToString();
            PauseTime = gc.PauseDurationMSec;
            PinnedObjectsSize = (double)gc.GetPinnedObjectSizes() / (1024 * 1024);
            PinnedObjectPercent = gc.GetPinnedObjectPercentage() >= 0 ? (double)gc.GetPinnedObjectPercentage() : 0;
            SuspensionTime = gc.SuspendDurationMSec;
            Issues = new Dictionary<string, double>();

            // per heap calculations
            HeapSize = HeapFragmentation = 0;
            SurvivalPercent = new Dictionary<string, double>();
            
            for (int gen = (int)Gens.Gen0; gen < (int)Gens.Gen0After; gen++)
            {
                HeapSize += gc.HeapStats != null ? gc.GenSizeAfterMB((Gens)gen) : 0;
                HeapFragmentation += gc.GenFragmentationMB((Gens)gen);
                var surv = gc.SurvivalPercent((Gens)gen);
                if (!Double.IsNaN(surv))
                {
                    SurvivalPercent.Add(((Gens)gen).ToString(), surv);
                }
            }

What is the recommended way forward? If I change to using .MaxGenCount we get Index OOBs on our traces when processing and choosing .GenPinObj just because it happens to be the same int value as what we were using previously seems awfully dirty.

[cshung]

@cshung & @brianrob

This PR performed the following breaking change:

We have code in our codebase like this:

        public TraceGCEx(TraceGC gc)
        {
            Number = gc.Number;
            RelativeTimestamp = gc.StartRelativeMSec;
            Generation = gc.Generation;
            Reason = gc.Reason.ToString();
            Type = gc.Type.ToString();
            PauseTime = gc.PauseDurationMSec;
            PinnedObjectsSize = (double)gc.GetPinnedObjectSizes() / (1024 * 1024);
            PinnedObjectPercent = gc.GetPinnedObjectPercentage() >= 0 ? (double)gc.GetPinnedObjectPercentage() : 0;
            SuspensionTime = gc.SuspendDurationMSec;
            Issues = new Dictionary<string, double>();

            // per heap calculations
            HeapSize = HeapFragmentation = 0;
            SurvivalPercent = new Dictionary<string, double>();
            
            for (int gen = (int)Gens.Gen0; gen < (int)Gens.Gen0After; gen++)
            {
                HeapSize += gc.HeapStats != null ? gc.GenSizeAfterMB((Gens)gen) : 0;
                HeapFragmentation += gc.GenFragmentationMB((Gens)gen);
                var surv = gc.SurvivalPercent((Gens)gen);
                if (!Double.IsNaN(surv))
                {
                    SurvivalPercent.Add(((Gens)gen).ToString(), surv);
                }
            }

What is the recommended way forward? If I change to using .MaxGenCount we get Index OOBs on our traces when processing and choosing .GenPinObj just because it happens to be the same int value as what we were using previously seems awfully dirty.

Fundamentally, the runtime introduced a new "generation" (technically, the pinned object heap is not a new generation, but it is convenient to represent it that way) in 5.0, so that bound needs to be dynamically adjusted depending on whether or not the trace was captured before 5.0 or not.

To make this task easier, there is a new property named NumGenerations in this PR, can you use that property instead of hard coding?

[brandonh-msft]

Thanks for that explanation, @cshung

To make this task easier, there is a new property named NumGenerations in this PR, can you use that property instead of hard coding?

I considered that, however even after updating to v3.0.4 the object we're using doesn't have that property :(

Any other ideas? I'm inheriting this code so not quite sure what the right thing is ;)

[cshung]

Any other ideas? I'm inheriting this code so not quite sure what the right thing is ;)

I tried to look into this, and it appears to me there are some disconnects.

The new property NumGenerations is added on the GCPerHeapHistoryTraceData. That class seems to be mostly concerned with parsing trace data in binary format. The new property is used mostly because the trace format changed to have variable size content because we have a variable number of generations, and therefore the parsing need to deal with that.

But it appears to me all you have is a TraceGC object instead. I assumed TraceGC is being built as part of the parsing process, but I am unfamiliar with those structures, in particular, I am not sure how you can reach into the corresponding GCPerHeapHistoryTraceData object that was used during the parsing process to get access to that. This will need more study.

You mentioned that there is an index out of bound error when you were experimenting with the code. Do you happen to know which array are you accessing? Can you reach into that array and read the length of that array? Or surface the length of that array as a property of the TraceGC class?

[brandonh-msft]

Yeah the blow-up happens in the loop I posted:

            for (int gen = (int)Gens.Gen0; gen < (int)Gens.Gen0After; gen++)
            {
                HeapSize += gc.HeapStats != null ? gc.GenSizeAfterMB((Gens)gen) : 0;
                HeapFragmentation += gc.GenFragmentationMB((Gens)gen);
                var surv = gc.SurvivalPercent((Gens)gen);

Do you need me to execute it and get details on what gc looks like? It doesn't appear these objects/methods are proper arrays but rather methods which, in turn, index into an array.

[cshung]

Yeah the blow-up happens in the loop I posted:

            for (int gen = (int)Gens.Gen0; gen < (int)Gens.Gen0After; gen++)
            {
                HeapSize += gc.HeapStats != null ? gc.GenSizeAfterMB((Gens)gen) : 0;
                HeapFragmentation += gc.GenFragmentationMB((Gens)gen);
                var surv = gc.SurvivalPercent((Gens)gen);

Do you need me to execute it and get details on what gc looks like? It doesn't appear these objects/methods are proper arrays but rather methods which, in turn, index into an array.

Yes, a stack trace with the TraceEvent bits included would be nice.

[brandonh-msft]

does that shed any light on things?

[cshung]

It looks like you have a GenSizeBeforeMB array that you could access, and the dimension is 4, which is probably the number you are looking for. What would happen if you replace Gen.Gen0After by the length of that array?

Edit: Scratch that, actually the array has a hard-coded size, and much of the code in TraceGC hard coded Gen3 - those feel like bugs to me.

[brandonh-msft]

Yeah, no good.

any other ideas?

[cshung]

This is not going to be very productive to chat here once a day or two, would you mind finding me on teams?

[brandonh-msft]