Add SPGO support and MIBC comparison in dotnet-pgo (#52765)

This allows dotnet-pgo to generate .mibc files using the sample data
stored in the trace that it is processing. It implements support for
both last branch record (LBR) data and normal IP samples. The latter can
be produced using PerfView as normal while the former currently requires
using xperf with LBR mode enabled. For posterity, to enable both logging
required .NET events and LBR, the following commands can be used (on
Windows):

```
xperf.exe -start "NT Kernel Logger" -on LOADER+PROC_THREAD+PMC_PROFILE -MinBuffers 4096 -MaxBuffers 4096 -BufferSize 4096 -pmcprofile BranchInstructionRetired -LastBranch PmcInterrupt -setProfInt BranchInstructionRetired 65537 -start clr -on e13c0d23-ccbc-4e12-931b-d9cc2eee27e4:0x40000A0018:0x5 -MinBuffers 4096 -MaxBuffers 4096 -BufferSize 4096
scenario.exe
xperf.exe -stop "NT Kernel Logger" -stop clr -d xperftrace.etl
```

SPGO does not currently do well with optimized code as the mapping
IP<->IL mappings the JIT produces there are not sufficiently accurate.
To collect data in tier-0 one can enable two environment variables
before running the scenario:
```
$env:COMPlus_TC_QuickJitForLoops=1
$env:COMPlus_TC_CallCounting=0
```

When samples are used the associated counts will not typically look
valid, i.e. they won't satisfy flow conservation. To remedy this,
dotnet-pgo performs a smoothing step after assigning samples to the
flow-graph of each method. The smoothing is based on [1] and the code
comes from Midori.

The commit adds some new commands to dotnet-pgo. The --spgo flag can be
specified to create-mibc to use samples to create the .mibc file. Also,
even if --spgo is specified, instrumented data will still be preferred
if available in the trace. If spgo is not specified, the behavior should
be the same as before.

--spgo-with-block-counts and --spgo-with-edge-counts control whether
dotnet-pgo outputs the smoothed block or edge counts (or both). By
default block counts are output. The JIT can use both forms of counts
but will be most happy if only one kind is present for each method.

--spgo-min-samples controls how many samples must be in each method
before smoothing is applied and the result included in the .mibc. SPGO
is quite sensitive to low sample counts and the produced results are not
good when the number of samples is low. By default, this value is 50.

The commit also adds a new compare-mibc command that allows to compare
two .mibc files. Usage is dotnet-pgo compare-mibc --input file1.mibc
--input file2.mibc. For example, comparing a .mibc produced via
instrumentation and one produced via SPGO (in tier-0) for some JIT
benchmarks produces the following:

```
Comparing instrumented.mibc to spgo.mibc
Statistics for instrumented.mibc
# Methods: 3490
# Methods with any profile data: 865
# Methods with 32-bit block counts: 0
# Methods with 64-bit block counts: 865
# Methods with 32-bit edge counts: 0
# Methods with 64-bit edge counts: 0
# Methods with type handle histograms: 184
# Methods with GetLikelyClass data: 0
# Profiled methods in instrumented.mibc not in spgo.mibc: 652

Statistics for spgo.mibc
# Methods: 1107
# Methods with any profile data: 286
# Methods with 32-bit block counts: 286
# Methods with 64-bit block counts: 0
# Methods with 32-bit edge counts: 0
# Methods with 64-bit edge counts: 0
# Methods with type handle histograms: 0
# Methods with GetLikelyClass data: 0
# Profiled methods in spgo.mibc not in instrumented.mibc: 73

Comparison
# Methods with profile data in both .mibc files: 213
  Of these, 213 have matching flow-graphs and the remaining 0 do not

When comparing the flow-graphs of the matching methods, their overlaps break down as follows:
100% █ (1.9%)
>95% █████████████████████████████████▌ (61.0%)
>90% ████████ (14.6%)
>85% ████▏ (7.5%)
>80% ████▋ (8.5%)
>75% █▊ (3.3%)
>70% █ (1.9%)
>65% ▎ (0.5%)
>60% ▎ (0.5%)
>55% ▏ (0.0%)
>50% ▏ (0.0%)
>45% ▏ (0.0%)
>40% ▎ (0.5%)
>35% ▏ (0.0%)
>30% ▏ (0.0%)
>25% ▏ (0.0%)
>20% ▏ (0.0%)
>15% ▏ (0.0%)
>10% ▏ (0.0%)
> 5% ▏ (0.0%)
> 0% ▏ (0.0%)
(using block counts)
```

I also made the dump command print some statistics about the .mibc that
was dumped. Hopefully some of this tooling can help track down #51908.

[1] Levin R., Newman I., Haber G. (2008) Complementing Missing and
Inaccurate Profiling Using a Minimum Cost Circulation Algorithm. In:
Stenström P., Dubois M., Katevenis M., Gupta R., Ungerer T. (eds) High
Performance Embedded Architectures and Compilers. HiPEAC 2008. Lecture
Notes in Computer Science, vol 4917. Springer, Berlin, Heidelberg.
https://doi.org/10.1007/978-3-540-77560-7_20

src/coreclr/tools/Common/Pgo/PgoFormat.cs

@@ -562,7 +562,9 @@ void MergeInSchemaElem(Dictionary<PgoSchemaElem, PgoSchemaElem> dataMerger, PgoS
                     switch (existingSchemaItem.InstrumentationKind)
                     {
                         case PgoInstrumentationKind.BasicBlockIntCount:
+                        case PgoInstrumentationKind.BasicBlockLongCount:
                         case PgoInstrumentationKind.EdgeIntCount:
+                        case PgoInstrumentationKind.EdgeLongCount:
                         case PgoInstrumentationKind.TypeHandleHistogramCount:
                             if ((existingSchemaItem.Count != 1) || (schema.Count != 1))
                             {

src/coreclr/tools/Common/TypeSystem/IL/FlowGraph.cs

@@ -0,0 +1,273 @@
+// Licensed to the .NET Foundation under one or more agreements.
+// The .NET Foundation licenses this file to you under the MIT license.
+
+using System;
+using System.Collections;
+using System.Collections.Generic;
+using System.Diagnostics;
+using System.Linq;
+using System.Text;
+using System.Threading.Tasks;
+
+namespace Internal.IL
+{
+    internal class BasicBlock : IEquatable<BasicBlock>
+    {
+        public BasicBlock(int start, int size)
+            => (Start, Size) = (start, size);
+
+        // First IL offset
+        public int Start { get; }
+        // Number of IL bytes in this basic block
+        public int Size { get; }
+
+        public HashSet<BasicBlock> Sources { get; } = new HashSet<BasicBlock>();
+        public HashSet<BasicBlock> Targets { get; } = new HashSet<BasicBlock>();
+
+        public override string ToString() => $"Start={Start}, Size={Size}";
+
+        public override bool Equals(object obj) => Equals(obj as BasicBlock);
+        public bool Equals(BasicBlock other) => other != null && Start == other.Start;
+        public override int GetHashCode() => HashCode.Combine(Start);
+
+        public static bool operator ==(BasicBlock left, BasicBlock right) => EqualityComparer<BasicBlock>.Default.Equals(left, right);
+        public static bool operator !=(BasicBlock left, BasicBlock right) => !(left == right);
+    }
+
+    internal class FlowGraph
+    {
+        private readonly int[] _bbKeys;
+
+        private FlowGraph(IEnumerable<BasicBlock> bbs)
+        {
+            BasicBlocks = bbs.OrderBy(bb => bb.Start).ToList();
+            _bbKeys = BasicBlocks.Select(bb => bb.Start).ToArray();
+        }
+
+        /// <summary>Basic blocks, ordered by start IL offset.</summary>
+        public List<BasicBlock> BasicBlocks { get; }
+
+        /// <summary>Find index of basic block containing IL offset.</summary>
+        public int LookupIndex(int ilOffset)
+        {
+            int index = Array.BinarySearch(_bbKeys, ilOffset);
+            if (index < 0)
+                index = ~index - 1;
+
+            // If ilOffset is negative (more generally, before the first BB)
+            // then binarySearch will return ~0 since index 0 is the first BB
+            // that's greater.
+            if (index < 0)
+                return -1;
+
+            // If this is the last BB we could be after as well.
+            BasicBlock bb = BasicBlocks[index];
+            if (ilOffset >= bb.Start + bb.Size)
+                return -1;
+
+            return index;
+        }
+
+        public BasicBlock Lookup(int ilOffset)
+            => LookupIndex(ilOffset) switch
+            {
+                -1 => null,
+                int idx => BasicBlocks[idx]
+            };
+
+        public IEnumerable<BasicBlock> LookupRange(int ilOffsetStart, int ilOffsetEnd)
+        {
+            if (ilOffsetStart < BasicBlocks[0].Start)
+                ilOffsetStart = BasicBlocks[0].Start;
+
+            if (ilOffsetEnd > BasicBlocks.Last().Start)
+                ilOffsetEnd = BasicBlocks.Last().Start;
+
+            int end = LookupIndex(ilOffsetEnd);
+            for (int i = LookupIndex(ilOffsetStart); i <= end; i++)
+                yield return BasicBlocks[i];
+        }
+
+        internal string Dump(Func<BasicBlock, string> getNodeAnnot, Func<(BasicBlock, BasicBlock), string> getEdgeAnnot)
+        {
+            var sb = new StringBuilder();
+            sb.AppendLine("digraph G {");
+            sb.AppendLine("  forcelabels=true;");
+            sb.AppendLine();
+            Dictionary<long, int> bbToIndex = new Dictionary<long, int>();
+            for (int i = 0; i < BasicBlocks.Count; i++)
+                bbToIndex.Add(BasicBlocks[i].Start, i);
+
+            foreach (BasicBlock bb in BasicBlocks)
+            {
+                string label = $"[{bb.Start:x}..{bb.Start + bb.Size:x})\\n{getNodeAnnot(bb)}";
+                sb.AppendLine($"  BB{bbToIndex[bb.Start]} [label=\"{label}\"];");
+            }
+
+            sb.AppendLine();
+
+            foreach (BasicBlock bb in BasicBlocks)
+            {
+                foreach (BasicBlock tar in bb.Targets)
+                {
+                    string label = getEdgeAnnot((bb, tar));
+                    string postfix = string.IsNullOrEmpty(label) ? "" : $" [label=\"{label}\"]";
+                    sb.AppendLine($"  BB{bbToIndex[bb.Start]} -> BB{bbToIndex[tar.Start]}{postfix};");
+                }
+            }
+
+            // Write ranks with BFS.
+            List<BasicBlock> curRank = new List<BasicBlock> { BasicBlocks.Single(bb => bb.Start == 0) };
+            HashSet<BasicBlock> seen = new HashSet<BasicBlock>(curRank);
+            while (curRank.Count > 0)
+            {
+                sb.AppendLine($"  {{rank = same; {string.Concat(curRank.Select(bb => $"BB{bbToIndex[bb.Start]}; "))}}}");
+                curRank = curRank.SelectMany(bb => bb.Targets).Where(seen.Add).ToList();
+            }
+
+            sb.AppendLine("}");
+            return sb.ToString();
+        }
+
+        public static FlowGraph Create(MethodIL il)
+        {
+            HashSet<int> bbStarts = GetBasicBlockStarts(il);
+
+            List<BasicBlock> bbs = new List<BasicBlock>();
+            void AddBB(int start, int count)
+            {
+                if (count > 0)
+                    bbs.Add(new BasicBlock(start, count));
+            }
+
+            int prevStart = 0;
+            foreach (int ofs in bbStarts.OrderBy(o => o))
+            {
+                AddBB(prevStart, ofs - prevStart);
+                prevStart = ofs;
+            }
+
+            AddBB(prevStart, il.GetILBytes().Length - prevStart);
+
+            FlowGraph fg = new FlowGraph(bbs);
+
+            // We know where each basic block starts now. Proceed by linking them together.
+            ILReader reader = new ILReader(il.GetILBytes());
+            foreach (BasicBlock bb in bbs)
+            {
+                reader.Seek(bb.Start);
+                while (reader.HasNext)
+                {
+                    Debug.Assert(fg.Lookup(reader.Offset) == bb);
+                    ILOpcode opc = reader.ReadILOpcode();
+                    if (opc.IsBranch())
+                    {
+                        int tar = reader.ReadBranchDestination(opc);
+                        bb.Targets.Add(fg.Lookup(tar));
+                        if (!opc.IsUnconditionalBranch())
+                            bb.Targets.Add(fg.Lookup(reader.Offset));
+
+                        break;
+                    }
+
+                    if (opc == ILOpcode.switch_)
+                    {
+                        uint numCases = reader.ReadILUInt32();
+                        int jmpBase = reader.Offset + checked((int)(numCases * 4));
+                        bb.Targets.Add(fg.Lookup(jmpBase));
+
+                        for (uint i = 0; i < numCases; i++)
+                        {
+                            int caseOfs = jmpBase + (int)reader.ReadILUInt32();
+                            bb.Targets.Add(fg.Lookup(caseOfs));
+                        }
+
+                        break;
+                    }
+
+                    if (opc == ILOpcode.ret || opc == ILOpcode.endfinally || opc == ILOpcode.endfilter || opc == ILOpcode.throw_ || opc == ILOpcode.rethrow)
+                    {
+                        break;
+                    }
+
+                    reader.Skip(opc);
+                    // Check fall through
+                    if (reader.HasNext)
+                    {
+                        BasicBlock nextBB = fg.Lookup(reader.Offset);
+                        if (nextBB != bb)
+                        {
+                            // Falling through
+                            bb.Targets.Add(nextBB);
+                            break;
+                        }
+                    }
+                }
+            }
+
+            foreach (BasicBlock bb in bbs)
+            {
+                foreach (BasicBlock tar in bb.Targets)
+                    tar.Sources.Add(bb);
+            }
+
+            return fg;
+        }
+
+        /// <summary>
+        /// Find IL offsets at which basic blocks begin.
+        /// </summary>
+        private static HashSet<int> GetBasicBlockStarts(MethodIL il)
+        {
+            ILReader reader = new ILReader(il.GetILBytes());
+            HashSet<int> bbStarts = new HashSet<int>();
+            bbStarts.Add(0);
+            while (reader.HasNext)
+            {
+                ILOpcode opc = reader.ReadILOpcode();
+                if (opc.IsBranch())
+                {
+                    int tar = reader.ReadBranchDestination(opc);
+                    bbStarts.Add(tar);
+                    // Conditional branches can fall through.
+                    if (!opc.IsUnconditionalBranch())
+                        bbStarts.Add(reader.Offset);
+                }
+                else if (opc == ILOpcode.switch_)
+                {
+                    uint numCases = reader.ReadILUInt32();
+                    int jmpBase = reader.Offset + checked((int)(numCases * 4));
+                    // Default case is at jmpBase.
+                    bbStarts.Add(jmpBase);
+
+                    for (uint i = 0; i < numCases; i++)
+                    {
+                        int caseOfs = jmpBase + (int)reader.ReadILUInt32();
+                        bbStarts.Add(caseOfs);
+                    }
+                }
+                else if (opc == ILOpcode.ret || opc == ILOpcode.endfinally || opc == ILOpcode.endfilter || opc == ILOpcode.throw_ || opc == ILOpcode.rethrow)
+                {
+                    if (reader.HasNext)
+                        bbStarts.Add(reader.Offset);
+                }
+                else
+                {
+                    reader.Skip(opc);
+                }
+            }
+
+            foreach (ILExceptionRegion ehRegion in il.GetExceptionRegions())
+            {
+                bbStarts.Add(ehRegion.TryOffset);
+                bbStarts.Add(ehRegion.TryOffset + ehRegion.TryLength);
+                bbStarts.Add(ehRegion.HandlerOffset);
+                bbStarts.Add(ehRegion.HandlerOffset + ehRegion.HandlerLength);
+                if (ehRegion.Kind.HasFlag(ILExceptionRegionKind.Filter))
+                    bbStarts.Add(ehRegion.FilterOffset);
+            }
+
+            return bbStarts;
+        }
+    }
+}

src/coreclr/tools/Common/TypeSystem/IL/ILOpcodeHelper.cs

@@ -50,6 +50,11 @@ public static bool IsBranch(this ILOpcode opcode)
             return false;
         }
 
+        public static bool IsUnconditionalBranch(this ILOpcode opcode)
+        {
+            return opcode == ILOpcode.br || opcode == ILOpcode.br_s || opcode == ILOpcode.leave || opcode == ILOpcode.leave_s;
+        }
+
         private static readonly byte[] s_opcodeSizes = new byte[]
         {
             1, // nop = 0x00,

src/coreclr/tools/dotnet-pgo/CommandLineOptions.cs

@@ -28,6 +28,10 @@ internal class CommandLineOptions
         public bool DisplayProcessedEvents;
         public bool ValidateOutputFile;
         public bool GenerateCallGraph;
+        public bool Spgo;
+        public bool SpgoIncludeBlockCounts;
+        public bool SpgoIncludeEdgeCounts;
+        public int SpgoMinSamples = 50;
         public bool VerboseWarnings;
         public jittraceoptions JitTraceOptions;
         public double ExcludeEventsBefore;
@@ -40,6 +44,7 @@ internal class CommandLineOptions
         public List<AssemblyName> IncludedAssemblies = new List<AssemblyName>();
         public bool DumpMibc = false;
         public FileInfo InputFileToDump;
+        public List<FileInfo> CompareMibc;
 
         public string[] HelpArgs = Array.Empty<string>();
 
@@ -189,6 +194,15 @@ void HelpOption()
 #endif
                 CommonOptions();
                 CompressedOption();
+
+                syntax.DefineOption(name: "spgo", value: ref Spgo, help: "Base profile on samples in the input. Uses last branch records if available and otherwise raw IP samples.", requireValue: false);
+                syntax.DefineOption(name: "spgo-with-block-counts", value: ref SpgoIncludeBlockCounts, help: "Include block counts in the written .mibc file. If neither this nor spgo-with-edge-counts are specified, then defaults to true.", requireValue: false);
+                syntax.DefineOption(name: "spgo-with-edge-counts", value: ref SpgoIncludeEdgeCounts, help: "Include edge counts in the written .mibc file.", requireValue: false);
+                syntax.DefineOption(name: "spgo-min-samples", value: ref SpgoMinSamples, help: $"The minimum number of total samples a function must have before generating profile data for it with SPGO. Default: {SpgoMinSamples}", requireValue: false);
+
+                if (!SpgoIncludeBlockCounts && !SpgoIncludeEdgeCounts)
+                    SpgoIncludeBlockCounts = true;
+
                 HelpOption();
             }
 
@@ -231,7 +245,7 @@ void HelpOption()
             {
                 HelpArgs = new string[] { "merge", "--help", "--output", "output", "--input", "input"};
 
-                InputFilesToMerge = DefineFileOptionList(name: "i|input", help: "If a reference is not located on disk at the same location as used in the process, it may be specified with a --reference parameter. Multiple --reference parameters may be specified. The wild cards * and ? are supported by this option.");
+                InputFilesToMerge = DefineFileOptionList(name: "i|input", help: "Input .mibc files to be merged. Multiple input arguments are specified as --input file1.mibc --input file2.mibc");
                 OutputOption();
 
                 IReadOnlyList<string> assemblyNamesAsStrings = null;
@@ -281,6 +295,14 @@ void HelpOption()
                     OutputFileName = new FileInfo(outputFile);
             }
 
+            var compareMibcCommand = syntax.DefineCommand(name: "compare-mibc", value: ref command, help: "Compare two .mibc files");
+            if (compareMibcCommand.IsActive)
+            {
+                HelpArgs = new[] { "compare-mibc", "--input", "first.mibc", "--input", "second.mibc" };
+                CompareMibc = DefineFileOptionList(name: "i|input", help: "The input .mibc files to be compared. Specify as --input file1.mibc --input file2.mibc");
+                if (CompareMibc.Count != 2)
+                    Help = true;
+            }
 
             if (syntax.ActiveCommand == null)
             {
@@ -363,7 +385,7 @@ private CommandLineOptions()
         private void ParseCommmandLineHelper(string[] args)
         {
             ArgumentSyntax argSyntax = ArgumentSyntax.Parse(args, DefineArgumentSyntax);
-            if (Help || (!FileType.HasValue && (InputFilesToMerge == null) && !DumpMibc))
+            if (Help || (!FileType.HasValue && (InputFilesToMerge == null) && !DumpMibc && CompareMibc == null))
             {
                 Help = true;
             }