Improve loading of Roslyn assemblies on validate package task #22277

safern · 2021-10-22T18:40:09Z

With: dotnet/runtime#59908 we realized various things:

We were loading the roslyn assemblies into the default Assembly Load Context because we were using Assembly.LoadFrom on .NET Core and that is a legacy API on Core which just loads the assembly to the default load context. That could lead to different problems one of them for example: if there is an MSBuild Task that runs before package validation and that task depends on Microsoft.CodeAnalysis and carries it's own copy of it with the package and it is the same assembly version, since it could be loaded into the same default Assembly Load Context, we could endup in a state where we load Microsoft.CodeAnalysis.CSharp from a different path and have miss match problems.
If the toolset package is referenced, we were only setting the RoslynAssembliesPath for projects with the .csproj or vbproj extensions, so for example in dotnet/runtime we have an .ilproj and another project with .pkgproj extension which because of this reason, ended using the roslyn from the SDK and causing this possible miss match described on point 1.
If package validation runs on a package that has no lib or ref assets, say a tools or a native package, API Compat would never be invoked so only Microsoft.CodeAnalysis would be loaded, leading to potential miss matches of Microsoft.CodeAnalysis.dll vs Microsoft.CodeAnalysis.CSharp.dll whenever we run package validation on a package on the same build where we do need to run API Compat (which is actually what was causing the issue in dotnet/runtime) even if loading the assemblies into the Task ALC rather than the default ALC. So to fix this we will defensively load both assemblies when either one of them is requested to make sure on the first instance of the task on the current MSBuild node, we load both assemblies from the same location regardless of if we need them or not into the task ALC.

What we encountered in dotnet/runtime was a combination of 2 and 3.

src/Tasks/Microsoft.NET.Build.Tasks/targets/Microsoft.NET.Compatibility.Common.targets

src/Compatibility/Microsoft.DotNet.Compatibility/ValidatePackage.cs

ericstj · 2021-10-22T21:55:31Z

src/Compatibility/Microsoft.DotNet.Compatibility/ValidatePackage.cs

+        private Assembly LoadRoslyn(AssemblyName name, Func<string, Assembly> loadFromPath)
+        {
+            const string codeAnalysisName = "Microsoft.CodeAnalysis";
+            if (name.Name.StartsWith(codeAnalysisName, StringComparison.Ordinal))


Do we care about some other random assembly that matches the Microsoft.CodeAnalysis prefix but isn't one we'd care about? It seems unlikely since we're only hooking this for the duration of the task execution so it should be only our code running in the AppDomain at the time. I think that's the case at least.

That's why I changed it to the prefix cause it seemed simple and I don't see any reason why the AppDomain nor the Task ALC (which is exclusively used by this task) would try to load another Roslyn assembly. That said though, I don't mind being more defensive and doing a strict comparison for the 2 we know we care 😊

I went the defensive side and explicitly checked for both assemblies as we had it.

ericstj

LGTM, modulo a small scenario to consider.

This fixes an issue where a non-CSProj built package would load MS.CA.dll , then a CSProj would try to load MS.CA.CS.dll of a different version and it's MS.CA types would not agree with those already bound to the task from previous load. Further details: dotnet/sdk#22277

…6144) * Update dependencies from https://github.com/dotnet/arcade build 20220222.7 Microsoft.DotNet.XUnitConsoleRunner , Microsoft.DotNet.CodeAnalysis , Microsoft.DotNet.Build.Tasks.Workloads , Microsoft.DotNet.Build.Tasks.Templating , Microsoft.DotNet.Build.Tasks.TargetFramework.Sdk , Microsoft.DotNet.Build.Tasks.Packaging , Microsoft.DotNet.Build.Tasks.Installers , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Build.Tasks.Archives , Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.ApiCompat , Microsoft.DotNet.XUnitExtensions , Microsoft.DotNet.GenAPI , Microsoft.DotNet.VersionTools.Tasks , Microsoft.DotNet.GenFacades , Microsoft.DotNet.SharedFramework.Sdk , Microsoft.DotNet.RemoteExecutor , Microsoft.DotNet.PackageTesting , Microsoft.DotNet.Helix.Sdk From Version 2.5.1-beta.22107.2 -> To Version 2.5.1-beta.22122.7 * Update dependencies from https://github.com/dotnet/emsdk build 20220308.1 Microsoft.NET.Workload.Emscripten.Manifest-6.0.100 From Version 6.0.2 -> To Version 6.0.3 * Fix weird character problem * Update dependencies from https://github.com/dotnet/emsdk build 20220308.2 Microsoft.NET.Workload.Emscripten.Manifest-6.0.100 From Version 6.0.2 -> To Version 6.0.4 * Update dependencies from https://github.com/dotnet/arcade build 20220309.8 Microsoft.DotNet.XUnitConsoleRunner , Microsoft.DotNet.CodeAnalysis , Microsoft.DotNet.Build.Tasks.Workloads , Microsoft.DotNet.Build.Tasks.Templating , Microsoft.DotNet.Build.Tasks.TargetFramework.Sdk , Microsoft.DotNet.Build.Tasks.Packaging , Microsoft.DotNet.Build.Tasks.Installers , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Build.Tasks.Archives , Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.ApiCompat , Microsoft.DotNet.XUnitExtensions , Microsoft.DotNet.GenAPI , Microsoft.DotNet.VersionTools.Tasks , Microsoft.DotNet.GenFacades , Microsoft.DotNet.SharedFramework.Sdk , Microsoft.DotNet.RemoteExecutor , Microsoft.DotNet.PackageTesting , Microsoft.DotNet.Helix.Sdk From Version 2.5.1-beta.22107.2 -> To Version 2.5.1-beta.22159.8 * On run package validation for CSProj-built packages This fixes an issue where a non-CSProj built package would load MS.CA.dll , then a CSProj would try to load MS.CA.CS.dll of a different version and it's MS.CA types would not agree with those already bound to the task from previous load. Further details: dotnet/sdk#22277 * Update dependencies from https://github.com/dotnet/emsdk build 20220311.2 Microsoft.NET.Workload.Emscripten.Manifest-6.0.100 , Microsoft.NET.Workload.Emscripten.Manifest-6.0.200 , Microsoft.NET.Workload.Emscripten.Manifest-6.0.300 From Version 6.0.4 -> To Version 6.0.4 * Update dependencies from https://github.com/dotnet/arcade build 20220311.1 Microsoft.DotNet.XUnitConsoleRunner , Microsoft.DotNet.CodeAnalysis , Microsoft.DotNet.Build.Tasks.Workloads , Microsoft.DotNet.Build.Tasks.Templating , Microsoft.DotNet.Build.Tasks.TargetFramework.Sdk , Microsoft.DotNet.Build.Tasks.Packaging , Microsoft.DotNet.Build.Tasks.Installers , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Build.Tasks.Archives , Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.ApiCompat , Microsoft.DotNet.XUnitExtensions , Microsoft.DotNet.GenAPI , Microsoft.DotNet.VersionTools.Tasks , Microsoft.DotNet.GenFacades , Microsoft.DotNet.SharedFramework.Sdk , Microsoft.DotNet.RemoteExecutor , Microsoft.DotNet.PackageTesting , Microsoft.DotNet.Helix.Sdk From Version 2.5.1-beta.22107.2 -> To Version 2.5.1-beta.22161.1 Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com> Co-authored-by: Steve Pfister <steve.pfister@microsoft.com> Co-authored-by: Alexander Köplinger <alex.koeplinger@outlook.com> Co-authored-by: Eric StJohn <ericstj@microsoft.com>

* Improve loading of Roslyn assemblies on validate package task * PR Feedback

Improve loading of Roslyn assemblies on validate package task

025ce21

safern requested review from ericstj, Anipik and joperezr October 22, 2021 18:40

dotnet-issue-labeler bot added the Area-ApiCompat label Oct 22, 2021

safern mentioned this pull request Oct 22, 2021

'The "Microsoft.DotNet.Compatibility.ValidatePackage" task failed unexpectedly' in libs legs dotnet/runtime#59908

Closed

Anipik reviewed Oct 22, 2021

View reviewed changes

src/Tasks/Microsoft.NET.Build.Tasks/targets/Microsoft.NET.Compatibility.Common.targets Show resolved Hide resolved

Anipik approved these changes Oct 22, 2021

View reviewed changes

ericstj reviewed Oct 22, 2021

View reviewed changes

joperezr reviewed Oct 22, 2021

View reviewed changes

src/Compatibility/Microsoft.DotNet.Compatibility/ValidatePackage.cs Show resolved Hide resolved

joperezr approved these changes Oct 22, 2021

View reviewed changes

safern enabled auto-merge (squash) October 22, 2021 21:22

ericstj reviewed Oct 22, 2021

View reviewed changes

ericstj approved these changes Oct 22, 2021

View reviewed changes

safern disabled auto-merge October 22, 2021 22:33

PR Feedback

4ba8e3e

safern force-pushed the ImproveRoslynLoadingPV branch from a9ddf5e to 4ba8e3e Compare October 22, 2021 22:41

safern enabled auto-merge (squash) October 22, 2021 22:46

safern merged commit 76217c7 into dotnet:main Oct 23, 2021

safern mentioned this pull request Oct 26, 2021

6.0 dotnet/runtime: Microsoft.DotNet.Compatibility.ValidatePackage fails, "System.MissingMethodException: Method not found: '...'" dotnet/source-build#2501

Open

safern deleted the ImproveRoslynLoadingPV branch October 26, 2021 17:59

This was referenced Oct 26, 2021

[release/6.0] Disable package validation in source-build for reliability dotnet/runtime#60881

Merged

Port PackageValidation flaky build failure fixes to 6.0 #22353

Closed

safern mentioned this pull request Nov 16, 2021

[release/6.0] Disable native ports package validation dotnet/runtime#61697

Merged

lbussell mentioned this pull request Jan 11, 2022

Remove runtime patch that disabled package validation dotnet/installer#13000

Merged

joperezr mentioned this pull request Mar 11, 2022

Improve loading of Roslyn assemblies on validate package task (#22277) #24339

Merged

joperezr pushed a commit that referenced this pull request Mar 14, 2022

Improve loading of Roslyn assemblies on validate package task (#22277)

964e9e5

* Improve loading of Roslyn assemblies on validate package task * PR Feedback

benvillalobos mentioned this pull request Mar 23, 2022

Detect arm64 hostfxr.dll #24512

Merged

ericstj mentioned this pull request May 17, 2023

[main] Update dependencies from dotnet/roslyn dotnet/msbuild#8674

Merged

This was referenced Dec 13, 2023

Configuration Schema generator dotnet/aspire#1383

Merged

Investigate loading ReadyToRun version of Roslyn in ConfigurationSchemaGenerator dotnet/aspire#1426

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve loading of Roslyn assemblies on validate package task #22277

Improve loading of Roslyn assemblies on validate package task #22277

safern commented Oct 22, 2021

ericstj Oct 22, 2021

safern Oct 22, 2021

safern Oct 22, 2021

ericstj left a comment

Improve loading of Roslyn assemblies on validate package task #22277

Improve loading of Roslyn assemblies on validate package task #22277

Conversation

safern commented Oct 22, 2021

ericstj Oct 22, 2021

Choose a reason for hiding this comment

safern Oct 22, 2021

Choose a reason for hiding this comment

safern Oct 22, 2021

Choose a reason for hiding this comment

ericstj left a comment

Choose a reason for hiding this comment