Redo pubternalyzer with Roslyn Operations #21996

roji · 2020-08-08T23:31:11Z

Rewrite most of the analyzer using Operations, for much better perf and VB.NET support.
Support some extra language scenarios which weren't being detected.

Fixes #19648
Fixes #20206

* Rewrite most of the analyzer using Operations, for much better perf and VB.NET support. * Support some extra language scenarios which weren't being detected. Fixes #19648 Fixes #20206

roji · 2020-08-08T23:42:34Z

...hapedQueryCompilingExpressionVisitor.CosmosProjectionBindingRemovingExpressionVisitorBase.cs

@@ -365,9 +365,12 @@ private BlockExpression AddIncludes(BlockExpression shaperBlock)
                var includeMethod = navigation.IsCollection ? _includeCollectionMethodInfo : _includeReferenceMethodInfo;
                var includingClrType = navigation.DeclaringEntityType.ClrType;
                var relatedEntityClrType = navigation.TargetEntityType.ClrType;
+#pragma warning disable EF1001 // Internal EF Core API usage.


I think you already know about these internal usages @AndriySvyryd, but just in case.

roji · 2020-08-08T23:43:04Z

src/EFCore.Relational/Migrations/Internal/MigrationsModelDiffer.cs

@@ -88,7 +88,9 @@ public class MigrationsModelDiffer : IMigrationsModelDiffer
        {
            Check.NotNull(typeMappingSource, nameof(typeMappingSource));
            Check.NotNull(migrationsAnnotations, nameof(migrationsAnnotations));
+#pragma warning disable EF1001 // Internal EF Core API usage.


/cc @bricelam

I'll defer to @AndriySvyryd on these since they're here for seed data

Filed #22063
Let's just say that the current approach is not as robust as it should be

roji · 2020-08-09T05:25:01Z

/cc @mvasani and @sharwell in case you're interested in taking a look

As per https://github.com/dotnet/roslyn-analyzers/blob/master/src/Microsoft.CodeAnalysis.Analyzers/ReleaseTrackingAnalyzers.Help.md

* Field, property and event declarations * Implemented interface

roji · 2020-08-11T13:39:00Z

@sharwell and @mavasani would it be possible to get a quick review of this analyzer after these changes are done? I'm particularly interested in any perf issues which stand out (this PR resolves some severe problems from before), but also interested in any general feedback.

BTW this takes a dependency on the recently-released 3.7.0 packages, does that have any possible bad consequences? Any idea which minimum VS version is required by this?

@AndriySvyryd I've done some considerable improvements after your review, you may want to check it out again.

roji · 2020-08-11T13:42:06Z

src/EFCore.Relational/Migrations/Internal/MigrationsModelDiffer.cs

@@ -137,7 +139,9 @@ public class MigrationsModelDiffer : IMigrationsModelDiffer
        ///     any release. You should only use it directly in your code with extreme caution and knowing that
        ///     doing so can result in application failures when updating to a new Entity Framework Core release.
        /// </summary>
+#pragma warning disable EF1001 // Internal EF Core API usage.


@bricelam same

src/EFCore.Analyzers/InternalUsageDiagnosticAnalyzer.cs

mavasani · 2020-08-13T11:58:31Z

src/EFCore.Analyzers/InternalUsageDiagnosticAnalyzer.cs

+                CSharpSyntax.InvocationExpressionSyntax s
+                    when s.Expression is CSharpSyntax.MemberAccessExpressionSyntax memberAccessSyntax
+                    => memberAccessSyntax.Name,
+                CSharpSyntax.MemberAccessExpressionSyntax s => s.Name,
+                CSharpSyntax.ObjectCreationExpressionSyntax s => s.Type,
+                CSharpSyntax.PropertyDeclarationSyntax s => s.Type,
+                CSharpSyntax.VariableDeclaratorSyntax declarator
+                    => declarator.Parent is CSharpSyntax.VariableDeclarationSyntax declaration
+                        ? declaration.Type
+                        : (SyntaxNode)declarator,
+                CSharpSyntax.TypeOfExpressionSyntax s => s.Type,
+
+                VBSyntax.InvocationExpressionSyntax s
+                    when s.Expression is VBSyntax.MemberAccessExpressionSyntax memberAccessSyntax
+                    => memberAccessSyntax.Name,
+                VBSyntax.MemberAccessExpressionSyntax s => s.Name,
+                VBSyntax.ObjectCreationExpressionSyntax s => s.Type,
+                VBSyntax.TypeOfExpressionSyntax s => s.Type,


It seems it would be better if you have different language specific analyzer assemblies for language specific code. Otherwise, your analyzer will cause a C# only solution to load all VB Roslyn assemblies and vice versa, which has an unnecessary assembly load overhead.

That's a good point - even if the assembly load overhead would only apply at build time...

The problem is that our analyzers package - Microsoft.EntityFrameworkCore.Analyzers - isn't typically referenced directly by user, but is brought in transitively as a dependency of Microsoft.EntityFrameworkCore. I know that ASP.NET also have a similar pattern, do you have any guidance on this, or any particular reason why the additional VB assembly load should be avoided?

If this is a big issue, we can drop the VB dependency altogether - it's only used to produce better/narrower diagnostic locations - the warnings themselves would still appear as we're using language-independent APIs here.

Normally the way this would be done is that your analyzer package will have 3 assemblies - Core, C# specific and VB specific. Then the MSBuild/SDK machinery will ensure that C# projects have analyzer references to first two and VB projects to the first and third. This way if someone has a C# only solution or a VB only solution, they do not get penalized by loading non-required language Roslyn assemblies (which can be quite a few as most of Roslyn layers are divided the same way). @jmarolf can probably help with setting up/guiding on the SDK pieces.

Oh nice, was not aware of this... Any pointer to an analyzer package which does this would be sufficient!

@jmarolf did this work for the Microsoft.CodeAnalysis.NetAnalyzers nuget package that we now insert into the .NET SDK. He can give pointers on how you can match his apporoach.

Scoping this out to #22085

Otherwise, your analyzer will cause a C# only solution to load all VB Roslyn assemblies and vice versa, which has an unnecessary assembly load overhead.

Note that it's slightly worse than this implies. Csc does not provide the VB assemblies, and vbc does not provide the C# assemblies. Command line use of an analyzer that references both will work if and only if the shared compiler server VBCSCompiler.exe is used, but that will occasionally fail to process a request and silently fall back to the language-specific executables, at which point the analyzers will silently fail to run.

src/EFCore.Analyzers/InternalUsageDiagnosticAnalyzer.cs

mavasani · 2020-08-13T12:01:52Z

src/EFCore.Analyzers/InternalUsageDiagnosticAnalyzer.cs

+        private static void AnalyzeInvocation(OperationAnalysisContext context, IInvocationOperation invocation)
+        {
+            // First check for any internal type parameters
+            foreach (var a in invocation.TargetMethod.TypeArguments)


Do we do similar analysis for generic type arguments of named types?

Not at the moment. Do you have guidance on how to get the generic type arguments of an ITypeSymbol?

Scoping out to #22086, any guidance on this would be appreciated.

src/EFCore.SqlServer/Metadata/Internal/SqlServerAnnotationProvider.cs

roji · 2020-08-16T10:12:39Z

This seems in a good enough state to merge, I've scoped out some possible additional work to external issues.

Youssef1313 · 2021-02-11T13:42:21Z

@roji, There was a performance complaint for this analyzer. See https://github.com/dotnet/core/issues/5672#issuecomment-776802704.

roji · 2021-02-11T17:02:48Z

@Youssef1313 thanks for pinging me, I was unaware of this. Let's wait to see what Jared's suggestion does - it's very likely to be the issue indeed.

Redo pubternalyzer with Roslyn Operations

cf3db52

* Rewrite most of the analyzer using Operations, for much better perf and VB.NET support. * Support some extra language scenarios which weren't being detected. Fixes #19648 Fixes #20206

roji force-pushed the AnalyzerStuff branch from 9314eb8 to cf3db52 Compare August 8, 2020 23:41

roji commented Aug 8, 2020

View reviewed changes

roji requested a review from a team August 9, 2020 05:25

AndriySvyryd approved these changes Aug 11, 2020

View reviewed changes

roji added 6 commits August 11, 2020 15:25

Detect subclassing via a symbol action instead of via syntax

8488397

Depend on Microsoft.CodeAnalysis.CSharp.Workspaces 3.7.0

f44a136

Fix symbol comparison method

28d090c

Add analyzer release tracking

e8dd3f5

As per https://github.com/dotnet/roslyn-analyzers/blob/master/src/Microsoft.CodeAnalysis.Analyzers/ReleaseTrackingAnalyzers.Help.md

Correct behavior for VB

80a452e

Implement some additional missing cases

b730e30

* Field, property and event declarations * Implemented interface

roji requested a review from dougbu as a code owner August 11, 2020 13:36

Suppress one more new warning

3514b51

roji commented Aug 11, 2020

View reviewed changes

smitpatel requested a review from AndriySvyryd August 11, 2020 16:18

Align Microsoft.CodeAnalysis versions across the solution

b27e8a7

AndriySvyryd reviewed Aug 11, 2020

View reviewed changes

src/EFCore.Analyzers/InternalUsageDiagnosticAnalyzer.cs Outdated Show resolved Hide resolved

AndriySvyryd reviewed Aug 11, 2020

View reviewed changes

src/EFCore.Analyzers/InternalUsageDiagnosticAnalyzer.cs Outdated Show resolved Hide resolved

AndriySvyryd approved these changes Aug 11, 2020

View reviewed changes

Address review comments

2fcd169