[8.0-rc.1] 'dbcontext optimize' does not preserve `convertsNulls` #31942

schuettecarsten · 2023-10-03T12:40:29Z

I upgraded my code from 7.0.11 to 8.0.0-rc.1 just to test compatibility. When creating an pre-compiled model using dotnet tool run dotnet-ef -- dbcontext optimize ...`, I get the following exception. It works when using 7.0.11 libraries.

System.NotSupportedException: Encountered a constant of unsupported type '<>c__DisplayClass2_0'. Only primitive constant nodes are supported.
   at Microsoft.EntityFrameworkCore.Query.Internal.LinqToCSharpSyntaxTranslator.<VisitConstant>g__GenerateValue|36_0(Object value, <>c__DisplayClass36_0&)
   at Microsoft.EntityFrameworkCore.Query.Internal.LinqToCSharpSyntaxTranslator.VisitConstant(ConstantExpression constant)
   at Microsoft.EntityFrameworkCore.Query.Internal.LinqToCSharpSyntaxTranslator.Translate[T](Expression node)
   at Microsoft.EntityFrameworkCore.Query.Internal.LinqToCSharpSyntaxTranslator.VisitMember(MemberExpression member)
   at Microsoft.EntityFrameworkCore.Query.Internal.LinqToCSharpSyntaxTranslator.Translate[T](Expression node)
   at Microsoft.EntityFrameworkCore.Query.Internal.LinqToCSharpSyntaxTranslator.TranslateList(IReadOnlyList`1 list)
   at Microsoft.EntityFrameworkCore.Query.Internal.LinqToCSharpSyntaxTranslator.TranslateMethodArguments(ParameterInfo[] parameters, IReadOnlyList`1 arguments)
   at Microsoft.EntityFrameworkCore.Query.Internal.LinqToCSharpSyntaxTranslator.VisitMethodCall(MethodCallExpression call)
   at Microsoft.EntityFrameworkCore.Query.Internal.LinqToCSharpSyntaxTranslator.Translate(Expression node)
   at Microsoft.EntityFrameworkCore.Query.Internal.LinqToCSharpSyntaxTranslator.VisitLambda[T](Expression`1 lambda)
   at Microsoft.EntityFrameworkCore.Query.Internal.LinqToCSharpSyntaxTranslator.TranslateCore(Expression node, ISet`1 collectedNamespaces, Boolean statementContext)
   at Microsoft.EntityFrameworkCore.Query.Internal.LinqToCSharpSyntaxTranslator.TranslateExpression(Expression node, ISet`1 collectedNamespaces)
   at Microsoft.EntityFrameworkCore.Design.Internal.CSharpHelper.Expression(Expression node, ISet`1 collectedNamespaces)
   at Microsoft.EntityFrameworkCore.Design.Internal.CSharpRuntimeAnnotationCodeGenerator.Create(ValueConverter converter, CSharpRuntimeAnnotationCodeGeneratorParameters parameters, ICSharpHelper codeHelper)
   at Microsoft.EntityFrameworkCore.Design.Internal.RelationalCSharpRuntimeAnnotationCodeGenerator.Create(CoreTypeMapping typeMapping, CSharpRuntimeAnnotationCodeGeneratorParameters parameters, ValueComparer valueComparer, ValueComparer keyValueComparer, ValueComparer providerValueComparer)
   at Microsoft.EntityFrameworkCore.Design.Internal.ICSharpRuntimeAnnotationCodeGenerator.Create(CoreTypeMapping typeMapping, IProperty property, CSharpRuntimeAnnotationCodeGeneratorParameters parameters)
   at Microsoft.EntityFrameworkCore.Scaffolding.Internal.CSharpRuntimeModelCodeGenerator.Create(IProperty property, String variableName, Dictionary`2 propertyVariables, CSharpRuntimeAnnotationCodeGeneratorParameters parameters)
   at Microsoft.EntityFrameworkCore.Scaffolding.Internal.CSharpRuntimeModelCodeGenerator.Create(IProperty property, Dictionary`2 propertyVariables, CSharpRuntimeAnnotationCodeGeneratorParameters parameters)
   at Microsoft.EntityFrameworkCore.Scaffolding.Internal.CSharpRuntimeModelCodeGenerator.CreateEntityType(IEntityType entityType, IndentedStringBuilder mainBuilder, IndentedStringBuilder methodBuilder, SortedSet`1 namespaces, String className, Boolean nullable)
   at Microsoft.EntityFrameworkCore.Scaffolding.Internal.CSharpRuntimeModelCodeGenerator.GenerateEntityType(IEntityType entityType, String namespace, String className, Boolean nullable)
   at Microsoft.EntityFrameworkCore.Scaffolding.Internal.CSharpRuntimeModelCodeGenerator.GenerateModel(IModel model, CompiledModelCodeGenerationOptions options)
   at Microsoft.EntityFrameworkCore.Scaffolding.Internal.CompiledModelScaffolder.ScaffoldModel(IModel model, String outputDir, CompiledModelCodeGenerationOptions options)
   at Microsoft.EntityFrameworkCore.Design.Internal.DbContextOperations.Optimize(String outputDir, String modelNamespace, String contextTypeName)
   at Microsoft.EntityFrameworkCore.Design.OperationExecutor.OptimizeContextImpl(String outputDir, String modelNamespace, String contextType)
   at Microsoft.EntityFrameworkCore.Design.OperationExecutor.OptimizeContext.<>c__DisplayClass0_0.<.ctor>b__0()
   at Microsoft.EntityFrameworkCore.Design.OperationExecutor.OperationBase.Execute(Action action)
Encountered a constant of unsupported type '<>c__DisplayClass2_0'. Only primitive constant nodes are supported.

I use -v option, but as no more details are shown after build completes, I cannot analyze what exactly goes wrong.

The text was updated successfully, but these errors were encountered:

ajcvickers · 2023-10-03T12:52:34Z

@schuettecarsten Please try with the latest daily build and if you still see an issue then please attach a small, runnable project or post a small, runnable code listing that reproduces what you are seeing so that we can investigate.

schuettecarsten · 2023-10-04T07:48:57Z

@ajcvickers I tried latest build (8.0.0-rtm.23503.6) and get the same error. The error seems to be caused by ValueConverter, I use several classes that are implemented like the example here: https://learn.microsoft.com/en-us/ef/core/modeling/value-conversions?tabs=data-annotations#pre-defined-conversions

Update: Root cause found. I have a custom ValueConverter that has this constructor:

	/// <summary>
	/// Initializes a new instance of the <see cref="CompressedBsonValueConverter{T}"/> class.
	/// </summary>
	public CompressedBsonValueConverter(CompressionLevel level, ConverterMappingHints? hints = default)
		: base(
			v => DoSerialize(v, level),
			v => DoDeserialize(v),
			hints)
	{
	}

This worked fine in v7 and now fails when creating the model code. I had to remove the level parameter from the DoSerialize call to make it work again. Unfortunately, the level value is lost. So, how to pass additional values to ValueConverters in v8?

schuettecarsten · 2023-10-04T09:38:31Z

@ajcvickers I also found that convertsNulls value is lost when the model code is generated.

	/// <summary>
	/// Initializes a new instance of the <see cref="CompressedBsonValueConverter{T}"/> class.
	/// </summary>
	public CompressedBsonValueConverter(CompressionLevel level, ConverterMappingHints? hints = default)
		: base(
			v => DoSerialize(v),
			v => DoDeserialize(v),
			false, // <-- convertsNulls
			hints)

The generated code is:

                converter: new ValueConverter<object, byte[]>(
                    (object v) => CompressedBsonValueConverter<object>.DoSerializeSmallestSize(v),
                    (Byte[] v) => CompressedBsonValueConverter<object>.DoDeserialize(v)),
[...]
                jsonValueReaderWriter: new JsonConvertedValueReaderWriter<object, byte[]>(
                    JsonByteArrayReaderWriter.Instance,
                    new ValueConverter<object, byte[]>(
                        (object v) => CompressedBsonValueConverter<object>.DoSerializeSmallestSize(v),
                        (Byte[] v) => CompressedBsonValueConverter<object>.DoDeserialize(v))));

The same code is generated if I change the convertsNulls parameter to true.

schuettecarsten · 2023-10-04T10:56:52Z

@ajcvickers Is is by design / expected that the code generator creates lots of code that seems to be unneccessary? Here is one example, the following code is generated for every SQL server column. This creates dozends of identical ValueComparer instances, lots of expressions and redundant compiled lambda functions....

            tenantID.TypeMapping = SqlServerLongTypeMapping.Default.Clone(
                comparer: new ValueComparer<long>(
                    (long v1, long v2) => v1 == v2,
                    (long v) => v.GetHashCode(),
                    (long v) => v),
                keyComparer: new ValueComparer<long>(
                    (long v1, long v2) => v1 == v2,
                    (long v) => v.GetHashCode(),
                    (long v) => v),
                providerValueComparer: new ValueComparer<long>(
                    (long v1, long v2) => v1 == v2,
                    (long v) => v.GetHashCode(),
                    (long v) => v));

For nullable columns, the code is more complex, but also always the same for each column of that type:

            defaultRecipientID.TypeMapping = SqlServerLongTypeMapping.Default.Clone(
                comparer: new ValueComparer<long?>(
                    (Nullable<long> v1, Nullable<long> v2) => v1.HasValue && v2.HasValue && (long)v1 == (long)v2 || !v1.HasValue && !v2.HasValue,
                    (Nullable<long> v) => v.HasValue ? ((long)v).GetHashCode() : 0,
                    (Nullable<long> v) => v.HasValue ? (Nullable<long>)(long)v : default(Nullable<long>)),
                keyComparer: new ValueComparer<long?>(
                    (Nullable<long> v1, Nullable<long> v2) => v1.HasValue && v2.HasValue && (long)v1 == (long)v2 || !v1.HasValue && !v2.HasValue,
                    (Nullable<long> v) => v.HasValue ? ((long)v).GetHashCode() : 0,
                    (Nullable<long> v) => v.HasValue ? (Nullable<long>)(long)v : default(Nullable<long>)),
                providerValueComparer: new ValueComparer<long?>(
                    (Nullable<long> v1, Nullable<long> v2) => v1.HasValue && v2.HasValue && (long)v1 == (long)v2 || !v1.HasValue && !v2.HasValue,
                    (Nullable<long> v) => v.HasValue ? ((long)v).GetHashCode() : 0,
                    (Nullable<long> v) => v.HasValue ? (Nullable<long>)(long)v : default(Nullable<long>)));

The same for SqlServerStringTypeMapping, SqlServerBoolTypeMapping, SqlServerByteArrayTypeMapping, GuidTypeMapping, ...

ajcvickers · 2023-10-04T16:46:48Z

/cc @AndriySvyryd

Fixes #31942 ### Description Updates to the compiled model resulted in losing the `convertsNulls` state of a value converter. ### Customer impact Silent regression from 7 resulting in different data returned. ### How found Customer reported on daily build. ### Regression Yes. ### Testing Added tests. ### Risk Low.

ajcvickers · 2023-10-04T19:02:36Z

Root cause found. I have a custom ValueConverter that has this constructor:

This looks like the same case as #31859.

schuettecarsten · 2023-10-04T20:36:04Z

@ajcvickers @AndriySvyryd
Thank you for the quick fix. So the generated code and the complexity for the TypeMapping stuff is by design?

AndriySvyryd · 2023-10-05T03:35:07Z

@ajcvickers @AndriySvyryd Thank you for the quick fix. So the generated code and the complexity for the TypeMapping stuff is by design?

Yes, this is part of our future Native AOT support #29754
Meanwhile it will further decrease the startup time.

schuettecarsten added the customer-reported label Oct 3, 2023

ajcvickers added type-bug area-model-building labels Oct 4, 2023

ajcvickers self-assigned this Oct 4, 2023

ajcvickers added Servicing-consider regression labels Oct 4, 2023

ajcvickers changed the title ~~[8.0-rc.1] 'dbcontext optimize' fails with 'Only primitive constant nodes are supported.', works in 7.0.11~~ [8.0-rc.1] 'dbcontext optimize' does not preserve convertsNulls Oct 4, 2023

ajcvickers mentioned this issue Oct 4, 2023

[release/8.0] Preserve convertsNulls in the compiled model #31967

Merged

AndriySvyryd added this to the 8.0.0 milestone Oct 4, 2023

AndriySvyryd added the closed-fixed The issue has been fixed and is/will be included in the release indicated by the issue milestone. label Oct 4, 2023

ajcvickers closed this as completed in #31967 Oct 5, 2023

ajcvickers added Servicing-approved and removed Servicing-consider labels Oct 11, 2023

ajcvickers removed their assignment Aug 31, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[8.0-rc.1] 'dbcontext optimize' does not preserve `convertsNulls` #31942

[8.0-rc.1] 'dbcontext optimize' does not preserve `convertsNulls` #31942

schuettecarsten commented Oct 3, 2023 •

edited

Loading

ajcvickers commented Oct 3, 2023

schuettecarsten commented Oct 4, 2023 •

edited

Loading

schuettecarsten commented Oct 4, 2023 •

edited

Loading

schuettecarsten commented Oct 4, 2023 •

edited

Loading

ajcvickers commented Oct 4, 2023

ajcvickers commented Oct 4, 2023

schuettecarsten commented Oct 4, 2023 •

edited

Loading

AndriySvyryd commented Oct 5, 2023

[8.0-rc.1] 'dbcontext optimize' does not preserve convertsNulls #31942

[8.0-rc.1] 'dbcontext optimize' does not preserve convertsNulls #31942

Comments

schuettecarsten commented Oct 3, 2023 • edited Loading

ajcvickers commented Oct 3, 2023

schuettecarsten commented Oct 4, 2023 • edited Loading

schuettecarsten commented Oct 4, 2023 • edited Loading

schuettecarsten commented Oct 4, 2023 • edited Loading

ajcvickers commented Oct 4, 2023

ajcvickers commented Oct 4, 2023

schuettecarsten commented Oct 4, 2023 • edited Loading

AndriySvyryd commented Oct 5, 2023

[8.0-rc.1] 'dbcontext optimize' does not preserve `convertsNulls` #31942

[8.0-rc.1] 'dbcontext optimize' does not preserve `convertsNulls` #31942

schuettecarsten commented Oct 3, 2023 •

edited

Loading

schuettecarsten commented Oct 4, 2023 •

edited

Loading

schuettecarsten commented Oct 4, 2023 •

edited

Loading

schuettecarsten commented Oct 4, 2023 •

edited

Loading

schuettecarsten commented Oct 4, 2023 •

edited

Loading