NotImplementedException thrown if predicate passed into a linq method from within an expression tree

[ghost]

Hi. I am running into a NotImplementedException exception any time I attempt to pass a predicate into any linq method, if it's called from inside an expression tree. Same issue with FirstOrDefault(...), OrderBy(...), Select(...) etc.

FirstOrDefault() with no parameters works fine.

I've just updated to the latest preview6 packages and still encounter the same trouble.

Cheers

---

EmployeesController.cs

[HttpGet]
public IQueryable<EmployeesDTO> Get()
{
    return _context.Employees.Select(EmployeesDTO.Projection);
}

---

This works fine:

namespace MyProject.Api.DTOs
{
    public class EmployeesDTO : DTO
    {
        public int LocalId { get; set; }
        public string Title { get; set; }

        public static readonly Expression<Func<Employees, EmployeesDTO>> Projection = x => new EmployeesDTO
        {            
            LocalId = x.LocalId,
            Title = x.EmployeesTitles
                .FirstOrDefault()
                .EmployeeTitle.Description,
        };
    }
}

This throws NotImplementedException:

namespace MyProject.Api.DTOs
{
    public class EmployeesDTO : DTO
    {
        public int LocalId { get; set; }
        public string Title { get; set; }

        public static readonly Expression<Func<Employees, EmployeesDTO>> Projection = x => new EmployeesDTO
        {            
            LocalId = x.LocalId,
            Title = x.EmployeesTitles
                .FirstOrDefault(t => t.CreatedDate == t.CreatedDate) 
                .EmployeeTitle.Description,
        };
    }
}

Exception

fail: Microsoft.AspNetCore.Server.Kestrel[13]
      Connection id "0HLNG6LJ8PIR5", Request id "0HLNG6LJ8PIR5:00000001": An unhandled exception was thrown by the application.
System.NotImplementedException: The method or operation is not implemented.
   at Microsoft.EntityFrameworkCore.Query.Pipeline.QueryableMethodTranslatingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression)
   at System.Linq.Expressions.MethodCallExpression.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at Microsoft.EntityFrameworkCore.Query.Pipeline.QueryableMethodTranslatingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression)
   at System.Linq.Expressions.MethodCallExpression.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at Microsoft.EntityFrameworkCore.Query.Pipeline.QueryableMethodTranslatingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression)
   at System.Linq.Expressions.MethodCallExpression.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at Microsoft.EntityFrameworkCore.Query.Pipeline.QueryableMethodTranslatingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression)
   at System.Linq.Expressions.MethodCallExpression.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at Microsoft.EntityFrameworkCore.Relational.Query.Pipeline.RelationalQueryableMethodTranslatingExpressionVisitor.TranslateSubquery(Expression expression)
   at Microsoft.EntityFrameworkCore.Relational.Query.Pipeline.RelationalSqlTranslatingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression)
   at System.Linq.Expressions.MethodCallExpression.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at Microsoft.EntityFrameworkCore.Relational.Query.Pipeline.RelationalSqlTranslatingExpressionVisitor.Translate(Expression expression)
   at Microsoft.EntityFrameworkCore.Relational.Query.Pipeline.RelationalProjectionBindingExpressionVisitor.Visit(Expression expression)
   at Microsoft.EntityFrameworkCore.Relational.Query.Pipeline.RelationalProjectionBindingExpressionVisitor.VisitMemberInit(MemberInitExpression memberInitExpression)
   at System.Linq.Expressions.MemberInitExpression.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at Microsoft.EntityFrameworkCore.Relational.Query.Pipeline.RelationalProjectionBindingExpressionVisitor.Visit(Expression expression)
   at Microsoft.EntityFrameworkCore.Relational.Query.Pipeline.RelationalProjectionBindingExpressionVisitor.Translate(SelectExpression selectExpression, Expression expression)
   at Microsoft.EntityFrameworkCore.Relational.Query.Pipeline.RelationalQueryableMethodTranslatingExpressionVisitor.TranslateSelect(ShapedQueryExpression source, LambdaExpression selector)
   at Microsoft.EntityFrameworkCore.Query.Pipeline.QueryableMethodTranslatingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression)
   at System.Linq.Expressions.MethodCallExpression.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at Microsoft.EntityFrameworkCore.Query.Pipeline.QueryCompilationContext2.CreateQueryExecutor[TResult](Expression query)
   at Microsoft.EntityFrameworkCore.Storage.Database.CompileQuery2[TResult](Expression query, Boolean async)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.CompileQueryCore[TResult](IDatabase database, Expression query, IModel model, Boolean async)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.<>c__DisplayClass9_0`1.<Execute>b__0()
   at Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQueryCore[TFunc](Object cacheKey, Func`1 compiler)
   at Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQuery[TResult](Object cacheKey, Func`1 compiler)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.Execute[TResult](Expression query)
   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryProvider.Execute[TResult](Expression expression)
   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryable`1.System.Collections.IEnumerable.GetEnumerator()
   at System.Text.Json.Serialization.JsonSerializer.HandleEnumerable(JsonClassInfo elementClassInfo, JsonSerializerOptions options, Utf8JsonWriter writer, WriteStack& state)
   at System.Text.Json.Serialization.JsonSerializer.Write(Utf8JsonWriter writer, Int32 flushThreshold, JsonSerializerOptions options, WriteStack& state)
   at System.Text.Json.Serialization.JsonSerializer.WriteAsyncCore(Object value, Type type, Stream utf8Json, JsonSerializerOptions options, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Mvc.Formatters.SystemTextJsonOutputFormatter.WriteResponseBodyAsync(OutputFormatterWriteContext context, Encoding selectedEncoding)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextResultFilterAsync>g__Awaited|29_0[TFilter,TFilterAsync](ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResultExecutedContextSealed context)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.ResultNext[TFilter,TFilterAsync](State& next, Scope& scope, Object& state, Boolean& isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeResultFilters()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|19_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Logged|17_1(ResourceInvoker invoker)
   at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
      Request finished in 1517.5037ms 500

Further technical details

VSCode 1.35.1 linux
OS: KDE Neon (Ubuntu 18.04 base)
Dabase Provider: Microsoft.EntityFrameworkCore.SqlServer

dotnet list package

   Top-level Package                                          Requested                     Resolved               
   > Ical.Net                                                 4.1.11                        4.1.11                 
   > Microsoft.AspNetCore.Mvc.NewtonsoftJson                  3.0.0-preview6.*              3.0.0-preview6.19307.2 
   > Microsoft.EntityFrameworkCore                            3.0.0-preview6.*              3.0.0-preview6.19304.10
   > Microsoft.EntityFrameworkCore.Design                     3.0.0-preview6.*              3.0.0-preview6.19304.10
   > Microsoft.EntityFrameworkCore.Proxies                    3.0.0-preview6.*              3.0.0-preview6.19304.10
   > Microsoft.EntityFrameworkCore.SqlServer                  3.0.0-preview6.*              3.0.0-preview6.19304.10
   > Microsoft.NETCore.App                              (A)   [3.0.0-preview6-27804-01, )   3.0.0-preview6-27804-01
   > Microsoft.VisualStudio.Web.CodeGeneration.Design         3.0.0-preview5-*              3.0.0-preview5-19264-04

dotnet --info

.NET Core SDK (reflecting any global.json):
 Version:   3.0.100-preview6-012264
 Commit:    be3f0c1a03

Runtime Environment:
 OS Name:     neon
 OS Version:  18.04
 OS Platform: Linux
 RID:         linux-x64
 Base Path:   /snap/dotnet-sdk/41/sdk/3.0.100-preview6-012264/

Host (useful for support):
  Version: 3.0.0-preview6-27804-01
  Commit:  fdf81c6faf

.NET Core SDKs installed:
  3.0.100-preview6-012264 [/snap/dotnet-sdk/41/sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.App 3.0.0-preview6.19307.2 [/snap/dotnet-sdk/41/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 3.0.0-preview6-27804-01 [/snap/dotnet-sdk/41/shared/Microsoft.NETCore.App]

To install additional .NET Core runtimes or SDKs:
  https://aka.ms/dotnet-download

[ajcvickers]

First, many thanks for trying preview 6. We really appreciate people trying the previews and filing issues on what they find.

If you haven't seen it already, there is some good information in the preview announcement post about the changes that are happening to LINQ queries for EF Core 3.0. Preview 6 is the first release containing these changes, which means we are both expecting things to be broken, but at the same time very grateful for everyone who tries the release and generated feedback.

@smitpatel In triage we came to the conclusion that this isn't something that worked in previous versions, but that it would likely work in 3.0. Is this correct, and if so is there an issue already tacking it?

[smitpatel]

We need to investigate this since it should already be working. There are few places which needs to work out to translate it correctly.

• FOD with predicate gets converted to where + FOD by nav rewrite
• Subquery member pushdown.

@arcooke - Can you verify the predicate passed into FirstOrDefault as it is always true? Is that typo or valid?

[ghost]

@ajcvickers Happy to help.. Oddly enough, I was on preview 5 when I first encountered this, and updated to 6 to see if it had been fixed. I just now reverted everything back to preview 5 (19227.1) to re-test, and now the same code posted above works fine. I may have temporarily had a version mismatch somewhere since I used wildcards for nuget package versions (3.0.0-preview5.*, 3.0.0-preview6.*, etc)

Preview 6 still breaks here, but 5 is working. Just verified with the identical code as posted above.

Working:

   > Microsoft.EntityFrameworkCore                            3.0.0-preview5.*              3.0.0-preview5.19227.1 
   > Microsoft.EntityFrameworkCore.Analyzers                  3.0.0-preview5.*              3.0.0-preview5.19227.1 
   > Microsoft.EntityFrameworkCore.Design                     3.0.0-preview5.*              3.0.0-preview5.19227.1 
   > Microsoft.EntityFrameworkCore.Proxies                    3.0.0-preview5.*              3.0.0-preview5.19227.1 
   > Microsoft.EntityFrameworkCore.SqlServer                  3.0.0-preview5.*              3.0.0-preview5.19227.1 

Not working:

   > Microsoft.EntityFrameworkCore                            3.0.0-preview6.*              3.0.0-preview6.19304.10
   > Microsoft.EntityFrameworkCore.Analyzers                  3.0.0-preview6.*              3.0.0-preview6.19304.10
   > Microsoft.EntityFrameworkCore.Design                     3.0.0-preview6.*              3.0.0-preview6.19304.10
   > Microsoft.EntityFrameworkCore.Proxies                    3.0.0-preview6.*              3.0.0-preview6.19304.10
   > Microsoft.EntityFrameworkCore.SqlServer                  3.0.0-preview6.*              3.0.0-preview6.19304.10

@smitpatel It is always true and was not a typo. I reduced the code down to its simplest possible form for testing. t.CreatedDate == t.CreatedDate .. I tested it that way since it should be functionally identical to calling FirstOrDefault() with no params. CreatedDate is defaulted to getdate() on insert from sql server so it always contains a valid datetime value.

[mhosman]

Same issue here!!!

[aybelanov]

Hi everyone!
I have the same issue after updating to preview6. Before it all worked witout issues (The next code is the native code NopCommerce).

This simple expression (and others similar ones too):
query = from p in query
from pc in p.ProductCategories.Where(pc => categoryIds.Contains(pc.CategoryId))
where categoryIds.Contains(pc.CategoryId)
select p;

calls next exception:
Microsoft.EntityFrameworkCore.Relational.Query.Pipeline.RelationalQueryableMethodTranslatingExpressionVisitor.TranslateSelectMany(ShapedQueryExpression source, LambdaExpression collectionSelector, LambdaExpression resultSelector)
Microsoft.EntityFrameworkCore.Query.Pipeline.QueryableMethodTranslatingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression)
System.Linq.Expressions.MethodCallExpression.Accept(ExpressionVisitor visitor)
System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
Microsoft.EntityFrameworkCore.Query.Pipeline.QueryableMethodTranslatingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression)
System.Linq.Expressions.MethodCallExpression.Accept(ExpressionVisitor visitor)
System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
Microsoft.EntityFrameworkCore.Query.Pipeline.QueryableMethodTranslatingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression)
System.Linq.Expressions.MethodCallExpression.Accept(ExpressionVisitor visitor)
System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
Microsoft.EntityFrameworkCore.Query.Pipeline.QueryableMethodTranslatingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression)
System.Linq.Expressions.MethodCallExpression.Accept(ExpressionVisitor visitor)
System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
Microsoft.EntityFrameworkCore.Query.Pipeline.QueryableMethodTranslatingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression)
System.Linq.Expressions.MethodCallExpression.Accept(ExpressionVisitor visitor)
System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
Microsoft.EntityFrameworkCore.Query.Pipeline.QueryableMethodTranslatingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression)
System.Linq.Expressions.MethodCallExpression.Accept(ExpressionVisitor visitor)
System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
Microsoft.EntityFrameworkCore.Query.Pipeline.QueryableMethodTranslatingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression)
System.Linq.Expressions.MethodCallExpression.Accept(ExpressionVisitor visitor)
System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
Microsoft.EntityFrameworkCore.Query.Pipeline.QueryableMethodTranslatingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression)
System.Linq.Expressions.MethodCallExpression.Accept(ExpressionVisitor visitor)
System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
Microsoft.EntityFrameworkCore.Query.Pipeline.QueryCompilationContext2.CreateQueryExecutor(Expression query)
Microsoft.EntityFrameworkCore.Storage.Database.CompileQuery2(Expression query, bool async)
Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.CompileQueryCore(IDatabase database, Expression query, IModel model, bool async)
Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler+<>c__DisplayClass9_0.b__0()
Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQueryCore(object cacheKey, Func<Func<QueryContext, TFunc>> compiler)
Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQuery(object cacheKey, Func<Func<QueryContext, TResult>> compiler)
Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.Execute(Expression query)
Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryProvider.Execute(Expression expression)
System.Linq.Queryable.Count(IQueryable source)

[ghost]

This is still an issue in Preview 9. Preview 5 is the newest version I'm able to use without running into this problem. Please look into this before releasing 3.0 .. this is critical functionality

This fails on preview 6, 7, 8 and 9

[smitpatel]

Generated SQL in 3.1

      SELECT [e1].[LocalId], (
          SELECT TOP(1) [e0].[Description]
          FROM [EmployeesTitles] AS [e]
          LEFT JOIN [EmployeeTitle] AS [e0] ON [e].[EmployeeTitleId] = [e0].[Id]
          WHERE [e1].[Id] = [e].[EmployeesId]) AS [Title]
      FROM [Employees] AS [e1]