Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parameter based optimization tries to order by constant #19019

Closed
smitpatel opened this issue Nov 21, 2019 · 1 comment · Fixed by #19168
Closed

Parameter based optimization tries to order by constant #19019

smitpatel opened this issue Nov 21, 2019 · 1 comment · Fixed by #19168
Labels
area-query closed-fixed The issue has been fixed and is/will be included in the release indicated by the issue milestone. type-bug
Milestone

Comments

@smitpatel
Copy link
Member

smitpatel commented Nov 21, 2019

Steps to reproduce

What steps can we follow to reproduce the issue?

We ❤ code! Include a complete code listing or attach a simplified project

        [ConditionalTheory]
        [MemberData(nameof(IsAsyncData))]
        public virtual Task OrderBy_StartsWith_with_null_parameter_as_argument(bool async)
        {
            var prm = (string)null;

            return AssertQuery(
                async,
                ss => ss.Set<Gear>().OrderBy(g => g.FullName.StartsWith(prm)).ThenBy(g => g.Nickname),
                ss => ss.Set<Gear>().OrderBy(g => false).ThenBy(g => g.Nickname),
                assertOrder: true);
        }

Got Exceptions? Include both the message and the stack trace

 Message: 
    Microsoft.Data.SqlClient.SqlException : A constant expression was encountered in the ORDER BY list, position 1.
  Stack Trace: 
    SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
    SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
    TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
    TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
    SqlDataReader.TryConsumeMetaData()
    SqlDataReader.get_MetaData()
    SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString, Boolean isInternal, Boolean forDescribeParameterEncryption, Boolean shouldCacheForAlwaysEncrypted)
    SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean isAsync, Int32 timeout, Task& task, Boolean asyncWrite, Boolean inRetry, SqlDataReader ds, Boolean describeParameterEncryptionRequest)
    SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, TaskCompletionSource`1 completion, Int32 timeout, Task& task, Boolean& usedCache, Boolean asyncWrite, Boolean inRetry, String method)
    SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method)
    SqlCommand.ExecuteReader(CommandBehavior behavior)
    SqlCommand.ExecuteDbDataReader(CommandBehavior behavior)
    DbCommand.ExecuteReader()
    RelationalCommand.ExecuteReader(RelationalCommandParameterObject parameterObject) line 404
    Enumerator.InitializeReader(DbContext _, Boolean result) line 176
    ExecutionStrategy.ExecuteImplementation[TState,TResult](Func`3 operation, Func`3 verifySucceeded, TState state) line 173
    ExecutionStrategy.Execute[TState,TResult](TState state, Func`3 operation, Func`3 verifySucceeded) line 159
    Enumerator.MoveNext() line 125
    LargeArrayBuilder`1.AddRange(IEnumerable`1 items)
    EnumerableHelpers.ToArray[T](IEnumerable`1 source)
    Enumerable.ToArray[TSource](IEnumerable`1 source)
    QueryAsserter`1.AssertQuery[TResult](Func`2 actualQuery, Func`2 expectedQuery, Func`2 elementSorter, Action`2 elementAsserter, Boolean assertOrder, Int32 entryCount, Boolean async, String testMethodName) line 87
    GearsOfWarQuerySqlServerTest.OrderBy_StartsWith_with_null_parameter_as_argument(Boolean async) line 7095
    --- End of stack trace from previous location ---

Further technical details

EF Core version: master
Database provider: (e.g. Microsoft.EntityFrameworkCore.SqlServer)
Target framework: (e.g. .NET Core 3.0)
Operating system:
IDE: (e.g. Visual Studio 2019 16.3)

@bmarder
Copy link

bmarder commented Dec 11, 2019

I encounter this same issue with the following query. Regression from 2.2. If the list has any elements, it works fine.

var ids = new List<int>();
await db.Blogs
    .OrderBy(x => ids.Contains(x.Id))
    .ToListAsync();

workaround solution

var ids = new List<int>();
await db.Blogs
    .OrderBy(x =>  ids.Any() && ids.Contains(x.Id))
    .ToListAsync();

maumar added a commit that referenced this issue Dec 13, 2019
- moving NullSemantics visitor after 2nd level cache - we need to know the parameter values to properly handle IN expressions wrt null semantics,
- NullSemantics visitor needs to go before SqlExpressionOptimizer and SearchCondition, so those two are also moved after 2nd level cache,
- moving optimizations that depend on knowing the nullability to NullSemantics visitor - optimizer now only contains optimizations that also work in 3-value logic, or when we know nulls can't happen,
- merging InExpressionValuesExpandingExpressionVisitor int NullSemantics visitor, so that we don't apply the rewrite for UseRelationalNulls,
- preventing NulSemantics from performing double visitation when computing non-nullable columns.

Resolves #11464
Resolves #15722
Resolves #18338
Resolves #18597
Resolves #18689
Resolves #19019
@maumar maumar added the closed-fixed The issue has been fixed and is/will be included in the release indicated by the issue milestone. label Dec 13, 2019
maumar added a commit that referenced this issue Dec 13, 2019
- moving NullSemantics visitor after 2nd level cache - we need to know the parameter values to properly handle IN expressions wrt null semantics,
- NullSemantics visitor needs to go before SqlExpressionOptimizer and SearchCondition, so those two are also moved after 2nd level cache,
- moving optimizations that depend on knowing the nullability to NullSemantics visitor - optimizer now only contains optimizations that also work in 3-value logic, or when we know nulls can't happen,
- merging InExpressionValuesExpandingExpressionVisitor int NullSemantics visitor, so that we don't apply the rewrite for UseRelationalNulls,
- preventing NulSemantics from performing double visitation when computing non-nullable columns.

Resolves #11464
Resolves #15722
Resolves #18338
Resolves #18597
Resolves #18689
Resolves #19019
maumar added a commit that referenced this issue Dec 16, 2019
- moving NullSemantics visitor after 2nd level cache - we need to know the parameter values to properly handle IN expressions wrt null semantics,
- NullSemantics visitor needs to go before SqlExpressionOptimizer and SearchCondition, so those two are also moved after 2nd level cache,
- moving optimizations that depend on knowing the nullability to NullSemantics visitor - optimizer now only contains optimizations that also work in 3-value logic, or when we know nulls can't happen,
- merging InExpressionValuesExpandingExpressionVisitor int NullSemantics visitor, so that we don't apply the rewrite for UseRelationalNulls,
- preventing NulSemantics from performing double visitation when computing non-nullable columns.

Resolves #11464
Resolves #15722
Resolves #18338
Resolves #18597
Resolves #18689
Resolves #19019
maumar added a commit that referenced this issue Dec 16, 2019
- moving NullSemantics visitor after 2nd level cache - we need to know the parameter values to properly handle IN expressions wrt null semantics,
- NullSemantics visitor needs to go before SqlExpressionOptimizer and SearchCondition, so those two are also moved after 2nd level cache,
- moving optimizations that depend on knowing the nullability to NullSemantics visitor - optimizer now only contains optimizations that also work in 3-value logic, or when we know nulls can't happen,
- merging InExpressionValuesExpandingExpressionVisitor int NullSemantics visitor, so that we don't apply the rewrite for UseRelationalNulls,
- preventing NulSemantics from performing double visitation when computing non-nullable columns.

Resolves #11464
Resolves #15722
Resolves #18338
Resolves #18597
Resolves #18689
Resolves #19019
maumar added a commit that referenced this issue Dec 16, 2019
- moving NullSemantics visitor after 2nd level cache - we need to know the parameter values to properly handle IN expressions wrt null semantics,
- NullSemantics visitor needs to go before SqlExpressionOptimizer and SearchCondition, so those two are also moved after 2nd level cache,
- moving optimizations that depend on knowing the nullability to NullSemantics visitor - optimizer now only contains optimizations that also work in 3-value logic, or when we know nulls can't happen,
- merging InExpressionValuesExpandingExpressionVisitor int NullSemantics visitor, so that we don't apply the rewrite for UseRelationalNulls,
- preventing NulSemantics from performing double visitation when computing non-nullable columns.

Resolves #11464
Resolves #15722
Resolves #18338
Resolves #18597
Resolves #18689
Resolves #19019
maumar added a commit that referenced this issue Dec 18, 2019
- moving NullSemantics visitor after 2nd level cache - we need to know the parameter values to properly handle IN expressions wrt null semantics,
- NullSemantics visitor needs to go before SqlExpressionOptimizer and SearchCondition, so those two are also moved after 2nd level cache,
- moving optimizations that depend on knowing the nullability to NullSemantics visitor - optimizer now only contains optimizations that also work in 3-value logic, or when we know nulls can't happen,
- merging InExpressionValuesExpandingExpressionVisitor int NullSemantics visitor, so that we don't apply the rewrite for UseRelationalNulls,
- preventing NulSemantics from performing double visitation when computing non-nullable columns.

Resolves #11464
Resolves #15722
Resolves #18338
Resolves #18597
Resolves #18689
Resolves #19019
maumar added a commit that referenced this issue Dec 31, 2019
- moving NullSemantics visitor after 2nd level cache - we need to know the parameter values to properly handle IN expressions wrt null semantics,
- NullSemantics visitor needs to go before SqlExpressionOptimizer and SearchCondition, so those two are also moved after 2nd level cache,
- combining NullSemantics with SqlExpressionOptimizer (kept SqlExpressionOptimizer name) - SearchCondition now applies some relevant optimization itself, so that we don't need to run full optimizer afterwards,
- merging InExpressionValuesExpandingExpressionVisitor into SqlExpressionOptimizer as well, so that we don't apply the rewrite for UseRelationalNulls,
- preventing NulSemantics from performing double visitation when computing non-nullable columns.

Resolves #11464
Resolves #15722
Resolves #18338
Resolves #18597
Resolves #18689
Resolves #19019
maumar added a commit that referenced this issue Jan 1, 2020
- moving NullSemantics visitor after 2nd level cache - we need to know the parameter values to properly handle IN expressions wrt null semantics,
- NullSemantics visitor needs to go before SqlExpressionOptimizer and SearchCondition, so those two are also moved after 2nd level cache,
- combining NullSemantics with SqlExpressionOptimizer (kept SqlExpressionOptimizer name) - SearchCondition now applies some relevant optimization itself, so that we don't need to run full optimizer afterwards,
- merging InExpressionValuesExpandingExpressionVisitor into SqlExpressionOptimizer as well, so that we don't apply the rewrite for UseRelationalNulls,
- preventing NulSemantics from performing double visitation when computing non-nullable columns.

Resolves #11464
Resolves #15722
Resolves #18338
Resolves #18597
Resolves #18689
Resolves #19019
maumar added a commit that referenced this issue Jan 2, 2020
- moving NullSemantics visitor after 2nd level cache - we need to know the parameter values to properly handle IN expressions wrt null semantics,
- NullSemantics visitor needs to go before SqlExpressionOptimizer and SearchCondition, so those two are also moved after 2nd level cache,
- combining NullSemantics with SqlExpressionOptimizer (kept SqlExpressionOptimizer name) - SearchCondition now applies some relevant optimization itself, so that we don't need to run full optimizer afterwards,
- merging InExpressionValuesExpandingExpressionVisitor into SqlExpressionOptimizer as well, so that we don't apply the rewrite for UseRelationalNulls,
- preventing NulSemantics from performing double visitation when computing non-nullable columns.

Resolves #11464
Resolves #15722
Resolves #18338
Resolves #18597
Resolves #18689
Resolves #19019
maumar added a commit that referenced this issue Jan 4, 2020
- moving NullSemantics visitor after 2nd level cache - we need to know the parameter values to properly handle IN expressions wrt null semantics,
- NullSemantics visitor needs to go before SqlExpressionOptimizer and SearchCondition, so those two are also moved after 2nd level cache,
- combining NullSemantics with SqlExpressionOptimizer (kept SqlExpressionOptimizer name) - SearchCondition now applies some relevant optimization itself, so that we don't need to run full optimizer afterwards,
- merging InExpressionValuesExpandingExpressionVisitor into SqlExpressionOptimizer as well, so that we don't apply the rewrite for UseRelationalNulls,
- preventing NulSemantics from performing double visitation when computing non-nullable columns.

Resolves #11464
Resolves #15722
Resolves #18338
Resolves #18597
Resolves #18689
Resolves #19019
maumar added a commit that referenced this issue Jan 7, 2020
- moving NullSemantics visitor after 2nd level cache - we need to know the parameter values to properly handle IN expressions wrt null semantics,
- NullSemantics visitor needs to go before SqlExpressionOptimizer and SearchCondition, so those two are also moved after 2nd level cache,
- combining NullSemantics with SqlExpressionOptimizer (kept SqlExpressionOptimizer name) - SearchCondition now applies some relevant optimization itself, so that we don't need to run full optimizer afterwards,
- merging InExpressionValuesExpandingExpressionVisitor into SqlExpressionOptimizer as well, so that we don't apply the rewrite for UseRelationalNulls,
- preventing NulSemantics from performing double visitation when computing non-nullable columns.

Resolves #11464
Resolves #15722
Resolves #18338
Resolves #18597
Resolves #18689
Resolves #19019
maumar added a commit that referenced this issue Jan 8, 2020
- moving NullSemantics visitor after 2nd level cache - we need to know the parameter values to properly handle IN expressions wrt null semantics,
- NullSemantics visitor needs to go before SqlExpressionOptimizer and SearchCondition, so those two are also moved after 2nd level cache,
- combining NullSemantics with SqlExpressionOptimizer (kept SqlExpressionOptimizer name) - SearchCondition now applies some relevant optimization itself, so that we don't need to run full optimizer afterwards,
- merging InExpressionValuesExpandingExpressionVisitor into SqlExpressionOptimizer as well, so that we don't apply the rewrite for UseRelationalNulls,
- preventing NulSemantics from performing double visitation when computing non-nullable columns.

Resolves #11464
Resolves #15722
Resolves #18338
Resolves #18597
Resolves #18689
Resolves #19019
maumar added a commit that referenced this issue Jan 8, 2020
- moving NullSemantics visitor after 2nd level cache - we need to know the parameter values to properly handle IN expressions wrt null semantics,
- NullSemantics visitor needs to go before SqlExpressionOptimizer and SearchCondition, so those two are also moved after 2nd level cache,
- combining NullSemantics with SqlExpressionOptimizer (kept SqlExpressionOptimizer name) - SearchCondition now applies some relevant optimization itself, so that we don't need to run full optimizer afterwards,
- merging InExpressionValuesExpandingExpressionVisitor into SqlExpressionOptimizer as well, so that we don't apply the rewrite for UseRelationalNulls,
- preventing NulSemantics from performing double visitation when computing non-nullable columns.

Resolves #11464
Resolves #15722
Resolves #18338
Resolves #18597
Resolves #18689
Resolves #19019
maumar added a commit that referenced this issue Jan 8, 2020
- moving NullSemantics visitor after 2nd level cache - we need to know the parameter values to properly handle IN expressions wrt null semantics,
- NullSemantics visitor needs to go before SqlExpressionOptimizer and SearchCondition, so those two are also moved after 2nd level cache,
- combining NullSemantics with SqlExpressionOptimizer (kept SqlExpressionOptimizer name) - SearchCondition now applies some relevant optimization itself, so that we don't need to run full optimizer afterwards,
- merging InExpressionValuesExpandingExpressionVisitor into SqlExpressionOptimizer as well, so that we don't apply the rewrite for UseRelationalNulls,
- preventing NulSemantics from performing double visitation when computing non-nullable columns.

Resolves #11464
Resolves #15722
Resolves #18338
Resolves #18597
Resolves #18689
Resolves #19019
maumar added a commit that referenced this issue Jan 8, 2020
- moving NullSemantics visitor after 2nd level cache - we need to know the parameter values to properly handle IN expressions wrt null semantics,
- NullSemantics visitor needs to go before SqlExpressionOptimizer and SearchCondition, so those two are also moved after 2nd level cache,
- combining NullSemantics with SqlExpressionOptimizer (kept SqlExpressionOptimizer name) - SearchCondition now applies some relevant optimization itself, so that we don't need to run full optimizer afterwards,
- merging InExpressionValuesExpandingExpressionVisitor into SqlExpressionOptimizer as well, so that we don't apply the rewrite for UseRelationalNulls,
- preventing NulSemantics from performing double visitation when computing non-nullable columns.

Resolves #11464
Resolves #15722
Resolves #18338
Resolves #18597
Resolves #18689
Resolves #19019
maumar added a commit that referenced this issue Jan 8, 2020
- moving NullSemantics visitor after 2nd level cache - we need to know the parameter values to properly handle IN expressions wrt null semantics,
- NullSemantics visitor needs to go before SqlExpressionOptimizer and SearchCondition, so those two are also moved after 2nd level cache,
- combining NullSemantics with SqlExpressionOptimizer (kept SqlExpressionOptimizer name) - SearchCondition now applies some relevant optimization itself, so that we don't need to run full optimizer afterwards,
- merging InExpressionValuesExpandingExpressionVisitor into SqlExpressionOptimizer as well, so that we don't apply the rewrite for UseRelationalNulls,
- preventing NulSemantics from performing double visitation when computing non-nullable columns.

Resolves #11464
Resolves #15722
Resolves #18338
Resolves #18597
Resolves #18689
Resolves #19019
maumar added a commit that referenced this issue Jan 8, 2020
- moving NullSemantics visitor after 2nd level cache - we need to know the parameter values to properly handle IN expressions wrt null semantics,
- NullSemantics visitor needs to go before SqlExpressionOptimizer and SearchCondition, so those two are also moved after 2nd level cache,
- combining NullSemantics with SqlExpressionOptimizer (kept SqlExpressionOptimizer name) - SearchCondition now applies some relevant optimization itself, so that we don't need to run full optimizer afterwards,
- merging InExpressionValuesExpandingExpressionVisitor into SqlExpressionOptimizer as well, so that we don't apply the rewrite for UseRelationalNulls,
- preventing NulSemantics from performing double visitation when computing non-nullable columns.

Resolves #11464
Resolves #15722
Resolves #18338
Resolves #18597
Resolves #18689
Resolves #19019
@maumar maumar closed this as completed in a5f3e26 Jan 8, 2020
@ajcvickers ajcvickers modified the milestones: 5.0.0, 5.0.0-preview1 Mar 13, 2020
@ajcvickers ajcvickers modified the milestones: 5.0.0-preview1, 5.0.0 Nov 7, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-query closed-fixed The issue has been fixed and is/will be included in the release indicated by the issue milestone. type-bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants