为什么运行时表达式会导致 Entity Framework Core 5 的缓存发生冲突？

Question

在我忘记它之前，我的执行上下文，我正在使用 .Net 5 和包：

Microsoft.EntityFrameworkCore.Design 5.0.6
Microsoft.EntityFrameworkCore.Relational 5.0.6
MySql.EntityFrameworkCore 5.0.3.1

我的主要目标是在需要检索实体时消除执行表达式的重复性任务，例如：

public class GetListEntity
{
   property int QueryProperty { get; set }
}

public class Entity
{
   property int Property { get; set }
}

public async Task<ActionResult> List(GetListEntity getListEntity)
{
   var restrictions = new List<Expression<Func<Entity>
   if (model.QueryProperty != null)
   { 
      restrictions.Add(e => e.Property == model.QueryProperty);
   }
   nonTrackedQueryableEntities = this.dbContext.Set<Entity>()
                                               .AsNoTracking();

   var expectedEntity = restrictions.Aggregate((sr, nr) => sr.And(nr)); //The And method is below as an extension
   var expectedNonTrackedQueryableEntities = nonTrackedQueryableEntities.Where(expectedEntity);

   // I will get the total first because the API was meant to paginate the responses.
   var total = await expectedNonTrackedQueryableEntities.CountAsync();
}


public static class ExpressionExtensions
{
    public static Expression<Func<T, bool>> Or<T>(this Expression<Func<T, bool>> selfExpression, Expression<Func<T, bool>> otherExpression)
    {
        return selfExpression.Compose(otherExpression, Expression.OrElse);
    }

    public static Expression<Func<T, bool>> And<T>(this Expression<Func<T, bool>> selfExpression, Expression<Func<T, bool>> otherExpression)
    {
        return selfExpression.Compose(otherExpression, Expression.AndAlso);
    }

    private static InvocationExpression Casting<T>(this Expression<Func<T, bool>> selfExpression, Expression<Func<T, bool>> otherExpression)
    {
        return Expression.Invoke(otherExpression, selfExpression.Parameters.Cast<Expression>());
    }

    private static Expression<Func<T, bool>> Compose<T>(this Expression<Func<T, bool>> selfExpression, Expression<Func<T, bool>> otherExpression, Func<Expression, Expression, Expression> merge)
    {
        var invocationExpression = selfExpression.Casting(otherExpression);
        return Expression.Lambda<Func<T, bool>>(merge(selfExpression.Body, invocationExpression), selfExpression.Parameters);
    }
}

我已经设法实现了我想要的，但让我们说......部分，因为如果我尝试连续至少两次查询数据库，我会得到这个异常：


System.ArgumentException: An item with the same key has already been added. Key: e
   at System.Collections.Generic.Dictionary`2.TryInsert(TKey key, TValue value, InsertionBehavior behavior)
   at Microsoft.EntityFrameworkCore.Query.ExpressionEqualityComparer.ExpressionComparer.CompareLambda(LambdaExpression a, LambdaExpression b)
   at Microsoft.EntityFrameworkCore.Query.ExpressionEqualityComparer.ExpressionComparer.Compare(Expression left, Expression right)
   at Microsoft.EntityFrameworkCore.Query.ExpressionEqualityComparer.ExpressionComparer.Compare(Expression left, Expression right)
   at Microsoft.EntityFrameworkCore.Query.ExpressionEqualityComparer.ExpressionComparer.CompareBinary(BinaryExpression a, BinaryExpression b)
   at Microsoft.EntityFrameworkCore.Query.ExpressionEqualityComparer.ExpressionComparer.Compare(Expression left, Expression right)
   at Microsoft.EntityFrameworkCore.Query.ExpressionEqualityComparer.ExpressionComparer.CompareLambda(LambdaExpression a, LambdaExpression b)
   at Microsoft.EntityFrameworkCore.Query.ExpressionEqualityComparer.ExpressionComparer.Compare(Expression left, Expression right)
   at Microsoft.EntityFrameworkCore.Query.ExpressionEqualityComparer.ExpressionComparer.CompareUnary(UnaryExpression a, UnaryExpression b)
   at Microsoft.EntityFrameworkCore.Query.ExpressionEqualityComparer.ExpressionComparer.Compare(Expression left, Expression right)
   at Microsoft.EntityFrameworkCore.Query.ExpressionEqualityComparer.ExpressionComparer.CompareExpressionList(IReadOnlyList`1 a, IReadOnlyList`1 b)
   at Microsoft.EntityFrameworkCore.Query.ExpressionEqualityComparer.ExpressionComparer.CompareMethodCall(MethodCallExpression a, MethodCallExpression b)
   at Microsoft.EntityFrameworkCore.Query.ExpressionEqualityComparer.ExpressionComparer.Compare(Expression left, Expression right)
   at Microsoft.EntityFrameworkCore.Query.ExpressionEqualityComparer.ExpressionComparer.CompareExpressionList(IReadOnlyList`1 a, IReadOnlyList`1 b)
   at Microsoft.EntityFrameworkCore.Query.ExpressionEqualityComparer.ExpressionComparer.CompareMethodCall(MethodCallExpression a, MethodCallExpression b)
   at Microsoft.EntityFrameworkCore.Query.ExpressionEqualityComparer.ExpressionComparer.Compare(Expression left, Expression right)
   at Microsoft.EntityFrameworkCore.Query.ExpressionEqualityComparer.ExpressionComparer.CompareExpressionList(IReadOnlyList`1 a, IReadOnlyList`1 b)
   at Microsoft.EntityFrameworkCore.Query.ExpressionEqualityComparer.ExpressionComparer.CompareMethodCall(MethodCallExpression a, MethodCallExpression b)
   at Microsoft.EntityFrameworkCore.Query.ExpressionEqualityComparer.ExpressionComparer.Compare(Expression left, Expression right)
   at Microsoft.EntityFrameworkCore.Query.ExpressionEqualityComparer.Equals(Expression x, Expression y)
   at Microsoft.EntityFrameworkCore.Query.CompiledQueryCacheKeyGenerator.CompiledQueryCacheKey.Equals(CompiledQueryCacheKey other)
   at Microsoft.EntityFrameworkCore.Query.RelationalCompiledQueryCacheKeyGenerator.RelationalCompiledQueryCacheKey.Equals(RelationalCompiledQueryCacheKey other)
   at MySql.EntityFrameworkCore.Query.Internal.MySQLCompiledQueryCacheKeyGenerator.MySQLCompiledQueryCacheKey.Equals(MySQLCompiledQueryCacheKey other)
   at MySql.EntityFrameworkCore.Query.Internal.MySQLCompiledQueryCacheKeyGenerator.MySQLCompiledQueryCacheKey.Equals(Object obj)
   at System.Collections.Concurrent.ConcurrentDictionary`2.TryGetValue(TKey key, TValue& value)
   at Microsoft.Extensions.Caching.Memory.MemoryCache.TryGetValue(Object key, Object& result)
   at Microsoft.Extensions.Caching.Memory.CacheExtensions.TryGetValue[TItem](IMemoryCache cache, Object key, TItem& value)
   at Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQuery[TResult](Object cacheKey, Func`1 compiler)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.ExecuteAsync[TResult](Expression query, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryProvider.ExecuteAsync[TResult](Expression expression, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.ExecuteAsync[TSource,TResult](MethodInfo operatorMethodInfo, IQueryable`1 source, Expression expression, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.ExecuteAsync[TSource,TResult](MethodInfo operatorMethodInfo, IQueryable`1 source, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.CountAsync[TSource](IQueryable`1 source, CancellationToken cancellationToken)'

根据跟踪，我设法发现 ORM 出于某种原因正在缓存我的表达式（并放置参数名称，在本例中为 'e'）并且第二次未能检测到键冲突查询数据库的类似表达式。我说是出于某种原因，因为这不是主要交易，但至少奇怪的是缓存涉及非跟踪查询，也许我在中间遗漏了一些东西。

为了理解我是如何来到这里的，我将把代码放在下面。

首先要在与查询实体列表相关的每个模型中实现一个接口，并公开扩展方法 ListRestrictions（几乎在底部）。

public interface IEntityFilter<TEntity>
{ 
}

下一步是定义属性以总结使用属性执行的操作并生成部分表达式以在扩展方法中使用：

    [AttributeUsage(AttributeTargets.Property, AllowMultiple = false)]
    public abstract class FilterByPropertyAttribute : Attribute
    {
        protected string FirstPropertyPath { get; }

        protected IEnumerable<string> NPropertyPath { get; }

        public FilterByPropertyAttribute(string firstPropertyPath, params string[] nPropertyPath)
        {
            this.FirstPropertyPath = firstPropertyPath;
            this.NPropertyPath = nPropertyPath;
        }

        protected MemberExpression GetPropertyExpression(ParameterExpression parameterExpression)
        {
            var propertyExpression = Expression.Property(parameterExpression, this.FirstPropertyPath);
            foreach (var propertyPath in this.NPropertyPath)
            {
                propertyExpression = Expression.Property(propertyExpression, propertyPath);
            }
            return propertyExpression;
        }

       public abstract Expression GetExpression(ParameterExpression parameterExpression, object propertyValue);
    }

并避免与可为空的结构进行比较


    public abstract class NonNullableValuePropertyFilterAttribute : FilterByPropertyAttribute
    {
        public NonNullableValuePropertyFilterAttribute(string firstPropertyPath, params string[] nPropertyPath)
            : base(firstPropertyPath, nPropertyPath)
        {
        }

        public override Expression GetExpression(ParameterExpression parameterExpression, object propertyValue)
        {
            var propertyExpression = this.GetPropertyExpression(parameterExpression);
            return this.GetExpression(propertyExpression, this.GetConvertedConstantExpression(propertyExpression, Expression.Constant(propertyValue)));
        }

        protected abstract Expression GetExpression(MemberExpression memberExpression, UnaryExpression unaryExpression);

        private UnaryExpression GetConvertedConstantExpression(MemberExpression memberExpression, ConstantExpression constantExpression)
        {
            var convertedConstantExpression = Expression.Convert(constantExpression, memberExpression.Type);
            return convertedConstantExpression;
        }
    }

具有已定义角色的属性将是：


    public class EqualPropertyFilterAttribute : NonNullableValuePropertyFilterAttribute
    {

        public EqualPropertyFilterAttribute(string firstPropertyPath, params string[] nPropertyPath)
            : base(firstPropertyPath, nPropertyPath)
        {
        }

        protected override Expression GetExpression(MemberExpression memberExpression, UnaryExpression unaryExpression)
        {
            return Expression.Equal(memberExpression, unaryExpression);
        }
    }

最后，扩展本身：

    public static class EntityFilterExtensions
    {
        public static List<Expression<Func<TEntity, bool>>> ListRestrictions<TEntity>(this IEntityFilter<TEntity> entityFilter)
        {
            var entityFilterType = entityFilter.GetType();            
            var propertiesInfo = entityFilterType.GetProperties()
                                                 .Where(pi => pi.GetValue(entityFilter) != null 
                                                              && pi.CustomAttributes.Any(ca => ca.AttributeType
                                                                                                 .IsSubclassOf(typeof(FilterByPropertyAttribute))));

            var expressions = Enumerable.Empty<Expression<Func<TEntity, bool>>>();
            if (propertiesInfo.Any())
            {
                var entityType = typeof(TEntity);
                var parameterExpression = Expression.Parameter(entityType, "e");
                expressions =  propertiesInfo.Select(pi =>
                {
                    var filterByPropertyAttribute = Attribute.GetCustomAttribute(pi, typeof(FilterByPropertyAttribute)) as FilterByPropertyAttribute;
                    var propertyValue = pi.GetValue(entityFilter);
                    var expression = filterByPropertyAttribute.GetExpression(parameterExpression, propertyValue);
                    return Expression.Lambda<Func<TEntity, bool>>(expression, parameterExpression);
                });
            }

            return expressions.ToList();
        }
    }

用法为：


public class GetListEntity : IEntityFilter<Entity>
{
   [EqualPropertyFilter(nameof(Entity.Property))]
   property int QueryProperty { get; set }
}

public class Entity
{
   property int Property { get; set }
}

public async Task<ActionResult> List(GetListEntity getListEntity)
{
   var restrictions = getListEntity.ListRestrictions();
   nonTrackedQueryableEntities = this.dbContext.Set<Entity>()
                                               .AsNoTracking();

   var expectedEntity = restrictions.Aggregate((sr, nr) => sr.And(nr));
   var expectedNonTrackedQueryableEntities = nonTrackedQueryableEntities .Where(expectedEntity);

   // I will get the total first because the API was meant to paginate the responses.
   var total = await expectedNonTrackedQueryableEntities.CountAsync();
}

并且要丢弃，如果我聚合表达式列表的非动态表达式，ORM 工作正常，当我使用动态表达式时，我在开始时遇到异常。

我找到了一个解决方法，在扩展方法中更改了这一行：


var parameterExpression = Expression.Parameter(entityType, "e");

对于这个：


var parameterExpression = Expression.Parameter(entityType, $"{entityType.Name}{entityFilter.GetHashCode()}");

我想知道为什么会发生这种情况，也许还有其他解决方法。我在任何 Github 存储库中打开一个线程之前发帖在这里，因为我仍然很好奇是否是我的错，因为在途中丢失了某些东西或错误。

Answer 1

从解释中可以清楚地看出，动态构建的谓词的 ParameterExpression 存在一些问题。最后它在使用的自定义表达式扩展方法之一中。

虽然从技术上讲它可以被认为是 ORM bug/issue，但它们在表达式树转换过程中必须解决非常复杂的事情，因此我们必须宽容并尽可能修复我们的代码。

构建动态查询表达式树时需要注意一些重要事项。

首先，使用的 ParameterExpression 的名称并不重要 - 它们由 reference 标识。只要所有参数都是由其他表达式正确引用的单独实例，就可以让所有参数都具有相同的名称（C# 编译器不允许您在编译时创建的名称）。

其次，一些在创建用于编译和执行为代码的表达式树时有意义的东西（比如在 LINQ to Objects 中）对于应该转换为其他东西的表达式树来说并不好（它们是有效的，但是使转换更加困难并导致 bugs/issues)。具体来说（导致问题的原因）是“调用”lambda 表达式。是的，有一个专用的 Expression.Invoke，但它导致几乎所有 IQueryable 实现都出现问题，因此最好通过“内联”它来模拟它，这意味着用实际替换主体内的参数实例表达式。

这是你的 ExpressionExtensions class 应用上述原则的修改版本：


public static partial class ExpressionExtensions
{
    public static Expression<Func<T, bool>> And<T>(this Expression<Func<T, bool>> left, Expression<Func<T, bool>> right)
        => Combine(left, right, ExpressionType.AndAlso);

    public static Expression<Func<T, bool>> Or<T>(this Expression<Func<T, bool>> left, Expression<Func<T, bool>> right)
        => Combine(left, right, ExpressionType.OrElse);

    private static Expression<Func<T, bool>> Combine<T>(Expression<Func<T, bool>> left, Expression<Func<T, bool>> right, ExpressionType type)
    {
        if (left is null) return right;
        if (right is null) return left;
        bool constValue = type == ExpressionType.AndAlso ? false : true;
        if ((left.Body as ConstantExpression)?.Value is bool leftValue)
            return leftValue == constValue ? left : right;
        if ((right.Body as ConstantExpression)?.Value is bool rightValue)
            return rightValue == constValue ? right : left;
        return Expression.Lambda<Func<T, bool>>(Expression.MakeBinary(type,
            left.Body, right.Invoke(left.Parameters[0])),
            left.Parameters);
    }

    public static Expression Invoke<T, TResult>(this Expression<Func<T, TResult>> source, Expression arg)
        => source.Body.ReplaceParameter(source.Parameters[0], arg);
}

它使用以下小帮手进行参数替换：

public static partial class ExpressionExtensions
{
    public static Expression ReplaceParameter(this Expression source, ParameterExpression parameter, Expression value)
        => new ParameterReplacer { Parameter = parameter, Value = value }.Visit(source);

    class ParameterReplacer : ExpressionVisitor
    {
        public ParameterExpression Parameter;
        public Expression Value;
        protected override Expression VisitParameter(ParameterExpression node)
            => node == Parameter ? Value : node;
    }
}

如评论中所确认，这解决了问题。

现在，无关，但作为奖励。对应该编译的表达式有意义的另一件事是 ConstantExpressions 的用法——它们被评估一次，然后在可能的许多地方使用。

然而，对于应该转换为 SQL 或类似的表达式树，使用 ConstantExpressions 会使每个查询不同，因此不可缓存。出于性能原因，最好使用被视为变量的表达式类型，从而允许缓存转换并参数化生成的SQL查询，因此客户端和数据库查询处理器可以重复使用“已编译”query/execution 计划。

这样做很容易。它不需要更改谓词的类型或您生成的方式。您只需将 ConstantExpression 替换为 ConstantExpression 的 member (property/field)。在你的情况下，这是更换

的问题

var propertyValue = pi.GetValue(entityFilter);

和

var propertyValue = Expression.Property(Expression.Constant(entityFilter), pi);

当然还有调整signatures/implementation（如果特定的表达式类型对方法不是必需的，通常尽量不要使用它们），例如

FilterByPropertyAttribute class:

public abstract Expression GetExpression(ParameterExpression parameter, Expression value);

NonNullableValuePropertyFilterAttribute class:


public override Expression GetExpression(ParameterExpression parameter, Expression value)
{
    var property = this.GetPropertyExpression(parameter);
    if (value.Type != property.Type)
        value = Expression.Convert(value, property.Type);
    return this.GetExpression(property, value);
}

protected abstract Expression GetExpression(MemberExpression member, Expression value);

EqualPropertyFilterAttribute class:

protected override Expression GetExpression(MemberExpression member, Expression value)
    => Expression.Equal(member, value);

包括用法在内的所有其他内容都保持不变。但是结果将是很好的参数化查询，就好像它是在编译时创建的一样。

为什么运行时表达式会导致 Entity Framework Core 5 的缓存发生冲突？

Why runtime Expressions cause collisions on the Cache of Entity Framework Core 5?

c#

mysql

reflection

entity-framework-core

.net-core