Fatal error. Releasing the double mapped memory failed

[KaJIbI4]

Hello!

There was such a problem today:

Fatal error. Releasing the double mapped memory failed
Repeat 2 times:
   at System.Reflection.Emit.RuntimeAssemblyBuilder.CreateDynamicAssembly(System.Runtime.CompilerServices.ObjectHandleOnStack, System.Reflection.NativeAssemblyNameParts*, System.Configuration.Assemblies.AssemblyHashAlgorithm, System.Reflection.Emit.AssemblyBuilderAccess, System.Runtime.CompilerServices.ObjectHandleOnStack)
   at System.Reflection.Emit.RuntimeAssemblyBuilder.CreateDynamicAssembly(System.Runtime.Loader.AssemblyLoadContext, System.Reflection.AssemblyName, System.Reflection.Emit.AssemblyBuilderAccess)
   at System.Reflection.Emit.RuntimeAssemblyBuilder..ctor(System.Reflection.AssemblyName, System.Reflection.Emit.AssemblyBuilderAccess, System.Runtime.Loader.AssemblyLoadContext, System.Collections.Generic.IEnumerable`1<System.Reflection.Emit.CustomAttributeBuilder>)
   at System.Reflection.Emit.AssemblyBuilder.DefineDynamicAssembly(System.Reflection.AssemblyName, System.Reflection.Emit.AssemblyBuilderAccess, System.Collections.Generic.IEnumerable`1<System.Reflection.Emit.CustomAttributeBuilder>, System.Reflection.Assembly)
   at Gridify.Syntax.GridifyTypeBuilder.GetTypeBuilder()
   at Gridify.QueryBuilders.LinqQueryBuilder`1[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].BuildQueryAccordingToValueType(System.Linq.Expressions.Expression, System.Linq.Expressions.ParameterExpression, System.Object, Gridify.Syntax.SyntaxNode, Gridify.Syntax.ValueExpressionSyntax)
   at Gridify.QueryBuilders.BaseQueryBuilder`2[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].BuildQuery(System.Linq.Expressions.Expression, System.Linq.Expressions.ParameterExpression, Gridify.Syntax.ValueExpressionSyntax, Gridify.Syntax.SyntaxNode, System.Func`2<System.String,System.Object>)
   at Gridify.QueryBuilders.BaseQueryBuilder`2[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].ConvertBinaryExpressionSyntaxToQuery(Gridify.Syntax.BinaryExpressionSyntax)
   at Gridify.QueryBuilders.BaseQueryBuilder`2[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].BuildQuery(Gridify.Syntax.ExpressionSyntax, Boolean)
   at Gridify.QueryBuilders.BaseQueryBuilder`2[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].BuildQuery(Gridify.Syntax.ExpressionSyntax, Boolean)
   at Gridify.QueryBuilders.BaseQueryBuilder`2[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].BuildQuery(Gridify.Syntax.ExpressionSyntax, Boolean)
   at Gridify.QueryBuilders.BaseQueryBuilder`2[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].BuildQuery(Gridify.Syntax.ExpressionSyntax, Boolean)
   at Gridify.GridifyExtensions.CreateQuery[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](Gridify.Syntax.SyntaxTree, Gridify.IGridifyMapper`1<System.__Canon>)
   at Gridify.QueryBuilder`1[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].ConvertConditionToExpression(System.String)
at System.Linq.Enumerable+SelectListIterator`2[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Fill(System.ReadOnlySpan`1<System.__Canon>, System.Span`1<System.__Canon>, System.Func`2<System.__Canon,System.__Canon>)
   at System.Linq.Enumerable+SelectListIterator`2[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].ToList()
   at Gridify.QueryBuilder`1[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].BuildFilteringExpression()

The case is happening in docker on Linux, .net 9, a MediatR, an Automapper, and Fluent validation are used, and the call location code is presented below:

     Must( f =>
           {
               try
               {
                   if ( string.IsNullOrEmpty( f ) )
                       return true;
                   _ = new QueryBuilder< MyEntity >().AddCondition( f ).BuildCompiled();
                   return true;
               }
               catch
               {
                   return false;
               }
           } ).
     WithMessage( "Incorrect filter or parameter values are beyond acceptable limits." );

The entire service crashed, of course we didn't get into the try catch block :(

The MyEntity entity has many types of fields, but they were filtered in the query only by

public string? MyField {get; set;}

Unfortunately, the exact filter has not been preserved in the logs, but it consists of blocks of the same type using the OR operator:

MyField=Text_12345|MyField=Text_21231|...

I understand that this validation option is slightly crooked, but it did a good job with incorrectly entered filters.We have not been able to reproduce anything similar with any filter options.
Perhaps you've come across something like this, or do you have any ideas?

[alirezanet]

Hi @KaJIbI4,
Interesting 🤔 … I haven’t seen this issue before. It would be great if you could reproduce it so we can gather more data on the root cause. Maybe try running your tests in parallel? There’s a slight chance it could be related to concurrent query modifications or usage or changing the GlobalConfigurations from different threads... Let me know what you find!

[KaJIbI4]

The initial error may not be related to the real problem, it's just that the process ran out of memory, and apparently it happened that way.
It looks like a memory leak. Here's a simplified example, but it reproduces 1-in-1 behavior without separate requests across all services. With an empty filter, everything is OK, the service lives at a certain memory level under load. In the presence of filters, it is through gridify that the constant growth of unmanaged memory appears.

I've tried different versions and the latest ones too, current:
.net8
postgres:15.0
"Microsoft.EntityFrameworkCore" Version="8.0.2"
"Npgsql.EntityFrameworkCore.PostgreSQL" Version="8.0.2"
"AutoMapper" Version="13.0.1"
"Gridify" Version="2.14.1"
"Gridify.EntityFramework" Version="2.14.1"
"MediatR" Version="12.2.0"

{   
    //-- mem leak with Filter=Name=abc --
    //-- no mem leak with empty Filter --
    var query = new GridifyQuery { Filter = request.Filter, OrderBy = request.OrderBy, Page = request.Page, PageSize = request.PageSize };
    var y = new Paging< Position >();
    for ( var i = 0; i < 10000; i++ )
    {
        y = await context.Positions.GridifyAsync( query, cancellationToken );
    }
    return new Paging< PositionDto > { Count = y.Count, Data = y.Data.Select( mapper.Map< Position, PositionDto > ) };

    //-- no leak --
    var x = new List< Position >();
    for ( var i = 0; i < 10000; i++ )
    {
        x = await context.Positions.Where( e => e.Name == "abc" ).ToListAsync( cancellationToken );
    }
    return new Paging< PositionDto > { Count = x.Count, Data = x.Select( mapper.Map< Position, PositionDto > ) };
}

If it doesn't bother you, check your environment to see if applying the Filter releases memory after itself.

There are about 10,000 real requests in the photo here. Thanks!

[alirezanet]

Yeah, I checked my own projects, and I don’t have this issue with Gridify. However, In your example, you're creating the GridifyQuery manually, whereas I usually pass the query through my controller’s action parameters. If I find some free time, I’ll test it with a memory profiler to see how it behaves.
But based on my experience (using Gridify for over five years in multiple projects), I have a feeling something else might be causing this issue in your project—though I can’t say for sure, especially because last year we had a lot of changes and new features.