Indices.GetAsync fails with UnexpectedTransportException if index uses a non-standard token filter

[imbelousov]

Elastic.Clients.Elasticsearch version: 8.17.1

Elasticsearch version: 8.17.4

.NET runtime version: 8

Operating system version: Windows 10

Description of the problem including expected versus actual behavior:
The client throws UnexpectedTransportException when deserializing the ES response, if the index contains a custom token filter (i.e., when "type" does not match any predefined value).

Steps to reproduce:

1. Install any plugin that includes a non-standard token filter.
2. Create an index manually and specify the custom filter in settings.index.analysis.filter section.
   Example:

{
  "my_index": {
    "settings": {
      "index": {
        "analysis": {
          "filter": {
            "my_custom_filter": {
              "type": "my_custom_filter"
            }
          }
        }
      }
    }
  }
}

3. Attempt to read the index settings using Indices.GetAsync API.

Expected behavior
Deserialization should not fail. The custom filter should be deserialized into a CLR object.

I found a workaround, although it feels a bit like a hack:

1. Create a custom extension of ElasticsearchClientSettings. This allows customization of the RequestResponseSerializer with a custom converter.

public class CustomElasticsearchClientSettings : ElasticsearchClientSettings
{
	public CustomElasticsearchClientSettings(NodePool nodePool,
		SourceSerializerFactory? sourceSerializer = null,
		SourceSerializerFactory? requestResponseSerializer = null)
		: base(nodePool, sourceSerializer)
	{
		if (requestResponseSerializer != null)
			UseThisRequestResponseSerializer = requestResponseSerializer(UseThisRequestResponseSerializer, this);
	}
}

2. Define a basic model for non-standard token filters.

public class CustomTokenFilter(string type)
	: ITokenFilter
{
	public string? Type { get; } = type;
}

3. Add a converter that deserializes an unknown filter into the common model.

public class CustomTokenFilterConverter : JsonConverter<ITokenFilter>
{
	public override ITokenFilter? Read(ref Utf8JsonReader reader,
		Type typeToConvert,
		JsonSerializerOptions options)
	{
		using var doc = JsonDocument.ParseValue(ref reader);
		var root = doc.RootElement;
		var type = root.GetProperty("type").GetString();
		return new CustomTokenFilter(type);
	}

	public override void Write(Utf8JsonWriter writer, ITokenFilter value, JsonSerializerOptions options)
	{
		writer.WriteStartObject();
		writer.WriteString("type", filter.Type);
		writer.WriteEndObject();
	}
}

4. Register the custom converter.

var settings = new CustomElasticsearchClientSettings(nodePool, requestResponseSerializer: (serializer, settings) =>
{
	serializer.TryGetJsonSerializerOptions(out var options);
	options.Converters.Add(new CustomTokenFilterConverter());
	return serializer;
});
var client = new ElasticsearchClient(settings);

[flobernd]

Hi @imbelousov , thanks for the detailed bug report and the workaround.

This indeed is a known issue where the client does not correctly handle non-exhaustive variants. There will be better handling for these cases in the future 🙂