Asserting OpenTelemetry data when using DistributedApplicationTestingBuilder

[DamianEdwards]

When using DistributedApplicationTestingBuilder in an integration test, because there is no dashboard process running, there is no OTLP endpoint hosted for the running resources to send OTel data to. We should consider having DistributedApplicationTestingBuilder host an OTLP endpoint and expose its data to allow assertion against it in a test, so that testing based on observing OTel data could be facilitated.

[albigi]

I think such a feature would also help a great deal when debugging test failures with complex invocation chains (which we can most of the times host entirely locally using DAPR and a bunch of lightweight containers such as Redis).

Having OTLP data to support the visualisation of the invocation chain can be an enormous time saver (compared to attaching the debugger to all the processes and step into all of them).

What we're currently trying to do is host the standalone dashboard using the docker image and have the apps send their OTLP data to it via OTLP exporter in the service defaults. we're not yet sure this will work as we're still fighting with it, will update in case we find a working solution, but it would be so useful to have telemetry support during test runs!

[samsp-msft]

This was a feature area kind of discussed by @lmolkova at OTel community day yesterday - using OTel to collect telemetry as part of a test run to be able to compare it and understand if its good or bad in comparison with previous runs.

[lmolkova]

there is also some otel tooling being developed to run validation against collected telemetry. let me find out more.

[martasp]

It is possible to get traces now by adding services.AddHostedService<DashboardWebApplication>(); and then get TracesViewModel from services using GetService<TracesViewModel>(). I remember there were issues with the private fields _app from DashboardWebApplication type that I needed so I used reflection.

Here's the code that works now. You can check out the full PR example dotnet/aspire-samples#361

    [Fact]
    public async Task EndpointCall_ShouldCaptureDistributedTracesAsASnapshot()
    {
        var appHost = await DistributedApplicationTestingBuilder.CreateAsync<Projects.AspireShop_AppHost>();

        appHost.Services.AddDashboardWebApplication();

        await using var app = await appHost.BuildAsync();

        await app.StartAsync();

        var tracesViewModel = app.Services.GetDashboardWebApplication().GetService<TracesViewModel>();

        var httpClient = app.CreateHttpClient("frontend");

        var tracesTree = await tracesViewModel.CaptureTraces(async () =>
        {
            var response = await httpClient.GetAsync("/");
            Assert.Equal(HttpStatusCode.OK, response.StatusCode);
        });

        await app.StopAsync();

        await Verify(tracesTree);
    }

It would be nice to just get TracesViewModel from services like this

    [Fact]
    public async Task EndpointCall_ShouldGenerateDistributedTraces()
    {
        // Arrange
        var appHost = await DistributedApplicationTestingBuilder.CreateAsync<Projects.AspireShop_AppHost>();
        await using var app = await appHost.BuildAsync();
        await app.StartAsync();

        //Getting aspire TracesViewModel service, but tracesViewModel is null?
        var tracesViewModel = app.Services.GetService<TracesViewModel>();
        //How can I get the aspire TracesViewModel service?

        // Act
        var httpClient = app.CreateHttpClient("frontend");
        var response = await httpClient.GetAsync("/");

        // Assert
        Assert.Equal(HttpStatusCode.OK, response.StatusCode);
        Assert.NotEmpty(tracesViewModel.GetTraces().Items);

        await app.StopAsync();
    }

But there are the problems:

• tracesViewModel is null
• How can I get the aspire TracesViewModel service easily?
• Do we want to expose the TracesViewModel service or should it be a raw open telemetry data service?

[STRUDSO]

I bumped into this issues while searching for a way to assert otel and then I later discovered this:

https://github.com/open-telemetry/opentelemetry-dotnet/blob/0c26ce2b3b909223672fe40d22e1f15750dabe0a/src/OpenTelemetry.Exporter.InMemory/InMemoryExporterHelperExtensions.cs#L18

Not sure how to plug this into

DistributedApplicationTestingBuilder

Yet, but will take a stab at it.