cloudwatch_logs properties aren't populated causing CloudWatch Tracer logs to not work

[moritonal]

Hi all

When you look at Traces in AWS CloudWatch, the Log section looks like this:

But if you click through to "View in CloudWatch Logs Insights` then apply the correct log-group it finds the appropriate logs, meaning the tracing header is being correctly set.

The X-Ray docs show that there is an aws field in the protocol of which only the xray part of this is filled in currently.

After investigation the aws field I see there are two fields cloudwatch_logs that are not populated. Luckily I know that an ECS instance has access to the ECS_CONTAINER_METADATA_URI_V4 endpoint.

When called from within an ECS instance in Far-gate, this responds with JSON in the following structure

{
    "Networks": [{
        ...
        "PrivateDNSName": "ip-10-0-157-102.eu-west-2.compute.internal",
        ...
    }],
    "LogOptions": {
        "awslogs-group": "..."
        "awslogs-region": "....",
        "awslogs-stream": "...."
    }
}

This endpoint includes enough information to fill in the cloudwatch_logs.group_name field via the awslogs-group field. I put together a branch here doing this.

cloudwatch_logs however requires two fields. log_group and arn. The problem is that whilst ECS_CONTAINER_METADATA_URI_V4 contains the awslogs-group, it does not contain the arn. It is very likely to be the same as the Labels['com.amazonaws.ecs.cluster'], which I use to calculate it in my branch which is successful.

However, my method for calculating the ARN from the cluster's ARN wouldn't if the logs were in a different account or region as the cluster. So I'm able to fill the Cloudwatch logs correctly with the logs, but this wouldn't work for some scenarios. One of two actions would fix this issue:

• Could the ECS_CONTAINER_METADATA_URI_V4 endpoint be updated to pass through the full ARN of the target log so it could be passed via X-Ray and used in CloudWatch Traces?

• Could the CloudWatch front-end be updated to handle only the log_group by itself? The front-end attempts to read a log-group by name if the arn isn't provided, or use the arn if it is.

Many thanks

Tom

[etiennechabert]

👍 to the fact that this is missing, from my perspective it just needs to be added to the Ruby SDK.

This is for example present in the JAVA SDK: https://github.com/aws/aws-xray-sdk-java/blob/398ec0542964b4d9b38c901385eda4d98da07cd4/aws-xray-recorder-sdk-core/src/main/java/com/amazonaws/xray/AWSXRayRecorder.java#L508-L510

And is allowing a seamless relation between logs and traces:

[etiennechabert]

I drafted this PR, taking example on how it was done for EC2:

#93

Let's see what the X-Ray / ECS teams have to say about it

[srprash]

The change has been release in v0.15.0. https://github.com/aws/aws-xray-sdk-ruby/releases/tag/0.15.0