Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"Container does not exist" error when getting PIDs for containerd metadata collector. #498

Open
tobz opened this issue Feb 13, 2025 · 0 comments
Labels
area/core Core functionality, event model, etc. type/investigation A potential issue that requires further investigation to quantify and categorize.

Comments

@tobz
Copy link
Member

tobz commented Feb 13, 2025

I noticed an error log in staging complaining about a container not existing when attempting to get the PIDs for it:

yyyy-MM-dd HH:mm:ss UTC | DATAPLANE | ERROR | (lib/saluki-env/src/workload/collectors/containerd.rs:157) | namespace:k8s.io,container_id:*,error:failed to make gRPC request: status: Unknown, message: "runc did not terminate successfully: exit status 1: container does not exist : unknown", details: [], metadata: MetadataMap { headers: {"content-type": "application/grpc"} } | Error getting PIDs for container.

Thinking about it, it seems perfectly reasonable that we're notified about a new container, but it exits fast enough that we can't get the PIDs for it first... but we should maybe look at the code path involved to make sure that we're not missing some other non-obvious failure mode.

We might also consider hiding these error messages (why complain about not being able to get PIDs for a container that indeed exited?) although it's also not clear if there's a good status field to key off of instead of having to manually parse the error message, etc.

@tobz tobz added area/core Core functionality, event model, etc. type/investigation A potential issue that requires further investigation to quantify and categorize. labels Feb 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/core Core functionality, event model, etc. type/investigation A potential issue that requires further investigation to quantify and categorize.
Projects
None yet
Development

No branches or pull requests

1 participant