Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pkg/util/log: Add support for OTLP sink #143049

Open
arjunmahishi opened this issue Mar 18, 2025 · 0 comments
Open

pkg/util/log: Add support for OTLP sink #143049

arjunmahishi opened this issue Mar 18, 2025 · 0 comments
Labels
C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-observability

Comments

@arjunmahishi
Copy link
Contributor

arjunmahishi commented Mar 18, 2025

Describe the solution you'd like
Implement an OTLP sink for CockroachDB logging that sends logs in OpenTelemetry format via gRPC (and optionally HTTP/JSON). It should expose configuration parameters (e.g., endpoint, port, auth headers) and integrate with the existing logging framework similar to HTTP and Fluent sinks.

OTLP leverages protocol buffers for efficient binary serialization, reducing payload sizes and enhancing performance compared to text-based formats. Its support for gRPC enables low-latency, high-throughput communication, while HTTP/JSON provides compatibility when needed.

Why we should do this

  • The SRE team is currently migrating Fluent Bit to OTel using the existing Fluent sink. Adding native OTLP support will optimise the logging pipeline further and help standardise the OpenTelemetry ecosystem.
  • Serverless log export (to customers) currently uses Fluent Bit as a log forwarder [ CRDB -> fluentbit -> otel-agent -> customer ]. Adding OTLP support would significantly simplify this pipeline.
  • Self hosted customers can also take advantage of this while setting up their logging pipeline.

References

Jira issue: CRDB-48633

@arjunmahishi arjunmahishi added C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-observability labels Mar 18, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-observability
Projects
None yet
Development

No branches or pull requests

1 participant