otel extension initial draft

advocacy_docs/pg_extensions/extensionrefs.json

@@ -128,5 +128,6 @@
     "plv8":"https://github.com/plv8/plv8",
     "aidb":"https://www.enterprisedb.com/docs/edb-postgres-ai/ai-accelerator/pipelines-overview/",
     "pgfs":"https://www.enterprisedb.com/docs/edb-postgres-ai/ai-accelerator/pgfs/",
-    "bluefin":"https://www.enterprisedb.com/docs/pg_extensions/advanced_storage_pack/"
+    "bluefin":"https://www.enterprisedb.com/docs/pg_extensions/advanced_storage_pack/",
+    "edb_otel": "https://www.enterprisedb.com/docs/pg_extensions/otel/"
 }

advocacy_docs/pg_extensions/index.mdx.in

@@ -18,6 +18,7 @@ navigation:
     - sqlprofiler
     - system_stats
     - wal2json
+    - otel
     - "#EDB Postgres Advanced Server only"
     - spl_check
     - edb_job_scheduler

advocacy_docs/pg_extensions/otel/configuring.mdx

@@ -0,0 +1,88 @@
+---
+title: Configuring EDB OTEL
+navTitle: Configuring
+---
+
+The following configuration options are available with edb_otel.
+
+## Permissions
+
+No permissions changes are required for edb_otel.
+
+## Prerequisites
+
+Before using edb_otel to export metrics, you must set up an OpenTelemetry collector endpoint that edb_otel can communicate with.  For detailed information on how to set up a quick test of the Opentelemetry endpoint using docker, see [OTLP Exporter Example](https://github.com/open-telemetry/opentelemetry-cpp/tree/main/examples/otlp).
+
+## GUCs
+
+The following GUC variables are available in edb_otel:
+
+| Variable                       | Description | Unit | Default | Minimum value |                                                                                                                                                                                                                                                |
+|--------------------------------|-------------|------|---------|---------------|
+| `edb_otel.edb_otel_endpoint`   | Specifies the OTEL endpoint URL. | | | |
+| `edb_otel.interval`            | Interval between two consecutive exports. It translates to the opentelemetry metric reader parameter `exportIntervalMillis`.| Milliseconds | 2 seconds | 1 second|  
+| `edb_otel.timeout`             | Specifies how long the export can run before it's cancelled. It translates to the opentelemetry metric reader parameter `exportTimeoutMillis`.| Milliseconds | 500 milliseconds | 100 milliseconds|
+| `edb_otel.worker_nap_time`     | Duration of background worker sleep time before it checks for metrics in the queue and processes them. | Milliseconds | 2 seconds | 1 second |
+| `edb_otel.metrics_queue_size`  | Interval between two consecutive exports. This is the maximum size of the metrics queue. If the queue reaches this size, any subsequent metrics are dropped until the queue is cleared while processing the metrics inside it.| MB | 100 | 100|
+
+## Configuring to be used by an SQL interface
+
+To use the edb_otel extension from a SQL interface, use the following steps:
+
+In `postgresql.conf`:
+
+1. Set the parameter `edb_otel.edb_otel_endpoint` to point to the collector endpoint.  Example: 
+```bash
+edb_otel.edb_otel_endpoint='localhost:4317'
+```
+2. Add `edb_otel` to the `shared_preload_libraries`. Example: 
+```bash
+shared_preload_libraries = '$libdir/dbms_pipe,$libdir/edb_gen,$libdir/dbms_aq,$libdir/edb_otel'
+```
+## Configuring to be used within C code
+
+To use the edb_otel extension from your C code, use the following steps:
+
+1. Ensure the `edb_otel_ext.h` file is within reach of your extension at build time. The file is placed in `$prefix/include/server/extension/edb_otel/` when the edb_otel dev/devel package is installed. Your Makefile must add `pg_config --includedir-server` to its include search paths, so your extension can use it as:
+```C
+#include "extension/edb_otel/edb_otel_ext.h"
+```
+This header contains the signature of the function that is used to send metrics to the OTEL endpoint.
+
+2. When the edb_otel extension is loaded, it places a function pointer in the rendezvous variable identified by `edb_otel_metric_func`. So the next step is to get that function pointer:
+```C
+edb_otel_metric_func_type * p_edb_otel_metric =
+    (edb_otel_metric_func_type *) find_rendezvous_variable("edb_otel_metric_func");
+```
+The function pointer may be stored in a session-long variable, but care should be taken in case edb_otel gets loaded after the calling code.
+
+Since the instrumented code must continue to work even when edb_otel isn't in use, if the function pointer returns NULL, no error should be raised, and the function must not be called.
+
+For example, instrumenting pg_stat_statements to send the number of plans may look similar to this:
+
+```C
+if (*p_edb_otel_metric) {
+    char val[100] = {0};
+    sprintf(val, "%ld", e->counters.calls[PGSS_PLAN]);
+    (*p_edb_otel_metric)("pg_stat_statements",
+                         "plans",
+                         EDB_OTEL_GAUGE,
+                         val,
+                         labels);
+}
+```
+
+The `edb_otel_metric` function has the following parameters:
+
+| Parameter(s)       | Input or output | Description                                                                                                                                                                       |
+|--------------------|-----------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `meter_name`       | Input           | A name to identify a group of metrics. |
+| `metric_name`      | Input           | A name to identify an actual measurement of the metric. |  
+| `metric_type`      | Input           | Identified by one of the enums in `edb_otel_ext.h`.|
+| `measurement`      | Input           | A string value.|
+| `labels`           | Input           | A jsonb string representing additional metadata to be associated with the metric, for example, attaching the query text to the measurement of `pg_stat_statements`.|
+
+!!!Note
+Care should be taken when calling the function that sends metrics. It's important to avoid using it in hot paths and while holding locks, since it currently enters third party code.  In the future, it may still perform non-trivial work, if only pushing data through a shm_mq to a background worker.
+!!!
+For a detailed example of this C API usage where `pg_stat_statements` was instrumented to send all metrics through OTEL, see this [example](https://github.com/EnterpriseDB/opentelemetry-work/blob/arthur/pg17/contrib/pg_stat_statements/pg_stat_statements.c).

advocacy_docs/pg_extensions/otel/index.mdx

@@ -0,0 +1,20 @@
+---
+title: 'EDB OTEL'
+indexCards: simple
+navigation:
+  - rel_notes
+  - installing
+  - uninstalling
+  - configuring
+  - using
+  - upgrading
+directoryDefaults:
+  product: EDB OTEL
+---
+
+EDB OTEL (edb_otel) is an extension that allows metrics to be sent through the [OpenTelemetry](https://opentelemetry.io) (OTEL) protocol.  It communicates with an OTEL endpoint and sends metrics to it.  Other extensions can use APIs to pass metrics to edb_otel, which will in turn redirect to the OTEL endpoint.
+
+
+edb_otel includes a background worker process which is responsible for managing the metric instruments and exporting the metrics to the opentelemetry metrics collector. The worker handles calls to the OpenTelemetry C++ SDK, preventing them from blocking the main backend process. Metric data, reported by other backends via a shared memory queue, is periodically retrieved and processed by the worker.
+
+Subsequent EDB extension releases will leverage edb_otel APIs for centralized routing OTEL metrics. 

advocacy_docs/pg_extensions/otel/installing.mdx

@@ -0,0 +1,92 @@
+---
+title: "Installing EDB OTEL"
+navTitle: Installing
+---
+
+To install and enable the edb_otel extension, use the following steps.
+
+The edb_otel extension is only available for PostgreSQL version 17 on RHEL 9.
+
+## Installing the package
+
+### Prerequisites
+
+Before you begin the installation process:
+
+- Install Postgres. See:
+  - [Installing EDB Postgres Advanced Server](/epas/latest/installing/)
+
+  - [Installing PostgreSQL](https://www.postgresql.org/download/) 
+
+  - [Installing EDB Postgres Extended Server](/pge/latest/installing/)
+
+- Set up the repository.
+
+  Setting up the repository is a one-time task. If you have already set up your repository, you don't need to perform this step.
+
+  To set up the repository, go to [EDB repositories](https://www.enterprisedb.com/repos-downloads) and follow the instructions provided there.
+
+
+### Install the package
+
+The syntax to install edb_otel on EDB Postgres Advanced Server and EDB Postgres Extended is:
+
+```shell
+# For EDB Postgres Advanced Server:
+sudo <package-manager> -y install edb-as<postgres_version>-edb_otel
+
+# For EDB Postgres Extended:
+sudo <package-maanger> -y install edb-postgresextended<postgres_version>-edb_otel
+```
+
+Where:
+
+- `<package-manager>` is the package manager used with your operating system:
+
+  | Package manager |             Operating system     |
+  | --------------- | -------------------------------- |
+  | dnf             | RHEL 9 and derivatives           |
+
+
+- `<postgres_version>` is the version of Postgres you're using. 
+
+For example, to install the latest version of edb_otel for EDB Postgres Advanced Server 17 on a RHEL 9 platform:
+
+```shell
+sudo dnf -y install edb-as17-edb_otel
+```
+
+The syntax to install edb_otel on PostgreSQL is:
+
+```shell
+# For RHEL 9 and its derivatives:
+sudo dnf -y install edb_otel_<postgres_version>
+```
+
+Where `<postgres_version>` is the version of PostgreSQL you're using.
+
+
+## Enabling the extension
+
+To enable the extension:
+
+1. Edit the `postgresql.conf` file, modifying the `shared_preload_libraries` parameter:
+
+    ```ini
+    shared_preload_libraries = '$libdir/edb_otel'
+    ```
+
+1. Restart the Postgres server.
+
+1. Create the edb_otel extension in your database with the following command: 
+
+   ```sql
+   # Connect to your database using psql
+   psql -d postgres -U enterprisedb
+   # Create the extension
+   CREATE EXTENSION edb_otel;
+   ```
+
+
+
+

advocacy_docs/pg_extensions/otel/rel_notes/edb_otel_1.0.0_rel_notes.mdx

@@ -0,0 +1,11 @@
+---
+title: Release notes for EDB OTEL version 1.0.0
+navTitle: "Version 1.0.0"
+---
+Release date: 28 March 2025
+
+This release of edb_otel includes:
+
+| Type        | Description                                                                                                                                                                                                                         |
+|-------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| Release | This is the initial release.|

advocacy_docs/pg_extensions/otel/rel_notes/index.mdx

@@ -0,0 +1,12 @@
+---
+title: 'EDB OTEL release notes'
+navTitle: "Release notes"
+indexCards: none
+---
+
+The edb_otel documentation describes the latest version of edb_otel, including minor releases and patches. These release notes cover what was new in each release. For new functionality introduced in a minor or patch release, there are also indicators in the content about the release that introduced the feature.
+
+|             Version              | Release Date |
+| -------------------------------- | ------------ |
+| [1.0.0](edb_otel_1.0.0_rel_notes) | 04 March 2025  |
+

advocacy_docs/pg_extensions/otel/uninstalling.mdx

@@ -0,0 +1,25 @@
+---
+title: "Uninstalling EDB OTEL"
+navTitle: Uninstalling
+---
+
+To uninstall edb_otel:
+
+1. Drop the edb_otel extension: 
+
+   ```sql
+   # Connect to your database using psql
+   psql -d postgres -U enterprisedb
+   # Drop the extension
+   DROP EXTENSION edb_otel;
+   ```
+
+1. Edit the `postgresql.conf` file on the server you want to remove the extension, deleting the `shared_preload_libraries` parameter line.
+
+   In Linux, remove: 
+
+   ```ini
+   shared_preload_libraries = `$libdir/edb_otel`
+   ```
+
+1. Restart the Postgres database server to apply the changes.

advocacy_docs/pg_extensions/otel/upgrading.mdx

@@ -0,0 +1,12 @@
+---
+title: "Upgrading EDB OTEL"
+navTitle: Upgrading
+---
+
+To upgrade edb_otel, use the `ALTER EXTENSION` command: 
+
+For example, to upgrade from version 1.0 to version 1.1 (not yet released):
+```
+# running as superuser
+ALTER EXTENSION edb_otel UPDATE TO '1.1';
+```