Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce gen_ai.request.seed as needed by cohere #1710

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .chloggen/genai-request-seed.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
change_type: 'enhancement'
component: gen_ai
note: Introduce gen_ai.request.seed and deprecated gen_ai.openai.request.seed

issues: [1710]
11 changes: 10 additions & 1 deletion docs/attributes-registry/gen-ai.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
- [GenAI Attributes](#genai-attributes)
- [OpenAI Attributes](#openai-attributes)
- [Deprecated GenAI Attributes](#deprecated-genai-attributes)
- [Deprecated OpenAI GenAI Attributes](#deprecated-openai-genai-attributes)

## GenAI Attributes

Expand All @@ -22,6 +23,7 @@ This document defines the attributes used to describe telemetry in the context o
| <a id="gen-ai-request-max-tokens" href="#gen-ai-request-max-tokens">`gen_ai.request.max_tokens`</a> | int | The maximum number of tokens the model generates for a request. | `100` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="gen-ai-request-model" href="#gen-ai-request-model">`gen_ai.request.model`</a> | string | The name of the GenAI model a request is being made to. | `gpt-4` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="gen-ai-request-presence-penalty" href="#gen-ai-request-presence-penalty">`gen_ai.request.presence_penalty`</a> | double | The presence penalty setting for the GenAI request. | `0.1` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="gen-ai-request-seed" href="#gen-ai-request-seed">`gen_ai.request.seed`</a> | int | Requests with same seed value more likely to return same result. | `100` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="gen-ai-request-stop-sequences" href="#gen-ai-request-stop-sequences">`gen_ai.request.stop_sequences`</a> | string[] | List of sequences that the model will use to stop generating further tokens. | `["forest", "lived"]` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="gen-ai-request-temperature" href="#gen-ai-request-temperature">`gen_ai.request.temperature`</a> | double | The temperature setting for the GenAI request. | `0.0` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="gen-ai-request-top-k" href="#gen-ai-request-top-k">`gen_ai.request.top_k`</a> | double | The top_k sampling setting for the GenAI request. | `1.0` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
Expand Down Expand Up @@ -88,7 +90,6 @@ Thie group defines attributes for OpenAI.
| Attribute | Type | Description | Examples | Stability |
|---|---|---|---|---|
| <a id="gen-ai-openai-request-response-format" href="#gen-ai-openai-request-response-format">`gen_ai.openai.request.response_format`</a> | string | The response format that is requested. | `json` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="gen-ai-openai-request-seed" href="#gen-ai-openai-request-seed">`gen_ai.openai.request.seed`</a> | int | Requests with same seed value more likely to return same result. | `100` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="gen-ai-openai-request-service-tier" href="#gen-ai-openai-request-service-tier">`gen_ai.openai.request.service_tier`</a> | string | The service tier requested. May be a specific tier, default, or auto. | `auto`; `default` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="gen-ai-openai-response-service-tier" href="#gen-ai-openai-response-service-tier">`gen_ai.openai.response.service_tier`</a> | string | The service tier used for the response. | `scale`; `default` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="gen-ai-openai-response-system-fingerprint" href="#gen-ai-openai-response-system-fingerprint">`gen_ai.openai.response.system_fingerprint`</a> | string | A fingerprint to track any eventual change in the Generative AI environment. | `fp_44709d6fcb` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
Expand Down Expand Up @@ -122,3 +123,11 @@ Describes deprecated `gen_ai` attributes.
| <a id="gen-ai-prompt" href="#gen-ai-prompt">`gen_ai.prompt`</a> | string | Deprecated, use Event API to report prompt contents. | `[{'role': 'user', 'content': 'What is the capital of France?'}]` | ![Deprecated](https://img.shields.io/badge/-deprecated-red)<br>Removed, no replacement at this time. |
| <a id="gen-ai-usage-completion-tokens" href="#gen-ai-usage-completion-tokens">`gen_ai.usage.completion_tokens`</a> | int | Deprecated, use `gen_ai.usage.output_tokens` instead. | `42` | ![Deprecated](https://img.shields.io/badge/-deprecated-red)<br>Replaced by `gen_ai.usage.output_tokens` attribute. |
| <a id="gen-ai-usage-prompt-tokens" href="#gen-ai-usage-prompt-tokens">`gen_ai.usage.prompt_tokens`</a> | int | Deprecated, use `gen_ai.usage.input_tokens` instead. | `42` | ![Deprecated](https://img.shields.io/badge/-deprecated-red)<br>Replaced by `gen_ai.usage.input_tokens` attribute. |

## Deprecated OpenAI GenAI Attributes

Describes deprecated `gen_ai.openai` attributes.

| Attribute | Type | Description | Examples | Stability |
|---|---|---|---|---|
| <a id="gen-ai-openai-request-seed" href="#gen-ai-openai-request-seed">`gen_ai.openai.request.seed`</a> | int | Deprecated, use `gen_ai.request.seed`. | `100` | ![Deprecated](https://img.shields.io/badge/-deprecated-red)<br>Replaced by `gen_ai.request.seed` attribute. |
2 changes: 1 addition & 1 deletion docs/gen-ai/openai.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ attributes and ones specific the OpenAI.
| [`gen_ai.request.model`](/docs/attributes-registry/gen-ai.md) | string | The name of the GenAI model a request is being made to. [2] | `gpt-4` | `Required` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| [`error.type`](/docs/attributes-registry/error.md) | string | Describes a class of error the operation ended with. [3] | `timeout`; `java.net.UnknownHostException`; `server_certificate_invalid`; `500` | `Conditionally Required` if the operation ended in an error | ![Stable](https://img.shields.io/badge/-stable-lightgreen) |
| [`gen_ai.openai.request.response_format`](/docs/attributes-registry/gen-ai.md) | string | The response format that is requested. | `json` | `Conditionally Required` if the request includes a response_format | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| [`gen_ai.openai.request.seed`](/docs/attributes-registry/gen-ai.md) | int | Requests with same seed value more likely to return same result. | `100` | `Conditionally Required` if the request includes a seed | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| [`gen_ai.openai.request.seed`](/docs/attributes-registry/gen-ai.md) | int | Deprecated, use `gen_ai.request.seed`. | `100` | `Conditionally Required` if the request includes a seed | ![Deprecated](https://img.shields.io/badge/-deprecated-red)<br>Replaced by `gen_ai.request.seed` attribute. |
| [`gen_ai.openai.request.service_tier`](/docs/attributes-registry/gen-ai.md) | string | The service tier requested. May be a specific tier, default, or auto. | `auto`; `default` | `Conditionally Required` [4] | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| [`gen_ai.openai.response.service_tier`](/docs/attributes-registry/gen-ai.md) | string | The service tier used for the response. | `scale`; `default` | `Conditionally Required` [5] | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| [`server.port`](/docs/attributes-registry/server.md) | int | GenAI server port. [6] | `80`; `8080`; `443` | `Conditionally Required` If `server.address` is set. | ![Stable](https://img.shields.io/badge/-stable-lightgreen) |
Expand Down
11 changes: 11 additions & 0 deletions model/gen-ai/deprecated/registry-deprecated.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -28,3 +28,14 @@ groups:
deprecated: "Removed, no replacement at this time."
brief: "Deprecated, use Event API to report completions contents."
examples: ["[{'role': 'assistant', 'content': 'The capital of France is Paris.'}]"]
- id: registry.gen_ai.openai.deprecated
type: attribute_group
brief: Describes deprecated `gen_ai.openai` attributes.
display_name: Deprecated OpenAI GenAI Attributes
attributes:
- id: gen_ai.openai.request.seed
stability: experimental
type: int
deprecated: Replaced by `gen_ai.request.seed` attribute.
brief: "Deprecated, use `gen_ai.request.seed`."
examples: [100]
10 changes: 5 additions & 5 deletions model/gen-ai/registry.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,11 @@ groups:
note: >
In some GenAI systems the encoding formats are called embedding types.
Also, some GenAI systems only accept a single format per request.
- id: gen_ai.request.seed
stability: experimental
type: int
brief: Requests with same seed value more likely to return same result.
examples: [100]
- id: gen_ai.response.id
stability: experimental
type: string
Expand Down Expand Up @@ -168,11 +173,6 @@ groups:
brief: >
Thie group defines attributes for OpenAI.
attributes:
- id: gen_ai.openai.request.seed
stability: experimental
type: int
brief: Requests with same seed value more likely to return same result.
examples: [100]
- id: gen_ai.openai.request.response_format
stability: experimental
type:
Expand Down
Loading