Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jonathan.chavez/debugging dist tracing #12234

Open
wants to merge 39 commits into
base: main
Choose a base branch
from
Open

Jonathan.chavez/debugging dist tracing #12234

Show file tree
Hide file tree
Changes from 38 commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
b4d6082
Add main classes for experiments sdk
jjxct Oct 24, 2024
f9e9296
Added more things but don't remember what
jjxct Oct 24, 2024
60f3ba5
Add network calls for main methods
jjxct Oct 28, 2024
d48942d
Add docstring
jjxct Oct 29, 2024
88f05d3
Format code
jjxct Oct 29, 2024
e73a897
Add custom exception classes
jjxct Oct 29, 2024
f8c9ef0
Move code to another directory
jjxct Oct 29, 2024
59577e1
Change experiments module export
jjxct Oct 29, 2024
402d402
Use f strings
jjxct Oct 29, 2024
2c281c5
Decouple running from evaluating
jjxct Oct 29, 2024
0e421da
Change parametrize function to make it simpler
jjxct Nov 4, 2024
173d2ae
Add test file, export the top level classes
Kyle-Verhoog Nov 5, 2024
044d696
fmt
Kyle-Verhoog Nov 5, 2024
ac634fa
Simplify http client, remove stdout printing
Kyle-Verhoog Nov 5, 2024
d29f081
fmt
Kyle-Verhoog Nov 5, 2024
dc119d0
more stdout cleanup, http status code checking
Kyle-Verhoog Nov 5, 2024
f018298
Add feedback from sync
jjxct Nov 8, 2024
351cd7a
Add error handling on tasks
jjxct Nov 10, 2024
2608ba5
fix import
jjxct Nov 10, 2024
5cbfd70
docstring
jjxct Nov 10, 2024
bed1261
Custom Exception classes
jjxct Nov 10, 2024
0cbc487
Merge remote-tracking branch 'origin/main' into jonathan.chavez/llm-e…
Kyle-Verhoog Nov 11, 2024
0928224
handle duration errors
jjxct Nov 11, 2024
cac1476
more stuff
jjxct Nov 12, 2024
436b1b6
Merge branch 'jonathan.chavez/llm-experiments' of github.com:DataDog/…
jjxct Nov 12, 2024
9024e14
support polymorphic i/o
jjxct Nov 14, 2024
a228c30
structure changes
jjxct Dec 6, 2024
b29fa1d
modifications to types
jjxct Dec 9, 2024
738cc07
remove unnecessary comments
jjxct Dec 9, 2024
1059172
fix code quality violations
jjxct Dec 9, 2024
965bdcb
add test comments
jjxct Dec 13, 2024
ba8e807
add error fields on evals
jjxct Jan 13, 2025
8e237e1
Merge branch 'main' of github.com:DataDog/dd-trace-py into jonathan.c…
jjxct Jan 13, 2025
2a73462
encode llm events in utf-8
jjxct Jan 15, 2025
9497ea8
tracing works
jjxct Jan 15, 2025
c05deb8
two buffers temporary
jjxct Jan 27, 2025
1b0800b
switch trace ingestion path
jjxct Jan 30, 2025
b8a2472
tracing stable
jjxct Feb 5, 2025
9ecc788
stabilize errors
jjxct Feb 11, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions ddtrace/llmobs/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,12 @@
from ddtrace.llmobs import LLMObs
LLMObs.enable()
"""

from ._llmobs import LLMObs

from ._experiments import Dataset
from ._experiments import Experiment
from ._experiments import task
from ._experiments import evaluator


__all__ = ["LLMObs"]
__all__ = ["LLMObs", "Dataset", "Experiment", "task", "evaluator"]
4 changes: 3 additions & 1 deletion ddtrace/llmobs/_constants.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,9 @@
INPUT_VALUE = "_ml_obs.meta.input.value"
INPUT_PARAMETERS = "_ml_obs.meta.input.parameters"
INPUT_PROMPT = "_ml_obs.meta.input.prompt"

EXPECTED_OUTPUT = "_ml_obs.meta.input.expected_output"
EXPERIMENT_INPUT = "_ml_obs.meta.input"
EXPERIMENT_OUTPUT = "_ml_obs.meta.output"
OUTPUT_DOCUMENTS = "_ml_obs.meta.output.documents"
OUTPUT_MESSAGES = "_ml_obs.meta.output.messages"
OUTPUT_VALUE = "_ml_obs.meta.output.value"
Expand Down
Loading
Loading