feat(langchain): auto-instrument with langgraph

[sabrenner]

Adds gated auto-instrumentation for langchain with langgraph

Complications with linking

The primary obstacle was that, unlike with LangGraph, where we had an intermediary function to patch between tasks executed by the graph, we don't have that for LangChain LCEL chains. Their steps are executed in a loop inside their invoke method, which blocks us from jumping in between the steps to make the links.

Additionally, LangChain elements that we trace are sometimes embedded in other Runnable types:

• RunnableBinding, which binds an instance inside of it
• RunnableParallel, which can run multiple Runnables (some of which we might trace) in parallel, which have the parallel items in a steps__ attribute

Something I tried to overcome this was to flatten/flatmap the items of the list of steps to extract these.

To overcome linking between steps, I recorded the instance of each traced Runnable, mapping its ID to its span, and vice versa, to be able to grab instances if needed. Additionally, for each Runnable item in the chain, I marked them as a chain step by adding them to a set of steps to later check against.

Setting links

Link setting is split into setting the input links ("to": "input") and output links ("to": "output")

Input Links

Input links are set by:

1. Identifying if the span represents a step in a chain. If not, set its input link as input --> input from the parent span.
2. If it does represent a step in the chain, find the index of the previous traced step in the chain (the chain instance is grabbed from the span to instance mapping referenced above), and setting it as the output --> input link. If the step contains multiple spans (ie from a RunnableParallel), add all of those spans as links with the same output --> input attribute

The index of the previously traced step in the chain (-1 if not found or is not a chain step) is returned for use in output linkage.

Output Links

Output links are set by:

1. If the span does not represent a step in a chain, or the parent span isn't a chain (ie has a steps attribute), set the output --> output link from the current span onto the parent span.
2. If the span does represent a step a chain, remove all previous span links on the span from the previous traced step, and set the new span link from the current span. We do this overwriting every time because we don't know ahead of time which step in the chain will be the last one we trace, so we have to remove previous span links if we find we need to add a new one.

Checklist

• PR author has checked that all the criteria below are met
• The PR description includes an overview of the change
• The PR description articulates the motivation for the change
• The change includes tests OR the PR description describes a testing strategy
• The PR description notes risks associated with the change, if any
• Newly-added code is easy to change
• The change follows the library release note guidelines
• The change includes or references documentation updates if necessary
• Backport labels are set (if applicable)

Reviewer Checklist

• Reviewer has checked that all the criteria below are met
• Title is accurate
• All changes are related to the pull request's stated goal
• Avoids breaking API changes
• Testing strategy adequately addresses listed risks
• Newly-added code is easy to change
• Release note makes sense to a user of the library
• If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
• Backport labels are set in a manner that is consistent with the release branch maintenance policy

[github-actions]

CODEOWNERS have been resolved as:

ddtrace/contrib/internal/langchain/patch.py                             @DataDog/ml-observability
ddtrace/contrib/internal/langgraph/patch.py                             @DataDog/ml-observability
ddtrace/llmobs/_integrations/base.py                                    @DataDog/ml-observability
ddtrace/llmobs/_integrations/langchain.py                               @DataDog/ml-observability
ddtrace/llmobs/_integrations/langgraph.py                               @DataDog/ml-observability

[pr-commenter]

Benchmarks

Benchmark execution time: 2025-02-06 23:07:56

Comparing candidate commit 2431cae in PR branch sabrenner/langchain-span-linking with baseline commit 1247ac2 in branch ``.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 394 metrics, 2 unstable metrics.

[sabrenner]

open to suggestions on the variable name here 😄

[sabrenner]

bit funky here, but this is associating spans with their instances. while this could be done directly in _llmobs_set_tags for individual spans, the span linking logic works by looking for the caller and identifying if it's a chain. To do that, we need a way to associate a different span with an isntance.

This is all sorta predicated on the fact that we don't have an intermediary for associating traced steps. Unlike the pregel_loop_tick which we could utilize for langgraph, all steps of a chain are iterated over directly in chain.invoke, with no means for us to jump between each step as they are being executed, only before and making these kinds of connections and setting these variables.

[datadog-dd-trace-py-rkomorn]

Datadog Report

Branch report: sabrenner/langchain-span-linking
Commit report: 98aca2a
Test service: dd-trace-py

✅ 0 Failed, 130 Passed, 1378 Skipped, 4m 44.97s Total duration (35m 12.56s time saved)