feat(llmobs): create LLMObs service (#8320)

This PR adds a LLMObs service (similar to CIVisibility) which does a
couple things (mostly base service setup, actual features to come in
future PRs):
- Holds on to a LLMObsWriter instance instead of LLMIntegration classes
holding onto their own instance
- Adds a trace processor which checks if an LLMObs span is contained in
a trace, then marks the trace to be sent to LLMObs (i.e. replaces
submitting LLMObs records with LLMObsSpan events). Note the actual trace
being submitted to LLMObs is not in this PR (will be done in this
[PR](#8339)).
- Enabled automatically on ddtrace-run, openai/bedrock integration
patching, provided `DD_LLMOBS_ENABLED=1`

**Note: no actual functionality has been introduced in this PR yet. This
PR just adds a base service setup for LLMObs.**

This PR also does a few other refactors:
- Moves all of `ddtrace/internal/llmobs` --> `ddtrace/llmobs/*`, mostly
as private files, and exposes only the `LLMObs` service class.
- Moves `ddtrace/internal/log_writer.py` -->
`ddtrace/llmobs/_log_writer.py`, as this is owned by the MLObs team.
- Creates a new test suite `llmobs` and moves all test files relevant to
moved files mentioned above to `tests/llmobs/*`.
- Adds the `llmobs` job to CircleCI.

## Checklist

- [x] Change(s) are motivated and described in the PR description
- [x] Testing strategy is described if automated tests are not included
in the PR
- [x] Risks are described (performance impact, potential for breakage,
maintainability)
- [x] Change is maintainable (easy to change, telemetry, documentation)
- [x] [Library release note
guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html)
are followed or label `changelog/no-changelog` is set
- [x] Documentation is included (in-code, generated user docs, [public
corp docs](https://github.com/DataDog/documentation/))
- [x] Backport labels are set (if
[applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting))
- [x] If this PR changes the public interface, I've notified
`@DataDog/apm-tees`.
- [x] If change touches code that signs or publishes builds or packages,
or handles credentials of any kind, I've requested a review from
`@DataDog/security-design-and-guidance`.

## Reviewer Checklist

- [x] Title is accurate
- [x] All changes are related to the pull request's stated goal
- [x] Description motivates each change
- [x] Avoids breaking
[API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces)
changes
- [x] Testing strategy adequately addresses listed risks
- [x] Change is maintainable (easy to change, telemetry, documentation)
- [x] Release note makes sense to a user of the library
- [x] Author has acknowledged and discussed the performance implications
of this PR as reported in the benchmarks PR comment
- [x] Backport labels are set in a manner that is consistent with the
[release branch maintenance
policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)

.circleci/config.templ.yml

@@ -549,6 +549,12 @@ jobs:
           pattern: "ci_visibility"
           trace_agent_url: ""
 
+  llmobs:
+    <<: *contrib_job
+    steps:
+      - run_test:
+          pattern: "llmobs"
+
   subprocess:
     <<: *contrib_job_small
     steps:

.github/CODEOWNERS

@@ -73,10 +73,11 @@ ddtrace/profiling                   @DataDog/profiling-python @DataDog/apm-core-
 tests/profiling                     @DataDog/profiling-python @DataDog/apm-core-python
 
 # MLObs
-ddtrace/internal/llmobs/                            @DataDog/ml-observability
+ddtrace/llmobs/                                     @DataDog/ml-observability
 ddtrace/contrib/openai                              @DataDog/ml-observability
 ddtrace/contrib/langchain                           @DataDog/ml-observability
 ddtrace/contrib/botocore/services/bedrock.py        @DataDog/ml-observability
+tests/llmobs                                        @DataDog/ml-observability
 tests/contrib/openai                                @DataDog/ml-observability
 tests/contrib/langchain                             @DataDog/ml-observability
 tests/contrib/botocore/test_bedrock.py              @DataDog/ml-observability

.riot/requirements/105602c.txt

@@ -0,0 +1,31 @@
+#
+# This file is autogenerated by pip-compile with Python 3.7
+# by the following command:
+#
+#    pip-compile --config=pyproject.toml --no-annotate --resolver=backtracking .riot/requirements/105602c.in
+#
+attrs==23.2.0
+coverage[toml]==7.2.7
+exceptiongroup==1.2.0
+hypothesis==6.45.0
+idna==3.6
+importlib-metadata==6.7.0
+iniconfig==2.0.0
+mock==5.1.0
+multidict==6.0.5
+opentracing==2.4.0
+packaging==23.2
+pluggy==1.2.0
+pytest==7.4.4
+pytest-cov==4.1.0
+pytest-mock==3.11.1
+pyyaml==6.0.1
+six==1.16.0
+sortedcontainers==2.4.0
+tomli==2.0.1
+typing-extensions==4.7.1
+urllib3==1.26.18
+vcrpy==4.4.0
+wrapt==1.16.0
+yarl==1.9.4
+zipp==3.15.0

.riot/requirements/16df1cc.txt → .riot/requirements/112818b.txt

@@ -2,18 +2,16 @@
 # This file is autogenerated by pip-compile with Python 3.12
 # by the following command:
 #
-#    pip-compile --no-annotate .riot/requirements/16df1cc.in
+#    pip-compile --no-annotate .riot/requirements/112818b.in
 #
 attrs==23.2.0
 coverage[toml]==7.4.0
 gevent==23.9.1
 greenlet==3.0.3
 httpretty==1.1.4
 hypothesis==6.45.0
-idna==3.6
 iniconfig==2.0.0
 mock==5.1.0
-multidict==6.0.4
 opentracing==2.4.0
 packaging==23.2
 pluggy==1.3.0
@@ -23,11 +21,7 @@ pytest-cov==4.1.0
 pytest-mock==3.12.0
 pytest-randomly==3.15.0
 python-json-logger==2.0.7
-pyyaml==6.0.1
 sortedcontainers==2.4.0
-vcrpy==5.1.0
-wrapt==1.16.0
-yarl==1.9.4
 zope-event==5.0
 zope-interface==6.1
 

.riot/requirements/1553fe5.txt → .riot/requirements/1130e9d.txt

@@ -2,7 +2,7 @@
 # This file is autogenerated by pip-compile with Python 3.8
 # by the following command:
 #
-#    pip-compile --no-annotate .riot/requirements/1553fe5.in
+#    pip-compile --no-annotate .riot/requirements/1130e9d.in
 #
 attrs==23.2.0
 coverage[toml]==7.4.0
@@ -11,11 +11,9 @@ gevent==23.9.1
 greenlet==3.0.3
 httpretty==1.1.4
 hypothesis==6.45.0
-idna==3.6
 importlib-metadata==7.0.1
 iniconfig==2.0.0
 mock==5.1.0
-multidict==6.0.4
 opentracing==2.4.0
 packaging==23.2
 pluggy==1.3.0
@@ -25,13 +23,8 @@ pytest-cov==4.1.0
 pytest-mock==3.12.0
 pytest-randomly==3.15.0
 python-json-logger==2.0.7
-pyyaml==6.0.1
 sortedcontainers==2.4.0
 tomli==2.0.1
-urllib3==1.26.18
-vcrpy==5.1.0
-wrapt==1.16.0
-yarl==1.9.4
 zipp==3.17.0
 zope-event==5.0
 zope-interface==6.1

.riot/requirements/1388f2d.txt

@@ -0,0 +1,27 @@
+#
+# This file is autogenerated by pip-compile with Python 3.8
+# by the following command:
+#
+#    pip-compile --no-annotate .riot/requirements/1388f2d.in
+#
+attrs==23.2.0
+coverage[toml]==7.4.0
+exceptiongroup==1.2.0
+hypothesis==6.45.0
+idna==3.6
+iniconfig==2.0.0
+mock==5.1.0
+multidict==6.0.5
+opentracing==2.4.0
+packaging==23.2
+pluggy==1.3.0
+pytest==7.4.4
+pytest-cov==4.1.0
+pytest-mock==3.12.0
+pyyaml==6.0.1
+sortedcontainers==2.4.0
+tomli==2.0.1
+urllib3==1.26.18
+vcrpy==6.0.1
+wrapt==1.16.0
+yarl==1.9.4

.riot/requirements/151fa3d.txt

@@ -0,0 +1,26 @@
+#
+# This file is autogenerated by pip-compile with Python 3.10
+# by the following command:
+#
+#    pip-compile --no-annotate .riot/requirements/151fa3d.in
+#
+attrs==23.2.0
+coverage[toml]==7.4.0
+exceptiongroup==1.2.0
+hypothesis==6.45.0
+idna==3.6
+iniconfig==2.0.0
+mock==5.1.0
+multidict==6.0.5
+opentracing==2.4.0
+packaging==23.2
+pluggy==1.3.0
+pytest==7.4.4
+pytest-cov==4.1.0
+pytest-mock==3.12.0
+pyyaml==6.0.1
+sortedcontainers==2.4.0
+tomli==2.0.1
+vcrpy==6.0.1
+wrapt==1.16.0
+yarl==1.9.4

.riot/requirements/153fdb0.txt

@@ -0,0 +1,24 @@
+#
+# This file is autogenerated by pip-compile with Python 3.11
+# by the following command:
+#
+#    pip-compile --no-annotate .riot/requirements/153fdb0.in
+#
+attrs==23.2.0
+coverage[toml]==7.4.0
+hypothesis==6.45.0
+idna==3.6
+iniconfig==2.0.0
+mock==5.1.0
+multidict==6.0.5
+opentracing==2.4.0
+packaging==23.2
+pluggy==1.3.0
+pytest==7.4.4
+pytest-cov==4.1.0
+pytest-mock==3.12.0
+pyyaml==6.0.1
+sortedcontainers==2.4.0
+vcrpy==6.0.1
+wrapt==1.16.0
+yarl==1.9.4

.riot/requirements/1397551.txt → .riot/requirements/1902e9f.txt

@@ -2,18 +2,16 @@
 # This file is autogenerated by pip-compile with Python 3.11
 # by the following command:
 #
-#    pip-compile --no-annotate .riot/requirements/1397551.in
+#    pip-compile --no-annotate .riot/requirements/1902e9f.in
 #
 attrs==23.2.0
 coverage[toml]==7.4.0
 gevent==23.9.1
 greenlet==3.0.3
 httpretty==1.1.4
 hypothesis==6.45.0
-idna==3.6
 iniconfig==2.0.0
 mock==5.1.0
-multidict==6.0.4
 opentracing==2.4.0
 packaging==23.2
 pluggy==1.3.0
@@ -23,11 +21,7 @@ pytest-cov==4.1.0
 pytest-mock==3.12.0
 pytest-randomly==3.15.0
 python-json-logger==2.0.7
-pyyaml==6.0.1
 sortedcontainers==2.4.0
-vcrpy==5.1.0
-wrapt==1.16.0
-yarl==1.9.4
 zope-event==5.0
 zope-interface==6.1
 

.riot/requirements/1ace2ad.txt

@@ -0,0 +1,27 @@
+#
+# This file is autogenerated by pip-compile with Python 3.9
+# by the following command:
+#
+#    pip-compile --no-annotate .riot/requirements/1ace2ad.in
+#
+attrs==23.2.0
+coverage[toml]==7.4.0
+exceptiongroup==1.2.0
+hypothesis==6.45.0
+idna==3.6
+iniconfig==2.0.0
+mock==5.1.0
+multidict==6.0.5
+opentracing==2.4.0
+packaging==23.2
+pluggy==1.3.0
+pytest==7.4.4
+pytest-cov==4.1.0
+pytest-mock==3.12.0
+pyyaml==6.0.1
+sortedcontainers==2.4.0
+tomli==2.0.1
+urllib3==1.26.18
+vcrpy==6.0.1
+wrapt==1.16.0
+yarl==1.9.4

.riot/requirements/1c854f5.txt → .riot/requirements/4cba75c.txt

@@ -2,7 +2,7 @@
 # This file is autogenerated by pip-compile with Python 3.7
 # by the following command:
 #
-#    pip-compile --config=pyproject.toml --no-annotate --resolver=backtracking .riot/requirements/1c854f5.in
+#    pip-compile --config=pyproject.toml --no-annotate --resolver=backtracking .riot/requirements/4cba75c.in
 #
 attrs==23.2.0
 coverage[toml]==7.2.7
@@ -11,11 +11,9 @@ gevent==22.10.2
 greenlet==3.0.3
 httpretty==1.1.4
 hypothesis==6.45.0
-idna==3.6
 importlib-metadata==6.7.0
 iniconfig==2.0.0
 mock==5.1.0
-multidict==6.0.4
 opentracing==2.4.0
 packaging==23.2
 pluggy==1.2.0
@@ -25,15 +23,9 @@ pytest-cov==4.1.0
 pytest-mock==3.11.1
 pytest-randomly==3.12.0
 python-json-logger==2.0.7
-pyyaml==6.0.1
-six==1.16.0
 sortedcontainers==2.4.0
 tomli==2.0.1
 typing-extensions==4.7.1
-urllib3==1.26.18
-vcrpy==4.4.0
-wrapt==1.16.0
-yarl==1.9.4
 zipp==3.15.0
 zope-event==5.0
 zope-interface==6.1

.riot/requirements/6fdb03a.txt

@@ -0,0 +1,24 @@
+#
+# This file is autogenerated by pip-compile with Python 3.12
+# by the following command:
+#
+#    pip-compile --no-annotate .riot/requirements/6fdb03a.in
+#
+attrs==23.2.0
+coverage[toml]==7.4.0
+hypothesis==6.45.0
+idna==3.6
+iniconfig==2.0.0
+mock==5.1.0
+multidict==6.0.5
+opentracing==2.4.0
+packaging==23.2
+pluggy==1.3.0
+pytest==7.4.4
+pytest-cov==4.1.0
+pytest-mock==3.12.0
+pyyaml==6.0.1
+sortedcontainers==2.4.0
+vcrpy==6.0.1
+wrapt==1.16.0
+yarl==1.9.4

.riot/requirements/eefd86a.txt → .riot/requirements/99214a1.txt

@@ -2,7 +2,7 @@
 # This file is autogenerated by pip-compile with Python 3.9
 # by the following command:
 #
-#    pip-compile --no-annotate .riot/requirements/eefd86a.in
+#    pip-compile --no-annotate .riot/requirements/99214a1.in
 #
 attrs==23.2.0
 coverage[toml]==7.4.0
@@ -11,11 +11,9 @@ gevent==23.9.1
 greenlet==3.0.3
 httpretty==1.1.4
 hypothesis==6.45.0
-idna==3.6
 importlib-metadata==7.0.1
 iniconfig==2.0.0
 mock==5.1.0
-multidict==6.0.4
 opentracing==2.4.0
 packaging==23.2
 pluggy==1.3.0
@@ -25,13 +23,8 @@ pytest-cov==4.1.0
 pytest-mock==3.12.0
 pytest-randomly==3.15.0
 python-json-logger==2.0.7
-pyyaml==6.0.1
 sortedcontainers==2.4.0
 tomli==2.0.1
-urllib3==1.26.18
-vcrpy==5.1.0
-wrapt==1.16.0
-yarl==1.9.4
 zipp==3.17.0
 zope-event==5.0
 zope-interface==6.1

.riot/requirements/88249c9.txt → .riot/requirements/b1a336f.txt

@@ -2,7 +2,7 @@
 # This file is autogenerated by pip-compile with Python 3.10
 # by the following command:
 #
-#    pip-compile --no-annotate .riot/requirements/88249c9.in
+#    pip-compile --no-annotate .riot/requirements/b1a336f.in
 #
 attrs==23.2.0
 coverage[toml]==7.4.0
@@ -11,10 +11,8 @@ gevent==23.9.1
 greenlet==3.0.3
 httpretty==1.1.4
 hypothesis==6.45.0
-idna==3.6
 iniconfig==2.0.0
 mock==5.1.0
-multidict==6.0.4
 opentracing==2.4.0
 packaging==23.2
 pluggy==1.3.0
@@ -24,12 +22,8 @@ pytest-cov==4.1.0
 pytest-mock==3.12.0
 pytest-randomly==3.15.0
 python-json-logger==2.0.7
-pyyaml==6.0.1
 sortedcontainers==2.4.0
 tomli==2.0.1
-vcrpy==5.1.0
-wrapt==1.16.0
-yarl==1.9.4
 zope-event==5.0
 zope-interface==6.1