[core] - Extract OpenTelemetry to a separate package #19734

maorleger · 2022-01-07T16:16:38Z

What

Rewrite core-tracing using an abstraction of an instrumenter
Remove @opentelemetry packages from core-tracing
Introduce a stateful client that can be used to interact with the underlying instrumenter
Introduce @azure/opentelemetry-instrumentation-azure-sdk package providing an OTel instrumenter
- Integrate this package with OpenTelemetry's instrumentation APIs to provide a plug-in that lights up OTel based tracing
Add a chaiAzureTrace chai plugin to allow for quick and simple validation of tracing logic for packages

Why

Core-tracing

This is essentially a rewrite of core-tracing to abstract away from @opentelemetry/api for multiple reasons:

Decreased bundle size
Making OTel tracing truly opt-in. If you don't use OTel, there's no reason for us to call OTel APIs. While the APIs is now stable, it has been problematic for us in the past.
No enums, no TS > 3.9 features to downlevel
Finally, @opentelemetry/api is very flexible and able to support many scenarios at the cost of what is a very wide API surface area. We can avoid much of this complexity.

new instrumentation package

With the changes to core-tracing, we now need a way to tie our abstraction to a concrete OTel implementation. A new package, @azure/opentelemetry-instrumentation-azure-sdk provides this hook using well-known registerInstrumentations API to light-up OTel based tracing when a user opts-in.

New chai plugin

This is our attempt at reducing the boilerplate around tracing, providing what is essentially a custom assertion wrapped in a chai plugin. While this may change significantly, I feel comfortable merging this in and making follow-up changes on main.

Callouts

There is still a lot of work to be done, but this is 90% of the way there (keeping in mind the 90-90 rule) - I feel that we are at a point where we can merge this in and continue work on main

merging: With enough approvals, I'd like to merge this in after next week's January package releases

Finally, when working with Karishma we drew this crude diagram to explain the integration points between packages. Hopefully this helps a bit so figured I'll share

TODO

CHANGELOG

sdk/core/core-tracing/review/core-tracing.api.md

maorleger · 2022-01-07T16:52:36Z

sdk/instrumentation/opentelemetry-instrumentation-azure-sdk/README.md

+
+## Next steps
+
+- TODO: no samples yet, so the link verification fails. Add link to samples...


I realize the TODOs here are less than ideal, but I would like to merge this with them - I am keeping a list of these TODOs and will create issues for them next week.

xirzec

So excited for this one! I left a few remarks (and reactions.) I'm also okay if you want to capture requested changes in an issue to handle in a separate PR since this one is so big and has been individually reviewed.

sdk/core/core-tracing/package.json

sdk/core/core-tracing/review/core-tracing.api.md

sdk/core/core-tracing/src/instrumenter.ts

Taken from #19734 (comment) - this demonstrates what that might look like This PR removes `TracingSpanContext` from our public types and relies on TracingContext and runtime checks to serialize and deserialize a TracingSpan

- Add the following interfaces: - TracingContext - Instrumenter - InstrumenterSpanOptions - TracingClient + TracingClientOptions - TracingSpan, TracingSpanContext, TracingSpanKind, TracingSpanOptions - Add the following methods: - useInstrumenter - allows plugging in an instrumenter implementation at runtime - parseTraceparentHeader / createRequestHeaders - parse and stringify TracingSpan IDs - createTracingClient - Remove `@opentelemetry/api` from `@azure/core-tracing` This is a complete rewrite of `@azure/core-tracing` to provide the following benefits 1. Abstract away from OTel's API, using a smaller surface area that provides (what we think) is a nicer abstraction for managing tracing 2. Make tracing completely opt-in - without installing the (yet-to-be-created) instrumentation package and configuring it the user will not have to deal with any of OpenTelemetry's complexity 3. Remove `@opentelemetry/api` from all packages to support #17068 for RLCs Due to the large change I am working on a feature branch, so this is not being merged to `main` Resolves #18650

azure-sdk · 2022-01-12T01:35:06Z

API changes have been detected in @azure/core-auth. You can review API changes here

azure-sdk · 2022-01-12T01:35:21Z

API changes have been detected in @azure/core-tracing. You can review API changes here

xirzec

I'm quite happy about this cleanup and looking forward to GAing core-tracing

sdk/core/core-http/package.json

This reverts commit 4fa61ed.

maorleger · 2022-01-13T23:21:59Z

test-utils failure is a known issue due to the recent recorder changes

[browser-tests] 12 01 2022 22:48:16.485:INFO [preprocessor.env]: Publishing variables:  [
[browser-tests]   'TEST_MODE',
[browser-tests]   'APPCONFIG_ENDPOINT',
[browser-tests]   'APPCONFIG_TEST_SETTING_KEY',
[browser-tests]   'APPCONFIG_TEST_SETTING_EXPECTED_VALUE',
[browser-tests]   'AZURE_CLIENT_ID',
[browser-tests]   'AZURE_CLIENT_SECRET',
[browser-tests]   'AZURE_TENANT_ID',
[browser-tests]   'RECORDINGS_RELATIVE_PATH'
[browser-tests] ]
[browser-tests] 
[browser-tests] START:
Trace: [Internal Error] 
    at D:\a\_work\1\s\common\tools\dev-tool\src\index.ts:10:11
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! @azure/template@1.0.11-beta.1 integration-test:browser: `dev-tool run test:browser`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the @azure/template@1.0.11-beta.1 integration-test:browser script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

Overriding and merging

maorleger · 2022-01-13T23:22:10Z

/check-enforcer override

maorleger · 2022-01-13T23:28:45Z

/azp run prepare-pipelines

Network 2022 05 01 (Azure#20695) * Adds base for updating Microsoft.Network from version stable/2022-01-01 to version 2022-05-01 * Updates readme * Updates API version in new specs and examples * add DisableTcpStateTracking on nic (Azure#19734) Co-authored-by: Dan Tu <dantu@microsoft.com> * Adding new VIP Swap APIs to support extension resource design (Azure#19698) * Adding new VIP Swap APIs to support extension resource design * fixing validation checks * minor fix * Reverting changes * Minor fix to add a new line * fix description (Azure#19987) Co-authored-by: Dan Tu <dantu@microsoft.com> * Add BillingType Property to ExpressRoutePorts (Azure#19932) * change port json * changes * add Virtual network gateway policy group api (Azure#19766) * finish update on connection configuration references * resolve reference error * finish prettier * resolve sematic error * finish update change * enablePrivateLinkFastPath property for VirtualNetworkGatewayConnection resource (Azure#20050) * Support for Per Rule Actions in Application Gateway WAF (Azure#20027) * Adding changes for per rule actions * Fixing spaces * prettier fix * Addressing PR comments * Address review comments * Add ColoLocation to ExpressRoutePort result (Azure#20033) * Add ColoLocation to ExpressRoutePort result * Fix automation code check error * Merge from network-2022-05-01 * Revert "Merge from network-2022-05-01" This reverts commit f0671a040b2bad684921f8f8ad4b53bb0f4b4a93. Co-authored-by: Jianqi Zhao <jianzhao@microsoft.com> * DDoS Per IP SKU + Custom Policy Deprecation Swagger PR (Azure#19985) * initial changes * example file changes * new files for new apis * adding a period * changing reference * prettier fixes * ddos fix * lint diff fixes * more lint diff errors * fixing operation id * changing bool to string enum * example changes * adding 202s as these are long running ops * examples Co-authored-by: Manas Chakka <manaschakka@microsoft.com> * Adding UnhealthyThreshold property to LB health probes, and updated examples with new property (Azure#20001) * Property to enable/disable OCSP revocation check on client certificate (Azure#20301) * add new property to enable/disable OCSP revocation check on client certificate for MTLS * revert breaking change of updating enum property * Add new parameter asn, geo and expressRouteAdvertise to CustomIPPrefix for new 2022-05-01 nrp sdk version (Azure#20266) * fix * fix * fix * fix * fix * fix Co-authored-by: Weiheng Li <weihl@microsoft.com> * Network 2022-05-01 Azure Virtual Network Manager Contributions (Azure#20440) * Add third NIP enum for avnm security admin config, AllowRulesOnly * add optional top parameters to various network manager post APIs * revise all example input subscriptions to all zero guid * change all top parameter max value to 100 * Revert "change all top parameter max value to 100" This reverts commit 12943d2f2b91b22f1dae232cb291a8551fedfdca. Co-authored-by: Jared Gorthy <jaredgorthy@microsoft.com> * Alpelled waf manifest 05 01 2022 (Azure#20047) * init * init * fix comments * fix err * try ref to appgw def * try ref to appgw def * try ref to appgw def * try ref to appgw def * update example * update example * update example * update state and action * update state and action * change ruleset name and add tiers * cammel case fix * cammel case fix * s_ruleid * s_RuleId * suppress s_RuleId camel case issue * suppress s_RuleId camel case issue * address comments * address comments * address comments * fix typo * cont fix * cont fix * cont fix * cont fix * cont fix * add pageable * add next link Co-authored-by: Alon Pelled <alpelled@microsoft.com> * Add prefixType property to CustomIPPrefix for NRP 2022-05-01 (Azure#20486) * add prefixType parameter * add example * Adding swagger changes for routemaps feature (Azure#20091) * initial changes for routemaps * run npm prettier script * adding examples * fixing validation errors * more validations * more fixes * trying to fix lint error * fix for match condition * renaming ActionType to fix lint * fix for python sdk generation * renaming Rule to RouteMapRule * modify examples to remove vnetroutes from non hub vnet conns * cleanup * Added Upper-case transform (Azure#20674) Co-authored-by: dtuCloud <tudan0103@gmail.com> Co-authored-by: Dan Tu <dantu@microsoft.com> Co-authored-by: shnaya434 <67019054+shnaya434@users.noreply.github.com> Co-authored-by: nimaller <71352534+nimaller@users.noreply.github.com> Co-authored-by: tyrannicrex <jiaczh@microsoft.com> Co-authored-by: Jesus Arango <jearango@microsoft.com> Co-authored-by: Sindhu Aluguvelli <sindhureddy216@gmail.com> Co-authored-by: Jianqi Zhao <archie.j.zhao@gmail.com> Co-authored-by: Jianqi Zhao <jianzhao@microsoft.com> Co-authored-by: Manas Chakka <mchakka@gmail.com> Co-authored-by: Manas Chakka <manaschakka@microsoft.com> Co-authored-by: bhavanabheem <109241669+bhavanabheem@users.noreply.github.com> Co-authored-by: biaogao <biga@microsoft.com> Co-authored-by: Weiheng Li <weihengli.tj@gmail.com> Co-authored-by: Weiheng Li <weihl@microsoft.com> Co-authored-by: jago2136 <42753636+jago2136@users.noreply.github.com> Co-authored-by: Jared Gorthy <jaredgorthy@microsoft.com> Co-authored-by: alon-microsoft <31221093+alon-microsoft@users.noreply.github.com> Co-authored-by: Alon Pelled <alpelled@microsoft.com> Co-authored-by: Eric Hoffmann <112216215+erichoffmannms@users.noreply.github.com> Co-authored-by: arganapathy <109313670+arganapathy@users.noreply.github.com> Co-authored-by: rahulbissa2727 <106511944+rahulbissa2727@users.noreply.github.com>

ghost added the Azure.Core label Jan 7, 2022

maorleger commented Jan 7, 2022

View reviewed changes

maorleger changed the title ~~Feature/core tracing rewrite~~ [core] - Extract OpenTelemetry to a separate package Jan 7, 2022

maorleger marked this pull request as ready for review January 7, 2022 17:37

maorleger requested review from benbp, bterlson, deyaaeldeen, HarshaNalluru, jeremymeng, mikeharder, praveenkuttappan, ramya-rao-a, sadasant, timovv, witemple-msft and xirzec as code owners January 7, 2022 17:37

maorleger requested a review from lmolkova January 7, 2022 18:57

xirzec reviewed Jan 7, 2022

View reviewed changes

maorleger mentioned this pull request Jan 11, 2022

[tracing] - Remove spanContext from our public API #19786

Merged

maorleger and others added 2 commits January 11, 2022 17:13

post-rebase rush update

f167f14

maorleger force-pushed the feature/core-tracing-rewrite branch from 766ad1d to f167f14 Compare January 12, 2022 01:19

Named types for span status union

419099b

wip

4fa61ed

xirzec approved these changes Jan 12, 2022

View reviewed changes

Azure deleted a comment from check-enforcer bot Jan 12, 2022

changelog for core-tracing

ab1c769

instrumentation changelog

ed382d7

maorleger commented Jan 12, 2022

View reviewed changes

sdk/core/core-http/package.json Outdated Show resolved Hide resolved

Revert "wip"

52944fc

This reverts commit 4fa61ed.

maorleger merged commit 9d78d16 into main Jan 13, 2022

maorleger deleted the feature/core-tracing-rewrite branch January 13, 2022 23:22

maorleger restored the feature/core-tracing-rewrite branch January 13, 2022 23:28

xirzec deleted the feature/core-tracing-rewrite branch April 20, 2022 22:09

lmolkova mentioned this pull request May 27, 2022

Azure SDK Tracing Nickel Semester (Jul-Dec21) Azure/azure-sdk#3202

Closed

36 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[core] - Extract OpenTelemetry to a separate package #19734

[core] - Extract OpenTelemetry to a separate package #19734

maorleger commented Jan 7, 2022 •

edited

Loading

maorleger Jan 7, 2022

xirzec left a comment

azure-sdk commented Jan 12, 2022

azure-sdk commented Jan 12, 2022

xirzec left a comment

maorleger commented Jan 13, 2022

maorleger commented Jan 13, 2022

maorleger commented Jan 13, 2022


		## Next steps

		- TODO: no samples yet, so the link verification fails. Add link to samples...

[core] - Extract OpenTelemetry to a separate package #19734

[core] - Extract OpenTelemetry to a separate package #19734

Conversation

maorleger commented Jan 7, 2022 • edited Loading

What

Why

Core-tracing

new instrumentation package

New chai plugin

Callouts

TODO

maorleger Jan 7, 2022

Choose a reason for hiding this comment

xirzec left a comment

Choose a reason for hiding this comment

azure-sdk commented Jan 12, 2022

azure-sdk commented Jan 12, 2022

xirzec left a comment

Choose a reason for hiding this comment

maorleger commented Jan 13, 2022

maorleger commented Jan 13, 2022

maorleger commented Jan 13, 2022

maorleger commented Jan 7, 2022 •

edited

Loading