OpenTelemetry tracing

[serbrech]

We would like to be able to enable tracing and see what the underlying amqp layer is doing as part of our application request tracing.
Today we have spans all the way to our application.Send, and we connect the trace on the receiver side by propagating the tracecontext onto the Message. But there is a lot happening on the client side in the AMQP and ServiceBus SDK layer that we would very much like to enable and see attached to our traces.

Some example of trace I would expect:
application Send -> [sbsdk.Send span if enabled in sbsdk] -> [amqp.Dial] -> amqp.NewSender -> sender.Send -> retry/attempt attributes]

• send info
• disposition info
• authentication info
• link attach

Basically frames that are rooted in a client func call and take a context, could all provide some tracing information.

Concrete Use case:

We've experienced high latency on some Send call in our application, but these are sparse and hard to reproduce.
Enabling amqp debug log in production is out of the question and not the right level of information.

Opentelemetry traces would help us narrow down what part of the chain is causing high latency.

Maybe it's a retry on the CBS authentication?
Maybe it's retries/timeouts on the send?
Maybe there won't be any retries and we would see a single Send span hanging?