[Backport] [2.x] - Add Tracing Instrumentation at Network and Rest layer

[Gaganjuneja]

Description

backport #9415 to 2.x

Related Issues

Resolves #[Issue number to be closed when this PR is merged]

Check List

• New functionality includes testing.
  • All tests pass
• New functionality has been documented.
  • New functionality has javadoc added
• Commits are signed per the DCO using --signoff
• Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[Gaganjuneja]

@reta, please take a look.

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/25170/
• CommitID: df16ce0
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Compatibility status:

Checks if related components are compatible with change df16ce0

Incompatible components

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/performance-analyzer-rca.git]

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/25179/
• CommitID: 854d207
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Compatibility status:

Checks if related components are compatible with change 854d207

Incompatible components

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git]

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: SUCCESS ✅
• URL: https://build.ci.opensearch.org/job/gradle-check/25182/
• CommitID: 854d207

[codecov]

Codecov Report

Merging #9939 (854d207) into 2.x (5258ff2) will increase coverage by 0.04%.
Report is 3 commits behind head on 2.x.
The diff coverage is 84.29%.

@@             Coverage Diff              @@
##                2.x    #9939      +/-   ##
============================================
+ Coverage     70.76%   70.81%   +0.04%     
- Complexity    58217    58308      +91     
============================================
  Files          4817     4820       +3     
  Lines        275286   275379      +93     
  Branches      40445    40456      +11     
============================================
+ Hits         194814   195010     +196     
+ Misses        63820    63732      -88     
+ Partials      16652    16637      -15     

Files Changed	Coverage Δ
.../opensearch/telemetry/tracing/noop/NoopTracer.java	72.72% <ø> (+18.18%)	⬆️
...g/opensearch/transport/nio/NioTransportPlugin.java	25.00% <ø> (ø)
...a/org/opensearch/common/network/NetworkModule.java	92.00% <ø> (ø)
...ain/java/org/opensearch/plugins/NetworkPlugin.java	100.00% <ø> (ø)
...racing/ThreadContextBasedTracerContextStorage.java	93.10% <0.00%> (+20.68%)	⬆️
...etry/tracing/listener/TraceableActionListener.java	58.82% <58.82%> (ø)
.../org/opensearch/telemetry/tracing/SpanBuilder.java	91.66% <66.66%> (+3.43%)	⬆️
...lemetry/tracing/channels/TraceableHttpChannel.java	70.58% <70.58%> (ø)
...lemetry/tracing/channels/TraceableRestChannel.java	84.21% <84.21%> (ø)
.../src/main/java/org/opensearch/common/Rounding.java	85.02% <92.85%> (-0.10%)	⬇️
... and 9 more

... and 500 files with indirect coverage changes