Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove cap of processors on remote store related threadpools #11432

Merged
merged 1 commit into from
Dec 7, 2023

Conversation

gbbafna
Copy link
Collaborator

@gbbafna gbbafna commented Dec 1, 2023

Description

Remote Store threadpools were capped to a max count. This leads to threadpool queue getting build up on bigger instances having more cores.

Related Issues

Resolves #[Issue number to be closed when this PR is merged]

Check List

  • New functionality includes testing.
    • All tests pass
      - [ ] New functionality has been documented.
      - [ ] New functionality has javadoc added
  • Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)
  • Commits are signed per the DCO using --signoff
  • Commit changes are listed out in CHANGELOG.md file (See: Changelog)
    - [] Public documentation issue/PR created

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
Copy link
Contributor

github-actions bot commented Dec 1, 2023

Compatibility status:

Checks if related components are compatible with change 51bc819

Incompatible components

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/sql.git]

Copy link
Contributor

github-actions bot commented Dec 1, 2023

❕ Gradle check result for 51bc819: UNSTABLE

  • TEST FAILURES:
      1 org.opensearch.smoketest.SmokeTestMultiNodeClientYamlTestSuiteIT.test {yaml=pit/10_basic/Delete all}
      1 org.opensearch.remotestore.RemoteIndexPrimaryRelocationIT.testPrimaryRelocationWhileIndexing

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

Copy link

codecov bot commented Dec 1, 2023

Codecov Report

Attention: 11 lines in your changes are missing coverage. Please review.

Comparison is base (21c0597) 71.28% compared to head (51bc819) 71.33%.
Report is 3 commits behind head on main.

Files Patch % Lines
...va/org/opensearch/repositories/s3/S3BlobStore.java 50.00% 3 Missing ⚠️
...a/org/opensearch/repositories/s3/S3Repository.java 50.00% 2 Missing ⚠️
...ch/repositories/s3/async/AsyncTransferManager.java 75.00% 1 Missing and 1 partial ⚠️
...s/replication/SegmentReplicationTargetService.java 92.85% 0 Missing and 2 partials ⚠️
...rg/opensearch/repositories/s3/S3BlobContainer.java 96.55% 0 Missing and 1 partial ⚠️
...earch/repositories/s3/async/AsyncPartsHandler.java 85.71% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main   #11432      +/-   ##
============================================
+ Coverage     71.28%   71.33%   +0.05%     
- Complexity    59033    59070      +37     
============================================
  Files          4893     4893              
  Lines        277753   277832      +79     
  Branches      40357    40367      +10     
============================================
+ Hits         197989   198194     +205     
+ Misses        63288    63120     -168     
- Partials      16476    16518      +42     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@gbbafna gbbafna added backport 2.x Backport to 2.x branch skip-changelog labels Dec 4, 2023
@sachinpkale sachinpkale changed the title Removing cap of processors on remote store related threadpools Remove cap of processors on remote store related threadpools Dec 5, 2023
@gbbafna gbbafna merged commit 4ad2d4c into opensearch-project:main Dec 7, 2023
63 of 125 checks passed
@gbbafna gbbafna deleted the remote_threadpool branch December 7, 2023 11:40
opensearch-trigger-bot bot pushed a commit that referenced this pull request Dec 7, 2023
Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
(cherry picked from commit 4ad2d4c)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
kotwanikunal pushed a commit that referenced this pull request Dec 7, 2023
… (#11511)

(cherry picked from commit 4ad2d4c)

Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
deshsidd pushed a commit to deshsidd/OpenSearch that referenced this pull request Dec 11, 2023
rayshrey pushed a commit to rayshrey/OpenSearch that referenced this pull request Mar 18, 2024
shiv0408 pushed a commit to Gaurav614/OpenSearch that referenced this pull request Apr 25, 2024
…earch-project#11432)

Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x Backport to 2.x branch skip-changelog
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants