[Backport 2.x] Add indexRandomForConcurrentSearch for tests #11204

jed326 · 2023-11-14T20:11:57Z

Description

Manual backport for the following commits:

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)
Commits are signed per the DCO using --signoff
Commit changes are listed out in CHANGELOG.md file (See: Changelog)
Public documentation issue/PR created

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

…ct#11118) * IndexStatsIT.java * SearchTimeoutIT.java * NaNSortingIT.java * NestedIT.java * RangeIT.java Signed-off-by: Jay Deng <jayd0104@gmail.com>

github-actions · 2023-11-14T20:32:39Z

Compatibility status:

Checks if related components are compatible with change bfddc44

Incompatible components

Incompatible components: [https://github.com/opensearch-project/performance-analyzer.git]

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/alerting.git]

github-actions · 2023-11-14T20:33:26Z

❌ Gradle check result for bdf2a14: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2023-11-14T21:05:52Z

❌ Gradle check result for 8bac796: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

* MultiSearchIT.java * SimpleNestedIT.java * SearchPreferenceIT.java * SimpleQueryStringIT.java * ScriptQuerySearchIT.java Signed-off-by: Jay Deng <jayd0104@gmail.com>

jed326 · 2023-11-14T21:22:01Z

https://build.ci.opensearch.org/job/gradle-check/29979/

[BUG] org.opensearch.repositories.azure.AzureBlobContainerRetriesTests.testWriteBlobWithRetries is faky #7934

github-actions · 2023-11-14T21:47:58Z

❌ Gradle check result for b8e5fc1: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2023-11-15T18:45:53Z

❌ Gradle check result for b8e5fc1: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2023-11-16T01:20:48Z

❌ Gradle check result for b8e5fc1: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

…ollow-up [opensearch-project#11130] (opensearch-project#11212) Signed-off-by: Ganesh Ramadurai <ganeshkrishna2396@gmail.com> Co-authored-by: Ganesh Ramadurai <ganeshkrishna2396@gmail.com>

Signed-off-by: Ganesh Ramadurai <ganeshkrishna2396@gmail.com> Signed-off-by: Ganesh Ramadurai <gramadur@amazon.com> Co-authored-by: Ganesh Ramadurai <ganeshkrishna2396@gmail.com>

* Fixing the tests for concurrent search Signed-off-by: Bansi Kasundra <kasundra@amazon.com> * Remove changes for non-flaky tests Signed-off-by: Bansi Kasundra <kasundra@amazon.com> --------- Signed-off-by: Bansi Kasundra <kasundra@amazon.com>

Updated flaky and failing test cases after changes to indexRandomForConcurrentSearch Remove unnecessary call to indexRandomForConcurrentSearch in testSearchAndRelocateConcurrently Signed-off-by: Ethan Emoto <eemoto@amazon.com>

github-actions · 2023-11-16T23:01:03Z

❕ Gradle check result for b8e5fc1: UNSTABLE

TEST FAILURES:

      1 org.opensearch.repositories.azure.AzureBlobContainerRetriesTests.testReadNonexistentBlobThrowsNoSuchFileException

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

codecov · 2023-11-16T23:01:56Z

Codecov Report

Attention: 49 lines in your changes are missing coverage. Please review.

Comparison is base (827099e) 70.88% compared to head (bfddc44) 70.91%.
Report is 13 commits behind head on 2.x.

Files	Patch %	Lines
.../index/search/comparators/HalfFloatComparator.java	0.00%	29 Missing ⚠️
...eldcomparator/HalfFloatValuesComparatorSource.java	25.00%	12 Missing ⚠️
...rg/opensearch/gradle/OpenSearchTestBasePlugin.java	0.00%	4 Missing ⚠️
...g/opensearch/tools/launchers/SystemJvmOptions.java	0.00%	4 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##                2.x   #11204      +/-   ##
============================================
+ Coverage     70.88%   70.91%   +0.02%     
- Complexity    58926    58935       +9     
============================================
  Files          4862     4864       +2     
  Lines        278389   278443      +54     
  Branches      40806    40812       +6     
============================================
+ Hits         197336   197450     +114     
+ Misses        64289    64224      -65     
- Partials      16764    16769       +5

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

jed326 · 2023-11-16T23:02:17Z

https://build.ci.opensearch.org/job/gradle-check/30116/

[BUG] AzureBlobContainerRetriesTests.testReadRangeBlobWithRetries flaky test failure #6090

github-actions · 2023-11-16T23:16:35Z

✅ Gradle check result for bfddc44: SUCCESS

sohami · 2023-11-17T00:53:59Z

Have manually set DCO to pass since I verified that each commit has the Signed-off-by line in it

Add indexRandomForConcurrentSearch method to tests: (opensearch-proje…

0539713

…ct#11118) * IndexStatsIT.java * SearchTimeoutIT.java * NaNSortingIT.java * NestedIT.java * RangeIT.java Signed-off-by: Jay Deng <jayd0104@gmail.com>

jed326 added the skip-changelog label Nov 14, 2023

jed326 mentioned this pull request Nov 14, 2023

Fix tests running with slice count of 1 for concurrent search use-case #11120

Closed

5 tasks

jed326 marked this pull request as ready for review November 14, 2023 20:13

jed326 force-pushed the backport-11181-11118 branch from bdf2a14 to 8bac796 Compare November 14, 2023 20:34

Add indexRandomForConcurrentSearch for tests: (opensearch-project#11181)

b8e5fc1

* MultiSearchIT.java * SimpleNestedIT.java * SearchPreferenceIT.java * SimpleQueryStringIT.java * ScriptQuerySearchIT.java Signed-off-by: Jay Deng <jayd0104@gmail.com>

jed326 force-pushed the backport-11181-11118 branch from 8bac796 to b8e5fc1 Compare November 14, 2023 21:21

This was referenced Nov 16, 2023

[Backport 2.x] Fixing the tests for concurrent search #11199

Closed

[Backport 2.x] Fixed flaky and failing test cases for concurrent search #11202

Closed

jed326 closed this Nov 16, 2023

jed326 reopened this Nov 16, 2023

jed326 marked this pull request as draft November 16, 2023 22:12

Gankris96 and others added 4 commits November 16, 2023 14:16

Fix tests for concurrent search with IndexRandomForConcurrentSearch f…

697fcf3

…ollow-up [opensearch-project#11130] (opensearch-project#11212) Signed-off-by: Ganesh Ramadurai <ganeshkrishna2396@gmail.com> Co-authored-by: Ganesh Ramadurai <ganeshkrishna2396@gmail.com>

Fix tests for concurrent search (opensearch-project#11130)

1a2a9eb

Signed-off-by: Ganesh Ramadurai <ganeshkrishna2396@gmail.com> Signed-off-by: Ganesh Ramadurai <gramadur@amazon.com> Co-authored-by: Ganesh Ramadurai <ganeshkrishna2396@gmail.com>

jed326 marked this pull request as ready for review November 16, 2023 22:29

sohami approved these changes Nov 17, 2023

View reviewed changes

sohami merged commit b33981e into opensearch-project:2.x Nov 17, 2023
17 of 18 checks passed

jed326 deleted the backport-11181-11118 branch March 12, 2024 22:13

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Backport 2.x] Add indexRandomForConcurrentSearch for tests #11204

[Backport 2.x] Add indexRandomForConcurrentSearch for tests #11204

jed326 commented Nov 14, 2023 •

edited

Loading

github-actions bot commented Nov 14, 2023 •

edited

Loading

github-actions bot commented Nov 14, 2023

github-actions bot commented Nov 14, 2023

jed326 commented Nov 14, 2023

github-actions bot commented Nov 14, 2023

github-actions bot commented Nov 15, 2023

github-actions bot commented Nov 16, 2023

github-actions bot commented Nov 16, 2023

codecov bot commented Nov 16, 2023 •

edited

Loading

jed326 commented Nov 16, 2023

github-actions bot commented Nov 16, 2023

sohami commented Nov 17, 2023

[Backport 2.x] Add indexRandomForConcurrentSearch for tests #11204

[Backport 2.x] Add indexRandomForConcurrentSearch for tests #11204

Conversation

jed326 commented Nov 14, 2023 • edited Loading

Description

Check List

github-actions bot commented Nov 14, 2023 • edited Loading

Compatibility status:

Incompatible components

Skipped components

Compatible components

github-actions bot commented Nov 14, 2023

github-actions bot commented Nov 14, 2023

jed326 commented Nov 14, 2023

github-actions bot commented Nov 14, 2023

github-actions bot commented Nov 15, 2023

github-actions bot commented Nov 16, 2023

github-actions bot commented Nov 16, 2023

codecov bot commented Nov 16, 2023 • edited Loading

Codecov Report

jed326 commented Nov 16, 2023

github-actions bot commented Nov 16, 2023

sohami commented Nov 17, 2023

jed326 commented Nov 14, 2023 •

edited

Loading

github-actions bot commented Nov 14, 2023 •

edited

Loading

codecov bot commented Nov 16, 2023 •

edited

Loading