[Tests] fix JUnit Reporter test #1338

kavilla · 2022-03-10T10:00:27Z

Description

Interesting flakiness in the test. After the jest upgrade, our tests
got more performant.

The JUnit Report integ test was actually expecting anything for a field
time. This time was calculated on elapsed time to the 0.00x value
anything faster would actually be undefined and at that point anything
fails because anything cannot be undefined.

tl;dr: This test became flakey because our tests now run too fast.

Signed-off-by: Kawika Avilla kavilla414@gmail.com

Issues Resolved

n/a

Check List

Interesting flakiness in the test. After the jest upgrade, our tests got more performant. The JUnit Report integ test was actually expecting `anything` for a field `time`. This `time` was calculated on elapsed time to the `0.00x` value anything faster would actually be undefined and at that point `anything` fails because `anything` cannot be undefined. tl;dr: This test became flakey because our tests now run too fast. Signed-off-by: Kawika Avilla <kavilla414@gmail.com>

mihirsoni

LGTM !! Thanks for the changes.

mihirsoni · 2022-03-10T12:14:21Z

src/dev/jest/integration_tests/__fixtures__/test.js

+it('fails', (done) => {
+  setTimeout(() => {
+    done();
+  }, 200);


will this timeout be enough ?

tmarkley · 2022-03-14T17:31:17Z

Interesting - did this just pop up? Was it only failing locally? We didn't have any failing integration tests in #1301.

kavilla · 2022-03-14T17:59:54Z

It would fail like 1 out 4 times locally and fail less frequently in github, probably saw it only twice in the CI but enough for me to check it out locally.

* Run GitHub workflow unit tests in band (opensearch-project#1306) * Fix JUnit Reporter test (opensearch-project#1338) * Add code coverage reporting (opensearch-project#1478) * Add release build verification workflow (opensearch-project#1502) * Add Windows CI to the workflow (opensearch-project#2966) * Add retrying flaky jest tests (opensearch-project#2967) * Record functional test artifacts in case of failure (opensearch-project#3190) * Improve yarn's performance in workflows by caching yarn's cache folder (opensearch-project#3194) * Fix the version of Chrome to 107 to match `chromedriver@107.0.1`, the last that works on NodeJS v10 Signed-off-by: Miki <miki@amazon.com>

* Run GitHub workflow unit tests in band (opensearch-project#1306) * Fix JUnit Reporter test (opensearch-project#1338) * Add code coverage reporting (opensearch-project#1478) * Add release build verification workflow (opensearch-project#1502) * Add Windows CI to the workflow (opensearch-project#2966) * Add retrying flaky jest tests (opensearch-project#2967) * Record functional test artifacts in case of failure (opensearch-project#3190) * Increase timeout waiting for the exiting of an optimizer worker (opensearch-project#3193) * Improve yarn's performance in workflows by caching yarn's cache folder (opensearch-project#3194) * Fix the version of Chrome to 107 to match `chromedriver@107.0.1`, the last that works on NodeJS v10 Signed-off-by: Miki <miki@amazon.com>

* Run GitHub workflow unit tests in band (opensearch-project#1306) * Fix JUnit Reporter test (opensearch-project#1338) * Add code coverage reporting (opensearch-project#1478) * Add release build verification workflow (opensearch-project#1502) * Add Windows CI to the workflow (opensearch-project#2966) * Add retrying flaky jest tests (opensearch-project#2967) * Record functional test artifacts in case of failure (opensearch-project#3190) * Increase timeout waiting for the exiting of an optimizer worker (opensearch-project#3193) * Improve yarn's performance in workflows by caching yarn's cache folder (opensearch-project#3194) * Fix the version of Chrome to 107 to match `chromedriver@107.0.1`, the last that works on NodeJS v10 * Fix unit tests of ExecutionContract throwing AbortError Signed-off-by: Miki <miki@amazon.com>

* Run GitHub workflow unit tests in band (opensearch-project#1306) * Fix JUnit Reporter test (opensearch-project#1338) * Add src/test_utils/ and src/core paths to the target clean patterns (opensearch-project#1442) * Add code coverage reporting (opensearch-project#1478) * Add release build verification workflow (opensearch-project#1502) * Add Windows CI to the workflow (opensearch-project#2966) * Add retrying flaky jest tests (opensearch-project#2967) * Record functional test artifacts in case of failure (opensearch-project#3190) * Increase timeout waiting for the exiting of an optimizer worker (opensearch-project#3193) * Improve yarn's performance in workflows by caching yarn's cache folder (opensearch-project#3194) * Fix the version of Chrome to 107 to match `chromedriver@107.0.1`, the last that works on NodeJS v10 * Fix unit tests of ExecutionContract throwing AbortError Signed-off-by: Miki <miki@amazon.com>

* Run GitHub workflow unit tests in band (opensearch-project#1306) * Fix JUnit Reporter test (opensearch-project#1338) * Add src/test_utils/ and src/core paths to the target clean patterns (opensearch-project#1442) * Add code coverage reporting (opensearch-project#1478) * Add release build verification workflow (opensearch-project#1502) * Add Windows CI to the workflow (opensearch-project#2966) * Add retrying flaky jest tests (opensearch-project#2967) * Record functional test artifacts in case of failure (opensearch-project#3190) * Increase timeout waiting for the exiting of an optimizer worker (opensearch-project#3193) * Improve yarn's performance in workflows by caching yarn's cache folder (opensearch-project#3194) * Fix the version of Chrome to 107 to match `chromedriver@107.0.1`, the last that works on NodeJS v10 * Prevent `AbortError` from breaking unit tests for ExecutionContract Signed-off-by: Miki <miki@amazon.com>

* Run GitHub workflow unit tests in band (#1306) * Fix JUnit Reporter test (#1338) * Add src/test_utils/ and src/core paths to the target clean patterns (#1442) * Add code coverage reporting (#1478) * Add release build verification workflow (#1502) * Add Windows CI to the workflow (#2966) * Add retrying flaky jest tests (#2967) * Record functional test artifacts in case of failure (#3190) * Increase timeout waiting for the exiting of an optimizer worker (#3193) * Improve yarn's performance in workflows by caching yarn's cache folder (#3194) * Fix the version of Chrome to 107 to match `chromedriver@107.0.1`, the last that works on NodeJS v10 * Prevent `AbortError` from breaking unit tests for ExecutionContract Signed-off-by: Miki <miki@amazon.com> Signed-off-by: Miki <miki@amazon.com>

…ject#3299) * Run GitHub workflow unit tests in band (opensearch-project#1306) * Fix JUnit Reporter test (opensearch-project#1338) * Add src/test_utils/ and src/core paths to the target clean patterns (opensearch-project#1442) * Add code coverage reporting (opensearch-project#1478) * Add release build verification workflow (opensearch-project#1502) * Add Windows CI to the workflow (opensearch-project#2966) * Add retrying flaky jest tests (opensearch-project#2967) * Record functional test artifacts in case of failure (opensearch-project#3190) * Increase timeout waiting for the exiting of an optimizer worker (opensearch-project#3193) * Improve yarn's performance in workflows by caching yarn's cache folder (opensearch-project#3194) * Fix the version of Chrome to 107 to match `chromedriver@107.0.1`, the last that works on NodeJS v10 * Prevent `AbortError` from breaking unit tests for ExecutionContract Signed-off-by: Miki <miki@amazon.com> Signed-off-by: Miki <amoo_miki@yahoo.com>

* Run GitHub workflow unit tests in band (opensearch-project#1306) * Fix JUnit Reporter test (opensearch-project#1338) * Add src/test_utils/ and src/core paths to the target clean patterns (opensearch-project#1442) * Add code coverage reporting (opensearch-project#1478) * Add release build verification workflow (opensearch-project#1502) * Add Windows CI to the workflow (opensearch-project#2966) * Add retrying flaky jest tests (opensearch-project#2967) * Record functional test artifacts in case of failure (opensearch-project#3190) * Increase timeout waiting for the exiting of an optimizer worker (opensearch-project#3193) * Improve yarn's performance in workflows by caching yarn's cache folder (opensearch-project#3194) * Fix the version of Chrome to 107 to match `chromedriver@107.0.1`, the last that works on NodeJS v10 * Prevent `AbortError` from breaking unit tests for ExecutionContract Signed-off-by: Miki <miki@amazon.com>

* Run GitHub workflow unit tests in band (opensearch-project#1306) * Fix JUnit Reporter test (opensearch-project#1338) * Add src/test_utils/ and src/core paths to the target clean patterns (opensearch-project#1442) * Add code coverage reporting (opensearch-project#1478) * Add release build verification workflow (opensearch-project#1502) * Add Windows CI to the workflow (opensearch-project#2966) * Add retrying flaky jest tests (opensearch-project#2967) * Record functional test artifacts in case of failure (opensearch-project#3190) * Increase timeout waiting for the exiting of an optimizer worker (opensearch-project#3193) * Improve yarn's performance in workflows by caching yarn's cache folder (opensearch-project#3194) * Fix the version of Chrome to 107 to match `chromedriver@107.0.1`, the last that works on NodeJS v10 * Prevent `AbortError` from breaking unit tests for ExecutionContract Backport PR: opensearch-project#3299 Signed-off-by: Miki <miki@amazon.com>

* Run GitHub workflow unit tests in band (#1306) * Fix JUnit Reporter test (#1338) * Add src/test_utils/ and src/core paths to the target clean patterns (#1442) * Add code coverage reporting (#1478) * Add release build verification workflow (#1502) * Add Windows CI to the workflow (#2966) * Add retrying flaky jest tests (#2967) * Record functional test artifacts in case of failure (#3190) * Increase timeout waiting for the exiting of an optimizer worker (#3193) * Improve yarn's performance in workflows by caching yarn's cache folder (#3194) * Fix the version of Chrome to 107 to match `chromedriver@107.0.1`, the last that works on NodeJS v10 * Prevent `AbortError` from breaking unit tests for ExecutionContract Signed-off-by: Miki <miki@amazon.com> Signed-off-by: Miki <amoo_miki@yahoo.com> Signed-off-by: Miki <miki@amazon.com> Signed-off-by: Miki <amoo_miki@yahoo.com>

kavilla requested a review from a team as a code owner March 10, 2022 10:00

kavilla added the test:integration label Mar 10, 2022

mihirsoni approved these changes Mar 10, 2022

View reviewed changes

mihirsoni reviewed Mar 10, 2022

View reviewed changes

ashwin-pc approved these changes Mar 10, 2022

View reviewed changes

kavilla merged commit f7f9c22 into opensearch-project:main Mar 10, 2022

kavilla deleted the avillk/junit-test-reporter branch March 10, 2022 20:03

AMoo-Miki mentioned this pull request Jan 20, 2023

Enhance the Build and Test workflow #3299

Merged

8 tasks

AMoo-Miki mentioned this pull request Jan 23, 2023

[Manual backport 1.3] Enhance the Build and Test workflow (#3299) #3310

Merged

8 tasks

ananzh mentioned this pull request Jan 24, 2023

[Manual Backport][1.3]Enhance the Build and Test workflow #3314

Closed

8 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Tests] fix JUnit Reporter test #1338

[Tests] fix JUnit Reporter test #1338

kavilla commented Mar 10, 2022 •

edited

Loading

mihirsoni left a comment

mihirsoni Mar 10, 2022

tmarkley commented Mar 14, 2022

kavilla commented Mar 14, 2022

[Tests] fix JUnit Reporter test #1338

[Tests] fix JUnit Reporter test #1338

Conversation

kavilla commented Mar 10, 2022 • edited Loading

Description

Issues Resolved

Check List

mihirsoni left a comment

Choose a reason for hiding this comment

mihirsoni Mar 10, 2022

Choose a reason for hiding this comment

tmarkley commented Mar 14, 2022

kavilla commented Mar 14, 2022

kavilla commented Mar 10, 2022 •

edited

Loading