Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

List out test failures in CI log #1737

Merged

Conversation

cliu123
Copy link
Member

@cliu123 cliu123 commented Apr 1, 2022

Signed-off-by: cliu123 lc12251109@gmail.com

Description

  • Category (Enhancement, New feature, Bug fix, Test fix, Refactoring, Maintenance, Documentation) Maintenance
  • Why these changes are required?
    Currently, CI log only gives the number of failing tests, but there is no easy way to find out what tests actually failed.
    With this change, CI log summarizes a list of all failing tests, which facilitates test failure investigation.
  • What is the old behavior before changes and new behavior after changes?

Old behavior:

2022-04-01T07:19:55.2791783Z 1133 tests completed, 13 failed, 46 skipped

New bahavior:

2022-04-01T06:09:12.2380647Z Failed tests for opensearch-security:
2022-04-01T06:09:12.2381069Z [org.opensearch.security.HttpIntegrationTests::testTenantInfo]
2022-04-01T06:09:12.2381586Z [org.opensearch.security.SecurityAdminMigrationTests::testSecurityMigrate]
2022-04-01T06:09:12.2571025Z [org.opensearch.security.SecurityAdminMigrationTests::testSecurityMigrate2]
2022-04-01T06:09:12.2572007Z [org.opensearch.security.auditlog.integration.BasicAuditlogTest::testRestMethod]
2022-04-01T06:09:12.2572879Z [org.opensearch.security.auditlog.integration.BasicAuditlogTest::testRestMethod]
2022-04-01T06:09:12.2573500Z [org.opensearch.security.dlic.dlsfls.DlsTest::testDlsWithMinDocCountZeroAggregations]
2022-04-01T06:09:12.2574262Z [org.opensearch.security.dlic.rest.api.ActionGroupsApiTest::testActionGroupsApi[0]]
2022-04-01T06:09:12.2574916Z [org.opensearch.security.dlic.rest.api.ActionGroupsApiTest::testActionGroupsApi[1]]
2022-04-01T06:09:12.2575545Z [org.opensearch.security.dlic.rest.api.MigrationTests::testSecurityMigrate[0]]
2022-04-01T06:09:12.2576207Z [org.opensearch.security.dlic.rest.api.MigrationTests::testSecurityValidateWithInvalidConfig[0]]
2022-04-01T06:09:12.2576865Z [org.opensearch.security.dlic.rest.api.MigrationTests::testSecurityValidate[0]]
2022-04-01T06:09:12.2577523Z [org.opensearch.security.dlic.rest.api.MigrationTests::testSecurityMigrateWithEmptyPassword[0]]
2022-04-01T06:09:12.2578192Z [org.opensearch.security.dlic.rest.api.MigrationTests::testSecurityMigrateInvalid[0]]
2022-04-01T06:09:12.2578798Z [org.opensearch.security.dlic.rest.api.MigrationTests::testSecurityMigrate[1]]
2022-04-01T06:09:12.2579460Z [org.opensearch.security.dlic.rest.api.MigrationTests::testSecurityValidateWithInvalidConfig[1]]
2022-04-01T06:09:12.2580115Z [org.opensearch.security.dlic.rest.api.MigrationTests::testSecurityValidate[1]]
2022-04-01T06:09:12.2580773Z [org.opensearch.security.dlic.rest.api.MigrationTests::testSecurityMigrateWithEmptyPassword[1]]
2022-04-01T06:09:12.2581439Z [org.opensearch.security.dlic.rest.api.MigrationTests::testSecurityMigrateInvalid[1]]
...
...

Testing

UTs

Check List

  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@cliu123 cliu123 requested a review from a team April 1, 2022 23:24
@cliu123 cliu123 changed the title Summarize all test failures List out test failures in CI log Apr 1, 2022
Signed-off-by: cliu123 <lc12251109@gmail.com>
@cliu123 cliu123 force-pushed the list_out_all_test_failures_in_log branch from c517381 to 7693538 Compare April 1, 2022 23:26
@cliu123 cliu123 merged commit c7e58e5 into opensearch-project:main Apr 3, 2022
@cliu123 cliu123 deleted the list_out_all_test_failures_in_log branch April 3, 2022 05:25
wuychn pushed a commit to ochprince/security that referenced this pull request Mar 16, 2023
Signed-off-by: cliu123 <lc12251109@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants