Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upload test report #5035

Merged
merged 64 commits into from
Jan 2, 2024
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
4a3c1cf
Add this branch for testing
Dec 24, 2023
9d55120
Add checks: write
Dec 24, 2023
657b39d
Add test report workflow
Dec 24, 2023
fc66fdf
Newline
Dec 24, 2023
e9b082a
Singular
Dec 24, 2023
e76188e
Testing workflow_run which only triggers off default branch
Dec 24, 2023
d17935d
comments
Dec 24, 2023
0851215
Comments
Dec 25, 2023
57df30e
Give write PR permission and use actions/download-artifact
Dec 25, 2023
147c068
Merge branch 'main' into upload-test-report
Dec 25, 2023
7a4d186
Remove upload/download
Dec 25, 2023
7abd754
Revert "Remove upload/download"
Dec 25, 2023
2ed1da1
Disable comment and checks in unit tests
Dec 25, 2023
d0ab66a
Add unit test report workflow
Dec 25, 2023
d19412d
Add comment
Dec 25, 2023
b608318
Sync update-test-report
Dec 25, 2023
2e81142
Trigger CI run
Dec 25, 2023
097847d
Merge branch 'main' into upload-test-report-1
Dec 25, 2023
2cee895
Use this branch
Dec 25, 2023
4cd4210
List dir
Dec 25, 2023
d54cdb1
Use default run-id
Dec 25, 2023
d642920
Simplify
Dec 25, 2023
4d66ba7
Simplify
Dec 25, 2023
ab9cdcb
Trigger CI
Dec 25, 2023
79274e3
Merge branch 'main' into upload-test-report-1
Dec 25, 2023
178390d
token
Dec 25, 2023
2b98179
Trigger CI
Dec 25, 2023
bed78f6
Specify runid
Dec 26, 2023
fb93374
Trigger CI
Dec 26, 2023
c3d58fd
Merge branch 'main' into upload-test-report-1
Dec 26, 2023
808c55a
run-id
Dec 26, 2023
0c7c518
Merge branch 'main' into upload-test-report-1
Dec 26, 2023
0dcb951
sync
Dec 26, 2023
490a0d3
Merge branch 'main' into upload-test-report-1
Dec 26, 2023
69e2408
Update test report workflow
Dec 26, 2023
7027254
Merge branch 'main' into upload-test-report-1
Dec 26, 2023
737acd8
Run gotip
Dec 26, 2023
627ba9a
Debug
Dec 26, 2023
a154993
Log path
Dec 26, 2023
f5fd5ba
install test tools
Dec 26, 2023
8bdd2f4
which go
Dec 26, 2023
0fdd51f
Remove make install test tools
Dec 26, 2023
e7b7a03
echo go
Dec 26, 2023
b38347f
echo path
Dec 26, 2023
3f9a937
echo contents goroot
Dec 26, 2023
52a612e
echo contents goroot
Dec 26, 2023
b9452aa
echo contents goroot
Dec 26, 2023
07034c2
echo which go
Dec 26, 2023
d9dc36d
Revert
Dec 26, 2023
24a6260
Echo gopath
Dec 26, 2023
07809e2
Export gopath
Dec 26, 2023
23a0223
Reinstate make install-test-tools
Dec 26, 2023
4df70d1
Remove install-test-tools
Dec 26, 2023
5971d1e
WIP
Dec 27, 2023
ffb5a58
Fail test
Dec 29, 2023
ff9d982
pipefail
Dec 30, 2023
2aac566
Separate test report from ci
Dec 30, 2023
7a8e42d
Fix test report failure
Dec 30, 2023
939f773
Merge remote-tracking branch 'upstream/main' into upload-test-report-1
Dec 30, 2023
b831ba6
Preven new test names from being created
Jan 1, 2024
80ac2b4
Merge remote-tracking branch 'upstream/main' into upload-test-report
Jan 1, 2024
40331a8
Cleanup
Jan 1, 2024
7a575e3
Update IDL
Jan 1, 2024
36a32c9
Fix lint
Jan 1, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions .github/workflows/ci-unit-tests-report.yml
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file needs to be merged into main first before we can see the results of the test report summary in PRs, which is why we can't see it in this PR.

Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
name: Unit Test Results

on:
workflow_run:
workflows: ["Unit Tests"]
types:
- completed

jobs:
unit-test-results:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is it necessary to separate this into a different workflow? Doing so requires all the juggling with artifact files. Can we not do all it at once directly in the unit test workflow, where we already call publish reports action?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is it necessary to separate this into a different workflow? Doing so requires all the juggling with artifact files. Can we not do all it at once directly in the unit test workflow, where we already call publish reports action?

It's a valid question, because one would think the action to execute the unit test and generate the results artifact is done as part of this repo's CI pipeline, so the relevant files should be easily accessible.

In fact, we're trying to do that now (publish test results as a comment to this PR) because, I agree, it's a lot simpler than juggling artifact uploads and downloads; but it's failing with this error message (from example run):

2024-01-01 12:55:11 +0000 - publish - INFO - This action is running on a pull_request event for a fork repository. Pull request comments and check runs cannot be created, so disabling these features. To fully run the action on fork repository pull requests, see https://github.com/EnricoMi/publish-unit-test-result-action/blob/v2.12.0/README.md#support-fork-repositories-and-dependabot-branches

I don't fully understand why, but from the log message, it almost sounds like a security measure to prevent PRs from another fork from writing to a PR owned by the "parent" repo, which seems to make sense.

Can you see any other alternative?

name: Unit Test Results
runs-on: ubuntu-latest

permissions:
checks: write

# Needed unless run with comment_mode: off.
pull-requests: write

# Required by download step to access artifacts API.
actions: read

steps:
- name: Harden Runner
uses: step-security/harden-runner@eb238b55efaa70779f274895e782ed17c84f2895 # v2.6.1
with:
egress-policy: audit # TODO: change to 'egress-policy: block' after couple of runs

- name: Download and Extract Artifacts
uses: dawidd6/action-download-artifact@v3
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we know what's different about it? It's preferable to use official actions like https://github.com/actions/download-artifact

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not specifically, but I did spend a good number of hours trying out actions/download-artifact without any luck. Oddly, it wasn't able to discover the uploaded files, but there may have been a silly mistake on my part 🤷🏼 Happy to take suggestions!

with:
run_id: ${{ github.event.workflow_run.id }}
path: artifacts

- name: Publish Test Results
uses: EnricoMi/publish-unit-test-result-action@v2
with:
commit: ${{ github.event.workflow_run.head_sha }}
event_file: artifacts/Event File/event.json
event_name: ${{ github.event.workflow_run.event }}
files: "artifacts/**/*.xml"
20 changes: 19 additions & 1 deletion .github/workflows/ci-unit-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ name: Unit Tests

on:
push:
branches: [main]
branches: [main, upload-test-report]

pull_request:
branches: [main]
Expand All @@ -14,6 +14,7 @@ concurrency:
# See https://github.com/ossf/scorecard/blob/main/docs/checks.md#token-permissions
permissions: # added using https://github.com/step-security/secure-workflows
contents: read
checks: write

jobs:
unit-tests:
Expand Down Expand Up @@ -41,6 +42,13 @@ jobs:
check_name: Unit Tests Summary
junit_files: junit-report.xml

- name: Upload Test Results
if: always()
uses: actions/upload-artifact@v3
with:
name: Test Results
path: junit-report.xml

- name: Setup CODECOV_TOKEN
uses: ./.github/actions/setup-codecov

Expand All @@ -52,3 +60,13 @@ jobs:
flags: unittests
fail_ci_if_error: true
token: ${{ env.CODECOV_TOKEN }}

event-file:
name: "Event File"
runs-on: ubuntu-latest
steps:
- name: Upload
uses: actions/upload-artifact@v3
with:
name: Event File
path: ${{ github.event_path }}