Skip to content

Commit

Permalink
ci: linux_hil: collect coverage info
Browse files Browse the repository at this point in the history
Capture code coverage information from running the integration HIL
tests against the Linux port, using `lcov` and `genhtml`. Save the
HTML report as an artifact. Comment on the Pull Request (if that
was the trigger) with overall coverage percentages.

Signed-off-by: Sam Friedman <sam@golioth.io>
  • Loading branch information
sam-golioth committed Mar 22, 2024
1 parent 6a4a533 commit 0d95c25
Showing 1 changed file with 71 additions and 7 deletions.
78 changes: 71 additions & 7 deletions .github/workflows/hil_test_linux.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,32 +26,96 @@ jobs:
- name: Compile
shell: bash
run: |
rm -rf test_binaries
mkdir -p test_binaries
export EXTRA_BUILD_ARGS=-DCONFIG_GOLIOTH_COAP_HOST_URI=${{ inputs.coap_gateway_url }}
for test in `ls tests/hil/tests`
do
cmake -B build -S tests/hil/platform/linux $EXTRA_BUILD_ARGS -DGOLIOTH_HIL_TEST=$test
make -j8 -C build
mv build/hil test_binaries/${test}-test
cmake -B build/$test -S tests/hil/platform/linux $EXTRA_BUILD_ARGS -DGOLIOTH_HIL_TEST=$test
make -j8 -C build/$test
done
- name: Setup Python dependencies
run: |
pip install --upgrade pip
pip install pytest pytest-timeout
pip install tests/hil/scripts/pytest-hil
pip install git+https://github.com/golioth/python-golioth-tools@v0.6.2
- name: Install Linux dependencies
run: |
sudo apt-get install lcov
- name: Run test
id: run_test
shell: bash
run: |
# Get coverage baseline
lcov -i -o baseline.info -c \
-d build/connection \
--include "*/golioth-firmware-sdk/src/*"
LCOV_FILES="-a baseline.info"
for test in `ls tests/hil/tests`
do
pytest --rootdir . tests/hil/tests/$test \
--board linux \
--fw-image test_binaries/${test}-test \
--fw-image build/${test}/hil \
--api-url ${{ inputs.api-url }} \
--api-key ${{ secrets[inputs.api-key-id] }} \
--timeout=600
lcov -c \
--directory build/${test} \
--include "*/golioth-firmware-sdk/src/*" \
-o ${test}.info
LCOV_FILES="$LCOV_FILES -a ${test}.info"
done
lcov $LCOV_FILES -o all_tests.info
genhtml -o coverage_html \
--rc genhtml_med_limit=50 \
--rc genhtml_hi_limit=80 \
all_tests.info
# Generate coverage summary and format into markdown table
coverage_summary=`lcov --summary all_tests.info`
coverage_summary=${coverage_summary#*$'\n'*$'\n'}
coverage_summary=${coverage_summary%% branches*}
coverage_summary="$(echo "$coverage_summary" | sed 's/\.*:/ |/g' | sed 's/^ /|/g' | sed 's/)/)|/g')"
# Echo coverage summary into GitHub Output, with title
echo "coverage_summary<<EOF" >> $GITHUB_OUTPUT
echo "# Code Coverage" >> $GITHUB_OUTPUT
echo "| Type | Coverage |" >> $GITHUB_OUTPUT
echo "| --- | --- |" >> $GITHUB_OUTPUT
echo "$coverage_summary" >> $GITHUB_OUTPUT
echo "EOF" >> $GITHUB_OUTPUT
- name: Find Comment
uses: peter-evans/find-comment@v1
if: github.event_name == 'pull_request'
id: fc
with:
issue-number: ${{ github.event.pull_request.number }}
comment-author: 'github-actions[bot]'
body-includes: Code Coverage
- name: Create code coverage comment
uses: peter-evans/create-or-update-comment@v1
if: |
github.event_name == 'pull_request' &&
steps.fc.outputs.comment-id == ''
with:
issue-number: ${{ github.event.pull_request.number }}
body: `${{ steps.run_test.outputs.coverage_summary }}`
- name: Update code coverage comment
uses: peter-evans/create-or-update-comment@v1
if: |
github.event_name == 'pull_request' &&
steps.fc.outputs.comment-id != ''
with:
comment-id: ${{ steps.fc.outputs.comment-id }}
body: `${{ steps.run_test.outputs.coverage_summary }}`
- name: Upload artifacts
uses: actions/upload-artifact@v4
with:
name: linux-hil-test-coverage
path: |
coverage_html/*

0 comments on commit 0d95c25

Please sign in to comment.