Developer guides – Github CI

We are using Github actions to manage our CI pipeline for automated testing. This page will shortly introduce the setup of the pipeline as well as the jobs that are run during its execution.

• Setup
• Jobs

General notice: Since we write (and read) files during our test pipeline, it can sometimes happen that the pipeline fails when it tries to remove output files to soon, as the recommerce process may still have it open. In many of these cases, simply restarting the failed job should fix the issue.

Setup

Our CI pipeline is configured to run on each commit and pull request. The specific implementation can be found in the .github/workflows/Pre-commmit and Tests.yml file. Currently, the pipeline runs three separate jobs, described in the following section.

As one run of the pipeline can take up to 30 minutes, any running action will be automatically cancelled if another commit is pushed to the same branch to save time and resources.

All our jobs are labelled with self-hosted in the category runs-on, which makes sure that the pipeline is only run on the virtual machine set up to run using resources of the EPIC chair of the HPI. If you need access to this VM, for debugging or maintenance reasons, please contact the EPIC chair directly, referring to the VM regarding the Github CI pipeline.

If you have the corresponding privileges for the repository, you can also view the state and edit the labels of the runner in the repository settings under Actions/Runners.

Jobs

At the moment, three separate jobs are run on each commit. These jobs must succeed before a pull request can be merged.

Pre-commit

The pre-commit job is the first and quickest job that is run in the pipeline. It runs the pre-commit run --all-files command on the codebase, checking for formatting and code style errors. If the pre-commit check fails at any point, so does the pipeline.

Please note that due to the setup of the pre-commit job, the job will fail if pre-commit attempts to make changes to a file, e.g. change the ordering of imports through the isort plugin. For this reason, we always recommend setting up pre-commit to run locally before each commit as well.

Webserver tests

During the second job, the tests written for the user interface/webserver are run. If any of the tests fail, so will the pipeline. The job is run with a verbosity level of -v 2, so if a test fails, the log output should contain enough information to debug the issue.

Recommerce tests

Lastly, all tests written for the recommerce simulation framework itself are run. This job can take in excess of 20 minutes to complete, as many combinations of the training of reinforcement learning agents are run to make sure all processes are working as expected, so bring some patience.