Tests tracking memory usage (the slow pipeline)

[j-i-l]

This approach actually includes 3 types of tests:

1. fast tests with testthat which run regularly
2. fast tests that are CRAN-like which run on changes (and change requests to) the default branch
3. slow tests that track memory usage

The first two are implemented (about to be - see furrer-lab/devel-abn#100 ), what remains is the tests that include the tracking of memory usage.

Originally posted by @j-i-l in #81

We want to run tests with valgrind enabled (what else?) if we have a release candidate.

Depending on what it is exactly that we want to track it might be enough to run R CMD check with --use-valgrind, in which case we could handle this by setting some variables in the existing github action CRAN_checks.

1. We should decide what sort of memory check we want to run
2. Implement the action accordingly

[matteodelucchi]

Comment:

• compare with rhub containers that use valgrind, how the R installation has to be configured with valgrind. Consider to build new containers for R with valgrind.
• To see what else, check this: https://cran.r-project.org/doc/manuals/R-exts.html#Checking-memory-access I assume CRAN applies all of them, but not sure what they actually are used for / if we also need to run all of them.

Decision:

• Start with implementation of Valgrind only, and keep the others for the future if necessary.

[j-i-l]

When to run and how to trigger

Memory checks should run only on 2 occasions:

1. Automatically when a new Pull Request of a release-... branch is opened (only 1x)
2. Whenever a Pull Request gets the label Memory::test. > Once the Test is done, the action should set the label to either Memory::failure or Memory::pass.

[j-i-l]

Status:

• Evoking R CMD check with both --use-gct and --use-valgrind is running for almost 48h now on my personal device.
• Running with --use-gct alone also exceeded the 6h threshold on the runner.

[j-i-l]

With devtools::test_file we could also follow a matrix-strategy in which we share the built .tar.gz as artifact, install, then test a single file.
Approach similar to: https://code.dblock.org/2021/09/03/generating-task-matrix-by-looping-over-repo-files-with-github-actions.html

[j-i-l]

Going to reopen this and add the on-label workflow directly.