Migrate to Compose v2

[scp93ch]

The docker-compose command is reaching end of life and we need to move to docker compose.

See https://docs.docker.com/compose/migrate/

[danshearer]

This URL given heavily references Docker Desktop, something we explicitly do not and cannot support, and in any case for which we have zero known users.

I propose we ignore Compose v2 (WONTFIX) until it becomes a bug causing us pain, because fixing it will required careful testing and therefore developer time, and without a timemachine we can't tell what the options will be when/if using Compose v1 becomes a bug. I have assigned myself this issue and propose closing it unless someone thinks of a reason why not.

It is true that there is a bug in INSTALL.md where Docker Desktop is acknowledged as a way to install Spyderisk, and I will remove that incorrect section. We don't test it, we don't use it, and it is a closed source platform that we can't assume users have any more than Windows.

[scp93ch]

I disagree with most of that @danshearer

The URL does not heavily reference Docker Desktop. I use Docker Desktop, though am happy to switch away from it if there is an equally simple solution for Windows.

At a glance it looks likely that we will not need to change our docker-compose.yml files, though possibly the variable definition fallback values might need looking at. It may be that people using Linux natively are already using Compose v2.

As it says "The final Compose V1 release, version 1.29.2, was May 10, 2021. These packages haven't received any security updates since then. Use at your own risk.". Is that not a good indication to upgrade?

The whole issue is about docker-compose, not Docker Desktop.

[danshearer]

I will just address the bit that jumps out at me here - I managed to miss that you use Docker Desktop on a daily basis, or had forgotten anyway. I thought we didn't have any users for it. So while we do still need to have that discussion about whether we can support Docker Desktop (i.e. by mentioning it at all in the docs) it isn't so obvious as if we had no users at all.

[danshearer]

... "The final Compose V1 release, version 1.29.2, was May 10, 2021. These packages haven't received any security updates since then. Use at your own risk.". Is that not a good indication to upgrade?

I agree we should have nice up to date toolchains, but whether we spend resources on this is about having a sense of the churn it creates vs the attack surface it reduces and that isn't obvious to me.

Does this perhaps belong in system-modeller-deployment, where the focus is on the tools to deploy Spyderisk in a safe and easy manner, presumably learning lessons to apply back in system-modeller?

[kenmeacham]

This issue has now become more serious, as it has now caused the test pipeline to break.The last successful execution of test-and-publish.yml was on 23 Jul 2024. Since then any commits to dev have produced failures. Generally the message has been "docker-compose: command not found", in the build part of the script.

[kenmeacham]

This issue has now become more serious, as it has now caused the test pipeline to break.The last successful execution of test-and-publish.yml was on 23 Jul 2024. Since then any commits to dev have produced failures. Generally the message has been "docker-compose: command not found", in the build part of the script.

It turns out that images used to run the pipelines stopped using "docker-compose" on 29 Jul 2024, so we are now forced to upgrade to v2:

actions/runner-images#9692

Further info at: https://docs.docker.com/compose/migrate/

[kenmeacham]

According to the advice, if using docker compose v2, the main change to make in pipeline scripts, etc is to replace "docker- compose" by "docker compose".

I have done this in the test-and-publish.yml script (direct fixes in dev), and this allows the script to run successfully again :)

Any existing branches will need this fix merged in, to allow tests to run successfully. I have done this so far on branch 191

[panositi]

our servers have explicitly installed docker-compose v1. I guess we can install docker compose v2 along side as this one is a plugin to docker.

To install docker compose v2 using the distribution packages is simple: sudo apt-get install docker-compose-v2

[kenmeacham]

@danshearer - I've requested a review of the related pull request from you, as Stephen says he'll be mostly away in the next couple of weeks. Any questions - just ask!

[scp93ch]

@danshearer can you review this please?