Reusable Docker image

[joshfrench]

I'm sure there are considerations that my own preferred workflow doesn't account for, so consider this PR more as a starting point for further discussion. But in brief, it's not strictly necessary to rebuild the Docker image if all you've updated is your keymap. This PR creates a reusable Docker image that can persist between firmware builds by simply mounting the config and firmware directories at runtime.

Pros:

• On my machine, building the Docker image takes ~120s. We can eliminate that if there's no need to update West or its config.
• Currently if the build fails, sometimes the zmk container is left around and needs to be manually removed. Using docker run --rm ... ensures the container is always cleaned up regardless of success.
• Makefile is arguably a better starting point for people who want to customize their build scripts -- targets can be added without clobbering the "official" workflow.

Cons:

• One side effect of running setup.sh every time is that zmk/west are always up to date. Making the Docker rebuild an optional step puts the responsibility on the end user to update those deps by manually rebuilding the image.
• Related to the above, it's not inherently obvious when the Docker image needs to be rebuilt. I've tried to address this in the Readme.
• Adds make to the mix, although if someone is already willing to run Docker locally I think it's a safe bet they have some familiarity with make.

Ultimately you could just instruct people to run make all (or even just make) and they'd get the same experience they do currently; this just adds a little more control over the build process for people who want it.

[ReFil]

Must admit i didn't do the local building stuff so i dont really know how it works

[gohanlon]

I haven't had a chance to try this yet as I haven't rebased to V2.0. (When was V2.0 "released", even?)

From reading, this PR looks great. +1 to using make and to avoiding the super-slow Docker build step. Thanks, @joshfrench!

[craftyguy]

I haven't had a chance to try this yet as I haven't rebased to V2.0. (When was V2.0 "released", even?)

they created a v2.0 branch, but nothing is tagged... which is a little confusing, because how do you even know what v2.0 is if they're pushing new patches to that branch??? anyways, maybe it's not meant to be "released" yet?

[ed-flanagan]

Commenting mostly out of curiosity, feel free to ignore

[ed-flanagan]

Might be out-of-scope, and I'm not familiar with west, but might it make sense to not have these as multiple layers¹? Only bringing up since this is about improving the image building process.

Footnotes

1. Hadolint DL3059 ↩

[joshfrench]

Oh, interesting. I combined these RUN lines but it did not make a difference in the image size; apparently it isn't doing too much in the way of transient files? I also don't know enough about west to say which of those commands are likely to bust the layer cache, so I'm inclined to keep them individually cache-able.

[joshfrench]

Ha, my very next run demonstrated this in action:

 => CACHED [4/7] RUN west init -l config
 => [5/7] RUN west update

The rule you linked to makes the tradeoff clear and I'm ok optimizing for speed in this case.

Putting easy cacheable and non-cacheable commands in the same layer (by using one RUN command) might have negative performance issues.

[ed-flanagan]

O sorry, it shouldn't do anything for the size/speed, should just effect the layers that Docker generates. I was thinking it may be a case similar to https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#apt-get (with subcommands like init and update).
Can avoid issues where earlier stages succeed & cached, resulting in later stages to have a(n) inconsistent/un-reproducible/non-updating state. I.e. they're actually coupled stages and should be apart of the same RUN

[ed-flanagan]

I had to apply #25 for v2 branch

[joshfrench]

same, i just omitted it here since there's an open PR for it.

[ReFil]

Pr has since been merged btw

[allanwind]

Let's get this merged so we can add podman support (revised PR #10).

[joshfrench]

🎉

[allanwind]

Works great.

Josh, can you test if more a more qualified url FROM docker.io/zmkfirmware/zmk-build-arm:stable works with docker? It's appears to be required for podman unless you add alias in /etc/containers/registers.conf.d. I can't tell from the docker documentation one way or another.

[joshfrench]

@allanwind Looks good:

% head -n 1 Dockerfile
FROM docker.io/zmkfirmware/zmk-build-arm:stable

% docker build --no-cache --tag zmk --file Dockerfile .
[+] Building 120.4s (12/12) FINISHED
 => [internal] load build definition from Dockerfile
 => => transferring dockerfile: 37B
 => [internal] load .dockerignore
 => => transferring context: 2B
 => [internal] load metadata for docker.io/zmkfirmware/zmk-build-arm:stable
 => [1/7] FROM docker.io/zmkfirmware/zmk-build-arm:stable@sha256:acea7abae2fe324e574e33f63761b1c20bb60714fb528a7987b093dc32206e06
 => [internal] load build context
 => => transferring context: 118B
 => CACHED [2/7] WORKDIR /app
 => [3/7] COPY config/west.yml config/west.yml
 => [4/7] RUN west init -l config
 => [5/7] RUN west update
 => [6/7] RUN west zephyr-export
 => [7/7] COPY bin/build.sh ./
 => exporting to image
 => => exporting layers
 => => writing image sha256:e4dcaeb3fe75bdc1f9f3215f609f407e67e7367b42cac4c83feabb5cda5f8978
 => => naming to docker.io/library/zmk

[allanwind]

Thanks bud!