cmake and automated release packaging with static musl builds

[jan-guenter]

Why cmake instead of a Makefile?

• it offers a more flexible build process which comes in handy when adapting to different systems and architectures
• cpack make packaging trivial and might speed up adoption by distro maintainers
• ctest will come in handy when test will be added
• the CMakeList.txt is more readable than a Makefile imho
• my cmake file is half the size and has all the features of the make file, minus the colorful output, plus packaging instructions and more compiler/linker checks

also see comments by @WSLUser in #28

Why static musl builds?

• in contrast to libc you can use getpwuid in static builds without hidden dynamic dependencies
• it has a lower minimum kernel requirement of 2.6.39 (earlier 2.6 versions might also work - needs to be tested)

Automated release builds

I added a GitHub workflow that is triggered whenever a tag matching 'v*..' is pushed.
This workflow will build static musl versions with the following toolchains:

• x86_64-linux-musl
• i686-linux-musl
• aarch64-linux-musl
• arm-linux-musleabi
• arm-linux-musleabihf
  There are more pre-built toolchains available but I lack the hardware to test the produced binaries, so I excluded them for now.

It will also build dynamically linked amd64 DEB packages for Ubuntu 20.04 (focal) and 21.04 (hirsute). (requested by @lucmichalski in #33 )
I'll add other distro packages and architectures shortly. My priorities are CentOS 8, Alpine Linux and Fedora.

After the packages have been built, the workflow will automatically create a GitHub release attaching all the packages.
It extracts the topmost part of the CHANGELOG.md and uses it as the release notes.

Notes

This is still a work in progress and any input/feedback is welcome.

I also incorporated the Makefile changes from the OSX branch into the cmake file, but I don't have Apple hardware to test it with.

If the STATIC option is set in the cmake configuration step it checks if getpwuid can be used in static builds and does NOT set the preprocessor define STATIC_BUILD accordingly. Perhaps a renaming of this define or an additional define might be sensible.

Currently the workflow immediately publishes the release, but it could also be configured to only create a draft that can be reviewed and edited before final release. Let me what you prefer.

This is actually the first time I ever used GitHub workflows. I could draw a bit from my experience with gitlab CI, but I'm very unsure if I utilized all the workflow features exactly as indented. I'm certain it can be improved a lot and I would appreciate if someone would have a look and review my workflow.

I added a *.yml entry to the .editorconfig for easier workflow editing. The indentation style corresponds to the existing continuous-build.yml.

To demonstrate the workflow I added a version tag v1.0.15-1 to my fork with runner and job debug logging enabled:
https://github.com/jan-guenter/btop/actions/runs/
https://github.com/jan-guenter/btop/releases/tag/v1.0.15-1

Todo list

• add install script, readme etc. to archive packages like the static builds
• add more static builds for additional architectures (see the list of available pre-built musl toochain images)
• add more distro packages (especially CentOS 8, Alpine Linux and Fedora)
• add architecture variants to the Ubuntu builds
• update build and installation instructions in README.md
• update snapcraft.yml to use cmake

[aristocratos]

@jan-guenter
Contributing guidelines

[jan-guenter]

@jan-guenter Contributing guidelines

yes, I've read them before opening the PR, Could you please elaborate which part I might have violated?

[aristocratos]

Split up multiple unrelated changes in multiple pull requests.

[jan-guenter]

Split up multiple unrelated changes in multiple pull requests.

Hm, ok, I would argue they are are not unrelated since the whole packaging process and the workflow depend on the use of cmake and cpack.
I'm currently at work, but I'll try to split it up the cmake from the workflow change this evening if that's what you require.
However I see this as a bit problematic since the switch to cmake would cause the current build workflow to fail unless we keep both Makefile and cmake which would mean double maintenance.

[aristocratos]

@jan-guenter
You're making the mistake of basing all the ideas in the pull request on the assumption that cmake would be implemented.

Having the option to easily build statically with musl sounds like a great idea.

Also adding some automated build packages for some distros sounds good, at least until some package maintainer decides to take it on (if they do).

I however don't see any compelling reason to switch over to cmake.

it offers a more flexible build process which comes in handy when adapting to different systems and architectures

No one have yet pointed out on which system/platform the current Makefile fails but cmake would succeed.

cpack make packaging trivial and might speed up adoption by distro maintainers

There's no hurry really.

ctest will come in handy when test will be added

You're making the assumption that tests will be added, and if they were, it really isn't that difficult to add an make test section in the Makefile.

the CMakeList.txt is more readable than a Makefile imho

Subjective statements aren't really that good of an argument.

my cmake file is half the size and has all the features of the make file, minus the colorful output, plus packaging instructions and more compiler/linker checks

minus the colorful output maybe plays a role there. But nonetheless, why is the Makefile size relevant?

[WSLUser]

I think it's great to add musl support but most distros utilize libc. I'd like to see a libc++ option. It should of had a decent amount of improvements ever since Microsoft open sourced STL under same license so some code could be re-used wherever it made sense to do so.

[WSLUser]

We can keep both Makefile and cmake. I'm sure @jan-guenter would ensure the cmake stays up to date if you remain insistent on keeping a Makefile. But I think users preferences should be considered, not just your own. We don't need forks showing up due to refusal to changes you don't think would be helpful. If Makefile really was so much better then why have so many projects switched to Cmake then? Good points were raised about why. I hope you'll reconsider them and do research into the benefits.

[joske]

Well I do not agree, it's his project, he gets to decide. This is how open source works. Anyone is free to fork if they disagree. But the people doing the work get to decide.

I see no reason to use CMake, or to change to any other build system when the current one works just fine. Just because it may be better for other projects doesn't mean everyone needs to switch. Make works fine for simple projects with only a handful of files.

[WSLUser]

Make works fine for simple projects with only a handful of files.

Sure, until you run into a system that Make doesn't support. Make doesn't support every architecture or Operating System and I think from a user perspective, if you're going to make this available for people to use, then you should make it available to everyone everywhere. Make won't do that. Cmake will however.

[aristocratos]

@WSLUser

Make doesn't support every architecture or Operating System

Neither does btop++ so this really is irrelevant.
What system are you imagining that is so constrained that it can't run make but somehow can run btop++?

I'm sorry but your arguments makes me think you don't know what you are talking about.

[WSLUser]

While btop++ may not support other platforms currently, that is no reason not to future proof so that in the future, someone could provide that support. I will let others advance the discussion for cmake. And again, I'd ask you to research it yourself.

[aristocratos]

@WSLUser

While btop++ may not support other platforms currently, that is no reason not to future proof so that in the future, someone could provide that support.

Again, what are these hypothetical future platforms that doesn't support make? I mean, even windows have multiple ways of running make.

I will let others advance the discussion for cmake.

Why not defend your arguments if it's something you want to see get implemented.

And again, I'd ask you to research it yourself.

I'd say that it would be your job to research and provide sources if you want to convince others of your points.

My decision on the build system is not set in stone, but I'm not swayed by personal opinions and claims not being backed up by actual facts.

[jan-guenter]

@aristocratos, thanks for your detailed reply earlier today.

You're making the mistake of basing all the ideas in the pull request on the assumption that cmake would be implemented.

Not exactly, moving to cmake was the first step for me to allow myself to easily package btop++. In my daily business I'm working with a lot of different Linux distros and have my local package caches I use to have the apps I use easily installable through the distros package managers.
So my effort to transform your Makefile into cmake was mainly for myself. The musl builds were just a byproduct of having to rewrite the build flow and me being familiar with the hassles of statically linking against libc I decided not to reuse the toolchains you used in your workflow but to utilize the alpine base musl toolchain images I'm familiar with.

Before I even started this, I read #28 and picked up on you rather negative stance to cmake; which by the way is totally fine with me. I created this PR draft to see if perhaps seeing cmake in action an having a look at a working cmake file would change your opinion on it. In the end it always is just a matter of personal preference and everybody is entitled to their own opinion, but there is no harm in trying to convince someone.

---

@WSLUser

I think it's great to add musl support but most distros utilize libc.

You missed the point here. The great thing about musl is that you can completely statically link against it which isn't true for glibc (see STATIC_BUILD define in btop++).

I'd like to see a libc++ option.

libc++ is still lacking a good amount of c++20 feature which are used by this project. (https://libcxx.llvm.org/Status/Cxx20.html)

We can keep both Makefile and cmake. I'm sure @jan-guenter would ensure the cmake stays up to date if you remain insistent on keeping a Makefile.

It makes no sense to keep both in the same project. If I @aristocratos decides to stick with a Makefile, I'll simply merge it to the master of my fork and you are free to use it if you like. I'll try to keep it up to date at least until all the major distros have upstream packages available and I don't need to package it myself any longer.

---

@joske

Well I do not agree, it's his project, he gets to decide. This is how open source works.

I totally agree with you, it is @aristocratos decision and his alone. I forked the project and added cmake for my personal benefit, but since I'm still convinced that it would benefit the project in the long run, I decided to draft this PR and invested the extra effort of creating a packaging workflow to demonstrate the usefulness of cmake even in the currenlty limited scope of this project.

Anyone is free to fork if they disagree.

Exactly, as mentioned above if the PR gets declined, I'll simply maintain it in my fork and that's totally fine.

---

@aristocratos

With all the general replies out of the way let me try to explain in more detail why I would choose cmake over a Makefile.
As I said before it's totally fine if you disagree since it simply is a matter of personal preference. Just for clarification I used to write a lot of Makefile in my almost 2 decades as a C++ developer and architect. I'm now working in DevOPs, helping our dev teams organizing their projects, automate tedious tasks and overall trying to make their life simpler.

I agree with you that in the current state of the project the Makefile is still easily maintainable and does everything you need in the current scope of the project. Even thou I would disagree with a lot that @WSLUser wrote, but his notion of future-proofing has its merits. There is a reason why most Makefile projects utilize separate configure scripts - segregation of duties. Currently you only make a handful of decisions about how to build the project and its therefore justifiable to have them in the Makefile.
However, I can only imagine what this fantastic project you created will grow into. Unless you are hell-bent to constrict this project to never have any external dependencies you will at some point sooner or later need to deal with externals. This means you will either have a configure script or a massively bloated Makefile.

CMake is not a build tool pre se. I cannot recall who it was but at CppCon some speaker described it as a meta build system, which hits the nail on its head. CMake offers a simple scripting language tailored to generate build script like Makefiles, or even full blown projects files for common IDEs. Some IDEs even support loading CMake files itself as they were projects (e.g. VisualStudio or CLion).

Many of the usual tedious tasks within configure scripts or Makefiles like detection of compiler features, libraries, systems and architectures are abstracted away from the developer and packages into reusable functions. When trying to achive similar things in Makefiles you are either copy and pasting solutions or reinventing the wheel. Take the musl support as an example, I'm able explicitly state my actual intent in the cmake file; to test if static linking produces a linker warning (https://github.com/jan-guenter/btop/blob/4c92e5dfb712f5808480b4ea54006f8356849c08/CMakeLists.txt#L82)

Now just imagine, in the future when clang finally supports ranges or someone deciding to utilize clang compatible implementations like Eric Nieblers range-v3 to be able to utilize clang and/or libc++. In a Makefile you have to take care of so many little things which are dependent on the platform and tooling. Those are tasks CMake excels at, by providing ready to use functions and Modules for those common project configuration tasks.
Let alone handling external libraries. If at some point you want to add an external library to this project, utilizing cmakes features for finding and using libraries are way more comfortable especially when talking about packaging where you might want to utilize or package maintainers being required to use the distros versions.

Another compelling reason for CMake is its wide adoption amongst other tools. Just the ability to auto generate a compile database, which can be utilized by so many so many great tools especially code analysis tools like clang-tidy or sonarqube, would be reason enough for me to switch. Such tools don't jus need the sources to work, but they need to know how they are built. Without knowing things like the preprocessor defines used many tools need to resort to exhaustive searches and will produce a lot of false positives or wore simply skip parts due to the lack of information.

Last but not least I want to reiterate my initial point of simple packaging. I'll not repeat everything I wrote earlier, but rather state the following question: Why manually write a RPM spec file, DEB control files etc. if your build system can do it for you?

---

Sorry for the wall of text, but my TL;DR is: I predict a great future for your wonderful project; in the past couple of days I got so much great response from my peers, to which I presented your tool and made it available on many of our systems (just dev and test environments so far; need to await experience and maturity before installing it on production environments).

I said what I wanted to say. It's completely up to you @aristocratos if you want to accept my PR or not. If not, please go ahead and close it, then I'll merge it into the master of my fork for the time being. If you like me to add a Makefile based musl build as a github workflow, I'm happy to help and could take do so on the weekend; just let me know.

[aristocratos]

@jan-guenter
First off great reply and really good points!

To begin with, the main reason for sticking with Make is because of familiarity, it feels like working in a regular shell script. Which might not be a positive for many people but considering the time I spent working on bashtop, it's very comfortable to me.
That's also one of the reasons I'm confident I can fix any issues (or most) that pop up with make/shell logic in the makfile.
This might mean it gets pretty large in the future, at least for people who have a normal sense of what a large shell script is :)

Unless you are hell-bent to constrict this project to never have any external dependencies you will at some point sooner or later need to deal with externals.

I wouldn't say there is really a set goal to not have any external dependencies, I just find the challenge of writing from scratch to be the most rewarding. And also have the benefit of being really efficient (if you get it right) when you don't get any extra bloat you don't need attached.

When trying to achive similar things in Makefiles you are either copy and pasting solutions or reinventing the wheel.

I actually don't mind that part, coming up with your own solution to a problem is a very good way of really learning the systems you are working with. Same as with the point above, that's the most fun part of programming for me.

---

I do appreciate you taking the time to write this PR and giving advice.
And I do agree with many of your points but for now while I'm still have some big chunks of code left to write (FreeBSD support, GPU monitoring support, etc) I'll rather stay with the build system I'm familiar with and can quickly iterate on.
If major tooling issues arise or keeping the Makfile working becomes more hassle than it's worth, I definitively don't see an issue of switching over to cmake. And I would be very thankful for any help you could offer in getting it up and working in that case (if you're still interested at that point).

---

If you like me to add a Makefile based musl build as a github workflow, I'm happy to help and could take do so on the weekend; just let me know.

That would be awesome and very much appreciated since it would solve the big getpwuid issue with static builds currently present.

[jan-guenter]

Sounds good. Here is what I'll do:
I'll merge my cmake branch into the master of my branch keeping the source and your Makefile untouched to avoid merge conflict.
Using some trigger or scheduler (not sure about the details yet) I'll continuously merge your master and automatically run my build workflow on every release tag you add unless the Makefile changes. In case of a Makefile change I'll simply fail the CI workflow which notifies me and I will incorporate those changes into my cmake file.
This way we should have a good starting point if, at some point, there is a reason for you to make the switch.

On the weekend I'll get on adding musl builds to your Makefile and CI workflow.
I'll now open feature request for the musl builds to clarify some details beforehand.

Closing the PR.