Build automation (RPM+ISO)

[axk4545]

Currently RPMs are updated, built and added to the repo manually. It would be nice to script this funtionality so that we can have changes pushed to github trigger rebuilding of RPMs and updating of the repo.
The GH sources affected are those for the scripts and possibly the branding.

For help ask about Fedora SOP in #fedora-admin and #fedora-devel on freenode

[axk4545]

Maybe rework GH repo to funtion similar to the fedora lookaside cache

[ct-martin]

I can help with this, hit me up another time and we can brainstorm

[ct-martin]

From discussion:

1. Someone commits to GitHub
2. Pull on the build box (using git pull)
3. Make srpm + rpm
4. Copy to /var/builds-to-sign or somewhere that has "chmod g+w" and "chown build-signers:build-signers" w/ people allowed to sign in that group
5. (long-term) send a message via email and/or IRC to notify things need signing
6. Things get signed manually
7. Run after-signing script which does the following:
8. Copy items to mirrors folders
9. (long-term) Git commit to the GitHub repo w/ a RITlug-BuildBox account
10. (long-term) Post a message via email and/or IRC announcing new items (tigeros-announce?)

[axk4545]

@ct-martin also assigning you though I can't since you aren't on the repo yet. poke me if you want to be added and I will discuss with team and eboard.

[axk4545]

Here are a few option:
Self hosted COPR
Bash scripts and git hooks
mock-scm config options to make a tarball

[jwflory]

Self hosted COPR

For what it's worth, I'm 👎 to a self-hosted COPR because it will be difficult for someone to maintain later on. I'd be concerned about what would happen in four years if we hosted our own COPR.

[ct-martin]

@jflory7 tbh, git+jenkins/ci+bash (aka, what we have now) seems like the best option

[axk4545]

True. It is likely though that the build system will be on our infra unless we can find a good hosted way to do RPM builds. Also needs to be able to function as a dnf repo or put the result in a valid dnf repo. Also signing. Aidan Kahrs abkahrs@gmail.com

…

On Oct 16, 2017 4:57 PM, "Justin W. Flory" ***@***.***> wrote: Self hosted COPR For what it's worth, I'm 👎 to a self-hosted COPR because it will be difficult for someone to maintain later on. I'd be concerned about what would happen in four years if we hosted our own COPR. — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#29 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADUyLFHaZT6kJpzwLQzJaerBDPmtXFtZks5ss8MygaJpZM4NJJ9h> .

[jwflory]

It is likely though that the build system will be on our infra unless we can find a good hosted way to do RPM builds.

Well, if we're hosting it on our own, I'd figure it would have to be hosted on our infrastructure, right? 😉

If COPR is something we want to use, I have a strong preference for using Fedora's COPR and creating a group for the TigerOS team to maintain any packages there. Hosting on our own introduces a significant amount of work, and it would require all of the team to contribute documentation for how to manage that. I encourage all of you to think too about how this will be maintained in five years from now.

[axk4545]

@jflory7 Licensing might be an issue. Also COPR may want to have one repo per package.

[ct-martin]

Status update:

• We have an Ansible playbook that installs all the needed programs and does part of the configuration.
• We (@axk4545) are rethinking CI, will continue once that is decided.

[axk4545]

@ct-martin Travis is a no go. This is gonna be interesting with COPR. Biggest concern is licensing. I would go with automating our self hosted stuff (maybe ala this substituting fedpkg for rpmbuild ) and the process there but be able to migrate to COPR with tito ala this

@jflory7 I would like to get your input on how to make either of those more sustainable

[axk4545]

for help with using tito as describe above in hosted COPR and getting COPR to work with our subdir based approach talk to clime in #fedora-buildsys on freenode

[axk4545]

ideas:

• Lightweight build system section here https://www.hogarthuk.com/?q=node/11
• https://wiki.jenkins-ci.org/display/JENKINS/RPM+Sign+Plugin
• https://github.com/jhrcz/jenkins-rpm-builder
• https://schneide.wordpress.com/2013/03/18/1-click-deployment-of-rpms-with-jenkins/
  @ct-martin

[axk4545]

ok. so we should probably use tito in some way to get around the subdir part. we can run tito with jenkins to trigger. signing can likely use the jenkins plugin and deployment is copying in some way from the output dir to the mirror/repo dir and fixing perms. tito will handle making the tarballs and bascially going from flat source files to RPMs and SRPMs.

[ct-martin]

Having issues with Jenkins, looking into GitLab Runner

[axk4545]

We now have IP addresses and can provision the webhost

[axk4545]

It will trigger based on push to master

[Tjzabel]

I think this is going to take a lot longer than the Beta release to complete. Thoughts on moving this to the final release?

[ct-martin]

@Tjzabel I'm not going to block the beta on it, but I would like to keep it on the beta milestone due to its importance

[Tjzabel]

@ct-martin sounds good.

Also, it is probably a good idea to split this issue into smaller issues so it is much easier to track.
It would be a good idea to have an issue for updating the builder + mirror Ansible playbooks, an issue for the GitLab ISO CI, and an issue for the RPM building CI

[ct-martin]

@Tjzabel I actually have a draft issue for updating the playbooks right now with some other stuff as well. I have a couple thoughts on how to split up the issue, let's talk about them in the meeting in a couple minutes

[Tjzabel]

Next steps are to do the write-up on how TigerOS CI is going to work. Current draft is located on my GitLab repo.

The goal is to implement CI on a single RPM package to make sure the process goes smoothly. Then, a TigerOS mirror will be set up on the GitLab public site, where we will be hosting CI on the builder, and possibly the mirror.

The draft documentation will be added into the wiki at some point.

[Tjzabel]

Furthermore, I have an RPM CI script in the works to do the RPM building.

However, this is still a rough script, and needs to be ironed out in order to make the RPM building process fully automated. At the moment, the release version and package name need to be added as parameters to the script, which are not yet found automatically.

In order to build an RPM package, the specific release is needed. We need to figure out a way to grep for the release version without manually putting it in.

[Tjzabel]

In addition, RPM CI and ISO CI are very different from one another, and so I believe these two CI issues should be separated to create a more easy-to-follow workflow.

[ct-martin]

@Tjzabel could you clarify the following? I have no idea what your mean, and my best guess doesn't get close to what I had planned.

Then, a TigerOS mirror will be set up on the GitLab public site, where we will be hosting CI on the builder, and possibly the mirror.

Also, I have access to the GitHub credentials now so I can proceed when I have time (probably next weekend)

[Tjzabel]

@ct-martin If it helps, I also have no idea what I was trying to write there 😄

I am going to update that comment now with what I actually meant to say.

[Tjzabel]

@ct-martin actually, I do know what I was saying there.

Essentially, there will be a GitLab instance that is mirroring what the GitHub repo has, so when there is a change to the repo, the GitLab CI will pull the change and do the automation. Correct me if this line of thinking isn't what you had in mind.

[ct-martin]

@Tjzabel ok, yes, that is what I had in mind. I was confused by "mirror" having multiple meanings. What do you think about using terminology along the lines of "TigerOS mirror" or "the mirror" for mirrors.ritlug.com (to maintain backwards-compatibility in terminology) and "GitLab repo mirror" for the GitLab CI/CD for external repositories? Wording doesn't have to be exact.

Thanks for the clarification

[Tjzabel]

@ct-martin I got the same confusion when I re-read my own comment.

I definitely think it is a good idea to create uniform terminology when referring to the different parts. Even GitLab mirror is enough imo, and TigerOS mirror for the packages mirror.

[ct-martin]

@Tjzabel I think it would be a good idea to use "GitLab repo mirror" (with "repo") since GitLab Runner will be automating things on the mirror soon. I'll file an issue to start a glossary and we can discuss on Wednesday

[ct-martin]

As per meeting we have decided to do Continuous Delivery to a new development mirror based on the devel branches of the respective repos. This means that for devel builds instead of getting put into a queueing folder, they will be signed and put straight on a development mirror separate from the current one. This will also require standardizing branch protection, which I will figure out and document.

[ct-martin]

I have installed GitLab Runner on the builder, will set up the pipelines another time.

[Tjzabel]

This is no longer in scope for a beta release.

[Tjzabel]

Update

Currently testing out the ISO-building Dockerfile in order to build the F29 ISO.
We will be one step closer to adding build automation if this succeeds 👍