Initial manifest file for flatpak creation

[deisi]

A rough but working initial flatpak manifest.

The idea of this pull request is that it can be reviewed early in the development process. Also I will need some help as a couple of things will needed to be decided on. I attached a list of Todos, but I haven't put too much thought in to it. Further I'm learning by doing here as I have only little experience with creating flatpaks.

What are your comments? How should this be approached? My Questions are:

• What build pipeline should be used?
• What Identifier (app-id) should be used?
  It must have the form of org.developer.programm but org.jabref.jabref seems a bit strange to me.
• Should there be daily/beta builds hooked into flathub (documentation is not too big on that feature https://blogs.gnome.org/alexl/2019/02/19/changes-in-flathub-land/)

---

• Change in CHANGELOG.md described
• Tests created for changes
• Manually tested changed features in running JabRef
• Screenshots added in PR description (for bigger UI changes)
• Ensured that the git commit message is a good one
• Check documentation status (Issue created for outdated help page at help.jabref.org?)
• Desktop file
• Icon file
• Correct app-id
• Build pipeline
  • flathub
  • Beta builds
  • Stable builds
• Github actions
• Sandbox permissions
• Automatic source download

[LyzardKing]

I have to admit I'm no flatpak expert, but I have a fist questions..
Does the "sdk-extensions" and relevant module add the jre to the flatpak?
If so it's not needed, since the jabref build system includes that in the build.
If you explicitly want to disable that you need to fetch the jre only, but I'm not sure it's available anymore in binary form. @Siedlerchr will correct me if I'm wrong.

[deisi]

I have to admit I'm no flatpak expert

I guess if there would be one amongst your lines, a package would already exist. :)

Does the "sdk-extensions" and relevant module add the jre to the flatpak?

Yes this adds the jre I did this according to https://github.com/flathub/org.freedesktop.Sdk.Extension.openjdk11

If so it's not needed, since the jabref build system includes that in the build.
If you explicitly want to disable that you need to fetch the jre only, but I'm not sure it's available anymore

This I don't understand. Do you mean the jdk is not needed but the jre is, or is neither nor needed? If latter, what do I have to fetch the jre binary for?

[LyzardKing]

None is needed.
The jdk would be needed only to build, but you're pulling the prebuilt package.
The jre is needed only if you install a version that doesn't already include it ;)
I think you can remove both and be done with it. Everything is done at build time basically.
if you want to reduce the size of the package and rely on the extension it's another thing, but you need to remove some parts of the package you're downloading...

[Siedlerchr]

For java 11 there is the jdk only required for manually building jabref from source.
The prebuilt packages are created using jlink , they include custom version of the java 11 runtime, e.g. all (or only) the needed classes and modules to run the application, so you don't need to specify this.

[LyzardKing]

Can I ask a couple of questions, to figure out how to better integrate this in the build system...
Where are you getting the tar file from?
To get the flatpak on flathub (I suppose that's the ultimate goal) does it have to live in the flathub github repo?

[LyzardKing]

https://github.com/flathub/flathub/wiki/App-Submission#how-to-submit-an-app
As far as I can tell if we want the app to be on flathub the best way to do that is to open this PR (we can test it here) on the flathub repo.
The sources part should be set to url: http://builds.jabref.org/master/JabRef-portable_linux.tar.gz
I'll see if that can work

[tobiasdiez]

Thanks for working on this!

Concerning your questions:

1. Should there be daily/beta builds hooked into flathub (documentation is not too big on that feature https://blogs.gnome.org/alexl/2019/02/19/changes-in-flathub-land/)

   According to your link

   This isn’t meant to be used for nightly builds, but for releases that has some level of testing and are expected to mostly work and be usable to non-developer end-users.

   Thus, daily builds are sadly not supported. But I think it would be valuable to get alphas/betas also into flatpak.

2. What build pipeline should be used?
   We use github actions for our nightly build, but have not yet created a workflow for releases. If you want to play around with github actions, then feel free to go ahead and create one for releases. However, for now it would be also fine to provide detailed instructions of how to push a new version to flatpak (looks like it's a simple git push).

3. What Identifier (app-id) should be used? It must have the form of org.developer.programm but org.jabref.jabref seems a bit strange to me.
   In this case, we have to use org.jabref.jabref (we sometimes use JabRef team as the developer but org.jabrefteam.jabref looks even worse in my opinion).

[LyzardKing]

Another thing to consider @Siedlerchr is if you want jabref to be available on the flatpak default store (flathub).
In that case the build file has to live on the flathub github organization, as a repo.
So it would not tie in with github actions, but it would pull directly the file http://builds.jabref.org/master/JabRef-portable_linux.tar.gz after every update.
Although it would have to be updated to provide the checksum after every update I believe..

[Siedlerchr]

Hm, I see this is a complicated process then.
From what I see flatpack only operates on tags or specific commits, e.g. releases

[LyzardKing]

I'm no expert, but this it how the official gnome packages use it, and it being a gnome project I gather that's the intended use...
Having the flatpak json file upstream serves no real purpose, since it cannot be pushed to the flathub store (unlike the snap)
They work in a different way, but I believe the best approach is to open a PR in the flathub organization (as linked in a previous commend) so that we can also get more experienced eyes on this.
There might be the possibility in the jabref github actions script to set the checksum and trigger a build, but this has to be asked to the flatpak folks.

[Siedlerchr]

There is an github action/docker for creating flatpack stuff. No idea if this can be used: https://github.com/bilelmoussaoui/flatpak-github-actions

[LyzardKing]

I think that's just for building the flatpak..
I asked on the flatpak irc channel if it's possible to push to flathub, like we can do with the snap

[LyzardKing]

Ok...it is possible...just going over the technicals now.. ;)

[LyzardKing]

Unfortunately bad news came out from IRC:
the upload tool ss only available for custom repos, not for flathub, so users wouldn't have access to it.
I think moving it under the flathub org in github is the best bet atm.
A rebuild can only be triggered by a commit to the flathub repo.
There is no way to trigger a rebuild from upstream.

[deisi]

pull directly the file http://builds.jabref.org/master/JabRef-portable_linux.tar.gz after every update.
Although it would have to be updated to provide the checksum after every update I believe.

Yep that was one of the problems I had when I started building this locally. One has to add a checksum otherwise flatpak doesn't download the source up on building. Therefore I just manually downloaded a version of the .tar.gz file and used that to locally get the manifest to build.
I think that answers

Where are you getting the tar file from?

as well. Like I said its a proof of concept ;) and step one is make it work. Optimization comes after that.

There is an github action/docker for creating flatpack stuff. No idea if this can be used: https://github.com/bilelmoussaoui/flatpak-github-actions

I started looking into that a little. I'm not familiar with github actions as it is a fairly new feature I believe. I have registered to the beta myself now. Maybe I can get it to work.

[deisi]

A rebuild can only be triggered by a commit to the flathub repo.

So that means github actions is useless here? Instead the manifest would life within the flathub repo? So it would basically be a separate project that for the moment has to be manually connected to the jabref builds?

[LyzardKing]

Yes.
The manifest would live in the flathub repo, and would have to be manually updated on every upstream release.

[tobiasdiez]

If I understood you guys correctly, then it is desirable to not only have a flatpak but to have JabRef directly in flathub. In this case, we have no option to provide the latest development version as flatpak but should concentrate on releases/alpha versions.

Does the following work?

• Change the manifest to work with JabRef 4.3.1 downloading from https://github.com/JabRef/jabref/releases
• Create PR against flathub
• Wait until this is accepted and we have our own flathub/jabref repository
• optional: Create beta branch there and add JabRef 5.0.0 alpha

Alternatively, we could wait until 5.0 is released and then go through the above process (depends on how hard it is to have a working flatpak for 4.3.1). The second question is how we minimize the work needed to update the flathub/jabref repository after a release. But in general this looks like a relatively straightforward task that can be automated (as a github action here in this rep):

• Checkout git source of flathub/jabref
• Update version, download url and hash
• Push

[LyzardKing]

I'd actually pull from builds.jabref, so you can build the flatpak with the jpackage...ideally with a non dynamic url, but it should work anyway
It should end up like we do with the snap...
I'd open a pr on flathub.
I tried changing slightly the manifest script, and I can get it to pull/extract from there by setting it as a file instead of archive, but I get an error saying it can't set the command JabRef...
Maybe @deisi can do a better job, since I'm just poking arond really...
The task is what you said...but IF you want to use the jpackage
update the version via the normal build, then update the hash only, push.

If instead you want to use the jar, you would need to add the java runtime, like @deisi did in the beginning, and extra libraries if needed

[deisi]

I get an error saying it can't set the command JabRef

So its building but its not finding the binary to run. This means it doesn't find the binary within its path. This concerns two lines.

tar -xvzf flatpak/JabRef-portable_linux.tar.gz -C /app/

witch unpacks the package into the /app/ folder of the flatpak, and
--env=PATH=/app/jre/bin:/usr/bin:/app/JabRef/bin
defining the PATH variable where the binary should be in /app/JabRef/bin/

just to check, after building you can open a shell within the flatpak container after creation with:
http://docs.flatpak.org/en/latest/debugging.html#running-debugging-tools
its basically
flatpak run --command=sh --devel org.jabref.jabref

and then you can have a look at the package under ls /app and so on.

[deisi]

Change the manifest to work with JabRef 4.3.1

I don't think its too difficult, but I thought I save some work there and just start at version 5. Does version 4 have this nice runtime thing as well that makes it just work without jre? If yes I guess it would just come down to chage the source and be done with it.

I'd open a pr on flathub.

can you point me to it so I can see your changes?

[tobiasdiez]

No, JabRef 4.* still relies on an installed java (to be exact: java 8), even the released alpha of JabRef 5 does not include the java. So we probably need to wait a bit until we release JabRef 5 (or a beta), or just provide a flatpak without publishing it to flathub.

[deisi]

@LyzardKing is it running for you now?

[LyzardKing]

I have a working version now...but I had to make some changes...
I'll make a review with the diff

[deisi]

sry for the slow response. All should be in

[koppor]

Reading http://docs.flatpak.org/en/latest/hosting-a-repository.html, we have to setup a github workflow pushing to a separte GitHub repository hosting the flatpack file only?

[LyzardKing]

That would be to host on a custom repository.
In the case of jabref I think the best way forward is to commit the flatpak to flathub, the "default" repo.
Then most (all) flatpak users would have access to jabref.

[LyzardKing]

This is the wiki describing the process.
The TLDR is: add a repo with the json to the flathub github organization.
The problem is updating every release with the proper checksum.
I'm not sure if that could be automated in github actions.
This is the part that the snap would simplify..
https://github.com/flathub/flathub/wiki/App-Submission

[deisi]

Maybe for now skip the CI Pipeline and create a repoa manually, so one could start testing this?

[LyzardKing]

If the flatpak json is pushed to a repo on flathub
then a bot of some sort (I know nothing about github bots) could:

• trigger at the end of the github actions script, when the builds are pushed to builds.jabref.org
• edit the json on flathub with the updated checksum
• that would trigger the rebuild on flathub

[koppor]

I cannot move on with this as is. - We need to build a separate zip file to keep the SHA sum stable. With a high-volume of updates, the checksum quickly gets invalid: https://flathub.org/builds/#/builders/32/builds/11265

Refs #5399

[LyzardKing]

Once version 5 is released this issue can be resolved, and flatpak can build the "stable" versions..
Another idea is to use a script to automate pushing the new checksum
This could be done after every build on master...on github actions
Then when the stable is released this script could be used for a -dev or -daily flatpak

[koppor]

@LyzardKing Got it with the checksums. During the approval process, this is difficult. Also during peak times, users could get error messages when installing... Maybe, we should "just" release more often new alpha/beta versions. This would have the advantage to have a clean changelog for users not wanting to update daily. Similar to slow ring and fast ring of Windows ^^.

[rugk]

And it seems you've made nearly all steps to get it into Flathub, but I still cannot find it.

[onsah]

I came here when I was trying to install flatpak version of Jabref. I wonder what is the current situation?

[koppor]

We still need to address flathub/flathub#1257 (review).

Think, we will release 5.1 first and then we can work on FlatPak again.