Build deb package #67

zmwangx · 2016-04-29T04:28:46Z

I have improved my deb package building script and figured out uploading from Travis to GitHub releases. If you are interested in one more installation channel, this is almost ready to be integrated. (By the way, I doubt that googler could make its way into Debian's official repositories, so IMO either this or a PPA is the best we can do.)

My script can build a deb package off any commit-ish, and .travis.yml has been set up to upload the artifact to each pushed tag. Here is an example uploaded automatically from https://travis-ci.org/zmwangx/googler/jobs/126560102:

(Update: See #67 (comment) for something better.)

Note that this PR cannot be merged as is. All commits except the .travis.yml one are ready, but .travis.yml needs to be adapted to use your API token and upload from jarun/googler instead of zmwangx/googler. I believe all you need to do is to create a token with public_repo access, encrypt the token with Travis's CLI client:

travis encrypt -r jarun/googler $TOKEN

and replace the current ciphertext with the ciphertext you get.

zmwangx · 2016-04-29T04:58:23Z

Okay, I made a mistake when I set the architecture to any instead of all, which led to non-universal packages. It's been fixed, and now packages work across architectures. New example from https://travis-ci.org/zmwangx/googler/jobs/126565197:

jarun · 2016-04-29T04:59:33Z

No problem! I'll review it.
And try the installation too! :)

jarun · 2016-04-29T05:03:47Z

From the archive contents it seems you are installing to /usr rather than /usr/local PREFIX which we use on other platforms.

zmwangx · 2016-04-29T05:09:57Z

That's a very simple change, but shouldn't deb packages install to /usr? I think Linux package managers manage /usr instead of /usr/local, right? The AUR package also installs to /usr: https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=googler.

zmwangx · 2016-04-29T05:12:00Z

tools/makedeb

+ #
+ # Note that date will be empty if the tag at question is a lightweight
+ # tag (i.e., non-annotated) because they don't carry any tagger
+ # information; it that case we simply fall back to the committer.


Aww hell, typo... Feels really bad when I need to rebase for a typo... I'll push a fixup commit for now and only squash after review.

jarun · 2016-04-29T05:14:28Z

Let's keep it common to AUR. It depends on us, not the package manager. But I don't think we would complain if someday googler is accepted officially on Ubuntu ;).

/usr it is! 👍

zmwangx · 2016-04-29T05:18:26Z

But I don't think we would complain if someday googler is accepted officially on Ubuntu ;).

Of course not...

I've never tried to submit anything / watch others submit anything to Debian/Ubuntu though, nor do I fancy doing that. Have you? (Honestly I'm not a Debian/Ubuntu fan. Stability my ass... I like Arch much better.)

jarun · 2016-04-29T05:20:18Z

I loved Lubuntu. Now I have a dual boot with Manjaro LXQt and Lubuntu.
Nowadays my system boots into Manjaro only.

But Ubuntu it is for my wife. Perspectives differ, and so does user competence, you see...

jarun · 2016-04-29T05:21:57Z

It'll take me sometime (by EoD my time) to test installation on Ubuntu 16.04 and Ubuntu 14.04. OK? Need the systems. :)

zmwangx · 2016-04-29T05:23:10Z

Perspectives differ

Of course, I personally just can't tolerate the outdatedness of packages...

Anyway, I do have a whole bunch of Ubuntu boxes. I have spun up five (with Vagrant) in the past two days...

It'll take me sometime (by EoD my time) to test installation on Ubuntu 16.04 and Ubuntu 14.04. OK? Need the systems. :)

No problem, this is by no means urgent.

jarun · 2016-04-29T15:46:28Z

Installs with minor errors... you need to fill 2 more fields:

$ sudo dpkg -i googler_2.4-testdeb-2-1_all.deb 
dpkg: warning: parsing file '/var/lib/dpkg/available' near line 35370 package 'min':
 missing description
dpkg: warning: parsing file '/var/lib/dpkg/available' near line 35370 package 'min':
 missing maintainer
Selecting previously unselected package googler.
(Reading database ... 121462 files and directories currently installed.)
Preparing to unpack googler_2.4-testdeb-2-1_all.deb ...
Unpacking googler (2.4-testdeb-2-1) ...
Setting up googler (2.4-testdeb-2-1) ...
Processing triggers for man-db (2.6.7.1-1ubuntu1) ...
$ cat /etc/issue
Ubuntu 14.04.4 LTS \n \l

jarun · 2016-04-29T15:46:53Z

Runs fine.

jarun · 2016-04-29T15:49:46Z

Also when I try to download the changelog from synaptic:

This change is not coming from a source that supports changelogs.

Failed to fetch the changelog for googler
URI was: http:///.changelog

Please add a changelog file in the same pull request. we'll update it from the next release.

zmwangx · 2016-04-29T15:52:40Z

If you look at the script, description and maintainer fields are there, and there is a changelog (just a pointer to the GitHub release page, I can't automatically generate a Debian-compliant changelog). And

vagrant-ubuntu-wily-64 8:49:13 [/tmp] >>>
curl -LO https://github.com/zmwangx/googler/releases/download/v2.4-testdeb-2/googler_2.4-testdeb-2-1_all.deb
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   608    0   608    0     0    883      0 --:--:-- --:--:-- --:--:--   883
100 18250  100 18250    0     0  12947      0  0:00:01  0:00:01 --:--:-- 36794

vagrant-ubuntu-wily-64 8:49:26 [/tmp] >>>
sudo dpkg -i googler_2.4-testdeb-2-1_all.deb
Selecting previously unselected package googler.
(Reading database ... 93513 files and directories currently installed.)
Preparing to unpack googler_2.4-testdeb-2-1_all.deb ...
Unpacking googler (2.4-testdeb-2-1) ...
Setting up googler (2.4-testdeb-2-1) ...
Processing triggers for man-db (2.7.4-1) ...

vagrant-ubuntu-wily-64 8:49:38 [/tmp] >>>
dpkg -s googler
Package: googler
Status: install ok installed
Priority: optional
Section: misc
Installed-Size: 36
Maintainer: Arun Prakash Jana <[email protected]>
Architecture: all
Version: 2.4-testdeb-2-1
Depends: python (>= 2.7)
Description: Google Search and News from the command-line
 See https://github.com/jarun/googler#readme.

vagrant-ubuntu-wily-64 8:52:13 [/tmp] >>>
dpkg -L googler
/.
/usr
/usr/bin
/usr/bin/googler
/usr/share
/usr/share/man
/usr/share/man/man1
/usr/share/man/man1/googler.1.gz
/usr/share/doc
/usr/share/doc/googler
/usr/share/doc/googler/copyright
/usr/share/doc/googler/README.md.gz
/usr/share/doc/googler/changelog.Debian.gz

vagrant-ubuntu-wily-64 8:52:23 [/tmp] >>>
zcat /usr/share/doc/googler/changelog.Debian.gz
googler (2.4-testdeb-2-1) UNRELEASED; urgency=medium

  * See full changelog at
    https://github.com/jarun/googler/releases/tag/v2.4-testdeb-2

 -- Arun Prakash Jana <[email protected]>  Fri, 29 Apr 2016 02:24:17 +0530

???

zmwangx · 2016-04-29T15:59:40Z

By the way, I thought about including a changelog in this PR, but vetoed that idea, because Debian's changelog format is very peculiar (see example above), and I see no reason why our changelog should follow Debian's format (you won't enjoy appending to those later). And if this ever acquires any more official status, our changelog will be pretty much useless anyway, it will need to be hand-crafted (to include ticket numbers and such). As an auto-generated deb I'm pretty happy with the current changelog I'm generating.

zmwangx · 2016-04-29T16:02:14Z

dpkg: warning: parsing file '/var/lib/dpkg/available' near line 35370 package 'min':
 missing description
dpkg: warning: parsing file '/var/lib/dpkg/available' near line 35370 package 'min':
 missing maintainer

Oh, so the package min is missing description and maintainer, not googler 😉

zmwangx · 2016-04-29T16:03:21Z

Re synaptic: I've no experience with that. What command did you run? Maybe it can't do anything because googler is from a standalone deb, not a repository?

jarun · 2016-04-29T16:47:28Z

Re synaptic: I've no experience with that. What command did you run?

As we are not going follow Debian changelog format, don't worry about it.

I'm checking what this package 'min' is :).

zmwangx · 2016-04-29T16:51:46Z

As we are not going follow Debian changelog format, don't worry about it.

Actually I am following the Debian changelog format. Just not incremental.

jarun · 2016-04-29T16:53:23Z

min is https://palmeral.github.io/min/.
I was trying out this browser sometime back. Just didn't install any other package after that. We are good.

zmwangx · 2016-04-29T16:55:14Z

👍

professorjamesmoriarty · 2016-04-29T17:06:41Z

The changelog is correct format, however it's never going to work unless its added to the "offical" repos. Synaptic just provides a broken button on packages anyway. The feature always been a bit bleh. This goes for the screenshot bit as well.

The deb packages generated are to spec (within reason) and I completely love this method.

zmwangx · 2016-04-29T17:08:13Z

The changelog is correct format, however it's never going to work unless its added to the "offical" repos.

Make sense.

professorjamesmoriarty · 2016-04-29T17:11:30Z

And for the path for aur package, Archlinux's policy is to use /usr/, altho you'll find some people just don't care and do anything they want. Debian (unless it has changed) had a policy of using /usr/ for any packages such as this one. However /usr/local/ was reserved for the "sys admin" or local user. It all sounds very silly I'm sure. But I think both are fine choices, If everyone prefers /usr/local/ then it's ok.

I can't foresee any huge issues with this, personally I'd go with /usr/ but I'm crazy. There are some cases with SOME distros that /usr/local/ isnt even in a users $PATH.

zmwangx · 2016-04-29T17:16:43Z

It all sounds very silly I'm sure.

Why? /usr is managed by the package manager and system updates can do whatever; /usr/local is reserved for admins and system updates can't screw with it. It's figured out a long time ago and very reasonable.

As for this one, the only reasonable argument against /usr would be that this is a standalone deb, rather than from a repository. But I don't think it matters much. Moreover, if we do install shell completions (not at the moment), stuff in /usr/local/share/zsh/site-packages won't be picked up by /usr/bin/zsh by default (need to add to fpath), so that's one thing in favor of /usr.

jarun · 2016-04-29T17:32:51Z

Guys! /usr it is! Be carefree ;).

In the production environment we may only want to build deb for tags, although testing the build process doesn't hurt.

We already CI_FORCE_TEST. Add CI_SKIP_TEST for feature parity. But when would it be useful? When we're playing with other parts of the CI system and not concerned about testing.

It's impossible to check exit status of command substitutions inside heredocs, and saving command substitution results in variables before we enter heredocs is not very elegant, so we instead signal a C&C PID in the env var $NOTIFY_PID (if available) on error inside helper functions.

zmwangx · 2016-04-30T19:38:37Z

Rebased on master again.

jarun · 2016-05-01T09:59:52Z

Done! Please verify once.

zmwangx · 2016-05-01T17:40:41Z

@jarun Your credentials seem to be bad (see https://travis-ci.org/jarun/googler/jobs/127056857). Are you sure the token you used has the public_repo scope? And you encrypted the token for jarun/googler (travis encrypt -r jarun/googler)?

zmwangx · 2016-05-01T17:43:08Z

By the way, instead of pushing to master, let's work on another branch, say, travis-upload, before everything is sorted out. Sometimes you do need some grinding before CIs do the right thing for you.

zmwangx · 2016-05-01T17:44:53Z

I have temporarily set CI_SKIP_TEST (https://travis-ci.org/jarun/googler/settings) so that Google doesn't get pissed.

jarun · 2016-05-01T17:53:59Z

I'll try it again.
Yes, this token didn't have public_repo access. Using a fresh one now. If this doesn't work as expected I'll create a new branch.

jarun · 2016-05-01T18:11:40Z

Check now.

zmwangx · 2016-05-01T18:25:15Z

Got it this time!

https://travis-ci.org/jarun/googler/jobs/127063645
https://github.com/jarun/googler/releases/tag/v2.3-testdeb

We can delete the tag once we verify that the deb works.

zmwangx · 2016-05-01T18:28:29Z

Works for me on both wily64 and wily32.

jarun · 2016-05-01T18:32:00Z

Worked fine here too!

zmwangx · 2016-05-01T18:47:10Z

👍 Now we need to add instructions to README. I suggest that we also retro-publish a v2.3 deb package so that we can update README before 2.4. I built one from https://github.com/zmwangx/googler/releases/tag/v2.3deb, which is v2.3 plus cherry-picked dfe0476 (the Makefile patch to make packaging script work). The deb:

googler-2.3deb-1-all.deb
https://transfer.sh/5kJcv/googler-2.3deb-1-all.deb
SHA-256: 4f4d1f1c1e624d456a1d3898a5b228b6e287bde133ccf5ced351e49b44f856b4

jarun · 2016-05-01T18:52:54Z

Please push directly. I can't keep my eyes open anymore. Have a great day! 👍

zmwangx · 2016-05-01T18:53:43Z

Good night.

jarun · 2016-05-01T19:02:34Z

I added your info in man and program help. Hope that's fine with you.

zmwangx · 2016-05-01T19:03:47Z

No problem, but now it looks like you didn't do anything in 2016... Which is not true.

jarun · 2016-05-01T19:06:00Z

Hehehehe! I'll add some hyphens then :).

zmwangx · 2016-05-01T19:07:45Z

Also, we (or rather, you) have rewritten most of the code, so I think "Modified" is unnecessary. A "Copyright" line like Henri's should do.

jarun · 2016-05-01T19:10:49Z

Let's leave it with Henri ;) for the time being. I wish he knows his tool is being maintained well. :)

zmwangx · 2016-05-01T19:12:30Z

Yeah, I'm just saying

Copyright (C) 2008 Henri Hakkinen
Copyright (C) 2015-2016 Arun Prakash Jana

is more accurate.

jarun · 2016-05-01T19:40:57Z

Done :)

jarun · 2016-05-02T14:19:03Z

Closing this as it's merged locally.

zmwangx mentioned this pull request Apr 29, 2016

Discussion continued from #64 #65

Closed

zmwangx added the do not merge label Apr 29, 2016

zmwangx force-pushed the build-deb branch from 9c73329 to 157245b Compare April 29, 2016 04:53

zmwangx reviewed Apr 29, 2016
View reviewed changes

zmwangx force-pushed the build-deb branch from cacbaa3 to ca6be2a Compare April 29, 2016 07:03

zmwangx added 4 commits April 30, 2016 12:37

tools/makedeb: Rework arg parsing and introduce --tag-only

608669a

In the production environment we may only want to build deb for tags, although testing the build process doesn't hurt.

tests/ci-test-wrapper: Add CI_SKIP_TEST env var

0f5aec7

We already CI_FORCE_TEST. Add CI_SKIP_TEST for feature parity. But when would it be useful? When we're playing with other parts of the CI system and not concerned about testing.

.travis.yml: Build deb package and upload to releases

95a23c7

zmwangx force-pushed the build-deb branch from 949d235 to 95a23c7 Compare April 30, 2016 19:38

zmwangx mentioned this pull request May 1, 2016

README.md: Add deb installation instructions #71

Merged

jarun closed this May 2, 2016

zmwangx deleted the build-deb branch May 18, 2016 06:04

lock bot locked and limited conversation to collaborators Nov 15, 2019

Build deb package #67

Build deb package #67

Conversation

zmwangx commented Apr 29, 2016 • edited Loading

zmwangx commented Apr 29, 2016 • edited Loading

jarun commented Apr 29, 2016 • edited Loading

jarun commented Apr 29, 2016

zmwangx commented Apr 29, 2016 • edited Loading

zmwangx Apr 29, 2016

Choose a reason for hiding this comment

jarun commented Apr 29, 2016 • edited Loading

zmwangx commented Apr 29, 2016

jarun commented Apr 29, 2016

jarun commented Apr 29, 2016

zmwangx commented Apr 29, 2016

jarun commented Apr 29, 2016

jarun commented Apr 29, 2016

jarun commented Apr 29, 2016

zmwangx commented Apr 29, 2016 • edited Loading

zmwangx commented Apr 29, 2016 • edited Loading

zmwangx commented Apr 29, 2016

zmwangx commented Apr 29, 2016

jarun commented Apr 29, 2016

zmwangx commented Apr 29, 2016

jarun commented Apr 29, 2016

zmwangx commented Apr 29, 2016

professorjamesmoriarty commented Apr 29, 2016

zmwangx commented Apr 29, 2016

professorjamesmoriarty commented Apr 29, 2016

zmwangx commented Apr 29, 2016 • edited Loading

jarun commented Apr 29, 2016

zmwangx commented Apr 30, 2016

jarun commented May 1, 2016

zmwangx commented May 1, 2016

zmwangx commented May 1, 2016

zmwangx commented May 1, 2016

jarun commented May 1, 2016 • edited Loading

jarun commented May 1, 2016

zmwangx commented May 1, 2016

zmwangx commented May 1, 2016

jarun commented May 1, 2016

zmwangx commented May 1, 2016

jarun commented May 1, 2016

zmwangx commented May 1, 2016

jarun commented May 1, 2016

zmwangx commented May 1, 2016

jarun commented May 1, 2016

zmwangx commented May 1, 2016 • edited Loading

jarun commented May 1, 2016

zmwangx commented May 1, 2016

jarun commented May 1, 2016

jarun commented May 2, 2016

zmwangx commented Apr 29, 2016 •

edited

Loading

zmwangx commented Apr 29, 2016 •

edited

Loading

jarun commented Apr 29, 2016 •

edited

Loading

zmwangx commented Apr 29, 2016 •

edited

Loading

jarun commented Apr 29, 2016 •

edited

Loading

zmwangx commented Apr 29, 2016 •

edited

Loading

zmwangx commented Apr 29, 2016 •

edited

Loading

zmwangx commented Apr 29, 2016 •

edited

Loading

jarun commented May 1, 2016 •

edited

Loading

zmwangx commented May 1, 2016 •

edited

Loading