aptly repo include reports an error when used for a backports distribution #278

npf · 2015-08-04T20:44:12Z

The changes file for a backported Debian packages (e.g. suffixed with ~bpo8+1) does not include a references for the orig.tar.gz tarball, nor is the file pushed by dput.

However, aptly tries to import the file.

Typical error output:

[!] Unable to import file /tmp/aptly259445817/oar_2.5.5~rc1.orig.tar.gz into pool: open /tmp/aptly259445817/oar_2.5.5~rc1.orig.tar.gz: no such file or directory
[!] Some files were skipped due to errors:
  oar_2.5.5~rc1-1~bpo8+1.dsc
ERROR: some files failed to be added

See the related changes files here: https://gist.github.com/npf/64a7432e6ae04e9fa0c6

The text was updated successfully, but these errors were encountered:

smira · 2015-08-04T20:58:34Z

.changes file references orig.tar.gz:

 f59d49bd7a25aa5647b47a7fbbfd29d7275a7f54 4803977 oar_2.5.5~rc1.orig.tar.gz

npf · 2015-08-04T21:06:04Z

Hello,
not the changes file for the bpo (there are 2 files in my gist)

npf · 2015-08-25T22:24:32Z

Hello,

The same issue occurs with a new package release of a same upstream version: by default, dpkg buildpackage does not ship the orig.tar.gz file again.

For instance, uploading oar_2.5.5~~rc1-3 (3rd package release for upstream version 2.5.5~~rc1) , I get the error:

[!] Unable to import file /tmp/aptly971643183/oar_2.5.5~rc1.orig.tar.gz into pool: open /tmp/aptly971643183/oar_2.5.5~rc1.orig.tar.gz: no such file or directory
[!] Some files were skipped due to errors:
  oar_2.5.5~rc1-3.dsc
ERROR: some files failed to be added

See the second and third files of https://gist.github.com/npf/64a7432e6ae04e9fa0c6

smira · 2015-08-26T20:05:50Z

This sounds like a bug in aptly, but don't have time right now to investigate

npf · 2015-09-24T10:41:48Z

Section 5.6.21 of https://www.debian.org/doc/debian-policy/ch-controlfields.html, gives some specifications. Hope this could help.
Thanks for your reply.

npf · 2017-07-04T15:32:02Z

Hello, I see that issue #370 is closed, however, for the case I reported here, I'd like to mention that I still see the problem with aptly 1.0.1:

$ aptly version
aptly version: 1.0.1

First I include packages for sid (no error):

aptly repo -repo=sid_beta -no-remove-files=true -force-replace=true include incoming/oar_2.5.8~rc4-1_amd64.changes
gpgv: Signature made Mon 26 Jun 2017 10:55:12 PM CEST
gpgv:                using RSA key 60600425A3FE263CA4DFDD9DC9406ECEDB9F6404
gpgv: Good signature from "XXX"
Loading repository sid_beta for changes file oar_2.5.8~rc4-1_amd64.changes...
[+] liboar-perl_2.5.8~rc4-1_amd64 added
[+] oar-common-dbgsym_2.5.8~rc4-1_amd64 added
[+] oar-common_2.5.8~rc4-1_amd64 added
[+] oar-doc_2.5.8~rc4-1_all added
[+] oar-node_2.5.8~rc4-1_amd64 added
[+] oar-restful-api-dbgsym_2.5.8~rc4-1_amd64 added
[+] oar-restful-api_2.5.8~rc4-1_amd64 added
[+] oar-server-dbgsym_2.5.8~rc4-1_amd64 added
[+] oar-server-mysql_2.5.8~rc4-1_amd64 added
[+] oar-server-pgsql_2.5.8~rc4-1_amd64 added
[+] oar-server_2.5.8~rc4-1_amd64 added
[+] oar-user-dbgsym_2.5.8~rc4-1_amd64 added
[+] oar-user-mysql_2.5.8~rc4-1_amd64 added
[+] oar-user-pgsql_2.5.8~rc4-1_amd64 added
[+] oar-user_2.5.8~rc4-1_amd64 added
[+] oar-web-status_2.5.8~rc4-1_all added
[+] oar_2.5.8~rc4-1_source added

Then, including some backports, I get the error at the end:

aptly repo -repo=stretch-backports_beta -no-remove-files=true -force-replace=true include incoming/oar_2.5.8~rc4-1~bpo9+1_amd64.changes
gpgv: Signature made Tue 27 Jun 2017 10:08:23 AM CEST
gpgv:                using RSA key 60600425A3FE263CA4DFDD9DC9406ECEDB9F6404
gpgv: Good signature from "XXX"
Loading repository stretch-backports_beta for changes file oar_2.5.8~rc4-1~bpo9+1_amd64.changes...
[+] liboar-perl_2.5.8~rc4-1~bpo9+1_amd64 added
[+] oar-common-dbgsym_2.5.8~rc4-1~bpo9+1_amd64 added
[+] oar-common_2.5.8~rc4-1~bpo9+1_amd64 added
[+] oar-doc_2.5.8~rc4-1~bpo9+1_all added
[+] oar-node_2.5.8~rc4-1~bpo9+1_amd64 added
[+] oar-restful-api-dbgsym_2.5.8~rc4-1~bpo9+1_amd64 added
[+] oar-restful-api_2.5.8~rc4-1~bpo9+1_amd64 added
[+] oar-server-dbgsym_2.5.8~rc4-1~bpo9+1_amd64 added
[+] oar-server-mysql_2.5.8~rc4-1~bpo9+1_amd64 added
[+] oar-server-pgsql_2.5.8~rc4-1~bpo9+1_amd64 added
[+] oar-server_2.5.8~rc4-1~bpo9+1_amd64 added
[+] oar-user-dbgsym_2.5.8~rc4-1~bpo9+1_amd64 added
[+] oar-user-mysql_2.5.8~rc4-1~bpo9+1_amd64 added
[+] oar-user-pgsql_2.5.8~rc4-1~bpo9+1_amd64 added
[+] oar-user_2.5.8~rc4-1~bpo9+1_amd64 added
[+] oar-web-status_2.5.8~rc4-1~bpo9+1_all added
[!] Unable to import file /tmp/aptly386260638/oar_2.5.8~rc4.orig.tar.gz into pool: open /tmp/aptly386260638/oar_2.5.8~rc4.orig.tar.gz: no such file or directory
[!] Some files were skipped due to errors:
  incoming/oar_2.5.8~rc4-1~bpo9+1.dsc
ERROR: some files failed to be added

Hope this is useful.
Thanks.

smira · 2017-07-06T21:01:01Z

Thanks for bringing this up, @npf. This has never been fixed, pushing to 1.2.0

wwentland · 2017-09-07T15:10:12Z

Does it work if you force the inclusion of the original source by passing -sa to dpkg-genchanges ? That way you should have a reference to the orig.tar.gz in the .changes file.

npf · 2017-09-08T12:20:08Z

Hi @BABILEN ,
Thanks for the tip ! the ERROR vanishes indeed.
(I'm using sbuild, so I actually added the --force-orig-source flag)

However, in my opinion, this is only a workaround, right ? This does not close the issue ?

aptly should be able to handle the cases where the original source is not included, which is the default behavior for backports, but also if not an initial package release of an upstream version of a software, e.g. software-x.y-r with r > 1, from what I understand ?

wwentland · 2017-09-13T10:43:19Z

The assumption is basically that the orig source is included if you upload a new package and, conversely, that you would only skip it if you have uploaded it previously (e.g. when uploading software-x.y-r you expect the orig source to have been included in software-x.y-1).

npf · 2017-09-18T15:18:40Z

@BABILEN : Yep agreed. Unless I'm mistaken the error appears even though the orig source was indeed uploaded previously with software-x-y-1.

smira · 2017-09-25T12:42:49Z

So basically when searching for matching files, we should expand our scope to aptly's internal package pool - if matching file is there, we can assume safely we can use it. If file is missing, we should give back error (like today). Does that make sense?

wwentland · 2017-09-26T09:09:51Z

@smira: That sounds about right. You essentially want to make sure that you can find the orig source tarball somewhere in the repo, but not enforce that it has to be included in the current upload if it is present already.

Sometimes source packages reference files already present in the pool. Allow for those file to be omitted when importing packages either via `repo add` or `repo include`. If file is missing, aptly would make an attempt to look up file in the package pool (by checksum) and use it. Fixes: #278

smira · 2017-09-29T20:47:53Z

Should be fixed now, fix available in nightly build aptly_1.1.1+35+g5ef45bd

wwentland · 2017-10-01T20:37:26Z

Cheers!

npf · 2017-10-06T20:28:53Z

Thanks for fixing!

npf changed the title ~~repo include reports an error when used for backport packages~~ aptly repo include reports an error when used for a backports distribution Aug 4, 2015

npf mentioned this issue Aug 4, 2015

support of changes files #71

Closed

smira added the bug label Aug 26, 2015

smira added this to the v0.9.6 milestone Aug 26, 2015

smira modified the milestones: v0.9.6, v0.9.7 Jan 24, 2016

smira mentioned this issue Apr 7, 2016

if orig file is not in changes file but is in dsc file, aptly fails to process incoming queue #370

Closed

smira modified the milestones: v0.9.7, v0.9.8 Apr 18, 2016

smira removed this from the v0.9.8 milestone Mar 27, 2017

smira added the 1.2.0 label Jul 6, 2017

smira mentioned this issue Sep 29, 2017

Allow using files from the pool while importing source packages #650

Merged

6 tasks

smira closed this as completed in #650 Sep 29, 2017

smira mentioned this issue Dec 5, 2017

Unable to resolve orig.tar.gz already in repository for update-only uploads #407

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

aptly repo include reports an error when used for a backports distribution #278

aptly repo include reports an error when used for a backports distribution #278

npf commented Aug 4, 2015

smira commented Aug 4, 2015

npf commented Aug 4, 2015

npf commented Aug 25, 2015

smira commented Aug 26, 2015

npf commented Sep 24, 2015

npf commented Jul 4, 2017 •

edited

Loading

smira commented Jul 6, 2017

wwentland commented Sep 7, 2017 •

edited

Loading

npf commented Sep 8, 2017 •

edited

Loading

wwentland commented Sep 13, 2017

npf commented Sep 18, 2017 •

edited

Loading

smira commented Sep 25, 2017

wwentland commented Sep 26, 2017

smira commented Sep 29, 2017

wwentland commented Oct 1, 2017

npf commented Oct 6, 2017

aptly repo include reports an error when used for a backports distribution #278

aptly repo include reports an error when used for a backports distribution #278

Comments

npf commented Aug 4, 2015

smira commented Aug 4, 2015

npf commented Aug 4, 2015

npf commented Aug 25, 2015

smira commented Aug 26, 2015

npf commented Sep 24, 2015

npf commented Jul 4, 2017 • edited Loading

smira commented Jul 6, 2017

wwentland commented Sep 7, 2017 • edited Loading

npf commented Sep 8, 2017 • edited Loading

wwentland commented Sep 13, 2017

npf commented Sep 18, 2017 • edited Loading

smira commented Sep 25, 2017

wwentland commented Sep 26, 2017

smira commented Sep 29, 2017

wwentland commented Oct 1, 2017

npf commented Oct 6, 2017

npf commented Jul 4, 2017 •

edited

Loading

wwentland commented Sep 7, 2017 •

edited

Loading

npf commented Sep 8, 2017 •

edited

Loading

npf commented Sep 18, 2017 •

edited

Loading