Error: unable to publish: unable to process packages: link ... : no such file or directory

[xuanngo2001]

Ran

aptly publish repo -architectures='amd64,i386' -passphrase='secret.' master_google-main master_google-main

It generates the following error:

Loading packages...
Generating metadata files and linking package files...
ERROR: unable to publish: unable to process packages: link /root/.aptly/pool/12/9a/google-chrome-stable_60.0.3112.78-1_amd64.deb /root/.aptly/public/master_google-main/pool/main/g/google-chrome-stable/google-chrome-stable_60.0.3112.78-1_amd64.deb: no such file or directory

Yes, the file google-chrome-stable_60.0.3112.78-1_amd64.deb is indeed not found.
but when I ran

>find . | grep google-chrome-stable_60.0.3112.78-1_amd64
./data/.aptly/pool/12/9a/c54d87724b58ea5cff53b05a4858_google-chrome-stable_60.0.3112.78-1_amd64.deb
./data/.aptly/pool/9a/64/c54d87724b58ea5cff53b05a4858_google-chrome-stable_60.0.3112.78-1_amd64.deb
./data/.aptly/public/master_google-main/pool/main/g/google-chrome-stable/c54d87724b58ea5cff53b05a4858_google-chrome-stable_60.0.3112.78-1_amd64.deb

Everything is handled by aptly, there is no manual intervention.
Why there is a prefix hash in front of google-chrome-stable_60.0.3112.78-1_amd64.deb?
If I delete everything and start over, the same issue happens but might be other package from other mirror.

Context

1. I mirror http://dl.google.com/linux/chrome/deb/ stable main: aptly mirror ....
2. Copy mirror to local repository: aptly repo import -architectures=amd64,i386 -with-deps=true google-main master_google-main Name
3. Publish local repository: aptly publish repo -architectures='amd64,i386' -passphrase='secret.' master_google-main master_google-main

Your Environment

aptly version: 1.0.1 or nightly build has the same problem.
Debian GNU/Linux 8 (jessie): 4.9.0-0.bpo.3-amd64

[smira]

@limelime this happens if you switch between nightly (future 1.1.0) and 1.0.1. Nightly stores packages in a bit different way, so 1.0.1 can't handle it. If you stick with one version and import/publish on the same version it should be fine.

Upgrade from 1.0.1 to nightly is supported and doesn't require any action, but going back is not supported.

[colincoghill]

I'm seeing this error from a fresh nightly build where I erased the repo and rebuilt it.

I'm not all that familiar with the source, but I'm wondering if

https://github.com/smira/aptly/blob/master/deb/publish.go#L591

isn't using the new filename format (with the checksum)

[smira]

@colincoghill what do you mean by erased the repo? I would like to see if you're hitting any bug vs. known issue that going backwards is not supported.

I'm not all that familiar with the source, but I'm wondering if
https://github.com/smira/aptly/blob/master/deb/publish.go#L591
isn't using the new filename format (with the checksum)

It does use new format, as now package files track their location. And nightly should never have paths which look like pool/12/9a/google-chrome-stable_60.0.3112.78-1_amd64.deb unless it was upgraded from <= 1.0.x

[colincoghill]

Got a much better test case, for me, this dies with:

ERROR: unable to publish: unable to process packages: stat /root/.aptly/pool/ed/18/libestr0_0.1.10-0adiscon1trusty1_amd64.deb: no such file or directory root@aptly:~/aptly# cat broken.sh

It dies the same way against many repos, I don't think it's anything to do with upstream.

# Create a blank Ubuntu Trusty VM
# install aptly nightly from github.

# Start with no aptly repo/db
# rm -rf /root/.aptly


# initial setup
gpg --no-default-keyring \
    --keyring trustedkeys.gpg \
    --keyserver keys.gnupg.net \
    --recv-keys 0F6DD8135234BF2B  

aptly mirror create \
    -architectures=amd64 \
    -filter="rsyslog" \
    -filter-with-deps \
    rsyslog-incoming \
    http://ppa.launchpad.net/adiscon/v8-stable/ubuntu trusty main

aptly mirror update rsyslog-incoming
aptly snapshot create initial-rsyslog from mirror rsyslog-incoming
aptly publish snapshot -architectures=amd64 -distribution="trusty" initial-rsyslog proposed


# refresh
aptly mirror update rsyslog-incoming
aptly snapshot create updated-rsyslog from mirror rsyslog-incoming
aptly publish switch trusty proposed updated-rsyslog

[smira]

@colincoghill error from above looks like you switched from nightly back to 1.0.1 version (guessing here, but most likely). Going back (nightly to pre-1.1.0) is not supported, while going forward is fine. You need to go back to nightly build (or 1.1.0 when it gets released)

[colincoghill]

I was careful to create a fresh VM that had never had an older version of aptly and only used the nightly build.

I've just repeated the process, but on MacOSX, on a machine that has never had aptly installed before.

Colins-MacBook-Pro:aptly colin$ aptly publish switch trusty proposed updated-rsyslog
Loading packages...
Generating metadata files and linking package files...
ERROR: unable to publish: unable to process packages: stat /Users/colin/.aptly/pool/ed/18/libestr0_0.1.10-0adiscon1trusty1_amd64.deb: no such file or directory
Colins-MacBook-Pro:aptly colin$ aptly version aptly version: 1.1.0

I'm absolutely certain this is not a case of mixing aptly versions.

[smira]

@colincoghill thanks, I can reproduce that... I'll take a look to see what's up there

[smira]

@colincoghill thanks for reporting it! 1.1.1 (just released) and latest nightly contain the fix for this problem