Add option -skip-existing-packages to mirror update to speed up download queue setup #520
Conversation
There was a problem hiding this comment.
👍 awesome work, thanks!
just one minor comment on ronn and man page
also it would be awesome to add new option to the web docs: https://github.com/aptly-dev/aptly-dev.github.io
man/aptly.1
Outdated
| @@ -1,7 +1,7 @@ | |||
| .\" generated with Ronn/v0.7.3 | |||
| .\" http://github.com/rtomayko/ronn/tree/0.7.3 | |||
| . | |||
| .TH "APTLY" "1" "March 2017" "" "" | |||
| .TH "APTLY" "1" "2017-03-23" "" "" | |||
There was a problem hiding this comment.
this seems to be generated with broken ronn, make man should install correct ronn fork for you and run man page generation
There was a problem hiding this comment.
Took a while to get ronn up and running. Apparently using mustache in version 1.0.4 caused errors, so I had to revert that to 1.0.3. Nevertheless, it now seems to work and I've fixed the man page in 4ddf85b.
|
I've opened the pull request aptly-dev/www.aptly.info#38 for the web docs. |
|
|
When updating a mirror, all package files are downloaded and a list of downloads is generated by checking which files are already on disc and have the right size. This can take some time, especially on storage systems like NAS or external hard drives when mirroring large repositories.
Requirements
All new code should be covered with tests, documentation should be updated. CI should pass.
Description of the Change
To speed up the download queue setup, the internal database for the mirror can be used. It is only updated after a successful mirror update and contains the list of packages of the mirror. Assuming nobody interfered and deleted files manually, we can use this database to check what packages need to be downloaded. This new functionality is enabled only when using the command line option -skip-existing-packages, otherwise the behavior is unchanged.
Checklist
AUTHORS