Add layer caching to kaniko #353
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
To add layer caching to kaniko, I added two flags: --cache and --use-cache. If --use-cache is set, then the cache will be used, and if --cache is specified then that repo will be used to store cached layers. If --cache isn't set, a cache will be inferred from the destination provided. Currently, caching only works for RUN commands. Before executing the command, kaniko checks if the cached layer exists. If it does, it pulls it and extracts it. It then adds those files to the snapshotter and append a layer to the config history. If the cached layer does not exist, kaniko executes the command and pushes the newly created layer to the cache. All cached layers are tagged with a stable key, which is built based off of: 1. The base image digest 2. The current state of the filesystem 3. The current command being run 4. The current config file (to account for metadata changes) I also added two integration tests to make sure caching works 1. Dockerfile_test_cache runs 'date', which should be exactly the same the second time the image is built 2. Dockerfile_test_cache_install makes sure apt-get install can be reproduced
cmd/executor/cmd/root.go
Outdated
| @@ -92,6 +92,8 @@ func addKanikoOptionsFlags(cmd *cobra.Command) { | |||
| RootCmd.PersistentFlags().BoolVarP(&opts.Reproducible, "reproducible", "", false, "Strip timestamps out of the image to make it reproducible") | |||
| RootCmd.PersistentFlags().StringVarP(&opts.Target, "target", "", "", "Set the target build stage to build") | |||
| RootCmd.PersistentFlags().BoolVarP(&opts.NoPush, "no-push", "", false, "Do not push the image to the registry") | |||
| RootCmd.PersistentFlags().StringVarP(&opts.Cache, "cache", "", "", "Specify a registry to use as a chace, otherwise one will be inferred from the destination provided") | |||
There was a problem hiding this comment.
fixed, thanks!
sharifelgamal
approved these changes
Sep 17, 2018
dlorenc
reviewed
Sep 24, 2018
cmd/executor/cmd/root.go
Outdated
| @@ -92,6 +92,8 @@ func addKanikoOptionsFlags(cmd *cobra.Command) { | |||
| RootCmd.PersistentFlags().BoolVarP(&opts.Reproducible, "reproducible", "", false, "Strip timestamps out of the image to make it reproducible") | |||
| RootCmd.PersistentFlags().StringVarP(&opts.Target, "target", "", "", "Set the target build stage to build") | |||
| RootCmd.PersistentFlags().BoolVarP(&opts.NoPush, "no-push", "", false, "Do not push the image to the registry") | |||
| RootCmd.PersistentFlags().StringVarP(&opts.Cache, "cache", "", "", "Specify a registry to use as a cache, otherwise one will be inferred from the destination provided") | |||
| RootCmd.PersistentFlags().BoolVarP(&opts.UseCache, "use-cache", "", true, "Use cache when building image") | |||
There was a problem hiding this comment.
Can we default this to false for now?
There was a problem hiding this comment.
for sure, done
Rename --use-cache to --cache, and --cache to --cache-repo to clarify what the flags are used for. Default caching to false.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
To add layer caching to kaniko, I added two flags: --cache and
--use-cache.
If --use-cache is set, then the cache will be used, and if --cache is
specified then that repo will be used to store cached layers. If --cache
isn't set, a cache will be inferred from the destination provided.
Currently, caching only works for RUN commands. Before executing the
command, kaniko checks if the cached layer exists. If it does, it pulls
it and extracts it. It then adds those files to the snapshotter and
append a layer to the config history. If the cached layer does not exist, kaniko executes the command and
pushes the newly created layer to the cache.
All cached layers are tagged with a stable key, which is built based off
of:
I also added two integration tests to make sure caching works
the second time the image is built
reproduced
Should fix #300