Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache keys don't match for builds with a cached copy command #899

Closed
cvgw opened this issue Dec 2, 2019 · 0 comments · Fixed by #914
Closed

Cache keys don't match for builds with a cached copy command #899

cvgw opened this issue Dec 2, 2019 · 0 comments · Fixed by #914
Assignees
Labels
area/caching For all bugs related to cache issues kind/bug Something isn't working priority/p1 Basic need feature compatibility with docker build. we should be working on this next.

Comments

@cvgw
Copy link
Contributor

cvgw commented Dec 2, 2019

Actual behavior
When a build contains a cached layer using the COPY command all following layers in the build will produce incorrect cache keys. Specifically, the cache key used to read from the cache and the cache key used to write to the cache will be different for each layer following the copy command.

Layers affected by this will never be able to read from the cache because the key used to write the layer to the cache is different than the key used to read from the cache.

The cause of this bug is due to CachingCopyCommand and CopyCommand returning different values for FilesUsedFromContext. The result of FilesUsedFromContext is added to the cache key.

Expected behavior
All commands always produce the same cache key during read and during write given the same input.

To Reproduce
Steps to reproduce the behavior:

  1. Any build with a cached copy command followed by any other caching command
  2. Observe that commands following the cached copy command are never cached

Additional Information

  • Dockerfile
FROM ubuntu:16.04
COPY foo.txt bar.txt
RUN apt-get update -y
  • Build Context
    echo 'meow' > foo.txt

Triage Notes for the Maintainers

Description Yes/No
Please check if this a new feature you are proposing
Please check if the build works in docker but not in kaniko
Please check if this error is seen when you use --cache flag
Please check if your dockerfile is a multistage dockerfile
@cvgw cvgw added kind/bug Something isn't working area/caching For all bugs related to cache issues priority/p1 Basic need feature compatibility with docker build. we should be working on this next. labels Dec 2, 2019
drappier-charles pushed a commit to drappier-charles/kaniko that referenced this issue Dec 11, 2019
Relates to GoogleContainerTools#899
When Kaniko is pushing a layer at the end of a Dockerfile, if previous line have COPY inside, the computeKey is not compute well.
It happen cause the cachingcopycommand (use for cached layer) doesn't support the FilesUsedFromContext method, it was using the base_command one.
I've fixed it by doing a copy paste of the copycommand, and by adding the buildcontext parameter.
I think It will need a rework to refactor it, but I don't know how to use CacheCopy instead of BaseCommand on line 169 (it crash at runtime)
@cvgw cvgw self-assigned this Dec 15, 2019
cvgw added a commit to cvgw/kaniko that referenced this issue Dec 15, 2019
* Update cached copy command to return the same result for
files used from context so that cached and uncached copy
commands produce the same cache key
* Update tests for fix
* Add test for cached run command key consistency
tejal29 added a commit that referenced this issue Dec 16, 2019
Fix #899 cached copy results in inconsistent key
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/caching For all bugs related to cache issues kind/bug Something isn't working priority/p1 Basic need feature compatibility with docker build. we should be working on this next.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant