-
Notifications
You must be signed in to change notification settings - Fork 2.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
bug: Yarn uses old cached local file version of a package #2649
Comments
There should be an open issue for that and I think a PR is in the works |
Alright, thanks! I just ran into it so I just wanted to make sure it was being taken care of. 😄 Thanks for all your hard work! |
I believe that I've come across the bug described in this ticket. I was going to submit a bug report and PR, but since this issue is open already will just add a comment (couldn't find the open issue that @bestander was referring to two comments up). I found that changing this code in EDIT: had originally pasted in code from if (yield _this2.config.isValidModuleDest(dest)) {
return _this2.fetchCache(dest, fetcher);
} ...to: if (yield _this2.config.isValidModuleDest(dest)) {
const metadata = yield _this2.config.readPackageMetadata(dest);
if (metadata.remote.resolved === ref.remote.resolved) {
return _this2.fetchCache(dest, fetcher);
}
} ...fixed the problem. Note that I did this the EDIT: This ticket is closed and the bug I was going to report no longer occurs in current if (await this.config.isValidModuleDest(dest)) {
const metadata = await this.config.readPackageMetadata(dest);
if (metadata.remote.resolved === ref.remote.resolved) {
return this.fetchCache(dest, fetcher);
}
} Just thought I'd post these findings here in case it's of any help to the efforts already underway. Steps for bug reproduction: $ ls
package.json.09fd3b55a61cb4288a479a1c956de4975ce1e92d package.json.b2ff01deeaef7bde62cee7539aad143a8df2a2be
$ cat package.json.09fd3b55a61cb4288a479a1c956de4975ce1e92d
{
"name": "bug",
"version": "0.0.1",
"description": "bug",
"dependencies": {
"urijs": "medialize/URI.js#09fd3b55a61cb4288a479a1c956de4975ce1e92d"
}
}
$ cat package.json.b2ff01deeaef7bde62cee7539aad143a8df2a2be
{
"name": "bug",
"version": "0.0.1",
"description": "bug",
"dependencies": {
"urijs": "medialize/URI.js#b2ff01deeaef7bde62cee7539aad143a8df2a2be"
}
}
$ ln -s package.json.b2ff01deeaef7bde62cee7539aad143a8df2a2be package.json
$ yarn cache clean
yarn cache v0.18.1
success Cleared cache.
✨ Done in 0.09s.
$ yarn_cache_dir=$( yarn cache dir | head -2 | tail -1 )
$ yarn
yarn install v0.18.1
info No lockfile found.
warning bug@0.0.1: No license field
[1/4] □□ Resolving packages...
[2/4] □□ Fetching packages...
[3/4] □□ Linking dependencies...
[4/4] □□ Building fresh packages...
success Saved lockfile.
✨ Done in 0.77s.
$ cat yarn.lock
# THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY.
# yarn lockfile v1
urijs@medialize/URI.js#b2ff01deeaef7bde62cee7539aad143a8df2a2be:
version "1.18.4"
resolved "https://codeload.github.com/medialize/URI.js/tar.gz/b2ff01deeaef7bde62cee7539aad143a8df2a2be"
$ yarn cache ls
yarn cache v0.18.1
Name Version Registry Resolved
urijs 1.18.4 npm https://codeload.github.com/medialize/URI.js/tar.gz/b2ff01deeaef7bde62cee7539aad143a8df2a2be
✨ Done in 0.06s.
$ cp -pr $yarn_cache_dir/npm-urijs-1.18.4/ ./urijs-b2ff01deeaef7bde62cee7539aad143a8df2a2be
$ rm -fr node_modules/
$ mv yarn.lock yarn.lock.b2ff01deeaef7bde62cee7539aad143a8df2a2be
$ yarn cache clean
yarn cache v0.18.1
success Cleared cache.
✨ Done in 0.07s.
$ rm package.json
$ ln -s package.json.09fd3b55a61cb4288a479a1c956de4975ce1e92d package.json
$ yarn
yarn install v0.18.1
info No lockfile found.
warning bug@0.0.1: No license field
[1/4] □□ Resolving packages...
[2/4] □□ Fetching packages...
[3/4] □□ Linking dependencies...
[4/4] □□ Building fresh packages...
success Saved lockfile.
✨ Done in 0.73s.
$ cat yarn.lock
# THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY.
# yarn lockfile v1
urijs@medialize/URI.js#09fd3b55a61cb4288a479a1c956de4975ce1e92d:
version "1.18.4"
resolved "https://codeload.github.com/medialize/URI.js/tar.gz/09fd3b55a61cb4288a479a1c956de4975ce1e92d"
$ yarn cache ls
yarn cache v0.18.1
Name Version Registry Resolved
urijs 1.18.4 npm https://codeload.github.com/medialize/URI.js/tar.gz/09fd3b55a61cb4288a479a1c956de4975ce1e92d
✨ Done in 0.06s.
$ cp -pr $yarn_cache_dir/npm-urijs-1.18.4/ ./urijs-09fd3b55a61cb4288a479a1c956de4975ce1e92d
$ rm yarn.lock
$ rm -fr node_modules/
$ rm package.json
$ ln -s package.json.b2ff01deeaef7bde62cee7539aad143a8df2a2be package.json
$ cp -p yarn.lock.b2ff01deeaef7bde62cee7539aad143a8df2a2be yarn.lock
$ yarn
yarn install v0.18.1
warning bug@0.0.1: No license field
[1/4] □□ Resolving packages...
[2/4] □□ Fetching packages...
[3/4] □□ Linking dependencies...
[4/4] □□ Building fresh packages...
✨ Done in 0.41s.
$ cat yarn.lock
# THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY.
# yarn lockfile v1
urijs@medialize/URI.js#b2ff01deeaef7bde62cee7539aad143a8df2a2be:
version "1.18.4"
resolved "https://codeload.github.com/medialize/URI.js/tar.gz/b2ff01deeaef7bde62cee7539aad143a8df2a2be"
$ diff -r node_modules/urijs/ urijs-b2ff01deeaef7bde62cee7539aad143a8df2a2be/
Only in urijs-b2ff01deeaef7bde62cee7539aad143a8df2a2be/: .yarn-metadata.json
Only in urijs-b2ff01deeaef7bde62cee7539aad143a8df2a2be/: .yarn-tarball.tgz
diff -r node_modules/urijs/CHANGELOG.md urijs-b2ff01deeaef7bde62cee7539aad143a8df2a2be/CHANGELOG.md
5,8d4
< ### master
<
< * prevent `new URI(null)` from blowing up - [PR #321](https://github.com/medialize/URI.js/issues/321)
<
diff -r node_modules/urijs/src/URI.js urijs-b2ff01deeaef7bde62cee7539aad143a8df2a2be/src/URI.js
64,69d63
< if (url === null) {
< if (_urlSupplied) {
< throw new TypeError('null is not a valid argument for URI');
< }
< }
<
diff -r node_modules/urijs/test/test.js urijs-b2ff01deeaef7bde62cee7539aad143a8df2a2be/test/test.js
22,26d21
< test('URI(null)', function() {
< raises(function() {
< URI(null);
< }, TypeError, 'Failing undefined input');
< });
$ diff -r node_modules/urijs/ urijs-09fd3b55a61cb4288a479a1c956de4975ce1e92d/
Only in urijs-09fd3b55a61cb4288a479a1c956de4975ce1e92d/: .yarn-metadata.json
Only in urijs-09fd3b55a61cb4288a479a1c956de4975ce1e92d/: .yarn-tarball.tgz |
@da70 I think the issue @bestander is referring to is #2165. |
Thanks, @akloeber. |
@akloeber Thanks. I didn't read the ticket closely enough...it appears to apply only to local dependencies. The bug I ran into is different. I'll open a new ticket if I don't find an existing one addressing it. |
Can't reproduce the bug in version 0.21.0-0 (current |
In case anyone using an older version of yarn stumbles across this ticket while looking for answers about caching irregularities, the bug I was trying to describe above was resolved by Add hash to cache path for non-NPM packages (#2074). |
@da70 Are you sure this fix is already contained in v0.20.3? In my cache folder the hashes are still missing for local file dependencies ( |
I checked the sources of the yarn version that is installed on my machine and the changes from #2074 were present there. I have no idea why the hashing does still not occur. This can be reproduced with:
This produces
Subsequent changes to in
|
@akloeber The reason you are seeing this behavior is because in the v0.20.3 code, the directory name only includes a hash if the package's let name = pkg.name;
let uid = pkg.uid;
if (pkg.registry) {
name = `${pkg.registry}-${name}`;
uid = pkg.version || uid;
}
const hash = pkg.remote.hash;
if (hash) {
uid += `-${hash}`;
} For Github dependencies, src/resolvers/exotics/file-resolver.js:resolve(): manifest._remote = {
type: 'copy',
registry,
hash: null,
reference: loc,
}; I don't think the |
@da70 Thanks for the clarification, that makes sense and solves this issue for deployment. For development I'm using linklocal anyway. Only the prefix |
@akloeber Yes from the point of view of someone looking at the cache it would probably be easier to have a greater variety of prefixes. I think The fetch request determines the registry for the dependency here in cli/commands/install.js by matching against registry filename, which in the use case you describe is "package.json", which matches the Maybe in the future there will be a clearer differentiation between package registry and package origin/location. |
11 months later, for me, using yarn 1.3.2 the problem is still there. Defining a dependency in
Note that there is no versioning! Running |
@JedI-O, this is by design. I recommend using npm for fast cacheable updates or use version tags in git if you want to force update. |
Well, I don't want yarn to clone the repo each time I run |
It would require to make network requests on every install, that would be
slow.
Also afaik Yarn stores the git hash in the lockfile to avoid accidental
updates, so yarn actually does an extra effort to prevent your scenario.
I suppose ‘update’ command could/should do what you want.
…On Wed, Jan 10, 2018 at 1:40 AM JedI-O ***@***.***> wrote:
Well, I don't want yarn to clone the repo each time I run yarn installeither.
But I want yarn to clone the repo each time it has changed. I had assumed
that yarn would (or should) compare the latest commits and check whether
the latest commit on the remote repo is newer than the latest commit
locally installed. If yes: clone again, if not: do nothing. In my opinion,
that should at least be an optional behavior.
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
<#2649 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/ACBdWNEbijkISieej5i7jrt7y1sSEFBmks5tJIWpgaJpZM4L4yjF>
.
|
This is still an issue with yarn workspaces. If multiple caret-references are made for the same dependency or indirect dependency, yarn picks the oldest. Then, yarn holds on to those old code-rotting dependencies and indirect dependencies. You're on Mars what works is to reference a strict range in package.json like:
and in the individual workspace package do: yarn 1.19.1 macOS bug Node.js v12.11.1 yarn workspaces |
ubuntu 16_04 with yarn 1.19.1
and I delete the copy in node_modules yarn should clone a new working copy. Instead I get the old copy back. Is this the same bug? |
Having the same issue @rimmartin had |
Do you want to request a feature or report a bug?
bug
What is the current behavior?
Yarn uses old cached version of a file package instead of newer one.
If the current behavior is a bug, please provide the steps to reproduce.
Install a package on filesystem, update contents (without changing version), reinstall and see old version.
What is the expected behavior?
To correctly detected the cached version is invalid (hash) and use the new one.
package.json
yarn.lock
Expected:
Actual (installed by yarn):
This works as expected with
npm
.yarn cache clean
fixes this (once) but it is still a huge annoyance to clean my yarn cache every time I want to test a local package.The text was updated successfully, but these errors were encountered: