yarn install's dedupe doesn't work like npm's dedupe

[wyuenho]

Do you want to request a feature or report a bug?
Bug

What is the current behavior?
yarn dedupe claims yarn install will dedupe, but it doesn't seem to produce the correct dependency tree.

If the current behavior is a bug, please provide the steps to reproduce.

yarn add backbone@^1.1.0 && yarn add "backbone.marionette@<3" && npm ls->

├── backbone@1.3.3
├─┬ backbone.babysitter@0.1.0
│ ├─┬ backbone@1.1.2
│ │ └── underscore@1.8.3
│ └── underscore@1.6.0
├── backbone.marionette@2.4.7
├─┬ backbone.wreqr@1.0.0
│ ├─┬ backbone@1.1.2
│ │ └── underscore@1.8.3
│ └── underscore@1.6.0
└── underscore@1.8.3

npm install "backbone@^1.1.0" "backbone.marionette@<3" && npm dedupe && npm ls ->

├── backbone@1.1.2
├─┬ backbone.marionette@2.4.7
│ ├── backbone.babysitter@0.1.0
│ └── backbone.wreqr@1.0.0
└── underscore@1.6.0

What is the expected behavior?
Judging from the output of yarn ls

The indirect dependencies of yarn should resolve to 1.1.2, and since the top level dependency spec requires backbone@^1.1.0, the dedupe'd result should be only 1 copy of backbone on the top-level having the version 1.1.0.

Please mention your node.js, yarn and operating system version.
node.js: 4.6.0
npm: 2.15.9
yarn: 0.15.1
OS: Mac OS X 10.11.6

[simonua]

Not saying you don't have a point here, but that's a relatively old version of npm. Have you compared this to a more recent 3.x version?

https://github.com/npm/npm/blob/master/CHANGELOG.md

[wyuenho]

This is the result from npm 3

├─┬ backbone@1.3.3
│ └── underscore@1.8.3
└─┬ backbone.marionette@2.4.7
  ├─┬ backbone.babysitter@0.1.0
  │ ├── backbone@1.1.2
  │ └── underscore@1.6.0
  └─┬ backbone.wreqr@1.0.0
    ├── backbone@1.1.2
    └── underscore@1.6.0

They are all different for some reason I don't understand.

[ljharb]

Note that in npm 3, npm ls iirc shows the dep graph, not the files on disk - iirc.

[wyuenho]

Right, the packages are flat on disk, but backbone is at 1.3.3 and underscore at 1.8.3 for npm 3, so it's probably incorrect for npm 3 anyway

[a-r-d]

I am seeing this also on a react project - with npm my bundle size output from webpack was 600k and when I tried to use yarn my bundle size jumped to 1.1 mb. When I looked at the dependencies using a webpack stats tool, I found six different versions of react had been packaged.

[wycats]

In my opinion this is another consequence of #579. Current status: a first pass at the algorithm I described in that issue 😄

I think this file-size regression is probably a good test for my first-pass algorithm, which might still miss global optimizations (packages on the registry that could unify deps but which weren't chosen for any dependency), but should do pretty well when local deduping possible.

[busches]

Can you reproduce this with yarn@latest? I'm not able to recreate the issue using the latest Yarn v0.17.10. Yarn correctly uses version 1.1.0 of backbone for all the resolutions:

"backbone@1.0.0 - 1.3.x", backbone@1.1.0, "backbone@>=0.9.9 <=1.3.x":
  version "1.1.0"
  resolved "http://repo.nwie.net:8080/nexus/content/groups/npm/backbone/-/backbone-1.1.0.tgz#a3c845ea707dc210aa12b0dc16fceca4bbc18a3e"
  dependencies:
    underscore ">=1.4.3"

[wyuenho]

Same problem on npm 2.

I don't know how you produced the above output, but I suspect that's because the latest backbone version is on 1.1.0 on your npm registry. Normally it should be 1.1.2.

[wyuenho]

Ahh sorry guys, I just found out the reason for that weird dep tree is because our internal npm server has pinned to an old version of backbone.babysitter and backbone.wreqr that have an upper bound of backbone<=1.1.2. Sorry for the confusion.