Size of node_modules bigger with Yarn compared to NPM

[danielkcz]

Do you want to request a feature or report a bug?
More likely feature, but might be hidden bug as well

What is the current behavior?
Installing this package.json yields these results:

Yarn: 206MB, 44365 files, 5638 folders
NPM: 171MB, 30309 files, 4574 folders

If the current behavior is a bug, please provide the steps to reproduce.
Everything is in the gist. Also I have zipped both installed folders to ensure you can see what I see.

What is the expected behavior?
Difference is not that big here yet, but I had to strip that package.json of some private modules, but has certainly growing nature. In all cases everything is working as it supposed to.

Please mention your node.js, yarn and operating system version.
Node 6.9.1, Yarn 0.17.7, Windows 10 x64

[danielkcz]

Once I have created the #2090 issue regarding .npmignore I went and tried to remove any modules that are installed from repository. Apparently it's not a source of problem, here are the results:

Yarn: 165MB, 43646 files, 5452 folders
NPM: 130MB, 29493 files, 4380 folders

And here is modified package.json for a reference.

[danielkcz]

Ok, I have pinpointed single module grunt-browserify@5.0.0 where Yarn installed folder is twice in size than with NPM. Hopefully it will help to find a difference more easily than with a bigger project.

Yarn: 15.6MB, 4863 files, 1159 folders
NPM: 7.84MB, 2781 files, 678 folders

Edit: Fun fact, NPM adds a lot of information to package.json of installed packages. Yet it still wins in size big time :)

[danielkcz]

Ok I think I have found main culprit, fsevents module. It's optional dependency of the chokidar module and it shouldn't be installed at all. In case of grunt-browserify it even gets installed twice with different version which gives those extra 8MB in size.

Found related issues to this: #1285, #1328

[sebmck]

Fixed #1286 and #1328.

[arch-mage]

I got 544 MB just for babel and it's followers for this package.json. Compared to npm, which just 135M just for babel and babel-*, I think it's too big.

here's du -shc node_modules/babel-* output for both on some es2015 transforms:

npm:

....
1.1M	node_modules/babel-plugin-transform-es2015-classes
1.1M	node_modules/babel-plugin-transform-es2015-computed-properties
1.1M	node_modules/babel-plugin-transform-es2015-destructuring
1.1M	node_modules/babel-plugin-transform-es2015-duplicate-keys
1.1M	node_modules/babel-plugin-transform-es2015-for-of
1.1M	node_modules/babel-plugin-transform-es2015-function-name
...
135M	total

yarn:

...
8.0M	node_modules/babel-plugin-transform-es2015-classes
7.9M	node_modules/babel-plugin-transform-es2015-computed-properties
7.9M	node_modules/babel-plugin-transform-es2015-destructuring
7.9M	node_modules/babel-plugin-transform-es2015-duplicate-keys
7.9M	node_modules/babel-plugin-transform-es2015-for-of
7.9M	node_modules/babel-plugin-transform-es2015-function-name
...
544M	total

yarn got almost four times bigger than npm.

[neutraali]

This is (literally) a really big issue, and a the main thing keeping us from switching over to Yarn for the moment. We're trying to keep folder sizes to a minimum with various docker-integrations, so at the moment it looks like Yarn would multiply our build and test times.

[EmaGht]

this package.json
installed with NPM:

installed with Yarn:

This is already a problem, but it gets better:

NPM install bundled with this webpack

4.7MB... ok.

Yarn install bundled with same webpack:

5.7MB... excuse me? xD

also, straigthforward things like
import 'jquery-slimscroll';
works only when bundling NPM-installed packages, Yarn-installed packages somehow breaks that functionality.

Basically i can't update to NPM 5 because of the bugs it has and i can't just use Yarn because it bloats my application (??) and makes everything stop working (???)

Come on...

[karser]

I also confirm this, yarn node_modules folder is up to 1.5 times bigger than npm one:

npm
du -h angularjs-project/node_modules/ -d 0 && du -h angular4-project2/node_modules/ -d 0 && du -h angular4-project1/node_modules/ -d 0
212.4M  angularjs-project/node_modules/
287.1M  angular4-project2/node_modules/
304.2M  angular4-project1/node_modules/

yarn
du -h angularjs-project/node_modules/ -d 0 && du -h angular4-project2/node_modules/ -d 0 && du -h angular4-project1/node_modules/ -d 0
260.5M  angularjs-project/node_modules/
389.5M  angular4-project2/node_modules/
472.2M  angular4-project1/node_modules/

There is a time benchmark:

npm install test
node: v8.9.3
npm: 5.5.1
angularjs-project: added 1020 packages in 59.32s
angular4-project2: added 1184 packages in 22.051s
angular4-project1: added 1268 packages in 19.245s

Overall: 1:42s

yarn test
node: v8.9.3
yarn: 1.3.2
angularjs-project: Done in 29.52s.
angular4-project2: Done in 35.50s.
angular4-project1: Done in 24.45s.

Overall: 1:31s

npm ci test
node: v8.9.3
node: 5.8.0
angularjs-project: added 1021 packages in 15.641s
angular4-project2: 1184 packages in 12.138s
angular4-project1: added 1269 packages in 7.113s

Overall: 0:36s

[jpickwell]

I don’t think this is an issue anymore, at least with Yarn 1.17.3. In fact, using the --link-duplicates option makes node_modules even smaller.

macOS 10.14.5
node 10.16.0
npm 6.9.2
yarn 1.17.3

node_modules => yarn install --link-duplicates
npm_ci_node_modules => npm ci
npm_node_modules => npm install
yarn_node_modules => yarn install

λ du -hs *node_modules
612M	node_modules
646M	npm_ci_node_modules
647M	npm_node_modules
630M	yarn_node_modules

[olingern]

Closing this. If this is is still an issue on 1.21.1, please comment and I'll reopen.