"npm run watch" runs lerna with too many threads

[jmoujaes]

Description

Hello! I would like to make a suggestion to reduce the number of threads used by lerna in the "npm run watch" script.

gatsby/package.json

Line 69 in 216ed95

"watch": "lerna run watch --no-sort --stream --concurrency 999"

I suggest using the default number of threads (4) instead of the current number (999). The reason is that my VM froze up when I tried to run "npm run watch".

Environment

Gatsby version: master
Node.js version: v8.9.4
Operating System: Ubuntu 16.04 VM running on a Windows host. Has 4GB of RAM allotted. Had 8 firefox tabs open as well as one terminal session in which I ran the "npm run watch" command.

Actual result

VM froze.

Expected behavior

VM shouldn't freeze.

Steps to reproduce

1. Configure Ubuntu 16.04 VM with 4GB of RAM. Start the VM

2. Open up 8 firefox tabs, browse to github in all of them.

3. Open up one terminal session.

4. Go to your forked gatsby project root.

5. Run "npm run watch"

6. Notice slowness or freezing.

Note

I'm happy to make a PR.
Thanks for reading!

[m-allanson]

Ahh, that's why watch makes my machine chug for a few minutes. Sounds like a good change to me?

[KyleAMathews]

We need a separate watch process for each package — you can't restrict it to 4 as watch processes don't quit. Four would start up and then... the others would never be able to start.

A better solution perhaps would be a way to select which packages to watch as most of the time you're only editing 1 or maybe 2-3 packages.

[jmoujaes]

I see. So the reason for the slowdown is that it is watching a lot of packages, not that the concurrency flag is 999.

Something like this may accomplish what you're saying
npm run watch -- --scope={gatsby,gatsby-cli}

That passes --scope={gatsby,gatsby-cli} to the watch script and appends it to the end. So it effectively becomes
lerna run watch --no-sort --stream --concurrency 999 --scope={gatsby,gatsby-cli}
which will only watch changes for those two packages.

A little bit of context around how I came to file the issue is I was going through the steps in contributing.md to try to work another bug. I ran "npm run watch" which made it watch all the packages.

Do you think it would be worthwhile to put a note in there saying that it is a CPU intensive operation and suggest passing the package names individually?

[m-allanson]

@jmoujaes that sounds like a good fix. However I think you'd initially need all the packages built, as some of them depend on each other.

Maybe we need an earlier step in the docs that creates the initial built files. Something like:

• Run npm run lerna-prepublish to create an initial build of each package
• Run npm run watch. The watch command can be resource intensive. To limit it to the packages you're working on, add a scope flag, like npm run watch -- --scope={gatsby,gatsby-cli}

[KyleAMathews]

@m-allanson the bootstrap script which you should run to setup the repo does run pre-publish

gatsby/package.json

Line 49 in d5d9815

"bootstrap": "npm-run-all -s check-versions lerna-prepublish",

Didn't know about scope! Yeah, we should add that to the docs — @jmoujaes could you edit the CONTRIBUTING document to change npm run watch to demonstrate how to scope watching to packages you'll be editing?

[KyleAMathews]

https://github.com/gatsbyjs/gatsby/blob/master/CONTRIBUTING.md#contributing-to-the-repo

[jmoujaes]

Created PR, please let me know if you have any suggestions.
#4020

[KyleAMathews]

Thanks @jmoujaes!

[jmoujaes]

For sure, thank you both for being responsive!