Long term caching and split app into main.js and vendors.js #3145

Ayc0 · 2017-09-17T19:01:15Z

Proposal for issue #2206

facebook-github-bot · 2017-09-17T19:01:20Z

Thank you for your pull request and welcome to our community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. In order for us to review and merge your code, please sign up at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need the corporate CLA signed.

If you have received this in error or have any questions, please contact us at cla@fb.com. Thanks!

facebook-github-bot · 2017-09-17T19:19:14Z

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Facebook open source project. Thanks!

Timer · 2017-09-17T22:30:37Z

packages/react-scripts/config/webpack.config.prod.js

@@ -64,7 +64,7 @@ module.exports = {
  // You can exclude the *.map files from the build during deployment.
  devtool: 'source-map',
  // In production, we only want to load the polyfills and the app code.
-  entry: [require.resolve('./polyfills'), paths.appIndexJs],
+  entry: [require.resolve('./polyfills'), paths.appIndexJs, paths.appVendors],


Can you explain this to me? I thought it needed to be like this:

{ entry: { vendors: require(paths.appVendors), app: [require.resolve('./polyfills'), paths.appIndexJs] } }

I've done this to remain coherent with the current code
I think that

{ entry: { vendors: require(paths.appVendors), app: paths.appIndexJs, polyfills: require.resolve('./polyfills'), } }

is easier to understand

That seems fine as long as the polyfills get included before the app, and vendors before the app too.

I don't fully understand webpack entries 😄 .

you can give either a string which indicates the location of a file (or a npm package), an array of those strings or an object.
If it's an array or a string, the name of the entry will be the name of the file, and if you give an object, the name is the key and the location is the value.

In this case, the vendors in an array so webpack's gonna concatenate the vendors into a single chunk called vendors.[hash].js and the app and the polyfills are linked to the location of the file

I haven't put the files in the right order, I'll change this

the order of the entries is handle by HtmlWebpackPlugin with its option chunksSortMode

From what I understand, NameAllModulesPlugin is required for when externals are added because they cause ids to shift.

Sure, it may not affect non-ejected users, but we'd like ejected users to have something that "just works".

the module NameAllModulesPlugin is deprecated and now we have to use NamedModulesPlugin

(In fact, it's not, I have to learn how to read...)

Haha you're alright! You're a total champ for your work on this PR; most people give up by now.

Dude, it's just a two day work and I already did the split vendor thing in one of my project haha 😀

Timer · 2017-09-17T22:31:06Z

If possible, I'd like to get as close as possible to this: https://medium.com/webpack/predictable-long-term-caching-with-webpack-d3eee1d3fa31

react-scripts-dangerous · 2017-09-17T23:25:01Z

Hello! I'm a bot that helps facilitate testing pull requests.

Your pull request (commit 9bfe59544077be82794bcbf5c34e0667259da36d) has been released on npm for testing purposes.

npm i react-scripts-dangerous@1.0.11-9bfe595.0
# or
yarn add react-scripts-dangerous@1.0.11-9bfe595.0
# or
create-react-app --scripts-version=react-scripts-dangerous@1.0.11-9bfe595.0 folder/

Note that the package has not been reviewed or vetted by the maintainers. Only install it at your own risk!

Thanks for your contribution!

Timer · 2017-09-18T02:46:00Z

We need a story for when src/vendors.js doesn't exist. Can we have it default to not creating a vendors bundle?
Or maybe default to react and react-dom?

Timer · 2017-09-18T02:55:38Z

packages/react-scripts/config/webpack.config.prod.js

+    // Thanks to this change the vendor hash will now always stay the same
+    new webpack.NamedModulesPlugin(),
+    // Ensure that every chunks have an actual name and not an id
+    // If the chunck has a name, this name is used


nit: chunck

react-scripts-dangerous · 2017-09-18T03:05:17Z

Hello! I'm a bot that helps facilitate testing pull requests.

Your pull request (commit 700470871e04b689563fdd3bf98c9db013b825f8) has been released on npm for testing purposes.

npm i react-scripts-dangerous@1.0.11-7004708.0
# or
yarn add react-scripts-dangerous@1.0.11-7004708.0
# or
create-react-app --scripts-version=react-scripts-dangerous@1.0.11-7004708.0 folder/

Note that the package has not been reviewed or vetted by the maintainers. Only install it at your own risk!

Thanks for your contribution!

Timer · 2017-09-18T03:10:16Z

I forced a release for you (by @react-scripts-dangerous) in case the bug in CI only manifests during a simulated release. Just trying to be of help!

I'm going to sleep for now, so good luck! Thanks for what you're doing. 😄

Ayc0 · 2017-09-18T03:13:24Z

Thanks Timer 👍

Ayc0 · 2017-09-18T07:44:03Z

Timer, I've tried many things, the NamedChunksPlugin was a bit buggy when dynamic imports occured in the code. And sorting entries with the HTML plugging cause the compilation to fail.
So I've made a tweak for the NamedChunksPlugin (I need to improve it) and remove the other one
I've tried to change the orders of the entries but it doesn't change the order of the <script> in the HTML file
If the vendor file doesn't exist, there is a fallback set to react and react-dom

Ayc0 · 2017-09-18T07:49:18Z

The issue with NamedChunksPlugin and dynamic imports is that the filename is .../node_modules/babel_loader/index.js?.../src/file.js but we only want to keep file

Timer · 2017-09-18T12:57:03Z

Is NameAllModulesPlugin deprecated or just use an old API? We could look at what it's doing and apply it ourselves with the new API.

I've tried to change the orders of the entries but it doesn't change the order of the <script> in the HTML file

So does the chunksSortMode option not work? Do we need to file a bug?

Ayc0 · 2017-09-18T13:14:27Z

For NameAllModulesPlugin, I don't know, I'll look further tonight

And the option chunksSortMode causes either a big during the compilation or when it's compiled, the react-app doesn't start. I'll also look tonight if we can force an order in the entries

Timer · 2017-09-18T13:56:06Z

packages/react-scripts/config/webpack.config.prod.js

+      }
+      return (chunk.mapModules
+        ? chunk.mapModules(m => m.request)
+        : chunk.modules.map(m => m.request)


I don't think we need to have fallback behavior for old webpack versions, or is this here for another reason?

mapModules isn't supported in Webpack 2.6 and modules are deprecated in Webpack 3

We don't need to worry about webpack 2.6 support 😄

It was the current version on CRA when I fork the repo
But now the v3 is used 😁

I'll pull the current version of CRA

Timer · 2017-09-18T15:29:50Z

Looks like you accidentally rebased in the wrong direction 😛

Timer · 2017-09-18T15:33:36Z

Fix via:

$ git pull # just in case, I see you're on two different computers
$ git remote add upstream https://github.com/facebookincubator/create-react-app.git
$ git fetch upstream
$ git rebase upstream/master
$ git push -f

Then, on your other computer, make sure you do (for the first time after you force push):

$ git pull --rebase

or, I didn't test this one but it should work too:

$ git status # make sure your directory is clean; if it's not, commit and use the above command
$ git fetch origin
$ git reset --hard origin/master

Ayc0 · 2017-09-18T15:40:15Z

Oh yeah, i've rebased by reflex, sorry

react-scripts-dangerous · 2017-09-18T16:52:04Z

Hello! I'm a bot that helps facilitate testing pull requests.

Your pull request (commit 11579fb9fb1848208be47edb53bba981b8e94ddc) has been released on npm for testing purposes.

npm i react-scripts-dangerous@1.0.14-11579fb.0
# or
yarn add react-scripts-dangerous@1.0.14-11579fb.0
# or
create-react-app --scripts-version=react-scripts-dangerous@1.0.14-11579fb.0 folder/

Note that the package has not been reviewed or vetted by the maintainers. Only install it at your own risk!

Thanks for your contribution!

Timer · 2017-09-18T17:24:25Z

packages/react-scripts/config/webpack.config.prod.js

+    // List of all the node modules that should be excluded from the app
+    vendors: fs.existsSync(paths.appVendors)
+      ? require(paths.appVendors)
+      : ['react', 'react-dom'],


@gaearon would you agree these are decent defaults, or would you rather disable the vendor bundle all together?

I'm not sure how many people use react-scripts without react. 😄

can you use react-scripts without react ?
If you want to use it with preact for instance, you have to eject so...

You wouldn't have to eject if you don't need to alias (external modules), right?

I've never used external modules so I have no ideas

I meant a node_modules package by external modules.

We can wait for @gaearon's input, but I think we'd just change:

entry: { // Load the app and all its dependencies main: paths.appIndexJs, // Add the polyfills polyfills: require.resolve('./polyfills'), // List of all the node modules that should be excluded from the app vendors: fs.existsSync(paths.appVendors) ? require(paths.appVendors) : ['react', 'react-dom'], },

to something like this:

entry: Object.assign( { // Load the app and all its dependencies main: paths.appIndexJs, // Add the polyfills polyfills: require.resolve('./polyfills'), }, fs.existsSync(paths.appVendors) ? // List of all the node modules that should be excluded from the app { vendors: require(paths.appVendors) } : {} ),

You're right, it makes more sense to only load the vendors if the file exists, it's more understandable this way

@Timer, I managed to import the entries in the right order with HtmlWebpackPlugin and the option chunksSortMode
What wasn't working before is that I've imported an entry before the entry called runtime. And if you place the entries here in a wrong order, it won't work. The right order is['runtime', 'vendors', 'polyfills', 'main']`
But as webpack loads the entries without any bugs, I think it auto detects which entries must be imported before the others so I don't know if it's really usefull to hardcode the order of the entries

Well if it's not needed we can remove it, but only if we're sure!

I'm not sure at all, it's just that if the order is wrong, it won't compile and this is the only arrangement that works, so it can't be by chance.

I've added the commit anyways, thus if we want to remove it, it's pretty easy and the work isn't lost

react-scripts-dangerous · 2017-09-19T06:42:00Z

Hello! I'm a bot that helps facilitate testing pull requests.

Your pull request (commit 64887f9f5d251256f6a381c3ec3ba992e148f976) has been released on npm for testing purposes.

npm i react-scripts-dangerous@1.0.14-64887f9.0
# or
yarn add react-scripts-dangerous@1.0.14-64887f9.0
# or
create-react-app --scripts-version=react-scripts-dangerous@1.0.14-64887f9.0 folder/

Note that the package has not been reviewed or vetted by the maintainers. Only install it at your own risk!

Thanks for your contribution!

Stanley-Jovel · 2017-09-20T18:22:43Z

Please merge this 🙏

Timer · 2017-09-21T14:32:56Z

packages/react-scripts/config/webpack.config.prod.js

+      // it loads the rest, then the vendors
+      // And then the polyfills before the main
+      // The others chunks will be loaded after
+      chunksSortMode: sortChunks(['runtime', 'vendors', 'polyfills', 'main']),


Why don't we remove this since we said that it automatically detects the correct order?
Let's not add this until a problem arises (if one does).

Less complexity is always better. 😄

Timer · 2017-09-21T14:33:15Z

packages/react-scripts/config/sortChunks.js

@@ -0,0 +1,45 @@
+// @remove-on-eject-begin


Per removing chunksSortMode, we can probably trash this file now.

Timer · 2017-09-21T14:33:29Z

packages/react-scripts/package.json

@@ -44,6 +44,7 @@
    "fs-extra": "3.0.1",
    "html-webpack-plugin": "2.29.0",
    "jest": "20.0.4",
+    "name-all-modules-plugin": "^1.0.1",


Please pin this package version.

Timer · 2017-09-21T14:37:38Z

packages/react-scripts/template/src/vendors.js

@@ -0,0 +1 @@
+module.exports = ['react', 'react-dom'];


How about we add a comment here explaining what adding an entry to this list does?

// This file controls which npm packages are extracted into a vendor bundle. // A vendor bundle is very useful for reducing your transmitted bundle size, as // dependencies which rarely change can be cached while you iterate on your application. // (edit this comment as you see fit and make it run to 80 cols please) module.exports = ['react', 'react-dom']; // You can delete this file if you'd prefer to not make a vendor bundle. // (edit this comment as you see fit and make it run to 80 cols please)

Timer · 2017-09-21T14:38:27Z

packages/react-scripts/config/webpack.config.prod.js

+
+    // Avoid having the vendors in the rest of the app
+    new webpack.optimize.CommonsChunkPlugin({
+      name: 'vendors',


Do we need to exclude this commons chunk if we don't specify vendors in entry? Or is it simply ignored?

I've tried this without the vendor entry and it seems to be okay

I've rechecked, and if the file exists, the vendor entry weights 40KB, and the main 10KB (can't remember the exact values), and without, webpack compiles and there is no vendors entry and main weighs 50KB

I'd like to exclude this for safety reasons; we're not sure how webpack might regress in the future.

Timer · 2017-09-21T15:47:22Z

packages/react-scripts/template/src/vendors.js

+// as dependencies which rarely change can be cached while you iterate on your
+// application.
+
+// If you want to add another vendor, just add in this array the name of the


This sounds backwards -- how about "If you want to add another npm package to the vendor bundle, just add its name in this array".

I'm not fluent in english so... 😁

No problem!

react-scripts-dangerous · 2017-09-21T17:12:31Z

Hello! I'm a bot that helps facilitate testing pull requests.

Your pull request (commit c40c666dcce74aad55f06e7e5640505d5b5e44ca) has been released on npm for testing purposes.

npm i react-scripts-dangerous@1.0.14-c40c666.0
# or
yarn add react-scripts-dangerous@1.0.14-c40c666.0
# or
create-react-app --scripts-version=react-scripts-dangerous@1.0.14-c40c666.0 folder/

Note that the package has not been reviewed or vetted by the maintainers. Only install it at your own risk!

Thanks for your contribution!

Ayc0 · 2017-09-21T21:55:21Z

@Timer do I need to change other things ?

Timer · 2017-09-21T22:29:46Z

packages/react-scripts/package.json

@@ -64,4 +65,4 @@
  "optionalDependencies": {
    "fsevents": "1.1.2"
  }
-}
+}


Can you re-add the ending newline? Sorry for such a trivial thing.

This reverts commit 64887f9f5d251256f6a381c3ec3ba992e148f976.

…s doesn't exist

Ayc0 · 2018-01-20T18:28:59Z

This PR is now useless
Webpack's just released automatic chunk plugin and chunk plugin are removed in webpack 4

Ayc0 · 2018-01-20T18:29:28Z

https://gist.github.com/sokra/1522d586b8e5c0f5072d7565c2bee693

gaearon · 2018-01-20T18:39:25Z

We're very thankful for all the work you've put into it ❤️
I'm sorry this didn't end up being merged.

If you'd like to send a PR updating us to webpack 4 beta (so we can get closer to trying this change) it would be very welcome!

gaearon · 2018-01-20T18:42:59Z

I filed #3878 for migrating to webpack 4.

facebook-github-bot added the CLA Signed label Sep 17, 2017

Timer reviewed Sep 17, 2017

View reviewed changes

Timer reviewed Sep 18, 2017

View reviewed changes

Timer mentioned this pull request Sep 20, 2017

Split vendor and app into separate files #2206

Closed

Timer previously requested changes Sep 21, 2017

View reviewed changes

Timer reviewed Sep 21, 2017

View reviewed changes

Ayc0 changed the title ~~Split vendor and app into separate files~~ Long term caching and split app into main.js and vendors.js Sep 21, 2017

Timer reviewed Sep 21, 2017

View reviewed changes

Ayc0 added 18 commits January 18, 2018 07:38

Only add the vendors if the file "src/vendors.js" exists

9e648d4

Ensure that major entries are loaded before the rest

00f160c

evert "Ensure that major entries are loaded before the rest"

186f289

This reverts commit 64887f9f5d251256f6a381c3ec3ba992e148f976.

Pin the version of name-all-modules-plugin

75cf8c9

Add comments in vendors.js

3ef3617

Change comments in vendors.js

f4a889d

Add newline at the end of package.json

88dd88e

Desactivate webpack.optimize.CommonsChunkPlugin(vendors) in vendors.j…

49ffbcb

…s doesn't exist

Change the way the optimize pluggin for the vendors is set or removed

6778d82

Change the way unamed chunks are named

f37f51d

Change commentaries

6277cd3

Check if every vendors are defined in the package.json

779faee

Add devDeps and peerDeps

910a5f7

Update comments

0d3b82c

Export code in a detectMissingVendors function of react-dev-utils

50782f3

Use json files instead of js files for vendors

7629f7a

merge with :next

b314c99

Add detectMissingVendors in files of react-dev-utils/package.json

0fdaac1

andriijas mentioned this pull request Jan 19, 2018

Allow analysis of generated bundle and dependencies #1858

Closed

gaearon closed this Jan 20, 2018

gaearon mentioned this pull request Jan 20, 2018

Update to webpack 4 and integrate automatic vendor code splitting #3878

Closed

andriijas mentioned this pull request Jan 23, 2018

Loading React + ReactDOM from cdn #2758

Closed

viankakrisna mentioned this pull request Jan 24, 2018

use dynamic-cdn-webpack-plugin for react and react-dom #3914

Closed

rashidul0405 mentioned this pull request Feb 20, 2018

Long term caching (vendor code splitting) vercel/next.js#3850

Closed

lock bot locked and limited conversation to collaborators Jan 20, 2019

Long term caching and split app into main.js and vendors.js #3145

Long term caching and split app into main.js and vendors.js #3145

Conversation

Ayc0 commented Sep 17, 2017

facebook-github-bot commented Sep 17, 2017

facebook-github-bot commented Sep 17, 2017

Choose a reason for hiding this comment

Ayc0 Sep 17, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Timer commented Sep 17, 2017

react-scripts-dangerous commented Sep 17, 2017

Timer commented Sep 18, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

react-scripts-dangerous commented Sep 18, 2017

Timer commented Sep 18, 2017

Ayc0 commented Sep 18, 2017

Ayc0 commented Sep 18, 2017

Ayc0 commented Sep 18, 2017

Timer commented Sep 18, 2017

Ayc0 commented Sep 18, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Timer commented Sep 18, 2017

Timer commented Sep 18, 2017 • edited Loading

Ayc0 commented Sep 18, 2017 • edited Loading

react-scripts-dangerous commented Sep 18, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Timer Sep 18, 2017 • edited Loading

Choose a reason for hiding this comment

Timer Sep 18, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

react-scripts-dangerous commented Sep 19, 2017

Stanley-Jovel commented Sep 20, 2017

Timer Sep 21, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Ayc0 Sep 21, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

react-scripts-dangerous commented Sep 21, 2017

Ayc0 commented Sep 21, 2017

Choose a reason for hiding this comment

Ayc0 commented Jan 20, 2018

Ayc0 commented Jan 20, 2018

gaearon commented Jan 20, 2018 • edited Loading

gaearon commented Jan 20, 2018

Ayc0 Sep 17, 2017 •

edited

Loading

Timer commented Sep 18, 2017 •

edited

Loading

Ayc0 commented Sep 18, 2017 •

edited

Loading

Timer Sep 18, 2017 •

edited

Loading

Timer Sep 18, 2017 •

edited

Loading

Timer Sep 21, 2017 •

edited

Loading

Ayc0 Sep 21, 2017 •

edited

Loading

gaearon commented Jan 20, 2018 •

edited

Loading