build: allow easier checking of permanent deoptimizations #12456

mscdex · 2017-04-17T04:32:12Z

As the title says, this adds a build target to allow for checking for permanent deoptimizations in node core simply by running the parallel and sequential tests. This is not a "complete" solution per-se because it currently creates (and ignores) the issues it causes with tests that fork processes due to the extra output on stdout, which some tests check. However, aside from that it should work correctly everywhere else.

There are a few deoptimization fixes I've left out in this PR:

domain.js
- runBound() and runIntercepted() have deopts that do not seem straight-forward to fix AFAICT because of the API we are exposing. The deopt reason for both is:
  
  Bad value context for arguments value
  
  I was able to fix this same issue in another spot by bind()ing the function (this makes it monomorphic), but I'm not sure we can do that in these two functions because this is being utilized and I am not sure how much of userland calls the created bound/intercept functions with custom this values.
internal/process.js
- v8BreakIterator() also has
  Bad value context for arguments value
  I'm ignoring it since it's deprecated.
punycode.js
- adapt() and decode() both have:
  Unsupported let compound assignment
  I'm ignoring these because it's third party code and the module is deprecated.
repl.js
- This is a strange one and I haven't had time to narrow it down since the deopt message does not include a function name, but the reason is:
  Invalid left-hand side in assignment
  I've read that this may be due to a Crankshaft limitation and that it's *possible* TurboFan may not have this same deopt issue.

There are a bunch of other permanent deopts, but they lie in the actual tests themselves and not in lib/. Currently this build target shows all deopts, no matter where they occur as it does not seem that we can always reliably detect whether a deopt is from a test or not. I am not sure if deopts in tests are worth going after, perhaps it could be a possibility for code-and-learn or similar events (although the deopt reasons may be too confusing for some audiences)?

CI: https://ci.nodejs.org/job/node-test-pull-request/7436/

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
commit message follows commit guidelines

Affected core subsystem(s)

build
lib
test

mscdex · 2017-04-17T04:34:10Z

The changes in fs might be considered semver-major-y because of the change in how the callback is detected. The new method of callback detection I used is also used in other parts of fs though FWIW.

mscdex · 2017-04-17T04:37:12Z

Also, I'm not sure if the vcbuild.bat changes are absolutely correct or done the best way, batch files are not my forte.

For *nix though all you need to do is make test-check-deopts to run the checks.

benjamingr · 2017-04-17T08:37:45Z

Great work! I think this is a good step and I'd love to see this as part of our build.

I am not sure if deopts in tests are worth going after, perhaps it could be a possibility for code-and-learn or similar events (although the deopt reasons may be too confusing for some audiences)?

It depends on whether or not they're actually in the tests or are caused by code calling code outside of the tests and would happen in user code when calling Node modules too necessarily.

I'd like to see how this looks on TF+Ignition. Will review actual code soon.

benjamingr · 2017-04-17T08:39:48Z

lib/fs.js

-  for (const key in source)
+  var target = {};
+  var key;
+  for (key in source)


Does for (var key in source) really deopt here?

No, just the const usage. I thought there was an issue with using var in a for-in at one time, but I could be remembering wrong. I will change those.

benjamingr · 2017-04-17T08:42:37Z

lib/fs.js

@@ -320,7 +321,7 @@ fs.existsSync = function(path) {
 };

 fs.readFile = function(path, options, callback) {
-  callback = maybeCallback(arguments[arguments.length - 1]);
+  callback = maybeCallback(callback || options);


I see why this might indeed be "semver-major-y" but it looks fine to me. The only difference is in the case callback is passed but not as a function at which point options would be passed and would be rethrown if real options passed.

I think a CITGM run would be sufficient here and if that passes I don't think we need to change it or semver-major it.

Also I'm very surprised arguments use only indexed access and length check is performed deopts.

I think it's also because arguments was being used in a function where the named arguments were being overwritten. This could be a Crankshaft-specific limitation though, I'm not sure.

benjamingr · 2017-04-17T08:42:52Z

lib/fs.js

@@ -1248,7 +1247,7 @@ function writeAll(fd, isUserFd, buffer, offset, length, position, callback_) {
 }

 fs.writeFile = function(path, data, options, callback) {
-  callback = maybeCallback(arguments[arguments.length - 1]);
+  callback = maybeCallback(callback || options);


ditto for comment above.

benjamingr · 2017-04-17T08:43:50Z

lib/internal/cluster/child.js

@@ -99,7 +99,7 @@ function shared(message, handle, indexesKey, cb) {
    delete handles[key];
    delete indexes[indexesKey];
    return close.apply(this, arguments);
-  };
+  }.bind(handle);


Mind explaining this one? Typically bind isn't that great at not causing deopts :D

Bad value context for arguments value

See: https://gist.github.com/Hypercubed/89808f3051101a1a97f3

benjamingr · 2017-04-17T08:49:52Z

Actual changes LGTM but I'd like to check it out and test it first.

I'd also land this in two parts - fixing the deopts and adding the flag, but I'd prefer defer that choice to you and the CTC.

mscdex · 2017-04-17T13:53:41Z

It depends on whether or not they're actually in the tests or are caused by code calling code outside of the tests and would happen in user code when calling Node modules too necessarily.

For the most part the deopts in tests are clearly labeled with absolute paths to the test file in the deopt message line, so that was how I knew the deopt was not actually happening in node core. However for some special cases, such as the vm tests where the filename can be overridden, this absolute path does not exist, making it possibly look like a node core deopt.

refack · 2017-04-17T14:03:06Z

How about for all the places you replaced const with var, you add a /** @const */ so the intention is saved, and could be checked by linters?

refack

Add /** @const */ where replacing const with var

mscdex · 2017-04-17T14:15:31Z

@refack Here are some issues I see with doing such things:

It can add a lot of noise, which could especially become a problem for functions designed to be inlineable in Crankshaft
Adding comments for one particular type of deopt and not all others doesn't seem particularly useful, and doing it for all deopt situations can be tedious. Combine this with the fact that some of these deopts may not even be a thing anymore after the switch to TurboFan + Ignition.
I think there is/was already efforts somewhere for linters to check for potentially deoptimized code without the need for any kind of hints/metadata.

My intention with this PR was to have something so that in case someone does change a var back to const for example, they can run make test-check-deopts afterwards and notice that there is a new deopt and that they should revert their change. This way we don't need to litter the source with unnecessary comments.

refack · 2017-04-17T14:24:47Z

Apart from automatic tools, there's also the future human reader.

It can add a lot of noise, which could especially become a problem for functions designed to be inlineable in Crankshaft

Comments? will add noise for Crankshaft? (sincere question, could you point me to more info?)

Adding comments for one particular type of deopt and not all others doesn't seem particularly useful, and doing it for all deopt situations can be tedious.

var => const seems like the most prevalent in this PR, and is a only one that causes loss of intention.

Combine this with the fact that some of these deopts may not even be a thing anymore after the switch to TurboFan + Ignition.

Not be a thing as in; not be deopts? so we'd revert this change?

I think there is/was already efforts somewhere for linters to check for potentially deoptimized code without the need for any kind of hints/metadata.

I just don't want the intention to be lost (some of your changes superficially seem syntactical, while other are semantic [reusing the vars])

mscdex · 2017-04-17T14:40:59Z

@refack

Comments? will add noise for Crankshaft? (sincere question, could you point me to more info?)

One way to cause a function to not be inlineable in Crankshaft is to exceed the maximum source length (default is 600 characters).

var => const seems like the most prevalent in this PR, and is a only one that causes loss of intention.

Fixes for other deopts could be viewed just as unclear (e.g. the .bind() addition in child.js to fix a different type of deopt).

Not be a thing as in; not be deopts? so we'd revert this change?

What I mean is that TurboFan + Ignition may not cause these deopts (they may be Crankshaft-specific), so it may/may not matter for the master branch (and node v8.x) after future V8 upgrades. If they are Crankshaft-specific, then they could be reverted if necessary.

I just don't want the intention to be lost (some of your changes superficially seem syntactical, while other are semantic [reusing the vars])

Any reuse of variables is due to duplicate const variable names. When they are converted to var the linter complains, so the variable definition needs to be lifted out.

refack · 2017-04-17T14:46:47Z

I thought that adding comments will be the perfect balance of gaining performance, and keeping readability/maintainability. If I'm so off target, I'll remove my objection.

One way to cause a function to not be inlineable in Crankshaft is to exceed the maximum source length (default is 600 characters).

Including comments?

What I mean is that TurboFan + Ignition may not cause these deopts (they may be Crankshaft-specific), so it may/may not matter for the master branch (and node v8.x) after future V8 upgrades. If they are Crankshaft-specific, then they could be reverted if necessary.

👍

Any reuse of variables is due to duplicate const variable names. When they are converted to var the linter complains, so the variable definition needs to be lifted out.

Ok. but, they are sill "loss of intention" cases.

refack · 2017-04-17T14:49:20Z

Fixes for other deopts could be viewed just as unclear (e.g. the .bind() addition in child.js to fix a different type of deopt).

In second thought that's actually even worse, as a future dev (or a current dev like me) will get confused and might want to remove it. IMHO non-intuitive changes should be commented.

refack · 2017-04-17T14:51:00Z

It can add a lot of noise, which could especially become a problem for functions designed to be inlineable in Crankshaft

One way to cause a function to not be inlineable in Crankshaft is to exceed the maximum source length (default is 600 characters).

Including comments?

@hashseed do you know if comments hinder inlining?

PR-URL: #12456 Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

gibfahn · 2017-05-15T17:22:52Z

@mscdex should this be backported to v6.x?

mscdex · 2017-05-15T20:45:12Z

@gibfahn Sure.

mscdex added build Issues and PRs related to build files or the CI. lib / src Issues and PRs related to general changes in the lib or src directory. test Issues and PRs related to the tests. labels Apr 17, 2017

mscdex removed the windows Issues and PRs related to the Windows platform. label Apr 17, 2017

mscdex force-pushed the build-check-deopts branch 2 times, most recently from 9585e11 to 0481405 Compare April 17, 2017 06:20

mscdex mentioned this pull request Apr 17, 2017

url: always show password for URL instances #12420

Closed

2 tasks

benjamingr reviewed Apr 17, 2017

View reviewed changes

mscdex force-pushed the build-check-deopts branch from 0481405 to 17e5d49 Compare April 17, 2017 13:46

refack previously requested changes Apr 17, 2017

View reviewed changes

evanlucas pushed a commit that referenced this pull request May 2, 2017

cluster: fix permanent deoptimizations

7feffa8

PR-URL: #12456 Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

evanlucas pushed a commit that referenced this pull request May 2, 2017

process: fix permanent deoptimizations

15bfec2

PR-URL: #12456 Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

evanlucas pushed a commit that referenced this pull request May 2, 2017

net: fix permanent deoptimizations

31874a7

PR-URL: #12456 Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

evanlucas pushed a commit that referenced this pull request May 2, 2017

readline: fix permanent deoptimizations

d411dd0

PR-URL: #12456 Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

evanlucas pushed a commit that referenced this pull request May 2, 2017

test: fix permanent deoptimizations

875c6fb

PR-URL: #12456 Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

evanlucas pushed a commit that referenced this pull request May 2, 2017

test: fix test filenames

4cd73b4

PR-URL: #12456 Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

evanlucas pushed a commit that referenced this pull request May 2, 2017

build: add target for checking for perm deopts

0f58599

PR-URL: #12456 Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

evanlucas pushed a commit that referenced this pull request May 2, 2017

stream: fix permanent deoptimizations

61a47da

PR-URL: #12456 Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

evanlucas pushed a commit that referenced this pull request May 2, 2017

fs: fix permanent deoptimizations

4fb6ae4

PR-URL: #12456 Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

evanlucas pushed a commit that referenced this pull request May 2, 2017

cluster: fix permanent deoptimizations

1e03da6

PR-URL: #12456 Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

evanlucas pushed a commit that referenced this pull request May 2, 2017

process: fix permanent deoptimizations

9821c2f

PR-URL: #12456 Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

evanlucas pushed a commit that referenced this pull request May 2, 2017

net: fix permanent deoptimizations

0853286

PR-URL: #12456 Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

evanlucas pushed a commit that referenced this pull request May 2, 2017

readline: fix permanent deoptimizations

1a33df5

PR-URL: #12456 Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

evanlucas pushed a commit that referenced this pull request May 2, 2017

test: fix permanent deoptimizations

d3e1d45

PR-URL: #12456 Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

evanlucas pushed a commit that referenced this pull request May 3, 2017

test: fix test filenames

9da719a

PR-URL: #12456 Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

evanlucas pushed a commit that referenced this pull request May 3, 2017

build: add target for checking for perm deopts

caf6506

PR-URL: #12456 Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

evanlucas pushed a commit that referenced this pull request May 3, 2017

stream: fix permanent deoptimizations

7d0adc6

PR-URL: #12456 Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

evanlucas pushed a commit that referenced this pull request May 3, 2017

fs: fix permanent deoptimizations

84484b7

PR-URL: #12456 Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

evanlucas pushed a commit that referenced this pull request May 3, 2017

cluster: fix permanent deoptimizations

933b6b5

PR-URL: #12456 Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

evanlucas pushed a commit that referenced this pull request May 3, 2017

process: fix permanent deoptimizations

b573f77

PR-URL: #12456 Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

evanlucas pushed a commit that referenced this pull request May 3, 2017

net: fix permanent deoptimizations

4a08288

PR-URL: #12456 Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

evanlucas pushed a commit that referenced this pull request May 3, 2017

readline: fix permanent deoptimizations

1ac331b

PR-URL: #12456 Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

evanlucas pushed a commit that referenced this pull request May 3, 2017

test: fix permanent deoptimizations

7af2e79

PR-URL: #12456 Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

gibfahn added baking-for-lts PRs that need to wait before landing in a LTS release. lts-watch-v6.x labels May 15, 2017

jasnell mentioned this pull request May 11, 2017

8.0.0 Release Proposal #12220

Closed

seishun mentioned this pull request Jul 17, 2017

fs: don't conflate data and callback in fs.appendFile #11607

Closed

3 tasks

MylesBorins removed the baking-for-lts PRs that need to wait before landing in a LTS release. label Aug 17, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

build: allow easier checking of permanent deoptimizations #12456

build: allow easier checking of permanent deoptimizations #12456

mscdex commented Apr 17, 2017 •

edited

Loading

mscdex commented Apr 17, 2017 •

edited

Loading

mscdex commented Apr 17, 2017

benjamingr commented Apr 17, 2017

benjamingr Apr 17, 2017

mscdex Apr 17, 2017

benjamingr Apr 17, 2017

mscdex Apr 17, 2017

benjamingr Apr 17, 2017

benjamingr Apr 17, 2017

mscdex Apr 17, 2017

benjamingr commented Apr 17, 2017

mscdex commented Apr 17, 2017

refack commented Apr 17, 2017

refack left a comment

mscdex commented Apr 17, 2017 •

edited

Loading

refack commented Apr 17, 2017

mscdex commented Apr 17, 2017 •

edited

Loading

refack commented Apr 17, 2017

refack commented Apr 17, 2017

refack commented Apr 17, 2017

gibfahn commented May 15, 2017

mscdex commented May 15, 2017

build: allow easier checking of permanent deoptimizations #12456

build: allow easier checking of permanent deoptimizations #12456

Conversation

mscdex commented Apr 17, 2017 • edited Loading

Checklist

Affected core subsystem(s)

mscdex commented Apr 17, 2017 • edited Loading

mscdex commented Apr 17, 2017

benjamingr commented Apr 17, 2017

benjamingr Apr 17, 2017

Choose a reason for hiding this comment

mscdex Apr 17, 2017

Choose a reason for hiding this comment

benjamingr Apr 17, 2017

Choose a reason for hiding this comment

mscdex Apr 17, 2017

Choose a reason for hiding this comment

benjamingr Apr 17, 2017

Choose a reason for hiding this comment

benjamingr Apr 17, 2017

Choose a reason for hiding this comment

mscdex Apr 17, 2017

Choose a reason for hiding this comment

benjamingr commented Apr 17, 2017

mscdex commented Apr 17, 2017

refack commented Apr 17, 2017

refack left a comment

Choose a reason for hiding this comment

mscdex commented Apr 17, 2017 • edited Loading

refack commented Apr 17, 2017

mscdex commented Apr 17, 2017 • edited Loading

refack commented Apr 17, 2017

refack commented Apr 17, 2017

refack commented Apr 17, 2017

gibfahn commented May 15, 2017

mscdex commented May 15, 2017

mscdex commented Apr 17, 2017 •

edited

Loading

mscdex commented Apr 17, 2017 •

edited

Loading

mscdex commented Apr 17, 2017 •

edited

Loading

mscdex commented Apr 17, 2017 •

edited

Loading