Fix up Compiler levels for PPC machines #1020

mhdawson · 2017-11-29T16:58:33Z

Compilers on PPC were update to 4.9 recently.

What I'd like to do is:

On BE just revert. We are already working on dropping BE from builds for 8.X and later. (I've removed from main download page, next step is to stop building in releases). (4.9 is only need to build later releases)
on LE leave it at 4.9, validate that the binaries run on machines with gcc 4.8 installed and if ok update the release machine to match.
Figure out how we have multiple compilers on each machine. We are likely to need this in the future and it would be good to know how we are going to accomplish this going forward.
Update the ansible scripts so they do the right thing on PPC again.

mhdawson · 2017-11-29T16:59:33Z

@jBarz @refack FYI

seishun · 2017-11-29T18:34:03Z

I assume BE will be dropped from CI as well?

gibfahn · 2017-11-29T19:08:26Z

I assume BE will be dropped from CI as well?

For newer release lines yes.

mhdawson · 2017-11-29T22:11:48Z

@seishun as gibfahn says, BE machines will still be in the CI and will be used to test the release lines before 8.X. The CI jobs will be updated to that for BE they don't run and simply pass as we have done in other similar instances.

mhdawson · 2017-11-29T22:43:36Z

Added this to release job

if [[ $ARCH == ppc64 && ${NODE_MAJOR_VERSION} -gt 7 ]]; then
  echo "Not building $disttype on PPC Big Endian slave $NODE_NAME as not supported for v8.x and later"
  exit 0
fi

So we'll no longer build on for releases 8.x and above on PPC Big Endian.

Will keep an eye on builds for next little while to make sure it does the right thing (I tested in a separate job and I believe it is good) and then add to test jobs.

mhdawson · 2017-11-29T22:44:34Z

Note that archi is ppc64le for the LE machines so PPC sill still build on PPC LE for all versions.

refack · 2017-11-30T03:06:56Z

Because of incompatibility with the libuv test jobs, I had to revert the default to gcc4.8 on all the 4 ppc machines (gcc4.9 is still installed).
Now we just need to add a condition that will pick gcc4.9 for node>=9

mhdawson · 2017-11-30T17:46:10Z

@refack be-1 seems broken now for Node.js test, while be-2 seems ok.

The failures are:

test-commit-plinux/nodes/ppcbe-ubuntu1404/out/Release/obj.target/v8_init/deps/v8/src/setup-isolate-full.o ../deps/v8/src/setup-isolate-full.cc
In file included from ../deps/v8/src/base/logging.h:8:0,
                 from ../deps/v8/src/setup-isolate-full.cc:7:
/usr/include/c++/4.8/cstring:41:28: fatal error: bits/c++config.h: No such file or directory
 #include <bits/c++config.h>
                            ^
compilation terminated.
make[2]: *** [/home/iojs/build/wor

mhdawson · 2017-11-30T17:47:03Z

Is it possible you missed some steps on be-1 ?

Can you also list the exact steps you followed so that we know how to update the ansible scripts to get to the same state ?

mhdawson · 2017-11-30T17:48:16Z

Good forward, one more request is that we only change 1 machine at a time, validate that all is ok for a few days and then update the second machine once we are confident in the changes.

mhdawson · 2017-11-30T17:51:41Z

Seems like only be-1 of the 2 BE machines were reverted.

root@test-osuosl-ubuntu1404-ppc64-be-2:~# gcc --version
gcc (Ubuntu 4.9.4-2ubuntu1~14.04.1) 4.9.4
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Last login: Thu Nov 30 17:49:56 2017 from 32.97.110.56
root@test-osuosl-ubuntu1404-ppc64-be-1:~# gcc --version
gcc (Ubuntu 4.8.5-2ubuntu1~14.04.1) 4.8.5
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

So seems the revert has not reverted back to the prior state.

Both the le machines look like they were reversted.

mhdawson · 2017-11-30T17:52:34Z

One thing I think may be related is that I think the default on PPCBE is 32 bit versus the 64 bit compiler and that may be related.

mhdawson · 2017-11-30T17:54:31Z

@refack is it possible when installing/reverting 'g++-4.8-multilib' may have been removed ?

mhdawson · 2017-11-30T17:57:51Z

I think that multilib may no longer be there:

root@test-osuosl-ubuntu1404-ppc64-be-1:~# dpkg -l g++-4.8-multilib
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                               Version                Architecture           Description
+++-==================================-======================-======================-==========================================================================
un  g++-4.8-multilib                   <none>                 <none>                 (no description available)

mhdawson · 2017-11-30T17:58:50Z

@refack I'd like to have the steps you used for the revert to understand the sequence to see if re-installing multilib makes sense.

refack · 2017-11-30T18:10:51Z

I did the following (build-essentials overwrites the gcc/g++ symlinks which are used by default)

apt-get install build-essential
apt-get install libtool automake m4 g++ gcc

RE multilib, if I remember correctly it's simply named gcc-multilib:

root@test-osuosl-ubuntu1404-ppc64-be-1:~# dpkg -l gcc-multilib
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                                           Version                      Architecture                 Description
+++-==============================================-============================-============================-=================================================================================================
ii  gcc-multilib                                   4:4.8.2-1ubuntu6             powerpc                      GNU C compiler (multilib files)

mhdawson · 2017-11-30T18:14:10Z

@refack those were the commands for the revert ?

refack · 2017-11-30T18:16:25Z

@mhdawson yes.

Also, I then run a libuv-test-commit-linux to see if it passes, it also prints gcc --version showing 4.8.
I then run node-test-commit-plinux to verify everything builds.

mhdawson · 2017-11-30T18:19:55Z

Not sure how to explain those results as the CI is failing every time on the be-1 https://ci.nodejs.org/job/node-test-commit-plinux/. For recent runs: red is be-1, green is be-2

mhdawson · 2017-11-30T18:21:04Z

be-2 is still gcc 4.9 though, so if the run you launched there is would have been ok.

root@test-osuosl-ubuntu1404-ppc64-be-2:~# gcc --version
gcc (Ubuntu 4.9.4-2ubuntu1~14.04.1) 4.9.4
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

mhdawson · 2017-11-30T18:22:31Z

I think it is most likely related to multilib so I guess I'll try re-installing g++-4.8-multilib as done by the ansible scripts.

mhdawson · 2017-11-30T18:26:40Z

apt-get told me it was not installed, and it installed 2 new packages. Testing out a build now on be-1

mhdawson · 2017-11-30T18:46:41Z

Seems to have resolved the issue when testing from the command line. Will re-enable in CI and kickoff some tests to validate there.

mhdawson · 2017-11-30T18:47:53Z

Job running on be-1 - https://ci.nodejs.org/job/node-test-commit-plinux/13600/nodes=ppcbe-ubuntu1404/

mhdawson · 2017-11-30T19:16:34Z

Job passed, so re-installed seems to have resolved.

@refack you reverted because of some issues with libuv, is there a CI job I should run on be-1 to validate that side is now ok as well ?

mhdawson · 2017-11-30T19:17:08Z

I'm guessing it is libuv-test-commit-linux, running that now

mhdawson · 2017-11-30T20:34:34Z

Run on libuv - https://ci.nodejs.org/view/libuv/job/libuv-test-commit-linux/647/nodes=ppcbe-ubuntu1404/ on be-1

mhdawson · 2017-11-30T20:57:06Z

libuv ran passed. @refack anything else I should check before updating be-2 in the same way. It seems to still have the 4.9 compiler.

refack · 2017-11-30T22:07:23Z

https://ci.nodejs.org/view/All/job/node-test-commit-v8-linux/, but I see you covered that.

mhdawson · 2017-11-30T22:14:11Z

@refack ok thanks.

refack · 2017-12-01T21:47:42Z

BTW I found Ben's comment about compatibility: nodejs/node#13466 (comment) (mentioned in #762)

mhdawson · 2017-12-01T21:49:43Z

Ok, don't see any failures on be-1 today, will plan to revert to 4.8 on be-2 on monday

mhdawson · 2017-12-04T22:37:15Z

Reverted be-2, local compile test was ok, running CI now https://ci.nodejs.org/job/node-test-commit-plinux/nodes=ppcbe-ubuntu1404/13674/

mhdawson · 2017-12-05T20:33:15Z

Created test-osuosl-ubuntu14-ppc64-le-3 for use in testing new ansible scripts/install of both 4.8 and 4.9 compilers at the same time.

seishun · 2018-01-04T19:06:46Z

@mhdawson what's the status of this? I assume dropping CI for BE is covered in #1068, but LE still uses an outdated compiler: https://ci.nodejs.org/job/node-test-commit-plinux/14374/nodes=ppcle-ubuntu1404/

mhdawson · 2018-01-04T22:15:11Z

@jBarz is working on validating a dual compiler config for the power machines, getting him access in #1032. With the holidays it is stretching out a bit longer than expected.

mhdawson · 2018-02-09T18:24:03Z

Making progress, landed new ansible scripts for PPC le, still need to roll out to all of the machines.

mhdawson · 2018-05-11T15:36:53Z

This is complete, closing.

mhdawson mentioned this issue Nov 29, 2017

ansible: use gcc 4.9 on Ubuntu 14.04 #797

Closed

mhdawson mentioned this issue Nov 30, 2017

V8 6.1 backports nodejs/node#17354

Closed

mhdawson mentioned this issue Dec 5, 2017

linuxOne: gcc upgrade #1023

Closed

maclover7 added the platform:ppc label Feb 13, 2018

seishun mentioned this issue Feb 28, 2018

ansible: use gcc 4.9 on cross compiling machine #1094

Merged

gibfahn mentioned this issue Mar 22, 2018

Latest NodeJS 4.9.0, 6.14.0, 8.11.0, 9.9.0 and 9.10.0 not running on PPCLE Red Hat Enterprise Linux Server release 7.3 nodejs/node#19530

Closed

mhdawson closed this as completed May 11, 2018

Fix up Compiler levels for PPC machines #1020

Fix up Compiler levels for PPC machines #1020

Comments

mhdawson commented Nov 29, 2017 • edited Loading

mhdawson commented Nov 29, 2017

seishun commented Nov 29, 2017

gibfahn commented Nov 29, 2017

mhdawson commented Nov 29, 2017 • edited Loading

mhdawson commented Nov 29, 2017

mhdawson commented Nov 29, 2017 • edited Loading

refack commented Nov 30, 2017

mhdawson commented Nov 30, 2017

mhdawson commented Nov 30, 2017

mhdawson commented Nov 30, 2017

mhdawson commented Nov 30, 2017 • edited Loading

mhdawson commented Nov 30, 2017

mhdawson commented Nov 30, 2017

mhdawson commented Nov 30, 2017

mhdawson commented Nov 30, 2017

refack commented Nov 30, 2017

mhdawson commented Nov 30, 2017

refack commented Nov 30, 2017

mhdawson commented Nov 30, 2017

mhdawson commented Nov 30, 2017

mhdawson commented Nov 30, 2017 • edited Loading

mhdawson commented Nov 30, 2017

mhdawson commented Nov 30, 2017

mhdawson commented Nov 30, 2017 • edited Loading

mhdawson commented Nov 30, 2017

mhdawson commented Nov 30, 2017

mhdawson commented Nov 30, 2017

mhdawson commented Nov 30, 2017

refack commented Nov 30, 2017

mhdawson commented Nov 30, 2017

refack commented Dec 1, 2017

mhdawson commented Dec 1, 2017

mhdawson commented Dec 4, 2017

mhdawson commented Dec 5, 2017

seishun commented Jan 4, 2018

mhdawson commented Jan 4, 2018

mhdawson commented Feb 9, 2018

mhdawson commented May 11, 2018

mhdawson commented Nov 29, 2017 •

edited

Loading

mhdawson commented Nov 29, 2017 •

edited

Loading

mhdawson commented Nov 29, 2017 •

edited

Loading

mhdawson commented Nov 30, 2017 •

edited

Loading

mhdawson commented Nov 30, 2017 •

edited

Loading

mhdawson commented Nov 30, 2017 •

edited

Loading