src: trace_event: secondary storage for metadata #20900

ofrobots · 2018-05-22T22:48:05Z

Metadata trace-events should be held in secondary storage so that they
can be periodically reemitted. This change establishes the secondary
storage and ensures that events are reemitted on each flush.

/cc @nodejs/trace-events

CI:

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
commit message follows commit guidelines

ofrobots · 2018-06-28T00:06:17Z

I am unable to build on AIX to reproduce the timeout that only happens on AIX.

~/tmp/ofrobots/aix61-ppc64 $ gmake
gmake -C out BUILDTYPE=Release V=1
gmake[1]: Entering directory `/home/iojs/tmp/ofrobots/aix61-ppc64/out'
node.target.mk:13: warning: overriding recipe for target `/home/iojs/tmp/ofrobots/aix61-ppc64/out/Release/node.exp'
cctest.target.mk:13: warning: ignoring old recipe for target `/home/iojs/tmp/ofrobots/aix61-ppc64/out/Release/node.exp'
  touch 4874ac81486ce32779d2dbfe562b8f1b6ec63989.intermediate
  LD_LIBRARY_PATH=/home/iojs/tmp/ofrobots/aix61-ppc64/out/Release/lib.host:/home/iojs/tmp/ofrobots/aix61-ppc64/out/Release/lib.target:$LD_LIBRARY_PATH; export LD_LIBRARY_PATH; cd ../.; mkdir -p /home/iojs/tmp/ofrobots/aix61-ppc64/out/Release/obj/gen/src/node/inspector/protocol; python deps/v8/third_party/inspector_protocol/CodeGenerator.py --jinja_dir deps/v8/third_party/inspector_protocol/.. --output_base "/home/iojs/tmp/ofrobots/aix61-ppc64/out/Release/obj/gen/src/" --config "/home/iojs/tmp/ofrobots/aix61-ppc64/out/Release/obj/gen/node_protocol_config.json"
  touch 48324d268abcc345f409e144bccee591e70bdbe8.intermediate
  LD_LIBRARY_PATH=/home/iojs/tmp/ofrobots/aix61-ppc64/out/Release/lib.host:/home/iojs/tmp/ofrobots/aix61-ppc64/out/Release/lib.target:$LD_LIBRARY_PATH; export LD_LIBRARY_PATH; cd ../deps/v8/gypfiles; mkdir -p /home/iojs/tmp/ofrobots/aix61-ppc64/out/Release/obj/gen/src/inspector/protocol /home/iojs/tmp/ofrobots/aix61-ppc64/out/Release/obj/gen/include/inspector; python ../third_party/inspector_protocol/CodeGenerator.py --jinja_dir ../third_party --output_base "/home/iojs/tmp/ofrobots/aix61-ppc64/out/Release/obj/gen/src/inspector" --config ../src/inspector/inspector_protocol_config.json
  g++ -pthread -Wl,-bbigtoc -maix64  -o /home/iojs/tmp/ofrobots/aix61-ppc64/out/Release/genccode /home/iojs/tmp/ofrobots/aix61-ppc64/out/Release/obj.host/genccode/deps/icu-small/source/tools/genccode/genccode.o /home/iojs/tmp/ofrobots/aix61-ppc64/out/Release/obj.host/genccode/tools/icu/no-op.o /home/iojs/tmp/ofrobots/aix61-ppc64/out/Release/obj.host/tools/icu/libicutools.a
ld: 0711-317 ERROR: Undefined symbol: .std::type_info::operator==(std::type_info const&) const
ld: 0711-345 Use the -bloadmap or -bnoquiet option to obtain more information.
collect2: error: ld returned 8 exit status
gmake[1]: *** [/home/iojs/tmp/ofrobots/aix61-ppc64/out/Release/genccode] Error 1
rm 4874ac81486ce32779d2dbfe562b8f1b6ec63989.intermediate 48324d268abcc345f409e144bccee591e70bdbe8.intermediate
gmake[1]: Leaving directory `/home/iojs/tmp/ofrobots/aix61-ppc64/out'
gmake: *** [node] Error 2

This is after I setup the environment as per select-compiler.sh.

I'm stuck at this point. @mhdawson @richardlau is there a way to get help with this?

The test that times out has a large number of FS/IO operations in it. It is quite possible that the AIX machines genuinely not able able to finish this work in 120s. Either way, I cannot build on the machine, so I am unable to debug.

richardlau · 2018-06-28T00:24:42Z

cc @nodejs/platform-aix @jBarz

ofrobots · 2018-06-28T01:35:43Z

@refack pointed me to the job config on jenkins which fills in more blanks for me. I can build now.

richardlau · 2018-06-28T01:56:01Z

Out of interest, what was the missing piece?

ofrobots · 2018-06-28T03:34:45Z

Building requires LIBPATH to be unset and CC to be set to gcc.

…

On Wed, Jun 27, 2018, 6:57 PM Richard Lau ***@***.***> wrote: Out of interest, what was the missing piece? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#20900 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAE0qSNw7dc-nfHJrPuPEde4lqhyOgHSks5uBDf2gaJpZM4UJgl1> .

ofrobots · 2018-06-28T05:50:06Z

The aix timeout should be fixed by #21335. On AIX the race manifests as an infinite loop on this particular test on erroneous shutdown.

EDIT: minimal repro: ./node --trace-events-enabled -e 'throw 42'.

jasnell · 2018-08-12T06:43:21Z

Ping @ofrobots

ofrobots · 2018-08-13T17:45:24Z

Thanks for the ping. I plan to come back to this later this week.

jasnell · 2018-09-10T18:14:35Z

Ping again :-)

ofrobots · 2018-09-28T22:10:00Z

I am planning to come back to this once #22865 is addressed, i.e. after libuv/libuv#2003, #23110, and #22938 land.

ofrobots · 2018-10-12T13:10:12Z

This is updated now given the cascade of blockers are now resolved. New CI: https://ci.nodejs.org/job/node-test-pull-request/17751/

Linter re-run: https://ci.nodejs.org/job/node-test-linter/22720/

Metadata trace-events should be held in secondary storage so that they can be periodically reemitted. This change establishes the secondary storage and ensures that events are reemitted on each flush.

ofrobots · 2018-10-12T15:14:18Z

CI is green. @jasnell assuming your LGTM from 5 months ago (!) still holds, I am going to land this later today. The rebase did not change this PR in substance.

addaleax · 2018-10-12T21:22:00Z

Landed in f5986a4

Metadata trace-events should be held in secondary storage so that they can be periodically reemitted. This change establishes the secondary storage and ensures that events are reemitted on each flush. PR-URL: #20900 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net>

addaleax · 2018-10-12T22:32:29Z

Should this be backported to v10.x-staging? If yes please follow the guide and raise a backport PR, if not let me know or add the dont-land-on label.

ofrobots · 2018-10-16T19:59:18Z

backport on #23700

Metadata trace-events should be held in secondary storage so that they can be periodically reemitted. This change establishes the secondary storage and ensures that events are reemitted on each flush. PR-URL: #20900 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net>

Metadata trace-events should be held in secondary storage so that they can be periodically reemitted. This change establishes the secondary storage and ensures that events are reemitted on each flush. PR-URL: nodejs#20900 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Backport-PR-URL: nodejs#23700

Metadata trace-events should be held in secondary storage so that they can be periodically reemitted. This change establishes the secondary storage and ensures that events are reemitted on each flush. Backport-PR-URL: #23700 PR-URL: #20900 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Backport-PR-URL: #23700

ofrobots requested a review from jasnell May 22, 2018 22:48

nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. labels May 22, 2018

jasnell approved these changes May 23, 2018

View reviewed changes

BridgeAR added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label May 24, 2018

addaleax removed the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Jun 7, 2018

ofrobots added the trace_events Issues and PRs related to V8, Node.js core, and userspace code trace events. label Jun 26, 2018

ofrobots force-pushed the metadata-buffer branch from 7392e98 to 210a0c4 Compare June 26, 2018 23:43

ofrobots added the blocked PRs that are blocked by other issues or PRs. label Jun 28, 2018

ofrobots removed the blocked PRs that are blocked by other issues or PRs. label Aug 13, 2018

jasnell added the stalled Issues and PRs that are stalled. label Sep 10, 2018

ofrobots force-pushed the metadata-buffer branch from 210a0c4 to 7bcd8fa Compare October 11, 2018 23:09

ofrobots removed the stalled Issues and PRs that are stalled. label Oct 11, 2018

ofrobots force-pushed the metadata-buffer branch from 7bcd8fa to f51983d Compare October 12, 2018 13:08

src: trace_event: secondary storage for metadata

2bde938

Metadata trace-events should be held in secondary storage so that they can be periodically reemitted. This change establishes the secondary storage and ensures that events are reemitted on each flush.

ofrobots force-pushed the metadata-buffer branch from f51983d to 2bde938 Compare October 12, 2018 13:26

ofrobots added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Oct 12, 2018

addaleax approved these changes Oct 12, 2018

View reviewed changes

addaleax closed this Oct 12, 2018

addaleax added the backport-requested-v10.x label Oct 12, 2018

ofrobots deleted the metadata-buffer branch October 13, 2018 05:39

ofrobots mentioned this pull request Oct 16, 2018

[v10.x backport] src: trace_event: secondary storage for metadata #23700

Closed

4 tasks

MylesBorins added backported-to-v10.x and removed backport-requested-v10.x labels Nov 12, 2018

codebytere mentioned this pull request Nov 27, 2018

v10.13.1 proposal #24675

Closed

codebytere mentioned this pull request Nov 29, 2018

v10.14.2 proposal #24727

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

src: trace_event: secondary storage for metadata #20900

src: trace_event: secondary storage for metadata #20900

ofrobots commented May 22, 2018 •

edited

Loading

ofrobots commented Jun 28, 2018

richardlau commented Jun 28, 2018

ofrobots commented Jun 28, 2018

richardlau commented Jun 28, 2018

ofrobots commented Jun 28, 2018 via email •

edited

Loading

ofrobots commented Jun 28, 2018 •

edited

Loading

jasnell commented Aug 12, 2018

ofrobots commented Aug 13, 2018

jasnell commented Sep 10, 2018

ofrobots commented Sep 28, 2018

ofrobots commented Oct 12, 2018 •

edited

Loading

ofrobots commented Oct 12, 2018

addaleax commented Oct 12, 2018

addaleax commented Oct 12, 2018

ofrobots commented Oct 16, 2018

src: trace_event: secondary storage for metadata #20900

src: trace_event: secondary storage for metadata #20900

Conversation

ofrobots commented May 22, 2018 • edited Loading

Checklist

ofrobots commented Jun 28, 2018

richardlau commented Jun 28, 2018

ofrobots commented Jun 28, 2018

richardlau commented Jun 28, 2018

ofrobots commented Jun 28, 2018 via email • edited Loading

ofrobots commented Jun 28, 2018 • edited Loading

jasnell commented Aug 12, 2018

ofrobots commented Aug 13, 2018

jasnell commented Sep 10, 2018

ofrobots commented Sep 28, 2018

ofrobots commented Oct 12, 2018 • edited Loading

ofrobots commented Oct 12, 2018

addaleax commented Oct 12, 2018

addaleax commented Oct 12, 2018

ofrobots commented Oct 16, 2018

ofrobots commented May 22, 2018 •

edited

Loading

ofrobots commented Jun 28, 2018 via email •

edited

Loading

ofrobots commented Jun 28, 2018 •

edited

Loading

ofrobots commented Oct 12, 2018 •

edited

Loading