Flaky Linux tests no longer flaky?

[mscdex]

Currently there are 3 tests marked as flaky on Linux:

• test-http-client-timeout-event
• test-child-process-buffering
• test-child-process-exit-code

They were all added in the initial flaky test list back in August with #2424. I've been looking back through the test history for the Linux CI nodes and I cannot find those tests ever failing in recent months. Perhaps they can be removed from the list now?

[jbergstroem]

Could you stress test them?

[mscdex]

I couldn't find out which Linux nodes these tests were supposedly flaky on, so I'm just picking a few.

First up: test-http-client-timeout-event:

• centos5-32: https://ci.nodejs.org/job/node-stress-single-test/nodes=centos5-32/239/
• ubuntu1504-64: https://ci.nodejs.org/job/node-stress-single-test/nodes=ubuntu1504-64/237/
• armv7-wheezy: https://ci.nodejs.org/job/node-stress-single-test/nodes=armv7-wheezy/240
• pi1-raspbian-wheezy (I'll never do that again until Cross-compile for (ARM) stress tests build#277 is resolved): https://ci.nodejs.org/job/node-stress-single-test/nodes=pi1-raspbian-wheezy/238

[Trott]

test-http-client-timeout-event was marked flaky because of failures on pi1-raspbian-wheezy. #2555

test-child-process-buffering was marked flaky because of failures on armv7-wheezy.
#2627 (If it's not flaky anymore, that issue can be closed, of course.)

Not sure what OS test-http-client-timeout-event test-child-process-exit-code was failing on so the scatter-shot method might need to be used there too. Although I don't think I've seen a yellow build in a long, long time on Linux, so it's probably not flaky now....

[mscdex]

@Trott With regard to your last remark, did you mean test-child-process-exit-code instead of test-http-client-timeout-event?

[Trott]

@mscdex Yes, that's exactly right. It was a cut and paste error. Sorry about that. I'll edit it...

[mscdex]

Ok, here's a stress test for armv7-wheezy and test-child-process-buffering: https://ci.nodejs.org/job/node-stress-single-test/247/nodes=armv7-wheezy/

Here's a stress test for pi1-raspbian-wheezy and test-http-client-timeout-event (I accidentally aborted the one I had going for 3 hours... argh): https://ci.nodejs.org/job/node-stress-single-test/248/nodes=pi1-raspbian-wheezy/

[jbergstroem]

Looking good so far!

[Trott]

I've been looking back through the test history for the Linux CI nodes and I cannot find those tests ever failing in recent months.

I'd be good with removing these three tests at this point. The only one that hasn't been run through the stress test is test-child-process-exit-code. Like you, I can't find any examples of it failing recently. If we remove it from the flaky list and it turns out that it is in fact still flaky (which seems unlikely to happen), we can put it back in as flaky.

[mscdex]

@Trott Sounds good to me.

[Trott]

Created PR #4519 for this. @mscdex If you already have a PR for it that I'm just not seeing or wanted to put it in yourself for whatever reason (better commit message, your own personal tracking of xyz, whatever), just go ahead and close mine.