Add job to test configure

[gengjiawen]

Related: nodejs/node#29414.
Just to make sure configure works as expected, no need to build.
Such as

./configure --ninja

cc @targos

[rvagg]

we could probably pull that off with a sharedlibs docker container. What would we be testing for? exit code or is there something else we can use to confirm success? if you can express it in bash that would be handy too.

[rvagg]

^ fwiw we do a bit of TAP output with bash in some of our tests that confirm various build properties, like when we compile with a shared openssl library we run process.versions and confirm that we got what we compiled and then echo some TAP output to stdout which is picked up Jenkins nicely.

[targos]

I'm not sure we can really do it. Sure, we can test for --ninja, but why not other flag combinations? It's not possible to cover everything and people using specific combinations usually find problems soon enough.

[gengjiawen]

I'm not sure we can really do it. Sure, we can test for --ninja, but why not other flag combinations? It's not possible to cover everything and people using specific combinations usually find problems soon enough.

Yeap, the key is to test frequently used script. Should we do a poll in Node.js collaborators ?

[sam-github]

So, I'm sympathetic to the unpleasant surprise that --ninja was broken, but it's a developer flag, only node.js devs use it, so one of us will notice its broken PDQ, and fix it. No big deal. The ICU, or no-openssl builds are done more by downstream, I think they'd be more useful to test. But what would be nice to have, and what someone has time to do, are not the same thing!

@gengjiawen Are you offering to implement this or hoping to convince someone else to do it? This issue seems a duplicate of nodejs/node#29415, which was hopefully seen by build wg members (I certainly saw it, as well as--ninja breaking, I always use ninja), but if new features need adding to the build, it would help to have more people to work on them!

[rvagg]

I don't mind adding a new job just to test config flags if someone wants to put in the effort to write some bash that will run them and assert something, printing tap. It's not a bad thing to test, just low priority so consider this issue a TODO for someone with the time and interest eh?

[sam-github]

Added help wanted. Also, a suggestion that if someone does do that, that it be PRed into nodejs/node as a test target (so it can be developed and trouble-shooted outside of CI).

[patrickhousley]

If I am reading the code correctly, it looks like there is already a test target tooltests. Changing that to @$(PYTHON) -m unittest discover -s ./test/tools and renaming the exists test to use underscores will result in being able to run all python tests in the test/tools directory in one target. I can PR this in if that works for you @sam-github

[sam-github]

@patrickhousley That tests the stuff in tools/ or in test/tools.... how is that related to #1914 (comment) ? Its ./configure --ninja and the like that were requested to be tested.

[patrickhousley]

Sorry, just getting started with the project here. The tests in test/tools test tools used to build the project so it seemed like the best place to put new test cases for configure or any of the other build tools/scripts.

[patrickhousley]

Am I over thinking this maybe? Should the test be as simple as running ./configure --ninja and testing the exit code? Would there be any concern with configure.py actually writing to the workspace?

[gengjiawen]

Am I over thinking this maybe? Should the test be as simple as running ./configure --ninja and testing the exit code? Would there be any concern with configure.py actually writing to the workspace?

I think just test ./configure --ninja should be okay.

[github-actions]

This issue is stale because it has been open many days with no activity. It will be closed soon unless the stale label is removed or a comment is made.

[gengjiawen]

close in favor of nodejs/node#29415.