Pipeline to run Exploratory fuzzing tool to catch codegen issues #59489
Conversation
Add pipeline to run exploratory tools and return back the result
|
Tagging subscribers to this area: @hoyosjs Issue DetailsAdd an "internal" pipeline to run exploratory tools and return back the repro files.
|
kunalspathak
changed the title
|
@dotnet/jit-contrib |
kunalspathak
changed the title
There was a problem hiding this comment.
Some comments/questions
There's a lot of code here, and not much in the way of "overall" documentation besides just reading all the code. I worry about that, in terms of maintainability. However, I presume this is just a first step, and you want to get something in and then work on it as we gain more experience with it, and add additional exploratory tools.
I have included |
|
|
||
| schedules: | ||
| - cron: "0 6 * * *" | ||
| displayName: Daily at 11:30 PM (UTC-8:00) |
There was a problem hiding this comment.
It seems like daily is too often, and weekly might make more sense. But I'm ok with this; we can change our mind about scheduling later.
There was a problem hiding this comment.
Ok, I will schedule it for weekly for now.
| @@ -12,10 +12,10 @@ steps: | |||
| inputs: | |||
| buildType: specific | |||
| project: 'internal' # 'internal' or 'public' | |||
| pipeline: 'superpmi-replay' | |||
There was a problem hiding this comment.
Is this file only used for testing?
There was a problem hiding this comment.
Yes, this is used to skip building coreclr/libraries and download these from previous runs. This helps in fast testing during pipeline development.
| antigen_bin_directory = path.join(tool_code_directory, "bin", "Release", "net5.0") | ||
|
|
||
| # build the tool | ||
| with ChangeDir(tool_code_directory): | ||
| dotnet_cmd = os.path.join(source_directory, "dotnet.cmd") | ||
| if not is_windows: | ||
| dotnet_cmd = os.path.join(source_directory, "dotnet.sh") | ||
| run_command([dotnet_cmd, "publish", "-c", "Release", "--self-contained", "-r", run_configuration, "-o", antigen_bin_directory], _exit_on_fail=True) | ||
|
|
||
| if not os.path.exists(path.join(antigen_bin_directory, "Antigen.dll")): | ||
| raise FileNotFoundError("Antigen.dll not present at {}".format(antigen_bin_directory)) | ||
|
|
||
| # copy antigen tool | ||
| print('Copying {} -> {}'.format(antigen_bin_directory, antigen_dst_directory)) | ||
| copy_directory(antigen_bin_directory, antigen_dst_directory, match_func=acceptable_copy) |
There was a problem hiding this comment.
I wonder if this should all be encapsulated in a script in the antigen source directory, so this script doesn't need to know how to build the tool. Something like <tool_code_directory>build.cmd <dotnet_tool/script_to_use> <antigen_dst_directory>.
There was a problem hiding this comment.
Sure, I will fix it in antigen and make related change here as a follow-up PR.
ghost
locked as resolved and limited conversation to collaborators
Add an "internal" pipeline to run exploratory tools and return back the repro files.