Move linker repo to runtime

[tlakollo]

Move linker repo to runtime repo

This issue tracks progress and discussion for moving the dotnet/linker repo into dotnet/runtime repo.

Progress

• Selected the location of the repository as runtime\src\tools\linker
• Ability to transfer Linker history to runtime using the consolidation script
• Add cecil as an arcade package in order to avoid having a submodule
  • Rename package to DotNet.Cecil
• Be able to run dotnet format to apply runtime formatting to linker repo
• Create subsets (tools.linker & tools.linkertests) in order to be able to run and test the repo
• Run in CI
• Rename linker to illink
• Add additional label
• Build a package from the new location or use live bits

Motivation

• Code Sharing. At this moment NativeAOT and linker share a lot of code via a straight copy. NativeAOT consumes the ILLink.Shared project which is a shared project between linker, trimming analyzer and NativeAOT. It also shares the same testing infrastructure and even shares additional files that are not included in the shared project since they are only shared between linker and NativeAOT (not for the trimming analyzer). The plan is that once we move the linker repository to runtime we can take advantage of having only one ILLink.Shared project, only one testing infrastructure, and we can improve code sharing between linker and NativeAOT, even having the same formats and analyzers applied to the codebases improve the current state of code sharing.
• Versioning history. Backward compatibility of tools like linker and NativeAOT is an ongoing issue (see PublishTrimmed should use a linker that matches the TargetFramework #3029) having both repositories in the runtime simplifies the solution for both of the tools.

Select the location for the linker project

So far there have some locations that seem relevant

• src\coreclr\tools\ folder, although it would be closer to the AOT tools linker is not strictly a part of the coreclr
• Create a new folder under src\tools for tools that are agnostic to the clr. Likely the best option.
• Maybe somewhere in the mono partition?

Avoiding having a cecil submodule

Part of the linker workflow is to have a .gitmodules file that specifies a mono/cecil module, for runtime this won't fit the runtime needs. So we need to maybe ship cecil packages from mono/cecil and consume them in the runtime to later be used by linker

Linker style of code format

Currently, linker repo uses mono style of code. This makes already codesharing with NativeAOT more difficult since we cannot share code as is. Linker already transition from the mono space to the dotnet space, which leaves in the air if it should still use the mono guidelines. Once moving to the runtime repo and giving that we want to share code, the most likely solution is to just use runtime format for that repo. Note: mono directories under runtime still use the mono style of code guidelines.

Legs for execution

Need a definition on when should ci build linker and run linker tests. Most likely will remain as a subset just like currently clr.tools and clr.toolstests, and run only if changes in the linker repo are made. For source build, building and testing the linker repo seems inevitable unless we create a different leg that only runs linker.

Add an additional label

Add an additional label to runtime to identify linker issues

Keep a history of the linker changes

There are some scripts that were used to consolidate different repositories into runtime, although they are tailored to work with certain repositories we could configure it to keep the history from linker (see https://github.com/dotnet/consolidation)

Migration of github issues

Previous runtime consolidation work had the help of GitHub to migrate all issues from the different repos to the new dotnet/runtime repo, and then lock down creation of additional issues, such that the repos will no longer be used for issue management. They also inhibit the creation of PRs to the main branch, effectively making old repos an archive for read-only review of history. Plan is to do the same for linker

Copy of dotnet/runtime#75278

[marek-safar]

/cc @mrvoorhe

[mrvoorhe]

Thanks for the heads up @marek-safar. This will create additional work for us, but we'll figure it out.

[agocke]

The first tier of this work is largely done. Closing as complete.

Anything else is follow-up.