Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Host packages should not use runtime.json restore infrastructure #49137

Open
ViktorHofer opened this issue Mar 4, 2021 · 16 comments
Open

Host packages should not use runtime.json restore infrastructure #49137

ViktorHofer opened this issue Mar 4, 2021 · 16 comments
Milestone

Comments

@ViktorHofer
Copy link
Member

ViktorHofer commented Mar 4, 2021

The host packages still use runtime.json which was the mechanism to define RID based package dependencies in the .NET Core 2.x era. That feature was never publicly exposed by NuGet and is discussed to be removed in NET 6. For more details see NuGet/Home#5862.

There are alternatives to runtime.json, i.e.:

  • KnownFrameworkPackage with known RIDs
  • A single package with multiple RID based folders (bigger package, but easier to maintain)
  • Dead ending the package if it isn't supposed to be used anymore.

dotnet/sdk still consumes the meta package via the runtime.json mechanism which should be considered when switching to a different restore mechanism.

Host packages:

cc @agocke @vitek-karas @ericstj

@ViktorHofer ViktorHofer added this to the 6.0.0 milestone Mar 4, 2021
@dotnet-issue-labeler dotnet-issue-labeler bot added the untriaged New issue has not been triaged by the area owner label Mar 4, 2021
@ghost
Copy link

ghost commented Mar 4, 2021

Tagging subscribers to this area: @vitek-karas, @agocke
See info in area-owners.md if you want to be subscribed.

Issue Details

The host packages still use runtime.json which was the mechanism to define RID based package dependencies in the .NET Core 2.x era. That feature was never publicly exposed by NuGet and is discussed to be removed in NET 6. For more details see NuGet/Home#5862.

There are alternatives to runtime.json, i.e.:

  • KnownFrameworkPackage with known RIDs
  • A single package with multiple RID based folders (bigger package, but easier to generate and maintain)
  • Dead ending the package if it isn't supposed to be used anymore.

cc @agocke @vitek-karas @ericstj

Author: ViktorHofer
Assignees: -
Labels:

area-Host

Milestone: 6.0.0

@jkoritzinsky
Copy link
Member

The single-package option will require us to add another “join” job like the CrossOS DAC job to get all the artifacts in one place.

@ericstj
Copy link
Member

ericstj commented Mar 4, 2021

IMHO we should just deprecate the runtime.json and make the only method of consumption be SDK and have it work the same way as runtime packs.

dotnet/sdk still consumes the meta package via the runtime.json mechanism

Is this just the internal repo transport or does it show up in the actual product?

@ViktorHofer
Copy link
Member Author

Is this just the internal repo transport or does it show up in the actual product?

It's not exposed in the product, it's just on the consumption side: https://github.com/dotnet/sdk/blob/0d50c13812b626999aef1853bf23295f38287807/src/Resolvers/Microsoft.DotNet.MSBuildSdkResolver/Microsoft.DotNet.MSBuildSdkResolver.csproj#L58. Some tests as well in the sdk repo.

@ViktorHofer
Copy link
Member Author

IMHO we should just deprecate the runtime.json and make the only method of consumption be SDK and have it work the same way as runtime packs.

Can you please share some more information on the work necessary for that?

@ericstj
Copy link
Member

ericstj commented Mar 4, 2021

This stuff: https://github.com/dotnet/installer/blob/944ec2eb293e807d5235439ca0186d9725856beb/src/redist/targets/GenerateBundledVersions.targets#L100-L120
Then the SDK consuming that and generating PackageDownloads, then manually probing the downloaded directory to raise the assets.

Essentially what you said above: KnownFrameworkPackage with known RIDs.

For any internal consumption we might need to see what the usage is to find a suitable replacement. Might be leveraging those targets in the SDK which compute the right RID-specific package. Might be just referencing everything and let nuget choose. The host isn't so big, so referencing everything isn't such a huge burden.

A single package with multiple RID based folders (bigger package, but easier to maintain)

Another alternative to this is meta-package with normal references that apply exclusively (see runtime.native.System.IO.Ports).

@ViktorHofer
Copy link
Member Author

Another alternative to this is meta-package with normal references that apply exclusively (see runtime.native.System.IO.Ports).

I prefer any option that doesn't require a join job (as Jeremy commented above #49137 (comment)).

Essentially what you said above: KnownFrameworkPackage with known RIDs.

Ok that needs closer inspection. If we add Known items to the defaultversions generated file, doesn't that mean, that the packages will always be downloaded? Anyway, someone will need to take a look and make a switch.

@ViktorHofer
Copy link
Member Author

When doing that it would also be good to apply two code clean-ups:

  • Change naming from runtime.* to <PackageId>.Runtime.<RID> as other runtime framework packages
  • Move off from the four pkgprojs to .proj ones and use the NuGet Pack Task instead.

@ericstj
Copy link
Member

ericstj commented Mar 4, 2021

I prefer any option that doesn't require a join job

Not suggesting a join. Suggesting meta-package that "knows" the id's and versions of child packages. Essentially same as we do today but put the dependencies in nuspec instead of runtime.json.

@omajid
Copy link
Member

omajid commented Mar 18, 2021

cc @tmds

@jeffschwMSFT jeffschwMSFT removed the untriaged New issue has not been triaged by the area owner label Jul 6, 2021
@agocke agocke modified the milestones: 6.0.0, 7.0.0 Aug 9, 2021
@glopesdev
Copy link

Essentially same as we do today but put the dependencies in nuspec instead of runtime.json.

@ericstj Is there a way to specify RID-specific dependencies in nuspec, to get the same download optimizations that are possible with runtime.json?

@ericstj
Copy link
Member

ericstj commented May 4, 2022

No, it's not possible. NuGet/Home#1660 dotnet/sdk#4552

kzu added a commit to devlooped/chromium that referenced this issue Jun 3, 2022
It is unfortunately not possible to conditionally reference native assets depending on the installed tool's runtime platform. This means we need to assume the dependency will be restored at the project level and look it up via the project.assets.json, rather than the runtime deps.

See
- How a tool project is installed/restored: https://github.com/dotnet/sdk/blob/main/src/Cli/dotnet/ToolPackage/ToolPackageInstaller.cs
- Native files not being automatically added to the runtime deps.json: dotnet/sdk#11373
- How to do that via custom targets, but this wouldn't be possible on the target machine when restoring the temp project created for global tools: https://github.com/ericstj/sample-code/blob/nativeLibSample/addNative/addNative.csproj
- How runtime.json might work but it's going away: dotnet/runtime#11404
- Really going away: dotnet/runtime#49137
- How it's still not solved for .net6: NuGet/Home#5862
kzu added a commit to devlooped/chromium that referenced this issue Jun 3, 2022
It is unfortunately not possible to conditionally reference native assets depending on the installed tool's runtime platform. This means we need to assume the dependency will be restored at the project level and look it up via the project.assets.json, rather than the runtime deps.

See
- How a tool project is installed/restored: https://github.com/dotnet/sdk/blob/main/src/Cli/dotnet/ToolPackage/ToolPackageInstaller.cs
- Native files not being automatically added to the runtime deps.json: dotnet/sdk#11373
- How to do that via custom targets, but this wouldn't be possible on the target machine when restoring the temp project created for global tools: https://github.com/ericstj/sample-code/blob/nativeLibSample/addNative/addNative.csproj
- How runtime.json might work but it's going away: dotnet/runtime#11404
- Really going away: dotnet/runtime#49137
- How it's still not solved for .net6: NuGet/Home#5862
@agocke agocke modified the milestones: 7.0.0, 8.0.0 Jul 26, 2022
@elinor-fung elinor-fung modified the milestones: 8.0.0, 9.0.0 Aug 15, 2023
@ViktorHofer
Copy link
Member Author

cc @elinor-fung, @agocke

With f76045c, the remaining host package that still uses the runtime.json infrastructure is Microsoft.NETCore.DotNetAppHost.

Related: #49233, #75966

Unfortunately another package got added into the installer/pkg umbrella which depends on this infra: https://github.com/dotnet/runtime/tree/main/src/installer/pkg/projects/Microsoft.DotNet.ILCompiler.

After both projects got converted to NuGet Pack task based ones, we will finally be able to delete all the ancient infrastructure around supporting runtime.json based packages under src/installer.

@agocke
Copy link
Member

agocke commented Jul 31, 2024

Is this done? I think the ILCompiler runtimejson is gone, I don't see one for DotNetAppHost either.

@jkotas
Copy link
Member

jkotas commented Aug 1, 2024

I still see runtime.json in https://nuget.info/packages/Microsoft.DotNet.ILCompiler/9.0.0-preview.6.24327.7. What's the package you have looked at?

This was recently discussed in #102908 . We have removed runtime.json from bundled source built ILCompiler packages, but kept it for regular ILCompiler package.

@agocke
Copy link
Member

agocke commented Aug 1, 2024

I was looking at the source code and misread the change to be the other direction (we only include runtime.json in source build).

OK, I'll keep this around.

@agocke agocke modified the milestones: 9.0.0, Future Aug 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: No status
Development

No branches or pull requests

9 participants