[tests] enable One .NET tests for F# #5040

jonathanpeppers · 2020-08-21T16:15:46Z

Context: xamarin/Xamarin.Android.FSharp.ResourceProvider#6

Xamarin.Android.FSharp.ResourceProvider 1.0.1 is now built for
netstandard2.0 and published to an internal feed.

We should be able to enable F# tests under a dotnet context now.

The _RemoveLegacyDesigner MSBuild target was running, when it
shouldn't be for F# projects and adding a Resource.designer.cs
@(Compile) item. I needed to add a $(Language) check.

One workaround is for:

dotnet/sdk#12954

We need to set $(ProduceReferenceAssembly) to false for .NET 5 F#
projects at the moment.

pjcollins · 2020-08-21T18:34:08Z

I think we may be able to revisit these two tests now as well:
https://github.com/xamarin/xamarin-android/blob/master/src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/AndroidUpdateResourcesTest.cs#L548
https://github.com/xamarin/xamarin-android/blob/master/src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/AndroidUpdateResourcesTest.cs#L583

I just disabled both ignores and ran them on Windows out of curiosity (non dotnet versions) and they worked, so we may be able to remove the Windows skip here as well?

jonathanpeppers · 2020-08-21T19:12:17Z

@pjcollins locally I'm getting this for F# .NET 5 Release builds now:

Errors
    ILLink error IL1012: IL Linker has encountered an unexpected error. Please report the issue at https://github.com/mono/linker/issues [C:\src\xamarin-android\bin\TestDebug\temp\CheckResourceDesignerIsCreatedTrueFSharp\UnnamedProject.fsproj]
Fatal error in IL Linker
Unhandled exception. System.ArgumentOutOfRangeException: Length cannot be less than zero. (Parameter 'length')
    at System.String.Substring(Int32 startIndex, Int32 length)
    at Mono.Linker.TypeReferenceExtensions.GetDisplayNameWithoutNamespace(TypeReference type)
    at Mono.Linker.TypeReferenceExtensions.GetDisplayName(TypeReference type)
    at Mono.Linker.Dataflow.DiagnosticUtilities.GetGenericParameterDescriptionForErrorMessage(GenericParameter genericParameter)
    at Mono.Linker.Dataflow.DiagnosticUtilities.GetMetadataTokenDescriptionForErrorMessage(IMetadataTokenProvider targetContext)
    at Mono.Linker.Dataflow.ReflectionMethodBodyScanner.GetValueDescriptionForErrorMessage(ValueNode value)
    at Mono.Linker.Dataflow.ReflectionMethodBodyScanner.RequireDynamicallyAccessedMembers(ReflectionPatternContext& reflectionContext, DynamicallyAccessedMemberTypes requiredMemberTypes, ValueNode value, IMetadataTokenProvider targetContext)
    at Mono.Linker.Dataflow.ReflectionMethodBodyScanner.ProcessGenericArgumentDataFlow(GenericParameter genericParameter, TypeReference genericArgument, IMemberDefinition source)
    at Mono.Linker.Steps.MarkStep.MarkGenericArgumentConstructors(IGenericInstance instance, IMemberDefinition sourceLocationMember)
    at Mono.Linker.Steps.MarkStep.MarkGenericArguments(IGenericInstance instance, IMemberDefinition sourceLocationMember)
    at Mono.Linker.Steps.MarkStep.GetOriginalType(TypeReference type, DependencyInfo reason, IMemberDefinition sourceLocationMember)
    at Mono.Linker.Steps.MarkStep.MarkType(TypeReference reference, DependencyInfo reason, IMemberDefinition sourceLocationMember)
    at Mono.Linker.Steps.MarkStep.MarkField(FieldDefinition field, DependencyInfo& reason)
    at Mono.Linker.Steps.MarkStep.MarkEntireType(TypeDefinition type, Boolean includeBaseTypes, DependencyInfo& reason, IMemberDefinition sourceLocationMember)
    at Mono.Linker.Steps.MarkStep.MarkEntireType(TypeDefinition type, Boolean includeBaseTypes, DependencyInfo& reason, IMemberDefinition sourceLocationMember)
    at Mono.Linker.Steps.MarkStep.MarkEntireAssembly(AssemblyDefinition assembly)
    at Mono.Linker.Steps.MarkStep.InitializeAssembly(AssemblyDefinition assembly)
    at Mono.Linker.Steps.MarkStep.Initialize()
    at Mono.Linker.Steps.MarkStep.Process(LinkContext context)
    at Mono.Linker.Pipeline.ProcessStep(LinkContext context, IStep step)
    at Mono.Linker.Pipeline.Process(LinkContext context)
    at Mono.Linker.Driver.Run(ILogger customLogger)
    at Mono.Linker.Driver.Main(String[] args)
The command exited with code -532462766.

We'll see if this happens on CI.

jonathanpeppers · 2020-08-21T21:58:51Z

Yeah, the linker is broken for F#...

Something happened here: dotnet/linker@095f30a

It seems like it expects generic types to always have a ` character, and that might not be the case in F#.

Context: dotnet#5040 In trying to enable our F# MSBuild tests, one fails with: GenerateCompressedAssembliesNativeSourceFiles Xamarin.Android.Common.targets(1933,3): error XAGCANSF7004: System.ArgumentException: An item with the same key has already been added. Key: [FSharp.Core.resources.dll, Xamarin.Android.Tasks.CompressedAssemblyInfo] at System.Collections.Generic.TreeSet`1.AddIfNotPresent(T item) at System.Collections.Generic.SortedDictionary`2.Add(TKey key, TValue value) at Xamarin.Android.Tasks.GenerateCompressedAssembliesNativeSourceFiles.GenerateCompressedAssemblySources() at Xamarin.Android.Tasks.GenerateCompressedAssembliesNativeSourceFiles.RunTask() at Xamarin.Android.Tasks.AndroidTask.Execute() Looking at some of the item groups earlier: C:\src\xamarin-android\packages\xamarin.android.fsharp.resourceprovider\1.0.1\lib\monoandroid81\Xamarin.Android.FSharp.ResourceProvider.Runtime.dll ... DestinationSubPath = Xamarin.Android.FSharp.ResourceProvider.Runtime.dll ... C:\src\xamarin-android\packages\fsharp.core\4.7.2\lib\netstandard2.0\cs\FSharp.Core.resources.dll ... DestinationSubDirectory = cs\ DestinationSubPath = cs\FSharp.Core.resources.dll ... C:\src\xamarin-android\packages\fsharp.core\4.7.2\lib\netstandard2.0\de\FSharp.Core.resources.dll ... DestinationSubDirectory = de\ DestinationSubPath = de\FSharp.Core.resources.dll ... Two instances of `FSharp.Core.resources.dll` is causing the above exception. Should we be using the `%(DestinationSubDirectory)` and `%(DestinationSubPath)` item metadata? Currently, we have some behavior for architecture-specific .NET assemblies: * `%(AbiDirectory)` is set for x86, armeabi-v7a, etc. * `%(IntermediateLinkerOutput)` is set to a full path taking `%(AbiDirectory)` into account. To clean things up here, we should just use `%(DestinationSubDirectory)` coming from dotnet/sdk and `%(DestinationSubPath)` and remove the item data that we invented. Other changes: * Usage of `%(AbiDirectory)` can use `%(DestinationSubDirectory)` or `%(DestinationSubPath)` where appropriate. * Any usage of `%(IntermediateLinkerOutput)` can use `$(MonoAndroidIntermediateAssemblyDir)%(DestinationSubPath)` instead. * The `<ResolveAssemblies/>` (legacy) and `<ProcessAssemblies/>` MSBuild tasks no longer need the `IntermediateAssemblyDirectory` property.

Context: dotnet#5040 In trying to enable our F# MSBuild tests, one fails with: GenerateCompressedAssembliesNativeSourceFiles Xamarin.Android.Common.targets(1933,3): error XAGCANSF7004: System.ArgumentException: An item with the same key has already been added. Key: [FSharp.Core.resources.dll, Xamarin.Android.Tasks.CompressedAssemblyInfo] at System.Collections.Generic.TreeSet`1.AddIfNotPresent(T item) at System.Collections.Generic.SortedDictionary`2.Add(TKey key, TValue value) at Xamarin.Android.Tasks.GenerateCompressedAssembliesNativeSourceFiles.GenerateCompressedAssemblySources() at Xamarin.Android.Tasks.GenerateCompressedAssembliesNativeSourceFiles.RunTask() at Xamarin.Android.Tasks.AndroidTask.Execute() Looking at some of the item groups earlier: C:\src\xamarin-android\packages\xamarin.android.fsharp.resourceprovider\1.0.1\lib\monoandroid81\Xamarin.Android.FSharp.ResourceProvider.Runtime.dll ... DestinationSubPath = Xamarin.Android.FSharp.ResourceProvider.Runtime.dll ... C:\src\xamarin-android\packages\fsharp.core\4.7.2\lib\netstandard2.0\cs\FSharp.Core.resources.dll ... DestinationSubDirectory = cs\ DestinationSubPath = cs\FSharp.Core.resources.dll ... C:\src\xamarin-android\packages\fsharp.core\4.7.2\lib\netstandard2.0\de\FSharp.Core.resources.dll ... DestinationSubDirectory = de\ DestinationSubPath = de\FSharp.Core.resources.dll ... Two instances of `FSharp.Core.resources.dll` is causing the above exception. Should we be using the `%(DestinationSubDirectory)` and `%(DestinationSubPath)` item metadata? Currently, we have some behavior for architecture-specific .NET assemblies: * `%(AbiDirectory)` is set for x86, armeabi-v7a, etc. * `%(IntermediateLinkerOutput)` is set to a full path taking `%(AbiDirectory)` into account. To clean things up here, we should just use `%(DestinationSubDirectory)` coming from dotnet/sdk and `%(DestinationSubPath)` and remove the item data that we invented. Other changes: * Usage of `%(AbiDirectory)` can use `%(DestinationSubDirectory)` or `%(DestinationSubPath)` where appropriate. * Any usage of `%(IntermediateLinkerOutput)` can use `$(MonoAndroidIntermediateAssemblyDir)%(DestinationSubPath)` instead. * The `<ResolveAssemblies/>` (legacy) and `<ProcessAssemblies/>` MSBuild tasks no longer need the `IntermediateAssemblyDirectory` property. * A new check was needed in `<ProcessAssemblies/>` to make sure we don't append `%(DestinationSubDirectory)` for the same assembly multiple times.

Context: #5040 In trying to enable our F# MSBuild tests, one fails with: GenerateCompressedAssembliesNativeSourceFiles Xamarin.Android.Common.targets(1933,3): error XAGCANSF7004: System.ArgumentException: An item with the same key has already been added. Key: [FSharp.Core.resources.dll, Xamarin.Android.Tasks.CompressedAssemblyInfo] at System.Collections.Generic.TreeSet`1.AddIfNotPresent(T item) at System.Collections.Generic.SortedDictionary`2.Add(TKey key, TValue value) at Xamarin.Android.Tasks.GenerateCompressedAssembliesNativeSourceFiles.GenerateCompressedAssemblySources() at Xamarin.Android.Tasks.GenerateCompressedAssembliesNativeSourceFiles.RunTask() at Xamarin.Android.Tasks.AndroidTask.Execute() Looking at some of the item groups earlier: C:\src\xamarin-android\packages\xamarin.android.fsharp.resourceprovider\1.0.1\lib\monoandroid81\Xamarin.Android.FSharp.ResourceProvider.Runtime.dll ... DestinationSubPath = Xamarin.Android.FSharp.ResourceProvider.Runtime.dll ... C:\src\xamarin-android\packages\fsharp.core\4.7.2\lib\netstandard2.0\cs\FSharp.Core.resources.dll ... DestinationSubDirectory = cs\ DestinationSubPath = cs\FSharp.Core.resources.dll ... C:\src\xamarin-android\packages\fsharp.core\4.7.2\lib\netstandard2.0\de\FSharp.Core.resources.dll ... DestinationSubDirectory = de\ DestinationSubPath = de\FSharp.Core.resources.dll ... Two instances of `FSharp.Core.resources.dll` is causing the above exception. Should we be using the `%(DestinationSubDirectory)` and `%(DestinationSubPath)` item metadata? Currently, we have some behavior for architecture-specific .NET assemblies: * `%(AbiDirectory)` is set for x86, armeabi-v7a, etc. * `%(IntermediateLinkerOutput)` is set to a full path taking `%(AbiDirectory)` into account. To clean things up here, we should just use `%(DestinationSubDirectory)` coming from dotnet/sdk and `%(DestinationSubPath)` and remove the item data that we invented. Other changes: * Usage of `%(AbiDirectory)` can use `%(DestinationSubDirectory)` or `%(DestinationSubPath)` where appropriate. * Any usage of `%(IntermediateLinkerOutput)` can use `$(MonoAndroidIntermediateAssemblyDir)%(DestinationSubPath)` instead. * The `<ResolveAssemblies/>` (legacy) and `<ProcessAssemblies/>` MSBuild tasks no longer need the `IntermediateAssemblyDirectory` property. * A new check was needed in `<ProcessAssemblies/>` to make sure we don't append `%(DestinationSubDirectory)` for the same assembly multiple times.

Context: xamarin/Xamarin.Android.FSharp.ResourceProvider#6 Xamarin.Android.FSharp.ResourceProvider 1.0.1 is now built for `netstandard2.0` and published to an internal feed. We should be able to enable F# tests under a `dotnet` context now. The `_RemoveLegacyDesigner` MSBuild target was running, when it shouldn't be for F# projects and adding a `Resource.designer.cs` `@(Compile)` item. I needed to add a `$(Language)` check. One workaround is for: dotnet/sdk#12954 We need to set `$(ProduceReferenceAssembly)` to `false` for .NET 5 F# projects at the moment.

jonathanpeppers · 2020-09-02T19:15:38Z

This looks good to merge, I opened an issue for the F# workarounds we have: #5076

jonathanpeppers requested review from dellis1972 and jonpryor as code owners August 21, 2020 16:15

jonathanpeppers requested a review from pjcollins August 21, 2020 16:15

jonathanpeppers force-pushed the dotnet-fsharp-resourceprovider branch from 689dc7e to 0564fac Compare August 21, 2020 19:09

jonathanpeppers marked this pull request as draft August 21, 2020 21:59

jonathanpeppers mentioned this pull request Aug 24, 2020

Linker fails for F# projects in .NET 5+ dotnet/linker#1448

Closed

jonathanpeppers force-pushed the dotnet-fsharp-resourceprovider branch from 0564fac to 27f399b Compare August 24, 2020 16:39

jonathanpeppers mentioned this pull request Aug 24, 2020

[One .NET] fixes for satellite assemblies #5044

Merged

jonathanpeppers force-pushed the dotnet-fsharp-resourceprovider branch from 27f399b to 9240c28 Compare August 24, 2020 20:14

jonathanpeppers force-pushed the dotnet-fsharp-resourceprovider branch from 9240c28 to b4c8bd7 Compare September 1, 2020 13:11

jonathanpeppers marked this pull request as ready for review September 1, 2020 13:11

jonathanpeppers force-pushed the dotnet-fsharp-resourceprovider branch from b4c8bd7 to b9686e7 Compare September 1, 2020 20:54

pjcollins approved these changes Sep 2, 2020

View reviewed changes

dellis1972 approved these changes Sep 2, 2020

View reviewed changes

jonathanpeppers mentioned this pull request Sep 2, 2020

[One .NET] workarounds for F# #5076

Closed

jonathanpeppers merged commit c31ad9f into dotnet:master Sep 2, 2020

jonathanpeppers deleted the dotnet-fsharp-resourceprovider branch September 2, 2020 19:16

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[tests] enable One .NET tests for F# #5040

[tests] enable One .NET tests for F# #5040

jonathanpeppers commented Aug 21, 2020

pjcollins commented Aug 21, 2020

jonathanpeppers commented Aug 21, 2020

jonathanpeppers commented Aug 21, 2020

jonathanpeppers commented Sep 2, 2020

[tests] enable One .NET tests for F# #5040

[tests] enable One .NET tests for F# #5040

Conversation

jonathanpeppers commented Aug 21, 2020

pjcollins commented Aug 21, 2020

jonathanpeppers commented Aug 21, 2020

jonathanpeppers commented Aug 21, 2020

jonathanpeppers commented Sep 2, 2020