Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Android] Introduce NetTraceToMibcConverter task & streamline testing targets #72394

Merged
merged 16 commits into from
Aug 4, 2022
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion eng/Subsets.props
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@
<DefaultMonoSubsets Condition="'$(MonoAOTEnableLLVM)' == 'true' and '$(MonoAOTLLVMDir)' == ''">mono.llvm+</DefaultMonoSubsets>
<DefaultMonoSubsets Condition="'$(TargetOS)' == 'Browser'">$(DefaultMonoSubsets)mono.wasmruntime+</DefaultMonoSubsets>
<DefaultMonoSubsets Condition="'$(MonoCrossAOTTargetOS)' != ''">$(DefaultMonoSubsets)mono.aotcross+</DefaultMonoSubsets>
<DefaultMonoSubsets>$(DefaultMonoSubsets)mono.runtime+mono.corelib+mono.packages</DefaultMonoSubsets>
<DefaultMonoSubsets>$(DefaultMonoSubsets)mono.runtime+mono.corelib+mono.packages+mono.tools+</DefaultMonoSubsets>

<DefaultLibrariesSubsets Condition="'$(BuildTargetFramework)' == '$(NetCoreAppCurrent)' or
'$(BuildTargetFramework)' == '' or
Expand Down Expand Up @@ -135,6 +135,7 @@
<SubsetName Include="Mono.AotCross" Description="The cross-compiler runtime for Mono AOT." />
<SubsetName Include="Mono.CoreLib" Description="The managed System.Private.CoreLib library for Mono." />
<SubsetName Include="Mono.Packages" Description="The projects that produce NuGet packages for the Mono runtime." />
<SubsetName Include="Mono.Tools" Description="Tooling that helps support Mono development and testing." />
<SubsetName Include="Mono.WasmRuntime" Description="The WebAssembly runtime." />
<SubsetName Include="Mono.MsCorDbi" Description="The implementation of ICorDebug interface." />
<SubsetName Include="Mono.Workloads" OnDemand="true" Description="Builds the installers and the insertion metadata for Blazor workloads." />
Expand Down Expand Up @@ -357,6 +358,10 @@
<ProjectToBuild Include="$(MonoProjectRoot)System.Private.CoreLib\System.Private.CoreLib.csproj" Category="mono" />
</ItemGroup>

<ItemGroup Condition="$(_subset.Contains('+mono.tools+'))">
<ProjectToBuild Include="$(CoreClrProjectRoot)tools\dotnet-pgo\dotnet-pgo.csproj;" Category="mono" />
</ItemGroup>

<ItemGroup Condition="$(_subset.Contains('+mono.workloads+'))">
<ProjectToBuild Include="$(WorkloadsProjectRoot)\workloads.csproj" Category="mono" />
</ItemGroup>
Expand Down
74 changes: 74 additions & 0 deletions eng/testing/tests.android.targets
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
<Project>
<PropertyGroup>
<BundleTestAppTargets>$(BundleTestAppTargets);BundleTestAndroidApp</BundleTestAppTargets>
</PropertyGroup>

<Import Project="$(MonoProjectRoot)\msbuild\android\build\AndroidApp.props" />
<Import Project="$(MonoProjectRoot)\msbuild\android\build\AndroidApp.InTree.targets" />

<PropertyGroup>
<AndroidBuildAppDependsOn>PrepareForAndroidBuildApp;$(AndroidBuildAppDependsOn);_CopyTestArchive</AndroidBuildAppDependsOn>

<BundleTestAndroidAppDependsOn>AndroidBuildApp</BundleTestAndroidAppDependsOn>
</PropertyGroup>

<!-- Target that kicks off the whole test build and run flow -->
<Target Name="BundleTestAndroidApp" DependsOnTargets="$(BundleTestAndroidAppDependsOn)" />

<Target Name="PrepareForAndroidBuildApp">
<Error Condition="!Exists('$(MicrosoftNetCoreAppRuntimePackRidDir)')" Text="MicrosoftNetCoreAppRuntimePackRidDir=$(MicrosoftNetCoreAppRuntimePackRidDir) doesn't exist" />

<WriteLinesToFile File="$(PublishDir)xunit-excludes.txt" Lines="$(XunitExcludesTxtFileContent)" Overwrite="true" />

<PropertyGroup>
<MainLibraryFileName Condition="'$(MainLibraryFileName)' == ''">AndroidTestRunner.dll</MainLibraryFileName>
steveisok marked this conversation as resolved.
Show resolved Hide resolved

<AndroidAppDir>$(PublishDir)</AndroidAppDir>
<AndroidAppBundleDir>$(BundleDir)</AndroidAppBundleDir>
</PropertyGroup>

<ItemGroup>
<AndroidEnv Condition="'$(XUnitSingleThreadedMode)' == 'true'" Include="XUNIT_SINGLE_THREADED">
<Value>1</Value>
</AndroidEnv>
<AndroidEnv Condition="'$(XUnitUseRandomizedTestOrderer)' == 'true'" Include="XUNIT_RANDOM_ORDER_SEED">
<Value>1883302047</Value>
</AndroidEnv>
<AndroidEnv Condition="'$(XUnitSingleThreadedMode)' == 'true'" Include="XUNIT_VERBOSE">
<Value>1</Value>
</AndroidEnv>
</ItemGroup>

<ItemGroup>
<AndroidAssembliesToBundle Include="@(NativeLibraries->'$(PublishDir)%(Identity)')">
<_InternalForceInterpret>true</_InternalForceInterpret>
<_IsNative>true</_IsNative>
</AndroidAssembliesToBundle>

<_PublishAssemblies Include="$(PublishDir)\**\*.dll" Exclude="$(PublishDir)\**\*.resources.dll" />
<_SatelliteAssemblies Include="$(PublishDir)\**\*.resources.dll" />

<AndroidAssembliesToBundle Include="@(_PublishAssemblies)">
Comment on lines +48 to +51
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit since publish assesmblies naming implis all assemblies, and it doesnt seem like its being used elsewhere, so is there a benefit to having it in a separate itemgroup rather than explicitly defining AndroidAssembliesToBundle?

Suggested change
<_PublishAssemblies Include="$(PublishDir)\**\*.dll" Exclude="$(PublishDir)\**\*.resources.dll" />
<_SatelliteAssemblies Include="$(PublishDir)\**\*.resources.dll" />
<AndroidAssembliesToBundle Include="@(_PublishAssemblies)">
<_SatelliteAssemblies Include="$(PublishDir)\**\*.resources.dll" />
<AndroidAssembliesToBundle Include="$(PublishDir)\**\*.dll" Exclude="@(_SatelliteAssemblies)">

Is there a particular reason we are keeping track of SatelliteAssemblies in an itemgroup?
If its not going to be used, maybe just
<AndroidAssembliesToBundle Include="$(PublishDir)\**\*.dll" Exclude="$(PublishDir)\**\*.resources.dll" />

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could be wrong, but I think the paths need expanded in an item first before you can condition them like we do in AndroidAssembliesToBundle

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah do you mean the metadata getting set properly?

I think it still works properly before getting expanded. With a console project with a.txt b.txt c.txt in the same directory

<ItemGroup>
   <TestItemGroup Include="$(MSBuildThisFileDirector)*.txt">
     <_InternalForceInterpret Condition="'%(FileName)%(Extension)' != 'b.txt'">true</_InternalForceInterpret>
   </TestItemGroup>
 </ItemGroup>

evaluates to
Screen Shot 2022-07-19 at 16 51 29

So it looks like the metadata is still set properly

<_InternalForceInterpret Condition="'$(UseMonoJustInterp)' == 'true' and '%(FileName)%(Extension)' != 'System.Private.CoreLib.dll'">true</_InternalForceInterpret>
<_IsNative>false</_IsNative>
</AndroidAssembliesToBundle>

<AndroidNativeFilesToBundle Include="$(PublishDir)\**\*.*" Exclude="$(PublishDir)\*.dll" />
mdh1418 marked this conversation as resolved.
Show resolved Hide resolved
</ItemGroup>
</Target>

<Target Name="_CopyTestArchive"
Condition="'$(ArchiveTests)' == 'true' and '$(IgnoreForCI)' != 'true'">
<Error Condition="'$(TestArchiveTestsDir)' == ''" Text="TestArchiveTestsDir property to archive the test folder must be set." />

<Message Importance="High" Text="PackageId: $(ApkPackageId)"/>
<Message Importance="High" Text="Instrumentation: net.dot.MonoRunner"/>
<Message Importance="High" Text="Apk: $(ApkBundlePath)"/>

<Copy SourceFiles="$(ApkBundlePath)"
DestinationFolder="$(TestArchiveTestsDir)"
SkipUnchangedFiles="true"
Condition="'$(ArchiveTests)' == 'true' and '$(IgnoreForCI)' != 'true'" />
</Target>

</Project>
122 changes: 11 additions & 111 deletions eng/testing/tests.mobile.targets
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,18 @@
<RunScriptOutputPath>$([MSBuild]::NormalizePath('$(BundleDir)', '$(RunScriptOutputName)'))</RunScriptOutputPath>

<PublishingTestsRun>true</PublishingTestsRun>
<BundleTestAppTargets>BundleTestAndroidApp</BundleTestAppTargets>
<PublishTestAsSelfContainedDependsOn>Publish</PublishTestAsSelfContainedDependsOn>
</PropertyGroup>

<PropertyGroup>
<RunAOTCompilation Condition="'$(TargetOS)' == 'iOS' or '$(TargetOS)' == 'tvOS'">true</RunAOTCompilation>
</PropertyGroup>

<PropertyGroup>
<DotnetPgoToolDir>$([MSBuild]::NormalizeDirectory('$(ArtifactsBinDir)', 'coreclr', '$(TargetOS).$(TargetArchitecture).$(Configuration)', 'dotnet-pgo'))</DotnetPgoToolDir>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: this could be unified with ILAsmToolPath which references the same path in Directory.Build.props

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since ILAsmToolPath is only active when it's source build or s390x, I made a CoreCLRToolPath property and used that instead.

<DotnetPgoToolPath>$([MSBuild]::NormalizePath('$(DotnetPgoToolDir)', 'dotnet-pgo'))</DotnetPgoToolPath>
</PropertyGroup>

<PropertyGroup Condition="'$(EnableAggressiveTrimming)' == 'true' or '$(EnableSoftTrimming)' == 'true'">
<PublishTrimmed>true</PublishTrimmed>
<!-- Suppress linker warnings as these are tests -->
Expand All @@ -39,6 +43,10 @@
<AdditionalXHarnessArguments Condition="'$(ExpectedExitCode)' != ''">$(AdditionalXHarnessArguments) --expected-exit-code $(ExpectedExitCode)</AdditionalXHarnessArguments>
</PropertyGroup>

<PropertyGroup>
<DiagnosticPorts Condition="'$(DiagnosticStartup)' != ''">$(DiagnosticPorts),$(DiagnosticStartup)</DiagnosticPorts>
steveisok marked this conversation as resolved.
Show resolved Hide resolved
</PropertyGroup>

<PropertyGroup Condition="'$(TargetOS)' == 'Android'">
<!-- The -arg flag for xharness passes the arguments along to the instrumentation app -->
<AdditionalXHarnessArguments Condition="'$(XUnitMethodName)' != ''">$(AdditionalXHarnessArguments) --arg=-m=$(XUnitMethodName)</AdditionalXHarnessArguments>
Expand All @@ -55,9 +63,8 @@
<Compile Include="$(RepoRoot)src\libraries\Common\tests\Tests\RandomizedTestOrderAssemblyInfo.cs" />
</ItemGroup>

<UsingTask Condition="'$(RunAOTCompilation)' == 'true'" TaskName="MonoAOTCompiler" AssemblyFile="$(MonoAOTCompilerTasksAssemblyPath)" />
<Import Condition="'$(RunAOTCompilation)' == 'true'" Project="$(MonoAOTCompilerDir)MonoAOTCompiler.props" />

<Import Project="$(MSBuildThisFileDirectory)tests.android.targets"
Condition="'$(TargetOS)' == 'Android'" />
<Import Project="$(MSBuildThisFileDirectory)tests.ioslike.targets"
Condition="'$(TargetsAppleMobile)' == 'true'" />
<Import Project="$(MSBuildThisFileDirectory)tests.wasm.targets"
Expand All @@ -67,113 +74,6 @@
<_MobileIntermediateOutputPath>$(IntermediateOutputPath)mobile</_MobileIntermediateOutputPath>
</PropertyGroup>

<UsingTask TaskName="RuntimeConfigParserTask"
AssemblyFile="$(MonoTargetsTasksAssemblyPath)"
Condition="'$(MonoTargetsTasksAssemblyPath)' != ''" />

<Target Name="GenerateRuntimeConfig" Condition="'$(TargetOS)' != 'Browser'"
Inputs="$(PublishDir)$(AssemblyName).runtimeconfig.json"
Outputs="$(PublishDir)runtimeconfig.bin">
<PropertyGroup>
<RuntimeConfigFilePath>$(PublishDir)$(AssemblyName).runtimeconfig.json</RuntimeConfigFilePath>
<ParsedRuntimeConfigFilePath>$(PublishDir)runtimeconfig.bin</ParsedRuntimeConfigFilePath>
</PropertyGroup>

<ItemGroup>
<RuntimeConfigReservedProperties Include="RUNTIME_IDENTIFIER"/>
<RuntimeConfigReservedProperties Include="APP_CONTEXT_BASE_DIRECTORY"/>
</ItemGroup>

<!-- Parse runtimeconfig.template.json file -->
<RuntimeConfigParserTask
RuntimeConfigFile="$(RuntimeConfigFilePath)"
OutputFile="$(ParsedRuntimeConfigFilePath)"
RuntimeConfigReservedProperties="@(RuntimeConfigReservedProperties)">
</RuntimeConfigParserTask>
</Target>

<!-- Generate a self-contained app bundle for Android with tests. -->
<UsingTask Condition="'$(TargetOS)' == 'Android'"
TaskName="AndroidAppBuilderTask"
AssemblyFile="$(AndroidAppBuilderTasksAssemblyPath)" />

<Target Condition="'$(TargetOS)' == 'Android'" Name="BundleTestAndroidApp" DependsOnTargets="GenerateRuntimeConfig">
steveisok marked this conversation as resolved.
Show resolved Hide resolved
<Error Condition="!Exists('$(MicrosoftNetCoreAppRuntimePackRidDir)')" Text="MicrosoftNetCoreAppRuntimePackRidDir=$(MicrosoftNetCoreAppRuntimePackRidDir) doesn't exist" />

<PropertyGroup>
<AndroidAbi Condition="'$(TargetArchitecture)' == 'arm64'">arm64-v8a</AndroidAbi>
<AndroidAbi Condition="'$(TargetArchitecture)' == 'arm'">armeabi-v7a</AndroidAbi>
<AndroidAbi Condition="'$(TargetArchitecture)' == 'x64'">x86_64</AndroidAbi>
<AndroidAbi Condition="'$(TargetArchitecture)' == 'x86'">x86</AndroidAbi>

<MainLibraryFileName Condition="'$(MainLibraryFileName)' == ''">AndroidTestRunner.dll</MainLibraryFileName>
</PropertyGroup>

<ItemGroup>
<_AndroidEnv Condition="'$(XUnitSingleThreadedMode)' == 'true'" Include="XUNIT_SINGLE_THREADED">
<Value>1</Value>
</_AndroidEnv>
<_AndroidEnv Condition="'$(XUnitUseRandomizedTestOrderer)' == 'true'" Include="XUNIT_RANDOM_ORDER_SEED">
<Value>1883302047</Value>
</_AndroidEnv>
<_AndroidEnv Condition="'$(XUnitSingleThreadedMode)' == 'true'" Include="XUNIT_VERBOSE">
<Value>1</Value>
</_AndroidEnv>
</ItemGroup>

<ItemGroup Condition="'$(RunAOTCompilation)' == 'true'">
<AotInputAssemblies Include="$(PublishDir)\*.dll">
<AotArguments>@(MonoAOTCompilerDefaultAotArguments, ';')</AotArguments>
<ProcessArguments>@(MonoAOTCompilerDefaultProcessArguments, ';')</ProcessArguments>
</AotInputAssemblies>
</ItemGroup>

<WriteLinesToFile File="$(PublishDir)xunit-excludes.txt" Lines="$(XunitExcludesTxtFileContent)" Overwrite="true" />

<MakeDir Directories="$(_MobileIntermediateOutputPath)"
Condition="'$(RunAOTCompilation)' == 'true'"/>
<RemoveDir Directories="$(BundleDir)" />

<MonoAOTCompiler Condition="'$(RunAOTCompilation)' == 'true'"
CompilerBinaryPath="@(MonoAotCrossCompiler->WithMetadataValue('RuntimeIdentifier','$(TargetOS.ToLowerInvariant())-$(TargetArchitecture.ToLowerInvariant())'))"
OutputDir="$(_MobileIntermediateOutputPath)"
Mode="Full"
OutputType="AsmOnly"
Assemblies="@(AotInputAssemblies)"
AotModulesTablePath="$(BundleDir)\modules.c"
IntermediateOutputPath="$(IntermediateOutputPath)"
UseLLVM="$(MonoEnableLLVM)"
LLVMPath="$(MonoAotCrossDir)">
<Output TaskParameter="CompiledAssemblies" ItemName="BundleAssemblies" />
</MonoAOTCompiler>

<AndroidAppBuilderTask
RuntimeIdentifier="$(RuntimeIdentifier)"
ProjectName="$(AssemblyName)"
MonoRuntimeHeaders="$(MicrosoftNetCoreAppRuntimePackNativeDir)include\mono-2.0"
Assemblies="@(BundleAssemblies)"
MainLibraryFileName="$(MainLibraryFileName)"
IncludeNetworkSecurityConfig="$(IncludeNetworkSecurityConfig)"
EnvironmentVariables="@(_AndroidEnv)"
ForceAOT="$(RunAOTCompilation)"
ForceInterpreter="$(MonoForceInterpreter)"
StripDebugSymbols="False"
OutputDir="$(BundleDir)"
AppDir="$(PublishDir)">
<Output TaskParameter="ApkPackageId" PropertyName="ApkPackageId" />
<Output TaskParameter="ApkBundlePath" PropertyName="ApkBundlePath" />
</AndroidAppBuilderTask>

<Message Importance="High" Text="PackageId: $(ApkPackageId)"/>
<Message Importance="High" Text="Instrumentation: net.dot.MonoRunner"/>
<Message Importance="High" Text="Apk: $(ApkBundlePath)"/>

<Copy SourceFiles="$(ApkBundlePath)"
DestinationFolder="$(TestArchiveTestsDir)"
SkipUnchangedFiles="true"
Condition="'$(ArchiveTests)' == 'true' and '$(IgnoreForCI)' != 'true'" />
</Target>

<!-- This .targets file is also imported by the runtime Trimming tests, and we want to be able to manually configure trimming in them so this
should be considered if we ever want to change the Condition of the ConfigureTrimming target -->
<Target Name="ConfigureTrimming" Condition="('$(EnableAggressiveTrimming)' == 'true' or '$(EnableSoftTrimming)' == 'true') And '$(SkipConfigureTrimming)' != 'true'" AfterTargets="AddTestRunnersToPublishedFiles">
Expand Down
13 changes: 13 additions & 0 deletions src/mono/msbuild/android/build/AndroidApp.InTree.props
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
<Project>
steveisok marked this conversation as resolved.
Show resolved Hide resolved
<!-- This depends on the root Directory.Build.props imported this file -->

<Import Project="$(MSBuildThisFileDirectory)AndroidApp.props" />
<PropertyGroup>
<TargetFramework>$(NetCoreAppCurrent)</TargetFramework>
<EnableTargetingPackDownload>false</EnableTargetingPackDownload>
<EnableRuntimePackDownload>false</EnableRuntimePackDownload>
<PublishTrimmed>true</PublishTrimmed>
<TrimMode>link</TrimMode>
<RunAnalyzers>false</RunAnalyzers>
</PropertyGroup>
</Project>
21 changes: 21 additions & 0 deletions src/mono/msbuild/android/build/AndroidApp.InTree.targets
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
<Project>
<!-- This depends on the root Directory.Build.targets imported this file -->
<UsingTask TaskName="MonoAOTCompiler" AssemblyFile="$(MonoAOTCompilerTasksAssemblyPath)" />
<UsingTask TaskName="NetTraceToMibcConverter" AssemblyFile="$(MonoTargetsTasksAssemblyPath)" />
<UsingTask TaskName="RuntimeConfigParserTask" AssemblyFile="$(MonoTargetsTasksAssemblyPath)" />
steveisok marked this conversation as resolved.
Show resolved Hide resolved

<Import Project="$(MSBuildThisFileDirectory)AndroidApp.targets" />

<!-- Use local runtime pack -->
<Target Name="UpdateRuntimePack" AfterTargets="ResolveFrameworkReferences">
<PropertyGroup>
<_LocalMicrosoftNetCoreAppRuntimePackDir>$(MicrosoftNetCoreAppRuntimePackDir)</_LocalMicrosoftNetCoreAppRuntimePackDir>
</PropertyGroup>
<ItemGroup>
<ResolvedRuntimePack PackageDirectory="$(_LocalMicrosoftNetCoreAppRuntimePackDir)"
Condition="'$(_LocalMicrosoftNetCoreAppRuntimePackDir)' != '' and
'%(ResolvedRuntimePack.FrameworkName)' == 'Microsoft.NETCore.App'" />
</ItemGroup>
<Message Text="Used runtime pack: %(ResolvedRuntimePack.PackageDirectory)" Importance="high" />
</Target>
</Project>
18 changes: 18 additions & 0 deletions src/mono/msbuild/android/build/AndroidApp.props
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
<Project>
<PropertyGroup>
<RuntimeIdentifier>$(TargetOS.ToLowerInvariant())-$(TargetArchitecture.ToLowerInvariant())</RuntimeIdentifier>
steveisok marked this conversation as resolved.
Show resolved Hide resolved
<UseMonoRuntime>true</UseMonoRuntime>
steveisok marked this conversation as resolved.
Show resolved Hide resolved
<UseMonoJustInterp Condition="'$(RunAOTCompilation)' == 'true' and '$(MonoForceInterpreter)' == 'true'">true</UseMonoJustInterp>

<AndroidBuildAppAfterThisTarget Condition="'$(AndroidBuildAppAfterThisTarget)' == ''">Publish</AndroidBuildAppAfterThisTarget>
mdh1418 marked this conversation as resolved.
Show resolved Hide resolved
<AndroidBuildAppDependsOn>
_InitializeCommonProperties;
_BeforeAndroidBuildApp;
_AndroidResolveReferences;
_AndroidPrepareProfiledAot;
_AndroidAotCompileApp;
_AndroidGenerateAppBundle;
_AfterAndroidBuildApp
steveisok marked this conversation as resolved.
Show resolved Hide resolved
</AndroidBuildAppDependsOn>
</PropertyGroup>
</Project>
Loading