Merge branch 'microsoft:main' into mhristova/enable-binskim-scan

docs/docfx/articles/dests-health-checks.md

@@ -252,7 +252,7 @@ endpoints.MapReverseProxy(proxyPipeline =>
 - `ReactivationPeriod` - period after which an unhealthy destination's passive health state is reset to `Unknown` and it starts receiving traffic again. Default value is `null` which means the period will be set by a `IPassiveHealthCheckPolicy`
 
 ### Built-in policies
-There is currently one built-in passive health check policy - [`TransportFailureRateHealthPolicy`](xref:Yarp.ReverseProxy.Health.TransportFailureRateHealthPolicyOptions). It calculates the proxied requests failure rate for each destination and marks it as unhealthy if the specified limit is exceeded. Rate is calculated as a percentage of failured requests to the total number of request proxied to a destination in the given period of time. Failed and total counters are tracked in a sliding time window which means that only the recent readings fitting in the window are taken into account.
+There is currently one built-in passive health check policy - [`TransportFailureRateHealthPolicy`](xref:Yarp.ReverseProxy.Health.TransportFailureRateHealthPolicyOptions). It calculates the proxied requests failure rate for each destination and marks it as unhealthy if the specified limit is exceeded. Rate is calculated as a percentage of failed requests to the total number of request proxied to a destination in the given period of time. Failed and total counters are tracked in a sliding time window which means that only the recent readings fitting in the window are taken into account.
 There are two sets of policy parameters defined globally and on per cluster level.
 
 Global parameters are set via the options mechanism using `TransportFailureRateHealthPolicyOptions` type with the following properties:
@@ -272,7 +272,7 @@ services.Configure<TransportFailureRateHealthPolicyOptions>(o =>
 ```
 
 Cluster-specific parameters are set in the cluster's metadata as follows:
-`TransportFailureRateHealthPolicy.RateLimit` - failure rate limit for a destination to be marked as unhealhty. The value is in range `(0,1)`. Default value is provided by the global `DefaultFailureRateLimit` parameter.
+`TransportFailureRateHealthPolicy.RateLimit` - failure rate limit for a destination to be marked as unhealthy. The value is in range `(0,1)`. Default value is provided by the global `DefaultFailureRateLimit` parameter.
 
 ### Design
 The main component is [PassiveHealthCheckMiddleware](xref:Yarp.ReverseProxy.Health.PassiveHealthCheckMiddleware) sitting in the request pipeline and analyzing responses returned by destinations. For each response from a destination belonging to a cluster with enabled passive health checks, `PassiveHealthCheckMiddleware` invokes an [IPassiveHealthCheckPolicy](xref:Yarp.ReverseProxy.Health.IPassiveHealthCheckPolicy) specified for the cluster. The policy analyzes the given response, evaluates a new destination's passive health state and calls [IDestinationHealthUpdater](xref:Yarp.ReverseProxy.Health.IDestinationHealthUpdater) to actually update [DestinationHealthState.Passive](xref:Yarp.ReverseProxy.Model.DestinationHealthState.Passive) value. The update happens asynchronously in the background and doesn't block the request pipeline. When a destination gets marked as unhealthy, it stops receiving new requests until it gets reactivated after a configured period. Reactivation means the destination's `DestinationHealthState.Passive` state is reset from `Unhealthy` to `Unknown` and the cluster's list of healthy destinations is rebuilt to include it. A reactivation is scheduled by `IDestinationHealthUpdater` right after setting the destination's `DestinationHealthState.Passive` to `Unhealthy`.
@@ -329,10 +329,10 @@ public class FirstUnsuccessfulResponseHealthPolicy : IPassiveHealthCheckPolicy
 ## Available destination collection
 Destinations health state is used to determine which of them are eligible for receiving proxied requests. Each cluster maintains its own list of available destinations on `AvailableDestinations` property of the [ClusterDestinationState](xref:Yarp.ReverseProxy.Model.ClusterDestinationsState) type. That list gets rebuilt when any destination's health state changes. The [IClusterDestinationsUpdater](xref:Yarp.ReverseProxy.Health.IClusterDestinationsUpdater) controls that process and calls an [IAvailableDestinationsPolicy](xref:Yarp.ReverseProxy.Health.IAvailableDestinationsPolicy) configured on the cluster to actually choose the available destinations from the all cluster's destinations. There are the following built-in policies provided and custom ones can be implemented if necessary.
 
-- `HealthyAndUnknown` - inspects each `DestinationState` and adds it on the available destination list if all of the following statements are TRUE. If no destinations are available then requests will get a 503 error. This is the default policy.
+- `HealthyAndUnknown` - inspects each `DestinationState` and adds it on the available destination list if all of the following statements are TRUE. If no destinations are available then requests will get a 503 error.
     - Active health checks are disabled on the cluster OR `DestinationHealthState.Active != DestinationHealth.Unhealthy`
     - Passive health checks are disabled on the cluster OR `DestinationHealthState.Passive != DestinationHealth.Unhealthy`
-- `HealthyOrPanic` - calls `HealthyAndUnknown` policy at first to get the available destinations. If none of them are returned from this call, it marks all cluster's destinations as available.
+- `HealthyOrPanic` - calls `HealthyAndUnknown` policy at first to get the available destinations. If none of them are returned from this call, it marks all cluster's destinations as available. This is the default policy.
 
 **NOTE**: An available destination policy configured on a cluster will be always called regardless of if any health check is enabled on the given cluster. The health state of a disabled health check is set to `Unknown`.
 

eng/Version.Details.xml

@@ -3,17 +3,17 @@
   <ProductDependencies>
   </ProductDependencies>
   <ToolsetDependencies>
-    <Dependency Name="Microsoft.DotNet.Arcade.Sdk" Version="8.0.0-beta.23316.6">
+    <Dependency Name="Microsoft.DotNet.Arcade.Sdk" Version="8.0.0-beta.23330.1">
       <Uri>https://github.com/dotnet/arcade</Uri>
-      <Sha>e2334b2be36919347923d0ec872a46acddb1e385</Sha>
+      <Sha>9fc23a576f443284d25cd4311f5778158ad289c9</Sha>
     </Dependency>
-    <Dependency Name="Microsoft.DotNet.Helix.Sdk" Version="8.0.0-beta.23316.6">
+    <Dependency Name="Microsoft.DotNet.Helix.Sdk" Version="8.0.0-beta.23330.1">
       <Uri>https://github.com/dotnet/arcade</Uri>
-      <Sha>e2334b2be36919347923d0ec872a46acddb1e385</Sha>
+      <Sha>9fc23a576f443284d25cd4311f5778158ad289c9</Sha>
     </Dependency>
-    <Dependency Name="Microsoft.DotNet.XUnitExtensions" Version="8.0.0-beta.23316.6">
+    <Dependency Name="Microsoft.DotNet.XUnitExtensions" Version="8.0.0-beta.23330.1">
       <Uri>https://github.com/dotnet/arcade</Uri>
-      <Sha>e2334b2be36919347923d0ec872a46acddb1e385</Sha>
+      <Sha>9fc23a576f443284d25cd4311f5778158ad289c9</Sha>
     </Dependency>
   </ToolsetDependencies>
 </Dependencies>

eng/Versions.props

@@ -13,6 +13,6 @@
   <!--Package versions-->
   <PropertyGroup>
     <MicrosoftBclTimeProviderVersion>8.0.0-preview.4.23220.7</MicrosoftBclTimeProviderVersion>
-    <MicrosoftDotNetXUnitExtensionsPackageVersion>8.0.0-beta.23316.6</MicrosoftDotNetXUnitExtensionsPackageVersion>
+    <MicrosoftDotNetXUnitExtensionsPackageVersion>8.0.0-beta.23330.1</MicrosoftDotNetXUnitExtensionsPackageVersion>
   </PropertyGroup>
 </Project>

eng/common/sdl/configure-sdl-tool.ps1

@@ -17,7 +17,9 @@ Param(
   # Optional: Additional params to add to any tool using PoliCheck.
   [string[]] $PoliCheckAdditionalRunConfigParams,
   # Optional: Additional params to add to any tool using CodeQL/Semmle.
-  [string[]] $CodeQLAdditionalRunConfigParams
+  [string[]] $CodeQLAdditionalRunConfigParams,
+  # Optional: Additional params to add to any tool using Binskim.
+  [string[]] $BinskimAdditionalRunConfigParams
 )
 
 $ErrorActionPreference = 'Stop'
@@ -69,22 +71,32 @@ try {
     $gdnConfigFile = Join-Path $gdnConfigPath "$toolConfigName-configure.gdnconfig"
 
     # For some tools, add default and automatic args.
-    if ($tool.Name -eq 'credscan') {
-      if ($targetDirectory) {
-        $tool.Args += "`"TargetDirectory < $TargetDirectory`""
+    switch -Exact ($tool.Name) {
+      'credscan' {
+        if ($targetDirectory) {
+          $tool.Args += "`"TargetDirectory < $TargetDirectory`""
+        }
+        $tool.Args += "`"OutputType < pre`""
+        $tool.Args += $CrScanAdditionalRunConfigParams
       }
-      $tool.Args += "`"OutputType < pre`""
-      $tool.Args += $CrScanAdditionalRunConfigParams
-    } elseif ($tool.Name -eq 'policheck') {
-      if ($targetDirectory) {
-        $tool.Args += "`"Target < $TargetDirectory`""
+      'policheck' {
+        if ($targetDirectory) {
+          $tool.Args += "`"Target < $TargetDirectory`""
+        }
+        $tool.Args += $PoliCheckAdditionalRunConfigParams
       }
-      $tool.Args += $PoliCheckAdditionalRunConfigParams
-    } elseif ($tool.Name -eq 'semmle' -or $tool.Name -eq 'codeql') {
-      if ($targetDirectory) {
-        $tool.Args += "`"SourceCodeDirectory < $TargetDirectory`""
+      {$_ -in 'semmle', 'codeql'} {
+        if ($targetDirectory) {
+          $tool.Args += "`"SourceCodeDirectory < $TargetDirectory`""
+        }
+        $tool.Args += $CodeQLAdditionalRunConfigParams
+      }
+      'binskim' {
+        if ($targetDirectory) {
+          $tool.Args += "`"Target < $TargetDirectory`""
+        }
+        $tool.Args += $BinskimAdditionalRunConfigParams
       }
-      $tool.Args += $CodeQLAdditionalRunConfigParams
     }
 
     # Create variable pointing to the args array directly so we can use splat syntax later.

eng/common/sdl/execute-all-sdl-tools.ps1

@@ -35,6 +35,7 @@ Param(
   [string[]] $CrScanAdditionalRunConfigParams,                                                   # Optional: Additional Params to custom build a CredScan run config in the format @("xyz:abc","sdf:1")
   [string[]] $PoliCheckAdditionalRunConfigParams,                                                # Optional: Additional Params to custom build a Policheck run config in the format @("xyz:abc","sdf:1")
   [string[]] $CodeQLAdditionalRunConfigParams,                                                   # Optional: Additional Params to custom build a Semmle/CodeQL run config in the format @("xyz < abc","sdf < 1")
+  [string[]] $BinskimAdditionalRunConfigParams,                                                  # Optional: Additional Params to custom build a Binskim run config in the format @("xyz < abc","sdf < 1")
   [bool] $BreakOnFailure=$False                                                                  # Optional: Fail the build if there were errors during the run
 )
 
@@ -107,7 +108,8 @@ try {
           -GuardianLoggerLevel $GuardianLoggerLevel `
           -CrScanAdditionalRunConfigParams $CrScanAdditionalRunConfigParams `
           -PoliCheckAdditionalRunConfigParams $PoliCheckAdditionalRunConfigParams `
-          -CodeQLAdditionalRunConfigParams $CodeQLAdditionalRunConfigParams
+          -CodeQLAdditionalRunConfigParams $CodeQLAdditionalRunConfigParams `
+          -BinskimAdditionalRunConfigParams $BinskimAdditionalRunConfigParams
         if ($BreakOnFailure) {
           Exit-IfNZEC "Sdl"
         }

eng/common/templates/steps/execute-sdl.yml

@@ -33,7 +33,7 @@ steps:
 
 - ${{ if ne(parameters.overrideParameters, '') }}:
   - powershell: ${{ parameters.executeAllSdlToolsScript }} ${{ parameters.overrideParameters }}
-    displayName: Execute SDL
+    displayName: Execute SDL (Overridden)
     continueOnError: ${{ parameters.sdlContinueOnError }}
     condition: ${{ parameters.condition }}
 

global.json

@@ -1,9 +1,9 @@
 {
   "sdk": {
-    "version": "8.0.100-preview.5.23303.2"
+    "version": "8.0.100-preview.7.23360.2"
   },
   "tools": {
-    "dotnet": "8.0.100-preview.5.23303.2",
+    "dotnet": "8.0.100-preview.7.23360.2",
     "runtimes": {
       "dotnet": [
         "6.0.14",
@@ -16,7 +16,7 @@
     }
   },
   "msbuild-sdks": {
-    "Microsoft.DotNet.Arcade.Sdk": "8.0.0-beta.23316.6",
-    "Microsoft.DotNet.Helix.Sdk": "8.0.0-beta.23316.6"
+    "Microsoft.DotNet.Arcade.Sdk": "8.0.0-beta.23330.1",
+    "Microsoft.DotNet.Helix.Sdk": "8.0.0-beta.23330.1"
   }
 }

src/ReverseProxy/Configuration/ConfigValidator.cs

@@ -482,7 +482,7 @@ private void ValidateHealthChecks(IList<Exception> errors, ClusterConfig cluster
         if (string.IsNullOrEmpty(availableDestinationsPolicy))
         {
             // The default.
-            availableDestinationsPolicy = HealthCheckConstants.AvailableDestinations.HealthyAndUnknown;
+            availableDestinationsPolicy = HealthCheckConstants.AvailableDestinations.HealthyOrPanic;
         }
 
         if (!_availableDestinationsPolicies.ContainsKey(availableDestinationsPolicy))