WebSockets over HTTP/2

[Tratcher]

We've heard some interest express for WebSockets support over HTTP/2 in YARP. This would require support in ASP.NET Core (Kestrel) and HttpClient. The protocol is quite different from original WebSockets.

RFC: https://datatracker.ietf.org/doc/html/rfc8441
Kestrel: dotnet/aspnetcore#7801
HttpClient: dotnet/runtime#69669

We'd also need to worry about protocol upgrades/downgrades between front-end and back-end.

[karelz]

Triage: We have interested partner team, but not yet committed. It helps with COGS.
We would need feature work in .NET Runtime (7.0+).

Forwarding is directly through SocketsHttpHandler, so the API pattern might be different from current ClientWebSocket.

[adityamandaleeka]

This should be happening for .NET 7 on both the server and client side, so let's make YARP take advantage of that in 2.0

[Tratcher]

@greenEkatherine as the first step here, can you test YARP 1.1 on .NET 7 with an Http/2 WebSocket client and capture the error here in this issue? It's expected to fail since Kestrel will accept the H2 WS request but YARP won't know what to do with it. We just want to document what that failure looks like.

[serpent5]

I'm running YARP 1.1 with .NET 7, attempting to proxy the Angular CLI development server's WebSocket connection. Here's some information that might be helpful:

Request Headers:

{
  "method": "GET",
  "url": "wss://localhost:5001/ng-cli-ws",
  "httpVersion": "",
  "headers": [
    {
      "name": "Pragma",
      "value": "no-cache"
    },
    {
      "name": "Origin",
      "value": "https://localhost:5001"
    },
    {
      "name": "Accept-Encoding",
      "value": "gzip, deflate, br"
    },
    {
      "name": "Host",
      "value": "localhost:5001"
    },
    {
      "name": "Accept-Language",
      "value": "en-GB,en-US;q=0.9,en;q=0.8"
    },
    {
      "name": "User-Agent",
      "value": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36 Edg/107.0.1418.42"
    },
    {
      "name": "Upgrade",
      "value": "websocket"
    },
    {
      "name": "Cache-Control",
      "value": "no-cache"
    },
    {
      "name": "Connection",
      "value": "Upgrade"
    },
    {
      "name": "Sec-WebSocket-Version",
      "value": "13"
    },
    {
      "name": "Sec-WebSocket-Extensions",
      "value": "permessage-deflate; client_max_window_bits"
    }
  ]
}

Here's the request as logged in ASP.NET Core:

[09:44:51 INF] Request starting HTTP/2 CONNECT https://localhost:5001/ng-cli-ws

Here's a stack-trace from YARP:

System.NotSupportedException: Unsupported request method 'CONNECT'.
   at Yarp.ReverseProxy.Forwarder.RequestUtilities.GetHttpMethod(String method)
   at Yarp.ReverseProxy.Forwarder.HttpForwarder.CreateRequestMessageAsync(HttpContext context, String destinationPrefix, HttpTransformer transformer, ForwarderRequestConfig requestConfig, Boolean isStreamingRequest, ActivityCancellationTokenSource activityToken)
   at Yarp.ReverseProxy.Forwarder.HttpForwarder.SendAsync(HttpContext context, String destinationPrefix, HttpMessageInvoker httpClient, ForwarderRequestConfig requestConfig, HttpTransformer transformer)

Let me know if I can provide more information. If I've misunderstood and this isn't what you're after, I can delete this comment.

[Tratcher]

@serpent5 we know, we haven't implemented that support for YARP yet.

Some workarounds in the meantime:

• Disable HTTP/2 in Kestrel.
• Add a middleware to reject CONNECT requests with HTTP_1_1_REQUIRED (0xd) (I need to try this...)
  • https://learn.microsoft.com/en-us/aspnet/core/fundamentals/servers/kestrel/http2?view=aspnetcore-6.0#reset
  • You can also try returning a 400 (I need to try this too...)

[serpent5]

Ok, thanks. I understand it's a known limitation. I was just trying to provide more information around the exception to see if it helped. Thanks for the workarounds.

In my testing, rejecting the request with HTTP_1_1_REQUIRED works well. Returning a 400 causes the client to keep retrying the request, and so it keeps failing.

[adambajguz]

@serpent5 we know, we haven't implemented that support for YARP yet.

I know it's a hard question but: do you have any estimated release date for Websockets over HTTP/2?

[Tratcher]

I know it's a hard question but: do you have any estimated release date for Websockets over HTTP/2?

Not specifically, but we should start working on it soon. Once it's merged you'll be able to try it from nightly builds and preview releases.

What's your interest/dependency on this feature?

[Garios]

We would need it for Server-Side-Blazor.

[Lakerfield]

To save people some time: a temporary middleware implementing the fix from above: https://gist.github.com/Lakerfield/0ea3a687974df5192d66a375d78ce0e6

    if (context.Request.Method == HttpMethods.Connect && context.Request.Protocol != HttpProtocol.Http11)
    {
      var resetFeature = context.Features.Get<IHttpResetFeature>();
      if (resetFeature != null)
      {
        //https://www.rfc-editor.org/rfc/rfc7540#page-51
        //HTTP_1_1_REQUIRED (0xd):  The endpoint requires that HTTP/1.1 be used instead of HTTP/2.
        resetFeature.Reset(errorCode: 0xd);
        return;
      }
    }

[meirkr]

Hi
Will this http2 support added recently means that it will support also an incoming http1.1 ws connection request from client while the outgoing (downstream service) can be http2 wss?

[MihaZupan]

Yes, you can do any combination of [1.1, 2.0] incoming to [1.1, 2.0] outgoing.

[Tratcher]

Yes

[meirkr]

Since which yarp version? I think it's not in latest published nuget 1.1

[Tratcher]

This will be in the upcoming 2.0-rc.1 release.