Local dev error with https on .NET 7: Unsupported request method 'CONNECT'

[xabikos]

Hey @davidnx

Thank you very much for the awesome library you created. It's exactly what I needed for a new project I am creating.

I have a problem though when trying to serve the application over HTTPS. You can see the error when try to access the ActualNextjsApp sample as well through HTTPS https://localhost:5003.

If I get it right, it's something around WebSockets and how the middleware forwards the requests to Next's dev server. The error we see from the server side is

fail: Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware[1]
      An unhandled exception has occurred while executing the request.
      System.NotSupportedException: Unsupported request method 'CONNECT'.
         at Yarp.ReverseProxy.Forwarder.RequestUtilities.GetHttpMethod(String method)
         at Yarp.ReverseProxy.Forwarder.HttpForwarder.CreateRequestMessageAsync(HttpContext context, String destinationPrefix, HttpTransformer transformer, ForwarderRequestConfig requestConfig, Boolean i
sStreamingRequest, ActivityCancellationTokenSource activityToken)
         at Yarp.ReverseProxy.Forwarder.HttpForwarder.SendAsync(HttpContext context, String destinationPrefix, HttpMessageInvoker httpClient, ForwarderRequestConfig requestConfig, HttpTransformer transfo
rmer)
         at NextjsStaticHosting.AspNetCore.Internals.ProxyToDevServerMiddleware.InvokeAsync(HttpContext context) in C:\temp\NextjsStaticHosting-AspNetCore\src\NextjsStaticHosting.AspNetCore\Internals\Pro
xyToDevServerMiddleware.cs:line 44
         at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
         at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
         at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddlewareImpl.Invoke(HttpContext context)

And then in the console of the browser we have the following error:

The weird behavior is that when I am changing something in the page then it's updated dynamically without requiring a refresh.

I need to serve my application over HTTPS even for local development as it uses Identity Server as Identity Provider and once you get out of HTTPS then you have all kind of problems with Cookies policies.

I would love if you could provide some guidance on how to solve this.

[davidnx]

@xabikos HTTPS is supported out of the box in the sample; the issue you are facing is likely caused by (unusual?) network configurations on your machine.

The error (Unsupported request method 'CONNECT') is the smoking gun. The CONNECT http verb is used when communicating with a proxy (e.g. if you are on Windows, you'd configure this in Internet Options --> Connections tab --> LAN settings --> Proxy server section).

Your browser seems to be sending a CONNECT request to https://localhost:5003, and that isn't right. CONNECT is only valid when communicating with a proxy server, which your app is not (technically we are a reverse proxy when operating in dev mode, but that is fundamentally different from an http proxy as described in the HTTP specs for the CONNECT verb)

Please share your findings and I'll be curious to hear it. I will close this in a few days if I don't hear back, since the issue as stated isn't in scope of what I can help with, but until then I'll look forward to your comments. Best of luck!

[xabikos]

@davidnx thanks for the reply.

I spent some more time on the issue, and I couldn't spot the error. I am on Windows 11 and I am in my personal PC so I am administrator, and I don't have any special configuration around proxy or anything.

After your suggestions I tried the samples but this time I cloned the repo in WSL and then started it from there. It works as expected without the error.

Then I came back to mine solution in Visual Studio and I started experiment with various profiles. In all my application the profiles look like this

{
  "profiles": {
    "http": {
      "commandName": "Project",
      "launchBrowser": true,
      "launchUrl": "swagger",
      "environmentVariables": {
        "ASPNETCORE_ENVIRONMENT": "Development"
      },
      "dotnetRunMessages": true,
      "applicationUrl": "http://localhost:5000"
    },
    "https": {
      "commandName": "Project",
      "launchBrowser": true,
      "launchUrl": "swagger",
      "environmentVariables": {
        "ASPNETCORE_ENVIRONMENT": "Development"
      },
      "dotnetRunMessages": true,
      "applicationUrl": "https://localhost:7000;http://localhost:5000"
    },
    "IIS Express": {
      "commandName": "IISExpress",
      "launchBrowser": true,
      "launchUrl": "swagger",
      "environmentVariables": {
        "ASPNETCORE_ENVIRONMENT": "Development"
      }
    },
    "Docker": {
      "commandName": "Docker",
      "launchBrowser": true,
      "launchUrl": "{Scheme}://{ServiceHost}:{ServicePort}/swagger",
      "publishAllPorts": true,
      "useSSL": true
    }
  },
  "$schema": "https://json.schemastore.org/launchsettings.json",
  "iisSettings": {
    "windowsAuthentication": false,
    "anonymousAuthentication": true,
    "iisExpress": {
      "applicationUrl": "http://localhost:58000",
      "sslPort": 44343
    }
  }
}

and I have 2 projects to start at the same time. When I start with https as profile I get the error as described in the first post. When I select the IIS Express profile everything works as expected. I am not an expert on understanding every small detail on the various hosting models.

I am waiting to get my macbook as I would like to test it there as well. I just want to be sure that the entire backend platform will be starting in a predictable way in all platforms, so when a FE engineer is involved should be able to start the backend and then just do normal development in the client-side app.

For now I feel you could close the issue unless you have some other ideas of what could cause the error when using https profile which uses only Kestrel underneath if not mistaken.

[davidnx]

@xabikos finally figured this out -- the issue is due to incomplete support for WebSockets over HTTP/2. You are using .NET 7, and .NET 7 added support for WebSockets over HTTP/2, but YARP isn't leveraging that yet. You are running into issues because ALL of the following conditions are true:

1. Your browser is attempting to negotiate WebSockets over HTTP/2 (normal these days)
2. Your web app tries, but fails to, support this (Kestrel on .NET 7 accepts the initial extended CONNECT request (see the RFC), but ends up failing -- known issue WebSockets over HTTP/2 microsoft/reverse-proxy#1375
3. This library produces a 400 Bad Request as a result, instead of rejecting the request with HTTP_1_1_REQUIRED

Fix any of those and you will eliminate the problem. Some options you can explore:

• Disable HTTP/2 and use only HTTP/1.1, at least in local development. See: ListenOptions.Protocols and set to HttpProtocols.Http1
• This work-around: WebSockets over HTTP/2 microsoft/reverse-proxy#1375 (comment)
• Downgrade your web app to .NET 6 (i.e. change TargetFramework in your csproj to net6.0)
• Wait until YARP support this scenario, then I will update this library to consume it

I will leave this open to track this until we are consuming a version of YARP that support this cleanly.

Thanks for reporting, this was an interesting investigation and I ended up learning useful things as a result. Hope this helps you!

[xabikos]

Hello @davidnx

Thanks so much for coming back to me and the really detailed explanation and recommendations on how to fix it.

I understand the explanation and the problem. I have some questions though.

I only experience the problem when I am on Windows and using Kestrel. This doesn't happen when I am using IIS express to host my application. If not mistaken in this set up I am still using YARP, through this library, but not Kestrel. Probably because if IIS is there an automatic fallback to HTTP/1.1 instead of 2. Next to it when I started the app inside WSL on windows with Kestrel I didn't experience the error too.

I am a bit confused based why I am getting this behavior. In the meantime, I will just downgrade to HTTP/1.1 for local development as it's a simple solution and will allow me to continue the development without any problem.

I am happy I helped to find and clarify this issue and thanks again for the awesome library you created.

[davidnx]

@xabikos:

This doesn't happen when I am using IIS express to host my application

Right, IIS Express doesn't even attempt to handle WebSockets on HTTP/2, so it doesn't repro the issue (not sure if it uses HTTP/2 at all, really).

---

If not mistaken in this set up I am still using YARP, through this library

Yep.

---

when I started the app inside WSL on windows with Kestrel I didn't experience the error too

Yeah, that's surprising. I don't have time to set up a repro now, but if you still have this running, it would be interesting to see on your browser's DevTools what protocol it is using to connect to your WSL process. Make sure you enable the Protocol column in the Network panel and see what shows up.

I will close this as the issue is understood and a viable workaround is identified. Feel free to keep the conversation going in any case.

[davidnx]

YARP 2.0 is still not released, so until then I have temporarily added the workaround recommended in microsoft/reverse-proxy#1375 (comment). I will publish a new Nuget soon with the fix.

[xabikos]

Thanks so much @davidnx for following this. Much appreciated!