[HTTP/3] Support for HTTP/3 multiple connections by ManickaP · Pull Request #101531 · dotnet/runtime

ManickaP · 2024-04-25T08:29:49Z

Feel free to review and discuss the API changes. This is NO MERGE until we have the APIs approved and all the TODOs resolved

The implementation takes H/2 connection pooling and slightly adjusts it for H/3 specific behaviors (see a diff between HttpConnectionPool.Http2.cs and HttpConnectionPool.Http3.cs).

The PR depends on 2 API reviews:

[API Proposal]: [HTTP/3] Support multiple HTTP/3 connections #101535: for EnableMultipleHttp3Connections
[API Proposal]: [QUIC] Support managing available streams on QuicConnection #101534: for QuicConnection changes

TODO: Add numbers from benchmarks when it runs
TODO: Stress run depends on dotnet/aspnetcore#55282

Fixes #51775
Fixes #54968
Fixes #68380
Resolves #101535
Resolves #101534

ghost · 2024-04-25T08:29:54Z

Note regarding the new-api-needs-documentation label:

This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

ManickaP · 2024-04-25T10:43:04Z

No noticeable regression on perf from main:

main

1x10000 (clients x threads)

crank --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/build/ci.profile.yml --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/scenarios/httpclient.benchmarks.yml --client.framework net9.0 --server.framework net9.0 --scenario httpclient-kestrel-get --profile intel-lin-app --profile amd-lin2-load --variable useHttpMessageInvoker=true --variable httpVersion=3.0 --variable useHttps=true --variable responseSize=256 --variable numberOfHttpClients=1 --variable concurrencyPerHttpClient=10000

| client                      |                                       |
| --------------------------- | ------------------------------------- |
| Max CPU Usage (%)           | 35                                    |
| Max Cores usage (%)         | 971                                   |
| Max Working Set (MB)        | 3,810                                 |
| Max Private Memory (MB)     | 4,415                                 |
| Build Time (ms)             | 6,988                                 |
| Start Time (ms)             | 0                                     |
| Published Size (KB)         | 74,115                                |
| Symbols Size (KB)           | 19                                    |
| .NET Core SDK Version       | 9.0.100-preview.5.24224.9             |
| ASP.NET Core Version        | 9.0.0-preview.4.24223.1+f18510c2fbdf  |
| .NET Runtime Version        | 9.0.0-preview.4.24223.11+d92ac1f892a7 |
| Processor Count             | 28                                    |
| First request duration (ms) | 519                                   |
| Requests                    | 345,446                               |
| Bad Status Code Requests    | 0                                     |
| Exceptions                  | 0                                     |
| Mean RPS                    | 23,069                                |

1x100 (clients x threads)

crank --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/build/ci.profile.yml --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/scenarios/httpclient.benchmarks.yml --client.framework net9.0 --server.framework net9.0 --scenario httpclient-kestrel-get --profile intel-lin-app --profile amd-lin2-load --variable useHttpMessageInvoker=true --variable httpVersion=3.0 --variable useHttps=true --variable responseSize=256 --variable numberOfHttpClients=1 --variable concurrencyPerHttpClient=100

| client                      |                                       |
| --------------------------- | ------------------------------------- |
| Max CPU Usage (%)           | 41                                    |
| Max Cores usage (%)         | 1,150                                 |
| Max Working Set (MB)        | 577                                   |
| Max Private Memory (MB)     | 1,119                                 |
| Build Time (ms)             | 4,926                                 |
| Start Time (ms)             | 0                                     |
| Published Size (KB)         | 74,115                                |
| Symbols Size (KB)           | 19                                    |
| .NET Core SDK Version       | 9.0.100-preview.5.24225.1             |
| ASP.NET Core Version        | 9.0.0-preview.4.24223.1+f18510c2fbdf  |
| .NET Runtime Version        | 9.0.0-preview.4.24223.11+d92ac1f892a7 |
| Processor Count             | 28                                    |
| First request duration (ms) | 520                                   |
| Requests                    | 989,468                               |
| Bad Status Code Requests    | 0                                     |
| Exceptions                  | 0                                     |
| Mean RPS                    | 65,955                                |

20x500 (clients x threads)

crank --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/build/ci.profile.yml --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/scenarios/httpclient.benchmarks.yml --client.framework net9.0 --server.framework net9.0 --scenario httpclient-kestrel-get --profile intel-lin-app --profile amd-lin2-load --variable useHttpMessageInvoker=true --variable httpVersion=3.0 --variable useHttps=true --variable responseSize=256 --variable numberOfHttpClients=20 --variable concurrencyPerHttpClient=500

| client                      |                                       |
| --------------------------- | ------------------------------------- |
| Max CPU Usage (%)           | 97                                    |
| Max Cores usage (%)         | 2,710                                 |
| Max Working Set (MB)        | 6,526                                 |
| Max Private Memory (MB)     | 7,268                                 |
| Build Time (ms)             | 2,544                                 |
| Start Time (ms)             | 0                                     |
| Published Size (KB)         | 74,115                                |
| Symbols Size (KB)           | 19                                    |
| .NET Core SDK Version       | 9.0.100-preview.5.24225.1             |
| ASP.NET Core Version        | 9.0.0-preview.4.24223.1+f18510c2fbdf  |
| .NET Runtime Version        | 9.0.0-preview.4.24223.11+d92ac1f892a7 |
| Processor Count             | 28                                    |
| First request duration (ms) | 518                                   |
| Requests                    | 4,976,883                             |
| Bad Status Code Requests    | 0                                     |
| Exceptions                  | 0                                     |
| Mean RPS                    | 331,779                               |

PR, multiple H3 connections OFF

1x10000 (clients x threads)

crank --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/build/ci.profile.yml --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/scenarios/httpclient.benchmarks.yml --client.framework net9.0 --server.framework net9.0 --scenario httpclient-kestrel-get --profile intel-lin-app --profile amd-lin2-load --variable useHttpMessageInvoker=true --variable httpVersion=3.0 --variable useHttps=true --variable responseSize=256 --variable numberOfHttpClients=1 --variable concurrencyPerHttpClient=10000 --client.options.outputFiles artifacts/bin/testhost/net9.0-linux-Release-x64/shared/Microsoft.NETCore.App/9.0.0/System.Net.Http.dll --client.options.outputFiles  artifacts/bin/testhost/net9.0-linux-Release-x64/shared/Microsoft.NETCore.App/9.0.0/System.Net.Quic.dll

| client                      |                                       |
| --------------------------- | ------------------------------------- |
| Max CPU Usage (%)           | 36                                    |
| Max Cores usage (%)         | 998                                   |
| Max Working Set (MB)        | 3,772                                 |
| Max Private Memory (MB)     | 4,371                                 |
| Build Time (ms)             | 2,866                                 |
| Start Time (ms)             | 0                                     |
| Published Size (KB)         | 74,115                                |
| Symbols Size (KB)           | 19                                    |
| .NET Core SDK Version       | 9.0.100-preview.5.24225.1             |
| ASP.NET Core Version        | 9.0.0-preview.4.24223.1+f18510c2fbdf  |
| .NET Runtime Version        | 9.0.0-preview.4.24223.11+d92ac1f892a7 |
| Processor Count             | 28                                    |
| First request duration (ms) | 596                                   |
| Requests                    | 342,841                               |
| Bad Status Code Requests    | 0                                     |
| Exceptions                  | 0                                     |
| Mean RPS                    | 22,914                                |

1x100 (clients x threads)

crank --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/build/ci.profile.yml --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/scenarios/httpclient.benchmarks.yml --client.framework net9.0 --server.framework net9.0 --scenario httpclient-kestrel-get --profile intel-lin-app --profile amd-lin2-load --variable useHttpMessageInvoker=true --variable httpVersion=3.0 --variable useHttps=true --variable responseSize=256 --variable numberOfHttpClients=1 --variable concurrencyPerHttpClient=100 --client.options.outputFiles artifacts/bin/testhost/net9.0-linux-Release-x64/shared/Microsoft.NETCore.App/9.0.0/System.Net.Http.dll --client.options.outputFiles  artifacts/bin/testhost/net9.0-linux-Release-x64/shared/Microsoft.NETCore.App/9.0.0/System.Net.Quic.dl

| client                      |                                       |
| --------------------------- | ------------------------------------- |
| Max CPU Usage (%)           | 46                                    |
| Max Cores usage (%)         | 1,296                                 |
| Max Working Set (MB)        | 587                                   |
| Max Private Memory (MB)     | 1,174                                 |
| Build Time (ms)             | 2,897                                 |
| Start Time (ms)             | 0                                     |
| Published Size (KB)         | 74,115                                |
| Symbols Size (KB)           | 19                                    |
| .NET Core SDK Version       | 9.0.100-preview.5.24225.1             |
| ASP.NET Core Version        | 9.0.0-preview.4.24223.1+f18510c2fbdf  |
| .NET Runtime Version        | 9.0.0-preview.4.24223.11+d92ac1f892a7 |
| Processor Count             | 28                                    |
| First request duration (ms) | 594                                   |
| Requests                    | 985,119                               |
| Bad Status Code Requests    | 0                                     |
| Exceptions                  | 0                                     |
| Mean RPS                    | 65,691                                |

20x500 (clients x threads)

rank --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/build/ci.profile.yml --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/scenarios/httpclient.benchmarks.yml --client.framework net9.0 --server.framework net9.0 --scenario httpclient-kestrel-get --profile intel-lin-app --profile amd-lin2-load --variable useHttpMessageInvoker=true --variable httpVersion=3.0 --variable useHttps=true --variable responseSize=256 --variable numberOfHttpClients=20 --variable concurrencyPerHttpClient=500 --client.options.outputFiles artifacts/bin/testhost/net9.0-linux-Release-x64/shared/Microsoft.NETCore.App/9.0.0/System.Net.Http.dll --client.options.outputFiles  artifacts/bin/testhost/net9.0-linux-Release-x64/shared/Microsoft.NETCore.App/9.0.0/System.Net.Quic.dll

| client                      |                                       |
| --------------------------- | ------------------------------------- |
| Max CPU Usage (%)           | 95                                    |
| Max Cores usage (%)         | 2,667                                 |
| Max Working Set (MB)        | 6,494                                 |
| Max Private Memory (MB)     | 7,246                                 |
| Build Time (ms)             | 2,660                                 |
| Start Time (ms)             | 0                                     |
| Published Size (KB)         | 74,115                                |
| Symbols Size (KB)           | 19                                    |
| .NET Core SDK Version       | 9.0.100-preview.5.24225.1             |
| ASP.NET Core Version        | 9.0.0-preview.4.24223.1+f18510c2fbdf  |
| .NET Runtime Version        | 9.0.0-preview.4.24223.11+d92ac1f892a7 |
| Processor Count             | 28                                    |
| First request duration (ms) | 594                                   |
| Requests                    | 5,334,724                             |
| Bad Status Code Requests    | 0                                     |
| Exceptions                  | 0                                     |
| Mean RPS                    | 355,707                               |

PR, multiple H3 connections ON

1x10000 (clients x threads)

crank --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/build/ci.profile.yml --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/scenarios/httpclient.benchmarks.yml --client.framework net9.0 --server.framework net9.0 --scenario httpclient-kestrel-get --profile intel-lin-app --profile amd-lin2-load --variable useHttpMessageInvoker=true --variable httpVersion=3.0 --variable useHttps=true --variable responseSize=256 --variable numberOfHttpClients=1 --variable concurrencyPerHttpClient=10000 --client.options.outputFiles artifacts/bin/testhost/net9.0-linux-Release-x64/shared/Microsoft.NETCore.App/9.0.0/System.Net.Http.dll --client.options.outputFiles  artifacts/bin/testhost/net9.0-linux-Release-x64/shared/Microsoft.NETCore.App/9.0.0/System.Net.Quic.dll

| client                      |                                      |
| --------------------------- | ------------------------------------ |
| Max CPU Usage (%)           | 92                                   |
| Max Cores usage (%)         | 2,572                                |
| Max Working Set (MB)        | 6,491                                |
| Max Private Memory (MB)     | 7,228                                |
| Build Time (ms)             | 2,736                                |
| Start Time (ms)             | 0                                    |
| Published Size (KB)         | 74,777                               |
| Symbols Size (KB)           | 19                                   |
| .NET Core SDK Version       | 9.0.100-preview.5.24272.19           |
| ASP.NET Core Version        | 9.0.0-preview.6.24272.7+49c1c68bf1ac |
| .NET Runtime Version        | 9.0.0-preview.5.24272.2+5c06e5d01fa0 |
| Processor Count             | 28                                   |
| First request duration (ms) | 594                                  |
| Requests                    | 4,325,325                            |
| Bad Status Code Requests    | 0                                    |
| Exceptions                  | 0                                    |
| Mean RPS                    | 288,664                              |

src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/Http3Connection.cs

src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/SocketsHttpHandler.cs

src/libraries/System.Net.Quic/src/System/Net/Quic/QuicConnection.cs

src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/ConnectHelper.cs

.../System.Net.Http/src/System/Net/Http/SocketsHttpHandler/ConnectionPool/HttpConnectionPool.cs

src/libraries/System.Net.Quic/src/System/Net/Quic/QuicConnection.cs

...m.Net.Http/src/System/Net/Http/SocketsHttpHandler/ConnectionPool/HttpConnectionPool.Http3.cs

ManickaP · 2024-06-20T08:29:51Z

/azp run runtime-libraries stress-http

azure-pipelines · 2024-06-20T08:30:02Z

Azure Pipelines successfully started running 1 pipeline(s).

ManickaP · 2024-06-20T08:30:07Z

/azp run runtime-libraries-coreclr outerloop

azure-pipelines · 2024-06-20T08:30:19Z

Azure Pipelines successfully started running 1 pipeline(s).

rzikm · 2024-06-20T11:28:03Z

src/libraries/System.Net.Quic/src/System/Net/Quic/QuicConnection.cs

+    private async void OnStreamCapacityIncreased(int bidirectionalIncrement, int unidirectionalIncrement)
+    {
+        // Bail out early to avoid queueing work on the thread pool as well as event args instantiation.
+        if (_streamCapacityCallback is null)
+        {
+            return;
+        }


Will a call to this function allocate a state machine even when no callbacks are registered? Maybe we can delegate the latter half of the function to nested private helper function.

It will "allocate" the state machine struct on the stack until the first await that yields.
In this case if _streamCapacityCallback is null, it won't allocate anything on the heap.

So I get that it's not worth any change 😉

src/libraries/System.Net.Quic/src/System/Net/Quic/QuicStream.cs

ManickaP · 2024-06-25T07:30:53Z

/azp run runtime-libraries stress-http

azure-pipelines · 2024-06-25T07:31:02Z

Azure Pipelines successfully started running 1 pipeline(s).

ManickaP · 2024-06-25T07:31:11Z

/azp run runtime-libraries-coreclr outerloop

azure-pipelines · 2024-06-25T07:31:19Z

Azure Pipelines successfully started running 1 pipeline(s).

ManickaP · 2024-06-25T10:58:32Z

If the pipeline gods grant this PR green, I'll merge this. Last chance to scream any feedback 😄

ManickaP · 2024-06-25T11:02:07Z

/azp run runtime-libraries-coreclr outerloop

azure-pipelines · 2024-06-25T11:02:20Z

Azure Pipelines successfully started running 1 pipeline(s).

ManickaP · 2024-06-25T11:02:53Z

/azp run runtime-libraries stress-http

azure-pipelines · 2024-06-25T11:03:02Z

Azure Pipelines successfully started running 1 pipeline(s).

ghost added area-System.Net.Http new-api-needs-documentation labels Apr 25, 2024

dotnet-policy-service bot assigned ManickaP Apr 25, 2024

ManickaP added the NO-MERGE The PR is not ready for merge yet (see discussion for detailed reasons) label Apr 25, 2024

ManickaP requested a review from a team April 25, 2024 08:30

ManickaP force-pushed the h3-mul-conn branch from 97fdf7a to e193fce Compare April 25, 2024 08:32

ManickaP mentioned this pull request Apr 25, 2024

[API Proposal]: [QUIC] Support managing available streams on QuicConnection #101534

Closed

rzikm reviewed Apr 25, 2024

View reviewed changes

MihaZupan reviewed Apr 25, 2024

View reviewed changes

ManickaP force-pushed the h3-mul-conn branch from 06fab49 to c9ef439 Compare May 14, 2024 12:29

This was referenced May 14, 2024

tracing/eventpipe/providervalidation/providervalidation/providervalidation.cmd test failure #101759

Closed

Intermittent build failure in AfterSourceBuild: "Could not write state file" #76488

Open

ManickaP force-pushed the h3-mul-conn branch from 71869db to 3b21bea Compare May 16, 2024 09:21

ManickaP added 5 commits June 19, 2024 10:37

Generated ref source

114e89a

SupportedOS ordering, because I cannot stand it different

01e75dc

Idle connection back where it was

fd285fe

Assert

7f6e3c4

Fix handling negative stream capacity and 0 reporting STREAMS_AVAILABLE

2710d18

This was referenced Jun 19, 2024

Build on Windows Fails sometimes with fatal error C1090: PDB API call failed #48070

Open

STJ NullPropertyNameFail test failing in CI #103715

Closed

rzikm reviewed Jun 20, 2024

View reviewed changes

src/libraries/System.Net.Quic/src/System/Net/Quic/QuicStream.cs Show resolved Hide resolved

Handling cummulative increments, canceled streams and more tests.

346a278

Removed logging

ede8407

This was referenced Jun 25, 2024

Failed USB connection via port 54050, error 61, in tvOS arm64 Release AllSubsets_Mono #82637

Closed

[browser] HalfTests.ExplicitConversion_FromSingle failing due to NaN != NaN #103347

Open

ManickaP merged commit c14e5dd into dotnet:main Jun 25, 2024

ManickaP deleted the h3-mul-conn branch June 25, 2024 14:46

ManickaP mentioned this pull request Jun 27, 2024

Add option to enable multuple H/3 connections aspnet/Benchmarks#1998

Merged

github-actions bot locked and limited conversation to collaborators Jul 26, 2024

Conversation

ManickaP commented Apr 25, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ghost commented Apr 25, 2024

Uh oh!

ManickaP commented Apr 25, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

main

PR, multiple H3 connections OFF

PR, multiple H3 connections ON

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ManickaP commented Jun 20, 2024

Uh oh!

azure-pipelines bot commented Jun 20, 2024

Uh oh!

ManickaP commented Jun 20, 2024

Uh oh!

azure-pipelines bot commented Jun 20, 2024

Uh oh!

rzikm Jun 20, 2024

Choose a reason for hiding this comment

Uh oh!

MihaZupan Jun 20, 2024

Choose a reason for hiding this comment

Uh oh!

ManickaP Jun 25, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ManickaP commented Jun 25, 2024

Uh oh!

azure-pipelines bot commented Jun 25, 2024

Uh oh!

ManickaP commented Jun 25, 2024

Uh oh!

azure-pipelines bot commented Jun 25, 2024

Uh oh!

ManickaP commented Jun 25, 2024

Uh oh!

ManickaP commented Jun 25, 2024

Uh oh!

azure-pipelines bot commented Jun 25, 2024

Uh oh!

ManickaP commented Jun 25, 2024

Uh oh!

azure-pipelines bot commented Jun 25, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ManickaP commented Apr 25, 2024 •

edited

Loading

ManickaP commented Apr 25, 2024 •

edited

Loading