Getting vxsort working on Linux amd64 by cshung · Pull Request #98712 · dotnet/runtime

cshung · 2024-02-20T18:55:29Z

This change make vxsort available for other x64 platforms other than Windows. Overall, the algorithm improved the pause time for ephemeral GCs by a few percents. Attached below are the numbers on Linux running GCPerfSim.

This change will increase binary size by a about 130k because VxSort is a lot of code. For NativeAOT, I chose to follow the Windows policy to enable it by default and provided the same opt-out.

It is also tested under Rosetta emulation to show that it will work even on machines without AVX instructions. Of course, VxSort will not be available there and we will use the introsort fallback.

Special thanks to @janvorli for helping with the compilation errors when I was stuck trying to fix them.

Summary

Name	ExecutionTime (MSec) - base	comparand	Δ%	% GC Pause Time - base	comparand	Δ%
normal	25,617.28	25,068.92	-2.14	27.19	25.85	-4.94
soh_pinning	31,713.70	30,949.50	-2.41	36.15	35.95	-0.54
poh	25,309.35	25,176.36	-0.53	26.74	26.18	-2.09
loh	26,479.87	26,054.81	-1.61	23.63	22.68	-4.02

Name	Mean Heap Size Before (MB) - base	comparand	Δ%
normal	6,654.48	6,522.71	-1.98
soh_pinning	7,850.94	7,863.67	0.16
poh	6,715.51	6,641.29	-1.11
loh	6,841.37	6,872.47	0.45

Name	Mean Ephemeral Pause (MSec) - base	comparand	Δ%
normal	27.79	25.98	-6.52
soh_pinning	15.68	15.21	-3.01
poh	27.14	26.37	-2.84
loh	26.31	24.55	-6.69

ghost · 2024-02-20T18:55:41Z

Tagging subscribers to this area: @dotnet/gc
See info in area-owners.md if you want to be subscribed.

Issue Details

null

Author:	cshung
Assignees:	cshung
Labels:	`area-GC-coreclr`
Milestone:	-

am11 · 2024-03-10T21:35:48Z

The increase in crossgen2 binary size is 133 KB on linux-x64 and 115 KB on osx-x64 (without the last commit). If the performance gain is worth the additional size, can revert 786e0aa and adjust the size test:

runtime/src/tests/nativeaot/SmokeTests/HardwareIntrinsics/Program.cs

Line 25 in 82dac28

upperBound = 1750 * 1024; // ~1.75 MB

1750 -> 1900

mangod9 · 2024-03-11T01:04:06Z

why is this affecting crossgen2 binary sizes? Is this changing any JIT behavior?

cshung · 2024-03-11T01:07:01Z

why is this affecting crossgen2 binary sizes? Is this changing any JIT behavior?

This change does not change any JIT behavior, it is just that the code for vxsort has a non-trivial size.

src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Unix.targets

src/coreclr/gc/gc.cpp

src/coreclr/gc/gcsvr.cpp

src/coreclr/gc/gcwks.cpp

src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Unix.targets

src/coreclr/inc/warning_macros.h

am11 · 2024-03-28T22:41:36Z

One more am11@0dd4664 (I was testing on arm64 machine earlier 😅).

src/native/minipal/utils.h

janvorli

LGTM, thank you!

Co-authored-by: Jan Vorlicek <janvorli@microsoft.com> Co-authored-by: Adeel Mujahid <3840695+am11@users.noreply.github.com>

…code` when building on debug mode on loongarch64/arm64/amd64-linux by clang<=13. Also delete `#ifndef FALLTHROUGH` block as dotnet#98336 added after dotnet#98712.

…code` when building on debug mode on loongarch64/arm64/amd64-linux by clang<14. Also delete `#ifndef FALLTHROUGH` block as dotnet#98336 added after dotnet#98712.

…code` (#100742) when building on debug mode on loongarch64/arm64/amd64-linux by clang<14. Also delete `#ifndef FALLTHROUGH` block as #98336 added after #98712.

Co-authored-by: Jan Vorlicek <janvorli@microsoft.com> Co-authored-by: Adeel Mujahid <3840695+am11@users.noreply.github.com>

…code` (dotnet#100742) when building on debug mode on loongarch64/arm64/amd64-linux by clang<14. Also delete `#ifndef FALLTHROUGH` block as dotnet#98336 added after dotnet#98712.

ghost assigned cshung Feb 20, 2024

cshung marked this pull request as draft February 20, 2024 18:55

ghost added the area-GC-coreclr label Feb 20, 2024

cshung force-pushed the public/linux-vxsort branch from 6b2ebc9 to 7dd898f Compare February 20, 2024 19:44

build-analysis bot mentioned this pull request Feb 26, 2024

Checkout failure: "Git fetch failed with exit code 128" dotnet/arcade#9009

Open

2 tasks

cshung force-pushed the public/linux-vxsort branch from 1abc327 to ce18101 Compare March 9, 2024 18:50

build-analysis bot mentioned this pull request Mar 10, 2024

Compiler crashes when failing to release Mutex #53420

Open

cshung marked this pull request as ready for review March 18, 2024 19:39

cshung requested a review from MichalStrehovsky as a code owner March 18, 2024 19:39

MichalStrehovsky reviewed Mar 19, 2024

View reviewed changes

src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Unix.targets Show resolved Hide resolved

cshung force-pushed the public/linux-vxsort branch from 7d93e88 to 55d788d Compare March 19, 2024 16:14

This was referenced Mar 19, 2024

Tracking issue for CI build timeouts #76454

Closed

RestApiException`1: The response contained an invalid status code 500 Internal Server Error dotnet/dnceng#2298

Closed

Maoni0 reviewed Mar 19, 2024

View reviewed changes

src/coreclr/gc/gc.cpp Outdated Show resolved Hide resolved

Maoni0 reviewed Mar 19, 2024

View reviewed changes

src/coreclr/gc/gcsvr.cpp Outdated Show resolved Hide resolved

Maoni0 reviewed Mar 19, 2024

View reviewed changes

src/coreclr/gc/gcwks.cpp Outdated Show resolved Hide resolved

cshung force-pushed the public/linux-vxsort branch 2 times, most recently from b555daf to fdcf757 Compare March 20, 2024 18:06

build-analysis bot mentioned this pull request Mar 20, 2024

Assert failure(PID 13812 [0x000035f4], Thread: 14128 [0x3730]): promoted_bytes (heap_number) == promoted #100035

Closed

MichalStrehovsky mentioned this pull request Mar 21, 2024

Trim unused interfaces #100000

Merged

jkotas reviewed Mar 21, 2024

View reviewed changes

src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Unix.targets Show resolved Hide resolved

cshung force-pushed the public/linux-vxsort branch from 5d0404a to 19e8e53 Compare March 28, 2024 21:17

am11 reviewed Mar 28, 2024

View reviewed changes

src/coreclr/inc/warning_macros.h Outdated Show resolved Hide resolved

am11 reviewed Mar 29, 2024

View reviewed changes

src/native/minipal/utils.h Outdated Show resolved Hide resolved

cshung force-pushed the public/linux-vxsort branch from 6b5b725 to 2046a84 Compare March 29, 2024 22:35

build-analysis bot mentioned this pull request Mar 30, 2024

[browser][mt] SignalRClientTests timeouts #100388

Closed

cshung mentioned this pull request Apr 1, 2024

The check "runtime (Build browser wasm li...) was stuck for 2 days with blank error message. dotnet/dnceng#2410

Closed

3 tasks

janvorli approved these changes Apr 2, 2024

View reviewed changes

Getting vxsort working on Linux amd64

68cf2dd

Co-authored-by: Jan Vorlicek <janvorli@microsoft.com> Co-authored-by: Adeel Mujahid <3840695+am11@users.noreply.github.com>

cshung force-pushed the public/linux-vxsort branch from 2046a84 to 68cf2dd Compare April 3, 2024 16:04

cshung merged commit c08bd7b into dotnet:main Apr 3, 2024

cshung deleted the public/linux-vxsort branch April 3, 2024 20:27

build-analysis bot mentioned this pull request Apr 3, 2024

System.Net.Http.Functional.Tests.HttpMetricsTest_Http11_Async_HttpMessageInvoker Test failure #96407

Closed

matouskozak mentioned this pull request Apr 4, 2024

[NAOT][Apple][mobile] Undefined symbols for architecture x86_64: do_vxsort_avx2 causing AppleAppBuilderTask to fail #100632

Closed

matouskozak pushed a commit to matouskozak/runtime that referenced this pull request Apr 30, 2024

Getting vxsort working on Linux amd64 (dotnet#98712)

7932f01

Co-authored-by: Jan Vorlicek <janvorli@microsoft.com> Co-authored-by: Adeel Mujahid <3840695+am11@users.noreply.github.com>

github-actions bot locked and limited conversation to collaborators May 4, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Getting vxsort working on Linux amd64#98712

Getting vxsort working on Linux amd64#98712
cshung merged 1 commit intodotnet:mainfrom
cshung:public/linux-vxsort

cshung commented Feb 20, 2024 •

edited

Loading

Uh oh!

ghost commented Feb 20, 2024

Uh oh!

am11 commented Mar 10, 2024

Uh oh!

mangod9 commented Mar 11, 2024

Uh oh!

cshung commented Mar 11, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

am11 commented Mar 28, 2024

Uh oh!

Uh oh!

janvorli left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Comments

Conversation

cshung commented Feb 20, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

ghost commented Feb 20, 2024

Uh oh!

am11 commented Mar 10, 2024

Uh oh!

mangod9 commented Mar 11, 2024

Uh oh!

cshung commented Mar 11, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

am11 commented Mar 28, 2024

Uh oh!

Uh oh!

janvorli left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Comments

cshung commented Feb 20, 2024 •

edited

Loading