Performance Regression: UTF8 Span Methods 3x slower on AMD Platforms (NetCore2.1 -> 3.1)

[jamescourtney]

Performance of some UTF8 string methods seems to have regressed in a major way on AMD platforms in .NET Core 3.1. Others may be affected as well.

Test:

    [SimpleJob(BenchmarkDotNet.Jobs.RuntimeMoniker.NetCoreApp21)]
    [SimpleJob(BenchmarkDotNet.Jobs.RuntimeMoniker.NetCoreApp31)]
    public class Benchmark
    {
        const string testString = "hello world";
        private static readonly Encoding Utf8NoBom = new UTF8Encoding(false);

        private readonly byte[] writeBuffer = new byte[1024];
        private readonly byte[] readBuffer = Utf8NoBom.GetBytes(testString);

        [Benchmark]
        public void WriteString()
        {
            Utf8NoBom.GetBytes(testString.AsSpan(), this.writeBuffer.AsSpan());
        }

        [Benchmark]
        public void ReadString()
        {
            Utf8NoBom.GetString(this.readBuffer.AsSpan());
        }

        static void Main(string[] args)
        {
            BenchmarkRunner.Run<Benchmark>();
        }
    }

AMD system results:

BenchmarkDotNet=v0.12.0, OS=Windows 10.0.18363
AMD Ryzen 9 3900X, 1 CPU, 24 logical and 12 physical cores
.NET Core SDK=3.1.101
  [Host]     : .NET Core 3.1.1 (CoreCLR 4.700.19.60701, CoreFX 4.700.19.60801), X64 RyuJIT
  Job-RNGYVQ : .NET Core 2.1.15 (CoreCLR 4.6.28325.01, CoreFX 4.6.28327.02), X64 RyuJIT
  Job-ILXJIL : .NET Core 3.1.1 (CoreCLR 4.700.19.60701, CoreFX 4.700.19.60801), X64 RyuJIT


|      Method |       Runtime |     Mean |    Error |   StdDev |
|------------ |-------------- |---------:|---------:|---------:|
| WriteString | .NET Core 2.1 | 24.68 ns | 0.187 ns | 0.175 ns |
|  ReadString | .NET Core 2.1 | 23.33 ns | 0.373 ns | 0.349 ns |
| WriteString | .NET Core 3.1 | 70.51 ns | 0.787 ns | 0.736 ns |
|  ReadString | .NET Core 3.1 | 86.46 ns | 0.153 ns | 0.143 ns |

Intel system results:

BenchmarkDotNet=v0.12.0, OS=Windows 10.0.18363
Intel Xeon CPU E5-1650 v4 3.60GHz, 1 CPU, 12 logical and 6 physical cores
.NET Core SDK=3.1.101
  [Host]     : .NET Core 3.1.1 (CoreCLR 4.700.19.60701, CoreFX 4.700.19.60801), X64 RyuJIT
  Job-POUEZF : .NET Core 2.1.15 (CoreCLR 4.6.28325.01, CoreFX 4.6.28327.02), X64 RyuJIT
  Job-JZNDHJ : .NET Core 3.1.1 (CoreCLR 4.700.19.60701, CoreFX 4.700.19.60801), X64 RyuJIT


|      Method |       Runtime |     Mean |    Error |   StdDev |
|------------ |-------------- |---------:|---------:|---------:|
| WriteString | .NET Core 2.1 | 34.38 ns | 0.709 ns | 1.204 ns |
|  ReadString | .NET Core 2.1 | 31.24 ns | 0.652 ns | 1.376 ns |
| WriteString | .NET Core 3.1 | 13.61 ns | 0.301 ns | 0.519 ns |
|  ReadString | .NET Core 3.1 | 29.86 ns | 0.623 ns | 1.340 ns |

[scalablecory]

@GrabYourPitchforks

[GrabYourPitchforks]

@tannergooding Is the Ryzen CPU listed here one of the models that suffers from poor pdep / pext performance? Those instructions can occur on the ASCII processing code paths.

I can investigate using some of the SIMD instructions here where possible (packuswb and the like), but it's almost certainly going to increase the latency in these code paths.

[jamescourtney]

That's really interesting, actually. I've learned something today!

Based on my reading, it looks as if this instruction is implemented in microcode on my chip, while Intel has a hardware implementation for it. Also understand why the CLR wouldn't want to be in the business of micro optimizations like this for different architectures.

Running a quick experiment using the Kernel32 APIs for MultiByteToWideChar were slightly quicker, but still not as fast as NetCore2.1.

Thanks for the responses here.

[tannergooding]

Yes. pdep/pext are implemented in Microcode for all zen microarchitectures.

You can see this in the x86 instruction scheduling model that AMD contributed for LLVM:

• Znver1 - https://github.com/llvm/llvm-project/blob/master/llvm/lib/Target/X86/X86ScheduleZnver1.td#L748-L752
• Znver2 - https://github.com/llvm/llvm-project/blob/master/llvm/lib/Target/X86/X86ScheduleZnver2.td#L748-L752

WriteMicrocoded is defaulted to Latency = 100: https://github.com/llvm/llvm-project/blob/master/llvm/lib/Target/X86/X86ScheduleZnver1.td#L487-L490.
Nearby comments indicate "We set long latency so as to block the entire pipeline."

This might be a good first usecase for #785. The savings on platforms with fast pdep/pext are significant, but so are the drawbacks for platforms with slow versions.

[GrabYourPitchforks]

@jamescourtney - Can you set the environment variable COMPlus_EnableBMI2=0 and see if that improves your performance? That will disable the BMI2-specific instructions that we use as an optimization.

(Tanner informs me that the environment variable processing logic is case-sensitive on Linux. It shouldn't matter on Windows.)

[jamescourtney]

Setting that environment variable resolves the issue completely. Write performance more than doubles over NetCore2.1:

BenchmarkDotNet=v0.12.0, OS=Windows 10.0.18363
AMD Ryzen 9 3900X, 1 CPU, 24 logical and 12 physical cores
.NET Core SDK=3.1.101
  [Host]     : .NET Core 3.1.1 (CoreCLR 4.700.19.60701, CoreFX 4.700.19.60801), X64 RyuJIT
  Job-YGYXSM : .NET Core 2.1.15 (CoreCLR 4.6.28325.01, CoreFX 4.6.28327.02), X64 RyuJIT
  Job-LLDHGN : .NET Core 3.1.1 (CoreCLR 4.700.19.60701, CoreFX 4.700.19.60801), X64 RyuJIT


|      Method |       Runtime |      Mean |     Error |    StdDev |
|------------ |-------------- |----------:|----------:|----------:|
| WriteString | .NET Core 2.1 | 24.480 ns | 0.2870 ns | 0.2684 ns |
|  ReadString | .NET Core 2.1 | 24.004 ns | 0.2095 ns | 0.1960 ns |
| WriteString | .NET Core 3.1 |  9.832 ns | 0.0429 ns | 0.0358 ns |
|  ReadString | .NET Core 3.1 | 23.654 ns | 0.1812 ns | 0.1606 ns |

[damageboy]

Just adding a small something to this discussion.
While it's early in my morning and I'm too groggy to actually look at the code calling PDEP...
If we had something like CPUID and willingness to use specific code paths depending on CPUID @ runtime, some PDEPs can be converted rather efficiently to beat AMD's microcode:

https://github.com/zwegner/zp7

[jkotas]

CPUID and willingness to use specific code paths depending on CPUID

The model for the hardware intrinsic is complex already. This would make it even more complex than what it is today.

My suggestion for how to fix this issue would be to make Bmi2.IsSupported false on AMD64 processors. Once/if the AMD64 processors get this performance bug fixed, we can update the runtime to detect the versions with the fix and start returning true from this method again.

[GrabYourPitchforks]

I'm torn on whether the runtime lying about the intrinsic being supported is the correct approach. That seems put the runtime into the position of making a value judgment. Up until now the intrinsics have been only raw capabilities checks.

[jkotas]

This should be a point in time problem. I expect AMD to fix this performance bug and then all this will be legacy that I everybody will try to forget about.

The reason why I have proposed implementing the workaround in the runtime is that it will be done in just one central place instead of spread over all places that use this instruction. As with any workaround, there may be corner cases where it may not be perfect, but that's ok.