JIT: Intrinsify UTF16->UTF8 conversion for string literals (Encoding.UTF8.TryGetBytes)

[EgorBo]

Closes #84659
The main goal is to make upcoming UTF8 string interpolation faster (to be on par with UTF16 one). If JIT sees that the source is a string literal (can be extended to cover RVA ROS<char>, "shifted" pointers to literal literals, static readonly string) and the data in it is ASCII only - it peforms the conversion in the JIT time and just copies it to the destination.

Benchmark:

byte[] _outputBuffer = new byte[1024];

[Benchmark]
public int TryGetBytes_const()
{
    Encoding.UTF8.TryGetBytes("hello", _outputBuffer.AsSpan(), out int written1);
    return written1;
}

Method	Toolchain	Mean	Ratio
TryGetBytes_const	\Core_Root\corerun.exe	0.3435 ns	0.06
TryGetBytes_const	\Core_Root_base\corerun.exe	5.4439 ns	1.00

cc @stephentoub

[ghost]

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Closes #84659
The main goal is to make upcoming UTF8 string interpolation faster (to be on par with UTF16 one). If JIT sees that the source is a string literal (can be extended to cover RVA ROS, "shifted" pointers to literal literals, static readonly string) and the data in it is ASCII only - it peforms the conversion in the JIT time and just copies it to the destination.

Benchmark:

byte[] _outputBuffer = new byte[1024];

[Benchmark]
public int TryGetBytes_2B()
{
    Encoding.UTF8.TryGetBytes("Hi", _outputBuffer.AsSpan(), out int written); 
    return written;
}

[Benchmark]
public int TryGetBytes_27B()
{
    Encoding.UTF8.TryGetBytes("MemoryMarshal.GetReference", _outputBuffer.AsSpan(), out int written); 
    return written;
}

[Benchmark]
public int TryGetBytes_122B()
{
    Encoding.UTF8.TryGetBytes(
        ".NET Runtime uses third-party libraries or other resources" +
        "that may be distributed under licenses different than the .NET", 
        _outputBuffer.AsSpan(), out int written); 
    return written;
}

Method	Toolchain	Mean
TryGetBytes_2B	\runtime-PR\corerun.exe	0.7416 ns
TryGetBytes_2B	\runtime\corerun.exe	4.9567 ns
TryGetBytes_27B	\runtime-PR\corerun.exe	0.8916 ns
TryGetBytes_27B	\runtime\corerun.exe	6.5213 ns
TryGetBytes_122B	\runtime-PR\corerun.exe	1.6277 ns
TryGetBytes_122B	\runtime\corerun.exe	7.9679 ns

With DynamicPGO=1, results are worse in the default mode (up to 10ns)

codegen diff for TryGetBytes_122B: https://www.diffchecker.com/0udd0z9q/
The actual copying is done via:

;; length check
vmovups  zmm0, zmmword ptr [reloc @RWD00]
vmovups  zmmword ptr [r9], zmm0
vmovups  zmm0, zmmword ptr [reloc @RWD64]
vmovups  zmmword ptr [r9+38H], zmm0
mov      eax, 120 ; bytes written

	
RWD00   dq      6E75522054454E2Eh, 65737520656D6974h, 2D64726968742073h, 696C207974726170h, 2073656972617262h, 726568746F20726Fh, 6372756F73657220h, 6D20746168747365h
RWD64   dq      6D20746168747365h, 6964206562207961h, 6574756269727473h, 207265646E752064h, 7365736E6563696Ch, 6572656666696420h, 206E61687420746Eh, 54454E2E20656874h

in this case - via two avx-512 loads from the data section.

For now it needs DynamicPGO being enabled to workaround #85209, I'll land a fix for separately (or a workaround). tldr: None of Encoding.UTF8.X apis are ever inlined without PGO despite UTF8 returning something with an exact type.

cc @stephentoub

TODO:
Tests

Author:	EgorBo
Assignees:	-
Labels:	area-CodeGen-coreclr
Milestone:	-

[EgorBo]

@jakobbotsch @SingleAccretion yet another gtSplitTree use case PTAL 🙂 not urgent

I recommend to enable "ignore whitespaces" mode. There were a couple of "refactorings" + a pretty simple expansion for GetUtf8Bytes, I'm only concerned whether I preserve side-effects and their order correctly.

[jakobbotsch]

@EgorBo Can you merge and rerun CI?

[jakobbotsch]

The JIT changes look good to me.

[EgorBo]

the penalty can be mitigated if we slightly improve IsKnownConstant in jit and will be able to detect cases where span points to a literal in advance

Is this complicated / expensive? If not, can we do that? I'd expect the common case for direct end-user usage of Encoding.UTF8.TryGetBytes to be with non-const inputs. The const input case I've been primarily interested in is with Utf8.TryWrite and the AppendLiteral method that'll be called by code emitted by the C# compiler for the literal portion of interpolated strings. If it's too complicated / expensive / etc. to avoid the overhead for non-const inputs with TryGetBytes, then we should just skip TryGetBytes altogether and have the Utf8.TryWriteInterpolatedStringHandler.AppendLiteral use this intrinsic directly.

@stephentoub It's tricky to implement but it should be doable. Calling this static method directly from interpolation should be a better solution because we won't have to rely on inlining just like Jan pointed out earlier, so I guess for now I'll remove its usage from TryGetValue till IsKnonwConstant is implemented

[stephentoub]

It's tricky to implement but it should be doable. Calling this static method directly from interpolation should be a better solution because we won't have to rely on inlining just like Jan pointed out earlier, so I guess for now I'll remove its usage from TryGetValue till IsKnonwConstant is implemented

Ok, sounds good.

[EgorBo]

/azp run runtime-coreclr pgo

[azure-pipelines]

No commit pushedDate could be found for PR 85328 in repo dotnet/runtime

[stephentoub]

@EgorBo, anything I can do to help land this?

[EgorBo]

@EgorBo, anything I can do to help land this?

I was thinking of filing a separate PR to improve IsKnownConstant first, but I think it can be landed as is too in case if you want to call this new static ReadUtf8 directly from the interpolation? (it's actually a preferrable option because NAOT can't inline Encoding.UTF8.TryGetBytes currently)

[stephentoub]

if you want to call this new static ReadUtf8 directly from the interpolation

Yes, I thought that's what we agreed on above, using it from AppendLiteral and not from TryGetBytes yet.

[EgorBo]

Merging then, I've verified that the API works as expected when it's used inside TryGetBytes and removed it from there so now this API is unused. The additional runtime tests for TryGetBytes won't hurt I guess.

[stephentoub]

I'll submit a pr. Thanks.

[teo-tsirpanis]

Why?