[Perf] Regressions on System.Text.RegularExpressions.Tests.Perf_Regex_Cache

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.22621
Queue	TigerWindows
Baseline	019d7580a27db97f1fbdcf0d26f7ae3fa54fc2d1
Compare	68511fd27fe4055ce5203742998ba12019dfcbd4
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Cache

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
IsMatch - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	84.63 ms	95.70 ms	1.13	0.06	False
IsMatch_Multithreading - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	47.24 ms	70.38 ms	1.49	0.29	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Cache*'

Details

System.Text.RegularExpressions.Tests.Perf_Regex_Cache.IsMatch(total: 40000, unique: 1600, cacheSize: 800)

ETL Files

Histogram

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Cache.IsMatch_Multithreading(total: 40000, unique: 1600, cacheSize: 800)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[LoopedBard3]

Caused by: #105746

Linux x64 Related: dotnet/perf-autofiling-issues#39743
Mono related regression on Linux AOT-llvm arm64: dotnet/perf-autofiling-issues#39828

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-text-regularexpressions
See info in area-owners.md if you want to be subscribed.

[steveharter]

@lateapexearlyspeed @stephentoub thoughts on these 2 cases? Appear to be Linux only.

System.Text.RegularExpressions.Tests.Perf_Regex_Cache.IsMatch_Multithreading(total: 40000, unique: 1600, cacheSize: 800)

source

and similar test here:

System.Text.RegularExpressions.Tests.Perf_Regex_Cache.IsMatch(total: 40000, unique: 1600, cacheSize: 800)

source

[jkurdek]

Related MonoJIT regression: dotnet/perf-autofiling-issues#39795

[lateapexearlyspeed]

thoughts on these 2 cases? Appear to be Linux only.

Hi @steveharter , just have a look at it and find following:

these 2 cases are about [Arguments(40_000, 1_600, 800)] so which means unique regexps size > cache size (so will trigger cache expulsion) and cache size > MaxExamineOnDrop (so RegexCache will use random subset to do cache expulsion).

Consider this test case also uses random regexp generation orders, so I feel improved LRU of RegexCache cannot give benefit for it.

For regression of these cases, I cannot find obvious reason with #105746 by now. The different behaviors I can find for these cases is: when RegexCache needs to expel 'oldest' item, previous code can skip some integer values exchange and timestamp for all cache items can keep low value (0, 1 maybe ) which were because all new coming regexps had timestamp value of 0, which causes subsequent regexp request (even if requested regexp was in cache and had high timestamp) resets its timestamp back to 0 + 1. Isn't this behavior unexpected ? Beside of these, I haven't found other difference in RegexCache level :(