High memory usages on "AMD EPYC 7R13 Processor" #99804
Description
Description
We have a Web API program running on AWS ECS Fargate. After upgrading to .NET 8, we observed abnormal high memory usage. The service has around 10 ECS tasks, and when running these tasks on 'AMD EPYC 7R13 Processor' cpus, memory always stays at a high level.
the container image based on "FROM --platform=$BUILDPLATFORM mcr.microsoft.com/dotnet/aspnet:8.0 AS base"
and get the osVersion and dotnet veriosn in app is
"osVersion": "Unix 5.10.209.198", // Environment.OSVersion
"version": "8.0.3", // Environment.Version
The app running on AMD EPYC CPUs seem to seem to have lazier GC behavior. They handle requests (cost) almost equally (via ELB load balancing).
-52 is AMD EPYC CPU
High memory usage task /proc/cpuinfo outputs
{
"cpuInfo": [
"processor\t: 0",
"vendor_id\t: AuthenticAMD",
"cpu family\t: 25",
"model\t\t: 1",
"model name\t: AMD EPYC 7R13 Processor",
"stepping\t: 1",
"microcode\t: 0xa0011d1",
"cpu MHz\t\t: 3597.517",
"cache size\t: 512 KB",
"physical id\t: 0",
"siblings\t: 2",
"core id\t\t: 0",
"cpu cores\t: 1",
"apicid\t\t: 0",
"initial apicid\t: 0",
"fpu\t\t: yes",
"fpu_exception\t: yes",
"cpuid level\t: 16",
"wp\t\t: yes",
"flags\t\t: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy cr8_legacy abm sse4a misalignsse 3dnowprefetch topoext invpcid_single ssbd ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 clzero xsaveerptr rdpru wbnoinvd arat npt nrip_save vaes vpclmulqdq rdpid",
"bugs\t\t: sysret_ss_attrs null_seg spectre_v1 spectre_v2 spec_store_bypass srso",
"bogomips\t: 5299.99",
"TLB size\t: 2560 4K pages",
"clflush size\t: 64",
"cache_alignment\t: 64",
"address sizes\t: 48 bits physical, 48 bits virtual",
"power management:",
"",
"processor\t: 1",
"vendor_id\t: AuthenticAMD",
"cpu family\t: 25",
"model\t\t: 1",
"model name\t: AMD EPYC 7R13 Processor",
"stepping\t: 1",
"microcode\t: 0xa0011d1",
"cpu MHz\t\t: 3597.209",
"cache size\t: 512 KB",
"physical id\t: 0",
"siblings\t: 2",
"core id\t\t: 0",
"cpu cores\t: 1",
"apicid\t\t: 1",
"initial apicid\t: 1",
"fpu\t\t: yes",
"fpu_exception\t: yes",
"cpuid level\t: 16",
"wp\t\t: yes",
"flags\t\t: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy cr8_legacy abm sse4a misalignsse 3dnowprefetch topoext invpcid_single ssbd ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 clzero xsaveerptr rdpru wbnoinvd arat npt nrip_save vaes vpclmulqdq rdpid",
"bugs\t\t: sysret_ss_attrs null_seg spectre_v1 spectre_v2 spec_store_bypass srso",
"bogomips\t: 5299.99",
"TLB size\t: 2560 4K pages",
"clflush size\t: 64",
"cache_alignment\t: 64",
"address sizes\t: 48 bits physical, 48 bits virtual",
"power management:",
""
]
}
Normal memory usage task /proc/cpuinfo outputs
{
"cpuInfo": [
"processor\t: 0",
"vendor_id\t: GenuineIntel",
"cpu family\t: 6",
"model\t\t: 85",
"model name\t: Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz",
"stepping\t: 4",
"microcode\t: 0x2007006",
"cpu MHz\t\t: 3182.459",
"cache size\t: 33792 KB",
"physical id\t: 0",
"siblings\t: 2",
"core id\t\t: 0",
"cpu cores\t: 1",
"apicid\t\t: 0",
"initial apicid\t: 0",
"fpu\t\t: yes",
"fpu_exception\t: yes",
"cpuid level\t: 13",
"wp\t\t: yes",
"flags\t\t: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single pti fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves ida arat pku ospke",
"bugs\t\t: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit mmio_stale_data retbleed gds",
"bogomips\t: 5000.01",
"clflush size\t: 64",
"cache_alignment\t: 64",
"address sizes\t: 46 bits physical, 48 bits virtual",
"power management:",
"",
"processor\t: 1",
"vendor_id\t: GenuineIntel",
"cpu family\t: 6",
"model\t\t: 85",
"model name\t: Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz",
"stepping\t: 4",
"microcode\t: 0x2007006",
"cpu MHz\t\t: 3182.857",
"cache size\t: 33792 KB",
"physical id\t: 0",
"siblings\t: 2",
"core id\t\t: 0",
"cpu cores\t: 1",
"apicid\t\t: 1",
"initial apicid\t: 1",
"fpu\t\t: yes",
"fpu_exception\t: yes",
"cpuid level\t: 13",
"wp\t\t: yes",
"flags\t\t: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single pti fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves ida arat pku ospke",
"bugs\t\t: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit mmio_stale_data retbleed gds",
"bogomips\t: 5000.01",
"clflush size\t: 64",
"cache_alignment\t: 64",
"address sizes\t: 46 bits physical, 48 bits virtual",
"power management:",
""
]
}
Reproduction Steps
Runs app in AWS ECS with the AMD EPYC CPU.
Expected behavior
The memory usage is the same on different CPUs.
Actual behavior
high memory usages on AMD EPYC CPU.
Regression?
No response
Known Workarounds
No response
Configuration
No response
Other information
No response