Skip to content

More cleanup for 2.0 #384

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
MichaConrad merged 6 commits into from
Mar 5, 2025
Merged

More cleanup for 2.0 #384

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
c019b12
removing obsolete examples
MichaConrad Mar 3, 2025
35b6877
adding unit test clear on a multi server setup.
MichaConrad Mar 3, 2025
ddd9ce5
more cleanup
MichaConrad Mar 4, 2025
b2ca92b
update to benchmarks
MichaConrad Mar 4, 2025
1905908
trying to get those test not to fail randomly in CI - this is all a b…
MichaConrad Mar 4, 2025
6c6d760
just strange
MichaConrad Mar 5, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
110 changes: 42 additions & 68 deletions Benchmarks.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,15 +10,13 @@ All CacheManager instances used in the benchmarks have only one cache handle con
We are using the same configuration for all benchmarks and running two jobs each, one for x86 and one for x64. Regarding the different platforms, the conclusion is obviously that x64 is always faster than the x86 platform, but x64 consumes slightly more memory of course.

```ini
BenchmarkDotNet=v0.9.1.0
OS=Microsoft Windows NT 6.2.9200.0
Processor=Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz, ProcessorCount=8
Frequency=3328117 ticks, Resolution=300.4702 ns
HostCLR=MS.NET 4.0.30319.42000, Arch=32-bit RELEASE

Type=PutWithRegionSingleBenchmark Mode=Throughput Platform=X64
Jit=RyuJit LaunchCount=1 WarmupCount=2
TargetCount=100
BenchmarkDotNet v0.14.0, Windows 11 (10.0.26100.3194)
12th Gen Intel Core i7-12700KF, 1 CPU, 20 logical and 12 physical cores
.NET SDK 9.0.200
[Host] : .NET 8.0.13 (8.0.1325.6609), X64 RyuJIT AVX2
Job-BIBDFC : .NET 8.0.13 (8.0.1325.6609), X64 RyuJIT AVX2

IterationCount=10 LaunchCount=1 WarmupCount=2
```

### Add
Expand All @@ -27,73 +25,47 @@ TargetCount=100
Redis will be a lot slower in this scenario because CacheManager waits for the response to be able to return the `bool` value if the key has been added or not.
In general, it is good to see how fast the Dictionary handle is compared to the `System.Runtime` one. One thing you cannot see here is that also the memory footprint of the Dictionary handle is much lower.

Method | Platform | Median | StdDev | Scaled |
-----------: |:-----------: |-----------: |----------: |-------: |
Dictionary | X64 | 1.7254 us | 0.0511 us | 1.00 |
Dictionary | X86 | 1.9563 us | 0.0399 us | 1.00 |
Runtime | X64 | 4.9839 us | 0.0778 us | 2.89 |
Runtime | X86 | 7.0324 us | 0.2012 us | 3.59 |
Redis | X64 | 56.6671 us | 1.7202 us | 32.84 |
Redis | X86 | 58.0775 us | 0.9517 us | 29.69 |

| Method | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio |
|----------- |-------------:|------------:|------------:|-------:|--------:|-------:|-------:|----------:|------------:|
| Dictionary | 121.0 ns | 2.00 ns | 1.04 ns | 1.00 | 0.01 | 0.0153 | - | 200 B | 1.00 |
| Runtime | 637.2 ns | 28.46 ns | 16.94 ns | 5.27 | 0.14 | 0.2384 | 0.0010 | 3120 B | 15.60 |
| MsMemory | 198.8 ns | 4.59 ns | 3.04 ns | 1.64 | 0.03 | 0.0260 | - | 340 B | 1.70 |
| Redis | 105,395.6 ns | 3,758.09 ns | 2,485.74 ns | 871.20 | 20.86 | - | - | 1256 B | 6.28 |

*Adding one item per run with using region*

Method | Platform | Median | StdDev | Scaled |
-----------: |:-----------: |-----------: |----------: |-------: |
Dictionary | X64 | 1.8094 us | 0.0386 us | 1.00 |
Dictionary | X86 | 2.5029 us | 0.1179 us | 1.00 |
Runtime | X64 | 6.6934 us | 0.1275 us | 3.70 |
Runtime | X86 | 9.2334 us | 0.1637 us | 3.69 |
Redis | X64 | 58.5355 us | 1.7054 us | 32.35 |
Redis | X86 | 61.0272 us | 1.4178 us | 24.38 |
| Method | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Allocated | Alloc Ratio |
|----------- |-------------:|------------:|------------:|-------:|--------:|-------:|----------:|------------:|
| Dictionary | 144.2 ns | 4.65 ns | 3.07 ns | 1.00 | 0.03 | 0.0231 | 304 B | 1.00 |
| Runtime | 342.6 ns | 18.01 ns | 11.91 ns | 2.38 | 0.09 | 0.0610 | 800 B | 2.63 |
| MsMemory | 164.1 ns | 5.74 ns | 3.80 ns | 1.14 | 0.03 | 0.0231 | 304 B | 1.00 |
| Redis | 139,200.0 ns | 4,863.92 ns | 2,894.44 ns | 966.01 | 27.25 | - | 1528 B | 5.03 |


### Put
*Put 1 item per run*
Redis is as fast as the other handles in this scenario because CacheManager uses fire and forget for those operations. For Put it doesn't matter to know if the item has been added or updated...

Method | Platform | Median | StdDev | Scaled |
-----------: |:-----------: |-----------: |----------: |-------: |
Dictionary | X64 | 1.6802 us | 0.0235 us | 1.00 |
Dictionary | X86 | 1.9445 us | 0.0341 us | 1.00 |
Runtime | X64 | 4.4431 us | 0.0651 us | 2.64 |
Runtime | X86 | 6.5231 us | 0.1063 us | 3.35 |
Redis | X64 | 2.6869 us | 0.0934 us | 1.60 |
Redis | X86 | 3.5490 us | 0.0848 us | 1.83 |

*Put 1 item per run with region*

Method | Platform | Median | StdDev | Scaled |
-----------: |:-----------: |-----------: |----------: |-------: |
Dictionary | X64 | 1.7401 us | 0.0365 us | 1.00 |
Dictionary | X86 | 2.4589 us | 0.1022 us | 1.00 |
Runtime | X64 | 6.1772 us | 0.3683 us | 3.55 |
Runtime | X86 | 9.9298 us | 0.5574 us | 4.04 |
Redis | X64 | 3.0807 us | 0.0906 us | 1.77 |
Redis | X86 | 3.6385 us | 0.2123 us | 1.48 |
| Method | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio |
|----------- |------------:|-----------:|-----------:|------:|--------:|-------:|-------:|----------:|------------:|
| Dictionary | 95.01 ns | 1.888 ns | 1.249 ns | 1.00 | 0.02 | 0.0122 | - | 160 B | 1.00 |
| Runtime | 887.35 ns | 19.929 ns | 13.181 ns | 9.34 | 0.18 | 0.4263 | 0.0095 | 5576 B | 34.85 |
| MsMemory | 172.40 ns | 5.030 ns | 3.327 ns | 1.81 | 0.04 | 0.0336 | - | 440 B | 2.75 |
| Redis | 4,136.37 ns | 301.420 ns | 199.371 ns | 43.54 | 2.07 | 0.0839 | 0.0610 | 1095 B | 6.84 |


### Get
*Get 1 item per run*
With `Get` operations we can clearly see how much faster an in-memory cache is, compared to the distributed variant. That's why it makes so much sense to use CacheManager with a first and secondary cache layer.

Method | Platform | Median | StdDev | Scaled |
-----------: |:-----------: |-----------: |----------: |-------: |
Dictionary | X64 | 169.8708 ns | 4.6186 ns | 1.00 |
Dictionary | X86 | 540.0879 ns | 8.9363 ns | 1.00 |
Runtime | X64 | 310.4025 ns | 8.2928 ns | 1.83 |
Runtime | X86 | 1,030.5911 ns | 8.0532 ns | 1.91 |
Redis | X64 | 56,917.3537 ns | 2,035.4765 ns | 335.06 |
Redis | X86 | 59,519.5459 ns | 1,527.9613 ns | 110.20 |

*Get 1 item per run with region*

Method | Platform | Median | StdDev | Scaled |
-----------: |:-----------: |-----------: |----------: |-------: |
Dictionary | X64 | 234.9676 ns | 7.9172 ns | 1.00 |
Dictionary | X86 | 600.0429 ns | 13.7090 ns | 1.00 |
Runtime | X64 | 497.4116 ns | 15.6998 ns | 2.12 |
Runtime | X86 | 1,191.6452 ns | 19.8484 ns | 1.99 |
Redis | X64 | 57,257.9868 ns | 1,705.0148 ns | 243.68 |
Redis | X86 | 61,789.3944 ns | 1,775.6064 ns | 102.97 |
| Method | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Allocated | Alloc Ratio |
|----------- |-------------:|-------------:|-------------:|---------:|--------:|-------:|----------:|------------:|
| Dictionary | 34.97 ns | 0.532 ns | 0.317 ns | 1.00 | 0.01 | - | - | NA |
| Runtime | 179.45 ns | 2.104 ns | 1.392 ns | 5.13 | 0.06 | 0.0153 | 200 B | NA |
| MsMemory | 75.12 ns | 0.338 ns | 0.201 ns | 2.15 | 0.02 | - | - | NA |
| Redis | 75,534.71 ns | 5,303.351 ns | 3,507.838 ns | 2,160.12 | 97.47 | 0.1221 | 2008 B | NA |


### Serializer comparison
Expand Down Expand Up @@ -149,12 +121,14 @@ Pretty simple but large enough to analyze the performance.
**Results:**


Method | Platform | Median | StdDev | Scaled | Scaled-SD |
-----: |:-------: |-----------: |----------: |-------: |-------: |
BinarySerializer | X64 | 62.4158 ms | 1.8668 ms | 2.20 | 0.07 |
JsonSerializer | X64 | 28.5521 ms | 0.4633 ms | 1.00 | 0.00 |
JsonGzSerializer | X64 | 102.5552 ms | 4.8584 ms | 3.60 | 0.18 |
ProtoBufSerializer | X64 | 11.1276 ms | 0.1252 ms | 0.39 | 0.01 |
| Method | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio |
|------------------------- |----------:|---------:|---------:|------:|--------:|--------:|-------:|----------:|------------:|
| JsonSerializer | 83.21 us | 1.163 us | 0.769 us | 1.00 | 0.01 | 14.8926 | 2.4414 | 191.16 KB | 1.00 |
| JsonGzSerializer | 346.37 us | 4.785 us | 3.165 us | 4.16 | 0.05 | 21.9727 | 2.9297 | 280.75 KB | 1.47 |
| ProtoBufSerializer | 39.18 us | 0.701 us | 0.463 us | 0.47 | 0.01 | 8.6060 | 1.0986 | 110.7 KB | 0.58 |
| BondBinarySerializer | 19.03 us | 0.398 us | 0.263 us | 0.23 | 0.00 | 4.7302 | 0.6714 | 60.53 KB | 0.32 |
| BondFastBinarySerializer | 19.42 us | 0.431 us | 0.285 us | 0.23 | 0.00 | 4.7607 | 0.7324 | 60.84 KB | 0.32 |
| BondSimpleJsonSerializer | 63.56 us | 1.132 us | 0.748 us | 0.76 | 0.01 | 11.9629 | 1.9531 | 153.35 KB | 0.80 |


As expected the protobuf serialization outperforms everything else by a huge margin!
Expand Down
1 change: 1 addition & 0 deletions benchmarks/CacheManager.Benchmarks/CacheManager.Benchmarks.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
</ItemGroup>
<ItemGroup>
<PackageReference Include="BenchmarkDotNet" Version="0.14.0" />
<PackageReference Include="Microsoft.Garnet" Version="1.0.57" />
</ItemGroup>
<PropertyGroup Condition=" '$(Configuration)' == 'Debug' ">
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
Expand Down
62 changes: 41 additions & 21 deletions benchmarks/CacheManager.Benchmarks/Program.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,33 +1,53 @@
using System;
using System.Diagnostics.CodeAnalysis;
using System.Net;
using System.Reflection;
using BenchmarkDotNet.Configs;
using BenchmarkDotNet.Jobs;
using BenchmarkDotNet.Running;
using Garnet;
using Garnet.server;
using Microsoft.Extensions.Logging;

namespace CacheManager.Benchmarks
namespace CacheManager.Benchmarks;

[ExcludeFromCodeCoverage]
public class Program
{
[ExcludeFromCodeCoverage]
public class Program
public static GarnetServer StartServer(ILoggerFactory loggerFactory = null)
{
var server = new GarnetServer(new GarnetServerOptions()
{
EnableLua = true,
LuaOptions = new LuaOptions(LuaMemoryManagementMode.Native, string.Empty, TimeSpan.FromSeconds(5)),
EndPoint = new IPEndPoint(IPAddress.Loopback, 6379)
},
loggerFactory: loggerFactory);

server.Start();

return server;
}

public static void Main(string[] args)
{
public static void Main(string[] args)
StartServer();

do
{
do
{
var config = ManualConfig.CreateMinimumViable()
.AddJob(Job.Default
.WithIterationCount(10)
.WithWarmupCount(2)
.WithLaunchCount(1))
.AddDiagnoser(BenchmarkDotNet.Diagnosers.MemoryDiagnoser.Default);

BenchmarkSwitcher
.FromAssembly(typeof(Program).GetTypeInfo().Assembly)
.Run(args, config);

Console.WriteLine("done!");
Console.WriteLine("Press escape to exit or any key to continue...");
} while (Console.ReadKey().Key != ConsoleKey.Escape);
}
var config = ManualConfig.CreateMinimumViable()
.AddJob(Job.Default
.WithIterationCount(10)
.WithWarmupCount(2)
.WithLaunchCount(1))
.AddDiagnoser(BenchmarkDotNet.Diagnosers.MemoryDiagnoser.Default);

BenchmarkSwitcher
.FromAssembly(typeof(Program).GetTypeInfo().Assembly)
.Run(args, config);

Console.WriteLine("done!");
Console.WriteLine("Press escape to exit or any key to continue...");
} while (Console.ReadKey().Key != ConsoleKey.Escape);
}
}
14 changes: 0 additions & 14 deletions samples/OutputCacheExample/App_Start/BundleConfig.cs
View file
Open in desktop

This file was deleted.

13 changes: 0 additions & 13 deletions samples/OutputCacheExample/App_Start/FilterConfig.cs
View file
Open in desktop

This file was deleted.

23 changes: 0 additions & 23 deletions samples/OutputCacheExample/App_Start/RouteConfig.cs
View file
Open in desktop

This file was deleted.

24 changes: 0 additions & 24 deletions samples/OutputCacheExample/Content/Site.css
View file
Open in desktop

This file was deleted.

70 changes: 0 additions & 70 deletions samples/OutputCacheExample/Controllers/HomeController.cs
View file
Open in desktop

This file was deleted.

1 change: 0 additions & 1 deletion samples/OutputCacheExample/Global.asax
View file
Open in desktop

This file was deleted.

Loading
Loading