Skip to content

Fix the bucket weight computation#1704

Merged
Maoni0 merged 5 commits intodotnet:mainfrom
cshung:public/fix-bucket-weights
May 4, 2021
Merged

Fix the bucket weight computation#1704
Maoni0 merged 5 commits intodotnet:mainfrom
cshung:public/fix-bucket-weights

Conversation

@cshung
Copy link
Contributor

@cshung cshung commented Mar 1, 2021

Built on the previous fix, this PR fixes the wrong bucket weight computation. The change is going to be much easier to review once we have done with the previous fix first.

Here are some experiments I ran to show how the weight computation fix works.

This is the base command line: -tc 6 -tagb 80.0 -tlgb 0.2 -sohsi 30 -lohsi 0 -pohsi 0 -sohsr 100-4000 -lohsr 102400-204800 -pohsr 100-204800 -sohpi 50 -lohpi 0 -sohfi 0 -lohfi 0 -pohfi 0 -allocType reference -testKind time

Note that the allocation ratio arguments are not given above, we will vary them in the following cases:

Case 1:
-lohar 10 -pohar 20

Without fix:
sohAllocatedBytes: 17387864196 (20.24%)
lohAllocatedBytes: 22174926546 (25.81%)
pohAllocatedBytes: 46337496222 (53.94%)

With fix:
sohAllocatedBytes: 83407696230 (97.10%)
lohAllocatedBytes: 862837482 (1.00%)
pohAllocatedBytes: 1628816178 (1.90%)

It can be easily seen that the weight was wrong by a large margin, and the fix corrected it.

Case 2:
-lohar 100 -pohar 200

Without fix:
Crashes with System.Exception: Values are not close

With fix:
sohAllocatedBytes: 60546753948 (70.49%)
lohAllocatedBytes: 8375351532 (9.75%)
pohAllocatedBytes: 16977259434 (19.76%)

This repros the Values are not close problem in the base case.

Case 3:
-lohar 200 -pohar 100

Without fix:
Crashes with System.Exception: Values are not close

With fix:
Crashes with System.Exception: Values are not close

The last experiment indicates the bug is not completely fixed, it repros even in the new version, so this is incomplete work, but the original implementation was off by so much that I would like to merge in this partial fix first.

@cshung cshung marked this pull request as draft March 1, 2021 21:40
@cshung cshung force-pushed the public/fix-bucket-weights branch 4 times, most recently from ed65f46 to 2b94e22 Compare March 6, 2021 03:34
Base automatically changed from master to main March 18, 2021 17:12
@cshung cshung force-pushed the public/fix-bucket-weights branch from 2b94e22 to f80a8a9 Compare March 21, 2021 02:29
@Maoni0
Copy link
Member

Maoni0 commented Apr 15, 2021

this changes the way we survive pretty dramatically, ie, it no longer allows variability in DoSurvive. this issue is to track getting that ability back.

@cshung cshung marked this pull request as ready for review April 15, 2021 22:42
@Maoni0 Maoni0 merged commit e473202 into dotnet:main May 4, 2021
@cshung cshung deleted the public/fix-bucket-weights branch May 4, 2021 16:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants