refactor: trim redundant BTreeMap benchmarks, add realistic access patterns by maksymar · Pull Request #424 · dfinity/stable-structures

maksymar · 2026-03-31T10:44:22Z

This PR reduces the type/size grid for non-critical BTreeMap use cases and adds some realistic access patterns.

insert, remove, get -- are the most popular use cases for BTreeMap, while contains_key, first_*, last_*, pop_* can be considered as secondary, therefore should not be tested on the full type/size grid to save benchmark runtime
new benchmarks cover common real-world patterns
- 100K-entry insert/get/remove for deeper trees
- key overwrite, get-miss, sequential insertion, small repeated range queries

github-actions · 2026-03-31T10:56:54Z

`canbench` 🏋 (dir: ./benchmarks/vec) `bc86897` 2026-03-31 11:29:37 UTC

✅ ./benchmarks/vec/canbench_results.yml is up to date
📦 canbench_results_vec.csv available in artifacts

---------------------------------------------------

Summary:
  instructions:
    status:   No significant changes 👍
    counts:   [total 16 | regressed 0 | improved 0 | new 0 | unchanged 16]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

  heap_increase:
    status:   No significant changes 👍
    counts:   [total 16 | regressed 0 | improved 0 | new 0 | unchanged 16]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

  stable_memory_increase:
    status:   No significant changes 👍
    counts:   [total 16 | regressed 0 | improved 0 | new 0 | unchanged 16]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

---------------------------------------------------
CSV results saved to canbench_results.csv

github-actions · 2026-03-31T10:56:55Z

`canbench` 🏋 (dir: ./benchmarks/btreeset) `bc86897` 2026-03-31 11:29:54 UTC

✅ ./benchmarks/btreeset/canbench_results.yml is up to date
📦 canbench_results_btreeset.csv available in artifacts

---------------------------------------------------

Summary:
  instructions:
    status:   No significant changes 👍
    counts:   [total 100 | regressed 0 | improved 0 | new 0 | unchanged 100]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

  heap_increase:
    status:   No significant changes 👍
    counts:   [total 100 | regressed 0 | improved 0 | new 0 | unchanged 100]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

  stable_memory_increase:
    status:   No significant changes 👍
    counts:   [total 100 | regressed 0 | improved 0 | new 0 | unchanged 100]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

---------------------------------------------------
CSV results saved to canbench_results.csv

github-actions · 2026-03-31T10:56:55Z

`canbench` 🏋 (dir: ./benchmarks/io_chunks) `bc86897` 2026-03-31 11:30:27 UTC

✅ ./benchmarks/io_chunks/canbench_results.yml is up to date
📦 canbench_results_io_chunks.csv available in artifacts

---------------------------------------------------

Summary:
  instructions:
    status:   No significant changes 👍
    counts:   [total 18 | regressed 0 | improved 0 | new 0 | unchanged 18]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

  heap_increase:
    status:   No significant changes 👍
    counts:   [total 18 | regressed 0 | improved 0 | new 0 | unchanged 18]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

  stable_memory_increase:
    status:   No significant changes 👍
    counts:   [total 18 | regressed 0 | improved 0 | new 0 | unchanged 18]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

---------------------------------------------------
CSV results saved to canbench_results.csv

github-actions · 2026-03-31T10:56:55Z

`canbench` 🏋 (dir: ./benchmarks/nns) `bc86897` 2026-03-31 11:29:55 UTC

✅ ./benchmarks/nns/canbench_results.yml is up to date
📦 canbench_results_nns.csv available in artifacts

---------------------------------------------------

Summary:
  instructions:
    status:   No significant changes 👍
    counts:   [total 16 | regressed 0 | improved 0 | new 0 | unchanged 16]
    change:   [max 0 | p75 0 | median -98 | p25 -439.05K | min -6.98M]
    change %: [max 0.00% | p75 0.00% | median -0.04% | p25 -0.16% | min -0.29%]

  heap_increase:
    status:   No significant changes 👍
    counts:   [total 16 | regressed 0 | improved 0 | new 0 | unchanged 16]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

  stable_memory_increase:
    status:   No significant changes 👍
    counts:   [total 16 | regressed 0 | improved 0 | new 0 | unchanged 16]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

---------------------------------------------------
CSV results saved to canbench_results.csv

github-actions · 2026-03-31T10:56:56Z

`canbench` 🏋 (dir: ./benchmarks/memory_manager) `bc86897` 2026-03-31 11:29:46 UTC

✅ ./benchmarks/memory_manager/canbench_results.yml is up to date
📦 canbench_results_memory-manager.csv available in artifacts

---------------------------------------------------

Summary:
  instructions:
    status:   No significant changes 👍
    counts:   [total 3 | regressed 0 | improved 0 | new 0 | unchanged 3]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

  heap_increase:
    status:   No significant changes 👍
    counts:   [total 3 | regressed 0 | improved 0 | new 0 | unchanged 3]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

  stable_memory_increase:
    status:   No significant changes 👍
    counts:   [total 3 | regressed 0 | improved 0 | new 0 | unchanged 3]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

---------------------------------------------------
CSV results saved to canbench_results.csv

github-actions · 2026-03-31T10:56:56Z

`canbench` 🏋 (dir: ./benchmarks/btreemap) `bc86897` 2026-03-31 11:31:05 UTC

✅ ./benchmarks/btreemap/canbench_results.yml is up to date
📦 canbench_results_btreemap.csv available in artifacts

---------------------------------------------------

Summary:
  instructions:
    status:   New benchmarks added ➕
    counts:   [total 224 | regressed 0 | improved 0 | new 28 | unchanged 196]
    change:   [max +263.82K | p75 0 | median -3.79K | p25 -12.64K | min -1.55M]
    change %: [max +0.70% | p75 0.00% | median -0.00% | p25 -0.00% | min -0.32%]

  heap_increase:
    status:   New benchmarks added ➕
    counts:   [total 224 | regressed 0 | improved 0 | new 28 | unchanged 196]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

  stable_memory_increase:
    status:   New benchmarks added ➕
    counts:   [total 224 | regressed 0 | improved 0 | new 28 | unchanged 196]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

---------------------------------------------------

Only significant changes:
| status | name                                      | calls |     ins |  ins Δ% | HI |  HI Δ% | SMI |  SMI Δ% |
|--------|-------------------------------------------|-------|---------|---------|----|--------|-----|---------|
|  new   | btreemap_v2_contains_vec_32_vec128        |       | 428.33M |         |  0 |        |   0 |         |
|  new   | btreemap_v2_first_key_value_blob_256_128  |       | 662.70M |         |  0 |        |   0 |         |
|  new   | btreemap_v2_first_key_value_blob_32_0     |       | 288.11M |         |  0 |        |   0 |         |
|  new   | btreemap_v2_first_key_value_blob_32_1024  |       | 445.38M |         |  0 |        |   0 |         |
|  new   | btreemap_v2_first_key_value_blob_32_128   |       | 275.34M |         |  0 |        |   0 |         |
|  new   | btreemap_v2_first_key_value_blob_8_128    |       | 291.94M |         |  0 |        |   0 |         |
|  new   | btreemap_v2_first_key_value_principal     |       | 367.25M |         |  0 |        |   0 |         |
|  new   | btreemap_v2_first_key_value_u64_u64       |       | 231.16M |         |  0 |        |   0 |         |
|  new   | btreemap_v2_first_key_value_vec_32_128    |       | 227.16M |         |  0 |        |   0 |         |
|  new   | btreemap_v2_first_key_value_vec_32_vec128 |       | 227.16M |         |  0 |        |   0 |         |
|  new   | btreemap_v2_get_100k_u64_u64              |       |   2.77B |         |  0 |        |   0 |         |
|  new   | btreemap_v2_get_miss_u64_u64              |       | 229.19M |         |  0 |        |   0 |         |
|  new   | btreemap_v2_insert_100k_u64_u64           |       |   4.76B |         |  0 |        |  60 |         |
|  new   | btreemap_v2_insert_overwrite_u64_u64      |       | 360.35M |         |  0 |        |   0 |         |
|  new   | btreemap_v2_insert_seq_u64_u64            |       | 473.39M |         |  0 |        |   8 |         |
|  new   | btreemap_v2_last_key_value_blob_256_128   |       | 641.75M |         |  0 |        |   0 |         |
|  new   | btreemap_v2_last_key_value_blob_32_0      |       | 220.91M |         |  0 |        |   0 |         |
|  new   | btreemap_v2_last_key_value_blob_32_1024   |       | 389.73M |         |  0 |        |   0 |         |
|  new   | btreemap_v2_last_key_value_blob_32_128    |       | 266.93M |         |  0 |        |   0 |         |
|  new   | btreemap_v2_last_key_value_blob_8_128     |       | 321.10M |         |  0 |        |   0 |         |
|  new   | btreemap_v2_last_key_value_principal      |       | 217.83M |         |  0 |        |   0 |         |
|  new   | btreemap_v2_last_key_value_u64_u64        |       | 245.30M |         |  0 |        |   0 |         |
|  new   | btreemap_v2_last_key_value_vec_32_128     |       | 221.15M |         |  0 |        |   0 |         |
|  new   | btreemap_v2_last_key_value_vec_32_vec128  |       | 221.15M |         |  0 |        |   0 |         |
|  new   | btreemap_v2_pop_first_vec_32_vec128       |       |   1.07B |         |  0 |        |   0 |         |
|  new   | btreemap_v2_pop_last_vec_32_vec128        |       |   1.04B |         |  0 |        |   0 |         |
|  new   | btreemap_v2_range_small_u64_u64           |       |  18.49M |         |  0 |        |   0 |         |
|  new   | btreemap_v2_remove_100k_u64_u64           |       |   6.92B |         |  0 |        |   0 |         |

ins = instructions, HI = heap_increase, SMI = stable_memory_increase, Δ% = percent change

---------------------------------------------------
CSV results saved to canbench_results.csv

add first/last benchmarks

53fc21f

maksymar changed the title ~~add first/last benchmarks~~ refactor: improve btreemap benchmarks Mar 31, 2026

reduced grid

b9aec58

maksymar mentioned this pull request Mar 31, 2026

perf: add direct-mapped node cache to BTreeMap #416

Open

add more cache related benchmarks

285ca9d

maksymar changed the title ~~refactor: improve btreemap benchmarks~~ refactor: trim redundant BTreeMap benchmarks, add realistic access patterns Mar 31, 2026

restore btreemap_v2_contains_10mib_values

850ba2b

maksymar marked this pull request as ready for review March 31, 2026 11:26

maksymar requested a review from a team as a code owner March 31, 2026 11:26

maksymar requested a review from sasa-tomic March 31, 2026 11:35

michael-weigelt approved these changes Mar 31, 2026

View reviewed changes

maksymar merged commit 46bf6a0 into main Mar 31, 2026
20 checks passed

maksymar deleted the maksym/benchmarks branch March 31, 2026 12:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor: trim redundant BTreeMap benchmarks, add realistic access patterns#424

refactor: trim redundant BTreeMap benchmarks, add realistic access patterns#424
maksymar merged 4 commits intomainfrom
maksym/benchmarks

maksymar commented Mar 31, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Mar 31, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Mar 31, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Mar 31, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Mar 31, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Mar 31, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Mar 31, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

maksymar commented Mar 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Mar 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

canbench 🏋 (dir: ./benchmarks/vec) bc86897 2026-03-31 11:29:37 UTC

Uh oh!

github-actions bot commented Mar 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

canbench 🏋 (dir: ./benchmarks/btreeset) bc86897 2026-03-31 11:29:54 UTC

Uh oh!

github-actions bot commented Mar 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

canbench 🏋 (dir: ./benchmarks/io_chunks) bc86897 2026-03-31 11:30:27 UTC

Uh oh!

github-actions bot commented Mar 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

canbench 🏋 (dir: ./benchmarks/nns) bc86897 2026-03-31 11:29:55 UTC

Uh oh!

github-actions bot commented Mar 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

canbench 🏋 (dir: ./benchmarks/memory_manager) bc86897 2026-03-31 11:29:46 UTC

Uh oh!

github-actions bot commented Mar 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

canbench 🏋 (dir: ./benchmarks/btreemap) bc86897 2026-03-31 11:31:05 UTC

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

maksymar commented Mar 31, 2026 •

edited

Loading

github-actions bot commented Mar 31, 2026 •

edited

Loading

`canbench` 🏋 (dir: ./benchmarks/vec) `bc86897` 2026-03-31 11:29:37 UTC

github-actions bot commented Mar 31, 2026 •

edited

Loading

`canbench` 🏋 (dir: ./benchmarks/btreeset) `bc86897` 2026-03-31 11:29:54 UTC

github-actions bot commented Mar 31, 2026 •

edited

Loading

`canbench` 🏋 (dir: ./benchmarks/io_chunks) `bc86897` 2026-03-31 11:30:27 UTC

github-actions bot commented Mar 31, 2026 •

edited

Loading

`canbench` 🏋 (dir: ./benchmarks/nns) `bc86897` 2026-03-31 11:29:55 UTC

github-actions bot commented Mar 31, 2026 •

edited

Loading

`canbench` 🏋 (dir: ./benchmarks/memory_manager) `bc86897` 2026-03-31 11:29:46 UTC

github-actions bot commented Mar 31, 2026 •

edited

Loading

`canbench` 🏋 (dir: ./benchmarks/btreemap) `bc86897` 2026-03-31 11:31:05 UTC