Skip to content

docs: update build performance benchmarks (3.6.0)#712

Open
github-actions[bot] wants to merge 2 commits intomainfrom
benchmark/build-v3.6.0-20260330-164400
Open

docs: update build performance benchmarks (3.6.0)#712
github-actions[bot] wants to merge 2 commits intomainfrom
benchmark/build-v3.6.0-20260330-164400

Conversation

@github-actions
Copy link
Copy Markdown
Contributor

Automated build benchmark update for 3.6.0 from workflow run #538.

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Mar 30, 2026

Greptile Summary

This automated PR adds 3.6.0 build benchmark results (WASM-only; native runner was null for this release) to both the README summary table and the detailed BUILD-BENCHMARKS.md file.

Key changes:

  • README: Column header updated to Latest (WASM), resolving the previously flagged engine-context ambiguity. Per-file metrics now reflect 3.6.0 WASM numbers (13.3 ms/file build, 547ms 1-file rebuild, ~665s estimated at 50 K files).
  • BUILD-BENCHMARKS.md: 3.6.0 row inserted into all history tables (build, incremental, query). A _3.5.x_ stub row explains the version gap. The "Raw totals (latest)" section now shows only the WASM block; the native block, engine-parity note, and empty native columns in the phase-breakdown and estimated-performance tables have been removed.
  • Benchmark data verified: All percentage-change indicators (↑12% build speed, ↑28% DB size, ↑12% 1-file rebuild, ↑11% fn-deps, etc.) are consistent with the embedded raw JSON and cross-checked against the prior WASM baseline (3.4.1 WASM).

Confidence Score: 5/5

Documentation-only update with internally consistent numbers; all prior review concerns have been addressed — safe to merge.

Both previously flagged issues (missing engine qualifier in README table, empty native columns in BUILD-BENCHMARKS.md) were resolved in commit 922b7c8. All percentage deltas and raw totals cross-check correctly against the embedded JSON benchmark data. No logic, code, or schema is touched — this is a pure docs update.

No files require special attention.

Important Files Changed

Filename Overview
README.md Performance summary table updated to 3.6.0 WASM-only numbers; column header now reads 'Latest (WASM)' addressing the engine-context concern from the prior review round.
generated/benchmarks/BUILD-BENCHMARKS.md 3.6.0 WASM entry added to all history tables; native raw-totals section and empty native phase columns removed; percentage deltas and raw numbers are internally consistent with the embedded JSON benchmark data.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[CI Workflow Run #538] -->|produces raw JSON| B[BENCHMARK_DATA block\nin BUILD-BENCHMARKS.md]
    B -->|generator script reads JSON| C{Engine?}
    C -->|wasm| D[3.6.0 WASM row added\nto history tables]
    C -->|native: null| E[Native section omitted\nfrom latest blocks]
    D --> F[Build history table\n13.3 ms/file up12%]
    D --> G[Incremental table\nno-op 14ms / 1-file 547ms]
    D --> H[Query table\nfn-deps 2.1ms up11%]
    D --> I[Phase breakdown table\nWASM-only columns]
    E --> J[Engine parity note removed]
    E --> K[Native columns dropped\nfrom phase & estimated tables]
    F & G & H & I & J & K --> L[BUILD-BENCHMARKS.md final]
    L -->|key metrics extracted| M[README.md summary table\nLatest WASM column]
Loading

Reviews (2): Last reviewed commit: "docs: clarify WASM-only engine context i..." | Re-trigger Greptile

README.md Outdated
Comment on lines +574 to +582
| Metric | Latest |
|---|---|
| Build speed (native) | **5.7 ms/file** |
| Build speed (WASM) | **11.9 ms/file** |
| Build speed | **13.3 ms/file** |
| Query time | **12ms** |
| No-op rebuild (native) | **14ms** |
| 1-file rebuild (native) | **316ms** |
| Query: fn-deps | **1.9ms** |
| No-op rebuild | **14ms** |
| 1-file rebuild | **547ms** |
| Query: fn-deps | **2.1ms** |
| Query: path | **1.9ms** |
| ~50,000 files (est.) | **~285.0s build** |
| ~50,000 files (est.) | **~665.0s build** |
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Engine context removed from summary table

The previous README table explicitly labelled rows as Build speed (native) and Build speed (WASM), making it clear which engine drove each number. In this update the labels are collapsed to just Build speed, No-op rebuild, and 1-file rebuild, but the underlying values are WASM-only (native is null for 3.6.0). Without the engine qualifier, a reader comparing against the prior version will see:

  • Build speed: 5.7 ms/file → 13.3 ms/file (+133% apparent regression — but the 5.7 figure was native)
  • 1-file rebuild: 316 ms → 547 ms (+73% apparent regression — but 316 ms was also native)

Consider adding a small clarification, e.g. Build speed (WASM) or a footnote noting that 3.6.0 results are WASM-only, so readers don't mistake the engine switch for a performance regression.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 922b7c8. Added (WASM) qualifier to the README summary table header (Latest (WASM)) so readers can immediately see these are WASM-only numbers and won't mistake the engine switch for a regression.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 922b7c8. Added \ qualifier to the README summary table header () so readers can immediately see these are WASM-only numbers and won't mistake the engine switch for a regression.

Comment on lines 66 to 76
| Resolve imports | n/a | 11 ms | n/a | 1.5 ms |
| Build edges | n/a | 178.5 ms | n/a | 33.8 ms |
| Structure | n/a | 40.2 ms | n/a | 25.4 ms |
| Roles | n/a | 80.3 ms | n/a | 45.8 ms |
| AST nodes | n/a | 391.3 ms | n/a | 9.1 ms |
| Complexity | n/a | 402.2 ms | n/a | 0.7 ms |
| CFG | n/a | 424.2 ms | n/a | 0.4 ms |
| Dataflow | n/a | 209.5 ms | n/a | 0.4 ms |

### Estimated performance at 50,000 files

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Empty native columns in phase breakdown table

Now that native benchmarking has been dropped for 3.6.0 ("native": null in the raw data), the "Build Phase Breakdown (latest)" table retains Native (build) and Native (1-file) columns filled entirely with n/a. These columns add visual noise without conveying information. Consider removing them so the table only shows the two WASM columns, which keeps it consistent with the rest of the "latest" raw-totals section that already dropped the native block.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 922b7c8. Removed the empty Native (build) and Native (1-file) columns from the phase breakdown table. It now shows just Full build and 1-file rebuild columns, with the section header updated to Build Phase Breakdown (latest -- WASM). Also cleaned up the estimated-performance table the same way.

Add (WASM) qualifier to README summary table header and phase
breakdown section, remove empty native columns that added visual
noise, and note the 3.5.x version gap in the historical table.
@carlos-alm
Copy link
Copy Markdown
Contributor

Addressed the version gap note (P2, outside diff): added a placeholder row in the historical table for 3.5.x indicating they were not benchmarked (patch-only releases). Fixed in 922b7c8.

@carlos-alm
Copy link
Copy Markdown
Contributor

@greptileai review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant