docs: update build performance benchmarks (3.6.0)#712
docs: update build performance benchmarks (3.6.0)#712github-actions[bot] wants to merge 2 commits intomainfrom
Conversation
Greptile SummaryThis automated PR adds 3.6.0 build benchmark results (WASM-only; native runner was Key changes:
Confidence Score: 5/5Documentation-only update with internally consistent numbers; all prior review concerns have been addressed — safe to merge. Both previously flagged issues (missing engine qualifier in README table, empty native columns in BUILD-BENCHMARKS.md) were resolved in commit 922b7c8. All percentage deltas and raw totals cross-check correctly against the embedded JSON benchmark data. No logic, code, or schema is touched — this is a pure docs update. No files require special attention. Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[CI Workflow Run #538] -->|produces raw JSON| B[BENCHMARK_DATA block\nin BUILD-BENCHMARKS.md]
B -->|generator script reads JSON| C{Engine?}
C -->|wasm| D[3.6.0 WASM row added\nto history tables]
C -->|native: null| E[Native section omitted\nfrom latest blocks]
D --> F[Build history table\n13.3 ms/file up12%]
D --> G[Incremental table\nno-op 14ms / 1-file 547ms]
D --> H[Query table\nfn-deps 2.1ms up11%]
D --> I[Phase breakdown table\nWASM-only columns]
E --> J[Engine parity note removed]
E --> K[Native columns dropped\nfrom phase & estimated tables]
F & G & H & I & J & K --> L[BUILD-BENCHMARKS.md final]
L -->|key metrics extracted| M[README.md summary table\nLatest WASM column]
Reviews (2): Last reviewed commit: "docs: clarify WASM-only engine context i..." | Re-trigger Greptile |
README.md
Outdated
| | Metric | Latest | | ||
| |---|---| | ||
| | Build speed (native) | **5.7 ms/file** | | ||
| | Build speed (WASM) | **11.9 ms/file** | | ||
| | Build speed | **13.3 ms/file** | | ||
| | Query time | **12ms** | | ||
| | No-op rebuild (native) | **14ms** | | ||
| | 1-file rebuild (native) | **316ms** | | ||
| | Query: fn-deps | **1.9ms** | | ||
| | No-op rebuild | **14ms** | | ||
| | 1-file rebuild | **547ms** | | ||
| | Query: fn-deps | **2.1ms** | | ||
| | Query: path | **1.9ms** | | ||
| | ~50,000 files (est.) | **~285.0s build** | | ||
| | ~50,000 files (est.) | **~665.0s build** | |
There was a problem hiding this comment.
Engine context removed from summary table
The previous README table explicitly labelled rows as Build speed (native) and Build speed (WASM), making it clear which engine drove each number. In this update the labels are collapsed to just Build speed, No-op rebuild, and 1-file rebuild, but the underlying values are WASM-only (native is null for 3.6.0). Without the engine qualifier, a reader comparing against the prior version will see:
- Build speed: 5.7 ms/file → 13.3 ms/file (+133% apparent regression — but the 5.7 figure was native)
- 1-file rebuild: 316 ms → 547 ms (+73% apparent regression — but 316 ms was also native)
Consider adding a small clarification, e.g. Build speed (WASM) or a footnote noting that 3.6.0 results are WASM-only, so readers don't mistake the engine switch for a performance regression.
There was a problem hiding this comment.
Fixed in 922b7c8. Added (WASM) qualifier to the README summary table header (Latest (WASM)) so readers can immediately see these are WASM-only numbers and won't mistake the engine switch for a regression.
There was a problem hiding this comment.
Fixed in 922b7c8. Added \ qualifier to the README summary table header () so readers can immediately see these are WASM-only numbers and won't mistake the engine switch for a regression.
| | Resolve imports | n/a | 11 ms | n/a | 1.5 ms | | ||
| | Build edges | n/a | 178.5 ms | n/a | 33.8 ms | | ||
| | Structure | n/a | 40.2 ms | n/a | 25.4 ms | | ||
| | Roles | n/a | 80.3 ms | n/a | 45.8 ms | | ||
| | AST nodes | n/a | 391.3 ms | n/a | 9.1 ms | | ||
| | Complexity | n/a | 402.2 ms | n/a | 0.7 ms | | ||
| | CFG | n/a | 424.2 ms | n/a | 0.4 ms | | ||
| | Dataflow | n/a | 209.5 ms | n/a | 0.4 ms | | ||
|
|
||
| ### Estimated performance at 50,000 files | ||
|
|
There was a problem hiding this comment.
Empty native columns in phase breakdown table
Now that native benchmarking has been dropped for 3.6.0 ("native": null in the raw data), the "Build Phase Breakdown (latest)" table retains Native (build) and Native (1-file) columns filled entirely with n/a. These columns add visual noise without conveying information. Consider removing them so the table only shows the two WASM columns, which keeps it consistent with the rest of the "latest" raw-totals section that already dropped the native block.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
There was a problem hiding this comment.
Fixed in 922b7c8. Removed the empty Native (build) and Native (1-file) columns from the phase breakdown table. It now shows just Full build and 1-file rebuild columns, with the section header updated to Build Phase Breakdown (latest -- WASM). Also cleaned up the estimated-performance table the same way.
Add (WASM) qualifier to README summary table header and phase breakdown section, remove empty native columns that added visual noise, and note the 3.5.x version gap in the historical table.
|
Addressed the version gap note (P2, outside diff): added a placeholder row in the historical table for 3.5.x indicating they were not benchmarked (patch-only releases). Fixed in 922b7c8. |
|
@greptileai review |
Automated build benchmark update for 3.6.0 from workflow run #538.