SMRTR ProgrammingSep 17, 2025Lobsters

Is GitHub Actions suitable for running benchmarks?

SMRTR summary

GitHub Actions can effectively run relative benchmarks by comparing two commits in the same job, canceling out hardware variability. Testing identical commits over 16 days showed this method reliably detects performance regressions of 50% or more with only 3.7% false positives. Optimizations like using mamba instead of conda and implementing ccache reduced benchmark time from nearly 2 hours to 1 hour 20 minutes, while on-demand triggering via PR labels prevents resource waste.

SMRTR provides this summary for quick context. The original article belongs to Lobsters.

Read the original article
SMRTR Programming

Get the next batch of curated stories in your inbox.

This archive is built from SMRTR newsletter stories. Subscribe for hand-picked stories without the extra noise.

Related Stories

Browse Programming
ProgrammingAug 23, 2026

Rust Glancer

Rust-analyzer's memory bloat stems from treating all 6,666 dependencies equally — a tiered, IntelliJ-style backend could fix that.