As far as I could tell, benchstat seems to implicitly make a location-shift assumption in its interpretation of the Mann-Whitney U test, and I don't see why that assumption would hold.
I just keep asking the agent to throw `hyperfine` and or /`criteron` at things and hoping it's enough.
Not a super clear sell here, to me, coming from this position of ignorance. I'd love to have better background, to compare, to know more. I'll try and see, I guess! This at least sounds not so bad:
> I built benchfmt.rs, a zero-dependency Rust benchmark runner with a thread-local heap allocation counter, pprof profiler integration, and the same CLI flags as go test -bench. For those who don’t trust a new benchmarking runtime (you shouldn’t from the start!), it also ships a Criterion and libtest converter that emits Go’s benchfmt format directly.
As far as I could tell, benchstat seems to implicitly make a location-shift assumption in its interpretation of the Mann-Whitney U test, and I don't see why that assumption would hold.
At last someone does this!
Heh thanks!
I wasn't the first one, Michael Pratt from the Go team did something similar 2y ago ((:
https://github.com/prattmic/benchfmt-convert
I just keep asking the agent to throw `hyperfine` and or /`criteron` at things and hoping it's enough.
Not a super clear sell here, to me, coming from this position of ignorance. I'd love to have better background, to compare, to know more. I'll try and see, I guess! This at least sounds not so bad:
> I built benchfmt.rs, a zero-dependency Rust benchmark runner with a thread-local heap allocation counter, pprof profiler integration, and the same CLI flags as go test -bench. For those who don’t trust a new benchmarking runtime (you shouldn’t from the start!), it also ships a Criterion and libtest converter that emits Go’s benchfmt format directly.
Wow this is sweet