Hyperfine is a great tool but when I was using it at Deno to benchmark startup time there was a lot of weirdness around the operating system apparently caching inodes of executables.
If you are looking at shaving sub 20ms numbers, be aware you may need to pull tricks on macos especially to get real numbers.
I've found pretty good results with the System Trace template in xcode instruments. You can also stack instruments, for example combining the file inspector with a virtual memory inspector.
I've run into some memory corruption with it sometimes, though, so be wary of that. Emerge tools has an alternative for iOS at least, maybe one day they'll port it to mac.
Hyperfine is a great tool but when I was using it at Deno to benchmark startup time there was a lot of weirdness around the operating system apparently caching inodes of executables.
If you are looking at shaving sub 20ms numbers, be aware you may need to pull tricks on macos especially to get real numbers.
I've found pretty good results with the System Trace template in xcode instruments. You can also stack instruments, for example combining the file inspector with a virtual memory inspector.
I've run into some memory corruption with it sometimes, though, so be wary of that. Emerge tools has an alternative for iOS at least, maybe one day they'll port it to mac.
What database product does the community commonly send benchmark results to? This tool is great, but I'd love to analyze results relationally.