Many moons ago I was preparing for an interview to join wasmer, back then they showed me that by running wasmer as a kernel extension in Linux they have made it run four orders of magnitude (10,000 times) faster. Given the WASI, I keep thinking more than the plugin usability of things like wasmer, what is the point of WASM not being accepted as a second binary in all operating systems? Windows could support .exe as well as .wasm for example and we would finally have a cross-plaform binary. Apple would never accept; Microsoft will do, and Linux can simply have,
Judging by the slow speed of node here, I wonder if it was run without forcing optimizations. It doesn't do OSR (on-stack replacement), meaning that a benchmark script doesn't get a chance to tier up from the baseline compiler to the fully optimized tier.
Running with
node --no-liftoff
avoids this problem.
(This is not an issue on the Web, where code must return to the event loop anyhow, allowing tiering up to work, and node is optimized for that kind of workflow.)
It's genuinely great to see WASM runtimes are getting closer to native code while being sandboxed, but these benchmarks assume you're not bundling a whole Runtime + GC + stdlib inside the sandbox.
One fairly revealing microbenchmark for WASM runtimes is `int8` dot products & angular/cosine distances.
(My) NumKong [1] has implementations targeting both vanilla AVX2/Haswell and AVX2-VNNI/Alder Lake, which makes it easy to see where runtimes and code generators leave performance on the table.
I started a few Wasmtime/Cranelift PRs around this, but didn’t manage to finish them :facepalm: Might be a fun weekend project for someone interested in backend/codegen work.
It'd be nice to see memory use. Ideally base usage, then cost of running 1 of a thing, then cost of running 50 if a thing.
Personally low my 50% speed vs native does not sound like the worst to me. I've spent years using perl, java (in v4-v7 days), node... These have various ways they are less good than native, and we collectively seem to accept that. Ideally yes wasm gets faster but a 2x is pretty ok!
Many moons ago I was preparing for an interview to join wasmer, back then they showed me that by running wasmer as a kernel extension in Linux they have made it run four orders of magnitude (10,000 times) faster. Given the WASI, I keep thinking more than the plugin usability of things like wasmer, what is the point of WASM not being accepted as a second binary in all operating systems? Windows could support .exe as well as .wasm for example and we would finally have a cross-plaform binary. Apple would never accept; Microsoft will do, and Linux can simply have,
Judging by the slow speed of node here, I wonder if it was run without forcing optimizations. It doesn't do OSR (on-stack replacement), meaning that a benchmark script doesn't get a chance to tier up from the baseline compiler to the fully optimized tier.
Running with
node --no-liftoff
avoids this problem.
(This is not an issue on the Web, where code must return to the event loop anyhow, allowing tiering up to work, and node is optimized for that kind of workflow.)
We have some more comprehensive benchmarks at https://r-carissimi.github.io/wasure/, with fairly different results.
libsodium is very numeric operation heavy IIRC
What are the numbers in that table? Is higher better or is higher worse? How does fully native C compare? What options are you using?
Edit: ah, higher numbers are better. But that fact is hidden way down in a block of text behind a collapsed details view.
Curious where my wasm2go would stand compared to wazero. I should test it myself. :)
https://github.com/ncruces/wasm2go
Seconding this as a brilliant project!
Shameless self promotion here but I’m currently using wasm2go to build a native Go runtime for MicroPython https://github.com/gregfurman/micropython-go
It's genuinely great to see WASM runtimes are getting closer to native code while being sandboxed, but these benchmarks assume you're not bundling a whole Runtime + GC + stdlib inside the sandbox.
One fairly revealing microbenchmark for WASM runtimes is `int8` dot products & angular/cosine distances.
(My) NumKong [1] has implementations targeting both vanilla AVX2/Haswell and AVX2-VNNI/Alder Lake, which makes it easy to see where runtimes and code generators leave performance on the table.
I started a few Wasmtime/Cranelift PRs around this, but didn’t manage to finish them :facepalm: Might be a fun weekend project for someone interested in backend/codegen work.
[1]: https://github.com/ashvardanian/NumKong
WAVM in Rust? WAVM does not yet support ARM MTE or RISC-V CHERI?
wasmtime-mte could be added to the benchmark.
It'd be nice to see memory use. Ideally base usage, then cost of running 1 of a thing, then cost of running 50 if a thing.
Personally low my 50% speed vs native does not sound like the worst to me. I've spent years using perl, java (in v4-v7 days), node... These have various ways they are less good than native, and we collectively seem to accept that. Ideally yes wasm gets faster but a 2x is pretty ok!