I wish the author would do the same for image decoding. iOS inability to draw a canvas with 4096x4097 dimensions means you can't use an Off screen canvas to go from file to RGBA array.
So, the JavaScript audio/video APIs already available in the browser are fully competent to build any kind of media player. The major limitation is that media players reliant on the browser APIs are limited to the codecs and containers supported by the browsers, which turns out to be a huge limitation.
A media library written for WASM with supplied codecs would not have that limitation, such that you could have an equivalent to VLC directly in a browser supporting immediate playback of any video/audio codec/container supplied in the WASM payload. The major advantage of a browser based application versus a desktop app is immediate playback of media via hyperlink without the soul crushing limitations Apple imposes on iOS.
I used to work at a place that developed a DAW in the browser, and yeah, we used a WASM Ogg Opus decoder to save on storage costs for our sound library. We earlier had to have mp3s for safari devices. With opus being opus we really wanted to provide it for the majority of devices as well as our server side streaming, and being able to provide opus support for safari ourselves was a godsend.
Edit: Should note that our decoder was developed in house and I have nothing to do with the project OP linked to.
Is that true even for mp3 though? Apparently it actually has a universal browser support up to MSIE 9 (!), while Ogg Vorbis and FLAC are now supported by all modern browsers after MSIE [1]. Opus seems to be the earliest format with less support than that, while this library only dates back to 2020.
Sure, but what if you don't want to use mp3... codecs aren't created equal. For anything latency sensitive mp3 is a non starter. For anything under bandwidth constraints, flac isn't great.
OPUS is a descendant of CELT (which never caught on, for reasons) which is an open/unhindered by patents low latency lossy codec suitable for VoIP and live streaming. It's relatively recent, but also, those use cases are more popular recently in the browser.
And to go even more extreme, some applications really care about bit-exactness in their decoders, which you don't get by relying on other vendors.
I mean, WebAssembly Opus decoder seems meaningful even to this day, but I wondered why earlier formats are there in the first place. The bit-exactness is a good argument but as you've said that's a very specific requirement, and the library only seems to care about performance and minified size anyway.
awesome! would love to help get distributions of these as Extism wasm plugins so they can be used in Go/Rust/PHP/Ruby/OCaml/Java/C++/C#/bunch more languages...
Hey, author of the wasm-audio-decoders library here. Actually, I wrote this library for this very reason to supplement my other library, icecast-metadata-player which is an Icecast (Shoutcast) browser client with ICY metadata support.
https://github.com/eshaz/icecast-metadata-js
WASM would be total overkill. The mpeg stream is readable natively in the browser — that’s how I served the client when I was running livebassmusicrightnow.
I wish the author would do the same for image decoding. iOS inability to draw a canvas with 4096x4097 dimensions means you can't use an Off screen canvas to go from file to RGBA array.
Interesting. What would be the use case for these? Audio editors?
So, the JavaScript audio/video APIs already available in the browser are fully competent to build any kind of media player. The major limitation is that media players reliant on the browser APIs are limited to the codecs and containers supported by the browsers, which turns out to be a huge limitation.
A media library written for WASM with supplied codecs would not have that limitation, such that you could have an equivalent to VLC directly in a browser supporting immediate playback of any video/audio codec/container supplied in the WASM payload. The major advantage of a browser based application versus a desktop app is immediate playback of media via hyperlink without the soul crushing limitations Apple imposes on iOS.
Browser support of some codecs is inconsistent, for example Safari's support for opus has been pretty bad/nonexistent (iirc this recently changed)
I used to work at a place that developed a DAW in the browser, and yeah, we used a WASM Ogg Opus decoder to save on storage costs for our sound library. We earlier had to have mp3s for safari devices. With opus being opus we really wanted to provide it for the majority of devices as well as our server side streaming, and being able to provide opus support for safari ourselves was a godsend.
Edit: Should note that our decoder was developed in house and I have nothing to do with the project OP linked to.
bandlab?
Is that true even for mp3 though? Apparently it actually has a universal browser support up to MSIE 9 (!), while Ogg Vorbis and FLAC are now supported by all modern browsers after MSIE [1]. Opus seems to be the earliest format with less support than that, while this library only dates back to 2020.
[1] https://caniuse.com/?search=audio%20format
Sure, but what if you don't want to use mp3... codecs aren't created equal. For anything latency sensitive mp3 is a non starter. For anything under bandwidth constraints, flac isn't great.
OPUS is a descendant of CELT (which never caught on, for reasons) which is an open/unhindered by patents low latency lossy codec suitable for VoIP and live streaming. It's relatively recent, but also, those use cases are more popular recently in the browser.
And to go even more extreme, some applications really care about bit-exactness in their decoders, which you don't get by relying on other vendors.
I mean, WebAssembly Opus decoder seems meaningful even to this day, but I wondered why earlier formats are there in the first place. The bit-exactness is a good argument but as you've said that's a very specific requirement, and the library only seems to care about performance and minified size anyway.
Being able to decode audio in a Web Worker rather than the main UI thread is also a nice use case.
This. Even FLAC support isn't great.
They look great for https://github.com/badaix/snapweb which is missing decoders for some formats.
awesome! would love to help get distributions of these as Extism wasm plugins so they can be used in Go/Rust/PHP/Ruby/OCaml/Java/C++/C#/bunch more languages...
Probably a good basis for a shoutcast client! So little info on it & I really doubt it's supported in any browsers. Even though it's mp3 with a little metadata sprinkled in every X bytes. https://www.radiotoolbox.com/community/forums/viewtopic.php?...
Hey, author of the wasm-audio-decoders library here. Actually, I wrote this library for this very reason to supplement my other library, icecast-metadata-player which is an Icecast (Shoutcast) browser client with ICY metadata support. https://github.com/eshaz/icecast-metadata-js
There are lots of docs and info available for shoutcast/icecast servers and clients. Here’s a couple implementations in javascript:
https://github.com/livebassmusicrightnow/even-nicercast/blob...
https://github.com/TooTallNate/node-icy
WASM would be total overkill. The mpeg stream is readable natively in the browser — that’s how I served the client when I was running livebassmusicrightnow.