We went through the exciting journey of optimizing Neural Amp Modeler DSP to run inference real-time on the tiny Daisy Seed Arm Cortex-M7. Think hand-rolled GEMM kernels for small matrices and other fun stuff :D
The actual effect transforming the audio is a neural network. The beauty is that you can record someone playing guitar through an ultra-rare amp/filter/whatnot (both clean signal and output) and then train the neural network to replicate that.
Neural Amp Models are small neural nets that have been trained to emulate the sound of classic guitar amps and effects pedals. The neural networks are trained on amp inputs and amp outputs (or effect inputs and effect outputs). Neural Amp Modeller Core is open source, licensed under an MIT license. TONE300.com provides both free online training of Neural net Models, and a massive collection community-generated high-quality Neural Amp Models. NAM has a large and very active online community.
It is obviously not possible to run huge LLMs in realtime; but it's been known for some time that very small neural network ML models can run in realtime, and can produce absolutely stunning simulations of real amps. The models are typically in the order of thousands of weights, not billions of weights. (Not actually sure what the weight count is for the A2 pico models that are being discussed in the original post. OP may be able to help with that).
These tiny neural network models not only accurately reproduce the sound of the original amps, but also manage to reproduce the feel of playing the original amp as well. The quality is dramatically better than most previous amp simulations (and entirely competitive with really high-end amp simulation technologies like Kemperer). It is breakthrough technology, for guitar pedals and amp simulators particularly, that literally changes everything in the music industry. The models are also relatively easy to train. TONE3000.com provides free online services for training models, and currently host a massive library of thousands of high-quality NAM models that are downloadable, free of charge.
The particularly interesting part of this report is that a single NAM model will be able to run on ridiculously tiny embedded processors. OP claims to have a 2nd-generation Pico NAM model running on a 500Mhz Cortex M7. First-generation Standard NAM models typically require a much more beefy processor: an ARM processor in the Pi 4 or Pi 5 sort of range (2.0Ghz Cortex A72, and 2.4Ghz Cortex A76 processors), or on x64 processors (an N100-class intel processor would be a good choice).
(Author of an open source project that uses Neural Amp Modeler Core technology to run NAM models on Raspberry Pis)
We went through the exciting journey of optimizing Neural Amp Modeler DSP to run inference real-time on the tiny Daisy Seed Arm Cortex-M7. Think hand-rolled GEMM kernels for small matrices and other fun stuff :D
What makes it Neural?
The actual effect transforming the audio is a neural network. The beauty is that you can record someone playing guitar through an ultra-rare amp/filter/whatnot (both clean signal and output) and then train the neural network to replicate that.
Neural Amp Models are small neural nets that have been trained to emulate the sound of classic guitar amps and effects pedals. The neural networks are trained on amp inputs and amp outputs (or effect inputs and effect outputs). Neural Amp Modeller Core is open source, licensed under an MIT license. TONE300.com provides both free online training of Neural net Models, and a massive collection community-generated high-quality Neural Amp Models. NAM has a large and very active online community.
It is obviously not possible to run huge LLMs in realtime; but it's been known for some time that very small neural network ML models can run in realtime, and can produce absolutely stunning simulations of real amps. The models are typically in the order of thousands of weights, not billions of weights. (Not actually sure what the weight count is for the A2 pico models that are being discussed in the original post. OP may be able to help with that).
These tiny neural network models not only accurately reproduce the sound of the original amps, but also manage to reproduce the feel of playing the original amp as well. The quality is dramatically better than most previous amp simulations (and entirely competitive with really high-end amp simulation technologies like Kemperer). It is breakthrough technology, for guitar pedals and amp simulators particularly, that literally changes everything in the music industry. The models are also relatively easy to train. TONE3000.com provides free online services for training models, and currently host a massive library of thousands of high-quality NAM models that are downloadable, free of charge.
The particularly interesting part of this report is that a single NAM model will be able to run on ridiculously tiny embedded processors. OP claims to have a 2nd-generation Pico NAM model running on a 500Mhz Cortex M7. First-generation Standard NAM models typically require a much more beefy processor: an ARM processor in the Pi 4 or Pi 5 sort of range (2.0Ghz Cortex A72, and 2.4Ghz Cortex A76 processors), or on x64 processors (an N100-class intel processor would be a good choice).
(Author of an open source project that uses Neural Amp Modeler Core technology to run NAM models on Raspberry Pis)
the joys of staring at assembly output
Pretty neat stuff. Thanks! I’ve never been a big amp -head, but am familiar with the board and appreciate the contributions!