There is not a single person mentioned on the website, github created 3 days ago, no real contact, everything hidden. Completely anonymous. Domain owner hidden.
I wonder how proprietary it all is though, since the BitNet b1.58 paper has been out for a couple years now: https://arxiv.org/abs/2402.17764
From the wikipedia on 1.58 bit llms: "BitNet derives its performance from being trained natively in 1.58 bit instead of being quantized from a full-precision model after training. Still, training is an expensive process, and it would be desirable to be able to somehow convert an existing model to 1.58 bits. In 2024, HuggingFace reported a way to gradually ramp up the 1.58-bit quantization in fine-tuning an existing model down to 1.58 bits."
I just wonder how many of these labs are basically following the huggingface recipe here and possibly tweaking it and releasing models without huge training costs.
Independent testing of prismml suggest quite a capability drop off outside of their cherry picked benchmarks. I'll be curious to see what this model achieves though.
"Neutrino-1 8B was trained natively in its shipping format. There is no full-precision product model that was rounded afterward: the ternary representation is the medium the weights learned in, and the training methods that hold this quality at this depth are the lab’s unpublished work. The findings below are the part that travels."
This statement seems misleading at best.
Both the model page and the release page are basically unintelligible - I don't have a ton of faith in the work here, at least PrismML write coherent releases for their models.
Edit: Another beautiful piece of prose here, I almost wonder if they used the 8b model to generate the content for this release...
"Across the 6.95B coded weights, 62.63% sit at zero and the remainder splits 18.68% plus to 18.69% minus: sign-balanced to a hundredth of a point with no constraint asking for it."
I really had high hopes for the larger Ternary Bonsai and it feels like there is scope to improve, but I get the sense (albeit a naïve, probably not fully informed sense) that improvement can perhaps only come by training directly into ternary.
AI doesn't want anything, so it doesn't care whether it conveys meaning in its writing. And, apparently the developers of this project also don't care whether it conveys meaning. They just assume we'll wade through the slop? I dunno.
Can’t say I’m a fan of containers for this. A big chunk of local LLM gains come (imo) from the open modular nature of llama.cpp and friends. Easy to modify. Easy to experiment.
Containers are the proprietary binary blob in hardware world equivalent
I just tune out. It’s not worth knowing, following every development in the field. If something works now it will probably work in 8 months even if it’s no longer the new hype thing. Who cares.
Not using any of it is also a valid option though it doesn’t satisfy your FOMO. But nothing ever will.
There is not a single person mentioned on the website, github created 3 days ago, no real contact, everything hidden. Completely anonymous. Domain owner hidden.
There's a really interesting trend of labs using "proprietary" methods to convert existing models to a compressed ternary format.
PrismML actually targeted the same Qwen 8b model and got it down to 1.75gb here: https://prismml.com/news/ternary-bonsai
I wonder how proprietary it all is though, since the BitNet b1.58 paper has been out for a couple years now: https://arxiv.org/abs/2402.17764
From the wikipedia on 1.58 bit llms: "BitNet derives its performance from being trained natively in 1.58 bit instead of being quantized from a full-precision model after training. Still, training is an expensive process, and it would be desirable to be able to somehow convert an existing model to 1.58 bits. In 2024, HuggingFace reported a way to gradually ramp up the 1.58-bit quantization in fine-tuning an existing model down to 1.58 bits."
The section from huggingface is here: https://huggingface.co/blog/1_58_llm_extreme_quantization#fi...
I just wonder how many of these labs are basically following the huggingface recipe here and possibly tweaking it and releasing models without huge training costs.
Independent testing of prismml suggest quite a capability drop off outside of their cherry picked benchmarks. I'll be curious to see what this model achieves though.
Unfortunately Fermion Research appears to entirely AI generate all of their content here, even for the research section: https://www.fermionresearch.com/research/neutrino-8b/
"Neutrino-1 8B was trained natively in its shipping format. There is no full-precision product model that was rounded afterward: the ternary representation is the medium the weights learned in, and the training methods that hold this quality at this depth are the lab’s unpublished work. The findings below are the part that travels."
This statement seems misleading at best.
Both the model page and the release page are basically unintelligible - I don't have a ton of faith in the work here, at least PrismML write coherent releases for their models.
Edit: Another beautiful piece of prose here, I almost wonder if they used the 8b model to generate the content for this release...
"Across the 6.95B coded weights, 62.63% sit at zero and the remainder splits 18.68% plus to 18.69% minus: sign-balanced to a hundredth of a point with no constraint asking for it."
I really had high hopes for the larger Ternary Bonsai and it feels like there is scope to improve, but I get the sense (albeit a naïve, probably not fully informed sense) that improvement can perhaps only come by training directly into ternary.
The content on that page is too AI-generated to make sense to me; I don't understand what the model is for.
AI doesn't want anything, so it doesn't care whether it conveys meaning in its writing. And, apparently the developers of this project also don't care whether it conveys meaning. They just assume we'll wade through the slop? I dunno.
Can’t say I’m a fan of containers for this. A big chunk of local LLM gains come (imo) from the open modular nature of llama.cpp and friends. Easy to modify. Easy to experiment.
Containers are the proprietary binary blob in hardware world equivalent
Largely outperformed by Ternary-Bonsai-8B by their own chart, doesn't seem clear what their special sauce is here.
There’s a new announcement every other day wrt models. How do y’all keep track of them all same know what’s decent? Good grief!
And if it’s decent today, it’s shit in eight months! I tool hop as much as the next dev but this is a bit much.
I just tune out. It’s not worth knowing, following every development in the field. If something works now it will probably work in 8 months even if it’s no longer the new hype thing. Who cares.
Not using any of it is also a valid option though it doesn’t satisfy your FOMO. But nothing ever will.
The real strength of an 8B model is efficiency. It will be interesting to see the balance between performance and inference cost.
AI slop site with AI slop research...
Blog populated with incoherent PR material generated by Yet Another AI.
Sigh...