41 points | by brainless 3 hours ago ago
4 comments
Can this produce a useful model? So far 1 bit quants have been less useful than smaller models that use the same memory
I recently found this 1.58-bit model for ASR and it's surprising good (and very fast), that being said it's not a LLM.
https://huggingface.co/moondream/parakeet-redux
Their paper shows this comes with huge quality loss, but that doesn't make it a negative result by any means
Has anyone tried this on apple silicon M1-5? Any benchmarks/comps?
Can this produce a useful model? So far 1 bit quants have been less useful than smaller models that use the same memory
I recently found this 1.58-bit model for ASR and it's surprising good (and very fast), that being said it's not a LLM.
https://huggingface.co/moondream/parakeet-redux
Their paper shows this comes with huge quality loss, but that doesn't make it a negative result by any means
Has anyone tried this on apple silicon M1-5? Any benchmarks/comps?