Likely much worse write endurance than DRAM, since it’s still flash underneath. The saving grace is that model weights are mostly read-heavy, so endurance may matter less than it sounds.
This is almost entirely dominated by the read circuitry and the data path: it’s still taking 1/6 of a second to read the whole chip, which means that the flash cells aren’t working hard at all. (And that pitting the full weights of a dense model on these chips while using anywhere near all the capacity is a nonstarter if you intent to stream the weights as you run inference.)
What will be the durability/lifetime properties of this technology in comparison to traditional DRAM?
Likely much worse write endurance than DRAM, since it’s still flash underneath. The saving grace is that model weights are mostly read-heavy, so endurance may matter less than it sounds.
I had to double check those figures on Sk Hynix office web site [1], and it is not a typo or wrong capital "B".
It really is 3TB per second.
I literally paused for 5 min and thought how is this even possible.
[1] https://news.skhynix.com/en/hbf-at-fms-2026/
This is almost entirely dominated by the read circuitry and the data path: it’s still taking 1/6 of a second to read the whole chip, which means that the flash cells aren’t working hard at all. (And that pitting the full weights of a dense model on these chips while using anywhere near all the capacity is a nonstarter if you intent to stream the weights as you run inference.)