Cerebras Disaggregated Inference

(cerebras.ai)

81 points | by emschwartz 9 hours ago ago

19 comments

  • ismailmaj an hour ago

    Glad they are upfront about their chips being a bad fit for prefill workloads, their previous article completely dodged the question: https://www.cerebras.ai/blog/how-we-deploy-trillion-paramete...

  • aabdi 8 hours ago

    5x throughput more than b4, same 10x tokens/s as compared to standard.

    looks like disagg/wide ep is applicable to cerebras chips as well. kind of amazing.

    https://www.cerebras.ai/blog/how-we-deploy-trillion-paramete...

    • oivaksef 7 hours ago

      They talk about Kimi - I would love to run that on cerebras, but it's not available from them directly. Is there a provider I can use this at?

  • hephaes7us 6 hours ago

    It sounds like this might mean they could do real prefix caching?

    Cerebras would be much more interesting to me if cache hits resulted in a commensurate savings on my end.

  • speedping 4 hours ago

    Cross-platform prefill/decode split is an incredible feat, kudos to them. I wonder how much time is spent transporting kv cache, and if the gains would be even higher for LLMs with ultra-compressed kv cache like the latest deepseek.

  • villgax 41 minutes ago

    Lol why would anyone want to manage multiple vendors for the remotest non-LLM work even. Not everything is an LLM

  • augment_me 8 hours ago

    Have they solved the issue yet where they need to keep running ghost tokens to keep the board hot to prevent it from cracking, which causes massive energy draw in idle, and reduces lifetime by 3 times the estimate?

    https://culperresearch.com/wp-content/uploads/2026/10/Culper...

    • eh8 8 hours ago

      The short seller report contains a bunch of errors:

      1. The CS-3 operates on average at <65c, well below its thermal limits. Considering the actual thermal parameters of the wafer, using the short seller's own temperature equation would imply the wafer can operate at 85c for 33 years

      2. "Ghost tokens" are not unique to Cerebras and the 3x lifespan reduction claim is based on extrapolating GPU figures which is a big logical leap

      3. "since the introduction of the CS-3 in 2024, less than half of one percent of all CS-3 wafers deployed have been decommissioned."

      https://cerebras.ai/blog/what-it-took-to-build-wafer-scale-a...

      Disclaimer: I work at Cerebras

      • augment_me 8 hours ago

        Very interesting! Thanks for answering with your take, and the link.

        Questions:

        1) What is the idle power draw of the chip? Is the 80% number wrong?

        Nvidia is idling at around 180W/15% of their max for B300. So saying that "it's not unique to Cerebras" is meaningless if we don't consider the power draw itself

        2) Decommissioned is not the same as RMA:d/refurbished/redeployed, so giving this number is kind of dodging the question IMO

        • wmf 8 hours ago

          None of these expensive chips should be idling.

          • augment_me 7 hours ago

            What a non-answer. And there should be perfect latency hiding, and there should always be predictable power-of-2 workloads, and all weights should always be homogenous, etc.

            • adgjlsfhk1 7 hours ago

              It's a reasonable answer. Consumer chips are idle 95% of the time, so idle power draw matters. AI datacenters are currently under capacity, so rather than optimizing for idle power draw, just have work to do.

              • augment_me 6 hours ago

                No it's not, it does not reflect reality. I have worked with both industry and public clusters. The average is around 50% idle-time, many university clusters sit at around 80% idle-time.

                The absolute best I have seen at a hyperscalar internal with predictable, constant workloads due to own projects, and cracked routing infra is 15% idle-time.

                • Tuna-Fish 3 hours ago

                  There are both latency-sensitive (serving tokens to users) and non-latency sensitive (doing RLAIF requests to help train models) workloads, and there are enough of the latter chasing compute that it should not be hard to get near perfect utilization. The non-latency sensitive workloads do pay a lot worse.

                  This might become a problem later, but right now, if you have AI compute idling, you are doing something very wrong.

      • latchkey 7 hours ago

        As far as I understand it, data can also be retrieved from the ghost tokens, so they are valuable information too. This offsets the cost of them.

      • lukeify 8 hours ago

        So which Cerebras statement is in error then?

        > First, in Cerebras’s preliminary IPO prospectuses filed through December 2025, the Company assigned its computer equipment useful lives of just 1.5 to 3 years – in-line with both our Arrhenius modeling and what many of the Company’s hardware customers have described to us.4 Just four months later, in the Company’s April 2026 S-1, Cerebras nearly doubled that range to 3 to 5 years – a change that significantly inflated reported margins. The Company appears to have never explained the reason for this change.

        Was the useful life of a wafer 1.5–3 years on your IPO filing or 3–5 years in your S-1?

        Hard to believe a company with such skilled engineers couldn't figure out the correct useful wafer life given existing industry data. Or (more likely) are we just looking at standard corporate BS?

        • augment_me 4 hours ago

          Both are wrong.

          On the physics of silicon lifetime, Cerebras is right and Culper's calculation is wrong. As Cerebras blog points out, Culper took a 5-7 year GPU economic depreciation schedule and divided it by a 3x thermal power-draw ratio to claim the wafers physically die in ~2 years.

          However in the blog, Feldman writes:

          > By the next update of our S-1, we had set out to build a private cloud; the reason was that AI native customers and developers asked us to. One consequence was that the mix of fixed assets we used internally had changed to include production systems we owned for the cloud business. We therefore modified our depreciation schedule to 3-5 years.

          However the timeline is:

          Mar 2024: CS-3 launches.

          Jun 2024: First confidential IPO filing (DRS) sets equipment useful life at 1.5–3 years.

          Aug 2024: Cerebras Cloud launches (followed by $383M in 2025 cloud CapEx).

          Dec 22, 2025: Updated DRS still certifies a 1.5–3 year useful life, 16 months after launching the cloud, contradicting the statement in the blog.

          Late Dec 2025: Signs the 750 MW OpenAI cloud deal (3.7-year average contract).

          Apr 17, 2026: Public S-1 (weeks before the May 2026 IPO) doubles useful life to 3–5 years, boosting 1H 2026 gross margins from 24.0% to 29.9%.

          ---

          So the blog is not having the right causal relationship here, if launching the developer cloud in August 2024 was the reason for extending useful life to 3–5 years, that change would have been in the December 2025 filing.

          Real reasons probably touch upon their deal with OpenAI, which had to be taken on OpenAI's terms because the IPO would not have happened with a 86% UAE-based G42 (which stalled the 2024 IPO over China ties). Cerebras needed a big US anchor customer to go public, and OpenAI opted to buy cloud capacity rather than buy and run the hardware itself(you can guess why). Once you have to capitalize massive owned cloud infra for a 3.7-year contract right before an IPO, stretching hardware depreciation from 1.5–3 years to 3–5 years is kind of free because all other cloud providers use a 3–6 year schedule, so it's easy to get an auditor to sign off on it.

        • wmf 8 hours ago

          That's answered in the blog.

    • bob1029 3 hours ago

      This reminds me of AMD when zen was taking off.

      If short sellers are so desperate that their crap is showing up on HN, it might be time to buy CBRS.