Checked out this company about a year ago and they only offered small models. Now I see they have GLM-fp8/Kimi and DeepSeek V4 Pro. Since workloads are predominantly cached input, I'm surprised to see no separate price for cached input vs uncached.
I hope the prices will drop significantly; with these prices you'll end up with thousands in monthly costs quickly.
Hopefully more hardware companies will be on the market in the coming years. If the Chinese eventually start competing with the current memory makers, maybe that will help.
Checked out this company about a year ago and they only offered small models. Now I see they have GLM-fp8/Kimi and DeepSeek V4 Pro. Since workloads are predominantly cached input, I'm surprised to see no separate price for cached input vs uncached. I hope the prices will drop significantly; with these prices you'll end up with thousands in monthly costs quickly. Hopefully more hardware companies will be on the market in the coming years. If the Chinese eventually start competing with the current memory makers, maybe that will help.
I train on AMD MI250X and managed to get Gemma 4 31B to work - but it took a lot of work on the software side.
This is very interesting, planning to write about it?
Also the vllm patch accompanying the blogpost: https://github.com/doublewordai/vllm-amd-blog-doubleword
We at doubleword are bullish for AMD for low-interactivity inference - it does just take a bigger lift on the software side...
Are you long AMD?
Interesting that you ask that as AMD hits another ATH.
Then you are definitely long on AMD.
Nice work! Would DeepSeek V4 Pro on 8xMI300X work with these patches?
Nice work and thanks for being a customer.
(CEO Hot Aisle)
I wish you guys could partner with Modular to get Mojo inference working on your hardware, e.g. https://www.modular.com/models/deepseek-v4-pro