Adding to my homelab stack, hopefully it doesn't overthink like the little model. Actually, hoping it thinks a bit less. Wait actually I'm really praying it reasons a bit more directly. But wait, I'm really sure that it must be a bit better.
My stack is basically deer-flow with Qwen3.5-122B-A10B; this hopefully will be a speed and intelligence improvement. Running deer-flow overnight on any research topic or verify clear scoped programming issue is really neat.
Also, heating my home during the winter is nice.
Oh, also, I use llamacpp with --reasoning-budget; very simple way to move on.
This is a good counter argument. But you have to note that this is after OpenAI cut Luna costs by 80%. If you compare launch pricing, Qwen probably comes out ahead on a cost-performance basis.
For world knowledge, you'd want it to find and reference the source material to be sure. At that point, it doesn't matter if the knowledge is embedded.
Waiting for llama.cpp support to land, but this might be a big deal for Strix Halo users.
6B active params helps around the memory bandwidth constraints, but a 128GB box can probably run the Q3/Q4 quants fairly easily with a decent context size. This might actually be better for strix users than 27B, which was already very good.
Aside from the pelican, I am sort of impressed by the fact that things are going the way in terms of really impressive small models.
Also I love how this uses N-gram embedding. I think that Longcat was the first one who used it (I submitted that submission on hackernews because I really just loved the idea of it that I understood), I am certainly more interested in local LLM models and its interesting how they are utilizing new architectures to do some really impressive optimizations!
(Do note that I created it using a free rate limited end-point that I found on the huggingface space section: victor-chat-with-qwen3-8-flash-next.hf.space)
How is input token efficiency/verbosity on this model? Has anyone tried? GLM 5.2 was doing lot of turns and thinking piling up input tokens in the context (compared to Claude and GPT models). Then Qwen3.8-27B was 2x of that. Both delivered good output results but those cumulative input token costs were not cheap. Note this is on our specific business workloads. Genuinely interested in other people's experience (if you are able to try it out).
If there’s news, then yes. This is a pretty great new release for those still stuck on Qwen3.6 35B A3B if they have enough memory but don’t have super powerful compute.
I wonder if I could get this running through vLLM on 6x Nvidia L4 - the 3.6 worked great on 4 cards but sadly TP6 just isn’t a thing and I don’t have 8 cards available, maybe it’s gonna be okay with like TP2 and MTP. I have no idea at this time, probably need to test out what even might be possible.
This particular release is interesting because it's a preview of qwen4 architecture. And, while benchmarks are iffy, this is a direct comparison, by the same team, with qwen3.8-27b that was pretty well received for a local model.
This "next" release adds a new concept, first public release with n-grams, I think. And it's in a MoE size that is likely to be very fast and cheap to serve (faster than 27b for sure). It's also well suited for inference on alternative compute (i.e. sparks, macs, etc) so it's relevant to local users.
Adding to my homelab stack, hopefully it doesn't overthink like the little model. Actually, hoping it thinks a bit less. Wait actually I'm really praying it reasons a bit more directly. But wait, I'm really sure that it must be a bit better.
That's low reasoning for a model, but max for a HN comment.
My stack is basically deer-flow with Qwen3.5-122B-A10B; this hopefully will be a speed and intelligence improvement. Running deer-flow overnight on any research topic or verify clear scoped programming issue is really neat.
Also, heating my home during the winter is nice.
Oh, also, I use llamacpp with --reasoning-budget; very simple way to move on.
Didn't expect it to beat 3.8 27B so cleanly.
Opus 4.6 Max self-hosted at 30 tok/s on a 5k Macbook in Aug 2026. The LLM timelines are crazy.
For comparison with hosted models, GPT 5.6 Luna scores 67% on DeepSWE, compared to 59% here for Qwen.
Luna is $0.20 / $1.20 vs $0.16 / $0.47 with Qwen.
This is a good counter argument. But you have to note that this is after OpenAI cut Luna costs by 80%. If you compare launch pricing, Qwen probably comes out ahead on a cost-performance basis.
The luna cost cuts were real though, not a one time promotion or something, due to some optimization (probably distillation?) that openai did.
I don't like these comparisons. Sure it is impressive, but it does not have a world knowledge of larger models. It has most of theirs intelligence.
For world knowledge, you'd want it to find and reference the source material to be sure. At that point, it doesn't matter if the knowledge is embedded.
If/when we can get larger context this will mostly be mitigated by these smaller models being able to search the internet.
Self-learning/improving would be even better but that's still a long way to go.
>Opus 4.6 Max self-hosted at 30 tok/s on a 5k Macbook in Aug 2026. The LLM timelines are crazy.
How much memory does this translate to and what quantization (if any) were applied?
128GB, 4-bit quantized.
It's in Unsloth Desktop already. Looks like it's 73GB, so 128GB Mac or Strix Halo etc will work. Exciting!
Download is available, but likely need to wait for an update, I get this which is understandable with the architectural change :
Original error: llama.cpp does not support this GGUF's model architecture ('qwen4exp')
Edit : Saw the pull request, should arrive soon enough https://github.com/ggml-org/llama.cpp/pull/27742
Waiting for llama.cpp support to land, but this might be a big deal for Strix Halo users.
6B active params helps around the memory bandwidth constraints, but a 128GB box can probably run the Q3/Q4 quants fairly easily with a decent context size. This might actually be better for strix users than 27B, which was already very good.
this is a new architecture (foreshadowing qwen 4)
> trained at just 1/9 the cost of Qwen3.7-Plus, while outperforming it across the board
https://x.com/Alibaba_Qwen/status/2092591393424515114
Pelican: https://gist.github.com/SerJaimeLannister/8fdef9c00175da0ca6...
Aside from the pelican, I am sort of impressed by the fact that things are going the way in terms of really impressive small models.
Also I love how this uses N-gram embedding. I think that Longcat was the first one who used it (I submitted that submission on hackernews because I really just loved the idea of it that I understood), I am certainly more interested in local LLM models and its interesting how they are utilizing new architectures to do some really impressive optimizations!
(Do note that I created it using a free rate limited end-point that I found on the huggingface space section: victor-chat-with-qwen3-8-flash-next.hf.space)
How is input token efficiency/verbosity on this model? Has anyone tried? GLM 5.2 was doing lot of turns and thinking piling up input tokens in the context (compared to Claude and GPT models). Then Qwen3.8-27B was 2x of that. Both delivered good output results but those cumulative input token costs were not cheap. Note this is on our specific business workloads. Genuinely interested in other people's experience (if you are able to try it out).
Haven't tried, would be surprised if it's any different.
It's new arch demo for future Qwen 4 family, but (as I understand) training recipe/data is same as any other 3.8 model.
Those benchmarks look seriously impressive.. considering how small of a MoE model this is.
FYI: nothing seems to be able to run this (easily) yet. llama.cpp, vllm etc I couldn't get working because of no support in the mainline version.
They are giving pointers to how to run it now using for example https://recipes.vllm.ai/Qwen/Qwen3.8-Flash-Next (and an especially provided vllm release).
It looks like this also undercuts the already absurdly inexpensive Deepseek Flash in pricing. Wild.
Where are you seeing that? At the bottom of this post from Qwen I see:
Qwen 3.8 flash: $0.16 / $0.47
Compared to
Deepseek 0723: $0.03 / $0.075
(units in USD/m tok)
0.03 / 0.075 ? Where can i get that prices? Especially during peak hours DS4flash became much more money hungry than last month.
https://api-docs.deepseek.com/quick_start/pricing
Great. It's also smaller than DSV4 Flash, so it makes sense that way.
looks like it's better than deepseek v4 flash
Will this be cheaper than DS4flash ?
Looks like a good model for strix halo
indeed, can't wait for it to be supported by llama.cpp (or other engines)?
do we really need breaking news about qwen posted every single day?
If there’s news, then yes. This is a pretty great new release for those still stuck on Qwen3.6 35B A3B if they have enough memory but don’t have super powerful compute.
I wonder if I could get this running through vLLM on 6x Nvidia L4 - the 3.6 worked great on 4 cards but sadly TP6 just isn’t a thing and I don’t have 8 cards available, maybe it’s gonna be okay with like TP2 and MTP. I have no idea at this time, probably need to test out what even might be possible.
This particular release is interesting because it's a preview of qwen4 architecture. And, while benchmarks are iffy, this is a direct comparison, by the same team, with qwen3.8-27b that was pretty well received for a local model.
This "next" release adds a new concept, first public release with n-grams, I think. And it's in a MoE size that is likely to be very fast and cheap to serve (faster than 27b for sure). It's also well suited for inference on alternative compute (i.e. sparks, macs, etc) so it's relevant to local users.
I and presumably quite a few others with AMD AI or Apple Mac platforms are very impacted by this.
:)
It is very relevant and for a certain group of us, far more impactful to our work the next month(s) than any blog post could be.
this is a new architecture (foreshadowing qwen 4)
> trained at just 1/9 the cost of Qwen3.7-Plus, while outperforming it across the board
https://x.com/Alibaba_Qwen/status/2092591393424515114
There are many topics, personalities and politicians we hear about daily who have no merit.
Qwen's advances do (currently) have merit.
This actually is meaningful news, I think. Pretty wide audience appeal in the local LLM space too.
yes there’s no shortage of online real estate