Do you expect this leapfrogging to persist for a while longer? The collective response to this tug of war seems overwhelmingly simplified and “gotcha!” meme-based. Public discourse overall appears to lean into a predictable lack of reservation for exponential leaps between companies, unknowns, and established methods.
Circling back to this because it connects directly to something playing out right now. Qwen 3.5's small models shipped and the headline is '4B beats 80B'. Sounds wild until you realise it's 4B dense vs ~3B active in the MoE. The architecture lesson in this post is exactly what people need before taking that claim at face value. I broke down the whole thing here: https://reading.sh/your-laptop-is-an-ai-server-now-370bad238461?sk=1cf7a4391e614720ecbd6e9bc3f076a2
Great deep dive and very helpful. Thanks for writing this!
Thank you so much, and of course!
Amazing article, a true gem. I would be interested to learn more about how DeepSeek used reinforcement learning to such great effect.
Working on that article now!
Great in depth article!
Thanks for the kind words!
Great in-depth article on MoE LLMs.
Thank you very much!
I agree with Dr. Bamania- this was very rewarding reading.
Thank you! I'm glad you found it helpful!
Do you expect this leapfrogging to persist for a while longer? The collective response to this tug of war seems overwhelmingly simplified and “gotcha!” meme-based. Public discourse overall appears to lean into a predictable lack of reservation for exponential leaps between companies, unknowns, and established methods.
Circling back to this because it connects directly to something playing out right now. Qwen 3.5's small models shipped and the headline is '4B beats 80B'. Sounds wild until you realise it's 4B dense vs ~3B active in the MoE. The architecture lesson in this post is exactly what people need before taking that claim at face value. I broke down the whole thing here: https://reading.sh/your-laptop-is-an-ai-server-now-370bad238461?sk=1cf7a4391e614720ecbd6e9bc3f076a2
Great article. I couldn't finish it in one go though. I have book marked it and will come back to it later.