Doesn't matter if you care about AI speed, AI capability, or AI cost... centralized AI has lost the lead and cannot regain it... and the free market is starting to move
Anyone still using the term ‘shizz’ is alright by me. I love this validation, I felt it coming when I read about Bittensor in 2022 and am poorer still because of being too early; however, now I’m vindicated and ready to win in AI again (I still have all those coins too).
What if closed model providers crack down on this and prohibit their models from being blended with others? If it's existential, I don't see why they wouldn't do this so long as they're at the frontier. And if large labs also crack down on distillation, I'm not sure how confident I am that pure open source models will continue to keep up.
After trying OpenClaw and was horrified by cost and security, I decided to see if I could build my own.
I built a hybrid solution using a multi tier approach. Gemini Pro for master architect, Gemini Flash for the orquestrator with LangGraph, and local models (Qwen coder and now gemma4) for coding chunks scaffolded by the architect. I then added an on demand tier I call the Think Tank which prompts Claude, ChatGPT, Grok and Gemini, pulls the responses and send them all back to the think tank for a second pass and then takes the results back into Gemini Pro to summarize and then act.
The quality and cost is shockingly superior to any single model alone.
I thought I would add that I use LiteLLM for a consistent interface to all the models. This makes it easy to code multi-model intelligence.
I also use llama-guard3 and docker sandboxes for security, chainlit for chat, Monaco for code display, and postgres/pgvector for gathering experience with semantic search for recall and context distillation support.
This piece pushed me to actually build the pattern, not just nod at it. I added a mode called Basna (Croatian word for Fable) to my open-source agent platform: a router reads the task, picks the smallest set of specialist agents that can answer it (1 for something trivial, a handful for something hard), runs them in parallel, and merges their outputs weighted by each agent's learned reliability per domain, so routing gets better over time.
The thing that clicked for me from your post: the win is in the routing, not the ensembling. Naive "ask 10 models and fuse" is the expensive cousin, slower and pricier than the monolith. The efficiency only shows up when a cheap router activates the minimal relevant subset and you don't pay a fusion tax unless you have to. So Basna only invokes a synthesizer when the agents genuinely disagree; when they converge it just takes the highest-reliability answer. No fusion call, no cost.
@Roman I think you're right that fusion is harder and less efficient for long-running agentic/coding tasks than for hard-questions or deep-research. That's exactly where I've kept it conservative: route narrow, merge cheap, only synthesize on conflict. Early days, but the network-source framing held up well enough that I shipped it.
Yes! And I shipped Basna to my Captain Claw. Results are insane! With the fusion approach results are really rich, and with additional analysis pass for blind spots is revealing! I have been testing it on DeepSeek models, and next test is to use small Qwens!
Does this vision not depend a lot on how many near-frontier open-weight followers exist ( specifically Chinese labs)? How does it run in a scenario where distillation gets much harder and China looser export controls do not solve China's compute problem in the next 3-4 years? Especially if capability gains keep eating up compute, like inference-time scaling, or creating many virtual RL environments to unlock new economically meaningful tasks, the gap keeps widening. So I'm wondering whether you'd defend the market will still push for world-level AI even if frontier narrows to 2-3 compute-rich, closed American labs, like what stops them from internalizing the ensemble gains themselves and closing off third-party blending, so it still concentrates on just a few nodes?
Er, not sure that's really how mainframes worked? Do you have referenes for this? Parallel computing is really tricky. Nobody actually measures mainframe performance in terms of "strength", so what are you really trying to suggest? Supercomputers are what happens when you add mainframes together -but having them distributed over the Internet at that time was absolute madness. For this new paradigm, i'm still not sure what adding the Internet really adds. Sure, run multiple models to cover for individual weaknesses... but I'm not conviced that running it twice on two huge frontier models is something that's going to take off in a big way/scale.
I am not clear on one of the basic things with this Network of neural models , and I know I may be missing, is - having a 100 faster cheaper computing power + model with less accuracy doesn't add value in use cases where performance as well as accuracy, security are paramount. The risk of reliance on a centralized model is just distributed and not mitigated, IMO, and for systems, where models are running, needing high availability this setup may not be able to meet the SLA ?.. first of all who can own the SLA... Thoughts
As an idiot PM (forgive me), I must be missing something. If the solution is adding Fable and Opus running in parallel and then processing the results, isn't this far more costly than simply running Fable itself?
Is this just an optional shift in: speed, accuracy, cost -- pick two?
There are permutations which are more expensive - but OpenRouter showed there are some permutations which still achieved Fable level quality but at half the price.
The fact that on OpenRouter's chart, DeepSeek V4 Pro (solo) is ahead of GPT 5.5 (solo), should make you suspicious of the choice of the benchmark and of how really useful the result is. Achieving useful "fusion" in long-running agentic coding/engineering tasks might be harder (and far less efficient in terms of cost) than in hard-questions-like and deep-research-like tasks.
Honestly I'd be doubtful of it if I haven't spent so much time in the last 12 months beating humanity's last exam over and over using the same strategy. This stuff holds up.
Don't use openrouter's fusion. Make your own ensanble in your aplication.
Qwen 3.7 + Deep Seek V4 Flash + GLM 5.1 outperforms both GPT 5.5 and Opus 4.8 at a fraction of the price for the tasks the end-user want in the app I'm building.
Anyone still using the term ‘shizz’ is alright by me. I love this validation, I felt it coming when I read about Bittensor in 2022 and am poorer still because of being too early; however, now I’m vindicated and ready to win in AI again (I still have all those coins too).
you're the shizz with the rizz Roy!
What if closed model providers crack down on this and prohibit their models from being blended with others? If it's existential, I don't see why they wouldn't do this so long as they're at the frontier. And if large labs also crack down on distillation, I'm not sure how confident I am that pure open source models will continue to keep up.
This is like asking "what if mainframe computers cracked down on being connected by telephone lines in the 1960s."
They'd just get left behind even faster. Look at Kimi K2.7... it's already parity with frontier AI systems.
We still have the silicon availability issue. Aren’t there only like 3 manufacturers in the world?
I learned this 6 months ago.
After trying OpenClaw and was horrified by cost and security, I decided to see if I could build my own.
I built a hybrid solution using a multi tier approach. Gemini Pro for master architect, Gemini Flash for the orquestrator with LangGraph, and local models (Qwen coder and now gemma4) for coding chunks scaffolded by the architect. I then added an on demand tier I call the Think Tank which prompts Claude, ChatGPT, Grok and Gemini, pulls the responses and send them all back to the think tank for a second pass and then takes the results back into Gemini Pro to summarize and then act.
The quality and cost is shockingly superior to any single model alone.
This is exactly the gateway!
I thought I would add that I use LiteLLM for a consistent interface to all the models. This makes it easy to code multi-model intelligence.
I also use llama-guard3 and docker sandboxes for security, chainlit for chat, Monaco for code display, and postgres/pgvector for gathering experience with semantic search for recall and context distillation support.
This piece pushed me to actually build the pattern, not just nod at it. I added a mode called Basna (Croatian word for Fable) to my open-source agent platform: a router reads the task, picks the smallest set of specialist agents that can answer it (1 for something trivial, a handful for something hard), runs them in parallel, and merges their outputs weighted by each agent's learned reliability per domain, so routing gets better over time.
The thing that clicked for me from your post: the win is in the routing, not the ensembling. Naive "ask 10 models and fuse" is the expensive cousin, slower and pricier than the monolith. The efficiency only shows up when a cheap router activates the minimal relevant subset and you don't pay a fusion tax unless you have to. So Basna only invokes a synthesizer when the agents genuinely disagree; when they converge it just takes the highest-reliability answer. No fusion call, no cost.
@Roman I think you're right that fusion is harder and less efficient for long-running agentic/coding tasks than for hard-questions or deep-research. That's exactly where I've kept it conservative: route narrow, merge cheap, only synthesize on conflict. Early days, but the network-source framing held up well enough that I shipped it.
This is the way!
Yes! And I shipped Basna to my Captain Claw. Results are insane! With the fusion approach results are really rich, and with additional analysis pass for blind spots is revealing! I have been testing it on DeepSeek models, and next test is to use small Qwens!
Does this vision not depend a lot on how many near-frontier open-weight followers exist ( specifically Chinese labs)? How does it run in a scenario where distillation gets much harder and China looser export controls do not solve China's compute problem in the next 3-4 years? Especially if capability gains keep eating up compute, like inference-time scaling, or creating many virtual RL environments to unlock new economically meaningful tasks, the gap keeps widening. So I'm wondering whether you'd defend the market will still push for world-level AI even if frontier narrows to 2-3 compute-rich, closed American labs, like what stops them from internalizing the ensemble gains themselves and closing off third-party blending, so it still concentrates on just a few nodes?
https://sup.ai/ shut down
Er, not sure that's really how mainframes worked? Do you have referenes for this? Parallel computing is really tricky. Nobody actually measures mainframe performance in terms of "strength", so what are you really trying to suggest? Supercomputers are what happens when you add mainframes together -but having them distributed over the Internet at that time was absolute madness. For this new paradigm, i'm still not sure what adding the Internet really adds. Sure, run multiple models to cover for individual weaknesses... but I'm not conviced that running it twice on two huge frontier models is something that's going to take off in a big way/scale.
I am not clear on one of the basic things with this Network of neural models , and I know I may be missing, is - having a 100 faster cheaper computing power + model with less accuracy doesn't add value in use cases where performance as well as accuracy, security are paramount. The risk of reliance on a centralized model is just distributed and not mitigated, IMO, and for systems, where models are running, needing high availability this setup may not be able to meet the SLA ?.. first of all who can own the SLA... Thoughts
As an idiot PM (forgive me), I must be missing something. If the solution is adding Fable and Opus running in parallel and then processing the results, isn't this far more costly than simply running Fable itself?
Is this just an optional shift in: speed, accuracy, cost -- pick two?
There are permutations which are more expensive - but OpenRouter showed there are some permutations which still achieved Fable level quality but at half the price.
The fact that on OpenRouter's chart, DeepSeek V4 Pro (solo) is ahead of GPT 5.5 (solo), should make you suspicious of the choice of the benchmark and of how really useful the result is. Achieving useful "fusion" in long-running agentic coding/engineering tasks might be harder (and far less efficient in terms of cost) than in hard-questions-like and deep-research-like tasks.
Honestly I'd be doubtful of it if I haven't spent so much time in the last 12 months beating humanity's last exam over and over using the same strategy. This stuff holds up.
Don't use openrouter's fusion. Make your own ensanble in your aplication.
Qwen 3.7 + Deep Seek V4 Flash + GLM 5.1 outperforms both GPT 5.5 and Opus 4.8 at a fraction of the price for the tasks the end-user want in the app I'm building.