After Coding Models: Towards the Plurality of Artificial Minds
Over the past few weeks I made dozens of phone calls and exchanged a lot with friends at the frontier of the various frontier labs, and have a stage summary about coding models, multimodality, and world models: towards the plurality of artificial minds
First, I have to admit the mistake of my judgment. In the past three years, on many occasions I vocally expressed that LLM is a dead end, that it cannot possibly solve multimodal intelligence, cannot possibly solve spatial intelligence, cannot possibly solve embodied intelligence, that the VLM, VLA route can never possibly work. Now it is proven that my judgment of the past three years was wrong. I think the capabilities that GPT-6 and Fable 5.5 now display, I think it must be said very directly, have basically already solved all the multimodal tasks of VLM; in 3D generation capability they already pose an enormous challenge to many native 3D models, for example World Labs; rather than end2end generating a gs, nerf, mesh, coding models can in fact make 3D scenes, and in detail control may do it even better; and recently, there is also Fable directly making images, making videos through coding, with results even better than many native image models and video models. This is a fact, and we need to admit it.
Why do coding models work, not only work for coding, but also work for image, work for video, work for 3D? I have always believed in the scaling law, believing that as long as you have unlimited compute, you could do anything; but why then did I previously judge that multimodal, spatial, embodied llm would not work? Because I thought the language representation of llm is a one-dimensional representation, while images need two dimensions, videos need three dimensions, the world is four dimensions; using a one-dimensional representation to express a four-dimensional world, the result can only be stochastic parrots and shortcut.
Obviously, this judgment of mine was wrong. But where was it wrong? I think there are two points: underestimated the speed of llm scaling; I always thought that you can always use 1D to overfit 2D, 3D, but that way you would be very inefficient, and by the time your llm overfit succeeded, my 2D, 3D models would long since have been made to work. Obviously, on this matter of scaling, I judged wrong. Second, overestimated the speed at which the reasoning of 2D, 3D models emerges. In 2025, Veo and Sora already showed some emerged capabilities, but we have not seen these capabilities be further amplified; of course, we also have not seen this matter of scaling be practiced in multimodal models, whether image, or video, or 3D, the way it was on llm.
Then how fierce actually is llm scaling? From the information I have, GPT-6 is a model trained on 100,000 GPUs in two weeks, with more than 1 billion images inside, more than 1 million hours of video data, and more than 1 million hours of embodied data. Such a model, to run one pretraining run, needs 1.3 billion to 1.6 billion US dollars, and the next 400,000-GPU model, to train once is 5 billion US dollars. The cost of one pretraining is larger than the funding amount of the largest world model company, AMI Labs.
However, the resources of humanity are limited. Humanity as a whole does not now have the confidence to give the world model road 400,000 GPUs and 5 billion US dollars to do scaling, because even if 5 billion US dollars are smashed in, it does not necessarily work; however, smashing the same resources into the one-dimensional coding model not only made coding work, but likewise also made work many things that other technical routes claim.
With things pushed to here, many people will wonder, whether in the future we only need GPT and Claude, whether AI only has OpenAI and Anthropic left. Menlo Ventures' Deedy Das wrote a piece a few days ago, saying neolab is a pseudo-concept; the only window left for a neolab to become OpenAI and Anthropic was in fact only the spring of 2024, that is, the time period of GPT-4; only at that time could a venture raise catch up with the expenditure needed for model scaling; what remains are only two ways out, verticalization and magic.
A few days ago in SF I had dinner with a VC friend, I asked how things had been lately, the answer was “What else is there to invest in? There is nothing to invest in anymore, just invest in OpenAI and Anthropic.” A friend next to us asked, “But the valuations are already two trillion?”, answer: “If you believe in the future only OpenAI and Anthropic are left, that's where the money should go”
So is this our projection of the future? I think a basic judgment has to be made: that is, is code representation the optimal, universal representation? Because essentially you can do anything with code, you can write code with code, you can make images with code, you can make videos with code, you can make 3D with code, you can make an interactive world model with code, in which this world is written by code, rather than coming out of diffusion, you can control robots with code.
Coming down this far, I cannot help marveling at how accurate Dario's judgment of things was: in 2011 he wrote in his phd thesis “intelligence will arise from the collective properties of large networks of relatively simple elements”, in 2016 in OAI special project he wrote “A program that can write other programs would be, for obvious reasons, very powerful.” Scaling and coding, both of these things he bet right.
But taking a step back, how do we make sense of this from our cognitive theories? How did coding just work? How did coding just become the most important link on the road to AGI?
In 2024 I wrote a small essay, about SayCan, discussing a bit how Fodor's Language of Thought optimizes embodied long-horizon planning; the specific claim is that the plans of all long-horizon tasks need LoT to solve, rather than internal model simulation, because LoT is obviously more efficient. After writing it I forgot about this thing.
What is LoT? LoT is an idea in cognitive science and philosophy of mind, that is, thought itself is composed of some kind of structure that is like language, but is not language, i.e., formal languages; of course coding is a kind of formal language, math is also a kind of formal language. This also constructed in philo of mind the cornerstone of so-called Symbolism. Given how successful deep learning has been, I have always scoffed at ideas like symbolism, but with the defection of some flag-bearers, they have also begun to admit that the reason llm thinking is so strong now is because they began LoT, began symbolic thinking.
According to Fodor's account, language of thought is all you need, symbolic representation is all you need, for thinking; all visual things do not count as thinking, merely low-level perception; in Liang Wenfeng's words, it is “multimodality is a pendant of intelligence”. Claude's tone is in fact also Fodor's tone.
I think Anthropic already has a cohesive pathway towards solving thinking, and maybe, to the Fodorians, they have already solved thinking. Because thinking is LoT, is the capability Fable now displays.
Then what is left that can be done? Going back to Fodor's own words, mind has three distinct problems: 1) logical thought 2) meaning 3) consciousness. I think the first one has already been solved. The second problem, meaning, that is, how meaning is formed, is in essence an embodiment problem, that is, Wittgenstein's saying, all meaning must refer in some form to things within the world in order to obtain meaning; the third problem, instead, is from Descartes onward, that there must be an inner theatre for consciousness to be possible (of course Hinton does not agree with this, in history he perhaps also has the standing to challenge Descartes)
With so many mystic statements, how concretely do they guide our practice in real life?
Anthropic has obviously already marched towards 2) and 3), their method should be, in the case of an llm backbone, adding a lot of multimodal, a lot of embodied data. It should be done very well. Then what do we do?
We should clearly draw how far coding intelligence can actually go. I think because coding can in fact solve all the things of the digital world, all the things on servers will become a subset of coding. math obviously is a subset of coding, AI4Science too. But the things in language, A and O may not necessarily be able to eat them, for example, Harvey, Legora instead have a lot of room.
But that is also where it stops.
Instead, multimodality needs to go raid the home base. At the birth of the computer, everyone wrote code in the terminal, just like writing in claude code; but in the end, the rise of Microsoft relied on GUI? Why do we need GUI? Because humans are visual animals. So in the end, 100% all operating systems will certainly become a streaming audio-video model possibly with a jev head to do api calls. This will rise together with AI glasses, eliminating Apple and iPhone, and OAI and Anthropic.
Secondly, the final model in the physical world cannot possibly be an llm model. Instead, the rapid improvement of the world-building capability of GPT and Fable only has benefits for world models, no harm. Because a world model is not necessarily a “big” model. In the future there will be many small models, different models; before, for each small model, reaching the threshold where intelligence is usable was too hard, the data was not enough, the results were not good. But in fact Qwen3-0.6B is the most widely used model. In the future, there will certainly be many on-the-edge small models, personalized models, but with enough intelligence, but without too much intelligence, in every person's life. If you say a 10T GPT, with 8T inside used for the representation of language, 2T for the representation of multimodality, maybe both can be done pretty well. But if you only have 5B parameters, you have no need to stuff the representation of language into it, and you also cannot stuff the representation of language in.
So exactly the opposite, world models, the multimodal route, need to go an anti-llm, anti-scaling law route; these are the two biggest consensuses of AI now. However, the future is shaped by people.
We need to demonstrate 1) intelligence density, with the same small parameter count, by continuously optimizing data, optimizing representation, our small models can surpass the so-called The Model in the domains people need, yet every household does not need a terminator but needs an R2-D2 - 2) the diversity of artificial minds, just like the diversity of biological minds, for example the earliest ElevenLabs was STT, TTS models, but now there are very good STS models, crossing over text, under the enabling of GPT/Fable, all kinds of vertical good models, instead, all have the possibility of being born 3) latency and local models are the most essential needs, the recent jev also shows everyone's enthusiasm for small models, fast models, specific models.
In essence, this is against the so-called The Model, Model 101; in substance it is anti-llm, anti-scaling, both non-consensus, but in fact the direction where the most can be done.