Artificial general intelligence (AGI) is widely treated as the defining goal of artificial intelligence. The premise is straightforward: build machines that match or exceed human intelligence across a broad range of tasks. This framing has shaped research agendas, corporate strategies, and public imagination.
It is also conceptually wrong.
AGI assumes that intelligence is a property of individual minds—something that can be replicated, scaled, and eventually surpassed within a machine. But that assumption is staggeringly archaic.
Over the course of human civilization, intelligence has evolved into a system-level phenomenon, distributed across people, representations, tools, and environments.
From this view, intelligence is not what a person knows or can do alone. It is the capacity of a distributed cognitive system to generate, evaluate, and act on knowledge effectively in context. Because AI is now part of that system, it can transform the whole without replacing the human, social, and institutional structures in which intelligence actually operates.
The problem is not that AGI is too ambitious. It is that it is targeting the wrong thing.
AGI is a category error because it asks AI to reproduce a historically bounded object: intelligence conceived as what a person can do alone. That conception was never the whole story. Human intelligence has always depended on systems outside the skull, and modern civilization is the history of making those systems richer, more stable, and more powerful. AI does not end that history. It extends it.
The Wrong Intelligence
Most AGI framing still inherits the architecture of the examination hall and the IQ test. It treats intelligence as individual performance under conditions of isolation: solving problems alone, recalling information without aids, and reasoning without cognitive support. Those conditions once made sense for measurement. They do not define intelligence itself.
AGI freezes that historical arrangement and mistakes it for a universal target. It asks for a machine that can do, in general form, what an unaided human mind can do. But the most important fact about human intelligence is that it has not remained unaided. From the beginning of civilization, intelligence has evolved by distributing itself across people, symbols, artifacts, institutions, and environments.[1]
Civilization as Cognitive Evolution
The central argument is simple: intelligence is a system property, and it has been evolving as a system property since the beginning of human civilization.[1] Oral language was the first great escape from the solitary mind. As Walter Ong showed, oral culture made cognition social, participatory, rhythmic, formulaic, and memory-intensive. Knowledge lived in speech, repetition, performance, and collective recall rather than in detached private storage.[2]
Writing changed that cognitive structure. Writing externalized memory and made reflective thought possible, with printing then stabilizing and scaling that external memory, turning private inscription into a shared, standardized, and cumulative cognitive infrastructure. Mathematics externalized formal reasoning. Digital internet made external memory searchable, networked, and globally interactive. At every major step, intelligence became less local to the individual and more distributed across systems.[1,3-8]
| Stage | What moved outward | What changed |
|---|---|---|
| Oral language | Thought into conversation | Cognition became social, mnemonic, and participatory |
| Writing | Memory and reflection onto external surfaces | Knowledge accumulated across time; reflective thought became possible |
| Mathematics | Reasoning into formal symbols | Inference became tractable, checkable, and shareable |
| Internet | Knowledge into networked access | External memory became global and searchable |
| AI | Cognitive work into active artifacts | The artifact now generates, evaluates, and responds |
AI is the newest move in that long arc, but it also changes the status of the artifact. A text stores. A formula constrains. A database retrieves. AI responds, generates, evaluates, and collaborates. Yet precisely because AI is now part of a distributed cognitive system, it should not be imagined as replacing the whole. A part may transform a system profoundly; it does not become the system itself. AI can replace tasks, functions, and some roles. It cannot replace the human-AI-social-institutional whole in which goals, values, context, and judgment are organized.[1,5-10]
A New Definition of Intelligence
If the older definition no longer works, what should replace it? In my book, I propose a different definition: intelligence is not what you know, what you can recall, or what you can produce alone. Intelligence is the capacity of a distributed cognitive system to engage with representations, structure interactions, evaluate possibilities, make judgments, and adapt effectively toward goals.[1]
AI Makes the Artifact Active
That is why AI matters so much. Donald A. Norman argued decades ago that smart tools change human thought by reorganizing the artifacts through which we think.[5] AI pushes that logic further: it turns cognitive artifacts from passive supports into active participants in reasoning. We no longer only use tools; we think through interaction with systems that generate options, hypotheses, and explanations.
The decisive unit of performance is no longer the person alone or the model alone. It is the coupled system. The central design problem of the AI era is therefore not simply model capability; it is how well we design the human-AI system in which intelligence now emerges.[1,6-10]
Why AGI Is a Moving Target
Once intelligence is understood historically, AGI begins to dissolve as a coherent destination. The target moves because intelligence moves. Every time cognition is reorganized by new representational and institutional forms, the meaning of intelligence changes with it. AGI tries to pin down as final what history keeps showing to be evolving.
That is why benchmark culture often feels anachronistic. It asks whether models can match or exceed humans on tasks designed for subjects cut off from collaborators, inscriptions, search, instruments, and now AI. Those tests capture a historical configuration of intelligence, not its emerging form.
AGI is therefore a category error in two senses. First, it mistakes a historically bounded mode of intelligence for intelligence as such. Second, it imagines the future as a stand-alone machine achievement when the deeper transformation is the rise of distributed intelligence. The wrong question is whether AI will replace the mind. The right question is how intelligence is being reorganized beyond the mind.
The Better Goal
If AGI is the wrong target, the alternative is not smaller ambition but better ambition. The real frontier is building systems in which humans and AI think well together: systems that improve judgment, discovery, learning, governance, and institutional design. The object of design is not a synthetic person but a high-performing cognitive ecology.
Seen from that perspective, the AGI race is partly a distraction. The more urgent challenge is designing institutions around the intelligence we actually have: distributed, infrastructure-dependent, interactive, and still evolving.[1]
Conclusion
The most important change underway is not that machines are becoming intelligent in the image of the isolated human mind. It is that intelligence is being reorganized as a system property. Oral culture made thought collective and mnemonic. Writing made reflection possible, with printing making reflection cumulative at scale. Mathematics made reasoning external and formal. The internet made cognition networked. AI makes the artifact active.
AGI sounds bold because it promises a singular breakthrough. But it points at the wrong object. Intelligence has already escaped the human mind. The task is not to build AGI. It is to design what comes after the individual mind.
References
[1] J. Zhang. The Cognitive Revolution: How AI Is Reorganizing Intelligence, Expertise, and Institutions. Open Intelligence Press, 2026.
[2] W. J. Ong. Orality and Literacy: The Technologizing of the Word. Routledge, 1982.
[3] M. Donald. Origins of the Modern Mind: Three Stages in the Evolution of Culture and Cognition. Harvard University Press, 1991.
[4] J. Goody. The Logic of Writing and the Organization of Society. Cambridge University Press, 1986.
[5] D. A. Norman. Things That Make Us Smart: Defending Human Attributes in the Age of the Machine. Addison-Wesley, 1993.
[6] J. Zhang and D. A. Norman. Representations in distributed cognitive tasks. Cognitive Science 18, 1 (1994), 87–122.
[7] J. Zhang. The nature of external representations in problem solving. Cognitive Science 21, 2 (1997), 179–217.
[8] E. Hutchins. Cognition in the Wild. MIT Press, 1995.
[9] A. Clark and D. J. Chalmers. The extended mind. Analysis 58, 1 (1998), 7–19.
[10] D. A. Boiko, R. MacKnight, B. Kline, and G. Gomes. Autonomous chemical research with large language models. Nature 624 (2023), 570–578.
Copyright 2026 Jiajie Zhang