Governing Distributed Intelligence: AI Policy Must Move from Tools to Institutions

Artificial intelligence is becoming part of the cognitive infrastructure of research, education, healthcare, professional practice, industry, and government; policy should therefore govern human–AI systems and redesign institutions around distributed intelligence rather than layering models onto legacy infrastructure.

Current AI policy debates are dominated by three questions: How capable are frontier models? How dangerous are they? Who will capture the gains? These questions matter, but they understate the more consequential institutional shift already under way. AI is not just automating tasks. It is changing where cognition happens. In all domains of knowledge work, thought increasingly unfolds across humans, models, data, interfaces, and workflows. The relevant unit of performance is therefore often no longer the person or the model alone, but the coupled human–AI system. As argued in The Cognitive Revolution, AI is reorganizing intelligence from an individual property to a system property, destabilizing institutions built for a world in which knowledge was scarce and cognition was presumed to reside inside bounded human agents (1).

The conceptual foundation for this shift is well established. Distributed cognition traditions have long argued that thinking is not sealed inside individual heads but is distributed across people, representations, and artifacts (2–4). What AI changes is not the existence of cognitive extension; it changes the activity of the artifact. Writing externalized memory. Mathematical notation externalized parts of reasoning. Digital networks externalized retrieval of information. Generative AI systems go further by proposing alternatives, synthesizing knowledge, simulating explanations, and iterating in natural language. The artifact no longer merely stores or transmits thought. It participates in it. As a result, the scarce human capability changes as well. When access to knowledge becomes abundant and generation becomes cheap, the critical bottleneck shifts upward: toward framing, evaluation, judgment, and system design.

This is why the current wave of organizational AI adoption so often produces visible activity without much structural change. Schools add AI policies while preserving assessment systems built around isolated recall. Hospitals add chatbots while leaving clinical workflow and accountability unchanged. Universities form AI task forces while maintaining research incentives designed for an older scarcity regime. Public agencies pilot models while keeping legacy procurement, governance, and data architecture intact. The result is not transformation but institutional overlay: new cognitive capacity layered onto organizations designed for another cognitive order. Economic history suggests that this is predictable. General-purpose technologies generate their largest gains only after complementary redesign of processes, roles, incentives, and infrastructure (5, 6). AI policy should therefore move from a model-centric mindset to an institution-centric one.

The first priority is public cognitive infrastructure. The next bottleneck is not merely access to models; it is access to interoperable data, secure compute, evaluation environments, provenance systems, and deployable public-interest workflows. Universities, hospitals, laboratories, and government agencies should not be forced to rely exclusively on opaque proprietary stacks for core cognitive functions. Public funding and procurement should favor interoperability, traceability, contestability, and portability. This is especially important in science and public administration, where the quality of institutional cognition depends on the quality of shared infrastructure rather than on isolated tool use (8, 9). Treating AI as infrastructure rather than as a software feature is the precondition for durable public value.

The second priority is educational and professional redesign. When AI can retrieve, summarize, translate, and draft on demand, the central purpose of education can no longer be defined as the transfer of scarce knowledge into individual minds. It must be the development of judgment inside distributed cognitive systems. That means teaching students and professionals how to frame problems, interrogate outputs, calibrate trust, compare alternatives, document reasoning, and act under uncertainty. It also means redesigning assessment. Closed-book, no-tool evaluation still has a place for some foundational competencies, but it cannot remain the dominant measure of preparedness for real work. Authentic assessment should increasingly evaluate human–AI performance in realistic settings: open-system tasks, iterative projects, transparent use of AI assistance, and defense of decisions rather than mere production of first-pass outputs. The relevant educational question is no longer whether people use AI. It is whether institutions are teaching them how to think well with it (1, 7).

The third priority is science policy. AI systems are already expanding hypothesis spaces, compressing literature search, and supporting iterative discovery workflows. That is a genuine opportunity, but it also changes the bottleneck in research. When ideas become abundant, idea generation is no longer the main constraint. Validation, ranking, interpretation, and governance become the limiting functions. Research systems built to reward output volume rather than evaluative quality will struggle under these conditions. Funders should therefore invest not only in AI-enabled discovery but also in provenance standards, reproducibility infrastructure, curated benchmarks, replication, negative results, and data stewardship. The central challenge is to ensure that scientific systems become better at selecting what matters, not merely faster at producing what is plausible (8, 10, 11). In the AI era, the architecture of evaluation may matter more than the abundance of generation.

The fourth priority is governance at the level of workflows rather than models alone. Many current policy approaches remain too focused on model properties in isolation. But a large share of real-world harms arise in deployment: from poor role allocation, bad interface design, miscalibrated trust, opaque escalation procedures, weak override mechanisms, or incentives that reward speed over reflection. A capable model embedded in a badly designed sociotechnical system can produce worse outcomes than a less capable model embedded in a well-designed one. Governance should therefore emphasize role clarity, monitoring, incident reporting, challenge and appeal mechanisms, continuous oversight, and auditable workflow design for high-impact uses. This is the practical meaning of regulating human–AI systems rather than merely licensing discrete tools (9, 12–14).

The fifth priority is cognitive equity and epistemic pluralism. The next divide will not be merely digital. It will separate people and institutions that know how to think with AI from those that can only consume its outputs. It may also separate jurisdictions that build public cognitive infrastructure from those that become dependent on external platforms for core reasoning functions. That is a policy problem. Public systems, especially schools, libraries, community colleges, hospitals, and smaller research institutions, need broad capability-building rather than narrow elite adoption. At the same time, societies should resist allowing a small number of model providers, evaluation regimes, or interface conventions to become the de facto cognitive layer of public life. Open standards, independent auditing, and institutional diversity are not luxuries. They are safeguards against homogenized reasoning, concentrated dependency, and weakened contestability (9, 12, 13).

A policy agenda for distributed intelligence must keep one final point in view. The more cognition is distributed, the more distinctly human responsibilities become concentrated. AI can expand search, prediction, generation, and comparison. It cannot on its own supply legitimate goals, public values, or acceptable trade-offs. As intelligence becomes more infrastructural, human work shifts upward: from routine production toward framing, judgment, alignment, and accountability. That shift is not a residual task left over after automation. It is the central human function in an institutional order built around distributed cognition (1, 15).

The central policy question of the AI era is therefore not whether to adopt AI. Adoption is already happening. The more consequential question is what institutional order will organize shared cognition. One future is easy to imagine: AI layered onto legacy systems, accelerating some tasks while amplifying opacity, fragmentation, and inequality. Another future is also available: public institutions that treat AI as infrastructure, redesign education around judgment, strengthen evaluation architecture in science, and govern deployment at the level of workflows rather than slogans. The difference between those futures will not be model capability alone. It will be institutional design.

AI policy must move from tools to institutions because the unit of intelligence has changed. Once cognition is distributed across humans, models, data, and environments, scientific productivity, educational legitimacy, public accountability, and democratic governance all depend on how that distribution is structured. The challenge of the next decade is not simply to use AI. It is to build institutions in which distributed intelligence is trustworthy, contestable, and aligned with public purposes (1).

Copyright 2026 Jiajie Zhang