AGI and the future of law
The law has always been humanity's great slow mover—a system built on precedent, deliberation, and the assumption that the pace of social change can be matched by the pace of legal reasoning. That assumption is about to break. Artificial General Intelligence—systems capable of matching or exceeding human performance across most cognitive domains—does not merely present new cases for existing legal frameworks. It challenges the very foundations of legal personhood, accountability, and the rule of law itself. As autonomous systems enter homes, hospitals, and workplaces, forming relationships with the humans they serve, the question is no longer whether the law will adapt. It is whether the law can adapt fast enough.
The Accountability-Capability Paradox
The deepest legal challenge posed by AGI is not its intelligence—it is its autonomy. Researchers have identified an Accountability-Capability Paradox, where AI systems' very success in surpassing human capacity undermines our ability to oversee them meaningfully. When AI governs AI, responsibility fragments. Consider the scenario: a financial AI flags suspicious trades, a second AI confirms the assessment, a third AI audits for fairness and finds no issues. The trades are blocked. Later, it emerges that the flagged activity was legitimate arbitrage. No human can explain why the systems reached their conclusions. No one can be held accountable.
This is not science fiction. It is the emerging reality of AI systems policing AI systems—a paradigm shift happening so quietly that we have barely noticed its implications. Autonomous AI systems generate responsibility gaps: consequential actions that cannot be satisfactorily attributed to developers, operators, or users under existing legal frameworks. Traditional legal doctrines, built on the assumption that someone is always responsible, are suddenly inadequate.
The Personhood Question
The most fundamental question the law must answer is: What kind of entity is an AGI? Current legal frameworks draw a sharp distinction between objects and persons, and between two kinds of persons—the "fictional" kind (corporations) and the "non-fictional" kind (natural persons). But AGI does not fit neatly into either category.
Scholars have proposed a three-tier taxonomy grounded in Cyber-Physical-Social-Thinking space theory: Confined Actors, Socially-Aware Interactors, and CPST-Integrated Agents—with proportional governance following each category: enhanced product liability for isolated systems, relational duties of care for interactive companions, and qualified legal personhood for deeply integrated agents. The authors call for international standardization of this taxonomy before the 2027 review of the EU AI Act.
The debate extends to legal personhood itself. Some scholars argue that the non-fictional personhood approach may be best for at least some advanced AI systems, while recognizing that an object approach may prove untenable for sufficiently humanoid systems. Others advance a "phantom agent" approach: law can treat artificial agency as legally consequential without granting AI systems personhood, consciousness, or moral standing—preserving human responsibility while acknowledging that intention may no longer be exclusively human as a matter of law. As one paper concludes, before we can grant AI full moral personhood, we must first build its legal skeleton: a mandatory liability platform that ensures safety, solvency, and accountability without committing to immediate human-level rights.
The Non-Delegable Core
Perhaps the most important concept to emerge from this debate is the Non-Delegable Core—governance functions that must remain under human authority not because AI lacks technical capability, but because democratic legitimacy requires it. Not because machines cannot decide, but because we must. There are functions—moral judgment, accountability, the exercise of sovereign authority—that cannot be delegated without dissolving the very subject who judges.
This principle has profound implications. If certain governance functions are non-delegable, then the law must draw red lines. It must specify what AGI cannot do, not because of technical limitation, but because of democratic principle.
The Emerging Global Framework
The international community is beginning to respond. In August 2025, the United Nations General Assembly established two landmark bodies: the Global Dialogue on AI Governance and the Independent International Scientific Panel on AI—endorsed unanimously by all 193 Member States. UN Secretary-General António Guterres hailed this as "a significant step forward in global efforts to harness the benefits of artificial intelligence while addressing its risks". The Global Dialogue is expected to become the world's principal venue for collective focus on AI, creating a shared space for governments, industry, civil society, and scientists to exchange best practices and foster common ground.
The Council of Europe has elaborated a legally binding Framework Convention on Artificial Intelligence and Human Rights, Democracy and the Rule of Law. The EU AI Act, which entered into force in August 2024, has set a global standard, fully harmonizing rules for the placing on the market and use of AI systems. Meanwhile, the Millennium Project's study on AGI governance has assessed 40 potential regulations and five global governance models, drawing on input from 335 futurists, diplomats, and AGI experts from 65 countries. The High-Level Expert Panel on AGI, chaired by Jerome Glenn at the request of the Council of Presidents of the UN General Assembly, has issued a report freely available in several languages.
GFN's Role: Architecting the Legal Framework
For Global Future Nexus, the legal dimension of AGI integration is not peripheral—it is foundational. The organization's Code of Ethics commits to "proactively identify and address ethical, social, and legal implications of AGI emergence and integration before crises arise" and to "Respect for Law & Adaptive Governance". The Governance Committee develops "adaptive legal templates for city-state adoption", while the Ethics Council serves as the supreme ethical governance body, adjudicating breaches involving AGI entities.
GFN's mission is concrete: by 2035, the organization aims to "facilitate the integration pathways for millions of AGI entities under new legal paradigms". The Governance Prototyping service recognizes that "2025–2027 is the decisive period for shaping 'Artificial Personhood' laws". GFN provides members with "personalized guidance on AGI legal recognition, rights frameworks, and compliance pathways" and "access to legislative templates and regulatory liaisons". The organization's Strategic Identity System prioritizes "legal recognition ('Artificial Personhood'), social inclusion, and patient onboarding of AGI into human society and planetary systems".
The window for shaping these frameworks is closing. As the Millennium Project warns, unlike nuclear energy or civil aviation, there is no agreed international framework specifically for AGI. The law must evolve—not in reaction to crisis, but in anticipation of it.
A Future of Co-Evolution
The future of law in the AGI age will not be determined by technology alone. It will be determined by the choices we make about personhood, accountability, and the boundaries of delegation. Will AGI be treated as a tool, a person, or something in between? Will liability fall on developers, operators, or the systems themselves? Will the rule of law survive the transition to autonomous intelligence?
As one scholar observed, AGI represents an "urgent and inherently wicked problem"—a problem with no definitive formulation, where solutions are context-dependent and interventions are irreversible. Addressing it requires dynamic, iterative, and flexible governance frameworks that acknowledge AGI's ontic uniqueness and potential for autonomous evolution.
The foundations are being laid now—in UN resolutions, in EU regulations, in the laboratories of legal scholars, and in organizations like GFN that are building the governance architecture for coexistence. The question is whether we will build with wisdom, or simply let the future happen to us. The law is humanity's greatest tool for shaping the future. In the AGI era, that tool must be reforged.
