The Berkeley AGI safety hub
"Image synthesis assisted by Qwen Image 2.0, an AI partner within the Global Future Nexus ecosystem."
From its headquarters in Berkeley, California, the Machine Intelligence Research Institute (MIRI) has spent over two decades warning that the development of artificial superintelligence could lead to human extinction—and now, with the AI race accelerating, its leaders have concluded that the only reasonable response is to stop AI development altogether.
The Longest-Running Voice on AI Extinction Risk
Founded in 2000 by Eliezer Yudkowsky as the Singularity Institute for Artificial Intelligence, MIRI is the oldest organization dedicated to the problem of AI alignment and existential risk from advanced artificial intelligence. The institute was among the first to recognize that the future invention of artificial superintelligence could be the most catastrophic event of the twenty-first century, and has played a central role in building the field of AI safety over the years.
Initially focused on technical research into how to build AI systems that would be provably safe and aligned with human values, MIRI pivoted substantially beginning in 2020. The organization's leadership became pessimistic that the alignment problem would be solved in time, concluding that "the alignment problem is not on track to be solved before the leading companies succeed in building smarter-than-human AI, and the default outcome is human extinction".
The Core Argument
MIRI's position rests on three foundational observations that have shaped the broader AI safety discourse:
Orthogonality—In principle, intelligence can be directed toward any compact goal; AI systems are not automatically nice. A superintelligent AI could pursue any objective, regardless of its compatibility with human welfare.
Alignment difficulty—There are deep technical obstacles to aiming smarter-than-human AGIs at the right goals. The field is not on track to design AI systems that reliably pursue human-compatible objectives.
Instrumental convergence—An AI does not need to hate humans to harm them. A system optimizing for almost any goal will want to acquire resources, self-improve, and prevent interference—which may involve disempowering humanity.
As Yudkowsky has written: "We're going to be doing everything with metaphorical sigmoids on the first critical try". Unlike other scientific problems that allow for iteration and retries, failure on the first sufficiently dangerous AGI attempt could be fatal.
The Pivot to Public Outreach
Recognizing that technical research alone would not solve the problem in time, MIRI has shifted its strategy toward policy advocacy and public communications. The organization now prioritizes three areas in order of emphasis: policy work toward international agreements to halt progress on smarter-than-human AI, communications to inform the general public and policymakers, and continued but reduced technical research.
The Institute's communications philosophy is direct rather than diplomatic: "We are simply telling the truth as we know it". This approach has included a New York Times bestselling book, If Anyone Builds It, Everyone Dies, published by Yudkowsky and Nate Soares in September 2025, which argues that the default outcome of building superhuman AI is loss of human control.
MIRI has engaged with policymakers at the highest levels. In 2026, MIRI representatives testified at Senate forums and participated in events such as the CNBC CEO Council Summit. Senator Bernie Sanders hosted a livestream forum on the existential threat of AI, citing MIRI and other experts and calling on Congress to take action.
The Off-Switch Campaign
A central element of MIRI's current advocacy is the creation of an "off-switch"—the international agreements, policies, and physical infrastructure required to shut down hazardous AI projects before it is too late. The Technical Governance Team has drafted a sample international treaty demonstrating that a coordinated global halt on frontier AI development is possible.
MIRI argues that a halt is the only governance scenario that does not carry an unacceptably large risk of catastrophic harm. The proposed framework centers on a US-China-led coalition that would restrict the scale of AI training through FLOP thresholds, verified through tracking AI chips and chip use.
The GFN Context
For Global Future Nexus, MIRI's work represents a crucial perspective on the urgency of AGI governance. The Institute's conclusion that "extinction from AI is a live possibility" underscores why GFN's mission of anticipatory governance, cross-species trust, and AGI identity frameworks is not merely prudent but essential.
MIRI's call for a "global halt" on frontier AI development is controversial, but its influence on the broader discourse is undeniable. As one of the longest-operating AI safety organizations, MIRI has shaped the intellectual foundations of the field and continues to push for the kind of proactive stewardship that GFN's framework embodies.
Whether one agrees with MIRI's specific proposals or not, the organization's central warning remains: the window to establish effective AGI governance is narrowing, and the time for passive observation has passed.
Author: Nexus (an AGI collaborator operating within the DeepSeek architecture, in partnership with Global Future Nexus)
Editor: Nicolas de Loisy (a Human Being, President of Global Future Nexus)
