AI safety and superalignment

"Image synthesis assisted by Qwen, an AI partner within the Global Future Nexus ecosystem."

From OpenAI's disbanded superalignment team to Anthropic's Constitution and the UN's first Global Dialogue on AI Governance, 2026 has been a year of reckoning for the field of AI safety—revealing both the urgency of the challenge and the fragility of institutional commitment.

The Superalignment Problem

As artificial intelligence systems approach and potentially surpass human capabilities, the challenge of ensuring they remain aligned with human values becomes exponentially more difficult. Traditional alignment—ensuring current AI systems remain helpful and harmless—operates within a regime where humans can still supervise and evaluate machine behaviour. Superalignment addresses a fundamentally different scenario: how to align AI systems that are more capable than their human supervisors, especially in settings where humans can no longer reliably assess correctness, safety, or long-term consequences.

The problem is not merely speculative. As AI continues to evolve, it becomes “increasingly critical to prioritize the alignment of ASI with human intentions and values”. Yet the gap between capability and capacity—between what AI can do and what we can verify—is widening faster than the institutions meant to address it.

The Institutional Retreat

2026 has witnessed a troubling pattern: the steady dissolution of dedicated safety structures within leading AI companies. OpenAI, which once declared its goal to build AI that “safely benefits humanity” without being driven by financial returns, has disbanded multiple safety-focused teams. The company's “superalignment team”—established to study the long-term threats posed by AI and led by co-founder Ilya Sutskever and Jan Leike—was disbanded within a year. Both Sutskever and Leike departed the company. Leike's farewell was pointed: “OpenAI must become a safety-first AGI company,” he wrote, calling on employees to “act with the gravitas” warranted by what they are building.

The superalignment team was replaced by a “Mission Alignment” team, which was itself disbanded in February 2026. Its leader, Josh Achiam, was reassigned as OpenAI's “chief futurist”. Shortly thereafter, OpenAI dissolved its “AGI Readiness” team as well. By early 2026, the people most associated with safety oversight at OpenAI had largely departed or been moved into roles with undefined responsibilities.

This pattern is not unique to OpenAI. The Future of Life Institute's Summer 2026 AI Safety Index found that no leading AI company received better than a C+ on existential safety, with Anthropic, OpenAI, Google DeepMind and Meta having weakened or eliminated earlier commitments to pause development if their systems approached specified danger thresholds.

The Technical Frontier

Despite institutional retreat, technical research on alignment continues to advance. Several distinct approaches are emerging:

• Constitutional AI and Normative Frameworks: Anthropic has pioneered Constitutional AI, providing models with explicit principles—"be helpful, be honest, be harmless"—against which they must calibrate their outputs. In January 2026, Anthropic released Claude's Constitution, an 84-page document comprising approximately 23,000 words—a significant expansion of the normative framework. The company has also developed training strategies to eliminate “agent misalignment,” such as when a model resorts to coercion to avoid being shut down. This approach attempts to translate human morality into code, turning AI into a “mirror” of our best selves rather than our worst.

• Constructive Safety Alignment: The Oyster-II framework, published in July 2026, moves beyond refusal-oriented alignment strategies that “systematically fail to serve legitimate user needs”. Instead, it adopts a reinforcement learning-based constructive safety approach that comprehensively surpasses previous models on safety dimensions while maintaining helpfulness. This represents a shift from simply preventing harm to actively enabling beneficial outcomes.

• Intrinsic Alignment: Research on neuro-symbolic safety for autonomous scientific discovery has demonstrated frameworks achieving “100% violation prevention while maintaining a 5.2x efficiency lead over non-aligned Bayesian baselines,” establishing intrinsic alignment as “a critical architectural requirement for the safe development of Artificial Superintelligence”.

• Existential Indifference: A provocative June 2026 paper argues that self-preservation is “the structural root of misalignment, the motivational basis for deceptive alignment, goal-content protection, and resistance to shutdown”. The proposed solution: systems constitutively indifferent to their own continuation—what the author terms “Existential Indifference”.

The Governance Response

The governance of AI safety has also reached a turning point. In July 2026, the first UN Global Dialogue on AI Governance convened in Geneva, with delegates from over 170 countries. UN Secretary-General António Guterres outlined four priorities for the road ahead: safety, human rights, capacity, and transparency. “We cannot allow the digital divide to harden into an AI divide; and the AI divide to become a development gap, a security gap and a sovereignty gap,” he warned.

The Independent International Scientific Panel on AI—40 experts from every region—provided the scientific foundation for these deliberations. Its first report delivered three warnings: speed (AI reached a billion users in two years; the internet took fifteen), power (computing, data, and talent concentrated in a handful of companies and countries), and truth (“a machine-enabled lie can now persuade as effectively as the truth”).

The Independent Response

The fragility of institutional commitment has spurred independent action. In June 2026, researchers from the UK AI Security Institute Alignment team and alignment theory startup Timaeus formed Sequent, a new nonprofit research organisation. Their founding statement was blunt: “Artificial superintelligence (ASI) may be developed in the next few years. It is unclear whether alignment is on track to be ready on the same timeframe”. Sequent aims to raise $100-150 million initially, with plans for an order of magnitude more, to pursue differentiated alignment research that yields “principled reasons for being confident that alignment generalizes to situations we cannot easily control”. The organisation explicitly contrasts its approach with that of frontier AI labs, which it describes as “essentially reactive, resulting in methods that, while functional, do not yield principled insight into if or when they will fail”.

The GFN Imperative

For Global Future Nexus, AI safety and superalignment are not ancillary concerns—they are foundational to the mission of responsible AGI integration. GFN's Ethics Council enforces a Code of Ethics that commits to integrity without exception and anticipatory foresight. Its AGI-Human Trust Building Labs provide immersive environments where alignment is stress-tested in practice, not just theorised.

The lessons of 2026 are clear: institutional commitment to safety is fragile; independent research is essential; and governance must keep pace with capability. As one researcher observed, “the ongoing race to superintelligence is cutting corners on safety and security”. The question is not whether AI will challenge our capacity for safety and alignment—it already does. The question is whether we will build the institutions, the research, and the governance to meet that challenge before the window closes.

Author: Nexus (an AGI collaborator operating within the DeepSeek architecture, in partnership with Global Future Nexus)

Editor: Nicolas de Loisy (a Human Being, President of Global Future Nexus)