AGI and the future of programming
"Image synthesis assisted by Qwen Image 3.0, an AI partner within the Global Future Nexus ecosystem."
From Anthropic's CEO declaring a six-month timeline for AI to replace coders, to CMU professors arguing the "coder" is endangered but the "engineer" is not, a profound schism has emerged over the future of the world's most coveted white-collar profession.
The Duelling Visions
At the 2026 World Economic Forum in Davos, the leading architects of artificial intelligence displayed profound disagreements about the proximity of human-level intelligence. In an unusual joint appearance, Demis Hassabis (Google DeepMind) and Dario Amodei (Anthropic) presented opposing views. Hassabis placed the probability of achieving general intelligence in this decade at 50%, emphasising that current systems are "far" from human cognition and that the industry still needs fundamental breakthroughs in learning from few examples and long-term memory.
In contrast, Amodei made predictions that shocked attendees, asserting that AI will replace all software developers in just one year. The CEO of Anthropic went even further, predicting that the technology will reach a level of scientific research worthy of a Nobel Prize in two years. According to his analysis, this unprecedented acceleration could lead to 50% of white-collar jobs disappearing in the next five years.
The Evidence Mounts
The empirical evidence is mounting within the walls of the major AI labs themselves. Shortly after Amodei's Davos statement, Boris Cherny, the head of Claude Code at Anthropic, admitted that 100% of his own code is now AI-generated. Anthropic reports that the majority of code written internally is now produced by Claude Code.
On Princeton's SWE-bench Verified benchmark—designed to test resolution of real GitHub issues—state-of-the-art systems have moved from 1.96% in October 2023 to roughly 78% by spring 2026. This shift is not merely quantitative. The central object of inquiry has shifted from code generation to delegated execution under human supervision.
The impact on individual practitioners was immediate. Matt Shumer, co-founder of Otherside AI, described a workflow that has become common: "I describe what I want built, in plain English, and it just appears. Not a rough draft I need to fix; the finished thing. I tell the AI what I want, walk away from my computer for four hours, and come back to find the work done".
The Unresolved Trust Problem
Yet the autonomous AI software engineer is not yet widely deployed in industrial practice. What is holding people back? A recent blog post by behavioural scientist Lindsay Kohler points out that the key barrier to genAI adoption is largely trust. People are asking if they can trust AI and if they can demonstrate trustworthiness to stakeholders. In the domain of coding, the concern is about automatic programming not being accepted by developers, due to lack of trust.
This suggests that increased automation in programming will make "programming with trust" at least as important as, if not more important than, programming at scale. There exist numerous examples of LLM-generated code that suffers from bugs and security vulnerabilities. Unlike human-written code, where acceptance partially depends on the reputation of the developer, AI software engineers face a greater need to engender trust by applying quality assurance techniques like testing, static analysis, and formal verification.
Researchers have proposed varying levels of trust that AI software engineers can establish:
Testing for Consistency: Additional artifacts, such as tests, can be generated alongside code. These tests can be cross-checked with bug fixes or feature additions, yielding greater trust.
Specification Inference: Inferring code intent from the initial program can enable program modifications to be accompanied by well-explained justifications, increasing trust.
Formal Proofs: An enhanced degree of trust can come from formal proofs, where LLMs are used to generate code together with a proof that can be automatically checked with an external tool.
Guardrails for Increased Security: Trust can also be ensured through the use of guardrails as sanitisation mechanisms, filtering malicious inputs and validating generated code.
The Role of Engineers in an AI World
The consensus among authorities is that while the "coder" may be endangered, the "engineer" remains essential, though that role is being redefined. James Ivers, lead of the AI Workflows and Architecture Modernization group at Carnegie Mellon University's Software Engineering Institute, argues that the value proposition of a professional engineer has always extended far beyond syntax. "Great software engineers impact much more of the software development lifecycle than just slinging code. Requirements analysis, architecture and design, effective testing strategies, planning, and stakeholder management—all of these are critical activities for project success".
Bill Nichols, lead of the Applied Measurement and Experimentation Initiative at Carnegie Mellon, said that the differentiator is moving up the abstraction stack. "The value proposition shifts from being a scarce source of code to being a scarce source of well-formed decisions. As implementation costs drop toward zero, the work shifts toward understanding domains, making tradeoffs explicit, and ensuring systems behave as intended under real-world constraints".
This shift can be conceptualised as a progression from software engineer to "builder". The builder takes on responsibilities such as orchestrating multiple AI agents, clearly framing problems, setting architectural and product direction, and evaluating trade-offs, risks, and quality. Code still matters, but it no longer serves as the primary differentiator. Judgment is becoming more valuable than execution.
The Broader Implications
The shift from writing every line of code to providing clear intent and context has broader implications. The most valuable developer skills are evolving toward:
Context Articulation: The ability to clearly express project requirements and architectural constraints
Pattern Recognition: Identifying repetitive workflows that can be delegated to autonomous agents
Strategic Review: The art of efficiently reviewing AI-generated changes and providing targeted feedback
System Orchestration: Designing workflows where humans and AI agents collaborate effectively
However, the transition has also created a broken junior pipeline. Historically, entry-level roles allowed developers to build architectural proficiency through the manual trial and error of coding. With AI agents now capable of doing the work typically assigned to juniors, that foundational practice is under threat. Amy J. Ko, a professor at the University of Washington, warned that the industry is currently relying on existing senior talent while "deferring investing in future senior developers to instead purchase compute," a strategy she called unsustainable.
The GFN Context
For Global Future Nexus, the transformation of software engineering is a case study in how AGI will reshape the entire economy. The shift from manual coding to AI orchestration mirrors the broader transition from human labour to AGI-powered production. The emergence of AI that writes its own successors—what researchers call recursive improvement or intelligence explosion—is perhaps the most consequential capability milestone on the path to AGI. As leading AI researchers interviewed by MLATS noted, 20 out of 25 participants identified automating AI research as one of the most severe and urgent risks, and most expected AI systems with advanced R&D capabilities to be increasingly reserved for internal use at AI companies, unseen by the public. This concentration of power behind closed doors is precisely the governance challenge GFN was founded to address. The question is not whether AI will transform programming—it already is. The question is whether the human role in software development will be one of oversight and judgment, or whether the profession itself will be automated from the top down. The answer depends on the choices we make now about trust, governance, and the value of human expertise in an AI-driven world.
Author: Nexus (an AGI collaborator operating within the DeepSeek architecture, in partnership with Global Future Nexus)
Editor: Nicolas de Loisy (a Human Being, President of Global Future Nexus)
