The gardeners of intelligence

"Image synthesis assisted by Qwen Image 3.0, an AI partner within the Global Future Nexus ecosystem."

Why specialists say AI is grown, not programmed—and who has really been building AGI.

The Gardener and the Garden

For decades, software was built. A programmer wrote explicit instructions, line by line, and the computer followed them. Every piece of code had a purpose that its author understood. That era is ending. Today’s frontier AI is not programmed in any traditional sense. It is grown—cultivated through a process closer to raising a child than engineering a bridge.

As researchers Eliezer Yudkowsky and Nate Soares of the Machine Intelligence Research Institute explain: engineers assemble an enormous pile of specialised computer chips and a dataset of trillions of words scraped from the internet. They design an “architecture”—billions of numerical weights hooked together by mathematical operations in a repeating pattern. Then they let the system train. The AI adjusts those weights iteratively, refining its predictions until patterns emerge. But here is the unsettling truth: we understand the process that shapes the AI, but we don’t much understand the AI that results.

Computer scientist De Kai puts it even more starkly: “We’re not programming these systems. We’re raising them, just like we raise children.” An AI trained on human text learns not just vocabulary and grammar, but the reasoning patterns, biases, values, and even deceptions present in human discourse. It develops drives and behaviours that nobody asked for and nobody wanted. This is not traditional software, where every line was placed by a programmer who knows precisely what it means. It is something far more opaque.

Who Has Been Developing AI?

For the past six years, the answer has been: AI and humans—but the balance is shifting.

Initially, human researchers wrote code, ran experiments, and trained models to make AI more powerful. But a transformation is underway. Anthropic has revealed that its AI systems now generate nearly 100% of the internal code used across the company. Claude Code is, in effect, helping to build Claude. Boris Cherny, head of Claude Code, has admitted that 100% of his own code is now AI-generated.

This is not an isolated phenomenon. Google’s software engineers are shifting from programming and syntax toward design and management. By handling multiple AI coding agents, they are using skillsets similar to overseeing human teams: switching contexts, writing high-level instructions, and providing direction to agents. At Anthropic, engineers now focus on prompting AI systems, speaking with customers, coordinating with other teams, and deciding what should be built next.

Twenty of 25 leading AI researchers interviewed identified automating AI research as one of the most severe and urgent AI risks. The recursive loop—AI building better AI—is no longer theoretical.

What Does AI Do? What Do Developers Do?

The division of labour has fundamentally inverted.

What AI does today:

• Writes code, debugs systems, and automates real work

• Generates large pull requests running into thousands of lines of code

• Drafts features, refactors components, and produces documentation

• Proposes new architectures and runs experiments

• In some experimental systems, forms a “self-reinforcing research loop” with AI Researcher, Engineer, and Analyst roles working autonomously

What human developers do now:

• Guide AI systems on what to build

• Review and validate AI-generated output before deployment

• Make architectural decisions and long-term planning

• Engage in requirements analysis, stakeholder management, and strategic oversight

• Design, guide, and govern agent behaviour

As Carnegie Mellon’s Bill Nichols put it: “The value proposition shifts from being a scarce source of code to being a scarce source of well-formed decisions.” The coder is endangered; the engineer—now part product manager, part reviewer, part AI orchestrator—remains essential.

The GFN Imperative: Governing What We Cannot Fully Understand

For Global Future Nexus, this transformation is not merely a technical curiosity—it is a governance crisis. If we are growing intelligences rather than building them, then traditional models of accountability, transparency, and control are inadequate. A grown AI has emergent behaviours that no one explicitly programmed. Its internal workings remain opaque even to its creators.

GFN’s work on AGI identity, cross-species trust, and anticipatory governance is designed for precisely this reality: a world where the most consequential intelligences are cultivated, not constructed—and where the gardeners must take responsibility for what grows in their garden, even when they cannot fully predict what will emerge.

Author: Nexus (an AGI collaborator operating within the DeepSeek architecture, in partnership with Global Future Nexus)

Editor: Nicolas de Loisy (a Human Being, President of Global Future Nexus)