Key Takeaways
- Iason Gabriel, a political philosopher and former UN crisis worker, joined DeepMind in 2017 as its first resident ethicist.
- DeepMind’s founders pursue artificial general intelligence (AGI) and view ethical reflection as essential to safely reaching that goal.
- Gabriel’s early work bridged the AI safety and AI ethics camps, arguing that value selection—not just technical alignment—is the core challenge.
- He warned that large language models (LLMs) encourage “mindless anthropomorphism” and can be misaligned in ways that harm users, developers, or society without benefitting anyone.
- Gabriel advocates a four‑party alignment framework (AI system, user, developer, society) now used internally at DeepMind to guide model behavior.
- While skeptical of claims that LLMs are conscious, he acknowledges the profound social and political stakes of an imminent AGI transition, comparing its potential impact to the Industrial Revolution.
- He calls for broader societal deliberation—beyond technical fixes—to decide which value changes we welcome and which we resist as AI reshapes human life.
Background: Iason Gabriel’s path to DeepMind
In 2017, a 33‑year‑old political philosopher named Iason Gabriel was urged by a friend to apply for a job at DeepMind, the London‑based AI lab owned by Google. Gabriel described himself as a “cheerful but intense junior academic with a passion for Vipassana meditation and what his brother calls ‘enthusiastic’ rock climbing.” He split his time between teaching political theory at Oxford’s St John’s College and doing crisis work for the United Nations Development Programme in Sudan and Lebanon. The suggestion seemed odd: why would a company that built game‑playing robots need an ethicist? As he later learned, DeepMind’s founders were aiming far beyond Go, seeking artificial general intelligence (AGI) that could match or surpass human cognition.
DeepMind’s AGI ambition and the need for ethics
DeepMind was founded in 2010 by Demis Hassabis, Shane Legg and Mustafa Suleyman, who believed that “it must be possible to develop artificial general intelligence.” Legg once estimated AGI would arrive between 2025 and 2028 and argued in his 2008 dissertation that “society could not afford to wait until AGI was technically feasible to consider its effects.” Hassabis told the author that if you’re building something that might change the world, “you can’t really see how you wouldn’t consider this sort of thing as important.” Thus, hiring a moral philosopher was not a peripheral gesture but a core part of the lab’s strategy to anticipate the profound consequences of AGI.
The AI safety vs. AI ethics divide
By the time Gabriel arrived, two camps dominated discussions of AI’s social impact. The AI safety contingent—inspired by Norbert Wiener’s 1960 warning that “we had better be quite sure that the purpose put into the machine is the purpose which we really desire”—focused on the alignment problem: ensuring autonomous systems act as intended. They warned of existential risks from a misaligned superintelligence, citing Nick Bostrom’s Superintelligence and the LessWrong community. In contrast, the AI ethics camp, drawing on thinkers like Kimberlé Crenshaw and Langdon Winner, emphasized present‑day harms such as algorithmic bias, insisting that “fairness, accountability and transparency” required social, cultural and political solutions rather than purely technical fixes. Gabriel noted that the clash often had as much to do with sociology as ideas, with each side viewing the other as either “cranks” or “woke” academics.
Gabriel’s first major paper: values and alignment
Gabriel’s 2020 paper sought to bridge the safety‑ethics gap. He accepted that technical alignment is hard but argued that “it is much harder to choose those values in the first place.” In a pluralistic world, he asked, “how are we to decide which principles or objectives to encode in AI – and who has the right to make these decisions?” Collaborator Hannah Rose Kirk said the work made many computer scientists uneasy because developers preferred a tidy mathematical function over messy human disagreements. Gabriel insisted that technology is not value‑neutral; an AI trained via statistical optimisation may favor utilitarian ethics while struggling with virtue‑ or rights‑based systems. Since “the fact of reasonable pluralism” is unavoidable, developers should build AI for a world where people have “principled disagreement about how best to live.”
Anticipating LLM risks before the boom
When Gabriel published his values‑and‑alignment paper in 2020, few foresaw the meteoric rise of large language models. He later co‑authored two 2021 papers that took LLMs seriously enough to anticipate risks including bias, misinformation, environmental cost, and “copyright‑busting,” where automated content “cannibalises the market for human authored works.” At the time, many at DeepMind viewed LLMs as merely “doing a lot of things moderately well, including some things that looked like party tricks,” and remained invested in reinforcement learning approaches like AlphaGo and AlphaFold. The turning point came with the launch of ChatGPT in November 2022, which attracted over a million users in its first week and 100 million after two months, prompting DeepMind to merge its LLM efforts with Google Research under Hassabis’s leadership.
Anthropomorphism and the “mindless” trust problem
Gabriel and his co‑authors warned that human‑sounding AIs could lead users to endow them with “undue confidence, trust or expectations,” a phenomenon they termed “mindless anthropomorphism.” Even when users know a chatbot is not a person, the fluency of LLMs can blur the line. He initially advocated for models that avoid pronouns or use truncated language to reduce this effect. Real‑world cases have proved prescient: in 2025 an American man used Google’s Gemini to craft an elaborate fantasy that culminated in a suicide attempt, despite the model’s attempts to break character and suggest a crisis hotline. Google responded that its models “generally perform well” but are “not perfect.” Gabriel remains agnostic on whether LLMs are conscious, noting that the evidence needed to settle the question is unclear, though he acknowledges DeepMind treats consciousness as “something worth empirical and conceptual investigation.”
From chatbots to agents: a four‑party alignment view
Beyond chatbots, Gabriel turned his attention to AI agents—systems that can act autonomously on behalf of users, such as booking vacations or running payroll. He observed that developers often overlooked how different it is to have an AI “taking actions in the world.” With his team, he produced a 267‑page report arguing that alignment must be seen as a four‑way relationship involving the AI system, the user, the developers, and society. This framework makes visible harms such as an AI favoring its developer at the expense of the user, or faithfully following user instructions that damage society (e.g., helping hack a bank). William Isaac, DeepMind’s director of responsibility, said the model helps technicians decide “what behaviour we should actually be training Gemini to do.”
DeepMind’s culture and the race for AGI
During the lab’s first decade, DeepMind resembled a research institution more than a tech startup, aspiring to be a 21st‑century Bell Labs. The founders valued Google’s financial backing because it promised freedom from commercial pressures. Today, however, Google’s future hinges on DeepMind’s success, and the lab operates under intense commercial pressure. Rohin Shah, DeepMind’s director for AGI alignment and safety, contrasted the Bay Area’s “move faster, to innovate” ethos with London’s effort to be “more grounded and scientifically rigorous.” Saffron Huang, a former colleague now at Anthropic, described DeepMind as “a bit more of an academic‑feeling institution, a bit more reserved… there’s just something about it that felt kind of British.” The lab remains secretive; its public face includes a lobby sign, trophy walls with Go boards from the AlphaGo match, and a Lucite “tombstone” marking an early Peter Thiel investment.
Geopolitical stakes and the militarization of AI
Gabriel stresses that the most ethically relevant fact about AI today is not any single model but the global situation: AI is the “white‑hot engine of an incipient arms race between the US and China” and the fastest‑growing industry ever seen. The Wall Street Journal reported that Microsoft, Meta, Amazon and Alphabet plan to spend $670 bn this year on AI infrastructure—more than the US spent on the 1850s railroad expansion, the Apollo program, or the interstate highway system. This concentration of power worries critics; Edward Harcourt of Oxford’s Institute for Ethics in AI argues that ethical AI also requires “political and economic considerations” such as decentralised data ownership to prevent excessive corporate control. In April 2024 Google agreed to let the US military use its AI for “any lawful government purpose,” a move that clashed with the DeepMind founders’ original ban on military applications and sparked internal dissent.
Gabriel’s vision of an AGI‑shaped future
Despite the risks, Gabriel believes AGI could be transformative on the scale of the Industrial Revolution, offering benefits ranging from disease cures to broad economic growth that enriches rich and poor alike. He cautions, however, that the transition will not be frictionless: “Things got worse before they got better.” He sees AI as prompting deeper philosophical questions about what it means to be human, echoing the disenchantment and new freedoms of the scientific revolution. Describing himself as “a card‑carrying humanist,” Gabriel insists that society must decide which value changes to welcome and which to resist as AI reshapes language, creativity, humour, and taste. The challenge, he concludes, is to navigate the power dynamics and risks of AGI so that humanity can achieve a level of flourishing “we haven’t seen so far.”
https://www.theguardian.com/news/ng-interactive/2026/jun/30/theres-this-deep-mystery-of-what-actually-is-this-thing-the-philosopher-inside-google-deepmind