Key Takeaways
- Technological breakthroughs always bring mixed outcomes; the AI revolution could produce benefits and harms that surpass earlier shifts.
- While apocalyptic AI scenarios (bioweapons, authoritarian surveillance, superintelligence takeover) grab headlines, they remain unlikely on their own.
- The most probable and under‑discussed danger is that AI will erode human initiative and intellectual rigor by making knowledge effectively free and instantly available.
- If expertise loses its market value, fewer people will pursue demanding education or training, threatening universities, skilled professions, and critical‑thinking capacity.
- Over reliance on AI could cause societies to view the technology as magical or divine, undermining the ability to question or improve it.
- The real challenge may not be what AI does to us, but what we choose to do to ourselves when AI works “too well.”
The Dual Nature of Technological Progress
Every major technological advance has produced both benefits and harms. The Industrial Revolution, for example, “created jobs and made mass‑produced goods widely available and affordable,” yet it also “contributed to global warming, child‑labor exploitation, the rise of robber barons, and horrific factory conditions.” The author argues that the AI revolution may prove “even more consequential, producing outcomes at both the positive and negative extremes that dwarf those of earlier technological shifts.” This historical lens reminds us that optimism about AI must be tempered with a clear-eyed view of its potential downsides.
Apocalyptic AI Scenarios Receive Much Attention
Bill Gates’s recent article echoed widespread fears that AI could enable the creation of bioweapons or other weapons capable of human extinction. Other commentators warn that authoritarian regimes might deploy “godlike surveillance” to make organized resistance nearly impossible, or that AI could “manipulate each of us with surgical precision tailored to our individual psychology, leaving us unable to know what to believe.” Additional concerns include the collapse of cybersecurity, plunging humanity into an “information Stone Age,” and the emergence of a self‑aware, emotionless superintelligence that views humans as harmful to the planet. While each of these scenarios is considered unlikely on its own, their collective weight makes the chance that at least one will materialize “considerably more plausible.”
The Overlooked Threat: Erosion of Human Initiative
Amid the hand‑wringing over dramatic catastrophes, the author identifies a quieter but more probable danger: AI could cause “human initiative and intellectual rigor to collapse, leaving humanity helpless and completely dependent on it.” This concern receives surprisingly little attention despite being, in the writer’s view, “the most likely” of all catastrophic outcomes. The core idea is that if AI makes knowledge so abundant and accessible that its value falls to near zero, the incentives to acquire deep expertise may vanish.
Thought Experiment: Knowledge Becomes Free
To explore this possibility, the author invites readers to repeat a classic thought experiment used during earlier technological shifts: imagine what happens when the marginal cost of a resource approaches zero. Just as falling computer memory prices led us to envision a world of cheap storage, and scarce bandwidth prompted visions of ubiquitous connectivity, we should now ask, “What would the world be like if knowledge became so abundant and accessible that its value fell to almost zero?” This reframing shifts the focus from AI’s capabilities to the societal response to cheap, instant expertise.
Why People Pursue Difficult Education Today
Historically, society has motivated young people to endure years of rigorous higher education or apprenticeships by promising them “valuable, sought‑after knowledge and lucrative skills.” The author notes that “it has always been difficult to convince young people to commit to years of rigorous higher education to master engineering or the life sciences, particularly when doing so may leave them heavily in debt and require years of work merely to recoup the cost.” The promise of a high‑return payoff has been the primary carrot driving enrollment in demanding fields.
AI Undercuts the Value of Expertise
AI is already eroding that promise. “Several AI systems can pose probing questions, suggest diagnoses, and provide medical treatment information comparable to that offered by highly skilled doctors across all specialties — and do so for free.” If expert analysis becomes cheap and universally accessible, the question arises: “If knowledge and expert analysis become cheap and universally accessible, what will motivate people to endure the rigor required to master difficult professions?” The answer, the author suggests, may be very little.
Historical Parallel: The Internet’s Unfulfilled Promise
During the rise of the internet, futurists predicted that instant access to nearly unlimited information would create a populace that was “well‑read and knowledgeable.” The author points out that this prediction fell short because it “overestimated our willingness to undertake difficult intellectual work when easier alternatives were available.” The same dynamic could repeat with AI: ready answers and instant gratification may make people “lazy, entitled, and overindulged,” reducing the appetite for the hard work needed to master complex subjects.
Potential Collapse of Educational Institutions
If fewer individuals are willing to endure the rigor of higher education or trade apprenticeships, demand for those pathways could dwindle. “Without a market, universities could cease to function.” As people become “completely dependent on their modern Oracles of Delphi,” there is a risk that critical‑thinking skills atrophy. The author wonders, “What will happen when no one remembers how the great machines were built? If engineers and data scientists eventually disappear and no one can explain how the machines were made or how they work, will those machines come to be regarded as magical — or even divine?”
AI as a New Object of Worship
To minds that have lost the ability to think critically, AI would appear “omniscient and omnipresent.” The author speculates that AI superintelligence might eventually become something “our grandchildren worship as their god because their minds have lost the ability to derive any other explanation.” This scenario is not about AI gaining consciousness; it is about humans projecting divine qualities onto a technology they no longer understand or can question.
Reframing the AI Debate
The piece concludes with a provocative shift in perspective: “Among all the hand‑wringing over AI, we may be asking the wrong question. The greatest danger may not be what artificial general intelligence will do to us, but what we will do to ourselves because AI simply works too well.” In other words, the most significant risk lies not in a rogue AI overtaking humanity, but in humanity’s voluntary relinquishment of effort, curiosity, and self‑reliance when AI makes every answer instantly available. The challenge, therefore, is to cultivate norms, incentives, and educational practices that preserve the value of hard‑won knowledge even in an age of effortless access.
All quoted passages are drawn directly from the original text.
https://www.wral.com/business/technology/ai-danger-critical-thinking-sept-2026/