It’s intelligent! (Who’s afraid of AGI?)
Artificial General Intelligence (AGI) raises hope and concern in equal measure. Capable of competing with humans across all areas of cognition, it promises to transform the world—but at what price? We dive into an uncertain future, which holds both promises and threats.
Artificial General Intelligence, by Julian Togelius, MIT Press, 2024.
AGI: What are we talking about?
We have five years left… According to Ray Kurzweil, the big shot of Google-made futurism, Artificial General Intelligence (AGI) will come to be by 2029. While these predictions date back to 2005, Kurzweil was recently interviewed and stands by his prophecy, even suggesting he wouldn’t be surprised if the great leap happened sooner.
What great leap are we talking about? What exactly do we mean when we talk about AGI? For Julian Togelius, AGI is the digital Holy Grail—a system capable of doing everything a human can, but without the headaches or sudden cravings for coffee. It’s AI with unparalleled flexibility, capable of solving diverse problems across varied contexts. This stands in stark contrast to today’s machines, the “narrow AIs,” which excel only within their limited domains: playing chess, recognizing cats in photos, and that’s about it.
Not all experts agree on the definition : Ben Goertzel, one of the pioneers of the AGI concept, describes it as a universal and conscious brain, while researcher Murray Shanahan envisions it as a machine with ultra-adaptable skills but lacking deep understanding.
However, they all agree on one point: AGI will not be a mere enhancement of generative AI, which has revolutionized how we create new content—texts, images, music, or videos. As powerful as it is, generative AI remains a specialized form of intelligence: it can produce original outputs, but it doesn’t understand the concepts it handles.
Ask ChatGPT
And it will candidly admit: “My expertise is limited to what I was trained for: providing relevant and useful answers based on a vast corpus of text. However, I cannot autonomously adapt to completely new or unfamiliar tasks. My decisions are not guided by internal goals but by probabilistic models.”
If you ask whether it could evolve into AGI, it responds pessimistically: “Becoming AGI would require major technological advances beyond what my current architecture allows.”
Togelius, for his part, also takes a cautious stance, pointing out that significant technical hurdles remain: insufficient computing resources, inefficient learning methods, and data too limited to train a machine capable of excelling in the unexpected. For now, AGI remains a distant dream.
How human?
Conscious and super-intelligent machines?
Let’s ImAGIne for a moment that a machine overcame all the barriers we just discussed and managed to match—or even surpass—human intelligence in adaptability. Would it feel? Would it be conscious? Might it one day not only mimic our behavior but also experience the world with the same subjective intensity as we do?
The topic is far from settled—and that’s hardly surprising, given that consciousness itself remains a mystery even in the biological context.
Broadly speaking, AI thinkers fall into three categories:
- Physicalists, who argue that consciousness can only emerge in a biological medium.
- Functionalists, who believe that as long as AI operates like a human brain, it could develop consciousness.
- Philosophical zombie theorists, who maintain that an AI system could perfectly simulate consciousness without actually experiencing it.
This debate, which is way more than a mere squabble among experts, raises profound ethical questions: if AI were to develop consciousness, wouldn’t it deserve certain rights, much like humans? Could we then unplug or modify it without its consent, without committing some form of injustice?
Asked about this topic, ChatGPT warns: “As an AI, I lack consciousness and subjectivity. However, it is wise to anticipate these debates and establish ethical frameworks before a real situation forces us to confront them.”
An optimized future?
If so many researchers and such vast budgets—$200 billion from GAFAM alone in 2024—are being poured into the race for AGI, it’s not—entirely—for glory. It’s also because the advent of Artificial General Intelligence would represent a decisive milestone in human history, offering disruptive benefits across numerous fields.
By freeing individuals from repetitive or purely technical tasks, AGI could not only save time and increase efficiency but also refocus human efforts on strategic and creative activities.
Example #1 : in industry
Machines could handle predictive maintenance and supply chain optimization, thereby cutting costs and minimizing disruptions. AGI could also automate the initial stages of design in complex fields like automotive engineering or architecture, and simulate thousands of prototypes within hours. This would allow human engineers to focus on aesthetic innovation, user experience, or creative solutions to align these models with societal needs.
Example #2: In the medical sector
The analytical capabilities of AGI could pave the way for highly personalized healthcare. By combining massive clinical data, image recognition skills and conversational abilities, AGI could deliver highly accurate diagnoses and develop tailored treatments and prevention strategies based on each patient’s genetic profile, lifestyle, and psychological traits. AGI could also take on a proactive role by identifying weak signals indicative of pandemics, which would enable faster and more targeted responses.
Similarly, in education, AGI could revolutionize learning with individualized curricula, adjusted in real time to match students’ progress and difficulties.
Example #3: For the planet
AGI could become a powerful ally in combating climate change and environmental risks by helping us understand ongoing disruptions, better preparing for them, and mitigating their effects wherever possible. By integrating data from diverse sources—satellites, sensors, scientific reports—AGI could design comprehensive strategies to reduce greenhouse gas emissions, restore degraded ecosystems, and sustainably manage critical resources like water and rare earth elements.
Or a nightmare for our civilization?
These promises should not overshadow the inherent dangers of such powerful tools. Among the catastrophic scenarios, the first that comes to mind is undoubtedly that of omnipotent AGIs.
Not only might they evolve toward a form of consciousness, but they could also become “super-intelligent” and capable of improving themselves exponentially, quickly escaping human control. They could redefine their own objectives in unpredictable ways, leaving humanity without a say in the matter…
A groundless fear? Not for some researchers at OpenAI, who wrote to their management to raise alarms about a previously secret AGI project—Q-star—that they believe “poses a threat to humanity.”
Risk #1: The black box
Even without reaching catastrophic extremes, the rise of AGI could sharply amplify the “black box” phenomenon that many users already criticize in today’s specialized AIs.
Current AI systems often develop biases that even their creators struggle to understand. With AGI, these blind spots are likely to grow. Will we be able to detect failures and deviations in time? And can we prevent AGI from making decisions that conflict with our values—all in the name of the very goals we assigned to it?
Risk #2: My job
The deployment of AGI could profoundly disrupt the job market. Unlike specialized AI, AGI would perform a wide range of tasks, from writing to strategic decision-making, thus putting even skilled jobs at risk. Industries like finance, medicine, and IT could be deeply transformed, leading to the displacement of human workers. This shift could exacerbate inequalities between highly skilled employees—who can collaborate with AGI—and those whose roles become obsolete, if they lack the resources or training to adapt to this new paradigm.
Risk #3: The Concentration of Power
Given their immense demands for data and computing power, AGIs won’t be accessible to the masses. Only a handful of major tech companies and wealthy nation-states will have the resources to develop and sustain them, meaning these entities will yield decisive influence over the rules of the global game.
The fact that Elon Musk announced a new €5 billion fundraising round for his AGI project xAI just days after aligning with the Trump administration does little to inspire confidence.
Risk #4: The environmental impact of AGI
The infrastructure required to power AGI will consume colossal amounts of energy. At a time when the climate crisis demands a shift to low-carbon lifestyles, the rise of AGI presents a difficult paradox. While AGI promises solutions to optimize resource use and reduce emissions, its own ecological footprint could exacerbate the very problems it is designed to solve.
AGI: The new pharmakon, both cure and poison… Other dangers also emerge: the exponential acceleration of an uncontrollable super-intelligence, loss of privacy and enhanced surveillance, misalignment with human values leading to perverse or dangerous outcomes, military conflicts fueled by AGI applications, and cognitive manipulation (shaping opinions and behaviors), among others.
We won’t give ChatGPT the final word, because it’s up to us humans to write it—with all our doubts, all our scruples.
© Copyright Business Digest - All rights reserved
