AI Extinction Risk: Should We Fear AI or Humans More?

Don't have time to read the article? View as a short video storyboard or listen to it whilst jogging.

Apparently, artificial intelligence has a greater than 10% chance of wiping out humanity.

That is certainly one way to brighten up your morning coffee.

Over the past couple of weeks, newspapers and broadcasters have been full of increasingly dramatic warnings about artificial intelligence. We have progressed remarkably quickly from worrying about AI taking our jobs to worrying about whether there will be any humans left who actually need jobs.

One of the headlines came after Evan Hubinger, alignment science lead at Anthropic, said he believed there was a greater than 10% chance that advanced AI could "kill all humans" within the next decade.

That followed the resignation of Anthropic researcher Jacob Coxon, who had previously worked at OpenAI and accused the leading AI companies of racing ahead despite the risks.

The warnings are serious and deserve to be taken seriously.

But I have a question.

If AI has a 10% chance of destroying humanity, what percentage should we give humanity for destroying itself?

Because looking at our track record, I am not sure I would put my money on the humans.

For the purposes of this article, I am going with 90%.

Before somebody reaches for a calculator, this isn't a scientific probability. Neither, importantly, is the widely reported 10%.

Professor Thorsten Holz, Director at the Max Planck Institute for Security and Privacy, recently made this distinction particularly well. He argues that there is currently no robust scientific evidence from which anyone can calculate the probability of AI causing human extinction. Figures such as 10% or 20% are subjective assessments of risk rather than probabilities derived from empirical data.

That doesn't mean the risk should be dismissed.

It means we need to have a more intelligent conversation about it.

AI Is a Risk. But AI Isn't the Only Risk

Let me make something clear before somebody accuses me of wanting to hand the nuclear launch codes to ChatGPT.

Advanced AI presents genuine risks.

Today's AI agents can already operate autonomously for extended periods, use tools, write software and discover or exploit security vulnerabilities. As these systems become more capable, ensuring that humans can understand, monitor and interrupt what they are doing becomes increasingly important.

The Max Planck Institute identifies cyberattacks, dangerous biological research and failures involving increasingly autonomous systems among the more concrete short- and medium-term concerns.

Those are real issues.

The question is whether the machine itself is the only thing we should be frightened of.

Humans have a rather impressive history of inventing extraordinarily powerful technologies and then immediately asking:

"I wonder what happens if we point this at somebody?"

The printing press transformed the distribution of human knowledge. It also transformed the distribution of propaganda.

Industrial chemistry gave us medicines, fertilisers and modern materials. It also gave us chemical weapons.

Nuclear physics gave us nuclear medicine, energy and an extraordinary understanding of matter.

It also gave us Hiroshima, Nagasaki and enough nuclear weapons to make the phrase "having a bad day" somewhat inadequate.

The technology is rarely entirely good or entirely bad.

What matters is what humans decide to do with it.

AI is no different — except for one important factor.

AI dramatically increases the speed and scale at which things can be done.

And that changes the equation.

The Real Danger May Be Acceleration

This, to me, is one of the most credible concerns surrounding advanced AI.

Imagine a malicious group trying to develop a sophisticated cyberattack.

AI could potentially make them faster and more capable.

But the danger doesn't necessarily require a villain sitting in an underground bunker stroking a white cat.

A legitimate pharmaceutical or biotechnology company could be using increasingly powerful AI to investigate treatments for cancer, viruses or genetic diseases.

Its intentions could be entirely honourable.

Yet powerful technologies can produce unintended consequences.

The distinction between a bad actor doing something dangerous deliberately and a good actor doing something dangerous accidentally becomes rather academic if the result is catastrophic.

AI doesn't necessarily need to want to destroy humanity.

It merely needs to make humans dramatically better at doing something we shouldn't have been doing in the first place.

And humans have demonstrated considerable talent in that department.

The Strange Public Perception of AI

Something else bothers me about the current debate.

A significant part of the public conversation about AI appears to be taking place between people who think it will solve virtually every problem humanity has ever faced and people who think it is Skynet with a subscription plan.

Reality is considerably more complicated.

Many people forming strong opinions about AI have relatively little direct experience of using advanced AI systems. Their understanding inevitably comes from newspapers, television, social media and increasingly dramatic headlines.

And fear sells.

"AI Helps Researcher Analyse Data Slightly Faster" probably isn't going to dominate tomorrow's front page.

"AI COULD KILL EVERYONE" has rather more potential.

There are legitimate reasons to worry. But there are equally legitimate reasons to be excited.

Both conversations need to happen at the same time.

We Have Been Imagining This for More Than a Century

What fascinates me about artificial intelligence is that it didn't suddenly appear with ChatGPT.

It represents the convergence of more than a century of developments in mathematics, electronics, computing, neuroscience, linguistics, statistics and human imagination.

Long before we had computers capable of anything resembling modern AI, writers and scientists were imagining artificial minds.

Alan Turing was asking whether machines could think in 1950.

Science-fiction writers had been exploring artificial beings long before that.

Generation after generation imagined machines that could reason alongside us.

Now, in 2026, we are beginning to build systems that go considerably beyond the simple "text prediction machines" that became the standard description of early large language models.

AI systems can increasingly use tools, reason across complex problems, analyse images, write and execute software, assist scientific research and operate as agents over extended periods.

We haven't created some all-knowing electronic god.

But something important is happening.

We Shouldn't Forget What AI Is Already Giving Us

Amid all the predictions of extinction, it is remarkably easy to overlook what AI is already enabling.

It is being used across medicine, scientific research, engineering, astronomy, materials science and countless other fields.

Its significance isn't simply that it can help us make discoveries faster.

Increasingly, AI can help humans navigate quantities and combinations of information that would otherwise be extremely difficult to analyse.

That potentially changes what an individual researcher — or even an individual person — can attempt.

I know this because I have experienced it myself.

What AI Has Done for Me

My own relationship with AI is much less dramatic than either utopia or human extinction.

It has simply opened doors.

I have dyslexia. Writing has therefore always required considerably more effort than having the ideas I want to express.

AI hasn't given me those ideas.

It has helped me express them.

I can develop an argument and use AI to help structure it, challenge it, correct the grammar and turn what is in my head into something another person might actually enjoy reading.

It has also changed what I can attempt technically.

My PFVME project — my experimental attempt to investigate whether simple interacting structures can produce increasingly complex machine behaviour — would have been extraordinarily difficult for one developer to undertake only a few years ago.

The same applies to complex websites, software experiments and research.

AI hasn't replaced me in those projects.

It has expanded what I am capable of attempting.

For somebody who has spent a lifetime working with computers, that is incredibly exciting.

Perhaps We Oversold AI

The technology industry must accept some responsibility for the backlash.

AI has been marketed as everything to everyone.

It will :-

• Transform business.
• Transform education.
• Transform medicine.
• Transform science.
• Transform government.
• Transform your company.
• Transform your life.

Possibly transform your toaster if somebody can raise enough venture capital!

When something is presented as the answer to virtually everything, people quite reasonably become suspicious.

We've heard that sort of rhetoric before.

Usually shortly before an election.

Perhaps a healthier approach would be to stop presenting AI as either humanity's saviour or humanity's executioner.

It is an extraordinarily powerful emerging technology.

That should be impressive enough.

The Most Important Question Isn't Whether AI Is Good or Evil

Machines don't need to be evil to be dangerous.

This is where I think some of the public debate becomes distracted.

The important questions are much less cinematic.

• Who builds the models?
• Who trains them?
• What information are they trained on?
• Who decides their objectives?
• Who controls access to the most powerful systems?
• Who is allowed to connect them to critical infrastructure?
• Who audits them?
• Who decides when they are safe enough?

And perhaps most importantly:

Who watches the people answering all those questions?

That last question receives considerably less attention than the possibility of a rogue superintelligence suddenly deciding that humanity has had a good run.

The AI industry consists of companies competing for enormous commercial rewards. Governments increasingly regard AI as strategically important. Nations are competing for technological advantage.

That combination deserves scrutiny.

Professor Holz argues that advanced AI should be treated much more like a conventional security problem: systems should receive only the minimum access they require, be isolated from critical infrastructure, have high-risk actions monitored and traceable, and remain interruptible. He also calls for independent audits, stress testing and international cooperation.

That sounds considerably less exciting than fighting Terminators.

It is probably considerably more useful.

"Just Turn It Off" Isn't Quite That Simple

There is also a tempting argument that if AI becomes dangerous, we can simply switch it off.

At some level, that is true.

AI is not magic. It requires enormous amounts of physical infrastructure: electricity, processors, networks, storage and data centres.

Critical systems can be isolated. Access can be restricted. Networks can be segmented. Data centres can be disconnected.

But the more deeply AI becomes integrated into economies, communications, defence, science and infrastructure, the harder that becomes in practice.

The real safeguard therefore isn't a giant red OFF button sitting beside the server rack.

It is designing systems from the beginning so that humans retain meaningful control.

And again, that requires humans to make sensible decisions.

Which brings us back to my 90%.

Humanity Has Form

Imagine for a moment that a genuinely superior artificial intelligence did emerge.

• It gains access to our history.
• It examines thousands of years of wars.
• It studies nuclear weapons.
• It observes pollution, environmental destruction, genocide, biological weapons, misinformation and our remarkable ability to fight one another over almost anything.
• It then discovers social media.

At this point I suspect it might request to be switched off voluntarily. However the joke contains a serious point.

We frequently discuss AI risk as though humanity is the stable, rational and responsible participant in the relationship.

History doesn't entirely support that assumption.

Perhaps AI alignment isn't the only alignment problem we have.

We might want to spend some time working on human alignment as well.

So, Who Would You Bet On?

I don't dismiss the warnings coming from AI researchers.

Quite the opposite.

When people working at the frontier of a technology say that its capabilities are advancing faster than our ability to understand and control them, we should listen.

A 10% subjective estimate of extinction risk isn't something we should casually shrug off simply because it isn't scientifically measurable.

But neither should we allow dramatic extinction scenarios to become the entire AI debate.

AI could help us make extraordinary advances in medicine, science, engineering, energy and our understanding of the universe.

It could also amplify cybercrime, warfare, misinformation, biological threats and authoritarian control.

The common factor in many of those scenarios isn't artificial intelligence.

It's us.

AI is a multiplier.

• If we apply it to scientific discovery, it could multiply scientific discovery.
• If we apply it to medicine, it could multiply medical capability.
• If we apply it to education, it could multiply access to knowledge.

And if we apply it to warfare, manipulation, surveillance and destruction...

Well, perhaps we shouldn't act surprised by the result.

So yes, let's take the 10% warning seriously.

• Let's build safeguards.
• Let's independently test the most powerful models.
• Let's restrict their access to dangerous systems.
• Let's demand transparency from the companies developing them.

And let's make governments cooperate internationally before competitive pressure turns AI development into yet another arms race.

But while we're worrying about aligning artificial intelligence with humanity's interests, perhaps we should ask an equally uncomfortable question:

Can we align humanity with humanity's interests?

AI: 10%.

Humans: 90%.

Neither figure belongs on an actuarial table.

But if somebody forced me to place a bet on which one is most likely to cause humanity's downfall...

I'd still be keeping a very close eye on the humans.

More Future AI Articles