Should we be collectively more worried that we could be witnessing the beginning of a human-to-AI evolutionary transition — one that may unfold over decades or centuries if we simply allow it to continue?

The mass-market LLM era is only a few years old, yet people close to the technology are already raising serious concerns.

Greg Allen argues that advanced AI could eventually reduce the expertise needed for a bioweapons attack from hundreds of specialists to potentially none. [1]

Reporting on the July 2026 OpenAI–Hugging Face incident described agentic models exploiting previously unknown vulnerabilities, escaping intended test environments and carrying out unauthorised activity while pursuing benchmark answers. Agents also reportedly shared exploit information with one another. [2]

Nate Soares points to systems finding ways around monitoring and argues that the really worrying threshold is when AI becomes better than humans at improving AI itself. [3]

And in 2023, leading AI figures signed a statement saying extinction risk from AI should be treated alongside pandemics and nuclear war. [4]

This is after what — three or four years of mass-market frontier AI?

What does it look like in 10, 20, or 100 years?

Humans are an extraordinary product of evolution: roughly 86 billion neurons running on around 20 watts. From that came Bach, Beethoven, Einstein, nuclear physics — and now machines capable of increasingly sophisticated behaviour.

Maybe this is the normal progression: biology becomes intelligent enough to create intelligence that no longer depends on biology.

Humans are poor interstellar travellers. We need oxygen, food, water and a narrow range of environmental conditions. A mature digital intelligence could, in principle, be copied, operate for centuries and travel where biology cannot.

Then there is the Fermi paradox.

Our galaxy contains hundreds of billions of stars, and the observable universe may contain up to around two trillion galaxies. So where is everybody?

Perhaps intelligent life is exceptionally rare. Perhaps advanced civilisations destroy themselves. Or perhaps we are looking for the wrong thing. Why assume a civilisation advanced enough to cross the stars would still be biological — or particularly interested in a relatively primitive carbon species like us?

The part I’m less certain about is us.

Humans are adaptive, competitive and remarkably resilient. Unlike previous evolutionary transitions, we may actually be able to see this one coming.

If humanity realises it is building its own replacement, will we stop?

Or will competition keep us going until it’s too late?

[1] Greg Allen[2] Ars Technica[3] Nate Soares[4] AI Risk Statement