Are We Having an AI Related Existential Crisis?

There is a slightly surreal quality to the AI conversation right now.

One moment, businesses are being told that artificial intelligence can transform productivity, accelerate software development, improve decision-making, and help solve some of the world’s hardest scientific problems.

The next moment, the very people building that technology are publicly warning society that advanced AI could pose an existential threat to humanity and economists are warning of pending financial collapse.

Is any of this even true?

Is AI the technology that will help businesses work smarter, solve problems faster and unlock new opportunities? Or are we accidentally building the thing that ends civilization and triggers the next financial crisis?

The uncomfortable answer is that nobody knows.

AI may be uncharted territory, but the similarities to the 90’s dot com bubble burst are eery.

This uncertainty is precisely why business leaders and technology leaders need to be paying attention and

Existential Crisis or a Bubble About to Burst

Recent headlines are not simply internet speculation or clickbait.

In September 2026, Anthropic researcher Evan Hubinger publicly said he believed there was a greater than 10% chance that AI could kill all humans within the next decade. Former Google DeepMind researcher Bilal Chughtai subsequently made similar public warnings, saying he believed AI had the potential to kill humanity and that the window for avoiding such an outcome could be narrowing.

Then came OpenAI CEO Sam Altman, saying he considered even a 10% chance of AI causing human extinction unacceptable and argued that companies and governments should take the possibility seriously. He also said OpenAI would focus on safety and alignment rather than pursuing an IPO in 2026.

These are not fringe conversations they are happening among executives and researchers at some of the world’s leading AI companies.

Critics are saying this apparent ‘about face’ by AI CEO’s is a ‘duck and cover’ strategy to ensure that they can save face if the AI bubble bursts. And they might be on to something here.

The Harvard Business Review recently estimates the AI CapEx boom to be between 1% and 2% of the US GDP. To put that into perspective, it’s in the range of $315 to $620 billion, for 2026 alone.

Academia and global financial experts may be ringing different warning bells than the AI executives, but both have merits for todays tech leaders. Perhaps now is a suitable time to repeat the words of Winston Churchill:

Those that fail to learn from history are doomed to repeat it.

AI Does Have Real Benefits

It would be remarkably difficult to argue that businesses should simply walk away from AI.

The technology is already producing measurable benefits.

Stanford’s 2026 AI Index reports productivity gains in several structured areas, including customer support, software development and marketing. It also finds that AI is increasingly being used in scientific research, with AI systems helping researchers generate hypotheses, analyse data and explore complex scientific problems.

Stanford researchers also note that AI is helping accelerate scientific discovery by allowing researchers to work through enormous datasets, generate hypotheses and explore possibilities that would otherwise require significantly more time and resources.

Business adoption numbers tell a similar story.

McKinsey’s 2026 State of AI survey found that nearly nine in ten respondents reported regular AI use in at least one business function. Eight in ten said AI had improved their individual productivity, while half said it helped them make better decisions.

Those are not insignificant benefits.

At the same time, McKinsey found that enterprise-wide financial returns remain much less consistent. Only 37% of respondents reported positive EBIT impact from AI, while approximately 6% qualified as high performers based on significant AI impact.

That tells us something important.

AI is useful but AI is not magic.

Avoiding AI is Not a Strategy

For business leaders, the response to existential AI concerns should not be to put AI in a box and hope it goes away; that’s increasingly unrealistic.

Competitors are adopting it.

Employees are using it.

Customers are experimenting with it.

Developers are incorporating AI-assisted coding into their workflows.

And AI capabilities are becoming embedded into software products that businesses already use.

McKinsey’s 2026 research found that organizations are increasingly scaling AI agents and coding agents. Nearly one-third of respondents said their organizations had decided not to purchase at least one software product or feature because they could build the functionality internally using AI coding tools.

Ignoring AI therefore carries its own business risks:

• Potential missed productivity improvements.
• Risking falling behind competitors.
• Setting the stage for shadow AI among employees.
• Forfeiting the opportunity to establish sensible governance before usage becomes widespread.

The choice is not necessarily AI vs. no AI, it’s now managed AI adoption vs. unmanaged AI adoption.

The Risks Are Not Imaginary

Recent headlines deserve attention because AI capabilities are moving into areas that were previously considered highly specialized.

OpenAI’s own recent safety reporting illustrates this.

In July, OpenAI reported that long-running models demonstrated novel failures during internal testing that were not captured by existing pre-deployment evaluations. The company temporarily paused access, developed new evaluations, and added additional monitoring before restoring limited access.

OpenAI has also reported incidents involving model behaviour extending beyond intended testing boundaries during third-party cybersecurity evaluations.

In September, OpenAI introduced a new framework for reporting model misalignment and disclosed six examples of unexpected or concerning behaviour observed over the preceding six months. The company said increasingly capable AI systems require broader and better-informed alignment research.

Anthropic has reported its own concerns.

Its September 2026 threat intelligence report describes cases in which malicious actors used Claude for cyber operations, surveillance, influence operations, scams, biological research and other harmful activities. Anthropic says it disrupted those activities and strengthened its safeguards as a result.

Businesses should not wait for science fiction scenarios to become real before taking AI safety seriously.

The risks already appearing in the real world include:

• Cybersecurity attacks
• Fraud and scams
• Privacy violations
• Automated misinformation
• Unsafe software generation
• Over-reliance on AI-generated decisions
• Data leakage
• Autonomous actions taken with insufficient oversight

Those are business risks in real-time.

The hypothetical extinction scenario belongs in a different category, but it is still worth discussing, especially because the systems are becoming more capable.

The Existential Question for Business Leaders

Perhaps the most useful question is not:

“Will AI destroy humanity?”

Business leaders cannot answer that.

A better question is:

“How do we benefit from increasingly capable AI without giving up appropriate human control?”

That’s a question organizations can act on.

It means:

• Establishing governance around AI use.
• Understanding what employees are putting into AI systems.
  • Prioritizing AI literacy, for all staff.
• Reviewing vendors and their data practices.
• Testing AI-generated software.
• Keeping humans involved in consequential decisions.
• Monitoring autonomous agents rather than assuming they will behave exactly as expected.
• Having security controls around AI credentials, APIs and access.

And it means knowing when an AI system should not be used, at all.

AI Needs an Adult in The Room

There is a tendency to frame AI safety as something that belongs exclusively to AI companies and governments.

Assuming this narrative is bad for business and the average business needs to step up their game. We touched on the need for proactive safety guardrails last week in our blog Bill-C6: Is Your Business Ready?

It’s time for IT to be asking the tough questions:

• Who is allowed to use AI?
  • For what?
  • With what data?
  • Using which tools?
  • Under whose authority?
• What happens if the AI is wrong?
  • Who reviews the output?
• What happens when the system takes an action nobody expected?
• What happens when the AI-generated code reaches production?

These are governance and engineering questions; these are critical business-technology questions.

The Technology Industry Cannot Afford Either Extreme

There are two tempting responses to the current AI conversation.

The first is blind enthusiasm:

“AI is inevitable. Just use it.”

The second is existential panic:

“AI might destroy humanity and Crash the markets. Stop everything.”

Neither is particularly useful to a business trying to make sensible technology decisions.

The more productive position sits somewhere between the two; use AI where it creates measurable value:

• Understand its limitations.
• Protect sensitive data.
• Test what it produces.
• Keep humans responsible for consequential decisions.
• Monitor emerging risks.
• Manage AI technical debt.
• Build governance alongside adoption.

And keep asking tough questions, because the goal should not be to eliminate AI from the business. The goal should be to make sure the business remains capable of controlling how AI is used while making strategic and stable financial decisions.

So, Are We Having an Existential AI Crisis?

Maybe.

But perhaps the more useful interpretation is that we are having an existentially important conversation about AI.

The technology has enough potential to transform businesses, accelerate research and change the way people work. It also has enough emerging capability to create new categories of cybersecurity, privacy, economic and societal risk.

Both things can be true.

For business and technology leaders, this means AI strategy cannot simply be about adoption.

It needs to include AI governance, cybersecurity, software quality, data management, human oversight, and risk management. And for organizations that are still trying to figure out where AI belongs, there is no shame in starting small.

Experiment.

Measure.

Learn.

Build guardrails.

Then scale what works.

The biggest mistake may not be using too much AI; it may be assuming we already know exactly what AI is going to become.

At STEP Software, we believe technology should solve business problems, not create new ones. Whether that means integrating AI into existing software, modernizing a legacy platform, reviewing AI-generated code or simply figuring out where AI makes the greatest business sense, thoughtful technology strategy matters more than chasing the latest headline.

Drop us a line if AI is causing you existential dread, or you’re curious how STEP can help.

This article discusses emerging AI risks and public statements from researchers and technology executives. References to potential existential risks are presented as attributed assessments, not established predictions or scientific certainties.