Yapay zeka kendini geliştirme korkuları neden 'varoluşsal' endişelere neden oluyor?

Fears over the safety of AI systems — and their potential to wipe out humanity — gained new, viral traction this week.

Evan Hubinger, an alignment lead at Anthropic, said on X that he thinks there is more than a 10% chance that AI could kill all humans within the next decade, after a colleague quit over safety fears.

More warnings from researchers at both Anthropic and OpenAI followed. Cue a social media frenzy.

But it was in Hubinger's reply to his own post that revealed where exactly his concerns lay.

"What I am worried about is superintelligence arising from recursive self-improvement, as we have said is happening faster than we thought," he said.

Recursive self-improvement, or RSI, is when AI itself helps improve the process of building new models, potentially leading to spiralling capability as better systems build better systems and so on.

The worry is that if AI takes control of how new models are trained, the very humans who initially built those systems could lose control.

Both OpenAI and Anthropic have in recent months said that this autonomous model improvement is happening faster than they thought.

"Our internal data shows Claude is accelerating AI development—a possible path to recursive self-improvement, or AI autonomously building a more capable successor," Anthropic posted on X in June.

"It's happening faster than we thought, and the implications deserve greater attention."

While AI hasn't hit the point of RSI yet, it's already accelerating the development of AI systems. Anthropic said in a blog post from August about RSI that its engineers on average ship eight times as much code per quarter as they did between 2021-2025.

"AI is already at the level where it can introduce some new ideas," Vincent Conitzer, professor of computer science at Carnegie Mellon University, told me.