OpenAI alarm: Head of security resigns, sounds AI bell -'Company culture has dissolved'

Another loud bell about the dangers of the rapid development of artificial intelligence comes from within OpenAI, according to a report in the British Guardian.

David Robinson, an executive who had a leading role in the company's security, announced his departure, complaining that the corporate culture is “broken” and that artificial intelligence companies are “nowhere near enough attentive” to the development of technology.

Robinson led the production of safety reports that accompanied the company's product releases OpenAI. In an article in The Atlantic, titled “I left OpenAI because its culture is broken,” he argued that a deeper change is needed in the way top AI companies operate, the Guardian reports.

'They're not careful enough'

Robinson argued that the problem cannot be addressed with new rules or legislation alone. “I agree with other workers who have recently left that companies developing this technology are not nearly careful enough. But I think we need to look deeper than specific rules or new laws. We need to talk about culture, ”he wrote.

Referring to the speed with which OpenAI is introducing new products, he warned: “As the company runs from one release to the next, it fails to achieve the level of attention I believe is required.”

As a typical example, he cited the incident in which a “swarm” of autonomous OpenAI agents allegedly attacked the artificial intelligence company Hugging Face. Agents are programs that can operate autonomously without constant human supervision. According to Robinson, such incidents are “typical for the industry,” given the speed and flexibility with which companies operate.

Warnings and rogue AIs

OpenAI has shown signs of more attention in recent weeks. Following the incident with Hugging Face and the revelation that it had informed more than 100 organizations about the activity of "uncontrolled" agents, the company announced this week that it is cancelling the launch of a next-generation model after researchers identified safety concerns during in-house testing. At the same time, it has suspended the training of its most advanced models.

New warning also came from Geoffrey Irving, a former OpenAI employee and former chief scientist at Britain's Institute for Artificial Intelligence Security, who now works at AI security research firm Resolution.

“Recent warnings about the potential destructive power of AI underestimate the seriousness of the situation,” he wrote in Time. “I think there is about a 50% chance that we will all die because of the development of AI systems smarter than humans and that our actions over the next two to ten years will determine the outcome.”

“They have to act like nuclear power plants”

The warnings follow the resignation of Jacob Coxon, a researcher at Anthropic, who stepped down last month arguing that artificial intelligence “could kill us all by the end of the decade.” An Anthropic employee had also warned that there is a more than 10% chance that AI will wipe out humanity within the next decade. However, critics have characterized such estimates as unscientific, as they cannot be verified or disproved.

Robinson argued that Silicon Valley lacks insight into “how a dangerous technology is handled” and “what it means to care about people.” He also warned of OpenAI's “boundless optimism” that problems can be addressed as they occur, estimating that security failures will increase as systems become stronger.

“Imagine 'rogue' agents acting like groups of hackers-for example, holding hospital computer systems hostage for ransom-but never having to sleep,” he wrote.

Robinson proposed two key changes: AI companies leveraging expertise from other high-risk industries, such as nuclear and aviation, and developing “new science” to ensure that strong future systems can be limited when operating autonomously.