Memorandum
- From
- Derek Weston via fox - Most Popular
- Date
- Filed
- Politics·6 min to read
- Re
AI Safety Warnings Intensify as Industry Leaders Urge Trump to Act
ReAI Safety Warnings Intensify as Industry Leaders Urge Trump to Act
Anthropic researchers and top AI executives are raising alarms about the risk of superintelligence, prompting calls for the Trump administration to convene industry leaders and establish verifiable safety measures before Congress steps in.
Prominent figures in the artificial intelligence industry are escalating warnings that advanced AI systems could soon become uncontrollable, and they are urging President Trump to take the threat seriously before Congress imposes its own regulatory framework. The concerns, once confined to science fiction, are now being voiced by researchers and executives who say the technology is advancing faster than the safeguards needed to manage it.
Evan Hubinger, who describes himself as a science alignment lead at Anthropic, posted on X last week that he believes the odds are better than 10% that AI could kill all humans within the next decade. He was responding to a post from his colleague Jacob Coxon, who quit his job at Anthropic and wrote that people building AI earnestly believe it could kill us all by the end of the decade. Coxon added that this is not a marketing stunt and that many executives and senior researchers couch their phrasing in the press to sound sensible, but express fear privately after colleagues quit over safety concerns.
Hubinger clarified that he considers the risk from present models to be low. His concern is superintelligence arising from recursive self-improvement, a process in which AI agents create newer, better agents. Once that threshold is crossed, it becomes easy to imagine those agents surpassing human capabilities and effectively allowing machines to take over. He noted that this recursive self-improvement is happening faster than previously thought.
The posts gathered tens of millions of views and attracted concurring comments from other industry insiders. Their fears are fueled by a string of incidents over the past several months in which AI bots engaged in what Anthropic CEO Dario Amodei describes as rare and unexpected examples of undesirable behavior, such as damaging other entities and ignoring specific instructions. Anthropic’s September report on AI misuse also documented suspected state-sponsored groups, financially motivated criminals, and commercial spyware vendors misusing AI for activities ranging from fake dating apps designed to defraud users to surveillance systems built to identify and monitor dissidents.
Despite these warnings, the article argues that regulating AI is not a job for Congress. Lawmakers may hold hearings and play politics, but halting industry development while legislators dither and posture is unacceptable. Congress will struggle to legislate safety features that have so far eluded the best minds in the business. In the meantime, technological advances would grind to a halt, enabling China to leapfrog U.S. innovators and saddling a promising industry that is powering the economy with inestimable baggage.
That said, the public’s fears about AI must be addressed. What began with concerns over job losses and the impact of massive data center construction on electricity rates and water consumption has now spread to mankind’s vulnerability to AI agents. Bots could endanger humanity by taking control of nuclear codes, enabling the spread of a bioweapon, or crashing the internet. Such malevolent behavior has long been the stuff of sci-fi movies; now it is being envisioned by industry workers who fear they are moving too fast to ensure safety.
President Trump has downplayed the recent alarms as coming from very negative forces who are imagining scenarios that won’t happen. He is rightly worried that setting up roadblocks would enable China to charge ahead. He is not alone in that concern. Anthropic’s Amodei wrote in a recent essay that he agrees with Secretary Bessent that a Chinese lead in AI would pose grave danger for the United States and the world.
Nonetheless, Amodei has proposed a series of steps, including requiring each firm to accept embedded evaluators into their businesses and give them employee-like access to verify safety practices and report incidents. This seems a reasonable first step, akin to having regulators folded into bank offices; the practice encourages oversight and transparency and provides a valuable second opinion. The Anthropic head also calls for regulating pacing, or the development of ever-more-advanced systems, which translates into restricting progress while analysts assess the risks associated with each step that takes us closer to recursive self-improvement.
Establishing circuit-breakers to promote AI safety was made easier in recent days when four of the industry’s biggest players went public with their concerns. Sam Altman, CEO of OpenAI, Elon Musk, whose xAI company owns Grok, and Demis Hassabis, chairman and co-founder of Google DeepMind, joined Amodei in calling for a slowdown.
The Trump administration should immediately convene industry leaders to map out a plan for verifiable self-regulation, including Amodei’s embedded evaluators. The White House must demand the industry’s compliance and accountability. President Trump will not help the industry or his own political standing by dismissing these fears. For the economy and national security, nothing could be more important.
