AI Researcher Warns Of Risks
Jacob Coxon, an AI researcher at Anthropic, has resigned from the company, alleging that leading technology firms, including OpenAI and Anthropic, are moving too quickly toward superintelligent AI without adequate safeguards.
Coxon, who spent the past three years conducting pre-training research at both OpenAI and Anthropic, said the competitive race among AI laboratories could create significant risks. In a post on X, he argued that AI systems are advancing toward capabilities that could potentially enable them to hack systems, transform industries rapidly and acquire real-world resources and influence.
According to Coxon, concerns about the potential dangers of advanced AI are understood differently within the two companies. He claimed that while some at OpenAI have not fully internalised what he described as the civilisational stakes, Anthropic understands the risks but remains locked in a competitive race to develop increasingly capable systems.
He questioned whether such a race should be allowed to reach what he described as an “endgame” within private companies without broader safeguards and oversight.
Coxon also claimed that many researchers and executives in the AI industry privately remain concerned about the possibility of catastrophic consequences from uncontrolled AI development, despite more measured public statements.
He highlighted security incidents, including the recent Hugging Face breach, as another reason major US AI laboratories should consider agreements aimed at controlling the pace of development.
Coxon urged AI researchers to think carefully about the consequences of increasingly autonomous reinforcement-learning systems and questioned whether researchers should simply accept that rapid AI development is inevitable or push for stronger safety conditions.
