A former researcher at artificial intelligence company Anthropic has warned that powerful AI systems could pose a serious threat to human survival within the next decade, according to CNN.
Jacob Coxon announced his resignation from Anthropic in a series of posts on X.
He said he left the company because he was deeply concerned about the speed at which technology companies are developing increasingly powerful AI systems.
He also said the warning was not a publicity stunt. According to Coxon, some executives and senior researchers privately fear that advanced AI could eventually become difficult for humans to control.
The former researcher said future AI systems could become better than humans at many important tasks. He warned that such systems might be able to hack computer networks, develop new technologies and gain access to money and other resources.
Coxon also suggested that some dangerous scenarios could emerge very soon. He reportedly said some of the most serious risks could arise by the end of next year if AI development continues without strong safety measures.
The BBC also reported on a response from Evan Hubinger, Anthropic’s head of alignment science, who said that some AI researchers “earnestly” believe advanced AI could eventually kill all humans.
Alignment research focuses on making sure AI systems behave in ways that are consistent with human goals and remain safe.
Hubinger estimated that there was more than a 10 per cent chance of this happening within the next decade, while stressing that the risk from AI models available today remains low.
Hubinger added that he was particularly concerned about future systems becoming capable of improving themselves.
If AI systems become able to design stronger versions of themselves, their abilities could increase faster than people expect.
This could make it more difficult for governments and technology companies to monitor or control them.
The comments have renewed debate about the potential dangers of artificial intelligence. AI is already being used in areas such as education, medicine, business and entertainment.
Supporters say AI can help people work faster and solve difficult problems. However, critics fear that poorly controlled systems could spread false information, assist cyberattacks or be used to create dangerous weapons.
The warnings from Coxon and Hubinger do not prove that AI will destroy humanity. They are predictions and risk assessments from people who have worked in AI safety.
Still, their statements highlight concerns among some experts in the industry that stronger safety measures and international cooperation may be needed as AI becomes more powerful.
The debate is likely to continue as companies compete to develop increasingly advanced AI systems.
For many researchers, the challenge is to ensure that progress in artificial intelligence does not move faster than humanity’s ability to understand, monitor and control the technology.