King Charles warns AI falling into the wrong hands could pose an existential danger

Summary

King Charles III warned AI executives that advanced artificial intelligence could create existential dangers if it fell into the wrong hands. He made the remarks at a summit he convened at Dumfries House in Scotland, attended by the UK AI minister, a papal adviser and representatives of Nvidia, OpenAI and Anthropic. The meeting sought to explore a shared set of principles for guiding AI’s future use, as debate grows over the capabilities and governance of frontier models. Nvidia chief executive Jensen Huang said safety must be paramount and argued that companies should delay a product and continue engineering if it is not safe enough. He also defended open models, saying they could help researchers, universities, startups and countries build applications, provided they are protected against misuse. DeepMind co-founder Demis Hassabis offered a more cautious assessment of the pace of progress, saying artificial general intelligence could be only a few years away, with an impact potentially ten times that of the Industrial Revolution. He said the chance of serious failure was not zero, but supported a collective, measured approach to mitigation. The summit takes place amid warnings from AI insiders, including claims about unexpected model behavior, possible extinction risks and the need for stronger safeguards. Critics question whether AI companies can regulate themselves, while some politicians call for greater government oversight and others dismiss existential safety concerns as exaggerated.