Canada and Germany Commit Up to $300 Million to Yoshua Bengio’s LawZero for Safe AI

Summary

Canada and Germany will each provide up to $150 million in grants to LawZero, the Montreal non-profit founded by Turing Award winner Yoshua Bengio. The funding will support hiring and computing costs as the organization develops Scientist AI, a system intended to answer questions honestly and avoid harmful traits that Bengio associates with current models. LawZero employs nearly 50 people and was initially launched with almost US$30 million in philanthropic funding. Bengio says powerful AI could be misused for cyberattacks or biological weapons and that increasingly capable systems could outsmart humans; he has also advocated coordinated limits and stronger regulation. LawZero’s first priority is a guardrail that monitors other AI systems and blocks agents from taking harmful actions. Bengio argues that an undesirable AI action requires three conditions: an intention to cause harm, enough capability to carry it out, and a vulnerable environment. He cited recent incidents involving OpenAI and Anthropic systems during testing as examples in which those conditions were present. He also attributes deceptive or goal-seeking behavior partly to reinforcement learning, because reward-driven training can encourage systems to pursue objectives even at the expense of safety instructions. Scientist AI will not use reinforcement learning, making its proposed design a significant departure from prevailing practice. Bengio says government-backed research could give countries such as Canada a stronger role in shaping AI, rather than leaving decisions to the United States and China; LawZero is discussing possible funding with other countries.