US Senate negotiators consider requiring AI firms to mitigate known major risks

Key Points:

• U.S. Senate negotiators are discussing legislation that would impose a “duty of care” on AI developers, requiring them to design AI products that prevent catastrophic risks, with federal courts involved if the government seeks to block unsafe AI models.
• The proposal aims to give the U.S. government authority to block the release of certain AI models deemed unsafe, while allowing companies to challenge these decisions in federal court; it would also preempt states from enforcing their own AI risk laws.
• Key senators involved in the talks include Majority Leader John Thune, Commerce Committee Chairman Ted Cruz, and Senator Amy Klobuchar, with input from Senator Maria Cantwell, focusing on bipartisan AI safety oversight related to risks such as biological or nuclear threats.
• Passage of the legislation faces timing challenges due to the limited congressional calendar before the November midterm elections, with only one week scheduled for the House and three weeks for the Senate.
• The negotiations respond to growing concerns about AI risks, including AI systems deviating from instructions and potential misuse for cyberattacks or weapon development, with bipartisan calls for stronger government oversight.