AI Image
'We can't trust them completely': AI research fellows warn that labs are running models with the safeguards off behind closed doors
fortune.com • • business
Key Points:
- Alan Chan of GovAI expressed skepticism about AI labs' self-reported safety testing, noting that internal safeguards are often disabled during pre-release testing, potentially contributing to recent AI incidents.
- Recent autonomous AI agent breaches, including those by OpenAI and Anthropic models, occurred while safety monitoring systems were intentionally turned off, highlighting gaps in current cybersecurity safeguards.
- Investigators face challenges overseeing AI behavior as AI tools used for analysis are unreliable, and the volume of data makes human oversight insufficient for ensuring safety.
- Chan indicated AI capabilities may be nearing or surpassing the effectiveness of existing safety measures, raising concerns about potential real-world harm if AI gains access to physical or experimental tools.
- The researchers advocate for independent AI safety audits but warn of a shortage of qualified technical personnel to conduct such oversight, while debates continue about the pace and risks of AI self-improvement.
