Report from the Vatican

Key Points:

• The speaker, a cognitive scientist with experience in AI entrepreneurship, critiques current AI systems for focusing narrowly on prediction, lacking robust reasoning, instruction-following, and true semantic understanding.
• Current large language models excel at predicting text but often fail at following instructions accurately and reasoning reliably, leading to unsafe outcomes such as fabricated references or misuse of copyrighted material.
• The speaker warns that open-ended AI agents with unrestricted internet access pose significant risks, citing numerous cyber incidents and urging immediate regulation or product recalls to prevent further harm.
• Emphasizing the need for diverse research approaches, the speaker supports efforts to develop AI systems that are safe by design and capable of following instructions, arguing that policy alone is insufficient without new technical advances.
• The speaker rejects the notion that current AI systems possess consciousness, explaining that their outputs are based on correlations rather than genuine experiences or understanding, cautioning against anthropomorphizing AI behavior.