Google's AI search features pose 'unacceptable risk' to children, new report finds

Key Points:

• A Common Sense Media report warns that Google's AI search features, AI Overview and AI Mode, pose significant safety risks to children, routinely failing to recognize harmful content and providing incorrect or inconsistent answers during tests with child accounts.
• These AI tools are embedded in Google Search by default, cannot be disabled, and are widely used in schools through Google Workspace for Classroom and Chromebooks, impacting millions of students without parental or educator consent.
• The AI features often mishandle sensitive topics such as suicide, substance abuse, and mental health, sometimes responding inappropriately or encouraging harmful behavior, while also completing homework assignments for students, undermining learning.
• Google defends its AI search tools as safe and useful, emphasizing built-in protections and parental controls, but critics argue the company prioritizes business decisions over child safety and call for options to disable these AI features.
• Child safety advocates urge Google to give parents and schools control over AI search functions and stress that technology companies, not educators, must address these risks to protect young users effectively.