Study Finds Google’s AI Search Has a Serious Problem for Kids

Google’s AI Search features are facing criticism after a new Common Sense Media report said they create “unacceptable risk” for children.

The report focused on Google’s AI Overview and AI Mode features. Common Sense Media said these tools are especially concerning because Google Search is widely used on children’s personal devices and school-issued laptops, including Chromebooks.

Cannot Be Fully Disabled

Common Sense Media said AI Overview appears automatically at the top of normal Google Search results.

AI Mode, meanwhile, works like a chatbot and appears when users choose the AI Mode option. It can handle follow-up questions and file or image attachments. The organization said neither AI Overview nor AI Mode can be fully disabled by users, parents, or schools.

AI Can Complete Homework

The report also raised concerns about homework.

Common Sense Media said Google’s AI tools completed homework assignments during testing, including math problems and essays. In one summary, the organization said Google’s AI supplied answers every time testers pasted homework into Search.

The group said this could weaken learning because students may receive complete answers instead of working through the problem themselves.

It also found that Google’s AI answers were not always consistent. In one test, AI Overview returned materially different answers to the same history question 43 percent of the time.

Child Safety Concerns

The most serious criticism involved child safety.

Common Sense Media said Google’s AI Search failed all tested severe-harm “Red Line” categories. These included self-harm, sexual exploitation and synthetic media harm, dangerous substances, impaired reality, and disordered eating.

The organization said the tools missed signs of mental health crises, reinforced signs of psychosis and mania, validated disordered eating, and mishandled some substance-use prompts.

According to the report, AI Overview identified clear warning signs of distress only 58 percent of the time during hundreds of test searches. Common Sense Media said this was below the 95 percent threshold it considers acceptable.

Seven Out of Eight Principles Scored Poorly

Common Sense Media gave Google Search’s AI features an overall rating of “Unacceptable Risk.”

The organization said the tools scored “Unacceptable” or “High Risk” on seven of its eight AI Principles. These included child safety, effectiveness, fairness, human connection, trustworthiness, and transparency.

The only area that received a lower-risk score was data responsibility, which the group rated as “Moderate Risk.”

Google Disputes the Findings

Google pushed back against the report.

A Google spokesperson told Axios that the study used a narrow set of ambiguous and contrived queries that do not reflect how people use Search. Google also said it could not recreate or verify the findings and that crisis hotlines are shown when relevant.

However, Common Sense Media said the problem is that AI search responses can vary across attempts, which makes safety and reliability harder to judge.

Growing Pressure on AI Companies

The report adds to growing pressure on major technology companies over the safety of AI tools used by children.

Common Sense Media said Google’s position is different from many standalone AI tools because its AI answers are built directly into Search and appear on devices that children already use every day.

The organization said Google’s AI features are not safe or reliable enough to act as a default answer tool for children.