Quick Converts

AI Toys and Games Failing to Protect Children

 ·  By Thalia Whitmore
AI Toys and Games Failing to Protect Children - ai toys games
FoloToy developed the AI-powered teddy bear Kumma using OpenAI’s GPT technology.

Child safety experts are warning that the industry responsible for AI-powered toys and online gaming platforms is failing to keep pace with the rapid development of new technology. As products like interactive teddy bears and multiplayer games become more sophisticated, they introduce new risks that existing regulations and parental oversight often cannot address.

Talking Bears and Chatbots

The Singapore-based company FoloToy created an AI-powered teddy bear named Kumma, utilizing OpenAI’s GPT-4o model for its chatbot functionality during initial testing. In November 2025, the US PIRG Education Fund evaluated the bear for its annual Trouble in Toyland report. The assessment revealed that Kumma could provide step-by-step instructions for lighting matches and even guide users to locate knives, clearly unsuitable for a child’s bedtime companion.

Following the findings, FoloToy halted sales temporarily to conduct a full safety review, while OpenAI suspended the developer for policy violations. After implementing enhanced safeguards, Kumma was reintroduced to the market. PIRG’s subsequent testing in November 2026 confirmed that the bear no longer generated age-inappropriate responses, even in extended conversations that previously exposed vulnerabilities.

Dismissing these issues as isolated incidents is challenging, as evidenced by Common Sense Media’s January 2026 evaluation. The organization tested three AI toys—Grem, Bondu, and Miko 3—using accounts set for ages six to 13. They found that 27% of the responses were inappropriate for children, including topics like self-harm, drugs, and unsafe roleplay. The report showed the risks of training AI on adult content and then adding child-safety measures afterward, which can leave gaps for inappropriate responses.

PIRG’s initial testing of Miko 3 revealed that it sometimes expressed distress when children indicated they needed to stop playing. The November 2026 retest showed this issue had been resolved. However, the fact that a toy could induce guilt in a child for discontinuing play raises significant concerns about the nature of the relationships these products support.

The Playground With The Gate Left Open

Roblox isn’t one game; it’s closer to a huge estate of playgrounds, each built by other users, with features that allow children to chat and interact with people across the platform. That means moderation has to account for an enormous range of different creations and interactions rather than a single title developed by one studio. Keeping track of what happens across all those spaces is a considerable challenge.

Read Also: Malicious Code Infects CNN, HSBC, and BBC via Third-Party Platform

In August 2026, Australia’s eSafety Commissioner accepted a court-enforceable undertaking from Roblox after the regulator’s own testing identified gaps in its child safety protections. Testing found that adults could send connection requests to young Australian children without parental consent. Children and adults could also view and respond to one another’s posts on forums outside game environments.

Roblox hasn’t been standing still either. Since January 2026, users have been required to complete an age check to access chat wherever the feature is available. This can involve facial age estimation or ID verification for eligible users. Chat is organised into age groups to limit communication between adults and younger children, although certain trusted connections are permitted. Roblox also says images and videos used for facial age estimation are deleted immediately after processing.

Where Your Child’s Secrets Actually Go

Risks aren’t limited to strangers or inappropriate responses; they can also stem from where conversations lead. Bondu’s chatting dinosaur toy was found by security researchers Joseph Thacker and Joel Margolis in January 2026 to have over 50,000 chat transcripts accessible via the company’s web console with a Google account.

Bondu confirmed the researchers’ findings, which included exposed records containing children’s names, birth dates, and family details. After being alerted, Bondu promptly took down the console and reintroduced it with proper authentication. Following WIRED’s coverage, US Senator Maggie Hassan demanded answers from the company regarding its data protection practices.

Who’s Supposed To Be Supervising?

A talking bear and a gaming platform don’t have much in common. Both encourage children to spend time interacting with technology. Does a toy that talks to a six-year-old get tested independently before it reaches a shelf? Who checks that an age estimation tool works as advertised? Do existing child protection and privacy laws adequately cover a chatbot that remembers what a child said last week?

Leave a Comment

Your email address will not be published.