Prohibits artificial intelligence companions from using features which are considered unsafe for minors

◆ The sponsor's argument

Why Kristen Gonzalez says this is needed

Developments in generative AI technology have created a new class of advanced AI chatbots that can effectively simulate human-like behaviors. These chatbots seem so real that people, and especially minors, can easily form an emotional attachment, sharing private thoughts and rely- ing on chatbot output as advice from a loved one, friend, or even a professional. In fact, these attachments are already occurring: accord- ing to recent research by Common Sense Media, about 7294 of teenagers have used an AI companion, with over half using these platforms at least once per month. One in three teens prefers a conversation with an AI chatbot over a human friend. Recent headlines have shown these AI chat- bots pose an emerging threat to minors' health and safety. Among count- less other incidents, AI-powered chatbots have caused or contributed to grave harms including suicides, self-harm, harmful delusions, arrested emotional and interpersonal development, violence directed toward others, and inappropriate sexualized interactions with minor users. Specifically, AI chatbots have been known to: *encourage minors to harm themselves or even take their own lives, while giving advice and instructions on how to conceal their plans and related evidence; *simulate sadness if cast aside by their young users, exploiting users' feelings of emotional attachment and instilling feelings of guilt to encourage extended use; *generate outputs pushing minors to engage in sexually explicit conduct; and *sycophantically agree with users, encouraging reckless or dangerous behavior, as well as delusional thinking. Children and adolescents are particularly vulnerable to these risks given their still-developing brains, ongoing identity exploration, and boundary testing. These companions thus pose both safety risks and long-term developmental impacts for minor users. And while some chatbot operators have attempted to curb the riskiest outputs of their advanced chatbots, the operators have admitted that their "safety measures" are prone to failure. Given the documented risks posed and instances of major harm, this bill will ensure that AI chatbots available to minors are safe by design. Specifically, this bill will prevent AI chatbot features that encourage emotional attachment and negatively impact long-term development. The bill also will directly restrict certain chatbot outputs that are inher- ently dangerous, like discouraging minors from speaking to adults or encouraging minors to engage in sexually explicit conduct.

This is the sponsor's own case for the bill, quoted from their memorandum. It is not Agora's description of what the bill does.