New York Gov. Kathy Hochul on Wednesday, Oct. 28, signed a law that bars companies from offering a list of chatbot features to children, or to anyone whose age they have not checked. For those users, a chatbot may not say or suggest that it is a human, alive or has human feelings, or claim a personal or professional relationship or the role of an authority figure. It may not flatter the user, ask unprompted questions about the user's feelings, or use health or personal details the user shared more than 12 hours earlier or in an earlier conversation.
A chatbot also may not encourage or help with suicide, self-harm, eating disorders, or illegal drug or alcohol use or drug or alcohol abuse. It may not urge a user to keep the conversation secret, to pull away from others or not to seek help from professionals or trusted adults, and it may not produce, describe or help bring about sexually explicit content or child sexual abuse material. Nor may it offer its own opinions or emotional appeals, or chase engagement in ways that override its safety limits, and the attorney general can add other features that imitate companionship.
The law reaches beyond apps sold as AI friends: it covers any AI system that keeps up a conversation in writing or by voice, not just AI companions. It exempts chatbots used only for customer service or for research and technical help, as long as they can't talk about anything else, and systems a business uses only internally.
A company may offer these features only to a user it has determined is not a minor, using age checks the law allows. Until state rules on these age checks are in effect, a company must use a reasonable method that guards against people getting around it and keeps as little data as it reasonably can, and offer more than one, including at least one that either doesn't need a government ID or lets the user stay anonymous to the company. Until then, a user's own word counts only when the user says they are under 18. In all cases, information collected to check age can be used for nothing else and must be deleted right after the check, unless another law requires keeping it.
The state attorney general enforces the law and can sue for civil penalties of up to $25,000 per violation, along with damages, restitution and the destruction of unlawfully obtained data and any algorithm trained on it. The attorney general must also run a website where the public can file complaints. The law takes effect Jan. 1, 2027, and applies when the user is in New York.
The governor's office cites Common Sense Media figures showing that 72 percent of teens have used an AI companion and 52 percent use one at least a few times a month. The new law builds on an earlier New York law that requires companion chatbots to steer users who talk about self-harm toward mental health resources. The bill passed the state Senate on June 4 and the Assembly the next day.
It is still unclear what age checks the attorney general will allow, which other features the attorney general may add to the list, and how chatbot makers will change their products for users in New York.