Meta says it will alert parents if teen AI chats suggest suicide risk
Meta is introducing new safety measures it says are designed to better protect teens using its AI tools, including a feature that it says will alerts parents if the company believes a teen's conversation with Meta AI suggests they may be considering suicide or self-harm.
The new "Parent Alerts" feature will begin rolling out Thursday to parents using Instagram's parental supervision tools in the United States, the United Kingdom, Australia and Canada. Meta said the feature will expand globally by the end of the year.
If an alert is sent, Meta said parents will also receive expert-backed resources to help them navigate conversations about suicide and self-harm with their teens.
According to Meta, the company developed a dedicated AI system to identify high-risk conversations after consulting parents and mental health experts on what types of interactions should warrant an alert. As an additional safeguard, every conversation flagged by AI will be reviewed by a human before a notification is sent.
Meta said that if a teen's intent is unclear, the company will "err on the side of caution."
Under current guidelines, 13 is the minimum age to use Meta's social media products, Facebook and Instagram. The company says users under the age of 18 are automatically placed into a 13+ "teen account" content setting, which it says "also applies to conversations with Meta AI."
The company also announced it is building the ability to contact first responders if a conversation with Meta AI suggests someone is at imminent risk of suicide. Meta said the effort builds on the more than 19,000 emergency referrals it made globally across Facebook and Instagram last year.
To improve how Meta AI responds to prompts related to suicide and self-injury, the company said it sought feedback from clinicians who specialize in teen mental health. Those updates are in addition to ongoing guidance from members of Meta's AI Wellbeing Expert Council, Suicide and Self-Harm Advisory Group and Youth Advisors.
Meta also said it has expanded its "Limited Content" setting to AI experiences. If parents enable the stricter content setting for their teen through Instagram's parental supervision tools, the teen's interactions with Meta AI will also be more limited.
Meta's announcement comes as other AI-focused companies have also taken steps they say are aimed at helping to protect teen users.
OpenAI announced parental controls with notifications for its teen users last year and uses an age prediction model on its ChatGPT product to apply a teen experience for users under 18 that prohibits ads and includes “extra safeguards for sensitive or age-inappropriate content,” according to the company.
Currently, ChatGPT is accessible for children ages 13 and older, and children between the ages of 13 to 18 must obtain parental consent before using ChatGPT, according to OpenAI.
Anthropic says its consumer product, Claude, is only available for users over the age of 18, stating in May that the product has, "safety systems in place to detect if people under 18 may be using Claude and we’ll disable accounts based on indicators of minor activity."
Meta's announcement about new safety measures for teens also comes as the company has faced increased scrutiny over its AI products.
Last week, after facing backlash from Hollywood, the company disabled its "Muse Image" feature, which allowed users to alter publicly posted images with AI. In a blog post, Meta said, "We've heard the feedback that this feature missed the mark, so it's no longer available."
The latest safety update also follows a lawsuit filed Tuesday by a group of 26 Meta employees who allege the company used AI systems to decide who to layoff; the suit alleges that workers on medical, parental and family leave were disproportionately targeted during a round of layoffs that began in May and affected about 10% of its workforce.
Meta denied the allegations, saying in a statement that the claims "lack merit and are not based on facts. Workforce management and organizational decisions were and are made by people, not AI."
If you or someone you know are experiencing suicidal, substance use or other mental health crises please call or text 988. You will reach a trained crisis counselor for free, 24 hours a day, seven days a week. You can also go to 988lifeline.org or dial the current toll free number 800-273-8255 [TALK].
