Meta wants teens to have age-appropriate experiences with AI and to keep parents informed if we become aware that their teen may be in distress.
The social media messaging app announced additional protections to help make sure that, when it comes to sensitive conversations related to suicide and self-harm, it is responding in the right way.
This means acknowledging teens’ feelings while helping them get the offline support they need, whether that’s from a parent, a professional helpline, or the emergency services.
Alerting Parents if Their Teen Discusses Suicide or Self-Harm With Meta AI
When a teen suggests they may be thinking about suicide or self-harm, Meta AI already directs them to crisis helplines and encourages them to reach out to a parent or another trusted adult like a counselor.
Now, the messaging app will also proactively alert supervising parents if their teen’s Meta AI chat suggests they may be at risk, based on signals developed with experts. Meta AI will share expert resources to help parents approach these conversations with their teens.
The messaging app worked with parents and experts to understand which AI conversations warrant an alert – such as those where a teen makes a clear reference to hurting themselves, even if that reference is subtle. Meta AI then built a dedicated AI system to identify these conversations.
The messaging platform understands how distressing these alerts may be for a parent to receive. That’s why, as we continue to improve our detection, all chats flagged by our AI will be manually reviewed before an alert is sent.
If a teen’s intent is ambiguous, Meta AI will err on the side of caution and alert the parent. While that means the messaging platform may sometimes notify parents when there may not be a real cause for concern, the messaging platform feel this is the right starting point, and we’ll continue to monitor to help make sure it is in the right place.
“While I believe that teens have a right to privacy, I also believe parents need to be informed if their teen may be at risk of hurting themselves. That’s why I advocated for this approach and support Meta’s decision to notify parents when, after careful review, it determines that a conversation with Meta AI contains indications of possible suicide or self-harm that warrant an alert. I appreciate how Meta struck the right balance; protecting teen privacy while ensuring parents have the information they need to support their teen.” – Larry Magid, CEO and Co-Founder of ConnectSafely
These alerts are live now for parents using Instagram parental supervision in the US, UK, Australia and Canada, and will be available for supervising parents globally by the end of the year.
They build on the alerts we already send supervising parents if their teen repeatedly searches for suicide or self-harm terms on Instagram within a short space of time.
Alerting the Emergency Services to Possible Suicide Risks
Meta AI is building the ability to contact emergency services if someone’s conversation, whether an adult or a teen, with Meta AI suggests that they may be at imminent risk of taking their own life.
This builds on the work the platform already does across Facebook and Instagram: when it becomes aware of a post suggesting a credible risk of suicide, it will alert emergency services.
Last year, Meta AI made over 19,000 such referrals around the world, helping first responders perform wellness checks on people who may be at risk of suicide.
Working With Experts to Improve Meta AI’s Responses on Suicide and Self-Harm
In addition to its ongoing consultation with its AI Wellbeing Expert Council, Suicide and Self-Harm Advisory Group, and Youth Advisors, the messaging platform sought feedback from over 75 mental health clinicians on how Meta AI responds to teens’ prompts about suicide and self-harm.
These clinicians, who all specialize in teen mental health, reviewed AI responses to hundreds of prompts and provided feedback on whether they were appropriate for teens, what worked well, and how they could be improved.
Meta AI is using that feedback to further improve how Meta AI interacts with teens on these sensitive topics — for example by making sure that Meta AI acknowledges the teens’ feelings when directing them to support resources, and doesn’t shut the conversation down too abruptly.
“I was struck by the rigor of Meta’s clinical review process. It examined not only immediate responses to suicide and self-harm concerns, but also the broader conversational context, appropriate follow-up, and the varying levels of risk that can exist even within high-risk situations. This kind of expert-informed, scenario-based refinement is essential to making AI experiences safer for teens.” – Dr. Ji-yeon Lee, Licensed Psychologist (USA) and Professor of Counseling Psychology at Hankuk University of Foreign Studies (S. Korea)
Launching the Stricter Limited Content Setting for Meta AI Chats
Teen Accounts are automatically placed into a 13+ content setting, which also applies to conversations with Meta AI.
In this default 13+ setting, Meta AI is designed to give age-appropriate responses and not comply with sensitive prompts. For example, Meta AI is trained not to engage in sexual or romantic conversations with teens or provide recipes for alcoholic drinks, directing teens to safer topics instead.
In October, the messaging platform announced Limited Content as a stricter content setting for parents who prefer a more restrictive experience for their teens on Instagram, and now that stricter setting will apply to AI experiences too.
This means that when parents opt their teens into the Limited Content setting on Instagram, it will also further limit the kinds of conversations teens can have with Meta AI. In the Limited Content setting, Meta AI will decline to respond to a broader range of prompts, further reducing the chance of potentially inappropriate conversations.
Don’t miss important articles during the week. Subscribe to Techbuild weekly digest for updates
