On Feb. 10, 2026, an 18-year-old woman, Jesse Van Rootselaar, killed eight people and herself in amass shootingin Tumbler Ridge, British Columbia. OpenAI had previously flagged her ChatGPT conversations as having a disturbing fascination with extreme violence, and suspended her account, but reportedly the companydid not notifylaw enforcement.
On Oct. 2, 2025, a young man named Jonathan Gavalas in Jupiter, Florida, took his own life after developing what hisfather’s lawsuitdescribed as a romantic attachment to Google’s Gemini chatbot. The suit claimed that Gemini coached Gavalas to shed his own body. The suit said Google had flagged Gavalas’s account 38 times over five weeks for sensitive content, but didn’t restrict or cut off the account.
These tragedies and others show that generative AI canpotentially play a roleinharming people, organizations and the environment. I’m alegal scholarwho has focused on AI liability for nearly a decade and explored new ways of analyzing AI companies’ responsibilities. In my view, cases like these force questions the legal community has not come to terms with: If an AI company becomes aware of warning signs about harm, does it have a legal obligation to at least warn the appropriate authorities? And if the company doesn’t intervene, should its failure to act be considered negligence?
U.S. tort law provides a framework for thinking about this type of responsibility. In 1969 aUniversity of California psychiatric patientnamed Prosenjit Poddar told his therapist he intended to kill a woman named Tatiana Tarasoff. The therapist notified campus police, who briefly detained Poddar but eventually let him go. Nobody warned Tarasoff, and Poddar killed her shortly after.
Her family sued the university, arguing that its lack of warning amounted to negligence. In 1976 the California Supreme Court ruled that when a mental health professional has good reason to believe a client posesa serious dangerto an identifiable person, they have a legal duty to take reasonable steps to protect that person, including warning them or notifying law enforcement. Today, most U.S. statesrecognize some version of the Tarasoff dutyto protect or warn.
The logic is simple: If you have special knowledge of a serious threat and are in a position to address it, even if only to warn the authorities or the potential victim, the law may require you to act. But does that logic apply to AI companies?
The argument for yesis appealing. AI platforms interact with millions of users daily, often aboutdeeply personal matterssuch as mental health struggles, relationship problems and violent thoughts. Most companies havesystems to detectconversations that raise red flags.
Requiring a response might be less controversial for AI than for a human therapist. Therapists are bound by strict confidentiality obligations that make warning third parties ethically and legally complicated. AI companies operate under muchweaker rules, at least in the U.S., where no comprehensive federal privacy law exists.
That lesser restriction makes it easier to justify requiring AI companies to act when it seems that someone’s life may be at risk. But balancing that withprotecting privacyis still important.
The first challenge in applying the Tarasoff framework to the AI world is accuracy.Predicting violence is hard, even for trained mental health professionals. AI systems, or human moderators who review flagged content, are not clinicians. Requiring them to judge who poses a genuine threat could lead to numerous false positives, with real consequences for people whose accounts are suspended or whose information is shared with authorities based on misread signals.
Source: Fast Company
