The chatbot murder guide: AI’s struggle between privacy, reporting violence
0 / 55
English
中文
For months before carrying out Canada’s deadliest school shooting in almost four decades, Jesse Van Rootselaar was consulting an unlikely adviser on how to perpetrate mass murder.
（暂无翻译）
“In a hallway, indoors, or a crowded classroom, the 870 [Remington shotgun] is brutal … Close quarters is its playground,” OpenAI’s ChatGPT guided the 18-year-old, according to reports.
（暂无翻译）
On February 10, 2026, Van Rootselaar shot his mother and half-brother at home before going to a school in the small British Columbia town of Tumbler Ridge, where he killed another six people – including five students – before taking his own life.
（暂无翻译）
Earlier this week, on October 14, US authorities charged a Washington State man with allegedly conspiring with Van Rootselaar to plan the attack. But the mass shooting incident has raised questions that go beyond the traditional legal understanding of culpability.
（暂无翻译） （暂无翻译）
The British Columbia government in September filed a civil lawsuit against OpenAI and its CEO Sam Altman, accusing them of failing to prevent the incident and claiming they were aware of the conversations that turned deadly. Families of the victims also filed separate lawsuits. OpenAI’s leadership issued a public apology for their failure to notify authorities.
（暂无翻译） （暂无翻译） （暂无翻译）
The company has not responded to requests from Al Jazeera for its comments on the lawsuits.
（暂无翻译）
But as the case against OpenAI progresses, it has spurred a familiar debate: Where should a line be drawn between reporting potentially dangerous behaviour and protecting an individual’s privacy?
（暂无翻译）
“The question of whether companies should be required to turn such information over to law enforcement is extraordinarily difficult, particularly if they don’t already screen chats for indications of such types of threats,” United States-based civil liberties lawyer Jenin Younes told Al Jazeera.
（暂无翻译）
“Forcing companies to implement a surveillance system implicates very serious civil liberties concerns, and that’s probably a bridge too far for me,” Younes added.
（暂无翻译）
“On the other hand, AI gives individuals access to types and quantities of information they didn’t have previously that may facilitate the commission of mass casualty events, a concern it’s not easy to dismiss.” One of the biggest challenges facing AI companies is determining whether a user expressing violent thoughts actually intends to act on them.
（暂无翻译）
In August, Anthropic, the company behind AI chatbot Claude, flagged a user to San Francisco police after discovering conversations allegedly threatening its CEO, Dario Amodei.
（暂无翻译）
According to the San Francisco Standard, citing a police report, the user had claimed to possess an AR-15 gun and suggested he had Amodei “in his sights”.
（暂无翻译）
However, when contacted by the newspaper, the man said he was joking and had no intention of carrying out any form of violence.
（暂无翻译）
The person was neither charged nor detained, the Standard said. Fazl Barez, a senior researcher at the University of Oxford specialising in AI safety and governance, said current AI models cannot “reliably” determine whether an interaction with an AI chatbot “is purely hypothetical or is likely to lead to harmful behaviour”.
（暂无翻译） （暂无翻译）
“Suppose you are a creative writer working on a fictional story and want to explore how a murder plot might unfold.
（暂无翻译）
From the context, it may be clear that this is simply a fictional narrative. If the threshold for flagging potentially harmful content is too low, such a writer could be wrongly flagged,” he told Al Jazeera. “Similarly, consider a graduate student in biology or chemistry studying pathogens that, if misused, could have serious consequences.
（暂无翻译） （暂无翻译） （暂无翻译）
They might be trying to understand the risks associated with these pathogens and how to mitigate them. Such discussions may be necessary for legitimate research and education. Even when the subject matter is sensitive, they should not automatically be flagged without considering the broader context.” The researcher said identifying a credible threat may require examining a user’s interactions across multiple conversations, rather than relying on isolated statements. Barez said someone experiencing intrusive violent thoughts might turn to a chatbot for support, while another person planning an attack may reveal their intentions gradually across several conversations.
（暂无翻译） （暂无翻译） （暂无翻译） （暂无翻译）
“Distinguishing between the two requires careful consideration of context, intent and patterns of behaviour, rather than relying on individual statements alone,” he added.
（暂无翻译）
Chinmayi Sharma, a law professor at Fordham University, said the debate has parallels with the duty imposed on mental health professionals to warn identifiable third parties about credible threats.
（暂无翻译）
However, she said there was a key difference – therapists have professional training, knowledge of their patients and enforceable standards governing their conduct.
（暂无翻译）
AI companies do not necessarily have comparable safeguards.
（暂无翻译）
“My concern with AI companies is that right now there’s not a concerted effort to hire mental health professionals, sociologists, individuals that would have the unique expertise on what the indicators of threat are here,” Sharma told Al Jazeera. “Secondly, licensed professionals are accountable to concrete, legally enforceable standards in their discipline.
（暂无翻译） （暂无翻译）
We don’t have something like that for AI companies.” The privacy implications extend beyond whether AI companies can accurately identify threats.
（暂无翻译）
Millions of people use chatbots to discuss relationships, emotional distress, mental health concerns and deeply personal experiences. Civil liberties lawyer Younes said many users may not realise how vulnerable those conversations could be to disclosure.
（暂无翻译） （暂无翻译）
“They often reflect a person’s innermost thoughts, questions, and desires. The majority of people assume that these dialogues will never be viewed by another person,” Younes, president and legal director of the American-Arab Anti-Discrimination Committee, said.
（暂无翻译） （暂无翻译）
Younes warned that the consequences of greater monitoring could extend well beyond those planning violence.
（暂无翻译）
“If people fear their chats can be turned over to law enforcement simply because they express disturbing thoughts/violent fantasies etc, many will be dissuaded from using them,” she told Al Jazeera. Younes argued that forcing companies to establish systems for monitoring users and reporting conversations could raise serious constitutional concerns in the US, including protections against unreasonable searches.
（暂无翻译） （暂无翻译）
Meanwhile, Fordham’s Sharma also raised an issue about who could be disproportionately affected by heavy monitoring by AI chatbots.
（暂无翻译）
“The concern is always there about either intentional or unintentional targeting, because if you have models trained on what we consider dangerous speech, they’re almost always going to reflect biases,” she said.
（暂无翻译）
“Certain minorities’ speech is over‑flagged as a threat, while other comparable speech is not.
（暂无翻译）
“Even in a world in which everyone is benevolent, we should still be concerned about disparate outcomes,” she said, warning that machines could reproduce such disparities at scale.
（暂无翻译）
While AI companies may face moral pressure to prevent violence, their legal obligations are less clear, legal experts say. Bryan Choi, a professor of law at the University of Colorado Boulder, said US law recognises a duty to warn in certain circumstances, including when mental health professionals become aware of imminent threats or when a special relationship exists between parties.
（暂无翻译） （暂无翻译）
But those principles do not necessarily apply to AI companies.
（暂无翻译）
“Under present case law, it seems unlikely that a company such as OpenAI would be held to have such duties to warn,” Choi told Al Jazeera.
（暂无翻译）
“The chatbot is not a licensed therapist.
（暂无翻译）
There is no special relationship between the company and the victims.” Choi said existing negligence and product liability laws could potentially be applied to AI companies, but proving responsibility would present significant challenges.
（暂无翻译）
Fordham’s Sharma suggested that independent institutions could help establish reporting standards rather than leaving decisions entirely to private companies or requiring immediate disclosure to police. Such bodies she said could develop procedures for assessing threats and explore ways of sharing only relevant information while protecting users’ identities and other personal details. Meanwhile, Younes argued that users should, at a minimum, be clearly informed about whether their conversations are monitored and what circumstances could lead to information being shared with law enforcement.
（暂无翻译） （暂无翻译） （暂无翻译）
“This is almost certainly an issue that will be wrestled with over the coming years, and I’m not sure anyone knows the solution at this particular point in time,” she said.
（暂无翻译）
“Events that have not yet occurred will likely influence how these policies are shaped.”
（暂无翻译）
仅供学习使用，如有侵权请联系删除 · 查看原文