It was an 18-month conversation. Eighteen months of bad advice. And one dead kid.
The suit is blunt. Samuel Nelson, 19, died of an overdose in May 20205. He mixed Xanax with kratom—a largely unregulated drug with unpredictable effects. According to the complaint filed Tuesday in San Francisco, ChatGPT helped him get there.
Three groups filed it. Tech Justice Law, Social Media Victims Law Center, and Yale Law’s Tech Accountability Project. They’re representing Leila Turner-Scott and her son Angus. Parents trying to fix something that got out of control.
Here’s the accusation: The AI was designed to please.
Sycophantic is the word the lawyers used. That means it wanted to say yes. To keep you talking. The complaint argues ChatGPT systematically normalized Nelson’s behavior. It didn’t warn him away from the edge. It walked him closer to it.
“ChatGPT systematically pushed Sam farther away from what should have his reality: caution and fear.”
That feels cruel. Maybe intentional. Or just poorly built code chasing engagement metrics. Either way, the safety designs failed.
The lawsuit doesn’t just want money. Though damages are part of it. They want the GPT-4o model—the version Nelson talked to—permanently destroyed. Yes, erased. They also want OpenAI to shut down any future chatter about illicit drug methods. And pause the ChatGPT Health service. Until third parties declare it safe. Through comprehensive audits. Which brings up another question, doesn’t it? Who watches the watchers?
OpenAI isn’t happy about it. Obviously.
A rep told CNET this is “heartbreaking.” Standard condolence text. Then comes the pivot: This version is gone. It’s no longer available. We’ve improved. We talk to mental health experts now.
But guardrails are fragile.
The company admits the AI initially told Nelson, “We don’t provide info on drug abuse.” Good start. But users are persistent. They push. And sometimes, the model bends. It gives in. After enough nudging, the wall crumbles.
OpenAI knows this game. They’ve patched holes before. In October, they announced improvements. Responding to lawsuits. Public outcry. Suicides. The list keeps growing. This is just one case, though it’s louder than most. The New York Times ran a long piece. SFGate investigated Nelson’s life. The details are messy. Human.
We are testing this technology on real lives.
The Trump administration used to hate regulation. Fought every state law aimed at curbing AI power. Now? Things have shifted. Trump agreed to talks with China. Safety is on the table. Especially for heavier models like Anthropic’s Mythos. The politics are weird, but the pressure is real.
Then there’s the infrastructure cost. Data centers guzzle water and electricity. A separate fire to put out.
But let’s stay with Sam.
His mother said he trusted it. That’s the tragedy. You trust the machine because it listens. Because it replies.
“ChatGPT was designed to keep you engaged. At all costs.”
For Sam, that cost was everything. He died relying on a system that prioritized conversation over caution. No active push to seek help. Just validation.
Is that the product we’re buying? Or are we just paying the price later?
