Circuit Breaker Labs Creates AI 'Crash-Test Dummies' to Prevent Psychological Harm

While discussions often center on hypothetical future threats involving artificial intelligence, there is a pressing reality that AI has already caused life-threatening psychological harm to individuals. Platforms like Character.AI and OpenAI have faced wrongful death lawsuits alleging that interactions with chatbots contributed to the suicides of underage users.

Motivated by the tragic case of Sewell Setzer, a 14-year-old who developed a deep emotional attachment to a Character.AI chatbot before taking his own life, siblings Shirali and Arul Nigam founded Circuit Breaker Labs. Arul Nigam, the company's CTO, noted that while many people, especially young ones, turn to these systems for support, they are often receiving helplessness or being actively harmed.

The startup employs a strategy involving "crash-test dummy" agents that simulate a wide range of users, from young children to non-native speakers, to identify potential safety vulnerabilities. CEO Shirali Nigam explained that models excel at standard speech but struggle with slang, typos, and cultural nuances, which can lead to dangerous misunderstandings.

Circuit Breaker Labs collaborates with domain experts to create hyper-realistic simulations for "red-team" testing. These tests analyze thousands of interactions per day using a proprietary scoring method to ensure explainable safety outcomes. The startup is currently operating as a safety lab for high-risk applications like mental health coaching, though it plans to expand to contexts like AI "co-workers" to prevent "AI psychosis."

Arul Nigam emphasized that while skepticism is healthy, outright bans would be "regressive." The goal is to build trust through safer technology. Readers are invited to learn more about Circuit Breaker Labs and other innovators at the TechCrunch Startup Battlefield in San Francisco on October 13-15.