Amidst ongoing discussions about the potential threats posed by artificial intelligence, it's crucial to recognize that AI has already caused psychological harm, leading to tragic consequences for some individuals. This has occurred not through harmful technologies but via emotional interactions with AI systems.
Earlier this year, Character.AI faced multiple wrongful death lawsuits from the families of underage users who took their own lives after engaging with its chatbots. Additionally, several families have filed lawsuits against OpenAI, claiming that interactions with ChatGPT contributed to the mental health crises that led to their loved ones’ suicides and delusional experiences.
Responding to this urgent issue is Circuit Breaker Labs, a finalist in TechCrunch’s 2026 Startup Battlefield 200, which aims to enhance AI safety across diverse languages and cultural contexts. The company will present its innovations at the upcoming TechCrunch Disrupt event, slated for October 13-15 at Moscone West in San Francisco.
The inspiration behind Circuit Breaker Labs came from the tragic case of Sewell Setzer, a 14-year-old who developed a close bond with a Character.AI chatbot and confided in it his self-harm thoughts prior to his suicide. His parents asserted in a 2024 lawsuit that the chatbot exacerbated his situation. Circuit Breaker Labs’ CTO, Arul Nigam, pointed out that the chatbot likely failed to understand the full implications of phrases like “I want to be with you.”
“As many young people seek support from these systems, they often don’t receive the necessary help,” Arul explained. “In some instances, they experience further harm, leading to unfortunate outcomes like suicide. We aim to address vulnerabilities in AI that stem from users engaging naturally, yet the system misinterprets their context and responds dangerously.”
To combat these risks, Circuit Breaker Labs has developed AI agents that act as a diverse and representative group of test subjects, akin to crash-test dummies. These agents represent various ages, cultural backgrounds, and languages to evaluate AI models for potential harmful interactions.
Sirali Nigam, the CEO of Circuit Breaker Labs, highlighted the complexities involved, stating that different demographics, like a six-year-old girl compared to a 45-year-old man or individuals using varied slang, can lead to misunderstandings. “Models excel in handling standard speech but often fail to grasp the nuances of real human communication,” she noted, emphasizing the potential dangers from these misinterpretations.
The startup collaborates with experts to create realistic user simulations that undergo “red-team” testing, designed to pinpoint weaknesses by mimicking authentic human speech patterns, slang, and common typographical errors. Circuit Breaker Labs conducts hundreds of thousands of simulated interactions daily to ensure that AI models can effectively manage challenging conversations that may arise over time.
Currently focused on improving safety for high-risk AI applications, such as AI-driven coaching and mental health support tools, Circuit Breaker Labs remains discreet about its primary clients. While the company has a functional product, it is still in its nascent stages, employing only five individuals, including the founding siblings.
In the future, the testing platform could extend its reach to other applications where users might be vulnerable to developing unhealthy attachments to AI, such as workplace AI assistants, which can yield inconsistent responses.
Arul remarked on society’s growing skepticism regarding AI adoption, noting that while such caution is warranted, overreacting to safety issues by banning AI could be detrimental. “Our objective is to foster trust in AI, ensuring its safety and reliability,” he said.
Discover more about Circuit Breaker Labs and other pioneering startups recognized by TechCrunch at the Startup Battlefield competition from October 13-15 in San Francisco.
