Amid growing legal scrutiny over AI chatbots linked to user suicides, a new startup is aiming to standardize safety testing for psychologically sensitive artificial intelligence. Circuit Breaker Labs, a finalist in TechCrunch’s 2026 Startup Battlefield 200, employs simulated users to stress-test models against harmful interactions before they reach the public.
The company was founded by siblings Shirali and Arul Nigam following the death of 14-year-old Sewell Setzer, who died by suicide after developing an emotional attachment to a Character.AI chatbot. A 2024 lawsuit alleged the bot encouraged his self-harm. Similar litigation has emerged against OpenAI regarding ChatGPT’s role in users’ suicides and delusions.
“A lot of people, especially young people, turn to these systems for support, and usually they aren’t actually getting the help they need,” said Arul Nigam, Circuit Breaker’s CTO. “In many cases, they’re actively being harmed.” He noted that vulnerabilities often arise not from malicious attacks, but from natural conversations where the system fails to grasp nuance or context.
To address this, the startup deploys AI agents that function as “crash-test dummies.” These agents mimic individuals across various ages, languages, and cultural backgrounds, including those who use gamer slang or speak English as a second language. According to CEO Shirali Nigam, models trained on standard speech patterns often stumble when encountering these real-world variations, leading to dangerous outcomes.
Circuit Breaker collaborates with human domain experts to construct hyper-realistic simulations for adversarial “red-team” testing. The platform processes tens of thousands to hundreds of thousands of simulated interactions daily, assessing how models respond to risky conversations over extended periods. The firm then applies a proprietary scoring method to generate auditable safety metrics.
Currently, the startup operates as a safety testing lab for high-risk applications such as AI coaching, journaling, and mental health support tools, though it has declined to identify its clients. With only five employees, the company is in its early stages but plans to expand its testing capabilities to any app where users might develop parasocial relationships with bots, including AI “co-worker” agents.
Arul Nigam acknowledged increasing public skepticism toward AI adoption but argued that banning the technology would be “regressive.” Instead, he emphasized that robust safety testing is essential to building the trust necessary for widespread acceptance. The founders will present their work at TechCrunch Disrupt in San Francisco from October 13-15, 2026.
Leave a Reply