A company founded by a Carnegie Mellon professor is betting that robots powered by generative AI can be trusted — if its software can simulate every way a person might trip, fall or collide with one.
Safeworld emerged from stealth today with a $12 million seed round, led by Shine Capital and a16z Speedrun, with additional funding from Box Group, Carnegie Mellon University Endowment, Innovation Endeavors and SV Angel. The company’s pitch is simple: before a robot gets deployed, Safeworld runs it through a digital twin of the factory, hospital or warehouse where it will live, populated with realistic human models. The idea is to catch failures that traditional testing misses.
Dr. Ding Zhao, who directs the Safe AI lab at Carnegie Mellon, founded Safeworld with veteran start-up executive Kyle Wong and machine learning engineer Simo Rachidi. Zhao has spent nearly his entire career working on AI safety, and he sees the current wave of robots as a fresh hazard.
“The safety challenge that we’re talking about is a combination of, one, really advanced generative AI probabilistic evals — how do you underwrite the risk of a probabilistic system?” Zhao says. “The second part that’s really hard is the trust part, and you need both to deploy a robot.”
Why Generative AI Makes Robots Harder to Predict
Today’s robots are built on generative AI models that make decisions in real time. That flexibility is the point — but it also means the machine’s behavior is not fully predictable.
Zhao compares the problem to autonomous driving. Companies like Tesla and Wayve face similar challenges: how to test a vehicle’s response to unexpected events on the road. But robots operate in unstructured environments, and each site has its own safety standards.
“One of the most common areas is if there is a blind corner in this particular factory,” Wong said. “What is the speed or what is the stopping distance that you need to make sure that this robot will not collide with a particular human? If a human is carrying boxes, for example, will the robot detect the human or not?”
Safeworld’s approach is to build a digital version of that corner, populate it with a simulation of the robot, and run thousands of scenarios where human models encounter the robot. The human models are designed to mimic unpredictable behavior, including tripping and falling.
“Tripping and falling is also a good example of something that we do a lot of testing with the simulation,” Wong said. “Otherwise, you would have to go and trip and fall for the robot, which is like a hard thing to be doing all the time.”
The Partner Building Solar Farm Robots
Gritt Robotics is one of Safeworld’s partners. Its CTO, Vishal Dugar, is developing the AI brain for robots that currently help workers install photovoltaic panels at industrial-scale solar farms, with plans to take on more complex construction tasks.
Dugar’s robots operate alongside human workers, and ensuring that its robotic arm doesn’t hit them is top of mind. He says formal verification — proving a system is safe with math and equations — is nearly impossible for these systems.
“It’s very hard to formally prove it by doing some math, writing some equations, and saying yeah, the system is verified to be safe,” Dugar says. “It necessarily has to be done empirically.”
The human models in Safeworld’s simulations must account for a wide range of behaviors and appearances. Dugar lists the possibilities:
- Kneeling
- Standing
- Tripping and falling
- Crouching
- Running
The company must also consider human variation in clothing, size, shape, height and skin color, Dugar says.
What the Money Will Fund
The $12 million round will fund further development of Safeworld’s platform and its deployment strategy. The company is still deciding whether to offer a platform for external users or a services-based approach.
Jonathan Lai, a partner at a16z Speedrun, framed the timing as urgent.
“The time to build an industry safety standard is now while robots are being designed and deployed,” Lai told TechCrunch. “By the time you have robots in households colliding with kids and causing safety incidents, that’s way too late.”
Zhao is confident his team is solving the right problem. He predicts Safeworld will be the first profitable company in the field.
“We’ll probably be the first profitable company in this field,” Zhao says. “Because if anyone wants to deploy, they need to pay us to handle the situation.”
The Trust Problem
The other half of Safeworld’s mission is trust. A robot’s software can pass every test and still fail the test that matters: does the person standing near it feel safe?
Zhao describes the challenge as two parts. The first is technical: how to evaluate a probabilistic system that learns from data. The second is building confidence that the evaluation was thorough.
“A lot of people are underestimating one how hard some of these edge cases are going to be to solve,” Zhao said. “It is not the robot in the vacuum, in the demo, that we are worried about. It is the robot that is deployed at scale, with people who potentially never operated a robot before.”
The founders believe robot-makers will want a third-party validator, even competitors, to share information about safety cases. That could create a market for Safeworld’s services.
What Comes Next
Safeworld’s technology is still evolving. The company is building its digital twins using physics engines like Genesis and MuJoCo, which allow the simulations to model realistic human movement. The goal is to make the virtual world feel as close to the real one as possible.
The company’s path to profitability depends on robot-makers paying for independent safety evaluations. Zhao’s prediction of first profitability is bold, but it rests on a bet that the industry will treat safety as a service rather than a cost of doing business.
For now, the company has the capital, the partners and the ambition. Whether it can convince people that a robot powered by generative AI won’t hurt them remains to be seen.
Source material: “Can Safeworld convince people that gen AI robots won’t hurt them?,” TechCrunch.
Get the Notebook.
The day's best stories and every fresh verdict, in plain English, in your inbox by seven. One email a day, no more.