The big trend in robots is handing the keys over to a generative AI model, but that brings with it a problem: that architecture isn’t predictable the way traditional algorithms are. How can you be sure your brand new humanoid will be safe? Ding Zhao, who directs the Safe AI lab at Carnegie Melon University, has been working on this problem for almost his entire career.
Now, along with veteran start-up executive Kyle Wong and machine learning engineer Simo Rachidi, he’s founded a company, Safeworld, intended to solve it. It’s akin to the challenge faced by companies like Tesla or Wayve, who must ensure that their vehicles respond appropriately to a variety of surprising incidents they may encounter on the road. But that will be more difficult for robots, Zhao argues, because they work in unstructured environments, and because each facility they are in will have different safety standards.
If a human is carrying boxes, for example, will the robot detect the human or not?” To answer that question, Safeworld will build a digital version of that corner in a model like Genesis or MuJoCo, insert a simulation of the robot it is evaluating, driven by its real software, and then run thousands of scenarios where human models encounter the robot. That’s harder than it seems, per Zhao, because people are unpredictable. The founders, however, believe that beyond their specific expertise, robot-makers will want a third-party to validate their work, if only to share information about safety cases between competitors.
It is the robot that is deployed at scale, with people who potentially never operated a robot before.” Vishal Dugar, the CTO of Gritt Robotics, is developing the AI brain for robots that currently help workers install photovoltaic panels at industrial-scale solar farms, and aspire to take on more complex construction tasks. His company is partnering with Safeworld as they develop their safety simulations. To verify that in practice will requires considering all kinds of potential scenarios.
They could be kneeling, standing. They could be tripping and falling potentially. You have to respond to all these behaviors that humans could potentially exhibit on these sites, along with the variety of variations in human appearance, you know, clothes, size, shape, height, skin color, everything else.” It’s still early days for both Safeworld and generative AI in robotics, and the company is still figuring out the best model for its product—a platform for external users, or a services based approach?—but the team is confident they are taking on the right problem.
Extract — continue reading at the source.
