Evidently, OpenAI is also stepping up its own efforts in robotics. Last week, Caitlin Kalinowski, who previously led development of virtual and augmented reality headsets at Meta, advertised on LinkedIn which would join OpenAI to work on hardware, including robotics.
Lachy Groom, a friend of OpenAI CEO Sam Altman and an investor and co-founder of Physical Intelligence, joins the team in the conference room to discuss the business aspect of the plan. The boyfriend is wearing an expensive looking hoodie and looks remarkably young. It highlights that Physical Intelligence has a long way to go to achieve a breakthrough in robot learning. “I just got on a call with Kushner,” he says, referring to Joshua Kushner, founder and managing partner of Thrive Capital, who led the startup’s initial investment round. He is also, of course, the brother of Donald Trump’s son-in-law, Jared Kushner.
Some other companies are now pursuing the same type of breakthrough. One called Skild, founded by roboticists at Carnegie Mellon University, raised $300 million in July. “Just as OpenAI built ChatGPT for language, we’re building a general-purpose brain for robots,” he says Deepak PathakSkild executive director and assistant professor at CMU.
Not everyone is sure that this can be achieved in the same way that OpenAI cracked the code of the AI language.
There is simply no Internet-scale repository of robot actions similar to the text and image data available for LLM training. Still, achieving a breakthrough in physical intelligence could require exponentially more data.
“Words in sequence are, dimensionally speaking, a small toy compared to all the movement and activity of objects in the physical world,” says Illah Nourbakhsh, a roboticist at CMU who is not involved in Skild. “The degrees of freedom we have in the physical world are much more than the letters of the alphabet.”
Ken Goldberg, a UC Berkeley scholar who works on the application of AI to robots, warns that the excitement building around the idea of a revolution of data-driven robots, as well as humanoids, is reaching proportions. exaggerated “To achieve expected performance levels, we will need ‘good old-fashioned engineering’, modularity, algorithms and metrics,” he says.
Russ Tedrakecomputer scientist at the Massachusetts Institute of Technology and vice president of robotics research at the Toyota Research Institute, says the success of LLMs has caused many roboticists, including himself, to reconsider their research priorities and focus on finding ways to carry out robotic learning more broadly. ambitious scale. But he admits formidable challenges remain.