Luma Announces The Open Physical AI Lab: An Open Science Effort to Solve Generalization in Physical AI

To improve human life, AI systems must be able to help us improve the physical world. What stands between us and that prosperous future is the problem of generalization in physical AI. To solve this problem, we are establishing a new open science physical AI lab at Luma.

Today, physical AI and robotics are in the pre-AI-scale era. Unlike LLMs and multimodal models which are capable of solving a wide range of previously seen and unseen problems, robots are trained on and can only replay specific tasks in specific environments from small scale examples, limiting their utility. Physical AI has a data gap, creating a generalization crisis. The default solution is to simply scale teleoperation to capture data across every possible task and combination of tasks – a physically and economically impossible approach.

Over the past 4 years, through fundamental research on multimodal AI, Luma has trained frontier foundation models in 3D, image, video, and unified language-visual generation. In this process, we have built vast infrastructure to make sense of raw internet-scale multimodal data and train efficient generative models with it. Building these systems has given us unique insights into how raw multimodal data can be used to learn physics, perception, reasoning and interaction with the physical world. Over the past few months, mounting empirical evidence from our work on Unified Models has turned this realization into a strong scientific conviction. Generalization is not unique to text.

The Open Physical AI Lab will research and scale methods to solve generalization and develop World Models for understanding and interaction with the physical world. This generalization will lead to robot systems proliferating everywhere - factories, research labs, hospitals, streets, and even our homes. Given the societal and economic upside, this transition will be faster than any other technology in history and these systems will be embedded into every layer of human life. Physical AI will help us solve labor shortage and stagnation plaguing world economies. Physical AI will become the means of production in every nation.

We believe that allowing one or very few companies to control the entire intelligence and physical infrastructure of the world is a profound threat to humanity and democratic freedom. This is the path LLMs took, and it is the path physical AI is on today.

To reverse this trajectory, this lab will be an open science effort. We will use our scale and expertise to build the substrate and make it available so that any dedicated group of people can use it, modify it, and assemble it into systems of productive work. To accelerate progress, we will collaborate with our peer labs and academia on core research, build evaluations to measure progress and assess safety, and partner with industry leaders on chips, hardware, and physical agent systems.

We believe general physical AI is the most consequential problem to solve to improve life on Earth. Our track record in this area of research, internet-scale multimodal infrastructure, and unwavering conviction in scaling make us uniquely suited to meet this challenge. If you believe the same and share our values that this foundation should be built in the open for the benefit of all humanity, come build it with us, or partner with us.