The artificial intelligence bubble has burst and winter is back, according to technology experts and professional skeptics following the market. Feifei Li doesn’t think so. In fact, Lee, who has earned the nickname “Godmother of AI,” is betting against it. She is on part-time leave from Stanford University to co-found a company called World Labs. While current generative AI is language-based, she sees a frontier where systems build complete worlds with physics, logic, and the rich details of physical reality. It’s an ambitious goal, and World Labs is on a fast track to raising funding, despite shady overlords who say advances in AI have reached a severe plateau. It will probably take a year for the startup to get a product out there, and it’s not entirely clear when or how well it will work if it does, but investors have raised $230 million. , reportedly valuing the startup at $1 billion.
About 10 years ago, Lee helped transform AI by creating ImageNet, a custom database of digital images that allowed neural networks to become significantly smarter. She believes that if AI is to create a real world (be it a realistic simulation or a fully imagined universe), today’s deep learning models will need similar enhancements. I feel that. A future George R.R. Martin might compose his dream worlds as prompts rather than prose, rendering them and walking through them. “The physical world of the computer is seen through the camera, and behind the camera is the computer’s brain,” Lee says. “To turn that vision into inference, generation, and ultimately interaction, we need to understand the physical structure, the physical dynamics of the physical world. And that technology is called spatial intelligence.” World Labs calls itself a spatial intelligence company, and its fate will determine whether those words become revolution or death.
Lee has been obsessed with spatial intelligence for years. While everyone was buzzing about ChatGPT, she and former student Justin Johnson were excitedly talking on the phone about the next version of AI. “The next 10 years will be about creating new content that takes computer vision, deep learning, and AI out of the internet world and embeds it in space and time,” says Johnson, now an assistant professor at the University of Michigan. .
Lee decided to start the company in early 2023 after a dinner with virtual networking pioneer Martin Casado, now a partner at Andreessen Horowitz. It’s a VC firm notorious for embracing AI like it’s a savior. Casado sees AI following the same path as computer games, which started with text, moved to 2D graphics, and now feature dazzling 3D images. Spatial intelligence drives change. Ultimately, he says, “you can throw your favorite book into the model and literally step inside it and watch it unfold in real time in an immersive way.” The first step to achieving that, Casado and Lee agreed, is to move from a large-scale language model to a large-scale world model.
Mr. Lee began building a team with Mr. Johnson as a co-founder. Casado suggested two more people. One was Christoph Lassner, who worked at Amazon, Meta’s Reality Labs, and Epic Games. He is the inventor of Pulsar, a rendering scheme that led to the famous technique called 3D Gaussian splatting. While this sounds like an indie band from MIT’s toga party, it’s actually a way to compose scenes rather than one-off objects. Another of Casado’s suggestions was Ben Mildenhall, who developed a powerful technology called NeRF (Neural Radiation Field), which converts 2D pixel images into 3D graphics. “We brought real-world objects into VR and made them look completely real,” he says. He left his position as a senior researcher at Google to join Lee’s team.
One obvious goal of large-scale world models would be to imbue robots with a sense of the world. It’s certainly in World Labs’ plans, but not for a while. The first step is to build a model with a deep understanding of three-dimensionality, physicality, and the concepts of space and time. Next comes the phase where the model supports augmented reality. After that, the company can work on robotics. If this vision becomes a reality, large-scale global models will improve self-driving cars, automated factories, and even humanoid robots.
