机器人初创公司如何破解数据短缺难题

It’s day four of Robot Week! You can click here to get 25% off an annual subscription.

On May 28, the startup Shift announced that it would clean any New York City apartment for free. In a launch video, cheerful young men scrubbed toilets, vacuumed floors, and wiped down counters.

The catch? Cleaners wore baseball caps with cameras mounted under the brims. The company planned to record the workers’ actions and sell the data to robotics companies.

While the whole deal might have been a gimmick — the scheduling website notes in an FAQ that the offer is only available for a “limited time” — it’s still a perfect encapsulation of one of the most important trends in robotics today.

Early LLMs were famously trained to “predict the next word” across billions of tokens of text scraped from the Internet. Most roboticists expect we’ll need something similar to train general-purpose robots: an Internet-scale database of everyday tasks that robots can learn from.

But right now, humanity doesn’t have anything like that. The largest openly available dataset of robots performing tasks, ABC-130K, only has 3,500 hours of task demonstrations.

Over the last few months, I’ve talked to dozens of founders, engineers, and robotics researchers about the need for demonstration data and the ways people are trying to get more of it. I visited a robotics lab at the University of Pennsylvania to try my hand at collecting robot data. During my spring trip to China, I watched men wearing virtual reality headsets puppet humanoid robots to open fridges, sweep trash, and move pillows around.

When I attended the Actuate conference in San Francisco in August, I was surprised by how many people there were working at data-collections startups.

A plethora of startups like Shift are trying to solve the data shortage by recording the actions of humans and converting the videos into training data for robots. Other companies are hiring humans to directly operate robots in labs, factories, and even people’s homes. Still others are hiring humans to perform everyday tasks while wearing gloves or exoskeletons that force the human to move in a robot-like way and capture rich data on the worker’s actions. Some large data-collection companies like Scale AI are experimenting with all of these strategies.

The ultimate goal is to develop robots that are good enough to operate (mostly) autonomously in the real world. Once that happens, robots could generate additional training data while doing useful work. This could lead to a flywheel where the companies with the best robots are able to generate the most high-quality data, allowing them to improve their robots even more.

But Deepak Pathak, the CEO of robotics startup Skild, told me that there’s a “chicken-and-egg problem” here. In order to generate high-quality data from deployments, robots need to be able to do some amount of useful work. And getting there will probably take a fair amount of data. So companies first need to figure out a scalable way to get robotics data without deploying robots commercially. The first company to figure this out could have a big advantage.

Getting the computer to make the data for you

Before we explore the strategies companies use to generate real-world training data, it’s worth asking why we need real-world data at all. Nearly a decade ago, Google DeepMind trained an AI to play Go entirely by self-play. After playing millions of games against itself, the model became better at Go than the top humans.

Could we do something similar for robots? Instead of training physical robots in the real world, maybe we could have virtual robots “teach themselves” to perform tasks through trial and error in a simulated environment. This approach actually does work for certain robotics tasks.

Clip from Figure’s blog post “Natural Humanoid Walk Using Reinforcement Learning” illustrating tens of robots walking in simulation with different parameters.

In March 2025, the humanoid robotics company Figure posted a high-level description of how it trains its robots to walk. Figure programmed a digital twin of its Figure 02 robot in a physics simulator and had that virtual robot try to walk over and over for millions of attempts. Each time, the robot received programmatic feedback — in a process called reinforcement learning — until the robot could walk in simulation. When Figure installed the resulting model on a physical robot, it could walk in the real world too.

When this process works, it’s the ideal way to train a robot.

It can be very fast: Figure said it was able to obtain “years of simulated demonstrations in a few hours.” This method can also result in a very robust model: after training for the equivalent of 1,000 years in a simulator, the foundation model company Skild produced a model that could control a quadruped robot even when engineers sawed its legs in half.

Basically every company today making a humanoid robot uses reinforcement learning in a simulated environment to teach it how to walk.1 Unfortunately, while this approach works well for locomotion tasks like walking and dancing, it doesn’t work as well for manipulation tasks, which involve more complex interactions with the environment.

Imagine trying to train a robot to hammer a nail. If a robot starts out acting entirely at random, it might go through millions of iterations without a single success. Reinforcement learning works by “rewarding” the model when it succeeds, but if the model never succeeds, there’s nothing to reinforce.

Developers can help the virtual robot by giving it fine-grained feedback that acts as a trail of breadcrumbs along the path to success. The robot might earn points for touching the hammer, more points for picking it up, still more for touching the nail with the hammer, and so forth. But this technique, known as “reward shaping,” is labor-intensive, doesn’t transfer well between tasks, and still may not produce good results.

In 2017, when prominent researchers tried to use reinforcement learning to teach a robot to hammer a nail in simulation, they couldn’t get it to work with just a “sparse” reward that judged whether the robot succeeded at the overall task. With help from shaped rewards, it took 50 hours of training for the robot model to learn — but the robot’s technique was still awkward:

A clip of different robot policies controlling a simulated hand, from Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations. Note that the reinforcement learning policy (center) grips the simulated hammer awkwardly compared to either a human-controlled demo (left) or the policy trained with the paper’s method that mixed demonstration data and reinforcement learning (right).

However, if the researchers provided 25 demonstrations of a human completing the task, the robot model learned how to do the task in about six hours — almost 10 times faster. And the robot wound up with better hammering technique.

While this paper is almost a decade old now, the basic observation is still true: in order to learn from trial and error, it’s helpful for the model to start with a certain level of basic competence so it succeeds at least some of the time. And one of the best ways to achieve basic competence is to have it first learn from human examples.

The sim-to-real gap

There’s another problem with trying to train a robot entirely in simulation: many aspects of the world are so complex that we don’t know how to simulate them with enough fidelity.

Take the hammer example again: one of the reasons the model learned to use an awkward grip that probably wouldn’t work in real life is that the simulator couldn’t model friction perfectly. This discrepancy between simulation and the real world — the sim-to-real gap — is one of the central challenges developers face in training robots in simulation.