How Workers Train Tesla’s Optimus Robot

Inside a glass-walled lab at Tesla’s engineering headquarters, dozens of workers act out the motions of everyday life: lifting a cup, wiping a table, pulling open a curtain.

They repeat each action hundreds of times during eight-hour shifts, and their work is captured by five cameras attached to their helmet and a heavy backpack. CEO Elon Musk sometimes stops by to watch, and Tesla investors visit regularly for demos.

It’s like being a “lab rat under a microscope,” one former worker told Business Insider.

The goal is simple: Teach Optimus, the company’s robot, how to move like a human.

Musk has identified Optimus as a crucial part of the business. During the company’s third-quarter earnings call, Musk said that it “has the potential to be the biggest product of all time” and said the company would eventually produce 1 million units per year. He has also projected that Optimus would account for around 80% of the automaker’s value one day.

Tesla intends for Optimus to perform a wide variety of tasks, including factory work, household chores, and caregiving. The work of the “data collection operators” is designed to pave the way for Optimus to replace human labor.

Business Insider spoke with five current and former workers to understand how Optimus is trained. They said the role is physically demanding, occasionally absurd, and always meticulous. They sprint, squat, and dance. If their movements are deemed not “human enough,” one worker said, their performance is critiqued.

A spokesperson for Tesla did not respond to a request for comment.

Tesla is one of many tech companies, including Figure and 1X, attempting to replicate complex human behavior in a robot. The ultimate goal, as with much of the AI industry, might be autonomy, but an army of flesh-and-blood workers props up the sleek demos and lofty promises.

‘Teaching a baby’

Training a robot to be human isn’t glamorous.

Most data collectors start out wiping down a table, sometimes for weeks. “You take a step, wipe the table, go into a reset pose, and do it all over again,” one former worker said. It’s “rinse and repeat until break time.”

The workers receive detailed documents about how to perform simple tasks, and they’re guided by a thick, constantly evolving manual, three workers said. They’re also paired up with a peer to ensure the tasks are performed correctly, five people said.

“Tesla is very strict on how everything needs to be done,” one worker said. “It can be very tough mentally and physically.”

Since data collection started, workers have largely used motion capture suits to teleoperate the robot — teaching it with manual movement. In June, the company told workers it was pivoting away from motion-capture suits and teleoperation and toward collecting data using only cameras, Business Insider previously reported. The change followed the departure of program director Milan Kovac.

The worker-mounted cameras point in all directions, helping precisely place the data collectors in the environment, workers said. They said the team was told it would be faster to scale data collection without the motion-capture suits.

Since the June shift, the company has incorporated multiple new training processes, including stationing cameras around workers while they perform tasks, three workers said.

Related stories

Jonathan Aitken, a robotics expert at the University of Sheffield, told Business Insider that the camera towers could supplement the data by providing a broader view of the environment.

Workers are sometimes outfitted with haptic gloves, which track the minute movements of their hands, three employees said. Musk has said Tesla has spent considerable energy trying to develop a human-like hand for Optimus, calling it “an incredibly difficult engineering challenge.”

Workers have also recorded themselves mirroring each other’s movements, three people said. At the company’s Fremont, California, plant, data collectors have organized vehicle parts and worked on conveyor belts while wearing the headset and backpack, two people said. Experts who spoke with Business Insider said that collecting different data points for the same task can be helpful for training.

Other tasks are so simple that a former worker likened them to “teaching a baby.” Two data collectors said they recorded themselves working on brain teasers designed for actual babies: putting rings on by size and color, or putting shapes into their corresponding slots.

Tesla has also begun using AI-generated prompts to help train the robot, three people said. During some training exercises, workers receive a series of AI-generated prompts via a headset, which is attached to the 30- to 40-pound backpack via wires. Though they’re often wearing the backpack, it sometimes sits nearby on a chair or table.

Workers said the exercises have included squatting, doing the “Chicken Dance,” acting like a gorilla, pretending to vacuum, sprinting for several yards, pretending to golf, and twerking. Workers are expected to perform each exercise within three to five seconds, the workers said. Two people recalled instances where some tasks made them uncomfortable, including AI-generated requests to crawl on all fours or remove a piece of clothing.

Aitken said the seemingly random tasks could help Tesla understand which areas it needs to improve on.

“How do you know you’ve covered the entire range of things you’ll need it to be able to do?” he said.

‘Cardio all day’

The work can take a physical toll, four people said. One former worker described it as “basically doing cardio all day.”

One former worker said they sustained a back injury while training the robot. They said the unbalanced weight of the backpack made it feel like they were constantly “walking with a limp.” They said they went on medical leave as a result.

“I lost feeling in my right leg and had a sharp pain in my back,” they said.

Workers said they saw colleagues sustain back and neck injuries due to the nature of the role.

Some people experienced problems during teleoperation, which can involve motion-capture suits and a virtual reality headset. A mix of poor image quality and the length of time workers spent wearing the gear triggered severe motion sickness, three people said. They said it could be especially disorienting when the robot fell over.

“You’d have the feeling you’re falling because you’re seeing through its eyes, but you’d be standing upright,” one worker said.

Related stories

For now, teleoperation is mostly reserved for investor visits.

“The investors want to see the bots moving in action,” one former worker said. “When we’re in mo-cap, we’re controlling the bots so it looks more fluid.”

Investors are sometimes accompanied by Musk, who has brought his 5-year-old son X to see the robot, two former workers said. One of the former workers compared the demos to “putting on a big show.”

“It felt like theater,” the former worker said.

Half the time, two workers said, the robot tumbles over when performing tasks that require bending or leaning, sometimes damaging expensive equipment. It’s typically strapped into the gantry — a supportive rig that keeps the robot upright — unless it’s performing tasks that involve traveling more than a few feet, three workers said.

Aitken said the robot should be able to remain upright with ease in a controlled environment like Tesla’s offices. “Having it stand up and maintain balance should be one of the first things you look at,” he said.

Musk said during Tesla’s recent earnings call that the robot is a 24/7 presence at the company’s engineering headquarters and walks around the office and escorts people to meeting rooms.

Kung fu and candy

At one point, more than 100 people worked on data collection, three people said. The company cut dozens of data collectors in September following twice-yearly performance reviews.

Workers are scored on how well they perform the tasks, five people said. They receive feedback on everything from bodily angles to positioning.

The data collectors have an online dashboard with grades based on data quality and quantity. Each worker is expected to gather at least four hours of usable video footage per shift, five people said. If the video footage is deemed unsatisfactory — if the worker’s positioning isn’t quite right, for instance — workers can be penalized.

For the workers who train it, Optimus’ performance metrics are less clear. When training relied primarily on teleoperation, data operators could press a button to see if Optimus would perform the task. (More often than not, it couldn’t, three people said.) Now, these evaluations are less common, and the workers said they have less transparency into the robot’s progress.

In company videos, Optimus can be seen walking, folding laundry, performing kung fu moves at the “Tron Ares” premiere, and handing out candy in Times Square.

A robot demo “is always the very best demo they could show you,” Alan Fern, an AI and robotics expert at Oregon State University, told Business Insider.

“When you see something like it performing kung fu, it looks like it’s doing something intelligent, which leads people to extrapolate its capabilities, but that’s just not true,” Fern said. “It’s just reacting to its environment. There is not a cognitive thought behind it.”

Musk, meanwhile, continues to tout a grand vision. “It won’t even seem like a robot,” he told investors in October. “It’ll seem like a person in a robot suit.”

For now, Optimus is still learning through repetition, trial and error, and endless hours of human labor.

Do you work for Tesla or have a tip? Contact this reporter via email at gkay@businessinsider.com or Signal at 248-894-6012. Use a personal email address, a nonwork device, and nonwork WiFi; here’s our guide to sharing information securely.