Skip to main content

The Dirty Secret of AI: Labs Pay XDOF for Robot Training Data

In a significant move signaling the intense race among leading AI laboratories to enable machines to interact within the physical world, OpenAI recent

5 min read6 views5 tags
Originally reported bytechcrunch

In a significant move signaling the intense race among leading AI laboratories to enable machines to interact within the physical world, OpenAI recently announced the revival of its robotics program, which had been suspended in 2021. This renewed focus, however, highlights a critical challenge: the AI industry currently lacks the vast, high-quality training data essential for developing capable robots, comparable to the datasets used for advanced language models.

This deficit is now fostering the emergence of a specialized infrastructure sector. Unlike large language models (LLMs), which are trained on extensive public text corpora, robotics demands data that precisely captures physical interactions. Such data is scarce, and existing sources like YouTube videos or footage from gig workers often prove to be low-fidelity and difficult to align with real-world physics.

XDOF (pronounced “ecks-doff”), a company recently emerging from stealth mode, is positioning itself to address this very challenge. It posits that the next major bottleneck in AI development will not be in models or processing chips, but rather in establishing the robust data feedback loops required to teach robots effective physical interaction.

The startup's mission is to develop sophisticated data pipelines, collection tools, and annotation systems that cutting-edge AI labs and robotics firms find challenging to build internally. To fuel this endeavor, XDOF has successfully secured $70 million in funding from prominent investors including Thrive Capital, Spark Capital, a16z, Lux, and WndrCo. According to co-founder and CEO Philippe Wu, XDOF, with its team of approximately 60 employees, is already collaborating with 20 customers, including several leading AI labs, though specific names remain undisclosed.

“All of the top labs are trying to pursue robotics,” Wu stated, emphasizing the urgency. He added, “We’ve already seen some of the downfalls of falling a little bit behind in the language model race … you don’t want to be in this type of situation where you pursue this technology too late, and everyone is in this boat where physical AI is the next frontier.”

Wu's personal journey led him to this problem during his PhD studies at UC Berkeley. His research centered on empowering robots to acquire skills from large-scale datasets, only to encounter a fundamental obstacle.

“We didn’t have large-scale data to work with,” he recounted to TechCrunch. He described it as “this chicken-and-egg problem — we first needed to actually collect data before we could even ask how to train a foundation model for robotics.”

This experience led Wu and his future XDOF co-founder and CTO, Fred Shentu, to develop GELLO, an innovative, low-cost teleoperation system. GELLO allowed human operators to control robotic arms, thereby generating crucial training data. Wu noted, “It ended up becoming a very influential paper in robotics, because a lot of people had similar needs and bottlenecks, and many started leveraging this type of device for data collection.”

Recognizing the significant market opportunity, Wu, Shentu, and third co-founder and Chief Operating Officer Nemo Jin officially launched XDOF in October 2024. Their aim was to establish a comprehensive data ecosystem for companies developing robotics models. Aware that merely providing data could be an unsustainable business, the company also prioritizes data cleaning, specialized tooling, and annotation, thereby creating a self-reinforcing feedback loop for robot training.

As an initial strategic move, XDOF is collaborating with UC Berkeley’s AI Research lab to unveil ABC, what they believe to be the largest collection of high-quality robot training data ever compiled. This dataset encompasses 130,000 trajectories of robot manipulation, 300 hours of simulation, and 100 hours of evaluations. Such extensive pre-training data has previously been unavailable to the academic community.

“We’ve seen in language, image generation, and other fields, that when models and data are released, the community achieves things that you wouldn’t necessarily have expected,” commented David McAllister, a Berkeley PhD student who played a key role in organizing the ABC release, in an interview with TechCrunch.

The XDOF team has already leveraged this data to train robots on various benchmark tasks, including folding T-shirts, flattening boxes, and precisely loading AirPods into their cases.

XDOF's strategy involves operating across a three-tiered data pyramid. The top tier, considered most valuable, comprises teleoperation data collected directly from robots in their deployment environments. The second tier involves teleoperated robots gathering more generalized data, akin to the GELLO project. The final tier focuses on "egocentric" data, collected by humans performing everyday tasks, for which XDOF intends to develop its own wearable sensors.

Wu emphasized the importance of data capture hardware, stating, “Your camera choice is going to affect the quality of your data — which is going to affect how your hand-tracking algorithm performs. If you don’t design the hardware well from the start, the data you collect might have very specific problems that you didn’t anticipate.”

The company plans to recruit and train a global workforce of teleoperators and egocentric data operators. This labor-intensive model naturally raises a pertinent question: Why aren't the major AI labs undertaking this extensive data production themselves?

Wu explained the scale of the undertaking: “You need a warehouse of hundreds of thousands of square feet with hundreds of robots. You need to maintain these robots, calibrate their physical parameters, and properly train operators.”

This level of infrastructure development demands a singular focus, substantial capital, and significant operational scale—resources that most AI labs prefer to outsource. This precise market gap is what XDOF aims to fill.

The company's name, XDOF, is a clever reference to "degrees of freedom," a robotics term describing the number of independent motions a robot can execute. For context, a human arm, from shoulder to wrist, possesses seven degrees of freedom, while Figure.AI’s latest humanoid robot boasts 30. The "X" in XDOF symbolizes the company's ambitious vision: “Arbitrary degrees of freedom, unlimited degrees of freedom,” as Wu articulated.

#AI News#XDOF#Robotics#Training Data#Physical AI
ES
Editorial StaffEditor

The Editorial Staff at AIChief is a team of professional content writers with extensive experience in AI and marketing. Founded in 2025, AIChief has quickly grown into the largest free AI resource hub in the industry.

View all posts
Reader feedback

What did you think of this story?

User Comments

Filter:
No comments yet. Be the first to comment!
Continue reading
View all news