Are brain waves the next unlock for physical AI?

The frontier of physical AI is a Jenga game in a warehouse in San Leandro, California.

That warehouse is occupied by Encord, a company that builds data tooling used to train AI models. Andrew Ceja is a pilot — the company’s term for its robotic trainers — and he’s carefully pulling wooden blocks from a tottering tower while wearing a headset with a camera that tracks what he sees. That alone is fairly common for collecting robot training data, but this headset also includes sensors that measure his brain waves as he carefully disassembles the block tower.

Encord is one of a growing number of startups betting the next real constraint on humanoid and warehouse robots will be the scarcity of real-world physical training data, and they are building a business not just to manage that data but to manufacture it. The brain wave headset Ceja is wearing was built by Zander Labs, a German neuroscience startup that’s betting measuring brain activity — to deduce mental states like error, intent, and surprise — can create a more useful dataset to train models.

The data bottleneck

The bet that generative AI can do for robots what it’s done for chatbots keeps running into the same wall: physical-world training data is hard to scale. Self-driving car companies collect it themselves, but that approach doesn't generalize. Training from video can work, but it lacks the fidelity of real-world data. Vineeth Velmurugan, Encord’s head of robot learning, says it will take a dataset roughly five times the size of YouTube’s video corpus to break through — a scale that explains why data-generation itself has become a business, not just a research problem.

Encord currently collects training data through two main methods:

• Egocentric video from workers wearing cameras, often augmented with additional angles and metrics
• Remote-operated robot data from leader-follower rigs where a human directly controls one arm and a second mimics its movements

When TechCrunch visited, pilots were using these rigs to create data for tasks like pouring coffee (very sloshy) and stacking poker chips. “Every humanoid company has asked us for these pieces,” Velmurugan says.

The brain wave experiment

Encord’s work with Zander is currently a trial run. The goal is to build an initial brain wave-tagged dataset, run it through customer robotics models, and evaluate if it actually improves performance before deciding whether to scale it up. Lukas Gehrke, a Zander neuroscientist supervising the work, says the amount of brain activity used at any point during a task offers clues for model builders trying to figure out when they need to deploy their highest-effort models.

Velmurugan calls this the “bleeding edge” of solving the robotics data bottleneck. He joined Encord from OpenAI’s robot lab and Berkshire Grey to build the company’s internal data-creation team. Encord was originally founded to help companies annotate machine-vision data and evaluate models; its pivot to manufacturing data was driven by customer demand.

Other modalities and economics

Encord is also experimenting with a set of sensors strapped to the forearm to detect electrical signals in muscles. Video taken of human hands manipulating objects typically doesn’t capture the entire hand, but Velmurugan hopes to build a 3D depiction of hand position based on the arm sensors, creating a more robust understanding for models.

Encord’s datasets are annotated with physical descriptions like “right hand tightens bolt” to aid LLM-based models. Velmurugan estimates this dense annotation is worth 100 times as much as “junky ego data” for specific tasks, and costs only 20 times more to produce — a good trade on paper. But “20 times more” is still real money, and that’s the catch: scraping text off the internet cost frontier labs next to nothing; generating physical training data does not, and that changes the economics of building these models.

A vantage point across the industry

Velmurugan says progress is being made — with Encord’s visibility into programs across the industry, he can see which data techniques are gaining traction before any single customer can. That will keep the dozen or so pilots at Encord’s facility busy. Both Ceja and Sofia Infante, another pilot who works on plugging ethernet cables into servers, are part of a burgeoning workforce developing the building blocks for neural networks.

Ceja had worked at a waste management company where he kept a robotic trash sorter in good working order. Now, as the Jenga tower topples, he says he enjoys the challenge of solving training tasks for robots: “It’s something new every day!”