The frontier of physical AI isn’t just a warehouse. It’s a Jenga tower.
In San Leandro, California, inside a nondescript industrial unit occupied by Encord, the game is getting heavier. Andrew Ceja isn’t just playing. He’s training a machine.
Ceja is what Encord calls a “pilot.” In plain English, he’s a robotic trainer. He wears a headset fitted with a camera to capture his perspective. But there’s something else strapped to his head. Sensors measuring his brain waves.
As he pulls wooden blocks from a tottering stack, the system records more than just the visual data. It captures his intent. His focus. The exact moment of error.
This isn’t science fiction. It’s the latest bet in the race to build functional humanoid robots.
Why physical AI data is harder than text
Generative AI revolutionized chatbots because the data was everywhere. You could scrape billions of text files from the internet. It cost almost nothing.
Physical AI doesn’t work that way.
The training data for robots simply does not exist at the scale needed. Not yet.
Vineeth Velmurugan knows this well. He heads robot learning at Encord. Before that, he was at OpenAI’s robotics lab and Berkshire Grey. He joined Encord because the old model broke down.
“The data simply does not exist,” he said.
Companies building machine-vision tools used to just manage data. Now they have to make it. End-to-end learning requires real-world manipulation data. Video alone? Not enough fidelity. Self-driving cars collect their own data, sure. But scaling that is a nightmare.
Velmurugan has a number that should scare anyone trying to replicate LLM economics for robots. You need a dataset roughly five times the size of YouTube’s entire video library.
How do you build that without bankrupting the lab?
You manufacture it. And you tag it with neuroscience.
Encord and Zander Labs: Measuring the mind
The headset Ceja wears comes from Zander Labs. It’s a German neuroscience startup. They don’t just watch you move. They measure brain activity to deduce mental states.
Surprise. Error. Intent.
This is a trial run between Encord and Zander. The goal is specific: create an initial brain wave-tagged dataset. Run it through customer robotics models. See if performance actually improves. If it does, they scale it up.
If it doesn’t, they pivot.
Lukas Gehrke, a neuroscientist at Zander supervising the work, sees the practical value immediately. The intensity of brain activity during a task tells model builders when the robot needs help. It marks the “high-effort” moments.
This is what Vineeth Velmurugan calls the “bleeding edge.”
It’s also expensive.
The economics of “junky ego data”
Most companies building robot brains are turning to “egocentric” data. Workers wear cameras. They perform tasks. Robots watch.
Encord does this too. They have pilots in factories around the globe. But in San Leandro, they are experimenting.
At one station, Sofia Infante maneuvers robotic arms to plug and unplug Ethernet cables from server racks. Data center operators dream of automating this. If only robots had the precision.
I tried it myself.
It’s hard.
Robot pincers lack the degrees of freedom of human fingers. They are clumsy. Infante has the dexterity. Her brain has the micro-adjustments. The robot needs to learn those adjustments. That’s the gap.
Another new modality involves sensors on the forearm. They detect electrical signals in muscles. Why? Because video of human hands often misses the hand itself. Obstructions happen. Angle limitations.
The arm sensors fill the gaps. They help create a 3D map of hand position even when the camera can’t see it.
But here’s the catch with the economics.
Encord annotates their datasets. “Right hand tightens bolt.” “Left hand pours coffee.” Dense annotation.
Velmurugan claims this level of detail is worth 100 times more than “junky ego data” for training specific tasks. It only costs 20 times more to make. On paper, it’s a good trade.
Twenty times more is still real money.
LLM makers scraped text for pennies. Physical data requires manufacturing. It requires human labor. It requires neuroscience gear. The economics are fundamentally broken compared to software.
Who is buying this data?
Encord works with leading robotics firms. Velmurugan can’t name them. But they are asking for specific skill datasets.
Fine-tuning.
At the Encord facility, leader-follower rigs are running. Paired robotic arms. One moves at the speed of human operators. The other mimics it.
Tasks include pouring coffee without sloshing. Stacking poker chips. Sorting fake flowers. Books. Plastic vegetables. Kitty litter trays. Bags of wire.
Every humanoid company wants these pieces.
Velmurugan sees it all from his vantage point. He sits between many robotics companies. He knows what’s working. He knows what’s failing. Startups and frontier labs alike are trying to figure out the same problems.
Which data techniques gain traction? Which ones fizzle?
Encord spots the winners before the customers even know they’re losing.
That keeps the pilots busy.
Ceja, Infante—they aren’t just annotators. They are the building blocks. They previously worked at Scale, another data annotation firm. Now they are here, pulling blocks from towers, plugging cables, pouring coffee.
Ceja used to maintain a robotic trash sorter at a waste management company. Technology was his ticket out of manual sorting.
Now he’s teaching machines how to think about stacking blocks.
“It’s something new every day,” he said.
The tower fell over. Again.
The sensors recorded it. The model learned.
But the data hunger doesn’t stop. It grows. Five times the size of YouTube. That’s not a near-term problem. It’s an industrial challenge.
Will brainwave tagging be the key? Or just another expensive sensor stack that slows down production?
We won’t know until the models actually work.
