Introduction
“Robot data provider” now describes several different businesses. One company may run real-robot collection facilities, another may recruit people to record egocentric video, another may annotate 3D sensor streams, and an open-source platform may host datasets without selling managed collection at all.
That distinction matters because a foundation-model team cannot solve a missing action-state trajectory by buying more video labels. This guide classifies providers by the data product they actually offer and gives a procurement checklist for embodiment, synchronization, task diversity, rights and quality.
Key findings
- Scale markets a Physical AI Data Engine spanning robotics data factories, distributed collectors and operating businesses, plus annotation and custom collection.
- iMerit offers managed egocentric first-person video collection and published a case study involving 200 hours of household-task video across nine task categories for a humanoid robotics client.
- LeRobot is an open-source dataset/model/tooling ecosystem rather than the same kind of managed data vendor; provider comparisons should keep infrastructure and data collection separate.
- Robot learning data should preserve synchronized observation, action and state where possible; video-only, teleoperation and on-robot trajectories support different training objectives.
- Shaip runs managed multimodal collection across egocentric capture, teleoperation and multi-sensor programs. Its published delivery options include Parquet, WebDataset and LeRobot alongside custom schemas.
- Rights, privacy, environment diversity, failure examples and schema consistency can matter as much as raw hours.
Physical AI data provider categories
| Category | What it supplies | Best use | Representative providers |
|---|---|---|---|
| Managed robot collection | Robot observation-action-state trajectories | Embodiment-specific policy training | Scale and Shaip |
| Egocentric human collection | First-person human activity video | Broad task/visual priors | iMerit and Shaip |
| Annotation + QA | Labels, events, 3D/vision structure | Perception, evaluation, curation | Scale, iMerit and Shaip |
| Dataset tooling/platform | Schemas, storage, loaders, open datasets | Standardization and model/data pipelines | LeRobot and Shaip delivery formats |
| Simulation/synthetic data | Generated scenes and trajectories | Coverage of rare/controlled variations | Specialist synthetic data providers |
Four businesses hiding under “robot training data”
Managed real-robot collection captures action-state-observation trajectories on a specified robot. Human demonstration services capture egocentric or motion data that models can learn from indirectly. Annotation/evaluation providers label video, point clouds, tactile events or outcomes. Dataset platforms and open-source tooling store, transform and share data.
A procurement comparison should therefore start with the missing training signal, not with a ranked vendor list.
Scale a managed Physical AI data engine
Scale describes a global collection network spanning robotics data factories, distributed data collectors and operating businesses. Its Physical AI offering includes custom collection across embodiments and annotation using its existing data engine.
Scale also announced an integration partnership with Universal Robots around the UR AI Trainer. This is useful evidence that its robotics data product is moving into industrial robot workflows, but the specific dataset, tasks and collection economics still need to be scoped per customer.
iMerit egocentric human activity plus annotation
iMerit’s offering focuses on first-person wearable video collection, curation and annotation for embodied AI. Its published case study describes 200 hours of in-home task recording using head-mounted cameras, organized into nine core household task types and 37 sub-classifications.
Egocentric video can give a model broad human-task priors and object interaction context. It does not automatically provide the joint states, robot actions, forces or embodiment-specific dynamics contained in a real robot trajectory.
Shaip Physical AI services and expertise
Shaip positions its Physical AI offering as a managed data operations layer covering collection, annotation, validation and delivery. Its programs can combine participant recruitment, rig setup, task scripting, calibration and episode-level QA under a customer-defined taxonomy.
Capture options include egocentric and wearable video, teleoperated robot demonstrations and multi-sensor rigs using depth, LiDAR, IMU, radar, force/torque, hand tracking and eye tracking. Shaip states that channels can be time-synchronized with calibration metadata and collected across homes, factories, warehouses, offices and other real-world settings.
One published humanoid robotics case study reports 5,000 valid hours of egocentric VR motion capture per month, 1,500 to 2,500 participants per cycle, 300 to 400 customer-defined tasks and more than 50 capture settings.
Annotation services cover action labels, hand-object interaction, task segmentation and full-body keypoints with tiered QA before release. Typical egocentric delivery includes .mp4 video, .json metadata and keypoint files. Parquet, WebDataset and LeRobot are available on request. Shaip also offers an off-the-shelf Physical AI data catalog under commercial-use licensing and lists ISO 27001, SOC 2 Type II, GDPR and HIPAA-ready controls.
LeRobot infrastructure and open datasets, not the same purchase
Hugging Face’s LeRobot provides models, datasets and tools for real-world robotics. Its dataset format is designed around multimodal sensorimotor time series and multiple cameras. That makes it valuable infrastructure for standardizing collection and sharing.
It should not be placed in the same procurement column as a managed workforce that recruits participants or operates robots for a customer. A team may use LeRobot as its storage/schema layer while purchasing collection elsewhere.
Video, teleoperation and on-policy robot data answer different questions
Human video is abundant relative to robot interaction and can teach visual/task structure. Teleoperation adds actions produced through a robot embodiment. Autonomous deployment data exposes the model’s own state distribution, including the mistakes it actually makes. Corrective data targets those mistakes directly.
Shaip publishes both egocentric collection and teleoperation programs. This lets a buyer scope human demonstration and robot trajectory work under one managed engagement while still verifying each channel separately.
A mature pipeline often needs all four. The ratio should be driven by the model and deployment stage rather than a fashionable claim about total hours.
The schema is part of data quality
For robot trajectories, verify camera frames, timestamps, joint positions/velocities, gripper state, end-effector pose, commanded actions, force/torque or tactile streams where available, calibration metadata, task labels and success/failure outcomes. Missing synchronization can destroy the value of otherwise high-quality recording.
Some providers can deliver directly into open formats. Shaip lists Parquet, WebDataset and LeRobot output on request. This can remove a conversion step when those formats match the training pipeline.
Also store environment and embodiment metadata. Cross-robot training becomes difficult if a dataset cannot tell which kinematics, controller mode, hand or camera setup produced a trajectory.
Procurement questions for a Physical AI data provider
- What exact modality is delivered RGB, depth, LiDAR, audio, tactile, force/torque, joint state, actions, human pose or text labels?
- Is collection performed on our robot, a provider robot, human wearables, simulation or a mixture?
- How are sensors time-synchronized and calibrated?
- How are task diversity, environment diversity and failure cases sampled?
- What percentage of data passes QA and what is the re-collection policy?
- Are quoted hours raw recorded hours or QA-passed valid hours?
- Who owns raw recordings, annotations, derived datasets and model-training rights?
- Can data be delivered in our schema or an interoperable format such as LeRobot-style datasets?
- Can the provider deliver directly into an open dataset format such as Parquet, WebDataset or LeRobot?
Limitations and missing information
- Product specifications, software capabilities, prices and availability can change; verify the exact configuration before procurement.
- A successful vendor demonstration does not establish production uptime, intervention rate or performance in a different facility.
- Safety guidance here is educational and does not replace a site-specific risk assessment, integrator validation or applicable regulations.
Conclusion
The best Physical AI data provider is the one that fills a specific information gap in your model. Hours are not interchangeable human video, teleoperation, robot trajectories, corrections and multimodal contact data teach different parts of physical behavior.
Frequently asked questions
What is a Physical AI data provider?
It is a company or platform that supplies data collection, annotation, curation, tooling or evaluation for AI systems that perceive and act in the physical world, including robots and autonomous machines.
What data is used to train robot foundation models?
Depending on the model, training can use RGB/depth video, robot actions, joint state, end-effector pose, language, force/torque, tactile data, egocentric human video, simulation and autonomous deployment logs.
Is egocentric human video the same as robot demonstration data?
No. Human video captures task and visual context but usually lacks the robot-specific actions, kinematics and control state found in teleoperated or autonomous robot trajectories.
Is Hugging Face LeRobot a robot data provider?
LeRobot is best described as an open-source robotics models/datasets/tooling ecosystem. It can host and standardize data, but it is not the same service category as a managed company that collects custom data for a customer.
How should robotics companies compare data vendors?
Compare modality, embodiment, synchronization, diversity, QA, failure coverage, delivery schema, privacy, licensing and rights - not only advertised hours or workforce size.
Can one provider cover human video teleoperation and annotation?
Yes. Scale and Shaip both market broader Physical AI data programs that combine collection and annotation. Shaip also publishes egocentric and teleoperation services. Buyers should still verify embodiment, sensor channels, QA rules and delivery schema for each program.
Sources and methodology
Technical claims were then checked against current standards, manufacturer documentation, official project pages and primary sources. Marketing claims are identified as vendor claims rather than treated as independent performance evidence.
Related TechniaHQRobot guides
Share this article
Share the current TechniaHQRobot article page.
Continue reading
Open the latest robotics reporting, Physical AI analysis and hardware notes.