10 Emerging Robotics Infrastructure Themes

We’ve seen humanoids do back flips, quadrupeds do parkour, and grippers that can delicately screw in a light bulb. As robots move from prototype to production, we’ve been asking: What do builders need right now to help accelerate this process?

We narrowed the answer down to ten key areas of innovation in three categories: the data and environments that teach a robot what to do, the hardware and networking that let it actually do it, and the software platforms that turn it all into a deployable system.

Data & Simulation

Large language models had the internet’s text to learn from. Robots have no equivalent. Berkeley Professor Ken Goldberg calls this the 100,000-year data gap in robotics. There’s no vast, ready-made archive of the physical world to train robot models to pick something up, walk across unpredictable terrain, or recover when it stumbles. Closing this data gap will require innovation:

• Data collection platforms: Robotics data can come from the real world or simulation. Real-world data includes video (egocentric, exocentric, etc.), universal manipulation interfaces (UMI), teleoperation, and production data. Collecting diverse high quality data is hard. The physical hardware rigs used to capture data, the data that is collected, and how it is annotated all determine how useful it is for training models. The “pick-and-shovel” platforms that provide production scale data sets and tooling to work with data represent a big opportunity.
• UMI Collection Devices: The Universal Manipulation Interface (UMI) framework lets humans teach robots manipulation skills using a handheld gripper. UMI devices need to be comfortable enough for a person to wear for training sessions and they have to capture data clean enough to transfer to a robot’s hardware. Getting these two things right might create the Tesla of robotics — an autonomous system passively trained without the user ever noticing. As Nvidia’s Jim Fan put it in the Robotics’ End Game YouTube, “Wearing these UMI or data wearables is still cumbersome, it’s not as seamless as just driving to work, so we need an FSD equivalent.”
• Simulators: Simulators offer a scalable method of data generation over real world data collection. Multi-physics simulators like NVIDIA’s Isaac Sim and MuJoCo can generate hours of training data in a few minutes. But these simulators still have limitations, such as accurately modeling deformable objects like clothing or liquids like water. Furthermore, policies trained in these idealized environments often fail when deployed due to unmodeled real-world factors, a mismatch known as the sim2real gap. Eka Robotics shows promising evidence of overcoming the limitations of simulation, particularly with their demonstration of picking a delicate raspberry. Faster and more precise multi-physics simulators will further reduce the data gap.
  
• World Models: A world model is an AI system that builds an internal representation of an environment. The model predicts how that environment changes over time in response to actions. Waymo has used world models successfully in self-driving to accelerate autonomy, simulating everything from rare events, like a car encountering an elephant, to open-ended what-if scenarios. Henry Yin and Naomi Xia at MOE Capital summarizes where robotics stands on this in his article, The Model That Dreams the World. A world model could be a better planner and a better simulator for robotics. By creating a more realistic representation of the environment, world models could help craft robot policies with a smaller sim2real gap.

Hardware & Networking

Robots are tightly coupled systems of hardware and software. In the digital world, text can be a standard input and output for models, which is relatively hardware agnostic. The robotics hardware stack consists of sensors, actuators, batteries, and GPUs. In robotics, inputs and outputs are hardware-dependent. A robot perceives the world through a variety of sensors (such as vision, lidar, and force) and outputs control instructions to a variety of motors and actuators. As a result, advances in hardware for robotics are important. These areas of opportunity include:

• Real Time Teleoperations: Teleoperation collects data, fine-tunes behaviors, and rescues a robot that’s gotten stuck in the field. But today’s systems have real constraints. These include network latency, connection instability, video encoding and decoding delays, and a reliance on visual-only feedback—since low-latency haptic loops are difficult to implement. Without haptics, an operator can’t feel physical contact, making it much harder to operate safely and accurately.  A solution that allows intuitive, long-range teleoperation from anywhere would be an enabler for robotic training and deployment.
• Onshoring: Rising geopolitical tensions and the broader race around AI are pushing companies to bring robotic supply chains and manufacturing onshore. Apart from a few exceptions, like Boston Dynamics, which manufactures humanoids and quadrupeds in Massachusetts, the bulk of robotics manufacturing and the supply chain runs through China. Unitree Robotics’ filings to go public show a robotics hardware manufacturer is a viable business model. This leaves a massive opening to build domestic capacity not only in FATP (final assembly, testing, and packaging), but also in core components like actuators and batteries.
• Actuators: Actuators are important, they enable motion. Proprioceptive actuation, pioneered by Professor Sangbae Kim, helped make locomotion successful. Yet we see limitations like motors having to be cooled at a humanoid marathon race in China. Actuators are incredibly important to dexterity as well. Human hands have 22 degrees of freedom, and building a robotic equivalent means packing small, powerful motors into a very tight space. Origami Robotics explains the challenge in their blog, The Dexterity Deadlock. Robotics would greatly benefit from actuators that have more torque, are lighter and consume less power.
• Tactile & Force Sensors: To succeed at human-like dexterity we need “to collect both the right data and learn the right thing”, Rodney Brooks writes in his article Why Today’s Humanoids Won’t Learn Dexterity. The right data must include touch and force sensing. The human hand is proof of this; it has ~17,000 mechanoreceptors and registers force across joints to control movement. Eka’s vision-force-action model demonstrates that incorporating force sensing enables robots to perform tasks previously thought impossible, such as screwing in a light bulb. Tactile feedback could further enhance dexterity. Tactile sensing can be done via vision, pressure, capacitance, etc. Determining the optimal sensor modality for learning and robust sensor design that can withstand use in deployment is an interesting opportunity. 

Software Tooling

MLOps emerged as the essential infrastructure for machine learning engineers to collect data, train models, and deploy them reliably. The stack to tune, evaluate, and deploy robots is rapidly evolving with incumbents like ROS and startups like Foxglove innovating at different layers of the stack. We see opportunities to build further especially at the fine-tuning, inference and guardrails layers.

• • Fine-tuning: Tools that take task-specific datasets, failure modes, and their corresponding recovery cases to continuously update and optimize models.
  • Evaluations & Benchmarking: Frameworks to measure how well a model or robot performs on a given task and to test for safety.
  • Replay & Testing: Tools that help virtually test robots in simulated environments, then replay logs to catch bugs and fix failures. 

• • Monitoring & Observability: Tools that stream, visualize, and log sensor data and joint poses in production, so issues can be caught and fixed quickly.

• • Guardrails: For real-time safety and security checks that enforce rules to ensure safe, reliable functionality.
  • Inference: Optimizations and hybrid runtime engines built to serve models across the cloud and directly on edge hardware, with minimal latency and power consumption.

• Fleet Orchestration: A fleet orchestration layer serves as the software ‘control plane’ that enables operators to deploy, manage, and monitor multiple robots in the field. This layer makes it easy to integrate diverse robot embodiments, as well as understand and fine-tune their capabilities. By taking high-level goals and making them operational, this layer breaks complex objectives into subtasks, assigns them to the right robots, and coordinates real-time execution and data-sharing across the fleet. 

We expect the technologies that reduce friction across the stack and help robots move to production will flourish over the next decade. If you’re building here, we’d like to hear from you.