Fall 2025 GRASP SFI: Tairan He, Carnegie Mellon University & NVIDIA, “Scalable Sim-to-Real Learning for General-Purpose Humanoid Skills”

Humanoids represent the most versatile robotic platform, capable of walking, manipulating, and collaborating with people in human-centered environments. Yet, despite recent advances, building humanoids that can operate reliably in the real world remains a fundamental challenge. Progress has been hindered by difficulties in whole-body control, robust perceptive reasoning, and bridging the sim-to-real gap.

In this talk, I will discuss how scalable simulation and learning can systematically overcome these barriers. I will present a research trajectory that advances humanoid capabilities along three dimensions:

1. Sim-to-Real Control: From real-time teleoperation (H2O) to dexterous loco-manipulation (OmniH2O), to a versatile generalist controller (HOVER), and agile transfer via dynamics alignment (ASAP), these works demonstrate increasingly dexterous and adaptable control.

2. Sim-to-Real Perception: With ABS, we show that robust real-world locomotion requires tightly coupling exteroceptive and proprioceptive sensing—shifting sim-to-real learning from blind skill execution to perception-driven control.

3. Future Directions: I will outline potential next steps in (1) Perceptive Loco-Manipulation: end-to-end visuomotor policies unifying perception, locomotion, and manipulation—and (2) Real-to-Sim Evaluation: using high-fidelity simulators environments to provide consistent evaluation protocols and benchmarks for real-world policy evaluation.

Taken together, these works explore how far scalable sim-to-real learning can advance humanoid capabilities across control and perception. While sim-to-real is not the only path toward reliable humanoids, this talk aims to test its limits—probing how scaling simulation and learning can push humanoids closer to functioning as capable partners in real-world environments.