Spring 2023 GRASP on Robotics: Jeannette Bohg, Stanford University, “Scaling Robot Learning for Long-Horizon Manipulation Tasks with Language, Logic and Youtube”

This was a hybrid event with in-person attendance in Wu and Chen and virtual attendance…

ABSTRACT

My long-term research goal is enable real robots to manipulate any kind of object such that they can perform many different tasks in a wide variety of application scenarios such as in our homes, in hospitals, warehouses, or factories. Many of these tasks will require long-horizon reasoning and sequencing of skills to achieve a goal state. While learning approaches promise generalization beyond what the robot has seen during training, they require large data collection – a challenge when operating on real robots and specifically for long-horizon tasks. In this talk, I will present our work on enabling long-horizon reasoning on real robots for a variety of different long-horizon tasks that can be solved by sequencing a large variety of composable skill primitives. We approach this problem from many different angles such as (i) using large-scale, language-annotated video datasets as a cheap data source for skill learning; (ii) sequencing these learned skill primitives to resolve geometric dependencies prevalent in long-horizon tasks; (iii) learning grounded predicates thereby enabling closed-loop, symbolic task planning.