Robotics Day at the Penn Wharton China Center – Day 2

Machine Learning for Robots: Perception, Planning and Motor Control 
Dr. Dan Lee | Director, GRASP (General Robotics, Automation, Sensing, and Perception) Lab

Evan C. Thompson Term Chair for Excellence in Teaching, Department of Electrical and Systems Engineering Machines today excel at seemingly complex games such as chess and trivia contests, yet still struggle with basic perceptual, planning, and motor tasks in the physical world. What are the appropriate representations needed to execute and adapt robust behaviors in real-time? I will present some examples of learning algorithms from my group that have been applied to robots for monocular visual odometry, high-dimensional trajectory planning, and legged locomotion.

These algorithms employ a variety of techniques central to machine learning: dimensionality reduction, online learning, and reinforcement learning. I will show and discuss applications of these algorithms to autonomous vehicles and humanoid robots.

Daniel Lee is the Evan C Thompson Term Chair, Raymond S. Markowitz Faculty Fellow, and Professor in the School of Engineering and Applied Science at the University of Pennsylvania.

He received his B.A. summa cum laude in Physics from Harvard University in 1990 and his Ph.D. in Condensed Matter Physics from the Massachusetts Institute of Technology in 1995. Before coming to Penn, he was a researcher at AT&T and Lucent Bell Laboratories in the Theoretical Physics and Biological Computation departments. He is a Fellow of the IEEE and has received the National Science Foundation CAREER award and the University of Pennsylvania Lindback award for distinguished teaching. He was also a fellow of the Hebrew University Institute of Advanced Studies in Jerusalem, an affiliate of the Korea Advanced Institute of Science and Technology, and organized the US-Japan National Academy of Engineering Frontiers of Engineering symposium.

As director of the GRASP Robotics Laboratory, his group focuses on understanding general computational principles in biological systems, and on applying that knowledge to build autonomous systems.

Legged Locomotion for the Urban-Desert Interface: from Robust Steady State to Reactive Transition Maneuvers
Dr. Dan Koditschek | Alfred Fitler Moore Professor, Department of Electrical and Systems Engineering

This talk will address some of the challenges facing legged robot platforms that aim to bring autonomously borne environmental sensory payloads into the mixed terrain encountered at the interface between the human engineered urban landscape and the complex desert substrate. On the one hand, the urban environment generally presents regular geometries and hard surfaces, punctuated by occasional slippery or fragile substrates. These conditions require robust steady state gaits of a kind that contemporary robotics has become increasingly capable of delivering. On the other hand, the desert environment presents highly irregular geometries and complex, poorly modeled substrates that demand a much more aggressive ability to transition between gaits and other modes of self-manipulation. The talk, an adaptation and extension of a plenary lecture given at the 2012 IEEE Workshop on the Future Intelligent Desert (Shanghai), will explore some of the new approaches we have begun to take toward this rather undeveloped area of legged locomotion science.

Daniel E. Koditschek is the Alfred Fitler Moore Professor of Electrical and Systems Engineering. Dr.Koditschek received his bachelorD5s degree in Engineering and Applied Science and his M.S. and Ph.D. degrees in Electrical Engineering in 1981 and 1983, all from Yale University. He served on the Yale Faculty in Electrical Engineering until moving to the University of Michigan a decade later. In January 2005, he moved to the University of Pennsylvania to assume the post of Chair of the Electrical and Systems Engineering Department, within the School of Engineering and Applied Science.

Koditschek’s research interests include robotics and, more generally, the application of dynamical systems theory to intelligent mechanisms. His archival journal and refereed conference publications, numbering well over 100, have appeared in a broad spectrum of venues ranging from the Transactions of the American Mathematical Society through The Journal of Experimental Biology, with a concentration in several of the IEEE journals and related transactions. 

On robotics for health care monitoring in elder care
Dr. Mark Yim | Department of Mechanical Engineering and Applied Mechanics

The promise of robot systems as initially imagined in science fiction is that of generic machines capable of doing a variety of tasks often mimicking humans. It turns out doing that can be very expensive. This talk will present some general principles towards designing low cost systems while also presenting specific examples ranging from mechatronic components (sensors and actuators), robotic components (grippers) to full systems (mobile manipulator and flying systems). In each case we will present some practical examples of methods that can be applied today.

Mark Yim is a professor at the University of Pennsylvania. His group designs and builds modular self-reconfigurable robots and has demonstrated robots that can transform into different shapes, jump, climb, manipulate objects and reassemble themselves after being kicked into pieces. Recently, his work has followed a theme of simplicity and low cost. His other research interests include product design, reactive art and architecture, origami, snake locomotion, flying robots, and self-assembling floating structures.

Honors include the Lindback Award for Distinguished Teaching (UPenn’s highest teaching honor); induction as a World Technology Network Fellow; and induction to MIT’s TR100 in 1999. He has over 40 patents issued (perhaps most prominent are related to the video game vibration control which resulted in over US$100,000,000 in litigation and settlements).

10:45 – 11:05

3D object recognition, localization, and reconstruction
Dr. Kostas Daniilidis | Associate Dean of Education, School of Applied Sciences and Engineering, Department of Computer and Information Science

In this talk, we address the problem of detection, localization, and reconstruction of 3D objects in cluttered scenes. Object exemplars are given in terms of 3D models and view exemplars. We learn and select appearance-based discriminative parts which are mapped onto the 3D model from the training set through a facility location optimization. The training set of 3D models is summarized into a sparse set of shapes from which we can generalize by linear combination. The main challenge is how to combine geometry and appearance: how to select part hypotheses and compute the 3D pose and shape at the same time. To achieve this, we optimize a function that minimizes simultaneously the geometric reprojection error as well the appearance matching of the parts. We apply the alternating direction method of multipliers (ADMM) to minimize the resulting convex function.

For specific classes of objects like surfaces of revolution, we prove how we can estimate 3D pose and shape from two views without any appearance information.

Kostas Daniilidis is Professor of Computer and Information Science at the University of Pennsylvania where he has been faculty since 1998. He currently the Associate Dean of Education.

He obtained his undergraduate degree in Electrical Engineering from the National Technical University of Athens, 1986, and his PhD in Computer Science from the University of Karlsruhe, 1992, under the supervision of Hans-Hellmut Nagel. His research interests are on visual motion and navigation, active perception, 3D object detection and localization, and panoramic vision. He was Associate Editor of IEEE Transactions on Pattern Analysis and Machine Intelligence from 2003 to 2007. He founded the series of IEEE Workshops on Omnidirectional Vision. In June 2006, he co-chaired with Pollefeys the Third Symposium on 3D Data Processing, Visualization, and Transmission, and he was Program co-Chair of the 11th European Conference on Computer Vision in 2010. He has been the director of the interdisciplinary GRASP laboratory from 2008 to 2013 and he is the Associate Dean for Graduate Education of Penn Engineering since 2013. He is an IEEE Fellow.

Social vision: social saliency and future localization
Dr. Jianbo Shi | Department of Computer and Information Science

In the first part of the talk, we focus on social vision, relating human motion and action with social attention. We predict social saliency, the likelihood of joint attention in 3D space. We learn from the social interaction captured by first person cameras, and apply the model in a third person image/video. Our representation does not require directional measurements such as gaze directions. A geometric analysis of social interactions in terms of existing qualitative studies such as F-formation and proxemics is presented.

In the second part of the talk, we future localization: to predict a set of plausible trajectories given a depth image. We predict paths avoiding obstacles, between objects, even paths turning around a corner into space behind objects. Inspired by proxemics, we represent the space around a person using an EgoSpace map, akin to an illustrated tourist map. We learn the relationship between the EgoSpace map and trajectory from first person video providing in-situ measurements of the future trajectory. We quantitatively evaluate our method to show predictive validity and apply to various real world scenes including walking, shopping, and social interactions.

Jianbo Shi studied Computer Science and Mathematics as an undergraduate at Cornell University where he received his B.A. in 1994. He received his Ph.D. degree in Computer Science from University of California at Berkeley in 1998. He joined The Robotics Institute at Carnegie Mellon University in 1999 as a research faculty, where he lead the Human Identification at Distance(HumanID) project, developing vision techniques for human identification and activity inference. In 2003 he joined University of Pennsylvania where he is currently a Professor of Computer and Information Science. In 2007, he was awarded the Longuet-Higgins Prize for his work on Normalized Cuts. His papers have been cited over 27,000 times. His current research focuses on human behavior analysis and image recognition-segmentation. His other research interests include image/video retrieval, 3D vision, and vision based desktop computing. His long-term interests center around a broader area of machine intelligence, he wishes to develop a “visual thinking” module that allows computers not only to understand the environment around us, but also to achieve cognitive abilities such as machine memory and learning.

Bioinspired, Adaptive Building Skins
Dr. Shu Yang | Department of Materials Science & Engineering

Nature provides us fascinating examples with remarkable optical effects, including the dazzling irridence on butterfly wings and dynamic underwater camouflage on Cephalopod skins that can change from transparency to red. Meanwhile, there have been tremendous interests in design of smart roofing, skylights, and architectural windows, which can block or reflect sunlight on scorching days, and return to a transparent state at a low lighting condition to improve light harvesting and capture free heat from the sun. The assembly of the existing devices, however, is often complex and costly.

Taking the cues from nature, we fabricate different material systems from colloidal particle dispersions in a solution and in a film, which can dramatically and reversibly change the optical properties from a highly transparent state to colorful displays and/or opaqueness in respond to proximity, lighting, motion, and mechanical stretching. By coupling the materials-environment response at the nano- and microscales with CMOS technology, we demonstrate autonomous tracking/imaging/sensing ability and feedback control systems as the first step toward energy efficient building skins.

Shu Yang is a Professor in the Departments of Materials Science & Engineering, and Chemical & Biomolecular Engineering at University of Pennsylvania. Her group is interested in synthesis, fabrication and assembly of polymers, liquid crystals, and colloids with precisely controlled size, shape, and geometry; investigating the dynamic tuning of their szie and structures, and the resulting unique optical, mechanical and surface/interface properties. Yang received her BS degree from Fudan University, China in 1992, and Ph. D. degree from Chemistry and Chemical Biology while researching in the Department of Materials Science and Engineering at Cornell University in 1999. She worked at Bell Laboratories, Lucent Technologies as a Member of Technical Staff before joining Penn in 2004. She is elected as Fellow of National Academy of Inventors (2014) and TR100 as one of the worldD5s top 100 young innovators under age of 35 by MIT’s Technology Review (2004). She was a recipient of ICI (1999) and Unilever (2001) student awards from American Chemical Society (ACS) for outstanding research in polymer science and engineering. She also served as Materials Research Society 2010 Fall meeting co-chair.

Lower Limb Exoskeleton Rehabilitation Robot
Dr. Duxin Liu | Shezhen Institutes of Advanced Technology (SIAT), CAS