GRASP Special Seminar – Yaser Ajmal Sheikh, Carnegie Mellon University, “Structure from Motion in the Wild: Towards 3D Reconstruction of Active Scenes from Moving Cameras”

Abstract: Our social environments abound with activity: faces expressing and emoting, people ambulating and gesticulating, and vehicles locomoting. These same environments are increasingly being imaged by cameras that people carry or wear. In this talk, I will discuss research into reconstructing active environments from cameras that may themselves be moving — a problem that requires the marriage of geometric reconstruction and statistical representation of time-varying data.I will first present an algorithm to reconstruct the motion of a collection of cameras constrained by an underlying articulated structure. The algorithm has been applied for motion capture using multiple body-mounted cameras. Results will be shown in settings where capture would be difficult with traditional motion capture systems, including walking outside and swinging on monkey bars. A central challenge in reconstructing active 3D structure is representation. I will present trajectory-based models of active objects and describe a closed form solution for the estimation of time-varying 3D structure. I will conclude by motivating the use of spatiotemporal models for representing active environments and describe the significant open problems in the area.