Spring 2012 GRASP Seminar – Jingyi Yu, University of Delaware, “Beyond Perspective Cameras: Multi-perspective Imaging, Reconstruction, Rendering and Projection”

In this talk, I will provide a complete framework for using multi-perspective imaging models in computer vision and graphics. Our multi-perspective framework consists of four key components: acquisition, reconstruction, rendering, and display. A multi-perspective camera captures a scene from multiple viewpoints in a single image. From the input image, intelligent software can recover 3D scene geometry using multi-perspective stereo matching algorithms or via shape-from-distortion approaches. Our solutions are particularly useful for reconstructing specular (reflective and refractive) surfaces such as dynamic 3D fluid surfaces which can also be viewed as general multi-perspective cameras. The recovered geometry, along with lighting and surface reflectance, can then be loaded into a new multi-perspective graphics pipeline for real-time rendering. Finally, we can visualize the rendering results on a special multi-perspective display that combines a single consumer projector and specially-shaped mirrors/lenses. Such displays will offer an unprecedented level of flexibility in terms of aspect ratio, size, field of view, etc.