Abstract: Stereo vision and optical flow methods attempt to measure scene depth and motion by tracking pixels over frames. To evaluate the performance of such methods, we need “ground truth” – the true depth or true object motion. In this talk I will describe different techniques for creating image datasets with ground truth, including structured lighting, laser and CT scanners, and hidden fluorescent texture. The Middlebury datasets are now well-established benchmarks in computer vision, and I will discuss both benefits and potential pitfalls of such benchmarks. I will also briefly touch on how data with ground truth can aid in developing new algorithms.