Mid-Level Vision for Robotics

This lecture explores the use of mid-level visual representations in robotics, focusing on manipulation and navigation tasks. The instructor presents experimental studies showing how incorporating mid-level perception significantly improves generalization and sample efficiency in learning visuomotor policies. The lecture discusses the benefits of using perceptual priors and the challenges of learning without them, highlighting the importance of carefully selecting mid-level perceptual skills. Various visual features, such as surface normals and depth, are analyzed for their impact on policy performance. Real-world testing and comparisons with state-of-the-art methods are also discussed.