Design and Implementation of Real-Time Multi-sensor Vision Systems (Keynote)

Implementation of high performance multi-camera / multi-sensor imaging systems that are required to produce real-time video output pose a large number of unique challenges to conventional digital design based on general-purpose processors or GPUs. In potential application areas ranging from machine vision, automotive, and virtual reality, the need for real-time operation with very limited latency dictates customized architectures that are not readily implementable using conventional approaches. In this talk, we will discuss the algorithms that are utilized in such multi-sensor platforms, the specialized architectures that allow streaming video processing, and system-level integration issues. Problems such as light-field reconstruction, multi-band blending, pixel-level interpolation, and rectification are presented, with detailed circuit and system level examples.