Tracking and Structure from Motion

Dense three-dimensional reconstruction of a scene from images is a very challenging task. In the structure from motion approach one of the key points is to compute depth maps which contain the distance of objects in the scene to a moving camera. Usually, this is achieved by finding correspondences in successive images and computing the distance by means of epipolar geometry. In this Master's thesis, a variational framework to solve the depth from motion problem for planar image sequences is proposed. Camera ego-motion estimation equations are derived and combined with the depth from motion estimation in a single algorithm. The method is successfully tested on synthetic images for general camera translation. Since it does not depend on the correspondance problem and because it is highly parallelizable, it is well adapted for real-time implementation. Further work in this thesis include a review of general variational methods in image processing, and in particular TV-L1 optical flow as well as its real-time implementation on the graphics processing unit.

Fast Structure from Motion for Planar Image Sequences

Luigi Bagnato, Pierre Vandergheynst, Andreas Weishaupt

Dense three-dimensional reconstruction of a scene from images is a challenging task. Usually, it is achieved by finding correspondences in successive images and computing the distance by means of epipolar geometry. In this paper, we propose a variational framework to solve the depth from motion problem for planar image sequences. We derive camera ego-motion estimation equations and we show how to combine the depth map and ego-motion estimation in a single algorithm. We successfully test our method on synthetic image sequences for general camera translation. Our method is highly parallelizable and thus well adapted for real-time implementation on the GPU.

2010

Beyond Cameras: Real-Time High-Resolution 3D and Panoramic MultiView Vision Systems with Their Applications

Bilal Demir

The cameras are invented by imitating the human visual system to capture the scene. The camera technologies have been substantially advanced in recent years. 108 MP resolution with 100x hybrid zoom has become standard features for smartphone flagships. In spite of the recent developments, the cameras are still restricted in terms of FoV, depth perception, pixel size, lens system, and more. That is why multicamera systems have progressively become prevalent. The multicamera systems can be a remedy to overcome the limitations that single cameras hold. The FoV can be raised to

360^\circ

through image stitching. The depth can be computed by stereo matching methods. The lens flaws and imperfections can be handled thanks to computer vision and image processing algorithms. Significant computational power is required to process a vast amount of pixels coming from multiple cameras. GPU and FPGA are promising platforms to implement computer vision and video processing applications due to their sophisticated parallelization features. FPGA platforms present more prevailing features, especially for real-time and portable vision applications. FPGAs provide less latency as they hold the connection to image sensors from a low level. FPGA enables the implementation of a system architecture dedicated to the target application. The compact systems can be designed by designing custom PCBs using only necessary ports. Also, FPGAs consume lower power and cheaper option compared to GPUs. On the other hand, GPUs propose more versatility and easy design and upgrade time. In the light of these observations, software and hardware integrated real-time high-resolution multi-view 3D and panoramic systems are presented in the scope of this thesis. Firstly, the depth estimation system is presented in the first part of the thesis. The proposed depth estimation system runs in real-time performance for up to 2K depth map resolution. The system adopts the trinocular scheme to address the occlusion problem. The pixel correspondence challenge in textureless-regions, from which the conventional stereo matching-based depth estimation systems suffer, is tackled by projecting artificial patterns through the integrated pico-projector. The application-specific system architecture ensures the high-performance depth map streaming. Secondly, the drone detection and tracking system is presented. The proposed drone detection system is capable of simultaneously monitoring 360° environment and detecting the drone from a long-range in real-time performance. The distributed architecture design enables ultra-high-resolution image processing. data coming from the hardware part. The GPU design is opted due to its high level of parallelization capability. The proposed system is appropriate to be employed for surveillance applications such as drone detection, passive radar system, vast terrain, and border control applications. Thirdly, the 3D stereoscopic panorama construction system is presented. The proposed system generated 2 separate panoramas for the left and right eyes to achieve 3D perception. The system offers the novel camera arrangement and the 3D panorama generation algorithm. The cameras are positioned to minimize the intra-panorama parallax while raising the inter-panorama parallax to augment 3D perception. Finally, the real-time vision systems are discussed with their pros and cons, and future predictions are presented in the conclusion part of the thesis.

EPFL2020

Beyond Cameras: Real-Time High-Resolution 3D and Panoramic MultiView Vision Systems with Their Applications

Bilal Demir

360^\circ

EPFL2020