Real-time high-resolution omnidirectional imaging platform for drone detection and tracking

Bilal Demir, Selman Ergünay, Yusuf Leblebici, Gökçen Nurlu, Vladan Popovic, Jean-Philippe Thiran
SPRINGER HEIDELBERG, 2019
Article

Résumé

Drones have become steadily affordable, which raises privacy and security concerns as well as interest in drone detection systems. On the other hand, drone detection is a challenging task due to small dimensions of drones, difficulty of long-distance detection, strict real-time constraints and necessity of wide angle coverage for drones. Although different radar and audio-assisted drone detection systems have been presented, they suffer from the cost, range, or interference problems. On the contrary, a long-range detection can be obtained by a vision-based system. Aiming that, we propose a real-time moving object detection and tracking system optimized for drone detection using 16 cameras with 20 MP resolution. The proposed system detects drones from short range and long range with 360 degrees surveillance coverage owing high-performance ultra-high-resolution (320 MP) video-processing capability. It is able to detect drones with 100 cm diameter from 700 m distance despite deceptive background. It is interference free, so multiple systems can properly operate in the vicinity without effecting each other. It integrates processing power of embedded systems with flexibility of software to generate a full platform for drone detection and tracking.

Official source

https://infoscience.epfl.ch/record/272089?ln=fr

À propos de ce résultat

Cette page est générée automatiquement et peut contenir des informations qui ne sont pas correctes, complètes, à jour ou pertinentes par rapport à votre recherche. Il en va de même pour toutes les autres pages de ce site. Veillez à vérifier les informations auprès des sources officielles de l'EPFL.

Concepts associés

Chargement

Publications associées

Chargement

Robust, High-Speed Interest Point Matching for Real-Time Applications

Michael Calonder

A typical Computer Vision system needs to process vast amounts of data as captured by one or more cameras, constantly testing the capabilities of today's hardware. Yet such systems face an ever-growing computational load caused by the more and more demanding applications which is further amplified by the trend to run the software on computationally weaker devices such as mobile phones, for example. Moreover, it is imperative for most real-world systems, be it for object detection or mobile robotics, to run in real-time so as to serve the purpose. Although applications of such systems can vary greatly, most systems share the same outset in the processing pipeline: They establish sparse point matches across multiple views of rigid scenes, which is known as the interest point matching problem. Being a very central problem, a number of algorithms offer fairly accurate solutions. However, they force their superordinate application to spend most of the available CPU time on matching, leaving only little room for subsequent tasks in the pipeline. In this thesis we propose two families of algorithms, Signatures and BRIEF, that address these difficulties. Both of them exhibit comparable accuracy to existing methods and improve up to two orders of magnitude on overall speed and one order of magnitude on memory consumption. Signatures and especially BRIEF are conceptually of downright simplicity, offering a competitive and easy-to-implement option for building a solid base for real-time applications with a stringent time budget. We demonstrate the effectiveness by implementing Signatures and BRIEF in four real-time systems. Technically, Signatures employ Randomized Trees to quantify the similarity of two image patches around arbitrary interest points, resulting in a robust description vector. Thanks to the binary tests that the Trees use for indexing image patches into the leaves, they are very efficient to evaluate which directly carries over to Signatures. BRIEF is the ultimate reduction of the binary test idea and uses such tests directly on a smoothed version of the patch to describe it. Doing so can be seen as computing thresholded random gradients resulting in binary description vectors which require less memory than their state-of-the-art counterparts and are particularly efficient to match —especially on recent hardware.

EPFL2010

Chargement

The standard cameras are designed to truthfully mimic the human eye and the visual system. In recent years, commercially available cameras are becoming more complex, and offer higher image resolutions than ever before. However, the quality of conventional imaging methods is limited by several parameters, such as the pixel size, lens system, the diffraction limit, etc. The rapid technological advancements, increase in the available computing power, and introduction of Graphics Processing Units (GPU) and Field-Programmable-Gate-Arrays (FPGA) open new possibilities in the computer vision and computer graphics communities. The researchers are now focusing on utilizing the immense computational power offered on the modern processing platforms, to create imaging systems with novel or significantly enhanced capabilities compared to the standard ones. One popular type of the computational imaging systems offering new possibilities is a multi-camera system. This thesis will focus on FPGA-based multi-camera systems that operate in real-time. The aim of themulti-camera systems presented in this thesis is to offer a wide field-of-view (FOV) video coverage at high frame rates. The wide FOV is achieved by constructing a panoramic image from the images acquired by the multi-camera system. Two new real-time computational imaging systems that provide new functionalities and better performance compared to conventional cameras are presented in this thesis. Each camera system design and implementation are analyzed in detail, built and tested in real-time conditions. Panoptic is a miniaturized low-cost multi-camera system that reconstructs a 360 degrees view in real-time. Since it is an easily portable system, it provides means to capture the complete surrounding light field in dynamic environment, such as when mounted on a vehicle or a flying drone. The second presented system, GigaEye II , is a modular high-resolution imaging system that introduces the concept of distributed image processing in the real-time camera systems. This thesis explains in detail howsuch concept can be efficiently used in real-time computational imaging systems. The purpose of computational imaging systems in the form of multi-camera systems does not end with real-time panoramas. The application scope of these cameras is vast. They can be used in 3D cinematography, for broadcasting live events, or for immersive telepresence experience. The final chapter of this thesis presents three potential applications of these systems: object detection and tracking, high dynamic range (HDR) imaging, and observation of multiple regions of interest. Object detection and tracking, and observation of multiple regions of interest are extremely useful and desired capabilities of surveillance systems, in security and defense industry, or in the fast-growing industry of autonomous vehicles. On the other hand, high dynamic range imaging is becoming a common option in the consumer market cameras, and the presented method allows instantaneous capture of HDR videos. Finally, this thesis concludes with the discussion of the real-time multi-camera systems, their advantages, their limitations, and the future predictions.

EPFL2016

Chargement

The cameras are invented by imitating the human visual system to capture the scene. The camera technologies have been substantially advanced in recent years. 108 MP resolution with 100x hybrid zoom has become standard features for smartphone flagships. In spite of the recent developments, the cameras are still restricted in terms of FoV, depth perception, pixel size, lens system, and more. That is why multicamera systems have progressively become prevalent. The multicamera systems can be a remedy to overcome the limitations that single cameras hold. The FoV can be raised to

360^\circ

through image stitching. The depth can be computed by stereo matching methods. The lens flaws and imperfections can be handled thanks to computer vision and image processing algorithms. Significant computational power is required to process a vast amount of pixels coming from multiple cameras. GPU and FPGA are promising platforms to implement computer vision and video processing applications due to their sophisticated parallelization features. FPGA platforms present more prevailing features, especially for real-time and portable vision applications. FPGAs provide less latency as they hold the connection to image sensors from a low level. FPGA enables the implementation of a system architecture dedicated to the target application. The compact systems can be designed by designing custom PCBs using only necessary ports. Also, FPGAs consume lower power and cheaper option compared to GPUs. On the other hand, GPUs propose more versatility and easy design and upgrade time. In the light of these observations, software and hardware integrated real-time high-resolution multi-view 3D and panoramic systems are presented in the scope of this thesis. Firstly, the depth estimation system is presented in the first part of the thesis. The proposed depth estimation system runs in real-time performance for up to 2K depth map resolution. The system adopts the trinocular scheme to address the occlusion problem. The pixel correspondence challenge in textureless-regions, from which the conventional stereo matching-based depth estimation systems suffer, is tackled by projecting artificial patterns through the integrated pico-projector. The application-specific system architecture ensures the high-performance depth map streaming. Secondly, the drone detection and tracking system is presented. The proposed drone detection system is capable of simultaneously monitoring 360° environment and detecting the drone from a long-range in real-time performance. The distributed architecture design enables ultra-high-resolution image processing. data coming from the hardware part. The GPU design is opted due to its high level of parallelization capability. The proposed system is appropriate to be employed for surveillance applications such as drone detection, passive radar system, vast terrain, and border control applications. Thirdly, the 3D stereoscopic panorama construction system is presented. The proposed system generated 2 separate panoramas for the left and right eyes to achieve 3D perception. The system offers the novel camera arrangement and the 3D panorama generation algorithm. The cameras are positioned to minimize the intra-panorama parallax while raising the inter-panorama parallax to augment 3D perception. Finally, the real-time vision systems are discussed with their pros and cons, and future predictions are presented in the conclusion part of the thesis.

EPFL2020

Chargement

Robust, High-Speed Interest Point Matching for Real-Time Applications

Michael Calonder

EPFL2010

360^\circ

EPFL2020

EPFL2016