Situational Awareness Using A Single Omnidirectional Camera

Luigi Bagnato, Eugenio De Vito, Laurent Jacques, Pierre Vandergheynst
2008
Projet étudiant

Résumé

To retrieve scene informations using a single omnidirectional camera, we have based our work on a shape from texture method proposed by Lindeberg. To do so, we have adapted the method of Lindeberg, that was developed for planar images, in order to use it on the sphere S2 . The mathematical tools we use are stereographic dilation to implement scale variations for the scale-space representation, and ﬁlter steerability on the sphere to decrease computational order. The texture distortions due to the pro jection from the real world to the image contain the informations that enable shape and orientation to be computed. A multi-scale texture descriptor, the windowed second moment matrix, that contains distorsions informations, is computed and analyzed, with some assumptions about the surface texture to retrieve surface orientation. We have used synthetic signals to evaluate the performances of the adapted method. The obtained results for the distance and shape estimations are good when the textures on the surfaces correspond to the assumptions, generally around the equatorial plane of S2 , but when we move away from the equator, the precision of the estimations decreases signiﬁcantly.

Official source

https://infoscience.epfl.ch/record/134761?ln=fr

À propos de ce résultat

Cette page est générée automatiquement et peut contenir des informations qui ne sont pas correctes, complètes, à jour ou pertinentes par rapport à votre recherche. Il en va de même pour toutes les autres pages de ce site. Veillez à vérifier les informations auprès des sources officielles de l'EPFL.

Concepts associés

Chargement

Publications associées

Chargement

Enactive robot vision

Mototaka Suzuki

The complexity of today's autonomous robots poses a major challenge for Artificial Intelligence. These robots are equipped with sophisticated sensors and mechanical abilities that allow them to enter our homes and interact with humans. For example, today's robots are almost all equipped with vision and several of them can move over rough terrain with wheels or legs. The methods developed so far in Artificial Intelligence, however, are not yet ready to cope with the complexity of the information gathered through the robot sensors and the need for rapid action in partially unknown and dynamic environments. In this thesis, I will argue that the apparent complexity of the environment and of the robot brain can be significantly simplified if perception, behavior, and learning are allowed to co-develop on the same time scale. In doing so, robots become sensitive to, and actively exploit, characteristics of the environment that they can tackle within their own computational and physical constraints. This line of work is grounded on philosophical and psychological research showing that perception is an active process mediated by behavior. However, computational models of active vision are very rare and often rely on architectures that are preprogrammed to detect certain characteristics of the environment. Previous work have shown that complex visual tasks, such as position and size invariant shape recognition as well as navigation, can be tackled with remarkably simple neural architectures generated by a coevolutionary process of active vision and feature selection. Behavioral machines equipped with primitive vision systems and direct pathways between visual and motor neurons were evolved while freely interacting with their environments. I proceed further on this line of investigation and describe the application of this methodology in three situations, namely car driving with an omnidirectional camera, goal-oriented navigation of a humanoid robot, and cooperative tasks by two agents. I will show that these systems develop sensitivity to a number of oriented, retinotopic, visual features – oriented edges, corners, height – and a behavioral repertoire to locate, bring, and keep these features in sensitive regions of the vision system that allow them to accomplish their goals. In a second set of experiments, I will show that active vision can be exploited by the robot to perform anticipatory exploration of the environment in a task that requires landmark-based navigation. Evolved robots exploit an internal expectation system that makes use of active exploration to check for expected events in the environment. I will describe a third set of experiments where, in addition to an evolutionary process, the visual system of the robot can develop receptive fields by means of unsupervised Hebbian learning and show that these receptive fields are significantly affected by the behavior of the system and differ from those predicted by most computational models of visual cortex. Finally, I will show that these robots replicate the performance deficiencies observed in experiments of motor deprivation with kitten when they are exposed to the same type of motor deprivations. Furthermore, the analyses of our robot brains suggest an explanation for the deficiencies observed in kitten that have not yet been fully understood.

EPFL2007

Chargement

Analysis of Multiview Omnidirectional Images in a Spherical Framework

Zafer Arican

With the increasing demand of information for more immersive applications such as Google Street view or 3D movies, the efficient analysis of visual data from cameras has gained more importance. This visual information permits to extract some crucial information in scenes such as similarity for recognition, 3D scene information, textures and patterns of objects. Multi-camera systems provide detailed visual information about a scene with images from different positions and viewing angles. The conventional perspective cameras that are commonly used in these systems however, have limited field of view. Therefore, they require either the deployment of many of these cameras or the capture of many images from different points to extract sufficient details about a scene. It increases the amount of data to be processed and the maintenance costs for such systems. Omnidirectional vision systems overcome this problem due to their 360-degree field of view and have found wide application areas in robotics, surveillance. These systems often use special refractive elements such as fisheye lenses or mirror-lense systems. The resulting images however inherit from the specific geometry of these units. Therefore, the analysis of these images with methods designed for perspective cameras results in degraded performance in omnivision systems. In this thesis, we focus on the analysis of multi-view omnidirectional images for efficient scene information extraction. We propose a novel spherical framework for omnidirectional image processing by exploiting the property that most omnidirectional images can be uniquely mapped on the sphere. We propose solutions for three common multiview image processing problems, namely feature detection, dense depth estimation and super-resolution for omnidirectional images. We first address the feature extraction problem in omnidirectional images. We develop a scale-invariant feature detection method which carefully handles the geometry of the images by performing the scale-space analysis directly on their native manifolds such as the parabola or the sphere. We then propose a new descriptor and a matching criteria that take into account the geometry and also eliminate the need for orientation computation. We also demonstrate that the proposed method can be used to match features in images captured by different types of sensors such as perspective, omnidirectional or spherical cameras. We then propose a dense depth estimation method to extract the 3D scene information from multiple omnidirectional images. We propose a graph-cut method adapted to the geometry of those images to minimize an energy function formulated for the dense depth estimation problem. We also propose a parallel graph-cut method that gives a significant speed improvement without a big penalty in accuracy. We show that the proposed method can be applied to multi-camera depth estimation and depth-based arbitrary view synthesis. Finally, we consider the multi-view omnidirectional images with the transformation of pure rotation. We address the view synthesis problem for these images in the framework of super-resolution. Considering also the inaccuracies in the rotation parameters, we solve an optimization problem that jointly estimates the rotation errors and reconstructs a high resolution omnidirectional image. We then extend the minimization problem with a regularization term for improved reconstruction quality with a reduced number of images. Results with both synthetic and real omnidirectional images suggest that the proposed method is a viable solution for super-resolution with omnidirectional images. Overall, this dissertation addresses three important issues of multiview omnidirectional image analysis and processing in a novel spherical framework. Our feature detection method can be used for the calibration of omnidirectional images as well as the feature matching in mobile and hybrid camera networks. Furthermore, our dense depth estimation method can impact the quality of 3D scene reconstruction and provide efficient solutions for view synthesis and multiview omnidirectional image coding. Finally, our super-resolution algorithm for omnidirectional images can promote the development of efficient acquisition systems for high resolution omnidirectional images

EPFL2010

Chargement

In the early stages of visual information processing one of the most fundamental and complex task is segmentation. This process is used to divide images or image sequences into meaningful regions or objects. Such a representation is definitely needed to go from the first rough description of the scene (i.e. light intensity) up to the semantic level, where one is able to discriminate shapes and other high-level visual primitives. The biological visual system performs this extremely difficult task with such ease that is still a major challenge for computer vision to reach this level of speed and accuracy. It is by now widely believed that a method inspired by biological vision has the best chances of accomplishing this goal. Therefore, our main goal in this thesis is to step on biologically inspired schemes to propose new and efficient techniques to solve the image segmentation problem. The starting point and key concept in this research, is the multiresolution paradigm. Indeed, the fact that the Human Visual System (HVS) processes the information in a multiresolution fashion is now well established. Several techniques have been introduced to model this behaviour. Scale-space is one of them and we will thus focus on such a formulation of the segmentation problem. Such a representation is composed by a stack of successive versions of the original data set at coarser scales. It is assumed that, the bigger the scale, the less information referred to local characteristics of the input data will appear. We also impose that general information applying to large scales will last through scale. Taking that into account, it is reasonable to think that local and high resolution scale information can be related to general and low resolution information. This will enable us to extract image structures. We thus propose to link the multiresolution approach to a computational model based on the analysis of the entire range of significant information across scales. The inherent multiscale image structure of the non-linear scale-space guides a coarse to fine segmentation using different strategies. The first one might be termed extrinsic because the non-linear scale-space is analyzed by an external process of linking information through scales. This leads to a completely adaptive scheme that can then be coupled to a labeling procedure in order to define a genuine segmentation. The second one could be labeled intrinsic because it adapts to the image through its inner properties and incorporate this in the labeling step of the main algorithm. These two approaches are particular examples of multiscale image segmentation. We also apply the same scale-space concept to define a segmentation variational formulation that leads to a region merging segmentation. In order to illustrate the proposed segmentation schemes, we apply our algorithms to gray-scale, noisy, color and texture images. Finally, it should be mentioned that the main problem addressed in this thesis, general image segmentation, plays an important role in the image understanding process, especially those that involve natural scenes rich in color and texture. The potential applications of the algorithms are thus numerous: medical image analysis, interactive multimedia developments, object based coding and compression, surveillance applications, image content based browsing and retrieval, etc.

EPFL2006

Chargement

Enactive robot vision

Mototaka Suzuki

EPFL2007

Analysis of Multiview Omnidirectional Images in a Spherical Framework

Zafer Arican

EPFL2010

EPFL2006