Specularities as an information source in the presence of texture

Pascal Lagger
EPFL, 2009
EPFL thesis

Abstract

The appearance of real world objects is the result of the combination of their own texture and the light rays striking them before reaching our eyes, or our camera. This appearance therefore brings very rich information about the objects shapes, materials, motions, and also about the environment that surrounds them. However, while the visual cortex of humans is extremely powerful at sorting out these different sources intricately fused in the objects appearance, it is very challenging to reproduce this capability on a computer. As a result, Computer Vision algorithms tend to make strong simplifying assumptions. For example, one may consider only objects made of Lambertian material, or untextured or static objects, or lighting environments limited to a single point source. For this thesis, we want to overcome these limitations, and be able to consider the most general case: textured objects exhibiting both Lambertian and specular reflectance, in motion in a general lighting environment. We first discuss the limits of a purely Lambertian approach, and show that specularities carry strong information, if one is able to properly exploit them. We then consider two different problems, for which we show how to use specularities even in presence of textured surfaces and uncalibrated lighting. The first problem is the recovery of photometric parameters of object surfaces as well as the lighting environment. We show how it can be done even in case of extended specularities reflections. The second problem we tackle is 3D objects tracking. This is a well-explored problem but for which specularities have never been used, and treated as noise. We show that they can actually improve accuracy to an order of magnitude that is not possible with commonly used cues such as texture alone.

Official source

https://infoscience.epfl.ch/record/135623?ln=en

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related concepts

Related publications

Pascal Lagger
EPFL, 2009
EPFL thesis

Abstract

Official source

https://infoscience.epfl.ch/record/135623?ln=en

About this result

Related concepts (17)

Related concepts

Related publications (7)

Related publications

A machine-learning algorithm correctly classifies cortical evoked potentials from both visual stimulation and electrical stimulation of the optic nerve

Eleonora Borda, Vivien Gaillet, Diego Ghezzi, Elodie Geneviève Zollinger

Objective. Optic nerve's intraneural stimulation is an emerging neuroprosthetic approach to provide artificial vision to totally blind patients. An open question is the possibility to evoke individual non-overlapping phosphenes via selective intraneural optic nerve stimulation. To begin answering this question, first, we aim at showing in preclinical experiments with animals that each intraneural electrode could evoke a distinguishable activity pattern in the primary visual cortex. Approach. We performed both patterned visual stimulation and patterned electrical stimulation in healthy rabbits while recording evoked cortical activity with an electrocorticogram array in the primary visual cortex. Electrical stimulation was delivered to the optic nerve with the intraneural array OpticSELINE. We used a support vector machine algorithm paired to a linear regression model to classify cortical responses originating from visual stimuli located in different portions of the visual field and electrical stimuli from the different electrodes of the OpticSELINE. Main results. Cortical activity induced by visual and electrical stimulation could be classified with nearly 100% accuracy relative to the specific location in the visual field or electrode in the array from which it originated. For visual stimulation, the accuracy increased with the separation of the stimuli and reached 100% for separation higher than 7 degrees. For electrical stimulation, at low current amplitudes, the accuracy increased with the distance between electrodes, while at higher current amplitudes, the accuracy was nearly 100% already for the shortest separation. Significance. Optic nerve's intraneural stimulation with the OpticSELINE induced discernible cortical activity patterns. These results represent a necessary condition for an optic nerve prosthesis to deliver vision with non-overlapping phosphene. However, clinical investigations will be required to assess the translation of these results into perceptual phenomena.

IOP PUBLISHING LTD2021

Human vision has evolved to make sense of a world in which elements almost never appear in isolation. Surprisingly, the recognition of an element in a visual scene is strongly limited by the presence of other nearby elements, a phenomenon known as visual crowding. Crowding impacts vision at all levels and is thus a versatile tool to understand the fundamental mechanisms of vision. For decades, visual crowding was perfectly well explained by traditional feedforward models of vision. In these models, vision starts with the detection of low-level features. This information is combined locally along the hierarchy of the visual cortex to build more and more complex feature detectors, until neurons respond selectively and robustly to complex objects. Crowding happens when nearby elements interfere in this local feature combination process and impair target recognition. However, recent studies have shown that crowding is not determined by local interactions but by the global configuration across the entire visual field. Depending on how elements group together, crowding can even almost disappear, a phenomenon called uncrowding. Hence, crowding is rather a complex, global and high-level phenomenon, that simple feedforward models cannot explain. In this thesis, I first analyse which models of crowding can explain uncrowding. I compare the performance of diverse models, selected according to different architectural and functional features, such as feedforward vs. recurrent architecture, local or global information processing, including a grouping stage or not. I show that the only model that reproduces human behaviour includes a dedicated recurrent grouping processing stage. Second, I show that global effects in crowding cannot be explained by low-level accounts. It was argued that the Texture Tiling model, based on a complex and high-dimensional pooling stage, may account for global effects in crowding, without requiring any recurrent grouping stage. To test this model, I use a large pool of recent crowding data. I show that the Texture Tiling model is equivalent to a simple pooling model and is thus as limited as these models. Next, I focus on deep neural networks, which are well in the spirit of the feedforward framework of vision and have become state-of-the-art models both in computer vision and neuroscience. I test whether AlexNet and ResNet-50, which have been proposed as realistic models of the visual system, exhibit uncrowding. I show that these networks do not reproduce uncrowding for principled reasons. Finally, I use a genetic algorithm to generate stimuli based on the performance of different models, i.e., in a bottom-up manner. The goal is to avoid using stimuli that favour models of grouping from the start. I compare the distribution of stimuli that are produced by the models to the ones that are produced by humans. I show that only the models that include grouping and segmentations processes behave like humans. Taken together, the results in my thesis highlight the importance of recurrent grouping and segmentation processes in human vision when large portions of the visual field are involved. These results can be used as direct guidelines for future models of vision, in order to constraint how recurrent processing should be incorporated to improve the performance of deep neural networks and other feedforward models of vision, and help them generalize to more complex visual inputs.

EPFL2021

Constraint Satisfaction Problems (CSPs) are ubiquitous in computer science. Many problems, ranging from resource allocation and scheduling to fault diagnosis and design, involve constraint satisfaction as an essential component. A CSP is given by a set of variables and constraints on small subsets of these variables. It is solved by finding assignments of values to the variables such that all constraints are satisfied. In its most general form, a CSP is combinatorial and complex. In this thesis, we consider constraint satisfaction problems with variables in continuous, numerical domains. Contrary to most existing techniques, which focus on computing a single optimal solution, we address the problem of computing a compact representation of the space of all solutions that satisfy the constraints. This has the advantage that no optimization criterion has to be formulated beforehand, and that the space of possibilities can be explored systematically. In certain applications, such as diagnosis and design, these advantages are crucial. In consistency techniques, the solution space is represented by labels assigned to individual variables or combinations of variables. When the labeling is globally consistent, each label contains only those values or combinations of values which appear in at least one solution. This kind of labeling is a compact, sound and complete representation of the solution space, and can be combined with other reasoning methods. In practice, computing a globally consistent labeling is too complex. This is usually tackled in two ways. One way is to enforce consistencies locally, using propagation algorithms. This prunes the search space and hence reduces the subsequent search effort. The other way is to identify simplifying properties which guarantee that global consistency can be enforced tractably using local propagation algorithms. When constraints are represented by mathematical expressions, implementing local consistency algorithms is difficult because it requires tools for solving arbitrary systems of equations. In this thesis, we propose to approximate feasible solution regions by 2k-trees, thus providing a means of combining constraints logically rather than numerically. This representation, commonly used in computer vision and image processing, avoids using complex mathematical tools. We propose simple and stable algorithms for computing labels of arbitrary degrees of consistency using this representation. For binary constraints, it is known that simplifying convexity properties reduces the complexity of solving a CSP. These properties guarantee that local degrees of consistency are sufficient to ensure global consistency. We show how, in continuous domains, these results can be generalized to ternary and in fact arbitrary n-ary constraints. This leads to polynomial-time algorithms for computing globally consistent labels for a large class of constraint satisfaction problems with continuous variables. We describe and justify our representation of constraints and our consistency algorithms. We also give a complete analysis of the theoretical results we present. Finally, the developed techniques are illustrated using practical examples.

EPFL1995

A machine-learning algorithm correctly classifies cortical evoked potentials from both visual stimulation and electrical stimulation of the optic nerve

Eleonora Borda, Vivien Gaillet, Diego Ghezzi, Elodie Geneviève Zollinger

IOP PUBLISHING LTD2021

EPFL1995

EPFL2021