Population receptive field size in (un)crowding: to isolate or to combine?

Traditional models posit that visual processing is local and feedforward. In this vein, crowding is explained to be the result of pooling of target and flanker features in early visual areas. Specifically, it has been suggested that when the target and flankers fall within the same receptive field, their features are pooled together, resulting in an irreversible loss of information and degraded performance. A recent fMRI study investigated the relation between crowding strength and population receptive field (pRF) sizes (He et al., 2019). The authors reported that stronger crowding coincides with larger pRF sizes in V2, suggesting that pooling occurs in V2. Here, we investigated pRF size in crowding, no crowding and uncrowding, in which adding flankers leads to improvements in target discrimination, contrary to what traditional models of crowding predict. We replicated previous findings of increased pRF size in crowding as compared to no crowding, with pronounced differences throughout early visual areas (V1 to V4, not only V2). Surprisingly, uncrowding coincides with the smallest pRF size, even compared to the no crowding condition, in which behavioral performance is best. Thus, pRF size is modulated by global context and the relationship between pRF size and behavioral performance is non-monotonic. Our findings suggest that pRF size is modulated by top-down feedback, which goes against the classic, purely feedforward models of crowding. We propose that pRFs are the means by which the brain either combines or isolates the target and the flankers – reflected in behavioral performance as crowding and uncrowding, respectively. [We would like to thank our funding sources: Swiss National Science Foundation (176153, http://p3.snf.ch/Project-176153; NCCR Synapsy, project grant numbers 32003B_135679, 32003B_159780, 324730_192755 and CRSK-3_190185), Leenaards Foundation, ROGER DE SPOELBERCH and Partridge Foundations.]

The role of population receptive field size and recurrent processing in learning to “de-crowd”

Bogdan Draganski, Michael Herzog, Maya Anna Jastrzebowska, Ayberk Ozkirli

In crowding, the perception of a target is impeded by surrounding clutter. While traditional models are feedforward and local, there is increasing behavioral and neural evidence for a critical role of recurrent processing across the visual hierarchy in crowding. Our recent fMRI findings suggest that higher visual areas, sensitive to global context, determine whether target and flanker features are combined within a single population receptive field (pRF) or rather the target is separated from the flankers in a smaller pRF, likely through recurrent processing. Here, we investigated the neural correlates of perceptual learning to “de-crowd”. Twelve participants trained on an orientation discrimination task under crowding, with the stimulus always presented in the top-right quadrant. Performance on the crowding condition improved substantially in the trained quadrant, but only to a lesser extent in an untrained quadrant. No significant improvements were seen in a no-crowding condition. We used fMRI to estimate pRF sizes in visual areas V1 to V4, separately in areas representing the four visual quadrants. We used dynamic causal modeling to compare bottom-up, top-down, and recurrent models of learning-related modulation of connectivity between visual areas. We found a substantial decrease in pRF size only in left ventral V3, corresponding to the trained visual quadrant. Learning to de-crowd modulated recurrent connectivity across the visual hierarchy, whereas only bottom-up connectivity was modulated in no-crowding. Our findings suggest that improvements in target discrimination under crowding are mediated by top-down processing, which determines the segmentation of the target and flankers through pRF size adjustment. [We would like to thank our funding sources: Swiss National Science Foundation (NCCR Synapsy, project grant numbers 32003B_135679, 32003B_159780, 324730_192755, CRSK-3_190185, 176153), the Leenaards Foundation, Fondation ROGER DE SPOELBERCH, and the Partridge Foundation.]

SAGE Publishing2021

Crowding and the importance of grouping and segmentation processes in human vision

Alban Bornet

Human vision has evolved to make sense of a world in which elements almost never appear in isolation. Surprisingly, the recognition of an element in a visual scene is strongly limited by the presence of other nearby elements, a phenomenon known as visual crowding. Crowding impacts vision at all levels and is thus a versatile tool to understand the fundamental mechanisms of vision. For decades, visual crowding was perfectly well explained by traditional feedforward models of vision. In these models, vision starts with the detection of low-level features. This information is combined locally along the hierarchy of the visual cortex to build more and more complex feature detectors, until neurons respond selectively and robustly to complex objects. Crowding happens when nearby elements interfere in this local feature combination process and impair target recognition. However, recent studies have shown that crowding is not determined by local interactions but by the global configuration across the entire visual field. Depending on how elements group together, crowding can even almost disappear, a phenomenon called uncrowding. Hence, crowding is rather a complex, global and high-level phenomenon, that simple feedforward models cannot explain. In this thesis, I first analyse which models of crowding can explain uncrowding. I compare the performance of diverse models, selected according to different architectural and functional features, such as feedforward vs. recurrent architecture, local or global information processing, including a grouping stage or not. I show that the only model that reproduces human behaviour includes a dedicated recurrent grouping processing stage. Second, I show that global effects in crowding cannot be explained by low-level accounts. It was argued that the Texture Tiling model, based on a complex and high-dimensional pooling stage, may account for global effects in crowding, without requiring any recurrent grouping stage. To test this model, I use a large pool of recent crowding data. I show that the Texture Tiling model is equivalent to a simple pooling model and is thus as limited as these models. Next, I focus on deep neural networks, which are well in the spirit of the feedforward framework of vision and have become state-of-the-art models both in computer vision and neuroscience. I test whether AlexNet and ResNet-50, which have been proposed as realistic models of the visual system, exhibit uncrowding. I show that these networks do not reproduce uncrowding for principled reasons. Finally, I use a genetic algorithm to generate stimuli based on the performance of different models, i.e., in a bottom-up manner. The goal is to avoid using stimuli that favour models of grouping from the start. I compare the distribution of stimuli that are produced by the models to the ones that are produced by humans. I show that only the models that include grouping and segmentations processes behave like humans. Taken together, the results in my thesis highlight the importance of recurrent grouping and segmentation processes in human vision when large portions of the visual field are involved. These results can be used as direct guidelines for future models of vision, in order to constraint how recurrent processing should be incorporated to improve the performance of deep neural networks and other feedforward models of vision, and help them generalize to more complex visual inputs.

EPFL2021

Biologically plausible unsupervised learning in shallow and deep neural networks

Bernd Albert Illing

The way our brain learns to disentangle complex signals into unambiguous concepts is fascinating but remains largely unknown. There is evidence, however, that hierarchical neural representations play a key role in the cortex. This thesis investigates biologically plausible models of unsupervised learning of hierarchical representations as found in the brain and modern computer vision models. We use computational modeling to address three main questions at the intersection of artificial intelligence (AI) and computational neuroscience.The first question is: What are useful neural representations and when are deep hierarchical representations needed? We approach this point with a systematic study of biologically plausible unsupervised feature learning in a shallow 2-layer networks on digit (MNIST) and object (CIFAR10) classification. Surprisingly, random features support high performance, especially for large hidden layers. When combined with localized receptive fields, random feature networks approach the performance of supervised backpropagation on MNIST, but not on CIFAR10. We suggest that future models of biologically plausible learning should outperform such random feature benchmarks on MNIST, or that such models should be evaluated in different ways.The second question is: How can hierarchical representations be learned with mechanisms supported by neuroscientific evidence? We cover this question by proposing a unifying Hebbian model, inspired by common models of V1 simple and complex cells based on unsupervised sparse coding and temporal invariance learning. In shallow 2-layer networks, our model reproduces learning of simple and complex cell receptive fields, as found in V1. In deeper networks, we stack multiple layers of Hebbian learning but find that it does not yield hierarchical representations of increasing usefulness. From this, we hypothesise that standard Hebbian rules are too constrained to build increasingly useful representations, as observed in higher areas of the visual cortex or deep artificial neural networks.The third question is: Can AI inspire learning models that build deep representations and are still biologically plausible? We address this question by proposing a learning rule that takes inspiration from neuroscience and recent advances in self-supervised deep learning. The proposed rule is Hebbian, i.e. only depends on pre- and post-synaptic neuronal activity, but includes additional local factors, namely predictive dendritic input and widely broadcasted modulation factors. Algorithmically, this rule applies self-supervised contrastive predictive learning to a causal, biological setting using saccades. We find that networks trained with this generalised Hebbian rule build deep hierarchical representations of images, speech and video.We see our modeling as a potential starting point for both, new hypotheses, that can be tested experimentally, and novel AI models that could benefit from added biological realism.

EPFL2021

Biologically plausible unsupervised learning in shallow and deep neural networks

Bernd Albert Illing

EPFL2021

Crowding and the importance of grouping and segmentation processes in human vision

Alban Bornet

EPFL2021

The role of population receptive field size and recurrent processing in learning to “de-crowd”

Bogdan Draganski, Michael Herzog, Maya Anna Jastrzebowska, Ayberk Ozkirli

SAGE Publishing2021