Lynn Schmittwilken | EPFL Graph Search

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

How crowding challenges (feedforward) convolutional neural networks

Michael Herzog, Ben Henrik Lönnqvist, Gregory Francis, Adrien Christophe Doerig, Alban Bornet, Lynn Schmittwilken

Are (feedforward) convolutional neural networks (CNNs) good models for the human visual system? Here, we used visual crowding as a well-controlled psychophysical test to probe CNNs. Visual crowding is a ubiquitous breakdown of object recognition in the human visual system, whereby targets become jumbled and unrecognisable in the presence of flanking objects. Humans exhibit several well-documented effects of crowding, such as invariance to size, where the size of the target and flanker letters may be changed without impacting the strength of crowding. We show that feedforward CNNs are unable to reproduce invariance to size, confusion between target and flanker identities, and importantly uncrowding, where paradoxically increasing the number of flankers improves performance. We investigate this phenomenon using a recurrent, neurally inspired model called LAMINART, which we find can reproduce uncrowding as observed in humans. Furthermore, we show that capsule networks, a recurrent family of CNNs with grouping and segmentation mechanisms, outperform any other models of uncrowding to date, demonstrating the importance of grouping and segmentation in mechanisms in visual information processing in general.

2021

Capsule networks as recurrent models of grouping and segmentation

Michael Herzog, Adrien Christophe Doerig, Bilge Sayim, Mauro Manassi, Lynn Schmittwilken

Classically, visual processing is described as a cascade of local feedforward computations. Feedforward Convolutional Neural Networks (ffCNNs) have shown how powerful such models can be. However, using visual crowding as a well-controlled challenge, we previously showed that no classic model of vision, including ffCNNs, can explain human global shape processing. Here, we show that Capsule Neural Networks (CapsNets), combining ffCNNs with recurrent grouping and segmentation, solve this challenge. We also show that ffCNNs and standard recurrent CNNs do not, suggesting that the grouping and segmentation capabilities of CapsNets are crucial. Furthermore, we provide psychophysical evidence that grouping and segmentation are implemented recurrently in humans, and show that CapsNets reproduce these results well. We discuss why recurrence seems needed to implement grouping and segmentation efficiently. Together, we provide mutually reinforcing psychophysical and computational evidence that a recurrent grouping and segmentation process is essential to understand the visual system and create better models that harness global shape computations.

2020

Capsule networks explain complex spatial processing

Michael Herzog, Adrien Christophe Doerig, Lynn Schmittwilken

Classically, visual processing is described as a cascade of local feedforward computations. This view has received striking support from the success of convolutional neural networks (CNNs). However, CNNs only roughly mimic human vision. For example, CNNs do not take the global spatial configuration of visual elements into account and thus fail at simple tasks such as explaining crowding and uncrowding. In crowding, the perception of a target deteriorates in the presence of neighboring elements. Classically, adding flanking elements is thought to always decreases performance. However, adding flankers even far away from the target can improve performance, depending on the global configuration (uncrowding). We showed previously that no classic model of crowding, including CNNs, can explain uncrowding. Here, we show that capsule networks, a type of deep network combining CNNs and object segmentation, explain both crowding and uncrowding. We trained capsule networks to recognize targets and groups of shapes. There were no crowding/uncrowding stimuli in the training set. When we subsequently tested the network on crowding/uncrowding stimuli, both crowding and uncrowding occurred. We show theoretically how crowding and uncrowding naturally emerge from neural dynamics in capsule networks. These powerful recurrent models offer a new framework to understand previously unexplained experimental results.

SAGE PUBLICATIONS LTD2019