Sigmoid function | EPFL Graph Search

A sigmoid function is a mathematical function having a characteristic "S"-shaped curve or sigmoid curve. A common example of a sigmoid function is the logistic function shown in the first figure and defined by the formula: :\sigma(x) = \frac{1}{1 + e^{-x}} = \frac{e^x}{e^x + 1}=1-\sigma(-x). Other standard sigmoid functions are given in the Examples section. In some fields, most notably in the context of artificial neural networks, the term "sigmoid function" is used as an alias for the logistic function. Special cases of the sigmoid function include the Gompertz curve (used in modeling systems that saturate at large values of x) and the ogee curve (used in the spillway of some dams). Sigmoid functions have domain of all real numbers, with return (response) value commonly monotonically increasing but could be decreasing. Sigmoid functions most often show a return value (y axis) in the range 0 to 1. Another commonly used range is from −1 t

Incorporation of Liquid-Crystal Light Valve Non-Linearities in Optical Multilayer Neural Networks

Sigmoidlike activation functions, as available in analog hardware, differ in various ways from the standard sigmoidal function because they are usually asymmetric, truncated, and have a non-standard gain. We present an adaptation of the backpropagation learning rule to compensate for these nonstandard sigmoids. This method is applied to multilayer neural networks with all-optical forward propagation and liquid-crystal light valves (LCLV) as optical thresholding devices. In this paper, the results of simulations of a backpropagation neural network with five different LCLV response curves as activation functions are presented. Although LCLV's perform poorly with the standard backpropagation algorithm, it is shown that our adapted learning rule performs well with these LCLV curves.

The Optical Society of America (OSA)1996

Asymmetric Metal-Dielectric Metacylinders and Their Potential Applications From Engineering Scattering Patterns to Spatial Optical Signal Processing

Romain Christophe Rémy Fleury, Ali Momeni, Mahdi Safari

We propose a bianisotropic hybrid metal-dielectric structure comprising dielectric and metallic cylindrical wedges wherein the composite metacylinder enables advanced control of electric, magnetic, and magnetoelectric resonances. We establish a theoretical framework in which the electromagnetic response of this meta-atom is described through the electric and magnetic multipole moments. The complete dynamic polarizability tensor, expressed in a compact form, is derived as a function of the Mie-scattering coefficients. Further, the constitutive parameters—determined analytically—illustrate the tunability of the structure’s frequency and strength of resonances in light of its high degree of geometric freedom. Flexibility in the design makes the proposed metacylinder a viable candidate for various applications in the microscopic (single meta-atom) and macroscopic (metasurface) levels. We show that the highly versatile bianisotropic meta-atom is amenable to being designed for the desired electromagnetic response, such as electric dipole-free and zero or near-zero (backward and forward) scattering at the microscopic level. In addition, we show that the azimuthal asymmetry gives rise to normal polarizability components, which are vital elements in synthesizing asymmetric optical transfer function at the macroscopic level. We conduct a precise inspection, from the microscopic to the macroscopic level, of the metasurface synthesis for emphasizing on the role of normal polarizability components for spatial optical signal processing. It is shown that this simple two-dimensional asymmetric meta-atom can perform first-order differentiation and edge detection at normal illumination. The results reported herein contribute toward improving the physical understanding of wave interaction with artificial materials composed of asymmetric elongated metal-dielectric inclusions and open the potential of its application in spatial signal and image processing.

2021

Leveraging Spiking Deep Neural Networks to Understand the Neural Mechanisms Underlying Selective Attention

Davide Zambrano

Spatial attention enhances sensory processing of goal-relevant information and improves perceptual sensitivity. Yet, the specific neural mechanisms underlying the effects of spatial attention on performance are still contested. Here, we examine different attention mechanisms in spiking deep convolutional neural networks. We directly contrast effects of precision (internal noise suppression) and two different gain modulation mechanisms on performance on a visual search task with complex real-world images. Unlike standard artificial neurons, biological neurons have saturating activation functions, permitting implementation of attentional gain as gain on a neuron's input or on its outgoing connection. We show that modulating the connection is most effective in selectively enhancing information processing by redistributing spiking activity and by introducing additional task-relevant information, as shown by representational similarity analyses. Precision only produced minor attentional effects in performance. Our results, which mirror empirical findings, show that it is possible to adjudicate between attention mechanisms using more biologically realistic models and natural stimuli.

MIT PRESS2022

Asymmetric Metal-Dielectric Metacylinders and Their Potential Applications From Engineering Scattering Patterns to Spatial Optical Signal Processing

Romain Christophe Rémy Fleury, Ali Momeni, Mahdi Safari

2021