Publications related to Softmax function

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Patrick Thiran, Negar Kiyavash, Saber Salehkaleybar

We study the performance of Stochastic Cubic Regularized Newton (SCRN) on a class of functions satisfying gradient dominance property with

1\le\alpha\le2

which holds in a wide range of applications in machine learning and signal processing. This conditio ...

NeurIPS2022

Face Reconstruction Fromdeep Facial Embeddings Using A Convolutional Neural Network

Sébastien Marcel, Hatef Otroshi Shahreza

State-of-the-art (SOTA) face recognition systems generally use deep convolutional neural networks (CNNs) to extract deep features, called embeddings, from face images. The face embeddings are stored in the system's database and are used for recognition of ...

IEEE2022

Memory of Motion for Initializing Optimization in Robotics

Teguh Santoso Lembono

Many robotics problems are formulated as optimization problems. However, most optimization solvers in robotics are locally optimal and the performance depends a lot on the initial guess. For challenging problems, the solver will often get stuck at poor loc ...

EPFL2022

Generalization Properties of NAS under Activation and Skip Connection Search

Volkan Cevher, Grigorios Chrysos, Fanghui Liu, Zhenyu Zhu

Neural Architecture Search (NAS) has fostered the automatic discovery of stateof- the-art neural architectures. Despite the progress achieved with NAS, so far there is little attention to theoretical guarantees on NAS. In this work, we study the generaliza ...

2022

Binary Perceptron: Efficient Algorithms Can Find Solutions in a RareWell-Connected Cluster

Emmanuel Abbé

It was recently shown that almost all solutions in the symmetric binary perceptron are isolated, even at low constraint densities, suggesting that finding typical solutions is hard. In contrast, some algorithms have been shown empirically to succeed in fin ...

ASSOC COMPUTING MACHINERY2022

Impulsive noise removal via a blind CNN enhanced by an iterative post-processing

Seyedeh Sahar Sadrizadeh, Hatef Otroshi Shahreza

In digital imaging, especially in the process of data acquisition and transmission, images are often affected by impulsive noise. Therefore, it is essential to remove impulsive noise from images before any further processing. Due to the remarkable performa ...

ELSEVIER2022

STORM+: Fully Adaptive SGD with Momentum for Nonconvex Optimization

Volkan Cevher, Ali Kavis

In this work we investigate stochastic non-convex optimization problems wherethe objective is an expectation over smooth loss functions, and the goal is to find an approximate stationary point. The most popular approach to handling such problems is varianc ...

2021

Learning Lipschitz-Controlled Activation Functions in Neural Networks for Plug-and-Play Image Reconstruction Methods

Michaël Unser, Dimitris Perdios, Pakshal Narendra Bohra, Alexis Marie Frederic Goujon, Sébastien Alexandre Emery

Ill-posed linear inverse problems are frequently encountered in image reconstruction tasks. Image reconstruction methods that combine the Plug-and-Play (PnP) priors framework with convolutional neural network (CNN) based denoisers have shown impressive per ...

2021

Yapa: Accelerated Proximal Algorithm For Convex Composite Problems

Mireille El Gheche, Giovanni Chierchia

Proximal splitting methods are standard tools for nonsmooth optimization. While primal-dual methods have become very popular in the last decade for their flexibility, primal methods may still be preferred for two reasons: acceleration schemes are more effe ...

IEEE2021

Estimating an extreme Bayesian network via scalings

Mario Krali

A recursive max-linear vector models causal dependence between its components by expressing each node variable as a max-linear function of its parental nodes in a directed acyclic graph and some exogenous innovation. Motivated by extreme value theory, inno ...

ELSEVIER INC2021

Softmax function

Graph Chatbot

Chat with Graph Search

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Face Reconstruction Fromdeep Facial Embeddings Using A Convolutional Neural Network

Memory of Motion for Initializing Optimization in Robotics

Generalization Properties of NAS under Activation and Skip Connection Search

Binary Perceptron: Efficient Algorithms Can Find Solutions in a RareWell-Connected Cluster

Impulsive noise removal via a blind CNN enhanced by an iterative post-processing

STORM+: Fully Adaptive SGD with Momentum for Nonconvex Optimization

Learning Lipschitz-Controlled Activation Functions in Neural Networks for Plug-and-Play Image Reconstruction Methods

Yapa: Accelerated Proximal Algorithm For Convex Composite Problems

Estimating an extreme Bayesian network via scalings

Face Reconstruction Fromdeep Facial Embeddings Using A Convolutional Neural Network

Memory of Motion for Initializing Optimization in Robotics

Learning Lipschitz-Controlled Activation Functions in Neural Networks for Plug-and-Play Image Reconstruction Methods

Binary Perceptron: Efficient Algorithms Can Find Solutions in a RareWell-Connected Cluster

Impulsive noise removal via a blind CNN enhanced by an iterative post-processing

Generalization Properties of NAS under Activation and Skip Connection Search

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Estimating an extreme Bayesian network via scalings

Yapa: Accelerated Proximal Algorithm For Convex Composite Problems

STORM+: Fully Adaptive SGD with Momentum for Nonconvex Optimization