Publication

Not All Samples Are Created Equal: Deep Learning with Importance Sampling

Related publications (39)

Graph Chatbot

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Distributed Momentum for Byzantine-resilient Stochastic Gradient Descent

Rachid Guerraoui, El Mahdi El Mhamdi, Sébastien Louis Alexandre Rouault

Byzantine-resilient Stochastic Gradient Descent (SGD) aims at shielding model training from Byzantine faults, be they ill-labeled training datapoints, exploited software/hardware vulnerabilities, or malicious worker nodes in a distributed setting. Two rece ...

2021

We study the consistency of the estimator in spatial regression with partial differential equa-tion (PDE) regularization. This new smoothing technique allows to accurately estimate spatial fields over complex two-dimensional domains, starting from noisy ob ...

2020

Deep Generative Models and Applications

Tatjana Chavdarova

Over the past few years, there have been fundamental breakthroughs in core problems in machine learning, largely driven by advances in deep neural networks. The amount of annotated data drastically increased and supervised deep discriminative models exceed ...

EPFL2020

Dynamical mean-field theory for stochastic gradient descent in Gaussian mixture classification

Florent Gérard Krzakala, Lenka Zdeborová, Francesca Mignacco

We analyze in a closed form the learning dynamics of stochastic gradient descent (SGD) for a single layer neural network classifying a high-dimensional Gaussian mixture where each cluster is assigned one of two labels. This problem provides a prototype of ...

Curran Associates, Inc.2020

Central limit theorems for multilevel Monte Carlo methods

Sebastian Krumscheid, Hakon Andreas Hoel

In this work, we show that uniform integrability is not a necessary condition for central limit theorems (CLT) to hold for normalized multilevel Monte Carlo (MLMC) estimators and we provide near optimal weaker conditions under which the CLT is achieved. In ...

2019

Comment: Models Are Approximations!

Anthony Christopher Davison, Erwan Fabrice Koch, Jonathan Koh Boon Han

This discussion focuses on areas of disagreement with the papers, particularly the target of inference and the case for using the robust 'sandwich' variance estimator in the presence of moderate mis-specification. We also suggest that existing procedures m ...

INST MATHEMATICAL STATISTICS2019

Introduction of optimisation problems in which the objective function is black box or obtaining the gradient is infeasible, has recently raised interest in zeroth-order optimisation methods. As an example finding adversarial examples for Deep Learning mode ...

2019

, ,

In this paper, we develop a stochastic-gradient learning algorithm for situations involving streaming data that arise from an underlying clustered structure. In such settings, the variance of gradient noise can be decomposed into the in-cluster variance si ...

IEEE2019

Local estimation of the global horizontal irradiance using an all-sky camera

Mario Paolone, Fabrizio Sossan, Enrica Scolari

Localised and high-frequency measurements of the global horizontal irradiance (GHI) are a key information to assess the level of power production of distributed photovoltaic generation. The paper presents a supervised machine learning-based procedure to es ...

PERGAMON-ELSEVIER SCIENCE LTD2018

Averaging Stochastic Gradient Descent on Riemannian Manifolds

Nicolas Henri Bernard Flammarion

We propose an estimator for the mean of a random vector in Rd that can be computed in time O(n3.5 + n2d) for n i.i.d. samples and that has error bounds matching the sub-Gaussian case. The only assumptions we make about the data distribution are that it has ...

2018