Publications related to Stochastic Zeroth-Order Optimisation Algorithms with Variance Reduction

Deep Learning Theory Through the Lens of Diagonal Linear Networks

In this PhD manuscript, we explore optimisation phenomena which occur in complex neural networks through the lens of

2

-layer diagonal linear networks. This rudimentary architecture, which consists of a two layer feedforward linear network with a diagonal ...

EPFL2024

Robust NAS under adversarial training: benchmark, theory, and beyond

Volkan Cevher, Grigorios Chrysos, Fanghui Liu, Yongtao Wu

Recent developments in neural architecture search (NAS) emphasize the significance of considering robust architectures against malicious data. However, there is a notable absence of benchmark evaluations and theoretical guarantees for searching these robus ...

2024

Scalable constrained optimization

Maria-Luiza Vladarean

Modern optimization is tasked with handling applications of increasingly large scale, chiefly due to the massive amounts of widely available data and the ever-growing reach of Machine Learning. Consequently, this area of research is under steady pressure t ...

EPFL2024

On the Generalization of Stochastic Gradient Descent with Momentum

Volkan Cevher, Kimon Antonakopoulos

While momentum-based accelerated variants of stochastic gradient descent (SGD) are widely used when training machine learning models, there is little theoretical understanding on the generalization error of such methods. In this work, we first show that th ...

Brookline2024

Deep learning approach for identification of H II regions during reionization in 21-cm observations - II. Foreground contamination

Jean-Paul Richard Kneib, Emma Elizabeth Tolley, Tianyue Chen, Michele Bianco

The upcoming Square Kilometre Array Observatory will produce images of neutral hydrogen distribution during the epoch of reionization by observing the corresponding 21-cm signal. However, the 21-cm signal will be subject to instrumental limitations such as ...

Oxford Univ Press2024

Predicting the long-term collective behaviour of fish pairs with deep learning

Francesco Mondada, Alexandre Massoud Alahi, Vaios Papaspyros

Modern computing has enhanced our understanding of how social interactions shape collective behaviour in animal societies. Although analytical models dominate in studying collective behaviour, this study introduces a deep learning model to assess social in ...

2024

Revisiting Character-level Adversarial Attacks for Language Models

Volkan Cevher, Grigorios Chrysos, Fanghui Liu, Yongtao Wu, Elias Abad Rocamora

Adversarial attacks in Natural Language Processing apply perturbations in the character or token levels. Token-level attacks, gaining prominence for their use of gradient-based methods, are susceptible to altering sentence semantics, leading to invalid adv ...

2024

Seeking the new, learning from the unexpected: Computational models of surprise and novelty in the brain

Alireza Modirshanechi

Human babies have a natural desire to interact with new toys and objects, through which they learn how the world around them works, e.g., that glass shatters when dropped, but a rubber ball does not. When their predictions are proven incorrect, such as whe ...

EPFL2024

Few-shot Learning for Efficient and Effective Machine Learning Model Adaptation

Arnout Jan J Devos

Machine learning (ML) enables artificial intelligent (AI) agents to learn autonomously from data obtained from their environment to perform tasks. Modern ML systems have proven to be extremely effective, reaching or even exceeding human intelligence.Althou ...

EPFL2024

A Geometric Unification of Distributionally Robust Covariance Estimators: Shrinking the Spectrum by Inflating the Ambiguity Set

Daniel Kuhn, Viet Anh Nguyen, Yves Rychener

The state-of-the-art methods for estimating high-dimensional covariance matrices all shrink the eigenvalues of the sample covariance matrix towards a data-insensitive shrinkage target. The underlying shrinkage transformation is either chosen heuristically ...

2024

Stochastic Zeroth-Order Optimisation Algorithms with Variance Reduction

Graph Chatbot

Chat with Graph Search

Deep Learning Theory Through the Lens of Diagonal Linear Networks

Robust NAS under adversarial training: benchmark, theory, and beyond

Scalable constrained optimization

On the Generalization of Stochastic Gradient Descent with Momentum

Deep learning approach for identification of H II regions during reionization in 21-cm observations - II. Foreground contamination

Predicting the long-term collective behaviour of fish pairs with deep learning

Revisiting Character-level Adversarial Attacks for Language Models

Seeking the new, learning from the unexpected: Computational models of surprise and novelty in the brain

Few-shot Learning for Efficient and Effective Machine Learning Model Adaptation

A Geometric Unification of Distributionally Robust Covariance Estimators: Shrinking the Spectrum by Inflating the Ambiguity Set

Deep Learning Theory Through the Lens of Diagonal Linear Networks

Robust NAS under adversarial training: benchmark, theory, and beyond

Scalable constrained optimization

A Geometric Unification of Distributionally Robust Covariance Estimators: Shrinking the Spectrum by Inflating the Ambiguity Set

Few-shot Learning for Efficient and Effective Machine Learning Model Adaptation

Deep learning approach for identification of H II regions during reionization in 21-cm observations - II. Foreground contamination

Predicting the long-term collective behaviour of fish pairs with deep learning

On the Generalization of Stochastic Gradient Descent with Momentum

Seeking the new, learning from the unexpected: Computational models of surprise and novelty in the brain

Revisiting Character-level Adversarial Attacks for Language Models