Publications de Wanyun Xie | EPFL Graph Search

CHAMELEON: A Flexible Data-mixing Framework for Language Model Pretraining and Finetuning

Training data mixtures greatly impact the generalization performance of large language models. Existing domain reweighting methods often rely on costly weight computations and require retraining when new data is introduced. To this end, we introduce a flex ...

2025

Training Deep Learning Models with Norm-Constrained LMOs

Volkan Cevher, Kimon Antonakopoulos, Thomas Michaelsen Pethick, Zhenyu Zhu, Wanyun Xie

In this work, we study optimization methods that leverage the linear minimization oracle (lmo) over a norm-ball. We propose a new stochastic family of algorithms that uses the lmo to adapt to the geometry of the problem and, perhaps surprisingly, show that ...

2025

Improving SAM Requires Rethinking its Optimization Formulation

Volkan Cevher, Kimon Antonakopoulos, Thomas Michaelsen Pethick, Wanyun Xie, Fabian Ricardo Latorre Gomez

This paper rethinks Sharpness-Aware Minimization (SAM), which is originally formulated as a zero-sum game where the weights of a network and a bounded perturbation try to minimize/maximize, respectively, the same differentiable loss. We argue that SAM shou ...

2024

Search for high-mass exclusive diphoton production with tagged protons in proton-proton collisions at √s=13 TeV

Matthias Finger, Qian Wang, Varun Sharma, Konstantin Androsov, Jan Steggemann, Ian Smith, Xin Chen, Rakesh Chawla, Michele De Palma, Matteo Galli

A search is presented for high-mass exclusive diphoton production via photon-photon fusion in proton-proton collisions at root s = 13 TeV in events where both protons survive the interaction. The analysis utilizes data corresponding to an integrated lumino ...

AMER PHYSICAL SOC2024

SAMPa: Sharpness-aware Minimization Parallelized

Volkan Cevher, Thomas Michaelsen Pethick, Wanyun Xie

Sharpness-aware minimization (SAM) has been shown to improve the generalization of neural networks. However, each SAM update requires sequentially computing two gradients, effectively doubling the per-iteration cost compared to base optimizers like SGD. We ...

2024

Mixed Nash for Robust Federated Learning

Volkan Cevher, Thomas Michaelsen Pethick, Wanyun Xie

We study robust federated learning (FL) within a game theoretic framework to alleviate the server vulnerabilities to even an informed adversary who can tailor training-time attacks (Fang et al., 2020; Xie et al., 2020a; Ozfatura et al., 2022; Rodríguez-Bar ...

2024

Stable Nonconvex-Nonconcave Training via Linear Interpolation

Volkan Cevher, Thomas Michaelsen Pethick, Wanyun Xie

This paper presents a theoretical analysis of linear interpolation as a principled method for stabilizing (large-scale) neural network training. We argue that instabilities in the optimization process are often caused by the nonmonotonicity of the loss lan ...

2023