Guillermo Ortiz Jimenez | EPFL Graph Search

I received my BSc. in telecommunications engineering (Valedictorian) from Universidad Politecnica de Madrid (UPM), Spain, in June 2015, and my MSc. in electrical engineering (Best Graduate, cum laude) from Delft University of Technology (TU Delft), The Netherlands, in August 2018. During my studies, I held positions at the Microwaves and Radar Group of UPM during 2015-2016 and at Philips Healthcare Research in Hamburg (Germany) during summer 2017. I am now working towards my PhD. thesis at the Signal Processing Laboratory 4 under the supervision of Prof. Pascal Frossard. My current research focuses on understanding deep learning by studying the complex interactions between datasets, architectures and optimization. In my work, I am providing a novel framework to design better and more reliable neural networks that exploit prior knowledge about the world. I am the recipient of a National Award for Excellence in Academic Performance from the Ministry of Education of Spain, a la Caixa Postgraduate Fellowship, and the Best Graduate award both at UPM and TU Delft.

A Structured Dictionary Perspective on Implicit Neural Representations

Beril Besbinar, Pascal Frossard, Guillermo Ortiz Jimenez, Gizem Yüce

Implicit neural representations (INRs) have recently emerged as a promising alternative to classical discretized representations of signals. Nevertheless, despite their practical success, we still do not understand how INRs represent signals. We propose a novel unified perspective to theoretically analyse INRs. Leveraging results from harmonic analysis and deep learning theory, we show that most INR families are analogous to structured signal dictionaries whose atoms are integer harmonics of the set of initial mapping frequencies. This structure allows INRs to express signals with an exponentially increasing frequency support using a number of parameters that only grows linearly with depth. We also explore the inductive bias of INRs exploiting recent results about the empirical neural tangent kernel (NTK). Specifically, we show that the eigenfunctions of the NTK can be seen as dictionary atoms whose inner product with the target signal determines the final performance of their reconstruction. In this regard, we reveal that meta-learning has a reshaping effect on the NTK analogous to dictionary learning, building dictionary atoms as a combination of the examples seen during meta-training. Our results permit to design and tune novel INR architectures, but can also be of interest for the wider deep learning theory community.

IEEE COMPUTER SOC2022

PRIME: A Few Primitives Can Boost Robustness to Common Corruptions

Pascal Frossard, Apostolos Modas, Seyed Mohsen Moosavi Dezfooli, Guillermo Ortiz Jimenez

Despite their impressive performance on image classification tasks, deep networks have a hard time generalizing to unforeseen corruptions of their data. To fix this vulnerability, prior works have built complex data augmentation strategies, combining multiple methods to enrich the training data. However, introducing intricate design choices or heuristics makes it hard to understand which elements of these methods are indeed crucial for improving robustness. In this work, we take a step back and follow a principled approach to achieve robustness to common corruptions. We propose PRIME, a general data augmentation scheme that relies on simple yet rich families of max-entropy image transformations. PRIME outperforms the prior art in terms of corruption robustness, while its simplicity and plug-and-play nature enable combination with other methods to further boost their robustness. We analyze PRIME to shed light on the importance of the mixing strategy on synthesizing corrupted images, and to reveal the robustness-accuracy trade-offs arising in the context of common corruptions. Finally, we show that the computational efficiency of our method allows it to be easily used in both on-line and off-line data augmentation schemes. Our code is available at https://github.com/amodas/PRIME-augmentations.

SPRINGER INTERNATIONAL PUBLISHING AG2022

What can linearized neural networks actually say about generalization?

Pascal Frossard, Seyed Mohsen Moosavi Dezfooli, Guillermo Ortiz Jimenez

For certain infinitely-wide neural networks, the neural tangent kernel (NTK) theory fully characterizes generalization, but for the networks used in practice, the empirical NTK only provides a rough first-order approximation. Still, a growing body of work keeps leveraging this approximation to successfully analyze important deep learning phenomena and design algorithms for new applications. In our work, we provide strong empirical evidence to determine the practical validity of such approximation by conducting a systematic comparison of the behavior of different neural networks and their linear approximations on different tasks. We show that the linear approximations can indeed rank the learning complexity of certain tasks for neural networks, even when they achieve very different performances. However, in contrast to what was previously reported, we discover that neural networks do not always perform better than their kernel approximations, and reveal that the performance gap heavily depends on architecture, dataset size and training task. We discover that networks overfit to these tasks mostly due to the evolution of their kernel during training, thus, revealing a new type of implicit bias.

2021