Publications by Kaicheng Yu | EPFL Graph Search

An Analysis of Super-Net Heuristics in Weight-Sharing NAS

Weight sharing promises to make neural architecture search (NAS) tractable even on commodity hardware. Existing methods in this space rely on a diverse set of heuristics to design and train the shared-weight backbone network, a.k.a. the super-net. Since he ...

IEEE COMPUTER SOC2022

From Human-Designed Convolutional Neural Networks Towards Robust Neural Architecture Search

Kaicheng Yu

Artificial intelligence has been an ultimate design goal since the inception of computers decades ago. Among the many attempts towards general artificial intelligence, modern machine learning successfully tackles many complex problems thanks to the progres ...

EPFL2021

Pyramid Architecture Search for Real-Time Image Deblurring

Wei Liu, Kaicheng Yu

Multi-scale and multi-patch deep models have been shown effective in removing blurs of dynamic scenes. However, these methods still suffer from one major obstacle: manually designing a lightweight and high-efficiency network is challenging and time-consumi ...

IEEE2021

Landmark Regularization: Ranking Guided Super-Net Training in Neural Architecture Search

Mathieu Salzmann, Kaicheng Yu

Weight sharing has become a de facto standard in neural architecture search because it enables the search to be done on commodity hardware. However, recent works have empirically shown a ranking disorder between the performance of stand-alone architectures ...

IEEE COMPUTER SOC2021

Evaluating the search phase of neural architecture search

Mathieu Salzmann, Martin Jaggi, Claudiu-Cristian Musat, Kaicheng Yu

Neural Architecture Search (NAS) aims to facilitate the design of deep networks fornew tasks. Existing techniques rely on two stages: searching over the architecture space and validating the best architecture. NAS algorithms are currently compared solely b ...

2020

Overcoming Multi-model Forgetting

Mathieu Salzmann, Anthony Christopher Davison, Martin Jaggi, Claudiu-Cristian Musat, Kaicheng Yu, Yassine Benyahia

We identify a phenomenon, which we refer to as multi-model forgetting, that occurs when sequentially training multiple deep networks with partially-shared parameters; the performance of previously-trained models degrades as one optimizes a subsequent one, ...

JMLR2019

Recurrent U-Net for Resource-Constrained Segmentation

Pascal Fua, Wei Wang, Mathieu Salzmann, Joachim Ludovic Hugonot, Kaicheng Yu

State-of-the-art segmentation methods rely on very deep networks that are not always easy to train without very large training datasets and tend to be relatively slow to run on standard GPUs. In this paper, we introduce a novel recurrent U-Net architecture ...

IEEE COMPUTER SOC2019

Statistically-Motivated Second-order Pooling

Mathieu Salzmann, Kaicheng Yu

Second-order pooling, a.k.a. bilinear pooling, has proven effective for deep learning based visual recognition. However, the resulting second-order networks yield a final representation that is orders of magnitude larger than that of standard, first-order ...

2018