Are you an EPFL student looking for a semester project?
Work with us on data science and visualisation projects, and deploy your project as an app on top of Graph Search.
Concept
Neural architecture search
Summary
Neural architecture search (NAS) is a technique for automating the design of artificial neural networks (ANN), a widely used model in the field of machine learning. NAS has been used to design networks that are on par or outperform hand-designed architectures. Methods for NAS can be categorized according to the search space, search strategy and performance estimation strategy used:
The search space defines the type(s) of ANN that can be designed and optimized.
The search strategy defines the approach used to explore the search space.
The performance estimation strategy evaluates the performance of a possible ANN from its design (without constructing and training it).
NAS is closely related to hyperparameter optimization and meta-learning and is a subfield of automated machine learning (AutoML).
Reinforcement learning (RL) can underpin a NAS search strategy. Barret Zoph and Quoc Viet Le applied NAS with RL targeting the CIFAR-10 dataset and achieved a network architecture that rivals the best manually-designed architecture for accuracy, with an error rate of 3.65, 0.09 percent better and 1.05x faster than a related hand-designed model. On the Penn Treebank dataset, that model composed a recurrent cell that outperforms LSTM, reaching a test set perplexity of 62.4, or 3.6 perplexity better than the prior leading system. On the PTB character language modeling task it achieved bits per character of 1.214.
Learning a model architecture directly on a large dataset can be a lengthy process. NASNet addressed this issue by transferring a building block designed for a small dataset to a larger dataset. The design was constrained to use two types of convolutional cells to return feature maps that serve two main functions when convoluting an input feature map: normal cells that return maps of the same extent (height and width) and reduction cells in which the returned feature map height and width is reduced by a factor of two. For the reduction cell, the initial operation applied to the cell’s inputs uses a stride of two (to reduce the height and width).
Official source
About this result
This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.
Related courses (8)
CS-502: Deep learning in biomedicine
Deep learning offers potential to transform biomedical research. In this course, we will cover recent deep learning methods and learn how to apply these methods to problems in biomedical domain.
PHYS-467: Machine learning for physicists
Machine learning and data analysis are becoming increasingly central in sciences including physics. In this course, fundamental principles and methods of machine learning will be introduced and practi
PHYS-754: Lecture series on scientific machine learning
This lecture presents ongoing work on how scientific questions can be tackled using machine learning. Machine learning enables extracting knowledge from data computationally and in an automatized way.
Related lectures (37)
Neural Networks: Multilayer Learning
Covers the fundamentals of multilayer neural networks and deep learning, including back-propagation and network architectures like LeNet, AlexNet, and VGG-16.
Backpropagation and Neural Networks
Covers the backpropagation algorithm for training neural networks and the representation of functions in multilayer networks.
Deep Learning: Theory and Applications
Explores the mathematics of deep learning, neural networks, and their applications in computer vision tasks, addressing challenges and the need for robustness.
Related publications (159)
Robust NAS under adversarial training: benchmark, theory, and beyond
Volkan Cevher, Grigorios Chrysos, Fanghui Liu, Yongtao Wu
Recent developments in neural architecture search (NAS) emphasize the significance of considering robust architectures against malicious data. However, there is a notable absence of benchmark evaluations and theoretical guarantees for searching these robus ...
2024Aggregating Spatial and Photometric Context for Photometric Stereo
Photometric stereo, a computer vision technique for estimating the 3D shape of objects through images captured under varying illumination conditions, has been a topic of research for nearly four decades. In its general formulation, photometric stereo is an ...
EPFL2024Coronal jets identification using Deep Learning as Image and Video Object Detection
This report presents a study on the development and application of a Region-based Convolutional Neural Network, Faster RCNN and a more complex one, TransVOD, to locate solar coronal jets using data from the Solar Dynamic Observatory (SDO). The study focus ...
2024Related people (34)
Related concepts (1)
Convolutional neural network
Convolutional neural network (CNN) is a regularized type of feed-forward neural network that learns feature engineering by itself via filters (or kernel) optimization. Vanishing gradients and exploding gradients, seen during backpropagation in earlier neural networks, are prevented by using regularized weights over fewer connections. For example, for each neuron in the fully-connected layer 10,000 weights would be required for processing an image sized 100 × 100 pixels.