Lecture
Scaling Language Models
Description
This lecture covers the scaling of language models, discussing the challenges and benefits of training and deploying large models. It explores the evolution of model sizes over the years, the impact of model size on performance, and the trade-offs between model size, data size, and compute resources. The instructor emphasizes the importance of scaling laws in estimating optimal model and dataset sizes for a fixed compute budget.
Official source
About this result
This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.