Are you an EPFL student looking for a semester project?
Work with us on data science and visualisation projects, and deploy your project as an app on top of Graph Search.
Lecture
Are you an EPFL student looking for a semester project?
Work with us on data science and visualisation projects, and deploy your project as an app on top of Graph Search.
This lecture covers the scaling of language models, discussing the challenges and benefits of training and deploying large models. It explores the evolution of model sizes over the years, the impact of model size on performance, and the trade-offs between model size, data size, and compute resources. The instructor emphasizes the importance of scaling laws in estimating optimal model and dataset sizes for a fixed compute budget.