Are you an EPFL student looking for a semester project?
Work with us on data science and visualisation projects, and deploy your project as an app on top of Graph Search.
Lecture
Are you an EPFL student looking for a semester project?
Work with us on data science and visualisation projects, and deploy your project as an app on top of Graph Search.
This lecture covers the evolution of modern Natural Language Processing (NLP) from GPT-2 to GPT-3, focusing on scaling, emergent behavior, in-context learning, chain-of-thought reasoning, and the development of ChatGPT. It discusses the transition from traditional fine-tuning to in-context learning, the significance of large-scale language models, and the impact of prompt engineering and prompt tuning. The instructor emphasizes the emergence of new efficient methods for adapting models to various tasks, including the use of reinforcement learning for instruction tuning. The lecture also explores the implementation of reinforcement learning algorithms like Proximal Policy Optimization (PPO) for model optimization.