Learning Policies from Non-Traditional Sources

This lecture explores the concept of learning interactive policies from non-traditional sources of data, focusing on intelligent and interactive autonomous systems. Topics covered include learning latent actions from state-action pairs, language disambiguation, and the use of language-informed latent actions. The lecture delves into the challenges of instruction following in natural language interaction, the development of assistive teleoperation methods, and the implementation of Language-Informed Latent Actions (LILA) for intuitive control mechanisms. Additionally, it discusses the PLATO framework for predicting latent affordances through object-centric pre-interaction and post-interaction phases, showcasing experiments in 2D and 3D manipulation tasks.