Analytics on Data at Rest and Data in Motion

This lecture covers the importance of combining data at rest (batch) with data in motion (streaming) in various applications such as financial transactions and web analytics. It explains the Lambda architecture, which allows handling both batch and stream-processing simultaneously. The lecture also delves into the complexities and alternatives of the Lambda architecture, emphasizing the challenges of maintaining two code bases in sync and ensuring consistent data quality. Additionally, it discusses the quality assessment of streams and batches, providing insights into the process of continuously processing data and periodically learning new models. The lecture concludes with a detailed overview of the final project requirements, including building a robust public transport route planner using the SBB dataset.