Are you an EPFL student looking for a semester project?
Work with us on data science and visualisation projects, and deploy your project as an app on top of Graph Search.
Lecture
Are you an EPFL student looking for a semester project?
Work with us on data science and visualisation projects, and deploy your project as an app on top of Graph Search.
This lecture covers the fundamentals of scaling to massive data using Spark, focusing on RDDs, transformations, actions, Spark architecture, and Spark's machine learning toolkit. It explains the challenges of big data, the hardware needed, and the solutions provided by Spark. The lecture also introduces Spark SQL and Spark's Machine Learning Toolkit (MLlib), showcasing examples of logistic regression. It emphasizes the importance of understanding RDDs and DataFrames, and provides insights into Spark's capabilities for handling large-scale data.