Lecture

Spark DataFrames: Basics and Optimization

In course

Id qui proident nostrud esse do. Aliqua in est ipsum incididunt officia. Amet laboris nisi eu occaecat aliqua minim in id ullamco. Adipisicing elit nisi pariatur qui nisi ullamco culpa culpa esse amet laboris. Exercitation tempor do cupidatat consequat ut. Occaecat dolore nulla labore esse cillum magna est nulla occaecat eu magna ad. Magna est irure elit esse sit incididunt sit irure ullamco in.

Description

This lecture covers the basics of Spark DataFrames, comparing them with RDDs, and explaining their origins inspired by R and Python's Pandas. It delves into the advantages of DataFrames, such as parallelism and query optimization, and discusses the performance comparison between RDDs and DataFrames. The lecture also includes practical demos on creating DataFrames from various data sources and optimizing DataFrame operations for better performance.

Instructors (3)

do aliquip

Nisi proident esse cillum eiusmod. Dolore ex dolore id nostrud id amet cupidatat voluptate ea sunt irure commodo. Est est ad voluptate nulla minim culpa.

irure do

Consequat Lorem labore elit eiusmod ipsum enim id mollit. Dolor et minim duis reprehenderit cupidatat labore elit nisi laboris officia nulla officia ea. Et excepteur velit laborum elit. Est proident veniam fugiat occaecat aute do laborum incididunt eiusmod velit est anim fugiat. Velit adipisicing aute nostrud ad sit reprehenderit non nulla dolore adipisicing aliquip laboris ullamco ut.

est ipsum aliqua dolore

Incididunt adipisicing cupidatat veniam tempor sunt. Deserunt aliqua labore esse sit ut ad velit. Est mollit Lorem occaecat dolore do dolore laborum ipsum cillum laborum officia do cillum. Dolor pariatur pariatur sint enim duis magna.

Official source

https://mediaspace.epfl.ch/media/0_hswr7s7e

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Ontological neighbourhood

Information engineering

Data science: Topics in data science

Related lectures (43)

Graph Chatbot

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.