Lecture

Spark SQL: Schema, Optimization, and UDFs

In course

Est dolore eiusmod nostrud quis aliqua. Id officia aute consequat est ut. Mollit quis do labore Lorem laborum aliquip et ea. Sunt officia ipsum anim eiusmod deserunt excepteur veniam irure. Consequat veniam fugiat eu dolore anim.

Description

This lecture covers the second generation of execution models for distributed computing, focusing on Spark SQL. It introduces the Spark Unified Stack, Spark SQL programming interface, data frames, DataFrame operators, and optimization principles. The instructor explains the optimization process, including logical and physical optimization, and the use of Catalyst Rules. Additionally, it delves into user-defined functions (UDFs) and their optimization, highlighting the advantages of Spark SQL over traditional SQL.

This video is available exclusively on Mediaspace for a restricted audience. Please log in to MediaSpace to access it if you have the necessary permissions.

Watch on Mediaspace

Instructor

deserunt Lorem

Lorem enim enim exercitation sunt eiusmod nisi amet amet. Quis esse excepteur anim qui cillum nostrud voluptate eiusmod esse enim excepteur velit consectetur mollit. Non commodo fugiat dolore ullamco incididunt ex duis consectetur quis. Adipisicing quis incididunt occaecat in qui cupidatat reprehenderit tempor id ex et quis aute. Exercitation reprehenderit cillum quis sunt.

Official source

https://mediaspace.epfl.ch/media/0_fi2lo1oz

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related lectures (32)

Graph Chatbot

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.