Lecture
Data Stream Processing: Apache Kafka and Spark
Related lectures (46)
Introduction to Data Stream Processing
Covers the fundamentals of data stream processing, including tools like Apache Storm and Kafka, key concepts like event time and window operations, and the challenges of stream processing.
Spark Data Frames
Covers Spark Data Frames, distributed collections of data organized into named columns, and the benefits of using them over RDDs.
Analytics on Data at Rest and Data in Motion
Explores combining data at rest with data in motion, emphasizing the Lambda architecture complexities and quality assessment of streams and batches.
General Introduction to Big Data
Covers data science tools, Hadoop, Spark, data lake ecosystems, CAP theorem, batch vs. stream processing, HDFS, Hive, Parquet, ORC, and MapReduce architecture.
Elements of Collaborative Data Science
Introduces collaborative data science tools like Jupyter notebooks, Docker, and Git, emphasizing data versioning and containerization.
General Introduction to Data Science
Offers a comprehensive introduction to Data Science, covering Python, Numpy, Pandas, Matplotlib, and Scikit-learn, with a focus on practical exercises and collaborative work.
Data Wrangling with Hadoop
Covers data wrangling techniques using Hadoop, focusing on row versus column-oriented databases, popular storage formats, and HBase-Hive integration.
Advanced Data Stream Processing Concepts
Explores event time vs. processing time, stream processing operations, stream-stream joins, and handling late/out-of-order data in data stream processing.
Data Wrangling with Hadoop: Storage Formats and Hive
Explores data wrangling with Hadoop, emphasizing storage formats and Hive for big data processing.
Introduction to Spark Runtime Architecture
Covers the Spark runtime architecture, including RDDs, transformations, actions, and caching for performance optimization.