Concept

Apache Hadoop

Related publications (33)

Communication trade-offs for Local-SGD with large step size

Synchronous mini-batch SGD is state-of-the-art for large-scale distributed machine learning. However, in practice, its convergence is bottlenecked by slow communication rounds between worker nodes. A natural solution to reduce communication is to use the " ...

NEURAL INFORMATION PROCESSING SYSTEMS (NIPS)2019

Don't cry over spilled records: Memory elasticity of data-parallel applications and its application to cluster scheduling

Willy Zwaenepoel, Calin Iorgulescu, Syed Mohammad Aunn Raza, Florin Dinu, Wajih Ul Hassan

Understanding the performance of data-parallel workloads when resource-constrained has significant practical importance but unfortunately has received only limited attention. This paper identifies, quantifies and demonstrates memory elasticity, an intrinsi ...

2017

Don't cry over spilled records: Memory elasticity of data-parallel applications and its application to cluster scheduling

Willy Zwaenepoel, Calin Iorgulescu, Syed Mohammad Aunn Raza, Florin Dinu, Wajih Ul Hassan

The USENIX Association2017

Optimized Distributed Hyperparameter Search and Simulation for Lung Texture Classification in CT Using Hadoop

Adrien Raphaël Depeursinge

Many medical image analysis tasks require complex learning strategies to reach a quality of image-based decision support that is sufficient in clinical practice. The analysis of medical texture in tomographic images, for example of lung tissue, is no excep ...

2016

Reliable and Real-Time Distributed Abstractions

David Kozhaya

The celebrated distributed computing approach for building systems and services using multiple machines continues to expand to new domains. Computation devices nowadays have additional sensing and communication capabilities, while becoming, at the same tim ...

EPFL2016

Big-Data Streaming Applications Scheduling Based on Staged Multi-armed Bandits

David Atienza Alonso, Karim Kanoun, Cem Tekin

Several techniques have been recently proposed to adapt Big-Data streaming applications to existing many core platforms. Among these techniques, online reinforcement learning methods have been proposed that learn how to adapt at run-time the throughput and ...

Institute of Electrical and Electronics Engineers2016

Big-Data Streaming Applications Scheduling Based on Staged Multi-Armed Bandits

David Atienza Alonso, Karim Kanoun, Cem Tekin

Institute of Electrical and Electronics Engineers2016

SigCO: Mining Significant Correlations via a Distributed Real-time Computation Engine

Karl Aberer, Jean Paul Calbimonte Perez, Tian Guo, Hao Zhuang

The dramatic rise of time-series data produced in a variety of contexts, such as stock markets, mobile sensing, sensor networks, data centre monitoring, etc., has fuelled the development of large-scale distributed real-time computation systems (e.g., Apach ...

2015

Fast Correlation Discovery for Large-Scale Streaming Time-Series Data

Karl Aberer, Saket Sathe, Tian Guo

The dramatic rise of streaming time-series data produced in a vari- ety of contexts, such as stock markets, mobile sensing, sensor net- works, data centre monitoring, etc., has fuelled the development of large-scale distributed real-time computation system ...

2014

RCMP: Enabling Efficient Recomputation Based Failure Resilience for Big Data Analytics

Florin Dinu

Data replication, the main failure resilience strategy used for big data analytics jobs, can be unnecessarily inefficient. It can cause serious performance degradation when applied to intermediate job outputs in multi-job computations. For instance, for I/ ...

2014

Graph Chatbot

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.