Publications associées à Hadoop

Communication trade-offs for Local-SGD with large step size

Synchronous mini-batch SGD is state-of-the-art for large-scale distributed machine learning. However, in practice, its convergence is bottlenecked by slow communication rounds between worker nodes. A natural solution to reduce communication is to use the " ...

NEURAL INFORMATION PROCESSING SYSTEMS (NIPS)2019

Don't cry over spilled records: Memory elasticity of data-parallel applications and its application to cluster scheduling

Willy Zwaenepoel, Calin Iorgulescu, Syed Mohammad Aunn Raza, Florin Dinu, Wajih Ul Hassan

Understanding the performance of data-parallel workloads when resource-constrained has significant practical importance but unfortunately has received only limited attention. This paper identifies, quantifies and demonstrates memory elasticity, an intrinsi ...

The USENIX Association2017

Optimized Distributed Hyperparameter Search and Simulation for Lung Texture Classification in CT Using Hadoop

Adrien Raphaël Depeursinge

Many medical image analysis tasks require complex learning strategies to reach a quality of image-based decision support that is sufficient in clinical practice. The analysis of medical texture in tomographic images, for example of lung tissue, is no excep ...

2016

Reliable and Real-Time Distributed Abstractions

David Kozhaya

The celebrated distributed computing approach for building systems and services using multiple machines continues to expand to new domains. Computation devices nowadays have additional sensing and communication capabilities, while becoming, at the same tim ...

EPFL2016

Big-Data Streaming Applications Scheduling Based on Staged Multi-armed Bandits

David Atienza Alonso, Karim Kanoun, Cem Tekin

Several techniques have been recently proposed to adapt Big-Data streaming applications to existing many core platforms. Among these techniques, online reinforcement learning methods have been proposed that learn how to adapt at run-time the throughput and ...

Institute of Electrical and Electronics Engineers2016

Big-Data Streaming Applications Scheduling Based on Staged Multi-Armed Bandits

David Atienza Alonso, Karim Kanoun, Cem Tekin

Several techniques have been recently proposed to adapt Big-Data streaming applications to existing many core platforms. Among these techniques, online reinforcement learning methods have been proposed that learn how to adapt at run-time the throughput and ...

Institute of Electrical and Electronics Engineers2016

SigCO: Mining Significant Correlations via a Distributed Real-time Computation Engine

Karl Aberer, Jean Paul Calbimonte Perez, Tian Guo, Hao Zhuang

The dramatic rise of time-series data produced in a variety of contexts, such as stock markets, mobile sensing, sensor networks, data centre monitoring, etc., has fuelled the development of large-scale distributed real-time computation systems (e.g., Apach ...

2015

Online indexing and distributed querying model-view sensor data in the cloud

Karl Aberer, Tian Guo, Hao Zhuang

As various kinds of sensors penetrate our daily life (e.g., sensor networks for environmental monitoring, GPS for localization and navigation), the efficient management of massive amount of sensor data becomes increasingly important at present. Many sensor ...

Springer-Verlag Berlin2014

DiNoDB: Efficient Large-Scale Raw Data Analytics

Anastasia Ailamaki, Marko Vukolic, Ioannis Alagiannis, Erietta Liarou, Yongchao Tian

Modern big data workflows, found in e.g., machine learning use cases, often involve iterations of cycles of batch analytics and interactive analytics on temporary data. Whereas batch analytics solutions for large volumes of raw data are well established (e ...

2014

Model-View Sensor Data Management in the Cloud

Karl Aberer, Tian Guo

Infinite nature of sensor data poses a serious challenge for query processing even in a cloud infrastructure. Model-based sensor data approximation reduces the amount of data for query processing, but all modeled segments need to be scanned, in the worst c ...

2013