Despite the natural parallelism across lookups, performance of distributed key-value stores is often limited due to load imbalance induced by heavy skew in the popularity distribution of the dataset. To avoid violating service level objectives expressed in terms of tail latency, systems tend to keep server utilization low and organize the data in micro-shards, which in turn provides units of migration and replication for the purpose of load balancing. These techniques reduce the skew, but incur additional monitoring, data replication and consistency maintenance overheads. This work shows that the trend towards extreme scale-out will further exacerbate the skew-induced load imbalance, and hence the overhead of migration and replication.
Andreas Mortensen, David Hernandez Escobar, Léa Deillon, Alejandra Inés Slagter, Eva Luisa Vogt, Jonathan Aristya Setyadji
Danick Briand, Nicolas Francis Fumeaux
Anna Fontcuberta i Morral, Alok Rudra, Santhanu Panikar Ramanandan, Joel René Sapera, Vladimir Dubrovskii, Sara Marti Sanchez