Diablo: A Distributed Analytical Blockchain Benchmark Framework Focusing on Real-World Workloads

The plethora of blockchain proposals raises the question of selecting the ideal blockchain for a given decentralised application (DApp). Most blockchain performance evaluations are obtained through tailored environments under unknown experimental settings documented through whitepapers or blog posts. Unfortunately, the few blockchain benchmark efforts typically generate workloads either from a central location, or, purely based on synthetic request patterns. We propose a benchmark framework, Diablo, to reason about the performance of blockchains under realistic workloads. Diablo is extensible, allowing DApp programmers to define new workloads, and blockchain designers to integrate and test their own blockchain. Diablo currently supports 6 blockchain implementations and 4 built-in workload DApps. Our results indicate that blockchains with lower fault tolerance can achieve higher performance, confirming that speculative execution of transactions might drastically impact performance. These results also show that one implementation can offer significantly better performance than another implementation of the same blockchain protocol.