Near Data Processing: Project Vision

This lecture introduces the concept of Near Data Processing (NDP) and its potential to overcome bottleneck issues in disaggregated compute and storage clusters. The instructor discusses the disproportionate bandwidths causing funnel bottlenecks, the scaling opportunities through NDP, existing NDP-aware approaches, challenges in synergetic NDP, and the co-design of storage layouts and data formats. The lecture concludes with the proposed work packages, including the NDP protocol, dynamic operator offloading, synergetic query execution, and the co-optimization of storage layout and data formats.