Optimizing Library Interactions: Functionality and Modularity

This lecture discusses the complexity of modern workloads, focusing on specialized data problems like IOT, OCR, ML, and NLP. It explores the challenges faced by compiler/library authors due to the lack of runtime information, leading to wasted instructions and inefficiencies. The presentation emphasizes the need for optimizing library interactions within DBMS, categorizing external libraries as relational operators, and addressing hardware resource underutilization. It also delves into the importance of strong boundaries between systems to avoid unnecessary checks and conversions. The lecture concludes with insights on instruction-level optimizations and identifying locations to log, highlighting the ongoing work in distributed query planning and storage tiering policy.