Deep Splines: Unifying Framework for Deep Neural Networks

This lecture by the instructor presents a functional framework for implementing and training deep neural networks with free-form activation functions, focusing on adaptive piecewise-linear splines. The lecture covers the variational formulation of inverse problems in imaging, supervised learning as a linear inverse problem, and the emergence of deep learning. It also discusses the use of deep ConvNets for biomedical image reconstruction, the challenges and opportunities of deep splines, and the implementation and training of deep splines. The lecture concludes with a discussion on the equivalence between ReLU and B-spline representations and the practical implications of using B-spline activation modules.