Singular Value Decomposition: Theory and Applications

This lecture covers the theory and applications of Singular Value Decomposition (SVD) in computational physics. It starts with Eigenvalue Decomposition (EVD) and then introduces SVD as a more general approach. The geometric interpretation of SVD is explained, showing how a unitary hypersphere is transformed into a hyper ellipsoid. The relation between SVD and EVD is discussed, highlighting the computational strategies for computing SVD. The lecture also demonstrates how to solve linear systems using SVD, providing examples and discussing the least squares solution. The concept of matrix pseudoinverse and its application in solving linear systems are presented, along with the least squares solution via pseudoinverse. The lecture concludes with examples of linear fits and polynomial fits using SVD.