This lecture covers the concept of polynomial approximation using orthonormal bases and orthogonal projection. It starts with building an orthonormal basis from a naive basis, then introduces the Gram-Schmidt orthonormalization procedure. The lecture also discusses Legendre polynomials and demonstrates orthogonal projection over a specific interval. Finally, it compares the error of the approximation using orthogonal projection with Taylor's series.