Implicit Bias in Machine Learning

This lecture explores implicit bias in machine learning, focusing on linearly separable datasets and the implicit bias of gradient descent. It also delves into the implicit bias for non-convex objectives, deep matrix completion, and the practical performance of stochastic gradient descent. The discussion extends to the double descent curve, underparametrized, interpolation threshold, and harmless interpolation in the overparametrized regime. Additionally, it covers stability in optimization algorithms, generalization bounds based on uniform stability, and the stability of stochastic gradient descent. The lecture concludes with an analysis of the effect of the number of iterations on the stability of SGD and the generalization error.