How close are the eigenvectors and eigenvalues of the sample and actual covariance matrices?

How many samples are sufficient to guarantee that the eigenvectors and eigenvalues of the sample covariance matrix are close to those of the actual covariance matrix? For a wide family of distributions, including distributions with finite second moment and distributions supported in a centered Euclidean ball, we prove that the inner product between eigenvectors of the sample and actual covariance matrices decreases proportionally to the respective eigenvalue distance and the number of samples. Our findings imply non-asymptotic concentration bounds for eigenvectors, eigenspaces, and eigenvalues and carry strong consequences for the nonasymptotic analysis of PCA and its applications. For instance, they provide conditions for separating components estimated from O(1) samples and show that even few samples can be sufficient to perform dimensionality reduction.