ImageNet performance correlates with pose estimation robustness and generalization on out-of-domain data

Neural networks are highly effective tools for pose estimation. However, robustness to outof-domain data remains a challenge, especially for small training sets that are common for real world applications. Here, we probe the generalization ability with three architecture classes (MobileNetV2s, ResNets, and EfficientNets). We developed a novel dataset of 30 horses that allowed for both “within-domain” and “out-of-domain” (unseen horse) benchmarking - this is a crucial test for robustness that current human pose estimation benchmarks do not directly address. We show that better ImageNet-performing architectures perform better on both within- and out-of-domain data if they are first pretrained on ImageNet. Our results demonstrate that transfer learning is beneficial for out-of-domain robustness.