Human biases in body measurement estimation

Body measurements, including weight and height, are key indicators of health. Being able to visually assess body measurements reliably is a step towards increased awareness of overweight and obesity and is thus important for public health. Nevertheless it is currently not well understood how accurately humans can assess weight and height from images, and when and how they fail. To bridge this gap, we start from 1,682 images of persons collected from the Web, each annotated with the true weight and height, and ask crowd workers to estimate the weight and height for each image. We conduct a faceted analysis taking into account characteristics of the images as well as the crowd workers assessing the images, revealing several novel findings: (1) Even after aggregation, the crowd's accuracy is overall low. (2) We find strong evidence of contraction bias toward a reference value, such that the weight of light people and the height of short people are overestimated, whereas the weight of heavy people and the height of tall people are underestimated. (3) We estimate workers' individual reference values using a Bayesian model, finding that reference values strongly correlate with workers' own height and weight, indicating that workers are better at estimating people similar to themselves. (4) The weight of tall people is underestimated more than that of short people; yet, knowing the height decreases the weight error only mildly. (5) Accuracy is higher on images of females than of males, but female and male workers are no different in terms of accuracy. (6) Crowd workers improve over time if given feedback on previous guesses. Finally, we explore various bias correction models for improving the crowd's accuracy, but find that this only leads to modest gains. Overall, this work provides important insights on biases in body measurement estimation as obesity-related conditions are on the rise.