Visual Attention and Perceptual Quality in Omnidirectional Imaging

Omnidirectional imaging has reached a level of widespread availability driven by recent advances in integrated circuit technology, image sensors, and computer graphics which allow now capturing, rendering and displaying of such type of immersive content in spatial resolutions sufficient to convey visual information directly to humans, as opposed to its previous use almost solely in computer vision for robotics and surveillance. In addition to its main property of covering full spherical field of view, omnidirectional imaging nowadays is an interactive multimedia; and, when experienced by means of virtual reality head-mounted displays, it achieves a remarkably high level of immersiveness. The paradigm, thus, has shifted toward human consumption of omnidirectional images and video.

Automatic prediction of salient regions in images is a well-developed topic in the field of computer vision. Yet, omnidirectional imaging brings new challenges to this topic, due to a different representation of visual information and additional degrees of freedom available to viewers. Having a model for visual attention in omnidirectional imaging is important to continue research in this subject. We develop such a model for interpreting experimental head-direction trajectories with a goal to construct a visual attention heat-map representing salient regions of an omnidirectional image. The developed model is further used in objective assessment of perceptual visual quality of omnidirectional visual content.

The problem of objectively measuring perceptual quality of omnidirectional visual content arises in many immersive imaging applications; and it is particularly important in compression and delivery. The interactive nature of this type of content limits the performance of earlier methods designed for static images or for video with a predefined dynamic. We aim to address the non-deterministic impact by using a statistical approach. In particular, we attempt to describe and analyze viewer interactions in omnidirectional imaging through estimation of visual attention. We propose an objective metric to measure perceptual quality of omnidirectional visual content considering visual attention information.

Additionally, we explore certain related extensions and applications in omnidirectional imaging. Firstly, we investigate a possible extension to 3+ degrees of freedom by considering an individual case of rendering narrow baseline light~filed images with limited translational interactions. We provide also results of extensive analysis of those iterations, including: circular histograms of directions of user movements, average vectors for a next perspective view, and charts of time spent on a view. Secondly, we look into privacy protection which is yet another field drawing more attention with the advances in image processing, visual and social media. We present a method for protecting user privacy in omnidirectional media, by removing parts of the content selected by the user, in a reversible manner. Results on distinct contents indicate our object removal methodology on the viewport enhances perceived quality, thereby improves privacy protection as the user is able to hide objects with less distortion in the overall image.