Venues in Social Media: Examining Ambiance Perception Through Scene Semantics

Yassir Benkhedda, Daniel Gatica-Perez, Darshan Santani
2017
Conference paper

Abstract

We address the question of what visual cues, including scene objects and demographic attributes, contribute to the automatic inference of perceived ambiance in social media venues. We first use a stateof- art, deep scene semantic parsing method and a face attribute extractor to understand how different cues present in a scene relate to human perception of ambiance on Foursquare images of social venues.We then analyze correlational links between visual cues and thirteen ambiance variables, as well as the ability of the semantic attributes to automatically infer place ambiance. We study the effect of the type and amount of image data used for learning, and compare regression results to previous work, showing that the proposed approach results in marginal-to-moderate performance increase for up to ten of the ambiance dimensions, depending on the corpus.

Official source

https://infoscience.epfl.ch/record/231121?ln=en

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related concepts

Related publications

InnerView: Learning Place Ambiance from Social Media Images

Daniel Gatica-Perez, Rui Hu, Darshan Santani

In the recent past, there has been interest in characterizing the physical and social ambiance of urban spaces to understand how people perceive and form impressions of these environments based on physical and psychological constructs. Building on our earlier work on characterizing ambiance of indoor places, we present a methodology to automatically infer impressions of place ambiance, using generic deep learning features extracted from images publicly shared on Foursquare. We base our methodology on a corpus of 45,000 images from 300 popular places in six cities on Foursquare. Our results indicate the feasibility to automatically infer place ambiance with a maximum R-2 of 0.53 using features extracted from a pre-trained convolutional neural network. We found that features extracted from deep learning with convolutional nets consistently outperformed individual and combinations of several low-level image features (including Color, GIST, HOG and LBP) to infer all the studied 13 ambiance dimensions. Our work constitutes a first study to automatically infer ambiance impressions of indoor places from deep features learned from images shared on social media.

ACM2016

, ,

We study the perception of ambiance of places captured in social media images by both machines and crowdworkers. This task is challenging due to the subjective nature of the ambiance construct as well as the large variety in layout, style, and visual characteristics of venues. For machine recognition of ambiance, we use state-of-the-art Residual Deep Convolutional Neural Networks (ResNets), followed by gradient-weighted class activation mapping (Grad-CAM) visualizations. This form of visual explanation obtained from the trained ResNet-50 models were assessed by crowdworkers based on a carefully designed crowdsourcing task, in which both visual ambiance cues of venues and subjective assessment of Grad-CAM results were collected and analyzed. The results show that paintings, photos, and decorative items are strong cues for artsy ambiance, whereas type of utensils, type of lamps and presence of flowers may indicate formal ambiance. Layout and design-related cues such as type of chairs, type of tables/tablecloth and type of windows are noted to have impact for both ambiances. Overall, the ambiance visual cue recognition results are promising, and the crowd-based assessment approach may motivate other studies on subjective perception of place attributes.

IEEE2018