Loud and Trendy: Crowdsourcing Impressions of Social Ambiance in Popular Indoor Urban Places

New research cutting across architecture, urban studies, and psychology is contextualizing the understanding of urban spaces according to the perceptions of their inhabitants. One fundamental construct that relates place and experience is ambiance, which is defined as "the mood or feeling associated with a particular place". We posit that the systematic study of ambiance dimensions in cities is a new domain for which multimedia research can make pivotal contributions. We present a study to examine how images collected from social media can be used for the crowdsourced characterization of indoor ambiance impressions in popular urban places. We design a crowdsourcing framework to understand suitability of social images as data source to convey place ambiance, to examine what type of images are most suitable to describe ambiance, and to assess how people perceive places socially from the perspective of ambiance along 13 dimensions. Our study is based on 50,000 Foursquare images collected from 300 popular places across six cities worldwide. The results show that reliable estimates of ambiance can be obtained for several of the dimensions. Furthermore, we found that most aggregate impressions of ambiance are similar across popular places in all studied cities. We conclude by presenting a multidisciplinary research agenda for future research in this domain. © 2015 ACM.

Computational Analysis of Urban Places Using Mobile Crowdsensing

Darshan Santani

In cities, urban places provide a socio-cultural habitat for people to counterbalance the daily grind of urban life, an environment away from home and work. Places provide an environment for people to communicate, share perspectives, and in the process form new social connections. Due to the active role of places to the social fabric of city life, it is important to understand how people perceive and experience places. One fundamental construct that relates place and experience is ambiance, i.e., the impressions we ubiquitously form when we go out. Young people are key actors of urban life, specially at night, and as such play an equal role in co-creating and appropriating the urban space. Understanding how places and their youth inhabitants interact at night is a relevant urban issue. Until recently, our ability to assess the visual and perceptual qualities of urban spaces and to study the dynamics surrounding youth experiences in those spaces have been limited partly due to the lack of quantitative data. However, the growth of computational methods and tools including sensor-rich mobile devices, social multimedia platforms, and crowdsourcing tools have opened ways to measure urban perception at scale, and to deepen our understanding of nightlife as experienced by young people. In this thesis, as a first contribution, we present the design, implementation and computational analysis of four mobile crowdsensing studies involving youth populations from various countries to understand and infer phenomena related to urban places and people. We gathered a variety of explicit and implicit crowdsourced data including mobile sensor data and logs, survey responses, and multimedia content (images and videos) from hundreds of crowdworkers and thousands of users of mobile social networks. Second, we showed how crowdsensed images can be used for the computational characterization and analysis of urban perception in indoor and outdoor places. For both place types, urban perception impressions were elicited for several physical and psychological constructs using online crowdsourcing. Using low-level and deep learning features extracted from images, we automatically inferred crowdsourced judgments of indoor ambiance with a maximum R2 of 0.53 and outdoor perception with a maximum R2 of 0.49. Third, we demonstrated the feasibility to collect rich contextual data to study the physical mobility, activities, ambiance context, and social patterns of youth nightlife behavior. Fourth, using supervised machine learning techniques, we automatically classified drinking behavior of young people in an urban, real nightlife setting. Using features extracted from mobile sensor data and application logs, we obtained an overall accuracy of 76.7%. While this thesis contributes towards understanding urban perception and youth nightlife patterns in specific contexts, our research also contributes towards the computational understanding of urban places at scale with high spatial and temporal resolution, using a combination of mobile crowdsensing, social media, machine learning, multimedia analysis, and online crowdsourcing.

EPFL2016

Looking at Cities in Mexico with Crowds

Daniel Gatica-Perez, Darshan Santani

Mobile and social technologies are providing new opportunities to document, characterize, and gather impressions of urban environments. In this paper, we present a study that examines urban perceptions of three cities in central Mexico (Guanajuato, Leon and Silao), which integrates a mobile crowdsourcing framework to collect geo-localized images of urban environments by a local youth community, and an online crowdsourcing platform (Amazon Mechanical Turk) to gather impressions of urban environments along twelve physical and psychological dimensions. Our study resulted in a collection of 7,000 geo-localized images containing outdoor scenes and views of each city's built environment, including touristic, historical, and residential neighbourhoods; and 156,000 individual judgments from MTurk. Statistical analyses show that outdoor environments can be reliably assessed with respect to most urban dimensions by the observers of crowdsourced images. Furthermore, a cross-city statistical analysis shows that outdoor urban places in Guanajuato (a touristic, cultural heritage site) are perceived as more quiet, picturesque and interesting compared to places in Leon and Silao, which are commercial and industrial hubs, respectively. In contrast Silao, is perceived to have lower accessibility than Leon. Finally, we investigate whether the perceptions of urban environments vary across different times of the day and found that places in the evening are perceived as less happy, pleasant and preserved, when compared to the same place in the morning. Through the use of collective action, participatory sensing and mobile crowdsourcing, our study engages citizens to understand socio-urban problems in their communities.

ACM2015

InnerView: Learning Place Ambiance from Social Media Images

Daniel Gatica-Perez, Rui Hu, Darshan Santani

In the recent past, there has been interest in characterizing the physical and social ambiance of urban spaces to understand how people perceive and form impressions of these environments based on physical and psychological constructs. Building on our earlier work on characterizing ambiance of indoor places, we present a methodology to automatically infer impressions of place ambiance, using generic deep learning features extracted from images publicly shared on Foursquare. We base our methodology on a corpus of 45,000 images from 300 popular places in six cities on Foursquare. Our results indicate the feasibility to automatically infer place ambiance with a maximum R-2 of 0.53 using features extracted from a pre-trained convolutional neural network. We found that features extracted from deep learning with convolutional nets consistently outperformed individual and combinations of several low-level image features (including Color, GIST, HOG and LBP) to infer all the studied 13 ambiance dimensions. Our work constitutes a first study to automatically infer ambiance impressions of indoor places from deep features learned from images shared on social media.

ACM2016

Computational Analysis of Urban Places Using Mobile Crowdsensing

Darshan Santani

EPFL2016