Lakmal Buddika Meegahapola

Generalization and Personalization of Mobile Sensing-Based Mood Inference Models: An Analysis of College Students in Eight Countries

Daniel Gatica-Perez, Lakmal Buddika Meegahapola

Mood inference with mobile sensing data has been studied in ubicomp literature over the last decade. This inference enables context-aware and personalized user experiences in general mobile apps and valuable feedback and interventions in mobile health apps. However, even though model generalization issues have been highlighted in many studies, the focus has always been on improving the accuracies of models using different sensing modalities and machine learning techniques, with datasets collected in homogeneous populations. In contrast, less attention has been given to studying the performance of mood inference models to assess whether models generalize to new countries. In this study, we collected a mobile sensing dataset with 329K self-reports from 678 participants in eight countries (China, Denmark, India, Italy, Mexico, Mongolia, Paraguay, UK) to assess the effect of geographical diversity on mood inference models. We define and evaluate country-specific (trained and tested within a country), continent-specific (trained and tested within a continent), country-agnostic (tested on a country not seen on training data), and multi-country (trained and tested with multiple countries) approaches trained on sensor data for two mood inference tasks with population-level (non-personalized) and hybrid (partially personalized) models. We show that partially personalized country-specific models perform the best yielding area under the receiver operating characteristic curve (AUROC) scores of the range 0.78-0.98 for two-class (negative vs. positive valence) and 0.76-0.94 for three-class (negative vs. neutral vs. positive valence) inference. Further, with the country-agnostic approach, we show that models do not perform well compared to country-specific settings, even when models are partially personalized. We also show that continent-specific models outperform multi-country models in the case of Europe. Overall, we uncover generalization issues of mood inference models to new countries and how the geographical similarity of countries might impact mood inference.

ASSOC COMPUTING MACHINERY2022

Sensing Eating Events in Context: A Smartphone-Only Approach

Daniel Gatica-Perez, Lakmal Buddika Meegahapola

While the task of automatically detecting eating events has been examined in prior work using various wearable devices, the use of smartphones as standalone devices to infer eating events remains an open issue. This paper proposes a framework that infers eating vs. non-eating events from passive smartphone sensing and evaluates it on a dataset of 58 college students. First, we show that time of the day and features from modalities such as screen usage, accelerometer, app usage, and location are indicative of eating and non-eating events. Then, we show that eating events can be inferred with an AUROC (area under the receiver operating characteristics curve) of 0.65 using subject-independent machine learning models, which can be further improved up to 0.81 for subject-dependent and 0.81 for hybrid models using personalization techniques. Moreover, we show that users have different behavioral and contextual routines around eating episodes requiring specific feature groups to train fully personalized models. These findings are of potential value for future mobile food diary apps that are context-aware by enabling scalable sensing-based eating studies using only smartphones; detecting under-reported eating events, thus increasing data quality in self report-based studies; providing functionality to track food consumption and generate reminders for on-time collection of food diaries; and supporting mobile interventions towards healthy eating practices.

2022

Smartphone Sensing for the Well-being of Young Adults: A Review

Daniel Gatica-Perez, Lakmal Buddika Meegahapola

Over the years, mobile phones have become versatile devices with a multitude of capabilities due to the plethora of embedded sensors that enable them to capture rich data unobtrusively. In a world where people are more conscious regarding their health and well-being, the pervasiveness of smartphones has enabled researchers to build apps that assist people to live healthier lifestyles, and to diagnose and monitor various health conditions. Motivated by the high smartphone coverage among young adults and the unique issues they face, in this review paper, we focus on studies that have used smartphone sensing for the well-being of young adults. We analyze existing work in the domain from two perspectives, namely Data Perspective and System Perspective. For both these perspectives, we propose taxonomies motivated from human science literature, which enable to identify important study areas. Furthermore, we emphasize the importance of diversity-awareness in smartphone sensing, and provide insights and future directions for researchers in ubiquitous and mobile computing, and especially to new researchers who want to understand the basics of smartphone sensing research targeting the well-being of young adults.

2021

Ten seconds of my nights: Exploring methods to measure brightness, loudness and attendance and their associations with alcohol use from video clips

Daniel Gatica-Perez, Lakmal Buddika Meegahapola, Skanda Muralidhar

Introduction Most evidence on associations between alcohol use behaviors and the characteristics of its social and physical context is based on self-reports from study participants and, thus, only account for their subjective impressions of the situation. This study explores the feasibility of obtaining alternative measures of loudness, brightness, and attendance (number of people) using 10-second video clips of real-life drinking occasions rated by human annotators and computer algorithms, and explores the associations of these measures with participants' choice to drink alcohol or not. Methods Using a custom-built smartphone application, 215 16-25-year-olds documented characteristics of 2,380 weekend night drinking events using questionnaires and videos. Ratings of loudness, brightness, and attendance were obtained from three sources, namely in-situ participants' ratings, video-based annotator ratings, and video-based computer algorithm ratings. Bivariate statistics explored differences in ratings across sources. Multilevel logistic regressions assessed the associations of contextual characteristics with alcohol use. Finally, model fit indices and cross-validation were used to assess the ability of each set of contextual measures to predict participants' alcohol use. Results Raw ratings of brightness, loudness and attendance differed slightly across sources, but were all correlated (r = .21 to .82, all p < .001). Participants rated bars/pubs as being louder (Cohen's d = 0.50 [95%-CI: 0.07-0.92]), and annotators rated private places as darker (d = 1.21 [95%-CI: 0.99-1.43]) when alcohol was consumed than when alcohol was not consumed. Multilevel logistic regressions showed that drinking in private places was more likely in louder (ORparticipants = 1.74 [CI: 1.31-2.32]; ORannotators = 3.22 [CI: 2.06-5.03]; ORalgorithm = 2.62 [CI: 1.83-3.76]), more attended (ORparticipants = 1.10 [CI: 1.03-1.18]; ORalgorithm = 1.19 [CI: 1.07-1.32]) and darker (OR = 0.64 [CI: 0.44-0.94]) situations. In commercial venues, drinking was more likely in darker (ORparticipants = 0.67 [CI: 0.47-0.94]; ORannotators = 0.53 [CI: 0.33-0.85]; ORalgorithm = 0.58 [CI: 0.37-0.88]) and louder (ORparticipants = 1.40 [CI: 1.02-1.92]; ORalgorithm = 2.45 [CI: 1.25-4.80]) places. Higher inference accuracies were found for the models based on the annotators' ratings (80% to 84%) and the algorithms' ratings (76% to 86%) than on the participants' ratings (69% to 71%). Conclusions Several contextual characteristics are associated with increased odds of drinking in private and commercial settings, and might serve as a basis for the development of prevention measures. Regarding assessment of contextual characteristics, annotators and algorithms might serve as appropriate substitutes of participants' in-situ impressions for correlational and regression analyses despite differences in raw ratings. Collecting contextual data by means of sensors or media files is recommended for future research.

PUBLIC LIBRARY SCIENCE2021

The Theory, Practice, and Ethical Challenges of Designing a Diversity-Aware Platform for Social Relations

Daniel Gatica-Perez, Lakmal Buddika Meegahapola

Diversity-aware platform design is a paradigm that responds to the ethical challenges of existing social media platforms. Available platforms have been criticized for minimizing users' autonomy, marginalizing minorities, and exploiting users' data for profit maximization. This paper presents a design solution that centers the well-being of users. It presents the theory and practice of designing a diversity-aware platform for social relations. In this approach, the diversity of users is leveraged in a way that allows like-minded individuals to pursue similar interests or diverse individuals to complement each other in a complex activity. The end users of the envisioned platform are students, who participate in the design process. Diversity-aware platform design involves numerous steps, of which two are highlighted in this paper: 1) defining a framework and operationalizing the "diversity" of students, 2) collecting "diversity" data to build diversity-aware algorithms. The paper further reflects on the ethical challenges encountered during the design of a diversity-aware platform.

ASSOC COMPUTING MACHINERY2021

Alone or With Others? Understanding Eating Episodes of College Students with Mobile Sensing

Daniel Gatica-Perez, Lakmal Buddika Meegahapola

Understanding food consumption patterns and contexts using mobile sensing is fundamental to build mobile health applications that require minimal user interaction to generate mobile food diaries. Many available mobile food diaries, both commercial and in research, heavily rely on self-reports, and this dependency limits the long term adoption of these apps by people. The social context of eating (alone, with friends, with family, with a partner, etc.) is an important self-reported feature that influences aspects such as food type, psychological state while eating, and the amount of food, according to prior research in nutrition and behavioral sciences. In this work, we use two datasets regarding the everyday eating behavior of college students in two countries, namely Switzerland (Nch=122) and Mexico (Nmx=84), to examine the relation between the social context of eating and passive sensing data from wearables and smartphones. Moreover, we design a classification task, namely inferring eating-alone vs. eating-with-others episodes using passive sensing data and time of eating, obtaining accuracies between 77% and 81%. We believe that this is a first step towards understanding more complex social contexts related to food consumption using mobile sensing.

Association for Computing Machinery2020

Jointly Optimizing Sensing Pipelines for Multimodal Mixed Reality Interaction

Lakmal Buddika Meegahapola

Natural human interactions for Mixed Reality Applications are overwhelmingly multimodal: humans communicate intent and instructions via a combination of visual, aural and gestural cues. However, supporting low-latency and accurate comprehension of such multimodal instructions (MMI), on resource-constrained wearable devices, remains an open chal-lenge, especially as the stale-of-the-art comprehension techniques for each individual modality increasingly utilize complex Deep Neural Network models. We demonstrate the possibility of overcoming the core limitation of latency-vs.-accuracy tradeoff by exploiting cross-modal dependencies-i.e., by compensating for the interior performance of one model with an increased accuracy of more complex model of a different modality. We present a sensor fusion architecture that performs MMI comprehension in a quasi-synchronous fashion, by fusing visual, speech and gestural input. The architecture is reconfigurable and supports dynamic modification of the complexity of the data processing pipeline for each individual modality in response to contextual changes. Using a representative "classroom" context and a set of tour conunon interaction primitives, we then demonstrate how the choices between low and high complexity models for each individual modality are coupled. In particular, we show that (a) a judicious combination of low and high complexity models across modalities can offer a dramatic 3-fold decrease in comprehension latency together with an increase similar to 10-15% in accuracy, and (b) the right collective choice of models is context dependent, with the performance of some model combinations being significantly more sensitive to changes in scene context or choice of interaction.

IEEE COMPUTER SOC2020

PokeME: Applying Context-Driven Notifications to Increase Worker Engagement in Mobile Crowd-Sourcing

Lakmal Buddika Meegahapola

In mobile crowd-sourcing systems, simply relying on people to opportunistically select and perform tasks typically leads to drawbacks such as low task acceptance/completion rates and undesirable spatial skews. In this paper, we utilize data from TASKer, a campus-based mobile crowd-sourcing platform, to empirically study and discover whether and how various context-aware notification strategies can help overcome such drawbacks. We first study worker interactions, in the absence of any notifications, to discover some spatio-temporal properties of task acceptance and completion. Based on these insights, we then experimentally demonstrate the effectiveness of two novel, non-personal, context-driven notification strategies, comparing the outcomes to two different baselines (no-notification and random-notification). Finally, using the data from the random-notification mechanism, we derive a classification model, incorporating several novel contextual features, that can predict a worker's responsiveness to notifications with high accuracy. Our work extends the crowd-sourcing literature by emphasizing the power of smart notifications for greater worker engagement.

ACM2020

Lakmal Buddika Meegahapola

Generalization and Personalization of Mobile Sensing-Based Mood Inference Models: An Analysis of College Students in Eight Countries

Sensing Eating Events in Context: A Smartphone-Only Approach

One More Bite? Inferring Food Consumption Level of College Students Using Smartphone Sensing and Self-Reports

Smartphone Sensing for the Well-being of Young Adults: A Review

Ten seconds of my nights: Exploring methods to measure brightness, loudness and attendance and their associations with alcohol use from video clips

The Theory, Practice, and Ethical Challenges of Designing a Diversity-Aware Platform for Social Relations

Alone or With Others? Understanding Eating Episodes of College Students with Mobile Sensing

Jointly Optimizing Sensing Pipelines for Multimodal Mixed Reality Interaction

PokeME: Applying Context-Driven Notifications to Increase Worker Engagement in Mobile Crowd-Sourcing

Alone or With Others? Understanding Eating Episodes of College Students with Mobile Sensing

The Theory, Practice, and Ethical Challenges of Designing a Diversity-Aware Platform for Social Relations

Generalization and Personalization of Mobile Sensing-Based Mood Inference Models: An Analysis of College Students in Eight Countries

Smartphone Sensing for the Well-being of Young Adults: A Review

Sensing Eating Events in Context: A Smartphone-Only Approach

One More Bite? Inferring Food Consumption Level of College Students Using Smartphone Sensing and Self-Reports

Ten seconds of my nights: Exploring methods to measure brightness, loudness and attendance and their associations with alcohol use from video clips

Jointly Optimizing Sensing Pipelines for Multimodal Mixed Reality Interaction

PokeME: Applying Context-Driven Notifications to Increase Worker Engagement in Mobile Crowd-Sourcing