Multimodal sentiment analysis | EPFL Graph Search

Multimodal sentiment analysis is a technology for traditional text-based sentiment analysis, which includes modalities such as audio and visual data. It can be bimodal, which includes different combinations of two modalities, or trimodal, which incorporates three modalities. With the extensive amount of social media data available online in different forms such as videos and images, the conventional text-based sentiment analysis has evolved into more complex models of multimodal sentiment analysis, which can be applied in the development of virtual assistants, analysis of YouTube movie reviews, analysis of news videos, and emotion recognition (sometimes known as emotion detection) such as depression monitoring, among others. Similar to the traditional sentiment analysis, one of the most basic task in multimodal sentiment analysis is sentiment classification, which classifies different sentiments into categories such as positive, negative, or neutral. The complexity of analyzing text,

Acquiring Broad Commonsense Knowledge for Sentiment Analysis Using Human Computation

Marina Boia

While artificial intelligence is successful in many applications that cover specific domains, for many commonsense problems there is still a large gap with human performance. Automated sentiment analysis is a typical example: while there are techniques that reasonably aggregate sentiments from texts in specific domains, such as online reviews of a particular product category, more general models have a poor performance. We argue that sentiment analysis can be covered more broadly by extending models with commonsense knowledge acquired at scale, using human computation. We study two sentiment analysis problems. We start with document-level sentiment classification, which aims to determine whether a text as a whole expresses a positive or a negative sentiment. We hypothesize that extending classifiers to include the polarities of sentiment words in context can help them scale to broad domains. We also study fine-grained opinion extraction, which aims to pinpoint individual opinions in a text, along with their targets. We hypothesize that extraction models can benefit from broad fine-grained annotations to boost their performance on unfamiliar domains. Selecting sentiment words in context and annotating texts with opinions and targets are tasks that require commonsense knowledge shared by all the speakers of a language. We show how these can be effectively solved through human computation. We illustrate how to define small tasks that can be solved by many independent workers so that results can form a single coherent knowledge base. We also show how to recruit, train, and engage workers, then how to perform effective quality control to obtain sufficiently high-quality knowledge. We show how the resulting knowledge can be effectively integrated into models that scale to broad domains and also perform well in unfamiliar domains. We engage workers through both enjoyment and payment, by designing our tasks as games played for money. We recruit them on a paid crowdsourcing platform where we can reach out to a large pool of active workers. This is an effective recipe for acquiring sentiment knowledge in English, a language that is known by the vast majority of workers on the platform. To acquire sentiment knowledge for other languages, which have received comparatively little attention, we argue that we need to design tasks that appeal to voluntary workers outside the crowdsourcing platform, based on enjoyment alone. However, recruiting and engaging volunteers has been more of an art than a problem that can be solved systematically. We show that combining online advertisement with games, an approach that has been recently proved to work well for acquiring expert knowledge, gives an effective recipe for luring and engaging volunteers to provide good quality sentiment knowledge for texts in French. Our solutions could point the way to how to use human computation to broaden the competence of artificial intelligence systems in other domains as well.

EPFL2016

EmotionWatch: Visualizing Fine-Grained Emotions in Event-Related Tweets

Renato Kempter, Claudiu-Cristian Musat, Pearl Pu Faltings, Valentina Sintsova

Spectators are increasingly using social platforms to express their opinions and share their emotions during big public events. Those reactions reveal the subjective perception of the event and extend its understanding. This has motivated us to develop a system to explore and visualize volume, patterns, and trends of user sentiments as they evolve over time. Previous work in sentiment analysis and opinion mining has addressed these issues. But the majority of them distinguish only two polarity categories, leaving a more detailed and insightful anal- ysis to be desired. In this paper, we suggest using a fine- grained, multi-category emotion model to classify and visualize users’ emotional reactions in public events. We describe EmotionWatch, a tool that constructs visual summaries of public emotions, and apply it to the 2012 Olympics as a test case. We report findings from a user study evaluating the usability of the tool and val- idating the emotion model. Results show that users prefer a more detailed inspection of public emotions over the simplified analysis. Despite its complexity, users were able to effectively grasp, understand, and interpret the emotional reactions using EmotionWatch. The same user study also pointed out few design improvements for the future development of analogous systems.

2014

EmotionWatch: Visualizing Fine-Grained Emotions in Event-Related Tweets

Renato Kempter, Claudiu-Cristian Musat, Pearl Pu Faltings, Valentina Sintsova

Spectators are increasingly using social platforms to express their opinions and share their emotions during big public events. Those reactions reveal the subjective perception of the event and extend its understanding. This has motivated us to develop a system to explore and visualize volume, patterns, and trends of user sentiments as they evolve over time. Previous work in sentiment analysis and opinion mining has addressed these issues. But the majority of them distinguish only two polarity categories, leaving a more detailed and insightful analysis to be desired. In this paper, we suggest using a fine-grained, multi-category emotion model to classify and visualize users’ emotional reactions in public events. We describe EmotionWatch, a tool that constructs visual summaries of public emotions, and apply it to the 2012 Olympics as a test case. We report findings from a user study evaluating the usability of the tool and validating the emotion model. Results show that users prefer a more detailed inspection of public emotions over the simplified analysis. Despite its complexity, users were able to effectively grasp, understand, and interpret the emotional reactions using EmotionWatch. The same user study also pointed out few design improvements for the future development of analogous systems.

2014

Acquiring Broad Commonsense Knowledge for Sentiment Analysis Using Human Computation

Marina Boia

EPFL2016