Correlation Between Measurable Facial expressions, Emotions, Usability, and Performance, a Preliminary Study

Date: 2024-06-10 to 2024-07-31
Role: Researcher, Developer, Designer

As part of a research internship at the University of Oviedo, together with Katerina Boni, I wrote a paper examining the relationship between facial expressions and website usability using the Facial Action Coding System (FACS) to analyze specific facial muscle movements (Action Units or AUs). Participants completed tasks on a custom-built website while their facial expressions were captured and usability metrics, such as the System Usability Scale (SUS), were recorded.

Key findings include a statistically significant correlation between the cheek-raiser movement (AU06) and perceived usability (SUS). Additionally, emotions such as fear and disgust negatively impacted user engagement and satisfaction. Despite a limited sample size, the study underscores the potential of leveraging affective computing to enhance usability evaluation and highlights the challenges of working with facial recognition in online environments.

The experiment included creating a fictional service called FreshBox, where users must navigate through the UI to achieve a certain goal. The project included developing the concept for the entire experiment, designing and building the interactive web experience for the project, and gathering and analyzing all the data collected during the experiment.

The paper has not been published.

Abstract

The optimization of website usability has become increasingly important as web-based platforms continue to evolve, aiming to meet user expectations and enhance satisfaction. Traditional methods of evaluating usability, relying on user feedback and performance metrics, often fail to capture the subjective experiences of users during website interactions. As a result, researchers have turned their attention to non-verbal cues, such as facial expressions, as potential indicators of user experiences. Facial expressions serve as a natural means of communication, conveying a wealth of emotional and cognitive information. While numerous research efforts have focused on measuring emotions through facial expressions, this narrow approach can lead to misinterpretations and overlook essential facets of user experiences. In contrast, the Facial Action Coding System (FACS), a widely adopted framework for analyzing facial expressions, offers a comprehensive approach. 

This project investigates the correlation between action units (AUs) of facial expressions and website usability, capturing facial images during participants’ interactions. Simultaneously, participants completed tasks on the website while providing usability ratings through a survey questionnaire. Furthermore, we examined the relationship between perceived usability and participants’ task performance. However, upon analyzing the data, no statistically significant results were found, indicating a lack of clear correlation between action units of facial expressions and website usability. Despite these non-significant outcomes, this research contributes to the field by acknowledging the complexities and challenges in exploring the relationship between facial expressions and usability.