Towards smart glasses for facial expression recognition using OMG and machine learning

Kiprijanovska, Ivana; Stankoski, Simon; Broulidakis, M. John; Archer, James; Fatoorechi, Mohsen; Gjoreski, Martin; Nduka, Charles; Gjoreski, Hristijan

doi:10.1038/s41598-023-43135-5

Towards smart glasses for facial expression recognition using OMG and machine learning

Journal

Scientific Reports

Date Issued

2023-09-25

Author(s)

Kiprijanovska, Ivana

Stankoski, Simon

Broulidakis, M. John

Archer, James

Fatoorechi, Mohsen

Gjoreski, Martin

Nduka, Charles

Gjoreski, Hristijan

DOI

10.1038/s41598-023-43135-5

Abstract

AbstractThis study aimed to evaluate the use of novel optomyography (OMG) based smart glasses, OCOsense, for the monitoring and recognition of facial expressions. Experiments were conducted on data gathered from 27 young adult participants, who performed facial expressions varying in intensity, duration, and head movement. The facial expressions included smiling, frowning, raising the eyebrows, and squeezing the eyes. The statistical analysis demonstrated that: (i) OCO sensors based on the principles of OMG can capture distinct variations in cheek and brow movements with a high degree of accuracy and specificity; (ii) Head movement does not have a significant impact on how well these facial expressions are detected. The collected data were also used to train a machine learning model to recognise the four facial expressions and when the face enters a neutral state. We evaluated this model in conditions intended to simulate real-world use, including variations in expression intensity, head movement and glasses position relative to the face. The model demonstrated an overall accuracy of 93% (0.90 f1-score)—evaluated using a leave-one-subject-out cross-validation technique.